2021-11-21T21:45:43.231 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-21T21:45:43.233 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba --verbose --archive /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819 --owner scheduled_teuthology@teuthology --description fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} -- /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/orig.config.yaml 2021-11-21T21:45:43.283 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-11-21T21:45:43.356 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819 branch: octopus description: fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: ceph-qa@ceph.io first_in_suite: false job_id: '6512819' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: gibba name: teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: octopus ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: debug ms: 1 debug osd: 20 osd objectstore: filestore osd op complaint time: 180 osd sloppy crc: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 911bc85f89d5c4ff92555237a37b5b774d7b1332 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: default sha1: 911bc85f89d5c4ff92555237a37b5b774d7b1332 workunit: branch: octopus sha1: 911bc85f89d5c4ff92555237a37b5b774d7b1332 owner: scheduled_teuthology@teuthology priority: 1000 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 sha1: 911bc85f89d5c4ff92555237a37b5b774d7b1332 sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 911bc85f89d5c4ff92555237a37b5b774d7b1332 targets: gibba016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbXkiIf+eb7zGtRPK3lWcFKzUuKOrgtMaTJTd36n1GegE0Yr1xXyD47mFwAh994XjmyywD0J7U6QI4B6makRyLIF4Pr60tZ7K6mnPzUDYvTcJHYJB2d6yOBgc7FqYzfYnWDWvFm1hrJ2QYPtqFoormplI9Xa9mNs12w3bdQH9E8NoSeLHk06/UeQwwWlkEY5qYgQeqqb0Dafaang2AMDep8v5MpQV0rxnaLKrv7krm2JexHuJIndmhjeRLFM9inpjZ1oSKLru4cC0a1EPrFPKfcQx07tIMI+7QvBoE+GSMKU3Tj0S3TsV6/nTlaSryFtKhikFLOWHF6IS8Lw7nwUhDYSEZ1X2oac3AiqPdTuaW+bFkXMQxVicb2/EwpjGZMx30B9ereV+Z8iUQl4Kz/Tmv2NsJKNan9A3sSF+b4LDxvEKbSliaKjkmVpEDl4/TMhpdPRly7llyvDtwgrNGgOC1O6xnob9QLe8KZTx2muLuAYCLPct/6gaGV4sdIl/jI08= gibba017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgM4jU37Z4i0t4d5zVA+fpbIZqWv4wZaq6viFXwkYqq8kBhca49H2EbhNKK0AYgvci0cdE9Zi5AmEYIjt7zDocjDrlAaAOnHJ9fGEv1eQUmLpJjj0cr43A2ziIdTmAy5mXk+/yZ5Ewj3lchN/NcszAxd0ahsIX5Pf6ZmjZO+W5mPvuHUzp8+kgx7FjfTB87Pfs4xRNK5oVDdaCKOFWJv9KqSUUjzzQlzMXpeNxru6xt4yLXvh7MzJw5IYF7PJDn8StkXu+mZSuD9I6vQ5Jfkw15UDfhXymT8I3dDgaSb7Kkf1JMGG34aMIiA5ooKwGEK4QYM4DOKXaY1OIWpEhMr3V6d3zKIey8Lqw8rAQn6qRAReyUNnOhBUARb0h5qKLMlBoTfq8GGLQ60gtlVZIORR2Arc2HJKlRblMpqzBkTsdfaremYKNzn+OXAzdl0Zo4j2pbis1zHREIhj0x/Tv91iQE/9xo0XMV3+EseNvTgIT+qOVCEpIOPYSA3llev1YAKs= gibba019.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIe+TTxY3Zgr8xdckrjIs412kMfHS2r40ciQq47T2ubn2KrPOng709Xa5yi2A6JfSKXxFq0vHq/nGE7NDoOGSxq9H6co/fgSMhNvzzwimt/YFcTf9BLnXlPOrsbAxl3P282zN48iSk9Mp7kE7gwXbAJRnsGydqWxm1ViOeUeSocmeohGziYAWytq+nTIH1Os+c1WFBmgPm97LzM2Vn/IkU5DXvPmhIfqN4wMq8zM1i7Vz6LVNW+ZhjmnzOJn9KkqAL4pyCaWoaUtr0nWSFlVQ39kROW9IlMgFTnCnvxO9QracTMV6LTjLqtVR49/4dedABRMzzbx3tJ4ERs/jjqOYFQJ/LhfxwdCW5/NgmMMG2wTHamzIeqB0qW/6v0qeSpSVBQi/EcmXKyhiO9UHtMT594c/6OkplG6sPgenEx9+nNhVB67kfUtIjz1oRUqXsJU1GxNuvdsMicAZjIITL73gMkq8csydPC6zf2jW2caFF2iqotjaSp6uwYZeVmXZXjb0= gibba036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKcULwJOz5ZQZp/E5QF+Cdla5PaLDPEvKPeItzDNuAxJYhjtDOr33hj34nuOJj1QTv4QoUmjwoqeJf+vB/PaFKOQb1JfjkusBLbVlGQuvXkphmlaRgpIYNXjWhBbvU29aL4cLlSWFOj6gOBfIKDn8UPlX1l6xl9DixM9rcZX6HI5rm73b358jNBx3qAxo+QH/j1rCvfQhyvUf6KU+47wQyWBAfm+YJcwgs5xxfqt8evozoU0/KoBiNy4885gnoDAdSrfF/z1qXSJaagKWvwdUnjkhm6VPNpxlJBQdI3sQF4ga+Tmgeqt618d5lFtLE35/HpofjvAh3N9s9AhdH8cxvCXwku0gS50NVfm5GVB8Obanh8XqFwXzWnKwZgO/aBC8+Z6zMWpXNoqj2Jl2/UYddtBuaSEjL2ClwYkPehPzbpvxySpwHqGgZ9smStMC4DkHD/79S8LYCN2oOqIUGqdLd5JhZwneSmCQq4s7ZDYClcx3jksrQAmP3Fr4C5LURTEU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: master teuthology_sha1: 6cfddd5e1c7d5ede1367cf3087c21af44eb88356 timestamp: 2021-11-19_04:10:03 tube: gibba user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.gibba.19943 2021-11-21T21:45:43.357 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa; will attempt to use it 2021-11-21T21:45:43.357 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks 2021-11-21T21:45:43.358 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-11-21T21:45:43.360 INFO:teuthology.task.internal:Checking packages... 2021-11-21T21:45:43.381 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '911bc85f89d5c4ff92555237a37b5b774d7b1332' 2021-11-21T21:45:43.381 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-11-21T21:45:43.382 INFO:teuthology.packaging:ref: None 2021-11-21T21:45:43.382 INFO:teuthology.packaging:tag: None 2021-11-21T21:45:43.382 INFO:teuthology.packaging:branch: octopus 2021-11-21T21:45:43.383 INFO:teuthology.packaging:sha1: 911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:45:43.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=octopus 2021-11-21T21:45:43.694 INFO:teuthology.task.internal:Found packages for ceph version 15.2.15-36-g911bc85f-1focal 2021-11-21T21:45:43.695 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-11-21T21:45:43.710 INFO:teuthology.task.internal:no buildpackages task found 2021-11-21T21:45:43.710 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-11-21T21:45:43.759 INFO:teuthology.task.internal:Saving configuration 2021-11-21T21:45:43.775 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-11-21T21:45:43.816 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-21T21:45:43.838 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'gibba016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819', 'up': True, 'machine_type': 'gibba', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-21 21:39:56.373408', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF+cS4vAqVILOcUw3UJ6rMrtJ04HsONFrCuSVnciEPi9'} 2021-11-21T21:45:43.860 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'gibba017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819', 'up': True, 'machine_type': 'gibba', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-21 21:39:56.374810', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEzXi7/pB4Z9eZVwLNSk4d39S26jYcR6WCTPW5CFwLyg'} 2021-11-21T21:45:43.881 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'gibba019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819', 'up': True, 'machine_type': 'gibba', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-21 21:39:56.375846', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG4P95R2jHYlqGmtchRUABPuLkUhN4+KGVHKDO1o4e6R'} 2021-11-21T21:45:43.904 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'gibba036.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819', 'up': True, 'machine_type': 'gibba', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-21 21:39:56.376877', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOSWNAfIQbYbC7ALksUpsZtHo3uIZWUCwf3QbchQkBpv'} 2021-11-21T21:45:43.905 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-11-21T21:45:43.915 INFO:teuthology.task.internal:roles: ubuntu@gibba016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2021-11-21T21:45:43.915 INFO:teuthology.task.internal:roles: ubuntu@gibba017.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-11-21T21:45:43.915 INFO:teuthology.task.internal:roles: ubuntu@gibba019.front.sepia.ceph.com - ['client.0'] 2021-11-21T21:45:43.915 INFO:teuthology.task.internal:roles: ubuntu@gibba036.front.sepia.ceph.com - ['client.1'] 2021-11-21T21:45:43.916 INFO:teuthology.run_tasks:Running task console_log... 2021-11-21T21:45:44.085 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1d4aac3e18>, signals=[15]) 2021-11-21T21:45:44.086 INFO:teuthology.run_tasks:Running task internal.connect... 2021-11-21T21:45:44.096 INFO:teuthology.task.internal:Opening connections... 2021-11-21T21:45:44.097 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T21:45:44.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:45:44.136 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T21:45:44.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:45:44.172 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba019.front.sepia.ceph.com 2021-11-21T21:45:44.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:45:44.208 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba036.front.sepia.ceph.com 2021-11-21T21:45:44.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:45:44.244 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-11-21T21:45:44.255 DEBUG:teuthology.orchestra.run.gibba016:> uname -m 2021-11-21T21:45:44.261 INFO:teuthology.orchestra.run.gibba016.stdout:x86_64 2021-11-21T21:45:44.261 DEBUG:teuthology.orchestra.run.gibba016:> cat /etc/os-release 2021-11-21T21:45:44.310 INFO:teuthology.orchestra.run.gibba016.stdout:NAME="Ubuntu" 2021-11-21T21:45:44.310 INFO:teuthology.orchestra.run.gibba016.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-21T21:45:44.310 INFO:teuthology.orchestra.run.gibba016.stdout:ID=ubuntu 2021-11-21T21:45:44.311 INFO:teuthology.orchestra.run.gibba016.stdout:ID_LIKE=debian 2021-11-21T21:45:44.311 INFO:teuthology.orchestra.run.gibba016.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-21T21:45:44.311 INFO:teuthology.orchestra.run.gibba016.stdout:VERSION_ID="20.04" 2021-11-21T21:45:44.311 INFO:teuthology.orchestra.run.gibba016.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-21T21:45:44.311 INFO:teuthology.orchestra.run.gibba016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-21T21:45:44.312 INFO:teuthology.orchestra.run.gibba016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-21T21:45:44.312 INFO:teuthology.orchestra.run.gibba016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-21T21:45:44.312 INFO:teuthology.orchestra.run.gibba016.stdout:VERSION_CODENAME=focal 2021-11-21T21:45:44.312 INFO:teuthology.orchestra.run.gibba016.stdout:UBUNTU_CODENAME=focal 2021-11-21T21:45:44.313 INFO:teuthology.lock.ops:Updating gibba016.front.sepia.ceph.com on lock server 2021-11-21T21:45:44.333 DEBUG:teuthology.orchestra.run.gibba017:> uname -m 2021-11-21T21:45:44.342 INFO:teuthology.orchestra.run.gibba017.stdout:x86_64 2021-11-21T21:45:44.343 DEBUG:teuthology.orchestra.run.gibba017:> cat /etc/os-release 2021-11-21T21:45:44.393 INFO:teuthology.orchestra.run.gibba017.stdout:NAME="Ubuntu" 2021-11-21T21:45:44.393 INFO:teuthology.orchestra.run.gibba017.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-21T21:45:44.394 INFO:teuthology.orchestra.run.gibba017.stdout:ID=ubuntu 2021-11-21T21:45:44.394 INFO:teuthology.orchestra.run.gibba017.stdout:ID_LIKE=debian 2021-11-21T21:45:44.394 INFO:teuthology.orchestra.run.gibba017.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-21T21:45:44.394 INFO:teuthology.orchestra.run.gibba017.stdout:VERSION_ID="20.04" 2021-11-21T21:45:44.395 INFO:teuthology.orchestra.run.gibba017.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-21T21:45:44.395 INFO:teuthology.orchestra.run.gibba017.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-21T21:45:44.395 INFO:teuthology.orchestra.run.gibba017.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-21T21:45:44.395 INFO:teuthology.orchestra.run.gibba017.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-21T21:45:44.396 INFO:teuthology.orchestra.run.gibba017.stdout:VERSION_CODENAME=focal 2021-11-21T21:45:44.396 INFO:teuthology.orchestra.run.gibba017.stdout:UBUNTU_CODENAME=focal 2021-11-21T21:45:44.396 INFO:teuthology.lock.ops:Updating gibba017.front.sepia.ceph.com on lock server 2021-11-21T21:45:44.414 DEBUG:teuthology.orchestra.run.gibba019:> uname -m 2021-11-21T21:45:44.423 INFO:teuthology.orchestra.run.gibba019.stdout:x86_64 2021-11-21T21:45:44.423 DEBUG:teuthology.orchestra.run.gibba019:> cat /etc/os-release 2021-11-21T21:45:44.470 INFO:teuthology.orchestra.run.gibba019.stdout:NAME="Ubuntu" 2021-11-21T21:45:44.470 INFO:teuthology.orchestra.run.gibba019.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-21T21:45:44.471 INFO:teuthology.orchestra.run.gibba019.stdout:ID=ubuntu 2021-11-21T21:45:44.471 INFO:teuthology.orchestra.run.gibba019.stdout:ID_LIKE=debian 2021-11-21T21:45:44.471 INFO:teuthology.orchestra.run.gibba019.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-21T21:45:44.471 INFO:teuthology.orchestra.run.gibba019.stdout:VERSION_ID="20.04" 2021-11-21T21:45:44.472 INFO:teuthology.orchestra.run.gibba019.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-21T21:45:44.472 INFO:teuthology.orchestra.run.gibba019.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-21T21:45:44.472 INFO:teuthology.orchestra.run.gibba019.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-21T21:45:44.472 INFO:teuthology.orchestra.run.gibba019.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-21T21:45:44.472 INFO:teuthology.orchestra.run.gibba019.stdout:VERSION_CODENAME=focal 2021-11-21T21:45:44.473 INFO:teuthology.orchestra.run.gibba019.stdout:UBUNTU_CODENAME=focal 2021-11-21T21:45:44.473 INFO:teuthology.lock.ops:Updating gibba019.front.sepia.ceph.com on lock server 2021-11-21T21:45:44.506 DEBUG:teuthology.orchestra.run.gibba036:> uname -m 2021-11-21T21:45:44.515 INFO:teuthology.orchestra.run.gibba036.stdout:x86_64 2021-11-21T21:45:44.516 DEBUG:teuthology.orchestra.run.gibba036:> cat /etc/os-release 2021-11-21T21:45:44.568 INFO:teuthology.orchestra.run.gibba036.stdout:NAME="Ubuntu" 2021-11-21T21:45:44.568 INFO:teuthology.orchestra.run.gibba036.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-21T21:45:44.568 INFO:teuthology.orchestra.run.gibba036.stdout:ID=ubuntu 2021-11-21T21:45:44.569 INFO:teuthology.orchestra.run.gibba036.stdout:ID_LIKE=debian 2021-11-21T21:45:44.569 INFO:teuthology.orchestra.run.gibba036.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-21T21:45:44.569 INFO:teuthology.orchestra.run.gibba036.stdout:VERSION_ID="20.04" 2021-11-21T21:45:44.569 INFO:teuthology.orchestra.run.gibba036.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-21T21:45:44.569 INFO:teuthology.orchestra.run.gibba036.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-21T21:45:44.570 INFO:teuthology.orchestra.run.gibba036.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-21T21:45:44.570 INFO:teuthology.orchestra.run.gibba036.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-21T21:45:44.570 INFO:teuthology.orchestra.run.gibba036.stdout:VERSION_CODENAME=focal 2021-11-21T21:45:44.570 INFO:teuthology.orchestra.run.gibba036.stdout:UBUNTU_CODENAME=focal 2021-11-21T21:45:44.571 INFO:teuthology.lock.ops:Updating gibba036.front.sepia.ceph.com on lock server 2021-11-21T21:45:44.589 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-11-21T21:45:44.605 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-11-21T21:45:44.666 INFO:teuthology.task.internal:Checking for old test directory... 2021-11-21T21:45:44.666 DEBUG:teuthology.orchestra.run.gibba016:> test '!' -e /home/ubuntu/cephtest 2021-11-21T21:45:44.670 DEBUG:teuthology.orchestra.run.gibba017:> test '!' -e /home/ubuntu/cephtest 2021-11-21T21:45:44.674 DEBUG:teuthology.orchestra.run.gibba019:> test '!' -e /home/ubuntu/cephtest 2021-11-21T21:45:44.677 DEBUG:teuthology.orchestra.run.gibba036:> test '!' -e /home/ubuntu/cephtest 2021-11-21T21:45:44.686 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-11-21T21:45:44.708 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-11-21T21:45:44.709 DEBUG:teuthology.orchestra.run.gibba016:> test -z $(ls -A /var/lib/ceph) 2021-11-21T21:45:44.712 DEBUG:teuthology.orchestra.run.gibba017:> test -z $(ls -A /var/lib/ceph) 2021-11-21T21:45:44.723 DEBUG:teuthology.orchestra.run.gibba019:> test -z $(ls -A /var/lib/ceph) 2021-11-21T21:45:44.728 DEBUG:teuthology.orchestra.run.gibba036:> test -z $(ls -A /var/lib/ceph) 2021-11-21T21:45:44.750 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-11-21T21:45:44.857 INFO:teuthology.run_tasks:Running task kernel... 2021-11-21T21:45:44.874 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2021-11-21T21:45:44.875 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-11-21T21:45:44.876 DEBUG:teuthology.orchestra.run.gibba016:> uname -r 2021-11-21T21:45:44.877 DEBUG:teuthology.orchestra.run.gibba017:> uname -r 2021-11-21T21:45:44.877 DEBUG:teuthology.orchestra.run.gibba019:> uname -r 2021-11-21T21:45:44.878 DEBUG:teuthology.orchestra.run.gibba036:> uname -r 2021-11-21T21:45:44.886 INFO:teuthology.orchestra.run.gibba019.stdout:5.4.0-88-generic 2021-11-21T21:45:44.887 INFO:teuthology.task.kernel:Running kernel on gibba019: 5.4.0-88-generic 2021-11-21T21:45:44.887 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get clean 2021-11-21T21:45:44.889 INFO:teuthology.orchestra.run.gibba036.stdout:5.4.0-88-generic 2021-11-21T21:45:44.890 INFO:teuthology.orchestra.run.gibba016.stdout:5.4.0-88-generic 2021-11-21T21:45:44.890 INFO:teuthology.orchestra.run.gibba017.stdout:5.4.0-88-generic 2021-11-21T21:45:44.890 INFO:teuthology.task.kernel:Running kernel on gibba036: 5.4.0-88-generic 2021-11-21T21:45:44.891 DEBUG:teuthology.orchestra.run.gibba036:> sudo apt-get clean 2021-11-21T21:45:44.891 INFO:teuthology.task.kernel:Running kernel on gibba016: 5.4.0-88-generic 2021-11-21T21:45:44.891 DEBUG:teuthology.orchestra.run.gibba016:> sudo apt-get clean 2021-11-21T21:45:44.892 INFO:teuthology.task.kernel:Running kernel on gibba017: 5.4.0-88-generic 2021-11-21T21:45:44.892 DEBUG:teuthology.orchestra.run.gibba017:> sudo apt-get clean 2021-11-21T21:45:45.003 DEBUG:teuthology.orchestra.run.gibba036:> sudo apt-get update 2021-11-21T21:45:45.006 DEBUG:teuthology.orchestra.run.gibba017:> sudo apt-get update 2021-11-21T21:45:45.022 DEBUG:teuthology.orchestra.run.gibba016:> sudo apt-get update 2021-11-21T21:45:45.062 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get update 2021-11-21T21:45:45.127 INFO:teuthology.orchestra.run.gibba036.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-11-21T21:45:45.129 INFO:teuthology.orchestra.run.gibba017.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-11-21T21:45:45.135 INFO:teuthology.orchestra.run.gibba016.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-11-21T21:45:45.238 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:45:45.239 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:45:45.246 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:45:45.308 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:45:45.314 INFO:teuthology.orchestra.run.gibba019.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-11-21T21:45:45.321 INFO:teuthology.orchestra.run.gibba017.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-11-21T21:45:45.328 INFO:teuthology.orchestra.run.gibba016.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-11-21T21:45:45.332 INFO:teuthology.orchestra.run.gibba036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-11-21T21:45:45.354 INFO:teuthology.orchestra.run.gibba036.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [306 kB] 2021-11-21T21:45:45.367 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-11-21T21:45:45.376 INFO:teuthology.orchestra.run.gibba016.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [987 kB] 2021-11-21T21:45:45.409 INFO:teuthology.orchestra.run.gibba017.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [987 kB] 2021-11-21T21:45:45.426 INFO:teuthology.orchestra.run.gibba036.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [987 kB] 2021-11-21T21:45:45.468 INFO:teuthology.orchestra.run.gibba036.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [184 kB] 2021-11-21T21:45:45.470 INFO:teuthology.orchestra.run.gibba036.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,880 B] 2021-11-21T21:45:45.471 INFO:teuthology.orchestra.run.gibba036.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [20.4 kB] 2021-11-21T21:45:45.472 INFO:teuthology.orchestra.run.gibba036.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [526 kB] 2021-11-21T21:45:45.476 INFO:teuthology.orchestra.run.gibba016.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [306 kB] 2021-11-21T21:45:45.481 INFO:teuthology.orchestra.run.gibba036.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [75.4 kB] 2021-11-21T21:45:45.482 INFO:teuthology.orchestra.run.gibba036.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:45.483 INFO:teuthology.orchestra.run.gibba036.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [653 kB] 2021-11-21T21:45:45.483 INFO:teuthology.orchestra.run.gibba016.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [184 kB] 2021-11-21T21:45:45.486 INFO:teuthology.orchestra.run.gibba016.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,880 B] 2021-11-21T21:45:45.487 INFO:teuthology.orchestra.run.gibba016.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [20.4 kB] 2021-11-21T21:45:45.487 INFO:teuthology.orchestra.run.gibba016.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [526 kB] 2021-11-21T21:45:45.493 INFO:teuthology.orchestra.run.gibba016.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [75.4 kB] 2021-11-21T21:45:45.494 INFO:teuthology.orchestra.run.gibba036.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [515 kB] 2021-11-21T21:45:45.499 INFO:teuthology.orchestra.run.gibba016.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:45.499 INFO:teuthology.orchestra.run.gibba016.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [653 kB] 2021-11-21T21:45:45.499 INFO:teuthology.orchestra.run.gibba036.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [107 kB] 2021-11-21T21:45:45.500 INFO:teuthology.orchestra.run.gibba036.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [12.9 kB] 2021-11-21T21:45:45.502 INFO:teuthology.orchestra.run.gibba017.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [306 kB] 2021-11-21T21:45:45.507 INFO:teuthology.orchestra.run.gibba017.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [184 kB] 2021-11-21T21:45:45.509 INFO:teuthology.orchestra.run.gibba016.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [515 kB] 2021-11-21T21:45:45.510 INFO:teuthology.orchestra.run.gibba017.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,880 B] 2021-11-21T21:45:45.511 INFO:teuthology.orchestra.run.gibba017.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [526 kB] 2021-11-21T21:45:45.518 INFO:teuthology.orchestra.run.gibba036.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.9 kB] 2021-11-21T21:45:45.518 INFO:teuthology.orchestra.run.gibba036.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,212 B] 2021-11-21T21:45:45.519 INFO:teuthology.orchestra.run.gibba016.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [107 kB] 2021-11-21T21:45:45.519 INFO:teuthology.orchestra.run.gibba036.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2021-11-21T21:45:45.520 INFO:teuthology.orchestra.run.gibba017.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [20.4 kB] 2021-11-21T21:45:45.520 INFO:teuthology.orchestra.run.gibba017.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [75.4 kB] 2021-11-21T21:45:45.521 INFO:teuthology.orchestra.run.gibba016.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [12.9 kB] 2021-11-21T21:45:45.522 INFO:teuthology.orchestra.run.gibba017.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:45.523 INFO:teuthology.orchestra.run.gibba017.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [653 kB] 2021-11-21T21:45:45.531 INFO:teuthology.orchestra.run.gibba017.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [515 kB] 2021-11-21T21:45:45.532 INFO:teuthology.orchestra.run.gibba016.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.9 kB] 2021-11-21T21:45:45.532 INFO:teuthology.orchestra.run.gibba016.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,212 B] 2021-11-21T21:45:45.533 INFO:teuthology.orchestra.run.gibba016.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2021-11-21T21:45:45.538 INFO:teuthology.orchestra.run.gibba017.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [107 kB] 2021-11-21T21:45:45.539 INFO:teuthology.orchestra.run.gibba017.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [12.9 kB] 2021-11-21T21:45:45.549 INFO:teuthology.orchestra.run.gibba017.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,212 B] 2021-11-21T21:45:45.550 INFO:teuthology.orchestra.run.gibba017.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.9 kB] 2021-11-21T21:45:45.550 INFO:teuthology.orchestra.run.gibba017.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2021-11-21T21:45:45.660 INFO:teuthology.orchestra.run.gibba017.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-11-21T21:45:45.661 INFO:teuthology.orchestra.run.gibba036.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-11-21T21:45:45.676 INFO:teuthology.orchestra.run.gibba016.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-11-21T21:45:45.702 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [987 kB] 2021-11-21T21:45:45.743 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-11-21T21:45:45.755 INFO:teuthology.orchestra.run.gibba017.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [563 kB] 2021-11-21T21:45:45.756 INFO:teuthology.orchestra.run.gibba036.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [563 kB] 2021-11-21T21:45:45.782 INFO:teuthology.orchestra.run.gibba016.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [563 kB] 2021-11-21T21:45:45.855 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [563 kB] 2021-11-21T21:45:45.903 INFO:teuthology.orchestra.run.gibba036.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,344 kB] 2021-11-21T21:45:45.926 INFO:teuthology.orchestra.run.gibba017.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,344 kB] 2021-11-21T21:45:45.970 INFO:teuthology.orchestra.run.gibba016.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,344 kB] 2021-11-21T21:45:46.021 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,344 kB] 2021-11-21T21:45:46.061 INFO:teuthology.orchestra.run.gibba036.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [276 kB] 2021-11-21T21:45:46.065 INFO:teuthology.orchestra.run.gibba017.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [276 kB] 2021-11-21T21:45:46.079 INFO:teuthology.orchestra.run.gibba036.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [14.5 kB] 2021-11-21T21:45:46.080 INFO:teuthology.orchestra.run.gibba036.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [569 kB] 2021-11-21T21:45:46.080 INFO:teuthology.orchestra.run.gibba017.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [14.5 kB] 2021-11-21T21:45:46.081 INFO:teuthology.orchestra.run.gibba017.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [569 kB] 2021-11-21T21:45:46.081 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [306 kB] 2021-11-21T21:45:46.095 INFO:teuthology.orchestra.run.gibba019.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [184 kB] 2021-11-21T21:45:46.101 INFO:teuthology.orchestra.run.gibba017.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [21.7 kB] 2021-11-21T21:45:46.101 INFO:teuthology.orchestra.run.gibba017.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [81.6 kB] 2021-11-21T21:45:46.102 INFO:teuthology.orchestra.run.gibba019.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,880 B] 2021-11-21T21:45:46.103 INFO:teuthology.orchestra.run.gibba019.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [526 kB] 2021-11-21T21:45:46.104 INFO:teuthology.orchestra.run.gibba016.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [276 kB] 2021-11-21T21:45:46.105 INFO:teuthology.orchestra.run.gibba017.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:46.105 INFO:teuthology.orchestra.run.gibba017.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [876 kB] 2021-11-21T21:45:46.106 INFO:teuthology.orchestra.run.gibba036.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [21.7 kB] 2021-11-21T21:45:46.106 INFO:teuthology.orchestra.run.gibba036.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [81.6 kB] 2021-11-21T21:45:46.109 INFO:teuthology.orchestra.run.gibba036.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:46.109 INFO:teuthology.orchestra.run.gibba036.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [876 kB] 2021-11-21T21:45:46.121 INFO:teuthology.orchestra.run.gibba016.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [14.5 kB] 2021-11-21T21:45:46.122 INFO:teuthology.orchestra.run.gibba016.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [569 kB] 2021-11-21T21:45:46.127 INFO:teuthology.orchestra.run.gibba019.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [20.4 kB] 2021-11-21T21:45:46.128 INFO:teuthology.orchestra.run.gibba019.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [75.4 kB] 2021-11-21T21:45:46.136 INFO:teuthology.orchestra.run.gibba036.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [647 kB] 2021-11-21T21:45:46.140 INFO:teuthology.orchestra.run.gibba017.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [647 kB] 2021-11-21T21:45:46.146 INFO:teuthology.orchestra.run.gibba016.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [21.7 kB] 2021-11-21T21:45:46.147 INFO:teuthology.orchestra.run.gibba016.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [81.6 kB] 2021-11-21T21:45:46.150 INFO:teuthology.orchestra.run.gibba016.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:46.150 INFO:teuthology.orchestra.run.gibba016.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [647 kB] 2021-11-21T21:45:46.157 INFO:teuthology.orchestra.run.gibba019.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [276 kB] 2021-11-21T21:45:46.160 INFO:teuthology.orchestra.run.gibba036.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [190 kB] 2021-11-21T21:45:46.161 INFO:teuthology.orchestra.run.gibba017.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [190 kB] 2021-11-21T21:45:46.165 INFO:teuthology.orchestra.run.gibba019.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:46.165 INFO:teuthology.orchestra.run.gibba019.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [653 kB] 2021-11-21T21:45:46.166 INFO:teuthology.orchestra.run.gibba036.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [19.5 kB] 2021-11-21T21:45:46.166 INFO:teuthology.orchestra.run.gibba017.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [19.5 kB] 2021-11-21T21:45:46.170 INFO:teuthology.orchestra.run.gibba016.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [876 kB] 2021-11-21T21:45:46.172 INFO:teuthology.orchestra.run.gibba019.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [14.5 kB] 2021-11-21T21:45:46.173 INFO:teuthology.orchestra.run.gibba019.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [569 kB] 2021-11-21T21:45:46.187 INFO:teuthology.orchestra.run.gibba017.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.5 kB] 2021-11-21T21:45:46.188 INFO:teuthology.orchestra.run.gibba017.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,204 B] 2021-11-21T21:45:46.188 INFO:teuthology.orchestra.run.gibba017.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,856 B] 2021-11-21T21:45:46.188 INFO:teuthology.orchestra.run.gibba017.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2021-11-21T21:45:46.189 INFO:teuthology.orchestra.run.gibba017.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [5,536 B] 2021-11-21T21:45:46.191 INFO:teuthology.orchestra.run.gibba036.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,204 B] 2021-11-21T21:45:46.192 INFO:teuthology.orchestra.run.gibba036.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.5 kB] 2021-11-21T21:45:46.193 INFO:teuthology.orchestra.run.gibba036.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,856 B] 2021-11-21T21:45:46.193 INFO:teuthology.orchestra.run.gibba036.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2021-11-21T21:45:46.193 INFO:teuthology.orchestra.run.gibba036.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [5,536 B] 2021-11-21T21:45:46.194 INFO:teuthology.orchestra.run.gibba019.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [515 kB] 2021-11-21T21:45:46.201 INFO:teuthology.orchestra.run.gibba016.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [190 kB] 2021-11-21T21:45:46.201 INFO:teuthology.orchestra.run.gibba019.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [21.7 kB] 2021-11-21T21:45:46.202 INFO:teuthology.orchestra.run.gibba019.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [81.6 kB] 2021-11-21T21:45:46.206 INFO:teuthology.orchestra.run.gibba019.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [528 B] 2021-11-21T21:45:46.206 INFO:teuthology.orchestra.run.gibba019.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [876 kB] 2021-11-21T21:45:46.207 INFO:teuthology.orchestra.run.gibba016.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [19.5 kB] 2021-11-21T21:45:46.216 INFO:teuthology.orchestra.run.gibba019.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [107 kB] 2021-11-21T21:45:46.221 INFO:teuthology.orchestra.run.gibba019.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [12.9 kB] 2021-11-21T21:45:46.225 INFO:teuthology.orchestra.run.gibba036.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [6,588 B] 2021-11-21T21:45:46.226 INFO:teuthology.orchestra.run.gibba036.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [3,292 B] 2021-11-21T21:45:46.231 INFO:teuthology.orchestra.run.gibba017.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [6,588 B] 2021-11-21T21:45:46.231 INFO:teuthology.orchestra.run.gibba017.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [3,292 B] 2021-11-21T21:45:46.232 INFO:teuthology.orchestra.run.gibba016.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,204 B] 2021-11-21T21:45:46.233 INFO:teuthology.orchestra.run.gibba016.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.5 kB] 2021-11-21T21:45:46.233 INFO:teuthology.orchestra.run.gibba016.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,856 B] 2021-11-21T21:45:46.234 INFO:teuthology.orchestra.run.gibba016.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2021-11-21T21:45:46.234 INFO:teuthology.orchestra.run.gibba016.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [6,588 B] 2021-11-21T21:45:46.236 INFO:teuthology.orchestra.run.gibba019.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [647 kB] 2021-11-21T21:45:46.255 INFO:teuthology.orchestra.run.gibba019.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [190 kB] 2021-11-21T21:45:46.259 INFO:teuthology.orchestra.run.gibba019.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [19.5 kB] 2021-11-21T21:45:46.260 INFO:teuthology.orchestra.run.gibba019.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.9 kB] 2021-11-21T21:45:46.261 INFO:teuthology.orchestra.run.gibba019.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,212 B] 2021-11-21T21:45:46.261 INFO:teuthology.orchestra.run.gibba019.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2021-11-21T21:45:46.268 INFO:teuthology.orchestra.run.gibba016.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [5,536 B] 2021-11-21T21:45:46.269 INFO:teuthology.orchestra.run.gibba016.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [3,292 B] 2021-11-21T21:45:46.288 INFO:teuthology.orchestra.run.gibba019.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.5 kB] 2021-11-21T21:45:46.289 INFO:teuthology.orchestra.run.gibba019.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,204 B] 2021-11-21T21:45:46.289 INFO:teuthology.orchestra.run.gibba019.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,856 B] 2021-11-21T21:45:46.290 INFO:teuthology.orchestra.run.gibba019.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2021-11-21T21:45:46.290 INFO:teuthology.orchestra.run.gibba019.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [6,588 B] 2021-11-21T21:45:46.307 INFO:teuthology.orchestra.run.gibba036.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [580 B] 2021-11-21T21:45:46.312 INFO:teuthology.orchestra.run.gibba017.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [580 B] 2021-11-21T21:45:46.324 INFO:teuthology.orchestra.run.gibba019.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [5,536 B] 2021-11-21T21:45:46.324 INFO:teuthology.orchestra.run.gibba019.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [3,292 B] 2021-11-21T21:45:46.350 INFO:teuthology.orchestra.run.gibba016.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [580 B] 2021-11-21T21:45:46.406 INFO:teuthology.orchestra.run.gibba019.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [580 B] 2021-11-21T21:45:48.225 INFO:teuthology.orchestra.run.gibba036.stdout:Fetched 8,413 kB in 1s (6,207 kB/s) 2021-11-21T21:45:48.254 INFO:teuthology.orchestra.run.gibba016.stdout:Fetched 8,413 kB in 1s (6,056 kB/s) 2021-11-21T21:45:48.311 INFO:teuthology.orchestra.run.gibba017.stdout:Fetched 8,413 kB in 1s (6,187 kB/s) 2021-11-21T21:45:48.593 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 8,413 kB in 2s (5,314 kB/s) 2021-11-21T21:45:49.181 INFO:teuthology.orchestra.run.gibba036.stdout:Reading package lists... 2021-11-21T21:45:49.192 DEBUG:teuthology.orchestra.run.gibba036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:45:49.228 INFO:teuthology.orchestra.run.gibba036.stdout:Reading package lists... 2021-11-21T21:45:49.288 INFO:teuthology.orchestra.run.gibba017.stdout:Reading package lists... 2021-11-21T21:45:49.290 INFO:teuthology.orchestra.run.gibba016.stdout:Reading package lists... 2021-11-21T21:45:49.299 DEBUG:teuthology.orchestra.run.gibba017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:45:49.301 DEBUG:teuthology.orchestra.run.gibba016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:45:49.336 INFO:teuthology.orchestra.run.gibba017.stdout:Reading package lists... 2021-11-21T21:45:49.337 INFO:teuthology.orchestra.run.gibba016.stdout:Reading package lists... 2021-11-21T21:45:49.338 INFO:teuthology.orchestra.run.gibba036.stdout:Building dependency tree... 2021-11-21T21:45:49.339 INFO:teuthology.orchestra.run.gibba036.stdout:Reading state information... 2021-11-21T21:45:49.422 INFO:teuthology.orchestra.run.gibba036.stdout:The following package was automatically installed and is no longer required: 2021-11-21T21:45:49.422 INFO:teuthology.orchestra.run.gibba036.stdout: libllvm11 2021-11-21T21:45:49.422 INFO:teuthology.orchestra.run.gibba036.stdout:Use 'sudo apt autoremove' to remove it. 2021-11-21T21:45:49.422 INFO:teuthology.orchestra.run.gibba036.stdout:The following additional packages will be installed: 2021-11-21T21:45:49.423 INFO:teuthology.orchestra.run.gibba036.stdout: linux-generic linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:49.423 INFO:teuthology.orchestra.run.gibba036.stdout: linux-headers-generic linux-image-5.4.0-90-generic 2021-11-21T21:45:49.423 INFO:teuthology.orchestra.run.gibba036.stdout: linux-modules-5.4.0-90-generic linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:49.424 INFO:teuthology.orchestra.run.gibba036.stdout:Suggested packages: 2021-11-21T21:45:49.424 INFO:teuthology.orchestra.run.gibba036.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-11-21T21:45:49.424 INFO:teuthology.orchestra.run.gibba036.stdout:Recommended packages: 2021-11-21T21:45:49.424 INFO:teuthology.orchestra.run.gibba036.stdout: thermald 2021-11-21T21:45:49.436 INFO:teuthology.orchestra.run.gibba017.stdout:Building dependency tree... 2021-11-21T21:45:49.436 INFO:teuthology.orchestra.run.gibba017.stdout:Reading state information... 2021-11-21T21:45:49.437 INFO:teuthology.orchestra.run.gibba016.stdout:Building dependency tree... 2021-11-21T21:45:49.437 INFO:teuthology.orchestra.run.gibba016.stdout:Reading state information... 2021-11-21T21:45:49.453 INFO:teuthology.orchestra.run.gibba036.stdout:The following NEW packages will be installed: 2021-11-21T21:45:49.454 INFO:teuthology.orchestra.run.gibba036.stdout: linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:49.454 INFO:teuthology.orchestra.run.gibba036.stdout: linux-image-5.4.0-90-generic linux-modules-5.4.0-90-generic 2021-11-21T21:45:49.454 INFO:teuthology.orchestra.run.gibba036.stdout: linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:49.455 INFO:teuthology.orchestra.run.gibba036.stdout:The following packages will be upgraded: 2021-11-21T21:45:49.455 INFO:teuthology.orchestra.run.gibba036.stdout: linux-generic linux-headers-generic linux-image-generic 2021-11-21T21:45:49.518 INFO:teuthology.orchestra.run.gibba017.stdout:The following package was automatically installed and is no longer required: 2021-11-21T21:45:49.518 INFO:teuthology.orchestra.run.gibba017.stdout: libllvm11 2021-11-21T21:45:49.518 INFO:teuthology.orchestra.run.gibba017.stdout:Use 'sudo apt autoremove' to remove it. 2021-11-21T21:45:49.520 INFO:teuthology.orchestra.run.gibba017.stdout:The following additional packages will be installed: 2021-11-21T21:45:49.520 INFO:teuthology.orchestra.run.gibba017.stdout: linux-generic linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:49.520 INFO:teuthology.orchestra.run.gibba017.stdout: linux-headers-generic linux-image-5.4.0-90-generic 2021-11-21T21:45:49.520 INFO:teuthology.orchestra.run.gibba017.stdout: linux-modules-5.4.0-90-generic linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:49.521 INFO:teuthology.orchestra.run.gibba017.stdout:Suggested packages: 2021-11-21T21:45:49.521 INFO:teuthology.orchestra.run.gibba017.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-11-21T21:45:49.521 INFO:teuthology.orchestra.run.gibba017.stdout:Recommended packages: 2021-11-21T21:45:49.521 INFO:teuthology.orchestra.run.gibba017.stdout: thermald 2021-11-21T21:45:49.521 INFO:teuthology.orchestra.run.gibba016.stdout:The following package was automatically installed and is no longer required: 2021-11-21T21:45:49.522 INFO:teuthology.orchestra.run.gibba016.stdout: libllvm11 2021-11-21T21:45:49.522 INFO:teuthology.orchestra.run.gibba016.stdout:Use 'sudo apt autoremove' to remove it. 2021-11-21T21:45:49.522 INFO:teuthology.orchestra.run.gibba016.stdout:The following additional packages will be installed: 2021-11-21T21:45:49.522 INFO:teuthology.orchestra.run.gibba016.stdout: linux-generic linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:49.523 INFO:teuthology.orchestra.run.gibba016.stdout: linux-headers-generic linux-image-5.4.0-90-generic 2021-11-21T21:45:49.523 INFO:teuthology.orchestra.run.gibba016.stdout: linux-modules-5.4.0-90-generic linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:49.523 INFO:teuthology.orchestra.run.gibba016.stdout:Suggested packages: 2021-11-21T21:45:49.523 INFO:teuthology.orchestra.run.gibba016.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-11-21T21:45:49.523 INFO:teuthology.orchestra.run.gibba016.stdout:Recommended packages: 2021-11-21T21:45:49.524 INFO:teuthology.orchestra.run.gibba016.stdout: thermald 2021-11-21T21:45:49.543 INFO:teuthology.orchestra.run.gibba017.stdout:The following NEW packages will be installed: 2021-11-21T21:45:49.544 INFO:teuthology.orchestra.run.gibba017.stdout: linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:49.544 INFO:teuthology.orchestra.run.gibba017.stdout: linux-image-5.4.0-90-generic linux-modules-5.4.0-90-generic 2021-11-21T21:45:49.545 INFO:teuthology.orchestra.run.gibba017.stdout: linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:49.545 INFO:teuthology.orchestra.run.gibba017.stdout:The following packages will be upgraded: 2021-11-21T21:45:49.545 INFO:teuthology.orchestra.run.gibba017.stdout: linux-generic linux-headers-generic linux-image-generic 2021-11-21T21:45:49.550 INFO:teuthology.orchestra.run.gibba016.stdout:The following NEW packages will be installed: 2021-11-21T21:45:50.118 INFO:teuthology.orchestra.run.gibba016.stdout: linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:50.118 INFO:teuthology.orchestra.run.gibba016.stdout: linux-image-5.4.0-90-generic linux-modules-5.4.0-90-generic 2021-11-21T21:45:50.118 INFO:teuthology.orchestra.run.gibba016.stdout: linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:50.120 INFO:teuthology.orchestra.run.gibba017.stdout:3 upgraded, 5 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:45:50.120 INFO:teuthology.orchestra.run.gibba017.stdout:Need to get 75.6 MB of archives. 2021-11-21T21:45:50.120 INFO:teuthology.orchestra.run.gibba017.stdout:After this operation, 378 MB of additional disk space will be used. 2021-11-21T21:45:50.121 INFO:teuthology.orchestra.run.gibba017.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-90-generic amd64 5.4.0-90.101 [14.8 MB] 2021-11-21T21:45:50.121 INFO:teuthology.orchestra.run.gibba036.stdout:3 upgraded, 5 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:45:50.121 INFO:teuthology.orchestra.run.gibba036.stdout:Need to get 75.6 MB of archives. 2021-11-21T21:45:50.122 INFO:teuthology.orchestra.run.gibba036.stdout:After this operation, 378 MB of additional disk space will be used. 2021-11-21T21:45:50.122 INFO:teuthology.orchestra.run.gibba036.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-90-generic amd64 5.4.0-90.101 [14.8 MB] 2021-11-21T21:45:50.123 INFO:teuthology.orchestra.run.gibba016.stdout:The following packages will be upgraded: 2021-11-21T21:45:50.123 INFO:teuthology.orchestra.run.gibba016.stdout: linux-generic linux-headers-generic linux-image-generic 2021-11-21T21:45:50.123 INFO:teuthology.orchestra.run.gibba016.stdout:3 upgraded, 5 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:45:50.123 INFO:teuthology.orchestra.run.gibba016.stdout:Need to get 75.6 MB of archives. 2021-11-21T21:45:50.124 INFO:teuthology.orchestra.run.gibba016.stdout:After this operation, 378 MB of additional disk space will be used. 2021-11-21T21:45:50.124 INFO:teuthology.orchestra.run.gibba016.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-90-generic amd64 5.4.0-90.101 [14.8 MB] 2021-11-21T21:45:50.124 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-11-21T21:45:50.125 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:45:50.165 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-11-21T21:45:50.270 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-11-21T21:45:50.271 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-11-21T21:45:50.354 INFO:teuthology.orchestra.run.gibba019.stdout:The following package was automatically installed and is no longer required: 2021-11-21T21:45:50.355 INFO:teuthology.orchestra.run.gibba019.stdout: libllvm11 2021-11-21T21:45:50.355 INFO:teuthology.orchestra.run.gibba019.stdout:Use 'sudo apt autoremove' to remove it. 2021-11-21T21:45:50.356 INFO:teuthology.orchestra.run.gibba019.stdout:The following additional packages will be installed: 2021-11-21T21:45:50.356 INFO:teuthology.orchestra.run.gibba019.stdout: linux-generic linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:50.356 INFO:teuthology.orchestra.run.gibba019.stdout: linux-headers-generic linux-image-5.4.0-90-generic 2021-11-21T21:45:50.357 INFO:teuthology.orchestra.run.gibba019.stdout: linux-modules-5.4.0-90-generic linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:50.357 INFO:teuthology.orchestra.run.gibba019.stdout:Suggested packages: 2021-11-21T21:45:50.357 INFO:teuthology.orchestra.run.gibba019.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-11-21T21:45:50.357 INFO:teuthology.orchestra.run.gibba019.stdout:Recommended packages: 2021-11-21T21:45:50.357 INFO:teuthology.orchestra.run.gibba019.stdout: thermald 2021-11-21T21:45:50.402 INFO:teuthology.orchestra.run.gibba019.stdout:The following NEW packages will be installed: 2021-11-21T21:45:50.403 INFO:teuthology.orchestra.run.gibba019.stdout: linux-headers-5.4.0-90 linux-headers-5.4.0-90-generic 2021-11-21T21:45:50.403 INFO:teuthology.orchestra.run.gibba019.stdout: linux-image-5.4.0-90-generic linux-modules-5.4.0-90-generic 2021-11-21T21:45:50.403 INFO:teuthology.orchestra.run.gibba019.stdout: linux-modules-extra-5.4.0-90-generic 2021-11-21T21:45:50.404 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages will be upgraded: 2021-11-21T21:45:50.404 INFO:teuthology.orchestra.run.gibba019.stdout: linux-generic linux-headers-generic linux-image-generic 2021-11-21T21:45:50.584 INFO:teuthology.orchestra.run.gibba019.stdout:3 upgraded, 5 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:45:50.585 INFO:teuthology.orchestra.run.gibba019.stdout:Need to get 75.6 MB of archives. 2021-11-21T21:45:50.585 INFO:teuthology.orchestra.run.gibba019.stdout:After this operation, 378 MB of additional disk space will be used. 2021-11-21T21:45:50.585 INFO:teuthology.orchestra.run.gibba019.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-90-generic amd64 5.4.0-90.101 [14.8 MB] 2021-11-21T21:45:50.751 INFO:teuthology.orchestra.run.gibba036.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-90-generic amd64 5.4.0-90.101 [8,994 kB] 2021-11-21T21:45:50.832 INFO:teuthology.orchestra.run.gibba017.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-90-generic amd64 5.4.0-90.101 [8,994 kB] 2021-11-21T21:45:50.835 INFO:teuthology.orchestra.run.gibba016.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-90-generic amd64 5.4.0-90.101 [8,994 kB] 2021-11-21T21:45:51.009 INFO:teuthology.orchestra.run.gibba036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-90-generic amd64 5.4.0-90.101 [39.4 MB] 2021-11-21T21:45:51.070 INFO:teuthology.orchestra.run.gibba016.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-90-generic amd64 5.4.0-90.101 [39.4 MB] 2021-11-21T21:45:51.070 INFO:teuthology.orchestra.run.gibba017.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-90-generic amd64 5.4.0-90.101 [39.4 MB] 2021-11-21T21:45:51.822 INFO:teuthology.orchestra.run.gibba019.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-90-generic amd64 5.4.0-90.101 [8,994 kB] 2021-11-21T21:45:52.094 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-90-generic amd64 5.4.0-90.101 [39.4 MB] 2021-11-21T21:45:52.547 INFO:teuthology.orchestra.run.gibba036.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.90.94 [1,900 B] 2021-11-21T21:45:52.549 INFO:teuthology.orchestra.run.gibba036.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.90.94 [2,580 B] 2021-11-21T21:45:52.549 INFO:teuthology.orchestra.run.gibba036.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90 all 5.4.0-90.101 [11.0 MB] 2021-11-21T21:45:52.554 INFO:teuthology.orchestra.run.gibba017.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.90.94 [1,900 B] 2021-11-21T21:45:52.554 INFO:teuthology.orchestra.run.gibba017.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.90.94 [2,580 B] 2021-11-21T21:45:52.554 INFO:teuthology.orchestra.run.gibba017.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90 all 5.4.0-90.101 [11.0 MB] 2021-11-21T21:45:52.597 INFO:teuthology.orchestra.run.gibba016.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.90.94 [1,900 B] 2021-11-21T21:45:52.598 INFO:teuthology.orchestra.run.gibba016.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.90.94 [2,580 B] 2021-11-21T21:45:52.599 INFO:teuthology.orchestra.run.gibba016.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90 all 5.4.0-90.101 [11.0 MB] 2021-11-21T21:45:52.842 INFO:teuthology.orchestra.run.gibba017.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90-generic amd64 5.4.0-90.101 [1,403 kB] 2021-11-21T21:45:52.873 INFO:teuthology.orchestra.run.gibba017.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.90.94 [2,448 B] 2021-11-21T21:45:52.947 INFO:teuthology.orchestra.run.gibba016.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90-generic amd64 5.4.0-90.101 [1,403 kB] 2021-11-21T21:45:52.987 INFO:teuthology.orchestra.run.gibba016.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.90.94 [2,448 B] 2021-11-21T21:45:53.090 INFO:teuthology.orchestra.run.gibba017.stdout:Fetched 75.6 MB in 3s (22.7 MB/s) 2021-11-21T21:45:53.128 INFO:teuthology.orchestra.run.gibba036.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90-generic amd64 5.4.0-90.101 [1,403 kB] 2021-11-21T21:45:53.165 INFO:teuthology.orchestra.run.gibba036.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.90.94 [2,448 B] 2021-11-21T21:45:53.211 INFO:teuthology.orchestra.run.gibba016.stdout:Fetched 75.6 MB in 3s (22.0 MB/s) 2021-11-21T21:45:53.234 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package linux-modules-5.4.0-90-generic. 2021-11-21T21:45:53.349 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package linux-modules-5.4.0-90-generic. 2021-11-21T21:45:53.460 INFO:teuthology.orchestra.run.gibba036.stdout:Fetched 75.6 MB in 4s (20.4 MB/s) 2021-11-21T21:45:53.519 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.90.94 [1,900 B] 2021-11-21T21:45:53.520 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.90.94 [2,580 B] 2021-11-21T21:45:53.520 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90 all 5.4.0-90.101 [11.0 MB] 2021-11-21T21:45:53.595 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package linux-modules-5.4.0-90-generic. 2021-11-21T21:45:53.784 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-90-generic amd64 5.4.0-90.101 [1,403 kB] 2021-11-21T21:45:53.800 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.90.94 [2,448 B] 2021-11-21T21:45:54.079 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 75.6 MB in 3s (22.3 MB/s) 2021-11-21T21:45:54.208 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-modules-5.4.0-90-generic. 2021-11-21T21:45:55.177 INFO:teuthology.orchestra.run.gibba017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138265 files and directories currently installed.) 2021-11-21T21:45:55.186 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../0-linux-modules-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:55.233 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:55.335 INFO:teuthology.orchestra.run.gibba016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138265 files and directories currently installed.) 2021-11-21T21:45:55.344 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../0-linux-modules-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:55.397 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:55.514 INFO:teuthology.orchestra.run.gibba036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138265 files and directories currently installed.) 2021-11-21T21:45:55.524 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../0-linux-modules-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:55.569 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:56.237 INFO:teuthology.orchestra.run.gibba019.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138265 files and directories currently installed.) 2021-11-21T21:45:56.247 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../0-linux-modules-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:56.299 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:56.608 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package linux-image-5.4.0-90-generic. 2021-11-21T21:45:56.616 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../1-linux-image-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:56.697 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:56.913 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package linux-image-5.4.0-90-generic. 2021-11-21T21:45:56.939 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../1-linux-image-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:56.952 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package linux-image-5.4.0-90-generic. 2021-11-21T21:45:56.960 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../1-linux-image-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:57.028 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:57.049 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:57.528 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-90-generic. 2021-11-21T21:45:57.553 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:57.594 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:57.714 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-image-5.4.0-90-generic. 2021-11-21T21:45:57.740 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../1-linux-image-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:57.851 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:57.891 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-90-generic. 2021-11-21T21:45:57.913 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-90-generic. 2021-11-21T21:45:57.916 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:57.938 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:57.957 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:58.005 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:45:58.750 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-90-generic. 2021-11-21T21:45:58.758 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:45:58.800 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:00.569 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../3-linux-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:00.672 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:01.014 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../3-linux-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:01.026 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:01.047 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../3-linux-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:01.150 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-image-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:01.165 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:01.166 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:01.495 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:01.520 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package linux-headers-5.4.0-90. 2021-11-21T21:46:01.520 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:01.546 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../5-linux-headers-5.4.0-90_5.4.0-90.101_all.deb ... 2021-11-21T21:46:01.578 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:01.619 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-image-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:01.644 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-image-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:01.740 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../3-linux-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:01.870 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:01.956 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package linux-headers-5.4.0-90. 2021-11-21T21:46:01.983 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../5-linux-headers-5.4.0-90_5.4.0-90.101_all.deb ... 2021-11-21T21:46:02.005 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package linux-headers-5.4.0-90. 2021-11-21T21:46:02.022 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:02.032 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../5-linux-headers-5.4.0-90_5.4.0-90.101_all.deb ... 2021-11-21T21:46:02.072 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:02.221 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:02.371 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-image-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:02.716 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-headers-5.4.0-90. 2021-11-21T21:46:02.743 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../5-linux-headers-5.4.0-90_5.4.0-90.101_all.deb ... 2021-11-21T21:46:02.782 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:04.961 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package linux-headers-5.4.0-90-generic. 2021-11-21T21:46:05.002 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../6-linux-headers-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:46:05.044 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:05.458 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package linux-headers-5.4.0-90-generic. 2021-11-21T21:46:05.498 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../6-linux-headers-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:46:05.540 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:05.625 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package linux-headers-5.4.0-90-generic. 2021-11-21T21:46:05.667 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../6-linux-headers-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:46:05.707 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:06.349 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-headers-5.4.0-90-generic. 2021-11-21T21:46:06.390 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../6-linux-headers-5.4.0-90-generic_5.4.0-90.101_amd64.deb ... 2021-11-21T21:46:06.432 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:06.620 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:06.764 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking linux-headers-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:07.096 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:07.102 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:07.210 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:07.231 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:07.247 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking linux-headers-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:07.392 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking linux-headers-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:07.595 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:07.702 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:07.732 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:07.872 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:08.053 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.90.94_amd64.deb ... 2021-11-21T21:46:08.062 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:08.188 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-headers-generic (5.4.0.90.94) ... 2021-11-21T21:46:08.198 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-headers-generic (5.4.0.90.94) over (5.4.0.88.92) ... 2021-11-21T21:46:08.314 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:08.494 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:08.562 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-headers-5.4.0-90 (5.4.0-90.101) ... 2021-11-21T21:46:08.646 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-headers-generic (5.4.0.90.94) ... 2021-11-21T21:46:08.664 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:08.687 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-modules-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:08.771 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:08.774 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-headers-generic (5.4.0.90.94) ... 2021-11-21T21:46:08.875 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:09.046 INFO:teuthology.orchestra.run.gibba017.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:09.046 INFO:teuthology.orchestra.run.gibba017.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:09.046 INFO:teuthology.orchestra.run.gibba017.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:09.047 INFO:teuthology.orchestra.run.gibba017.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:09.174 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:09.453 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-headers-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:09.490 INFO:teuthology.orchestra.run.gibba016.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:09.490 INFO:teuthology.orchestra.run.gibba016.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:09.490 INFO:teuthology.orchestra.run.gibba016.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:09.491 INFO:teuthology.orchestra.run.gibba016.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:09.571 INFO:teuthology.orchestra.run.gibba036.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:09.571 INFO:teuthology.orchestra.run.gibba036.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:09.572 INFO:teuthology.orchestra.run.gibba036.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:09.572 INFO:teuthology.orchestra.run.gibba036.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:09.604 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-headers-generic (5.4.0.90.94) ... 2021-11-21T21:46:09.642 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:09.685 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:09.731 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:09.909 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-image-generic (5.4.0.90.94) ... 2021-11-21T21:46:10.019 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up linux-generic (5.4.0.90.94) ... 2021-11-21T21:46:10.171 INFO:teuthology.orchestra.run.gibba017.stdout:Processing triggers for linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:10.311 INFO:teuthology.orchestra.run.gibba017.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-11-21T21:46:10.311 INFO:teuthology.orchestra.run.gibba017.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:10.408 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-image-generic (5.4.0.90.94) ... 2021-11-21T21:46:10.450 INFO:teuthology.orchestra.run.gibba019.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:10.450 INFO:teuthology.orchestra.run.gibba019.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:10.450 INFO:teuthology.orchestra.run.gibba019.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:10.450 INFO:teuthology.orchestra.run.gibba019.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:10.453 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-image-generic (5.4.0.90.94) ... 2021-11-21T21:46:10.509 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up linux-generic (5.4.0.90.94) ... 2021-11-21T21:46:10.580 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up linux-generic (5.4.0.90.94) ... 2021-11-21T21:46:10.599 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-modules-extra-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:10.611 INFO:teuthology.orchestra.run.gibba016.stdout:Processing triggers for linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:10.723 INFO:teuthology.orchestra.run.gibba036.stdout:Processing triggers for linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:10.770 INFO:teuthology.orchestra.run.gibba016.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-11-21T21:46:10.771 INFO:teuthology.orchestra.run.gibba016.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:10.845 INFO:teuthology.orchestra.run.gibba036.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-11-21T21:46:10.845 INFO:teuthology.orchestra.run.gibba036.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:11.416 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-image-generic (5.4.0.90.94) ... 2021-11-21T21:46:11.534 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-generic (5.4.0.90.94) ... 2021-11-21T21:46:11.670 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for linux-image-5.4.0-90-generic (5.4.0-90.101) ... 2021-11-21T21:46:11.793 INFO:teuthology.orchestra.run.gibba019.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-11-21T21:46:11.793 INFO:teuthology.orchestra.run.gibba019.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:30.163 INFO:teuthology.orchestra.run.gibba036.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-11-21T21:46:30.163 INFO:teuthology.orchestra.run.gibba036.stdout:Sourcing file `/etc/default/grub' 2021-11-21T21:46:30.171 INFO:teuthology.orchestra.run.gibba036.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:30.194 INFO:teuthology.orchestra.run.gibba036.stdout:Generating grub configuration file ... 2021-11-21T21:46:30.496 INFO:teuthology.orchestra.run.gibba017.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-11-21T21:46:30.497 INFO:teuthology.orchestra.run.gibba017.stdout:Sourcing file `/etc/default/grub' 2021-11-21T21:46:30.503 INFO:teuthology.orchestra.run.gibba017.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:30.535 INFO:teuthology.orchestra.run.gibba017.stdout:Generating grub configuration file ... 2021-11-21T21:46:30.541 INFO:teuthology.orchestra.run.gibba036.stdout:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:30.574 INFO:teuthology.orchestra.run.gibba036.stdout:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:30.896 INFO:teuthology.orchestra.run.gibba017.stdout:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:30.946 INFO:teuthology.orchestra.run.gibba017.stdout:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:31.003 INFO:teuthology.orchestra.run.gibba036.stdout:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:31.015 INFO:teuthology.orchestra.run.gibba036.stdout:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:31.127 INFO:teuthology.orchestra.run.gibba036.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:31.140 INFO:teuthology.orchestra.run.gibba036.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:31.265 INFO:teuthology.orchestra.run.gibba016.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-11-21T21:46:31.265 INFO:teuthology.orchestra.run.gibba016.stdout:Sourcing file `/etc/default/grub' 2021-11-21T21:46:31.285 INFO:teuthology.orchestra.run.gibba016.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:31.290 INFO:teuthology.orchestra.run.gibba016.stdout:Generating grub configuration file ... 2021-11-21T21:46:31.353 INFO:teuthology.orchestra.run.gibba017.stdout:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:31.365 INFO:teuthology.orchestra.run.gibba017.stdout:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:31.488 INFO:teuthology.orchestra.run.gibba017.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:31.501 INFO:teuthology.orchestra.run.gibba017.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:31.694 INFO:teuthology.orchestra.run.gibba016.stdout:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:31.749 INFO:teuthology.orchestra.run.gibba016.stdout:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:32.149 INFO:teuthology.orchestra.run.gibba019.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-11-21T21:46:32.149 INFO:teuthology.orchestra.run.gibba019.stdout:Sourcing file `/etc/default/grub' 2021-11-21T21:46:32.165 INFO:teuthology.orchestra.run.gibba019.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:32.192 INFO:teuthology.orchestra.run.gibba019.stdout:Generating grub configuration file ... 2021-11-21T21:46:32.248 INFO:teuthology.orchestra.run.gibba016.stdout:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:32.262 INFO:teuthology.orchestra.run.gibba016.stdout:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:32.387 INFO:teuthology.orchestra.run.gibba016.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:32.401 INFO:teuthology.orchestra.run.gibba016.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:32.555 INFO:teuthology.orchestra.run.gibba019.stdout:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:32.595 INFO:teuthology.orchestra.run.gibba019.stdout:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:33.049 INFO:teuthology.orchestra.run.gibba019.stdout:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:33.063 INFO:teuthology.orchestra.run.gibba019.stdout:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:33.191 INFO:teuthology.orchestra.run.gibba019.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:33.205 INFO:teuthology.orchestra.run.gibba019.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:33.464 INFO:teuthology.orchestra.run.gibba017.stdout:done 2021-11-21T21:46:33.776 INFO:teuthology.orchestra.run.gibba036.stdout:done 2021-11-21T21:46:33.778 DEBUG:teuthology.orchestra.run.gibba017:> dpkg -s linux-image-generic 2021-11-21T21:46:33.805 INFO:teuthology.orchestra.run.gibba017.stdout:Package: linux-image-generic 2021-11-21T21:46:33.805 INFO:teuthology.orchestra.run.gibba017.stdout:Status: install ok installed 2021-11-21T21:46:33.805 INFO:teuthology.orchestra.run.gibba017.stdout:Priority: optional 2021-11-21T21:46:33.806 INFO:teuthology.orchestra.run.gibba017.stdout:Section: kernel 2021-11-21T21:46:33.806 INFO:teuthology.orchestra.run.gibba017.stdout:Installed-Size: 18 2021-11-21T21:46:33.806 INFO:teuthology.orchestra.run.gibba017.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:33.807 INFO:teuthology.orchestra.run.gibba017.stdout:Architecture: amd64 2021-11-21T21:46:33.807 INFO:teuthology.orchestra.run.gibba017.stdout:Source: linux-meta 2021-11-21T21:46:33.807 INFO:teuthology.orchestra.run.gibba017.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:33.807 INFO:teuthology.orchestra.run.gibba017.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:33.808 INFO:teuthology.orchestra.run.gibba017.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:33.808 INFO:teuthology.orchestra.run.gibba017.stdout:Recommends: thermald 2021-11-21T21:46:33.808 INFO:teuthology.orchestra.run.gibba017.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:33.808 INFO:teuthology.orchestra.run.gibba017.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:33.808 INFO:teuthology.orchestra.run.gibba017.stdout: available. 2021-11-21T21:46:33.809 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-88-generic Expected: 5.4.0-90-generic 2021-11-21T21:46:33.809 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-21T21:46:33.809 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-21T21:46:33.810 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2021-11-21T21:46:33.810 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba017.front.sepia.ceph.com, path=None, version=distro) 2021-11-21T21:46:33.810 DEBUG:teuthology.orchestra.run.gibba017:> sudo apt-get clean 2021-11-21T21:46:33.883 DEBUG:teuthology.orchestra.run.gibba017:> sudo apt-get update 2021-11-21T21:46:34.051 DEBUG:teuthology.orchestra.run.gibba036:> dpkg -s linux-image-generic 2021-11-21T21:46:34.068 INFO:teuthology.orchestra.run.gibba036.stdout:Package: linux-image-generic 2021-11-21T21:46:34.068 INFO:teuthology.orchestra.run.gibba036.stdout:Status: install ok installed 2021-11-21T21:46:34.068 INFO:teuthology.orchestra.run.gibba036.stdout:Priority: optional 2021-11-21T21:46:34.068 INFO:teuthology.orchestra.run.gibba036.stdout:Section: kernel 2021-11-21T21:46:34.069 INFO:teuthology.orchestra.run.gibba036.stdout:Installed-Size: 18 2021-11-21T21:46:34.069 INFO:teuthology.orchestra.run.gibba036.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:34.069 INFO:teuthology.orchestra.run.gibba036.stdout:Architecture: amd64 2021-11-21T21:46:34.069 INFO:teuthology.orchestra.run.gibba036.stdout:Source: linux-meta 2021-11-21T21:46:34.070 INFO:teuthology.orchestra.run.gibba036.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:34.070 INFO:teuthology.orchestra.run.gibba036.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:34.070 INFO:teuthology.orchestra.run.gibba036.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:34.070 INFO:teuthology.orchestra.run.gibba036.stdout:Recommends: thermald 2021-11-21T21:46:34.071 INFO:teuthology.orchestra.run.gibba036.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:34.071 INFO:teuthology.orchestra.run.gibba036.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:34.071 INFO:teuthology.orchestra.run.gibba036.stdout: available. 2021-11-21T21:46:34.071 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-88-generic Expected: 5.4.0-90-generic 2021-11-21T21:46:34.072 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-21T21:46:34.072 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-21T21:46:34.072 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2021-11-21T21:46:34.072 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba036.front.sepia.ceph.com, path=None, version=distro) 2021-11-21T21:46:34.073 DEBUG:teuthology.orchestra.run.gibba036:> sudo apt-get clean 2021-11-21T21:46:34.096 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:46:34.096 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:46:34.167 DEBUG:teuthology.orchestra.run.gibba036:> sudo apt-get update 2021-11-21T21:46:34.170 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:46:34.253 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:46:34.390 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:46:34.391 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:46:34.402 INFO:teuthology.orchestra.run.gibba016.stdout:done 2021-11-21T21:46:34.465 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:46:34.547 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:46:34.745 DEBUG:teuthology.orchestra.run.gibba016:> dpkg -s linux-image-generic 2021-11-21T21:46:34.762 INFO:teuthology.orchestra.run.gibba016.stdout:Package: linux-image-generic 2021-11-21T21:46:34.762 INFO:teuthology.orchestra.run.gibba016.stdout:Status: install ok installed 2021-11-21T21:46:34.762 INFO:teuthology.orchestra.run.gibba016.stdout:Priority: optional 2021-11-21T21:46:34.763 INFO:teuthology.orchestra.run.gibba016.stdout:Section: kernel 2021-11-21T21:46:34.763 INFO:teuthology.orchestra.run.gibba016.stdout:Installed-Size: 18 2021-11-21T21:46:34.763 INFO:teuthology.orchestra.run.gibba016.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:34.763 INFO:teuthology.orchestra.run.gibba016.stdout:Architecture: amd64 2021-11-21T21:46:34.763 INFO:teuthology.orchestra.run.gibba016.stdout:Source: linux-meta 2021-11-21T21:46:34.764 INFO:teuthology.orchestra.run.gibba016.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:34.764 INFO:teuthology.orchestra.run.gibba016.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:34.764 INFO:teuthology.orchestra.run.gibba016.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:34.764 INFO:teuthology.orchestra.run.gibba016.stdout:Recommends: thermald 2021-11-21T21:46:34.765 INFO:teuthology.orchestra.run.gibba016.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:34.765 INFO:teuthology.orchestra.run.gibba016.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:34.765 INFO:teuthology.orchestra.run.gibba016.stdout: available. 2021-11-21T21:46:34.765 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-88-generic Expected: 5.4.0-90-generic 2021-11-21T21:46:34.766 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-21T21:46:34.766 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-21T21:46:34.766 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2021-11-21T21:46:34.766 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba016.front.sepia.ceph.com, path=None, version=distro) 2021-11-21T21:46:34.767 DEBUG:teuthology.orchestra.run.gibba016:> sudo apt-get clean 2021-11-21T21:46:34.835 DEBUG:teuthology.orchestra.run.gibba016:> sudo apt-get update 2021-11-21T21:46:35.048 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:46:35.049 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:46:35.081 INFO:teuthology.orchestra.run.gibba019.stdout:done 2021-11-21T21:46:35.123 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:46:35.158 INFO:teuthology.orchestra.run.gibba017.stdout:Reading package lists... 2021-11-21T21:46:35.169 DEBUG:teuthology.orchestra.run.gibba017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:46:35.205 INFO:teuthology.orchestra.run.gibba017.stdout:Reading package lists... 2021-11-21T21:46:35.205 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:46:35.284 INFO:teuthology.orchestra.run.gibba036.stdout:Reading package lists... 2021-11-21T21:46:35.295 DEBUG:teuthology.orchestra.run.gibba036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:46:35.306 INFO:teuthology.orchestra.run.gibba017.stdout:Building dependency tree... 2021-11-21T21:46:35.317 INFO:teuthology.orchestra.run.gibba017.stdout:Reading state information... 2021-11-21T21:46:35.331 INFO:teuthology.orchestra.run.gibba036.stdout:Reading package lists... 2021-11-21T21:46:35.385 INFO:teuthology.orchestra.run.gibba017.stdout:linux-image-generic is already the newest version (5.4.0.90.94). 2021-11-21T21:46:35.386 INFO:teuthology.orchestra.run.gibba017.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:46:35.386 INFO:teuthology.orchestra.run.gibba017.stdout: libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:46:35.386 INFO:teuthology.orchestra.run.gibba017.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:46:35.386 INFO:teuthology.orchestra.run.gibba017.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:46:35.386 INFO:teuthology.orchestra.run.gibba017.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:46:35.395 DEBUG:teuthology.orchestra.run.gibba019:> dpkg -s linux-image-generic 2021-11-21T21:46:35.400 INFO:teuthology.orchestra.run.gibba017.stdout:0 upgraded, 0 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:46:35.402 DEBUG:teuthology.orchestra.run.gibba017:> dpkg -s linux-image-generic 2021-11-21T21:46:35.437 INFO:teuthology.orchestra.run.gibba036.stdout:Building dependency tree... 2021-11-21T21:46:35.437 INFO:teuthology.orchestra.run.gibba036.stdout:Reading state information... 2021-11-21T21:46:35.455 INFO:teuthology.orchestra.run.gibba017.stdout:Package: linux-image-generic 2021-11-21T21:46:35.455 INFO:teuthology.orchestra.run.gibba017.stdout:Status: install ok installed 2021-11-21T21:46:35.456 INFO:teuthology.orchestra.run.gibba017.stdout:Priority: optional 2021-11-21T21:46:35.456 INFO:teuthology.orchestra.run.gibba017.stdout:Section: kernel 2021-11-21T21:46:35.456 INFO:teuthology.orchestra.run.gibba017.stdout:Installed-Size: 18 2021-11-21T21:46:35.456 INFO:teuthology.orchestra.run.gibba017.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:35.457 INFO:teuthology.orchestra.run.gibba017.stdout:Architecture: amd64 2021-11-21T21:46:35.457 INFO:teuthology.orchestra.run.gibba017.stdout:Source: linux-meta 2021-11-21T21:46:35.457 INFO:teuthology.orchestra.run.gibba017.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:35.457 INFO:teuthology.orchestra.run.gibba017.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:35.457 INFO:teuthology.orchestra.run.gibba017.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:35.458 INFO:teuthology.orchestra.run.gibba017.stdout:Recommends: thermald 2021-11-21T21:46:35.458 INFO:teuthology.orchestra.run.gibba017.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:35.458 INFO:teuthology.orchestra.run.gibba017.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:35.458 INFO:teuthology.orchestra.run.gibba017.stdout: available. 2021-11-21T21:46:35.459 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:46:35.461 INFO:teuthology.orchestra.run.gibba019.stdout:Package: linux-image-generic 2021-11-21T21:46:35.461 INFO:teuthology.orchestra.run.gibba019.stdout:Status: install ok installed 2021-11-21T21:46:35.461 INFO:teuthology.orchestra.run.gibba019.stdout:Priority: optional 2021-11-21T21:46:35.461 INFO:teuthology.orchestra.run.gibba019.stdout:Section: kernel 2021-11-21T21:46:35.462 INFO:teuthology.orchestra.run.gibba019.stdout:Installed-Size: 18 2021-11-21T21:46:35.462 INFO:teuthology.orchestra.run.gibba019.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:35.462 INFO:teuthology.orchestra.run.gibba019.stdout:Architecture: amd64 2021-11-21T21:46:35.462 INFO:teuthology.orchestra.run.gibba019.stdout:Source: linux-meta 2021-11-21T21:46:35.462 INFO:teuthology.orchestra.run.gibba019.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:35.463 INFO:teuthology.orchestra.run.gibba019.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:35.463 INFO:teuthology.orchestra.run.gibba019.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:35.463 INFO:teuthology.orchestra.run.gibba019.stdout:Recommends: thermald 2021-11-21T21:46:35.463 INFO:teuthology.orchestra.run.gibba019.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:35.464 INFO:teuthology.orchestra.run.gibba019.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:35.464 INFO:teuthology.orchestra.run.gibba019.stdout: available. 2021-11-21T21:46:35.464 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-88-generic Expected: 5.4.0-90-generic 2021-11-21T21:46:35.464 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-21T21:46:35.465 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-21T21:46:35.465 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2021-11-21T21:46:35.465 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba019.front.sepia.ceph.com, path=None, version=distro) 2021-11-21T21:46:35.465 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get clean 2021-11-21T21:46:35.469 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.Xk4NZahyIb 2021-11-21T21:46:35.469 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Xk4NZahyIb 2021-11-21T21:46:35.515 INFO:teuthology.orchestra.run.gibba036.stdout:linux-image-generic is already the newest version (5.4.0.90.94). 2021-11-21T21:46:35.515 INFO:teuthology.orchestra.run.gibba036.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:46:35.515 INFO:teuthology.orchestra.run.gibba036.stdout: libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:46:35.516 INFO:teuthology.orchestra.run.gibba036.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:46:35.516 INFO:teuthology.orchestra.run.gibba036.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:46:35.516 INFO:teuthology.orchestra.run.gibba036.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:46:35.523 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.Xk4NZahyIb 2021-11-21T21:46:35.531 INFO:teuthology.orchestra.run.gibba036.stdout:0 upgraded, 0 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:46:35.532 DEBUG:teuthology.orchestra.run.gibba036:> dpkg -s linux-image-generic 2021-11-21T21:46:35.539 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get update 2021-11-21T21:46:35.542 INFO:teuthology.orchestra.run.gibba036.stdout:Package: linux-image-generic 2021-11-21T21:46:35.542 INFO:teuthology.orchestra.run.gibba036.stdout:Status: install ok installed 2021-11-21T21:46:35.543 INFO:teuthology.orchestra.run.gibba036.stdout:Priority: optional 2021-11-21T21:46:35.543 INFO:teuthology.orchestra.run.gibba036.stdout:Section: kernel 2021-11-21T21:46:35.543 INFO:teuthology.orchestra.run.gibba036.stdout:Installed-Size: 18 2021-11-21T21:46:35.543 INFO:teuthology.orchestra.run.gibba036.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:35.544 INFO:teuthology.orchestra.run.gibba036.stdout:Architecture: amd64 2021-11-21T21:46:35.544 INFO:teuthology.orchestra.run.gibba036.stdout:Source: linux-meta 2021-11-21T21:46:35.544 INFO:teuthology.orchestra.run.gibba036.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:35.544 INFO:teuthology.orchestra.run.gibba036.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:35.544 INFO:teuthology.orchestra.run.gibba036.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:35.545 INFO:teuthology.orchestra.run.gibba036.stdout:Recommends: thermald 2021-11-21T21:46:35.545 INFO:teuthology.orchestra.run.gibba036.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:35.545 INFO:teuthology.orchestra.run.gibba036.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:35.545 INFO:teuthology.orchestra.run.gibba036.stdout: available. 2021-11-21T21:46:35.546 DEBUG:teuthology.orchestra.run.gibba036:> mktemp 2021-11-21T21:46:35.563 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.Xk4NZahyIb is 10KB 2021-11-21T21:46:35.581 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.Xk4NZahyIb 2021-11-21T21:46:35.589 DEBUG:teuthology.orchestra.run.gibba017:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:35.591 INFO:teuthology.orchestra.run.gibba036.stdout:/tmp/tmp.oof3hv9WNq 2021-11-21T21:46:35.592 DEBUG:teuthology.orchestra.run.gibba036:> sudo cp /boot/grub/grub.cfg /tmp/tmp.oof3hv9WNq 2021-11-21T21:46:35.613 DEBUG:teuthology.orchestra.run.gibba036:> sudo chmod 0666 /tmp/tmp.oof3hv9WNq 2021-11-21T21:46:35.649 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:46:35.650 DEBUG:teuthology.orchestra.run.gibba017:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-11-21T21:46:35.650 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:35.704 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-90-generic 2021-11-21T21:46:35.705 DEBUG:teuthology.orchestra.run.gibba017:> sudo update-grub 2021-11-21T21:46:35.710 DEBUG:teuthology.orchestra.remote:gibba036:/tmp/tmp.oof3hv9WNq is 10KB 2021-11-21T21:46:35.730 DEBUG:teuthology.orchestra.run.gibba036:> rm -fr /tmp/tmp.oof3hv9WNq 2021-11-21T21:46:35.739 DEBUG:teuthology.orchestra.run.gibba036:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:35.755 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:46:35.755 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:46:35.803 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:46:35.804 DEBUG:teuthology.orchestra.run.gibba036:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-11-21T21:46:35.804 DEBUG:teuthology.orchestra.run.gibba036:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:35.829 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:46:35.853 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-90-generic 2021-11-21T21:46:35.854 DEBUG:teuthology.orchestra.run.gibba036:> sudo update-grub 2021-11-21T21:46:35.911 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:46:36.221 INFO:teuthology.orchestra.run.gibba016.stdout:Reading package lists... 2021-11-21T21:46:36.233 DEBUG:teuthology.orchestra.run.gibba016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:46:36.269 INFO:teuthology.orchestra.run.gibba016.stdout:Reading package lists... 2021-11-21T21:46:36.373 INFO:teuthology.orchestra.run.gibba016.stdout:Building dependency tree... 2021-11-21T21:46:36.374 INFO:teuthology.orchestra.run.gibba016.stdout:Reading state information... 2021-11-21T21:46:36.453 INFO:teuthology.orchestra.run.gibba016.stdout:linux-image-generic is already the newest version (5.4.0.90.94). 2021-11-21T21:46:36.454 INFO:teuthology.orchestra.run.gibba016.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:46:36.454 INFO:teuthology.orchestra.run.gibba016.stdout: libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:46:36.454 INFO:teuthology.orchestra.run.gibba016.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:46:36.454 INFO:teuthology.orchestra.run.gibba016.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:46:36.455 INFO:teuthology.orchestra.run.gibba016.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:46:36.468 INFO:teuthology.orchestra.run.gibba016.stdout:0 upgraded, 0 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:46:36.469 DEBUG:teuthology.orchestra.run.gibba016:> dpkg -s linux-image-generic 2021-11-21T21:46:36.524 INFO:teuthology.orchestra.run.gibba016.stdout:Package: linux-image-generic 2021-11-21T21:46:36.524 INFO:teuthology.orchestra.run.gibba016.stdout:Status: install ok installed 2021-11-21T21:46:36.525 INFO:teuthology.orchestra.run.gibba016.stdout:Priority: optional 2021-11-21T21:46:36.525 INFO:teuthology.orchestra.run.gibba016.stdout:Section: kernel 2021-11-21T21:46:36.525 INFO:teuthology.orchestra.run.gibba016.stdout:Installed-Size: 18 2021-11-21T21:46:36.526 INFO:teuthology.orchestra.run.gibba016.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:36.526 INFO:teuthology.orchestra.run.gibba016.stdout:Architecture: amd64 2021-11-21T21:46:36.526 INFO:teuthology.orchestra.run.gibba016.stdout:Source: linux-meta 2021-11-21T21:46:36.527 INFO:teuthology.orchestra.run.gibba016.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:36.527 INFO:teuthology.orchestra.run.gibba016.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:36.527 INFO:teuthology.orchestra.run.gibba016.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:36.527 INFO:teuthology.orchestra.run.gibba016.stdout:Recommends: thermald 2021-11-21T21:46:36.527 INFO:teuthology.orchestra.run.gibba016.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:36.528 INFO:teuthology.orchestra.run.gibba016.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:36.528 INFO:teuthology.orchestra.run.gibba016.stdout: available. 2021-11-21T21:46:36.528 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:46:36.532 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.e2WvNkqHCQ 2021-11-21T21:46:36.533 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /boot/grub/grub.cfg /tmp/tmp.e2WvNkqHCQ 2021-11-21T21:46:36.540 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.e2WvNkqHCQ 2021-11-21T21:46:36.617 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.e2WvNkqHCQ is 10KB 2021-11-21T21:46:36.633 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.e2WvNkqHCQ 2021-11-21T21:46:36.638 DEBUG:teuthology.orchestra.run.gibba016:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:36.692 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:46:36.693 DEBUG:teuthology.orchestra.run.gibba016:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-11-21T21:46:36.693 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:36.740 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-90-generic 2021-11-21T21:46:36.741 DEBUG:teuthology.orchestra.run.gibba016:> sudo update-grub 2021-11-21T21:46:36.769 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-11-21T21:46:36.781 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-11-21T21:46:36.816 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-11-21T21:46:36.917 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-11-21T21:46:36.917 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-11-21T21:46:36.994 INFO:teuthology.orchestra.run.gibba019.stdout:linux-image-generic is already the newest version (5.4.0.90.94). 2021-11-21T21:46:36.994 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:46:36.995 INFO:teuthology.orchestra.run.gibba019.stdout: libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:46:36.995 INFO:teuthology.orchestra.run.gibba019.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:46:36.995 INFO:teuthology.orchestra.run.gibba019.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:46:36.995 INFO:teuthology.orchestra.run.gibba019.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:46:37.009 INFO:teuthology.orchestra.run.gibba019.stdout:0 upgraded, 0 newly installed, 0 to remove and 65 not upgraded. 2021-11-21T21:46:37.011 DEBUG:teuthology.orchestra.run.gibba019:> dpkg -s linux-image-generic 2021-11-21T21:46:37.064 INFO:teuthology.orchestra.run.gibba019.stdout:Package: linux-image-generic 2021-11-21T21:46:37.064 INFO:teuthology.orchestra.run.gibba019.stdout:Status: install ok installed 2021-11-21T21:46:37.065 INFO:teuthology.orchestra.run.gibba019.stdout:Priority: optional 2021-11-21T21:46:37.065 INFO:teuthology.orchestra.run.gibba019.stdout:Section: kernel 2021-11-21T21:46:37.065 INFO:teuthology.orchestra.run.gibba019.stdout:Installed-Size: 18 2021-11-21T21:46:37.065 INFO:teuthology.orchestra.run.gibba019.stdout:Maintainer: Ubuntu Kernel Team 2021-11-21T21:46:37.065 INFO:teuthology.orchestra.run.gibba019.stdout:Architecture: amd64 2021-11-21T21:46:37.066 INFO:teuthology.orchestra.run.gibba019.stdout:Source: linux-meta 2021-11-21T21:46:37.066 INFO:teuthology.orchestra.run.gibba019.stdout:Version: 5.4.0.90.94 2021-11-21T21:46:37.066 INFO:teuthology.orchestra.run.gibba019.stdout:Provides: virtualbox-guest-modules (= 6.1.26-dfsg-3~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.12) 2021-11-21T21:46:37.067 INFO:teuthology.orchestra.run.gibba019.stdout:Depends: linux-image-5.4.0-90-generic, linux-modules-extra-5.4.0-90-generic, linux-firmware, intel-microcode, amd64-microcode 2021-11-21T21:46:37.067 INFO:teuthology.orchestra.run.gibba019.stdout:Recommends: thermald 2021-11-21T21:46:37.067 INFO:teuthology.orchestra.run.gibba019.stdout:Description: Generic Linux kernel image 2021-11-21T21:46:37.067 INFO:teuthology.orchestra.run.gibba019.stdout: This package will always depend on the latest generic kernel image 2021-11-21T21:46:37.068 INFO:teuthology.orchestra.run.gibba019.stdout: available. 2021-11-21T21:46:37.068 DEBUG:teuthology.orchestra.run.gibba019:> mktemp 2021-11-21T21:46:37.072 INFO:teuthology.orchestra.run.gibba019.stdout:/tmp/tmp.dyIXR7PQ8Q 2021-11-21T21:46:37.072 DEBUG:teuthology.orchestra.run.gibba019:> sudo cp /boot/grub/grub.cfg /tmp/tmp.dyIXR7PQ8Q 2021-11-21T21:46:37.106 INFO:teuthology.orchestra.run.gibba017.stderr:Sourcing file `/etc/default/grub' 2021-11-21T21:46:37.108 INFO:teuthology.orchestra.run.gibba017.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:37.125 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 0666 /tmp/tmp.dyIXR7PQ8Q 2021-11-21T21:46:37.127 INFO:teuthology.orchestra.run.gibba017.stderr:Generating grub configuration file ... 2021-11-21T21:46:37.153 INFO:teuthology.orchestra.run.gibba036.stderr:Sourcing file `/etc/default/grub' 2021-11-21T21:46:37.155 DEBUG:teuthology.orchestra.remote:gibba019:/tmp/tmp.dyIXR7PQ8Q is 10KB 2021-11-21T21:46:37.161 INFO:teuthology.orchestra.run.gibba036.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:37.177 INFO:teuthology.orchestra.run.gibba036.stderr:Generating grub configuration file ... 2021-11-21T21:46:37.209 DEBUG:teuthology.orchestra.run.gibba019:> rm -fr /tmp/tmp.dyIXR7PQ8Q 2021-11-21T21:46:37.215 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:37.235 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:46:37.235 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-11-21T21:46:37.235 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-11-21T21:46:37.306 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-90-generic 2021-11-21T21:46:37.306 DEBUG:teuthology.orchestra.run.gibba019:> sudo update-grub 2021-11-21T21:46:37.435 INFO:teuthology.orchestra.run.gibba017.stderr:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:37.449 INFO:teuthology.orchestra.run.gibba017.stderr:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:37.456 INFO:teuthology.orchestra.run.gibba036.stderr:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:37.470 INFO:teuthology.orchestra.run.gibba036.stderr:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:37.870 INFO:teuthology.orchestra.run.gibba017.stderr:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:37.884 INFO:teuthology.orchestra.run.gibba017.stderr:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:37.934 INFO:teuthology.orchestra.run.gibba016.stderr:Sourcing file `/etc/default/grub' 2021-11-21T21:46:37.935 INFO:teuthology.orchestra.run.gibba036.stderr:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:37.945 INFO:teuthology.orchestra.run.gibba016.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:37.948 INFO:teuthology.orchestra.run.gibba036.stderr:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:37.973 INFO:teuthology.orchestra.run.gibba016.stderr:Generating grub configuration file ... 2021-11-21T21:46:37.999 INFO:teuthology.orchestra.run.gibba017.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:38.013 INFO:teuthology.orchestra.run.gibba017.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:38.062 INFO:teuthology.orchestra.run.gibba036.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:38.076 INFO:teuthology.orchestra.run.gibba036.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:38.270 INFO:teuthology.orchestra.run.gibba016.stderr:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:38.284 INFO:teuthology.orchestra.run.gibba016.stderr:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:38.584 INFO:teuthology.orchestra.run.gibba019.stderr:Sourcing file `/etc/default/grub' 2021-11-21T21:46:38.589 INFO:teuthology.orchestra.run.gibba019.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-11-21T21:46:38.617 INFO:teuthology.orchestra.run.gibba019.stderr:Generating grub configuration file ... 2021-11-21T21:46:38.753 INFO:teuthology.orchestra.run.gibba016.stderr:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:38.768 INFO:teuthology.orchestra.run.gibba016.stderr:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:38.889 INFO:teuthology.orchestra.run.gibba017.stderr:done 2021-11-21T21:46:38.890 DEBUG:teuthology.orchestra.run.gibba017:> sudo shutdown -r now 2021-11-21T21:46:38.891 INFO:teuthology.orchestra.run.gibba016.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:38.906 INFO:teuthology.orchestra.run.gibba016.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:38.952 INFO:teuthology.orchestra.run.gibba019.stderr:Found linux image: /boot/vmlinuz-5.4.0-90-generic 2021-11-21T21:46:38.967 INFO:teuthology.orchestra.run.gibba019.stderr:Found initrd image: /boot/initrd.img-5.4.0-90-generic 2021-11-21T21:46:39.468 INFO:teuthology.orchestra.run.gibba019.stderr:Found linux image: /boot/vmlinuz-5.4.0-88-generic 2021-11-21T21:46:39.483 INFO:teuthology.orchestra.run.gibba019.stderr:Found initrd image: /boot/initrd.img-5.4.0-88-generic 2021-11-21T21:46:39.609 INFO:teuthology.orchestra.run.gibba019.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-11-21T21:46:39.624 INFO:teuthology.orchestra.run.gibba019.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-11-21T21:46:39.769 INFO:teuthology.orchestra.run.gibba036.stderr:done 2021-11-21T21:46:39.771 DEBUG:teuthology.orchestra.run.gibba036:> sudo shutdown -r now 2021-11-21T21:46:39.822 INFO:teuthology.orchestra.run.gibba016.stderr:done 2021-11-21T21:46:39.824 DEBUG:teuthology.orchestra.run.gibba016:> sudo shutdown -r now 2021-11-21T21:46:40.502 INFO:teuthology.orchestra.run.gibba019.stderr:done 2021-11-21T21:46:40.504 DEBUG:teuthology.orchestra.run.gibba019:> sudo shutdown -r now 2021-11-21T21:47:08.919 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2021-11-21T21:47:08.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba017.front.sepia.ceph.com' 2021-11-21T21:47:08.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:47:09.774 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2021-11-21T21:47:09.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba036.front.sepia.ceph.com' 2021-11-21T21:47:09.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:47:09.827 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2021-11-21T21:47:09.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba016.front.sepia.ceph.com' 2021-11-21T21:47:09.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:47:10.506 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2021-11-21T21:47:10.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba019.front.sepia.ceph.com' 2021-11-21T21:47:10.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:48:27.363 DEBUG:teuthology.orchestra.remote:timed out 2021-11-21T21:48:28.332 DEBUG:teuthology.orchestra.remote:timed out 2021-11-21T21:48:28.333 DEBUG:teuthology.orchestra.remote:timed out 2021-11-21T21:48:28.843 DEBUG:teuthology.orchestra.remote:timed out 2021-11-21T21:48:57.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba017.front.sepia.ceph.com' 2021-11-21T21:48:57.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:48:57.692 DEBUG:teuthology.orchestra.run.gibba017:> true 2021-11-21T21:48:58.198 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@gibba017.front.sepia.ceph.com' 2021-11-21T21:48:58.198 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-90-generic"... 2021-11-21T21:48:58.199 DEBUG:teuthology.orchestra.run.gibba017:> uname -r 2021-11-21T21:48:58.207 INFO:teuthology.orchestra.run.gibba017.stdout:5.4.0-90-generic 2021-11-21T21:48:58.207 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-90-generic vs 5.4.0-90-generic 2021-11-21T21:48:58.207 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-11-21T21:48:58.208 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-11-21T21:48:58.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba036.front.sepia.ceph.com' 2021-11-21T21:48:58.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:48:58.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba016.front.sepia.ceph.com' 2021-11-21T21:48:58.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:48:58.635 DEBUG:teuthology.orchestra.run.gibba036:> true 2021-11-21T21:48:58.688 DEBUG:teuthology.orchestra.run.gibba016:> true 2021-11-21T21:48:58.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba019.front.sepia.ceph.com' 2021-11-21T21:48:58.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:48:59.141 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@gibba036.front.sepia.ceph.com' 2021-11-21T21:48:59.142 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.4.0-90-generic"... 2021-11-21T21:48:59.142 DEBUG:teuthology.orchestra.run.gibba036:> uname -r 2021-11-21T21:48:59.151 INFO:teuthology.orchestra.run.gibba036.stdout:5.4.0-90-generic 2021-11-21T21:48:59.152 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-90-generic vs 5.4.0-90-generic 2021-11-21T21:48:59.152 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-11-21T21:48:59.152 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-11-21T21:48:59.155 DEBUG:teuthology.orchestra.run.gibba019:> true 2021-11-21T21:48:59.192 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@gibba016.front.sepia.ceph.com' 2021-11-21T21:48:59.192 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-90-generic"... 2021-11-21T21:48:59.192 DEBUG:teuthology.orchestra.run.gibba016:> uname -r 2021-11-21T21:48:59.209 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2021-11-21T21:48:59.210 DEBUG:teuthology.orchestra.run.gibba017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-11-21T21:48:59.241 INFO:teuthology.orchestra.run.gibba016.stdout:5.4.0-90-generic 2021-11-21T21:48:59.241 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-90-generic vs 5.4.0-90-generic 2021-11-21T21:48:59.242 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-11-21T21:48:59.242 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-11-21T21:48:59.278 INFO:teuthology.orchestra.run.gibba017.stdout:ttyS1 2021-11-21T21:48:59.322 DEBUG:teuthology.parallel:result is None 2021-11-21T21:48:59.659 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@gibba019.front.sepia.ceph.com' 2021-11-21T21:48:59.660 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.4.0-90-generic"... 2021-11-21T21:48:59.660 DEBUG:teuthology.orchestra.run.gibba019:> uname -r 2021-11-21T21:48:59.668 INFO:teuthology.orchestra.run.gibba019.stdout:5.4.0-90-generic 2021-11-21T21:48:59.668 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-90-generic vs 5.4.0-90-generic 2021-11-21T21:48:59.669 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-11-21T21:48:59.669 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-11-21T21:49:00.153 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2021-11-21T21:49:00.153 DEBUG:teuthology.orchestra.run.gibba036:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-11-21T21:49:00.221 INFO:teuthology.orchestra.run.gibba036.stdout:ttyS1 2021-11-21T21:49:00.243 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2021-11-21T21:49:00.243 DEBUG:teuthology.orchestra.run.gibba016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-11-21T21:49:00.263 DEBUG:teuthology.parallel:result is None 2021-11-21T21:49:00.295 INFO:teuthology.orchestra.run.gibba016.stdout:ttyS1 2021-11-21T21:49:00.344 DEBUG:teuthology.parallel:result is None 2021-11-21T21:49:00.670 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2021-11-21T21:49:00.671 DEBUG:teuthology.orchestra.run.gibba019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-11-21T21:49:00.729 INFO:teuthology.orchestra.run.gibba019.stdout:ttyS1 2021-11-21T21:49:00.763 DEBUG:teuthology.parallel:result is None 2021-11-21T21:49:00.764 INFO:teuthology.run_tasks:Running task internal.base... 2021-11-21T21:49:00.783 INFO:teuthology.task.internal:Creating test directory... 2021-11-21T21:49:00.784 DEBUG:teuthology.orchestra.run.gibba016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-11-21T21:49:00.788 DEBUG:teuthology.orchestra.run.gibba017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-11-21T21:49:00.791 DEBUG:teuthology.orchestra.run.gibba019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-11-21T21:49:00.794 DEBUG:teuthology.orchestra.run.gibba036:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-11-21T21:49:00.803 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-11-21T21:49:00.826 INFO:teuthology.run_tasks:Running task internal.archive... 2021-11-21T21:49:00.925 INFO:teuthology.task.internal:Creating archive directory... 2021-11-21T21:49:00.926 DEBUG:teuthology.orchestra.run.gibba016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-11-21T21:49:00.929 DEBUG:teuthology.orchestra.run.gibba017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-11-21T21:49:00.933 DEBUG:teuthology.orchestra.run.gibba019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-11-21T21:49:00.936 DEBUG:teuthology.orchestra.run.gibba036:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-11-21T21:49:00.988 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-11-21T21:49:01.025 INFO:teuthology.task.internal:Enabling coredump saving... 2021-11-21T21:49:01.026 DEBUG:teuthology.orchestra.run.gibba016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-11-21T21:49:01.029 DEBUG:teuthology.orchestra.run.gibba017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-11-21T21:49:01.033 DEBUG:teuthology.orchestra.run.gibba019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-11-21T21:49:01.036 DEBUG:teuthology.orchestra.run.gibba036:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-11-21T21:49:01.051 INFO:teuthology.orchestra.run.gibba017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.051 INFO:teuthology.orchestra.run.gibba016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.054 INFO:teuthology.orchestra.run.gibba017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.055 INFO:teuthology.orchestra.run.gibba016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.059 INFO:teuthology.orchestra.run.gibba019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.064 INFO:teuthology.orchestra.run.gibba036.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.067 INFO:teuthology.orchestra.run.gibba036.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.069 INFO:teuthology.orchestra.run.gibba019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-21T21:49:01.069 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-11-21T21:49:01.117 INFO:teuthology.task.internal:Configuring sudo... 2021-11-21T21:49:01.117 DEBUG:teuthology.orchestra.run.gibba016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-11-21T21:49:01.120 DEBUG:teuthology.orchestra.run.gibba017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-11-21T21:49:01.123 DEBUG:teuthology.orchestra.run.gibba019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-11-21T21:49:01.126 DEBUG:teuthology.orchestra.run.gibba036:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-11-21T21:49:01.136 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-11-21T21:49:01.211 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-11-21T21:49:01.212 DEBUG:teuthology.orchestra.run.gibba016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-11-21T21:49:01.215 DEBUG:teuthology.orchestra.run.gibba017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-11-21T21:49:01.219 DEBUG:teuthology.orchestra.run.gibba019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-11-21T21:49:01.222 DEBUG:teuthology.orchestra.run.gibba036:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-11-21T21:49:01.231 DEBUG:teuthology.orchestra.run.gibba016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-21T21:49:01.241 DEBUG:teuthology.orchestra.run.gibba016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-21T21:49:01.294 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:49:01.295 DEBUG:teuthology.orchestra.run.gibba016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-11-21T21:49:01.320 DEBUG:teuthology.orchestra.run.gibba017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-21T21:49:01.330 DEBUG:teuthology.orchestra.run.gibba017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-21T21:49:01.335 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:49:01.335 DEBUG:teuthology.orchestra.run.gibba017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-11-21T21:49:01.399 DEBUG:teuthology.orchestra.run.gibba019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-21T21:49:01.409 DEBUG:teuthology.orchestra.run.gibba019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-21T21:49:01.419 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:49:01.420 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-11-21T21:49:01.487 DEBUG:teuthology.orchestra.run.gibba036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-21T21:49:01.497 DEBUG:teuthology.orchestra.run.gibba036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-21T21:49:01.501 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:49:01.501 DEBUG:teuthology.orchestra.run.gibba036:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-11-21T21:49:01.516 DEBUG:teuthology.orchestra.run.gibba016:> sudo service rsyslog restart 2021-11-21T21:49:01.520 DEBUG:teuthology.orchestra.run.gibba017:> sudo service rsyslog restart 2021-11-21T21:49:01.523 DEBUG:teuthology.orchestra.run.gibba019:> sudo service rsyslog restart 2021-11-21T21:49:01.526 DEBUG:teuthology.orchestra.run.gibba036:> sudo service rsyslog restart 2021-11-21T21:49:01.603 INFO:teuthology.run_tasks:Running task internal.timer... 2021-11-21T21:49:01.614 INFO:teuthology.task.internal:Starting timer... 2021-11-21T21:49:01.614 INFO:teuthology.run_tasks:Running task pcp... 2021-11-21T21:49:01.679 INFO:teuthology.run_tasks:Running task selinux... 2021-11-21T21:49:01.744 DEBUG:teuthology.task.selinux:Excluding gibba016: OS 'ubuntu' does not support SELinux 2021-11-21T21:49:01.745 DEBUG:teuthology.task.selinux:Excluding gibba017: OS 'ubuntu' does not support SELinux 2021-11-21T21:49:01.745 DEBUG:teuthology.task.selinux:Excluding gibba019: OS 'ubuntu' does not support SELinux 2021-11-21T21:49:01.745 DEBUG:teuthology.task.selinux:Excluding gibba036: OS 'ubuntu' does not support SELinux 2021-11-21T21:49:01.746 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-11-21T21:49:01.746 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-11-21T21:49:01.746 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-11-21T21:49:01.747 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-11-21T21:49:01.836 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-11-21T21:49:01.837 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-11-21T21:49:01.855 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2021-11-21T21:49:01.858 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit gibba016.front.sepia.ceph.com,gibba017.front.sepia.ceph.com,gibba019.front.sepia.ceph.com,gibba036.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-11-21T21:49:02.266 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-11-21T21:49:03.072 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-11-21T21:49:06.085 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-11-21T21:49:06.220 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:06 +0000 (0:00:00.155) 0:00:00.155 ******* 2021-11-21T21:49:06.235 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:06 +0000 (0:00:00.015) 0:00:00.171 ******* 2021-11-21T21:49:06.251 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:06 +0000 (0:00:00.015) 0:00:00.186 ******* 2021-11-21T21:49:06.269 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:06 +0000 (0:00:00.017) 0:00:00.204 ******* 2021-11-21T21:49:08.145 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-11-21T21:49:08.147 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba036.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba036.front.sepia.ceph.com] 2021-11-21T21:49:08.214 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:01.945) 0:00:02.150 ******* 2021-11-21T21:49:08.231 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba019.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba019.front.sepia.ceph.com] 2021-11-21T21:49:08.244 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba017.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba017.front.sepia.ceph.com] 2021-11-21T21:49:08.258 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba016.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba016.front.sepia.ceph.com] 2021-11-21T21:49:08.337 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.122) 0:00:02.272 ******* 2021-11-21T21:49:08.362 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.024) 0:00:02.297 ******* 2021-11-21T21:49:08.385 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.023) 0:00:02.320 ******* 2021-11-21T21:49:08.394 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-11-21T21:49:08.395 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:49:08.426 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.040) 0:00:02.361 ******* 2021-11-21T21:49:08.467 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:49:08.491 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.065) 0:00:02.427 ******* 2021-11-21T21:49:08.498 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:49:08.500 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:49:08.528 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.036) 0:00:02.464 ******* 2021-11-21T21:49:08.551 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.023) 0:00:02.487 ******* 2021-11-21T21:49:08.937 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-11-21T21:49:08.939 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} ok: [gibba016.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-11-21T21:49:08.941 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-11-21T21:49:08.963 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.412) 0:00:02.899 ******* 2021-11-21T21:49:08.986 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:08 +0000 (0:00:00.021) 0:00:02.921 ******* 2021-11-21T21:49:09.010 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.024) 0:00:02.945 ******* 2021-11-21T21:49:09.020 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-11-21T21:49:09.057 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.047) 0:00:02.993 ******* 2021-11-21T21:49:09.068 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-11-21T21:49:09.070 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:09.095 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.037) 0:00:03.031 ******* 2021-11-21T21:49:09.124 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.028) 0:00:03.059 ******* 2021-11-21T21:49:09.132 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:09.133 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:09.158 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.034) 0:00:03.094 ******* 2021-11-21T21:49:09.181 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.022) 0:00:03.117 ******* 2021-11-21T21:49:09.284 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-11-21T21:49:09.285 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [gibba017.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:49:09.309 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.128) 0:00:03.245 ******* 2021-11-21T21:49:09.332 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.022) 0:00:03.267 ******* 2021-11-21T21:49:09.339 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:49:09.340 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:49:09.365 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.033) 0:00:03.301 ******* 2021-11-21T21:49:09.387 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:09 +0000 (0:00:00.021) 0:00:03.323 ******* 2021-11-21T21:49:10.126 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-11-21T21:49:10.128 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "size": 182, "state": "file", "uid": 0} 2021-11-21T21:49:10.151 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:10 +0000 (0:00:00.764) 0:00:04.087 ******* 2021-11-21T21:49:10.160 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "size": 182, "state": "file", "uid": 0} 2021-11-21T21:49:10.161 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "size": 182, "state": "file", "uid": 0} 2021-11-21T21:49:10.163 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "size": 182, "state": "file", "uid": 0} 2021-11-21T21:49:10.188 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:10 +0000 (0:00:00.037) 0:00:04.124 ******* 2021-11-21T21:49:10.216 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:10 +0000 (0:00:00.027) 0:00:04.151 ******* 2021-11-21T21:49:10.239 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:10 +0000 (0:00:00.023) 0:00:04.174 ******* 2021-11-21T21:49:10.628 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:10.630 INFO:teuthology.task.ansible.out:TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoH 2021-11-21T21:49:10.632 INFO:teuthology.task.ansible.out:Dq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-11-21T21:49:10.658 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-11-21T21:49:10.661 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-11-21T21:49:10.694 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-11-21T21:49:10.696 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.c 2021-11-21T21:49:10.698 INFO:teuthology.task.ansible.out:om\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ak 2021-11-21T21:49:10.700 INFO:teuthology.task.ansible.out:lap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-11-21T21:49:10.809 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** 2021-11-21T21:49:10.811 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2021-11-21T21:49:10.814 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-11-21T21:49:10.828 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-11-21T21:49:10.830 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:10 +0000 (0:00:00.589) 0:00:04.764 ******* 2021-11-21T21:49:11.325 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] 2021-11-21T21:49:11.338 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] 2021-11-21T21:49:11.349 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] 2021-11-21T21:49:11.370 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] 2021-11-21T21:49:11.390 INFO:teuthology.task.ansible.out: TASK [set_fact] **************************************************************** 2021-11-21T21:49:11.391 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:11 +0000 (0:00:00.561) 0:00:05.326 ******* 2021-11-21T21:49:11.417 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-11-21T21:49:11.439 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-11-21T21:49:11.460 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-11-21T21:49:11.471 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-11-21T21:49:11.476 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-11-21T21:49:11.629 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:11 +0000 (0:00:00.238) 0:00:05.564 ******* 2021-11-21T21:49:11.650 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:11 +0000 (0:00:00.021) 0:00:05.586 ******* 2021-11-21T21:49:11.672 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:11 +0000 (0:00:00.021) 0:00:05.607 ******* 2021-11-21T21:49:11.693 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:11 +0000 (0:00:00.021) 0:00:05.629 ******* 2021-11-21T21:49:12.264 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba016.front.sepia.ceph.com] 2021-11-21T21:49:12.325 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-11-21T21:49:12.327 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:12 +0000 (0:00:00.631) 0:00:06.261 ******* 2021-11-21T21:49:12.338 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] 2021-11-21T21:49:12.373 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] 2021-11-21T21:49:12.384 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] 2021-11-21T21:49:12.416 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:12 +0000 (0:00:00.091) 0:00:06.352 ******* 2021-11-21T21:49:12.481 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:12 +0000 (0:00:00.064) 0:00:06.416 ******* 2021-11-21T21:49:12.509 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:12 +0000 (0:00:00.028) 0:00:06.444 ******* 2021-11-21T21:49:13.014 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-11-21T21:49:13.016 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.217182", "end": "2021-11-21 21:49:12.991012", "rc": 0, "start": "2021-11-21 21:49:12.773830", "stderr": "", "stderr_lines": [], "stdout": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "stdout_lines": ["d72e06d3c78c8d9639a8ca75effa043ee468910f"]} 2021-11-21T21:49:13.038 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.529) 0:00:06.974 ******* 2021-11-21T21:49:13.047 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.230313", "end": "2021-11-21 21:49:13.003660", "rc": 0, "start": "2021-11-21 21:49:12.773347", "stderr": "", "stderr_lines": [], "stdout": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "stdout_lines": ["d72e06d3c78c8d9639a8ca75effa043ee468910f"]} 2021-11-21T21:49:13.078 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.040) 0:00:07.014 ******* 2021-11-21T21:49:13.088 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.253453", "end": "2021-11-21 21:49:13.024449", "rc": 0, "start": "2021-11-21 21:49:12.770996", "stderr": "", "stderr_lines": [], "stdout": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "stdout_lines": ["d72e06d3c78c8d9639a8ca75effa043ee468910f"]} 2021-11-21T21:49:13.089 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.226344", "end": "2021-11-21 21:49:13.044383", "rc": 0, "start": "2021-11-21 21:49:12.818039", "stderr": "", "stderr_lines": [], "stdout": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "stdout_lines": ["d72e06d3c78c8d9639a8ca75effa043ee468910f"]} 2021-11-21T21:49:13.119 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.040) 0:00:07.054 ******* 2021-11-21T21:49:13.143 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.023) 0:00:07.078 ******* 2021-11-21T21:49:13.152 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-11-21T21:49:13.153 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.004742", "end": "2021-11-21 21:49:12.890065", "failed_when_result": false, "rc": 0, "start": "2021-11-21 21:49:12.885323", "stderr": "", "stderr_lines": [], "stdout": "be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec", "stdout_lines": ["be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec"]} 2021-11-21T21:49:13.178 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.035) 0:00:07.114 ******* 2021-11-21T21:49:13.197 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.009428", "end": "2021-11-21 21:49:13.388081", "failed_when_result": false, "rc": 0, "start": "2021-11-21 21:49:13.378653", "stderr": "", "stderr_lines": [], "stdout": "be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec", "stdout_lines": ["be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec"]} 2021-11-21T21:49:13.225 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.046) 0:00:07.161 ******* 2021-11-21T21:49:13.246 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008477", "end": "2021-11-21 21:49:12.984354", "failed_when_result": false, "rc": 0, "start": "2021-11-21 21:49:12.975877", "stderr": "", "stderr_lines": [], "stdout": "be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec", "stdout_lines": ["be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec"]} 2021-11-21T21:49:13.247 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.276 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.050) 0:00:07.211 ******* 2021-11-21T21:49:13.301 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.025) 0:00:07.237 ******* 2021-11-21T21:49:13.309 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-11-21T21:49:13.311 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.024053", "end": "2021-11-21 21:49:13.641757", "failed_when_result": false, "rc": 0, "start": "2021-11-21 21:49:13.617704", "stderr": "", "stderr_lines": [], "stdout": "be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec", "stdout_lines": ["be157e19c0bfe2e4e07c6f889b0dbfb8dd2c65ec"]} 2021-11-21T21:49:13.313 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.338 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.036) 0:00:07.274 ******* 2021-11-21T21:49:13.363 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.024) 0:00:07.298 ******* 2021-11-21T21:49:13.487 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.510 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.147) 0:00:07.446 ******* 2021-11-21T21:49:13.549 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-11-21T21:49:13.550 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.577 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.066) 0:00:07.513 ******* 2021-11-21T21:49:13.606 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.607 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.767 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.190) 0:00:07.703 ******* 2021-11-21T21:49:13.794 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.026) 0:00:07.730 ******* 2021-11-21T21:49:13.803 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-11-21T21:49:13.805 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [Remove managed_admin_users from managed_users] *************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.833 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.038) 0:00:07.768 ******* 2021-11-21T21:49:13.857 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.023) 0:00:07.792 ******* 2021-11-21T21:49:13.864 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-11-21T21:49:13.866 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.891 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.034) 0:00:07.827 ******* 2021-11-21T21:49:13.899 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-11-21T21:49:13.900 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.900 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.926 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.034) 0:00:07.862 ******* 2021-11-21T21:49:13.949 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.022) 0:00:07.884 ******* 2021-11-21T21:49:13.956 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-11-21T21:49:13.957 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:13.982 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:13 +0000 (0:00:00.033) 0:00:07.918 ******* 2021-11-21T21:49:13.989 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-11-21T21:49:13.991 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:14.016 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.033) 0:00:07.951 ******* 2021-11-21T21:49:14.023 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-11-21T21:49:14.024 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:14.049 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.033) 0:00:07.985 ******* 2021-11-21T21:49:14.072 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.022) 0:00:08.008 ******* 2021-11-21T21:49:14.079 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-11-21T21:49:14.081 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:14.105 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.033) 0:00:08.041 ******* 2021-11-21T21:49:14.113 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-11-21T21:49:14.114 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:14.144 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.038) 0:00:08.079 ******* 2021-11-21T21:49:14.151 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-11-21T21:49:14.152 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:14.177 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.033) 0:00:08.113 ******* 2021-11-21T21:49:14.200 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.022) 0:00:08.136 ******* 2021-11-21T21:49:14.219 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:14.246 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.046) 0:00:08.182 ******* 2021-11-21T21:49:14.253 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-11-21T21:49:14.255 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:49:14.280 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.033) 0:00:08.216 ******* 2021-11-21T21:49:14.318 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:49:14.348 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.067) 0:00:08.283 ******* 2021-11-21T21:49:14.356 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:49:14.382 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.034) 0:00:08.318 ******* 2021-11-21T21:49:14.400 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:49:14.427 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:14 +0000 (0:00:00.044) 0:00:08.362 ******* 2021-11-21T21:49:14.475 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-11-21T21:49:14.477 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-11-21T21:49:14.540 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-11-21T21:49:14.562 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-11-21T21:49:14.587 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-11-21T21:49:14.620 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-11-21T21:49:14.632 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-11-21T21:49:14.665 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-11-21T21:49:14.687 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-11-21T21:49:14.720 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-11-21T21:49:14.733 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-11-21T21:49:14.754 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-11-21T21:49:14.776 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-11-21T21:49:14.809 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-11-21T21:49:14.832 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-11-21T21:49:14.855 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-11-21T21:49:14.877 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-11-21T21:49:14.899 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-11-21T21:49:14.922 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-11-21T21:49:14.945 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-11-21T21:49:14.998 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-11-21T21:49:15.010 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-11-21T21:49:15.011 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-11-21T21:49:15.052 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-11-21T21:49:15.084 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-11-21T21:49:15.096 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-11-21T21:49:15.108 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-11-21T21:49:15.151 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-11-21T21:49:15.183 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-11-21T21:49:15.195 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-11-21T21:49:15.197 INFO:teuthology.task.ansible.out: ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-11-21T21:49:15.241 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-11-21T21:49:15.273 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-11-21T21:49:15.295 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-11-21T21:49:15.297 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-11-21T21:49:15.339 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-11-21T21:49:15.372 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-11-21T21:49:15.385 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-11-21T21:49:15.398 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-11-21T21:49:15.431 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-11-21T21:49:15.474 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-11-21T21:49:15.497 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-11-21T21:49:15.529 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-11-21T21:49:15.530 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-11-21T21:49:15.584 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-11-21T21:49:15.586 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-11-21T21:49:15.628 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-11-21T21:49:15.661 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-11-21T21:49:15.672 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-11-21T21:49:15.684 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-11-21T21:49:15.749 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-11-21T21:49:15.761 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-11-21T21:49:15.773 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-11-21T21:49:15.805 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-11-21T21:49:15.849 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-11-21T21:49:15.861 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-11-21T21:49:15.862 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-11-21T21:49:15.895 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-11-21T21:49:15.940 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-11-21T21:49:15.952 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-11-21T21:49:15.953 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-11-21T21:49:15.986 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-11-21T21:49:16.042 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-11-21T21:49:16.054 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-11-21T21:49:16.056 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-11-21T21:49:16.089 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-11-21T21:49:16.143 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-11-21T21:49:16.144 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-11-21T21:49:16.145 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-11-21T21:49:16.177 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-11-21T21:49:16.231 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-11-21T21:49:16.243 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-11-21T21:49:16.244 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-11-21T21:49:16.277 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-11-21T21:49:16.330 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-11-21T21:49:16.342 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-11-21T21:49:16.344 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-11-21T21:49:16.378 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-11-21T21:49:16.432 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-11-21T21:49:16.444 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-11-21T21:49:16.466 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-11-21T21:49:16.498 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-11-21T21:49:16.530 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-11-21T21:49:16.542 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-11-21T21:49:16.553 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-11-21T21:49:16.596 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-11-21T21:49:16.617 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-11-21T21:49:16.629 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-11-21T21:49:16.651 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-11-21T21:49:16.704 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-11-21T21:49:16.716 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-11-21T21:49:16.727 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-11-21T21:49:16.749 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-11-21T21:49:16.812 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-11-21T21:49:16.824 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-11-21T21:49:16.826 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-11-21T21:49:16.878 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-11-21T21:49:16.911 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-11-21T21:49:16.923 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-11-21T21:49:16.924 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-11-21T21:49:16.967 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-11-21T21:49:17.000 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-11-21T21:49:17.022 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-11-21T21:49:17.023 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-11-21T21:49:17.055 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-11-21T21:49:17.111 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-11-21T21:49:17.113 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-11-21T21:49:17.156 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-11-21T21:49:17.167 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-11-21T21:49:17.200 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-11-21T21:49:17.201 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-11-21T21:49:17.245 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-11-21T21:49:17.256 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-11-21T21:49:17.299 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-11-21T21:49:17.301 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-11-21T21:49:17.353 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-11-21T21:49:17.354 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-11-21T21:49:17.397 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-11-21T21:49:17.429 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-11-21T21:49:17.441 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-11-21T21:49:17.454 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-11-21T21:49:17.497 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-11-21T21:49:17.519 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-11-21T21:49:17.541 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-11-21T21:49:17.543 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-11-21T21:49:17.586 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-11-21T21:49:17.619 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-11-21T21:49:17.631 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-11-21T21:49:17.632 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-11-21T21:49:17.686 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-11-21T21:49:17.718 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-11-21T21:49:17.719 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-11-21T21:49:17.721 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-21T21:49:17.784 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-11-21T21:49:17.817 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-21T21:49:17.818 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:17.819 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-21T21:49:17.873 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-21T21:49:17.905 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-11-21T21:49:17.906 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:17.918 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:17.982 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:17.993 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-11-21T21:49:18.005 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-11-21T21:49:18.006 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-11-21T21:49:18.069 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-11-21T21:49:18.091 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-11-21T21:49:18.092 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-11-21T21:49:18.094 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-11-21T21:49:18.167 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-11-21T21:49:18.179 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-11-21T21:49:18.191 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-11-21T21:49:18.203 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-11-21T21:49:18.256 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-11-21T21:49:18.278 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-11-21T21:49:18.290 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-11-21T21:49:18.301 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-11-21T21:49:18.303 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:18.355 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-11-21T21:49:18.377 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-11-21T21:49:18.388 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-11-21T21:49:18.389 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-11-21T21:49:18.442 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-11-21T21:49:18.475 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-11-21T21:49:18.487 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-21T21:49:18.499 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-11-21T21:49:18.501 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:18.553 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-11-21T21:49:18.564 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-21T21:49:18.576 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:18.598 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-21T21:49:18.653 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-21T21:49:18.656 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:18.678 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-11-21T21:49:18.700 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:18.743 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-11-21T21:49:18.755 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:18.767 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-11-21T21:49:18.799 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-11-21T21:49:18.842 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-11-21T21:49:18.864 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-11-21T21:49:18.865 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-11-21T21:49:18.887 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-11-21T21:49:18.941 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-11-21T21:49:18.954 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-11-21T21:49:18.956 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-11-21T21:49:18.978 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-11-21T21:49:19.032 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-11-21T21:49:19.044 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-11-21T21:49:19.055 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-11-21T21:49:19.088 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-11-21T21:49:19.132 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-11-21T21:49:19.133 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-11-21T21:49:19.156 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-11-21T21:49:19.179 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-11-21T21:49:19.233 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-11-21T21:49:19.234 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-11-21T21:49:19.257 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-11-21T21:49:19.300 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-11-21T21:49:19.333 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-11-21T21:49:19.345 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-11-21T21:49:19.347 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:19.357 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-11-21T21:49:19.390 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-11-21T21:49:19.422 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-11-21T21:49:19.444 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-11-21T21:49:19.446 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-11-21T21:49:19.489 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-11-21T21:49:19.511 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-11-21T21:49:19.533 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-11-21T21:49:19.535 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-11-21T21:49:19.577 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-11-21T21:49:19.610 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-11-21T21:49:19.632 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-11-21T21:49:19.633 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-11-21T21:49:19.676 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-11-21T21:49:19.698 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-11-21T21:49:19.720 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-11-21T21:49:19.721 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-11-21T21:49:19.764 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-11-21T21:49:19.797 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-11-21T21:49:19.820 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-11-21T21:49:19.821 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-11-21T21:49:19.864 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-11-21T21:49:19.897 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-11-21T21:49:19.919 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-11-21T21:49:19.920 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-11-21T21:49:19.963 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-11-21T21:49:19.985 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-11-21T21:49:20.008 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-11-21T21:49:20.020 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-11-21T21:49:20.063 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-11-21T21:49:20.074 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-11-21T21:49:20.107 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-11-21T21:49:20.119 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-11-21T21:49:20.152 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-11-21T21:49:20.174 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:20.196 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-11-21T21:49:20.207 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-11-21T21:49:20.251 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-11-21T21:49:20.263 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-11-21T21:49:20.295 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:20.319 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-11-21T21:49:20.362 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:20.363 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-11-21T21:49:20.385 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-11-21T21:49:20.407 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-21T21:49:20.450 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-11-21T21:49:20.451 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-11-21T21:49:20.484 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-11-21T21:49:20.538 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-11-21T21:49:20.549 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-11-21T21:49:20.550 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-11-21T21:49:20.572 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-11-21T21:49:20.641 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-11-21T21:49:20.643 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-11-21T21:49:20.656 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-11-21T21:49:20.669 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-11-21T21:49:20.742 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-11-21T21:49:20.753 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-11-21T21:49:20.756 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-11-21T21:49:20.766 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-11-21T21:49:20.840 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-11-21T21:49:20.852 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-11-21T21:49:20.853 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-11-21T21:49:20.865 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-11-21T21:49:20.929 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-11-21T21:49:20.951 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-11-21T21:49:20.952 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-11-21T21:49:20.995 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-11-21T21:49:21.028 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-11-21T21:49:21.040 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-11-21T21:49:21.051 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-11-21T21:49:21.084 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-11-21T21:49:21.117 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-11-21T21:49:21.139 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-11-21T21:49:21.150 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-11-21T21:49:21.173 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-11-21T21:49:21.206 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-11-21T21:49:21.228 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-11-21T21:49:21.240 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-11-21T21:49:21.273 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-11-21T21:49:21.305 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-11-21T21:49:21.316 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-11-21T21:49:21.339 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-11-21T21:49:21.361 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-11-21T21:49:21.394 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-11-21T21:49:21.416 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-11-21T21:49:21.427 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-11-21T21:49:21.460 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-11-21T21:49:21.493 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-11-21T21:49:21.525 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-11-21T21:49:21.536 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-11-21T21:49:21.548 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-11-21T21:49:21.582 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-11-21T21:49:21.614 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-11-21T21:49:21.637 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-11-21T21:49:21.648 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-11-21T21:49:21.681 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-11-21T21:49:21.714 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-11-21T21:49:21.726 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-11-21T21:49:21.748 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-11-21T21:49:21.812 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-11-21T21:49:21.813 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-11-21T21:49:21.825 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-11-21T21:49:21.847 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-11-21T21:49:21.901 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-11-21T21:49:21.923 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-11-21T21:49:21.934 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-11-21T21:49:21.978 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-11-21T21:49:22.000 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-11-21T21:49:22.011 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-11-21T21:49:22.034 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-11-21T21:49:22.066 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-11-21T21:49:22.089 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-11-21T21:49:22.101 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-11-21T21:49:22.134 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-11-21T21:49:22.167 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-11-21T21:49:22.189 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-11-21T21:49:22.200 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-11-21T21:49:22.222 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-11-21T21:49:22.286 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-11-21T21:49:22.287 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-11-21T21:49:22.288 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-11-21T21:49:22.320 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-11-21T21:49:22.373 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-11-21T21:49:22.386 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-11-21T21:49:22.387 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-11-21T21:49:22.420 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-11-21T21:49:22.473 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-11-21T21:49:22.474 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-11-21T21:49:22.476 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-11-21T21:49:22.507 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-11-21T21:49:22.571 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-11-21T21:49:22.572 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-11-21T21:49:22.605 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-11-21T21:49:22.606 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-11-21T21:49:22.670 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-11-21T21:49:22.671 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-11-21T21:49:22.693 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-11-21T21:49:22.694 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-11-21T21:49:22.757 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-11-21T21:49:22.769 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-11-21T21:49:22.791 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-11-21T21:49:22.792 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-11-21T21:49:22.856 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-11-21T21:49:22.867 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-11-21T21:49:22.910 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-11-21T21:49:22.922 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-11-21T21:49:22.944 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-11-21T21:49:22.956 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-11-21T21:49:23.009 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-11-21T21:49:23.010 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-11-21T21:49:23.032 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-11-21T21:49:23.055 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-11-21T21:49:23.098 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-11-21T21:49:23.110 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-11-21T21:49:23.121 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-11-21T21:49:23.154 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-11-21T21:49:23.196 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-11-21T21:49:23.208 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-11-21T21:49:23.219 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-11-21T21:49:23.252 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-11-21T21:49:23.295 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-11-21T21:49:23.296 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-11-21T21:49:23.318 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-11-21T21:49:23.340 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-11-21T21:49:23.394 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-11-21T21:49:23.395 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-11-21T21:49:23.406 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-11-21T21:49:23.438 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-11-21T21:49:23.481 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-11-21T21:49:23.492 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-11-21T21:49:23.504 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-11-21T21:49:23.526 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-11-21T21:49:23.579 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-11-21T21:49:23.581 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-11-21T21:49:23.593 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:23.625 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-11-21T21:49:23.668 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-11-21T21:49:23.669 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-11-21T21:49:23.691 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-11-21T21:49:23.713 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-11-21T21:49:23.768 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:23.769 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-11-21T21:49:23.781 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-11-21T21:49:23.813 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-11-21T21:49:23.856 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-11-21T21:49:23.857 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-11-21T21:49:23.879 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-11-21T21:49:23.922 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:23.954 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-21T21:49:23.955 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-11-21T21:49:23.977 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-11-21T21:49:24.020 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-11-21T21:49:24.053 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-11-21T21:49:24.054 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-11-21T21:49:24.076 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-11-21T21:49:24.108 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-11-21T21:49:24.140 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-11-21T21:49:24.162 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-11-21T21:49:24.163 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-11-21T21:49:24.206 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-11-21T21:49:24.227 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-11-21T21:49:24.260 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-11-21T21:49:24.261 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-11-21T21:49:24.314 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-11-21T21:49:24.326 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-11-21T21:49:24.348 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-11-21T21:49:24.359 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-11-21T21:49:24.402 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-11-21T21:49:24.413 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-11-21T21:49:24.446 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-11-21T21:49:24.447 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-11-21T21:49:24.521 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-11-21T21:49:24.533 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-11-21T21:49:24.534 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-11-21T21:49:24.576 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-11-21T21:49:24.609 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-11-21T21:49:24.631 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-11-21T21:49:24.632 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-11-21T21:49:24.666 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-11-21T21:49:24.699 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-11-21T21:49:24.721 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-11-21T21:49:24.722 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-11-21T21:49:24.765 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-11-21T21:49:24.797 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-11-21T21:49:24.819 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-11-21T21:49:24.831 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-11-21T21:49:24.864 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-11-21T21:49:24.896 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-11-21T21:49:24.918 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-11-21T21:49:24.930 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-11-21T21:49:24.952 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-11-21T21:49:24.995 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-11-21T21:49:25.007 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-11-21T21:49:25.018 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-11-21T21:49:25.051 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-11-21T21:49:25.093 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-11-21T21:49:25.105 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-11-21T21:49:25.117 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-11-21T21:49:25.150 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-11-21T21:49:25.194 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-11-21T21:49:25.195 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-11-21T21:49:25.207 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-11-21T21:49:25.281 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-11-21T21:49:25.293 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-11-21T21:49:25.304 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-11-21T21:49:25.337 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-11-21T21:49:25.380 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-11-21T21:49:25.392 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-11-21T21:49:25.404 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-11-21T21:49:25.437 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-11-21T21:49:25.481 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-11-21T21:49:25.493 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-11-21T21:49:25.494 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-11-21T21:49:25.537 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-11-21T21:49:25.580 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-11-21T21:49:25.581 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-11-21T21:49:25.604 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-11-21T21:49:25.637 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-11-21T21:49:25.680 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-11-21T21:49:25.681 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-11-21T21:49:25.694 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-11-21T21:49:25.737 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-11-21T21:49:25.781 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-11-21T21:49:25.781 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-11-21T21:49:25.793 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-11-21T21:49:25.826 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-11-21T21:49:25.879 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-11-21T21:49:25.880 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-11-21T21:49:25.893 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-11-21T21:49:25.927 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-11-21T21:49:25.971 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-11-21T21:49:25.984 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-11-21T21:49:25.986 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-11-21T21:49:26.020 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-11-21T21:49:26.075 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-11-21T21:49:26.086 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-11-21T21:49:26.087 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-11-21T21:49:26.110 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-11-21T21:49:26.163 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-11-21T21:49:26.186 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-11-21T21:49:26.197 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-11-21T21:49:26.198 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-11-21T21:49:26.284 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-11-21T21:49:26.295 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-11-21T21:49:26.296 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-11-21T21:49:26.329 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-11-21T21:49:26.383 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-11-21T21:49:26.384 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-11-21T21:49:26.396 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-11-21T21:49:26.421 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-11-21T21:49:26.472 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-11-21T21:49:26.494 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-11-21T21:49:26.495 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-11-21T21:49:26.517 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-11-21T21:49:26.571 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-11-21T21:49:26.582 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-11-21T21:49:26.595 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-11-21T21:49:26.606 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-11-21T21:49:26.660 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-11-21T21:49:26.682 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-11-21T21:49:26.694 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-11-21T21:49:26.705 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-11-21T21:49:26.759 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-11-21T21:49:26.782 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-11-21T21:49:26.794 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-11-21T21:49:26.805 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-11-21T21:49:26.849 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-11-21T21:49:26.881 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-11-21T21:49:26.882 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-11-21T21:49:26.905 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-11-21T21:49:26.948 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-11-21T21:49:26.982 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-11-21T21:49:26.983 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-11-21T21:49:26.995 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-11-21T21:49:27.052 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-11-21T21:49:27.063 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-11-21T21:49:27.117 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-11-21T21:49:27.129 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-11-21T21:49:27.151 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-11-21T21:49:27.163 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-11-21T21:49:27.217 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-11-21T21:49:27.218 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "move_home": false, "name": "dachary", "shell": "/bin/bash", "state": "present", "uid": 10102} 2021-11-21T21:49:27.241 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-11-21T21:49:27.253 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-11-21T21:49:27.306 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-11-21T21:49:27.307 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "move_home": false, "name": "amctagga", "shell": "/bin/bash", "state": "present", "uid": 10103} 2021-11-21T21:49:27.340 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-11-21T21:49:27.352 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-11-21T21:49:27.406 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-11-21T21:49:27.407 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "move_home": false, "name": "kevinz", "shell": "/bin/bash", "state": "present", "uid": 10104} 2021-11-21T21:49:27.450 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-11-21T21:49:27.461 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "move_home": false, "name": "dachary", "shell": "/bin/bash", "state": "present", "uid": 10102} 2021-11-21T21:49:27.494 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-11-21T21:49:27.495 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "move_home": false, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "uid": 10105} 2021-11-21T21:49:27.549 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "move_home": false, "name": "dachary", "shell": "/bin/bash", "state": "present", "uid": 10102} 2021-11-21T21:49:27.560 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "move_home": false, "name": "amctagga", "shell": "/bin/bash", "state": "present", "uid": 10103} 2021-11-21T21:49:27.582 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "move_home": false, "name": "dachary", "shell": "/bin/bash", "state": "present", "uid": 10102} 2021-11-21T21:49:27.594 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "move_home": false, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "uid": 10106} 2021-11-21T21:49:27.648 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "move_home": false, "name": "amctagga", "shell": "/bin/bash", "state": "present", "uid": 10103} 2021-11-21T21:49:27.649 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "move_home": false, "name": "kevinz", "shell": "/bin/bash", "state": "present", "uid": 10104} 2021-11-21T21:49:27.682 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "move_home": false, "name": "amctagga", "shell": "/bin/bash", "state": "present", "uid": 10103} 2021-11-21T21:49:27.684 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "move_home": false, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "uid": 10107} 2021-11-21T21:49:27.746 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "move_home": false, "name": "kevinz", "shell": "/bin/bash", "state": "present", "uid": 10104} 2021-11-21T21:49:27.747 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "move_home": false, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "uid": 10105} 2021-11-21T21:49:27.769 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "move_home": false, "name": "kevinz", "shell": "/bin/bash", "state": "present", "uid": 10104} 2021-11-21T21:49:27.781 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "move_home": false, "name": "lflores", "shell": "/bin/bash", "state": "present", "uid": 10108} 2021-11-21T21:49:27.835 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "move_home": false, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "uid": 10105} 2021-11-21T21:49:27.847 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "move_home": false, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "uid": 10106} 2021-11-21T21:49:27.858 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "move_home": false, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "uid": 10105} 2021-11-21T21:49:27.870 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "move_home": false, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "uid": 10109} 2021-11-21T21:49:27.933 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "move_home": false, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "uid": 10106} 2021-11-21T21:49:27.944 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "move_home": false, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "uid": 10107} 2021-11-21T21:49:27.956 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "move_home": false, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "uid": 10106} 2021-11-21T21:49:27.957 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "move_home": false, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "uid": 10110} 2021-11-21T21:49:28.033 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "move_home": false, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "uid": 10107} 2021-11-21T21:49:28.044 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "move_home": false, "name": "lflores", "shell": "/bin/bash", "state": "present", "uid": 10108} 2021-11-21T21:49:28.045 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "move_home": false, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "uid": 10107} 2021-11-21T21:49:28.057 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "move_home": false, "name": "luisrico", "shell": "/bin/bash", "state": "present", "uid": 10111} 2021-11-21T21:49:28.121 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "move_home": false, "name": "lflores", "shell": "/bin/bash", "state": "present", "uid": 10108} 2021-11-21T21:49:28.132 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "move_home": false, "name": "lflores", "shell": "/bin/bash", "state": "present", "uid": 10108} 2021-11-21T21:49:28.143 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "move_home": false, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "uid": 10109} 2021-11-21T21:49:28.144 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "move_home": false, "name": "matan", "shell": "/bin/bash", "state": "present", "uid": 10112} 2021-11-21T21:49:28.218 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "move_home": false, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "uid": 10109} 2021-11-21T21:49:28.230 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "move_home": false, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "uid": 10109} 2021-11-21T21:49:28.231 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "move_home": false, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "uid": 10110} 2021-11-21T21:49:28.233 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "move_home": false, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "uid": 10113} 2021-11-21T21:49:28.317 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "move_home": false, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "uid": 10110} 2021-11-21T21:49:28.318 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "move_home": false, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "uid": 10110} 2021-11-21T21:49:28.330 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "move_home": false, "name": "luisrico", "shell": "/bin/bash", "state": "present", "uid": 10111} 2021-11-21T21:49:28.331 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "move_home": false, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "uid": 10114} 2021-11-21T21:49:28.415 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "move_home": false, "name": "luisrico", "shell": "/bin/bash", "state": "present", "uid": 10111} 2021-11-21T21:49:28.416 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "move_home": false, "name": "luisrico", "shell": "/bin/bash", "state": "present", "uid": 10111} 2021-11-21T21:49:28.418 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "move_home": false, "name": "matan", "shell": "/bin/bash", "state": "present", "uid": 10112} 2021-11-21T21:49:28.418 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "move_home": false, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "uid": 10115} 2021-11-21T21:49:28.516 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "move_home": false, "name": "matan", "shell": "/bin/bash", "state": "present", "uid": 10112} 2021-11-21T21:49:28.517 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "move_home": false, "name": "matan", "shell": "/bin/bash", "state": "present", "uid": 10112} 2021-11-21T21:49:28.518 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "move_home": false, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "uid": 10116} 2021-11-21T21:49:28.519 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "move_home": false, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "uid": 10113} 2021-11-21T21:49:28.604 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "move_home": false, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "uid": 10113} 2021-11-21T21:49:28.605 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10118, "groups": "sudo", "home": "/home/arjsharm", "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "move_home": false, "name": "arjsharm", "shell": "/bin/bash", "state": "present", "uid": 10117} 2021-11-21T21:49:28.616 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "move_home": false, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "uid": 10113} 2021-11-21T21:49:28.617 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "move_home": false, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "uid": 10114} 2021-11-21T21:49:28.704 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "move_home": false, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "uid": 10114} 2021-11-21T21:49:28.717 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "move_home": false, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "uid": 10114} 2021-11-21T21:49:28.739 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10119, "groups": "sudo", "home": "/home/melissali", "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "move_home": false, "name": "melissali", "shell": "/bin/bash", "state": "present", "uid": 10118} 2021-11-21T21:49:28.740 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "move_home": false, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "uid": 10115} 2021-11-21T21:49:28.793 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "move_home": false, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "uid": 10115} 2021-11-21T21:49:28.805 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "move_home": false, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "uid": 10115} 2021-11-21T21:49:28.827 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10120, "groups": "sudo", "home": "/home/ionutbalutoiu", "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "move_home": false, "name": "ionutbalutoiu", "shell": "/bin/bash", "state": "present", "uid": 10119} 2021-11-21T21:49:28.839 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "move_home": false, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "uid": 10116} 2021-11-21T21:49:28.882 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "move_home": false, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "uid": 10116} 2021-11-21T21:49:28.904 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "move_home": false, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "uid": 10116} 2021-11-21T21:49:28.927 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10121, "groups": "sudo", "home": "/home/pdiazbou", "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "move_home": false, "name": "pdiazbou", "shell": "/bin/bash", "state": "present", "uid": 10120} 2021-11-21T21:49:28.939 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10118, "groups": "sudo", "home": "/home/arjsharm", "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "move_home": false, "name": "arjsharm", "shell": "/bin/bash", "state": "present", "uid": 10117} 2021-11-21T21:49:28.981 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10118, "groups": "sudo", "home": "/home/arjsharm", "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "move_home": false, "name": "arjsharm", "shell": "/bin/bash", "state": "present", "uid": 10117} 2021-11-21T21:49:29.003 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10118, "groups": "sudo", "home": "/home/arjsharm", "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "move_home": false, "name": "arjsharm", "shell": "/bin/bash", "state": "present", "uid": 10117} 2021-11-21T21:49:29.015 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10122, "groups": "sudo", "home": "/home/xuwang", "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "move_home": false, "name": "xuwang", "shell": "/bin/bash", "state": "present", "uid": 10121} 2021-11-21T21:49:29.026 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10119, "groups": "sudo", "home": "/home/melissali", "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "move_home": false, "name": "melissali", "shell": "/bin/bash", "state": "present", "uid": 10118} 2021-11-21T21:49:29.079 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10119, "groups": "sudo", "home": "/home/melissali", "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "move_home": false, "name": "melissali", "shell": "/bin/bash", "state": "present", "uid": 10118} 2021-11-21T21:49:29.101 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10119, "groups": "sudo", "home": "/home/melissali", "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "move_home": false, "name": "melissali", "shell": "/bin/bash", "state": "present", "uid": 10118} 2021-11-21T21:49:29.113 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10123, "groups": "sudo", "home": "/home/blaine", "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "move_home": false, "name": "blaine", "shell": "/bin/bash", "state": "present", "uid": 10122} 2021-11-21T21:49:29.115 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10120, "groups": "sudo", "home": "/home/ionutbalutoiu", "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "move_home": false, "name": "ionutbalutoiu", "shell": "/bin/bash", "state": "present", "uid": 10119} 2021-11-21T21:49:29.168 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10120, "groups": "sudo", "home": "/home/ionutbalutoiu", "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "move_home": false, "name": "ionutbalutoiu", "shell": "/bin/bash", "state": "present", "uid": 10119} 2021-11-21T21:49:29.211 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10120, "groups": "sudo", "home": "/home/ionutbalutoiu", "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "move_home": false, "name": "ionutbalutoiu", "shell": "/bin/bash", "state": "present", "uid": 10119} 2021-11-21T21:49:29.213 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10121, "groups": "sudo", "home": "/home/pdiazbou", "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "move_home": false, "name": "pdiazbou", "shell": "/bin/bash", "state": "present", "uid": 10120} 2021-11-21T21:49:29.267 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10121, "groups": "sudo", "home": "/home/pdiazbou", "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "move_home": false, "name": "pdiazbou", "shell": "/bin/bash", "state": "present", "uid": 10120} 2021-11-21T21:49:29.300 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10122, "groups": "sudo", "home": "/home/xuwang", "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "move_home": false, "name": "xuwang", "shell": "/bin/bash", "state": "present", "uid": 10121} 2021-11-21T21:49:29.312 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10121, "groups": "sudo", "home": "/home/pdiazbou", "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "move_home": false, "name": "pdiazbou", "shell": "/bin/bash", "state": "present", "uid": 10120} 2021-11-21T21:49:29.376 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10122, "groups": "sudo", "home": "/home/xuwang", "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "move_home": false, "name": "xuwang", "shell": "/bin/bash", "state": "present", "uid": 10121} 2021-11-21T21:49:29.388 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10123, "groups": "sudo", "home": "/home/blaine", "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "move_home": false, "name": "blaine", "shell": "/bin/bash", "state": "present", "uid": 10122} 2021-11-21T21:49:29.401 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10122, "groups": "sudo", "home": "/home/xuwang", "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "move_home": false, "name": "xuwang", "shell": "/bin/bash", "state": "present", "uid": 10121} 2021-11-21T21:49:29.465 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10123, "groups": "sudo", "home": "/home/blaine", "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "move_home": false, "name": "blaine", "shell": "/bin/bash", "state": "present", "uid": 10122} 2021-11-21T21:49:29.488 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10123, "groups": "sudo", "home": "/home/blaine", "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "move_home": false, "name": "blaine", "shell": "/bin/bash", "state": "present", "uid": 10122} 2021-11-21T21:49:29.718 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10124, "groups": "sudo", "home": "/home/choffman", "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "name": "choffman", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10123} 2021-11-21T21:49:29.865 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:29 +0000 (0:00:15.438) 0:00:23.801 ******* 2021-11-21T21:49:29.935 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-11-21T21:49:29.959 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:29 +0000 (0:00:00.094) 0:00:23.895 ******* 2021-11-21T21:49:29.986 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-11-21T21:49:29.988 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10124, "groups": "sudo", "home": "/home/choffman", "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "name": "choffman", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10123} 2021-11-21T21:49:29.990 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10124, "groups": "sudo", "home": "/home/choffman", "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "name": "choffman", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10123} 2021-11-21T21:49:30.220 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10124, "groups": "sudo", "home": "/home/choffman", "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "name": "choffman", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10123} 2021-11-21T21:49:30.223 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-11-21T21:49:30.224 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@desktop 2021-11-21T21:49:30.226 INFO:teuthology.task.ansible.out:QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-11-21T21:49:30.228 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed208 2021-11-21T21:49:30.229 INFO:teuthology.task.ansible.out:2caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": 2021-11-21T21:49:30.231 INFO:teuthology.task.ansible.out: "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d 2021-11-21T21:49:30.232 INFO:teuthology.task.ansible.out:3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZY 2021-11-21T21:49:30.234 INFO:teuthology.task.ansible.out:M7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95 2021-11-21T21:49:30.234 INFO:teuthology.task.ansible.out:579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 2021-11-21T21:49:30.236 INFO:teuthology.task.ansible.out:4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp" 2021-11-21T21:49:30.237 INFO:teuthology.task.ansible.out:, "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f36 2021-11-21T21:49:30.239 INFO:teuthology.task.ansible.out:24b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9Jje 2021-11-21T21:49:30.241 INFO:teuthology.task.ansible.out:nDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, 2021-11-21T21:49:30.241 INFO:teuthology.task.ansible.out:{"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shy 2021-11-21T21:49:30.243 INFO:teuthology.task.ansible.out:amsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a2 2021-11-21T21:49:30.244 INFO:teuthology.task.ansible.out:8ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "z 2021-11-21T21:49:30.246 INFO:teuthology.task.ansible.out:l31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}]}, "changed": false} 2021-11-21T21:49:30.271 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-11-21T21:49:30.349 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.389) 0:00:24.285 ******* 2021-11-21T21:49:30.372 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.022) 0:00:24.307 ******* 2021-11-21T21:49:30.399 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.027) 0:00:24.335 ******* 2021-11-21T21:49:30.424 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.025) 0:00:24.360 ******* 2021-11-21T21:49:30.433 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-11-21T21:49:30.480 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.056) 0:00:24.416 ******* 2021-11-21T21:49:30.515 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.033) 0:00:24.450 ******* 2021-11-21T21:49:30.621 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-11-21T21:49:30.752 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@desktop 2021-11-21T21:49:30.754 INFO:teuthology.task.ansible.out:QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-11-21T21:49:30.755 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed208 2021-11-21T21:49:30.757 INFO:teuthology.task.ansible.out:2caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": 2021-11-21T21:49:30.759 INFO:teuthology.task.ansible.out: "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d 2021-11-21T21:49:30.761 INFO:teuthology.task.ansible.out:3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZY 2021-11-21T21:49:30.762 INFO:teuthology.task.ansible.out:M7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95 2021-11-21T21:49:30.763 INFO:teuthology.task.ansible.out:579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 2021-11-21T21:49:30.765 INFO:teuthology.task.ansible.out:4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp" 2021-11-21T21:49:30.766 INFO:teuthology.task.ansible.out:, "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f36 2021-11-21T21:49:30.769 INFO:teuthology.task.ansible.out:24b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9Jje 2021-11-21T21:49:30.770 INFO:teuthology.task.ansible.out:nDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, 2021-11-21T21:49:30.771 INFO:teuthology.task.ansible.out:{"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shy 2021-11-21T21:49:30.773 INFO:teuthology.task.ansible.out:amsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a2 2021-11-21T21:49:30.775 INFO:teuthology.task.ansible.out:8ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "z 2021-11-21T21:49:30.777 INFO:teuthology.task.ansible.out:l31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}]}, "changed": false} TASK [Create all users without sudo access.] *********************************** 2021-11-21T21:49:30.779 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-11-21T21:49:30.783 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@desktop 2021-11-21T21:49:30.785 INFO:teuthology.task.ansible.out:QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-11-21T21:49:30.787 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed208 2021-11-21T21:49:30.789 INFO:teuthology.task.ansible.out:2caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": 2021-11-21T21:49:30.790 INFO:teuthology.task.ansible.out: "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d 2021-11-21T21:49:30.792 INFO:teuthology.task.ansible.out:3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZY 2021-11-21T21:49:30.793 INFO:teuthology.task.ansible.out:M7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 2021-11-21T21:49:30.795 INFO:teuthology.task.ansible.out:4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp" 2021-11-21T21:49:30.797 INFO:teuthology.task.ansible.out:, "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f36 2021-11-21T21:49:30.799 INFO:teuthology.task.ansible.out:24b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9Jje 2021-11-21T21:49:30.800 INFO:teuthology.task.ansible.out:nDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shy 2021-11-21T21:49:30.802 INFO:teuthology.task.ansible.out:amsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a2 2021-11-21T21:49:30.804 INFO:teuthology.task.ansible.out:8ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e52 2021-11-21T21:49:30.804 INFO:teuthology.task.ansible.out:99298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "z 2021-11-21T21:49:30.806 INFO:teuthology.task.ansible.out:l31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}]}, "changed": false} 2021-11-21T21:49:30.837 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.322) 0:00:24.772 ******* 2021-11-21T21:49:30.875 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.037) 0:00:24.810 ******* 2021-11-21T21:49:30.906 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:30 +0000 (0:00:00.031) 0:00:24.841 ******* 2021-11-21T21:49:31.143 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@desktop 2021-11-21T21:49:31.145 INFO:teuthology.task.ansible.out:QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-11-21T21:49:31.146 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed208 2021-11-21T21:49:31.148 INFO:teuthology.task.ansible.out:2caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": 2021-11-21T21:49:31.150 INFO:teuthology.task.ansible.out: "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebb 2021-11-21T21:49:31.150 INFO:teuthology.task.ansible.out:a6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d 2021-11-21T21:49:31.152 INFO:teuthology.task.ansible.out:3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZY 2021-11-21T21:49:31.153 INFO:teuthology.task.ansible.out:M7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 2021-11-21T21:49:31.155 INFO:teuthology.task.ansible.out:4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp" 2021-11-21T21:49:31.157 INFO:teuthology.task.ansible.out:, "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f36 2021-11-21T21:49:31.159 INFO:teuthology.task.ansible.out:24b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9Jje 2021-11-21T21:49:31.160 INFO:teuthology.task.ansible.out:nDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shy 2021-11-21T21:49:31.162 INFO:teuthology.task.ansible.out:amsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a2 2021-11-21T21:49:31.163 INFO:teuthology.task.ansible.out:8ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "z 2021-11-21T21:49:31.165 INFO:teuthology.task.ansible.out:l31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}]}, "changed": false} 2021-11-21T21:49:31.184 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:31 +0000 (0:00:00.278) 0:00:25.120 ******* 2021-11-21T21:49:31.409 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-11-21T21:49:31.411 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"after": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "attempts": 1, "before": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "changed": false, "remote_url_changed": false} 2021-11-21T21:49:31.434 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:31 +0000 (0:00:00.249) 0:00:25.370 ******* 2021-11-21T21:49:31.579 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"after": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "attempts": 1, "before": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "changed": false, "remote_url_changed": false} 2021-11-21T21:49:31.604 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:31 +0000 (0:00:00.169) 0:00:25.539 ******* 2021-11-21T21:49:31.612 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"after": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "attempts": 1, "before": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "changed": false, "remote_url_changed": false} 2021-11-21T21:49:31.639 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:31 +0000 (0:00:00.034) 0:00:25.574 ******* 2021-11-21T21:49:31.770 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:31.772 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-11-21T21:49:31.855 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-11-21T21:49:31.857 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"after": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "attempts": 1, "before": "d72e06d3c78c8d9639a8ca75effa043ee468910f", "changed": false, "remote_url_changed": false} 2021-11-21T21:49:31.881 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:31 +0000 (0:00:00.242) 0:00:25.816 ******* 2021-11-21T21:49:31.890 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:31.891 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-11-21T21:49:31.945 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-11-21T21:49:31.978 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-11-21T21:49:31.990 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-11-21T21:49:32.044 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-11-21T21:49:32.066 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-11-21T21:49:32.068 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-11-21T21:49:32.078 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-11-21T21:49:32.132 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-11-21T21:49:32.164 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-11-21T21:49:32.165 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-11-21T21:49:32.233 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-11-21T21:49:32.235 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys" 2021-11-21T21:49:32.237 INFO:teuthology.task.ansible.out:, "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-11-21T21:49:32.266 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-11-21T21:49:32.268 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3Nz 2021-11-21T21:49:32.270 INFO:teuthology.task.ansible.out:aC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-11-21T21:49:32.331 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-11-21T21:49:32.342 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-11-21T21:49:32.364 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-11-21T21:49:32.387 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-11-21T21:49:32.390 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:32.431 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-11-21T21:49:32.433 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-11-21T21:49:32.453 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-11-21T21:49:32.507 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-11-21T21:49:32.509 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-11-21T21:49:32.519 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-11-21T21:49:32.531 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-11-21T21:49:32.543 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-11-21T21:49:32.545 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-11-21T21:49:32.695 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-11-21T21:49:32.697 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr 2021-11-21T21:49:32.699 INFO:teuthology.task.ansible.out:1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2c 2021-11-21T21:49:32.700 INFO:teuthology.task.ansible.out:t1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-11-21T21:49:32.709 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-11-21T21:49:32.752 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-11-21T21:49:32.754 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-11-21T21:49:32.774 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-11-21T21:49:32.785 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:32.797 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-11-21T21:49:32.799 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-11-21T21:49:32.810 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-11-21T21:49:32.823 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:32.876 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-11-21T21:49:32.878 INFO:teuthology.task.ansible.out:on", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:32.888 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:32.902 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-11-21T21:49:32.904 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-11-21T21:49:32.923 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-11-21T21:49:32.925 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-11-21T21:49:32.998 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-11-21T21:49:33.009 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-11-21T21:49:33.011 INFO:teuthology.task.ansible.out:on", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2 2021-11-21T21:49:33.012 INFO:teuthology.task.ansible.out:NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:33.021 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:33.033 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-11-21T21:49:33.034 INFO:teuthology.task.ansible.out:on", "validate_certs": true} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:33.044 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:33.097 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-11-21T21:49:33.100 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-11-21T21:49:33.108 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-11-21T21:49:33.119 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:33.131 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-11-21T21:49:33.153 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-11-21T21:49:33.209 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-11-21T21:49:33.232 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-11-21T21:49:33.233 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+K 2021-11-21T21:49:33.235 INFO:teuthology.task.ansible.out:b7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-11-21T21:49:33.255 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-11-21T21:49:33.256 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-11-21T21:49:33.319 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-11-21T21:49:33.321 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-11-21T21:49:33.322 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-11-21T21:49:33.331 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:33.332 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-11-21T21:49:33.342 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:33.344 INFO:teuthology.task.ansible.out: ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-11-21T21:49:33.428 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-11-21T21:49:33.441 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-11-21T21:49:33.452 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-11-21T21:49:33.527 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich 2021-11-21T21:49:33.529 INFO:teuthology.task.ansible.out:", "validate_certs": true} 2021-11-21T21:49:33.550 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-11-21T21:49:33.551 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-11-21T21:49:33.561 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-11-21T21:49:33.563 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-11-21T21:49:33.626 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-11-21T21:49:33.628 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-11-21T21:49:33.660 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-11-21T21:49:33.661 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAA 2021-11-21T21:49:33.663 INFO:teuthology.task.ansible.out:BAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, 2021-11-21T21:49:33.665 INFO:teuthology.task.ansible.out:"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-11-21T21:49:33.727 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-11-21T21:49:33.728 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-11-21T21:49:33.749 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-11-21T21:49:33.751 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-11-21T21:49:33.771 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-11-21T21:49:33.814 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-11-21T21:49:33.825 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-11-21T21:49:33.847 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-11-21T21:49:33.870 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-11-21T21:49:33.913 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-11-21T21:49:33.914 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-11-21T21:49:33.947 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-11-21T21:49:33.969 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-11-21T21:49:33.971 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-11-21T21:49:34.033 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQ 2021-11-21T21:49:34.034 INFO:teuthology.task.ansible.out:v306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-11-21T21:49:34.044 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQ 2021-11-21T21:49:34.046 INFO:teuthology.task.ansible.out:v306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-11-21T21:49:34.066 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-11-21T21:49:34.068 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3P 2021-11-21T21:49:34.069 INFO:teuthology.task.ansible.out:xJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-11-21T21:49:34.141 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-11-21T21:49:34.173 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-11-21T21:49:34.175 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-11-21T21:49:34.185 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-11-21T21:49:34.249 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-11-21T21:49:34.272 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-11-21T21:49:34.284 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-11-21T21:49:34.285 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-11-21T21:49:34.338 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-11-21T21:49:34.371 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-11-21T21:49:34.372 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-11-21T21:49:34.404 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQ 2021-11-21T21:49:34.406 INFO:teuthology.task.ansible.out:v306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-11-21T21:49:34.436 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-11-21T21:49:34.469 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-11-21T21:49:34.470 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-11-21T21:49:34.534 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-11-21T21:49:34.536 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-11-21T21:49:34.557 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-11-21T21:49:34.589 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-11-21T21:49:34.642 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-11-21T21:49:34.653 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-11-21T21:49:34.655 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-11-21T21:49:34.664 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-11-21T21:49:34.666 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-11-21T21:49:34.686 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-11-21T21:49:34.688 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-11-21T21:49:34.739 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-11-21T21:49:34.750 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-11-21T21:49:34.762 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-11-21T21:49:34.784 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-11-21T21:49:34.838 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-11-21T21:49:34.849 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-11-21T21:49:34.860 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-11-21T21:49:34.892 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-11-21T21:49:34.955 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-11-21T21:49:34.957 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-11-21T21:49:34.967 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-11-21T21:49:35.010 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-11-21T21:49:35.063 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-11-21T21:49:35.095 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-11-21T21:49:35.106 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-11-21T21:49:35.107 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-11-21T21:49:35.109 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-11-21T21:49:35.150 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:35.193 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:35.204 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:35.237 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-11-21T21:49:35.239 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-11-21T21:49:35.303 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-11-21T21:49:35.305 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-11-21T21:49:35.336 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-11-21T21:49:35.347 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-11-21T21:49:35.390 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-11-21T21:49:35.411 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-11-21T21:49:35.433 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-11-21T21:49:35.435 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-11-21T21:49:35.477 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-11-21T21:49:35.479 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-11-21T21:49:35.499 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-11-21T21:49:35.500 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-11-21T21:49:35.501 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-11-21T21:49:35.533 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-11-21T21:49:35.534 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-11-21T21:49:35.586 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-11-21T21:49:35.588 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-11-21T21:49:35.597 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-11-21T21:49:35.599 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-11-21T21:49:35.630 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-11-21T21:49:35.630 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-11-21T21:49:35.694 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-11-21T21:49:35.705 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-11-21T21:49:35.758 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:35.759 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-11-21T21:49:35.793 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-11-21T21:49:35.814 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-11-21T21:49:35.859 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-11-21T21:49:35.861 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-11-21T21:49:35.894 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-11-21T21:49:35.950 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-11-21T21:49:35.983 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-11-21T21:49:36.038 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHO 2021-11-21T21:49:36.039 INFO:teuthology.task.ansible.out:RxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:36.060 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHO 2021-11-21T21:49:36.062 INFO:teuthology.task.ansible.out:RxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:36.083 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHO 2021-11-21T21:49:36.085 INFO:teuthology.task.ansible.out:RxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:36.129 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-11-21T21:49:36.131 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-11-21T21:49:36.157 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-11-21T21:49:36.159 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-11-21T21:49:36.160 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-11-21T21:49:36.216 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-11-21T21:49:36.218 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-11-21T21:49:36.220 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-11-21T21:49:36.231 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-11-21T21:49:36.233 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-11-21T21:49:36.234 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-11-21T21:49:36.279 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-11-21T21:49:36.280 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-11-21T21:49:36.281 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-11-21T21:49:36.353 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-11-21T21:49:36.365 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-11-21T21:49:36.418 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-11-21T21:49:36.429 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-11-21T21:49:36.503 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-11-21T21:49:36.526 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-11-21T21:49:36.537 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-11-21T21:49:36.570 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-11-21T21:49:36.614 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-11-21T21:49:36.640 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-11-21T21:49:36.664 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-11-21T21:49:36.688 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-11-21T21:49:36.722 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-11-21T21:49:36.744 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-11-21T21:49:36.777 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-11-21T21:49:36.812 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHO 2021-11-21T21:49:36.814 INFO:teuthology.task.ansible.out:RxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:36.824 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-11-21T21:49:36.846 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-11-21T21:49:36.923 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-11-21T21:49:36.924 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-11-21T21:49:36.926 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvyd 2021-11-21T21:49:36.928 INFO:teuthology.task.ansible.out:wgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-11-21T21:49:36.935 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-11-21T21:49:36.937 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-11-21T21:49:36.958 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-11-21T21:49:36.959 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-11-21T21:49:37.021 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-11-21T21:49:37.023 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-11-21T21:49:37.066 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-11-21T21:49:37.077 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-11-21T21:49:37.121 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-11-21T21:49:37.123 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-11-21T21:49:37.154 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-11-21T21:49:37.186 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-11-21T21:49:37.219 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-11-21T21:49:37.221 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-11-21T21:49:37.262 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-11-21T21:49:37.264 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-11-21T21:49:37.274 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-11-21T21:49:37.275 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-11-21T21:49:37.316 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-11-21T21:49:37.328 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-11-21T21:49:37.360 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-11-21T21:49:37.372 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-11-21T21:49:37.415 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-11-21T21:49:37.448 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-11-21T21:49:37.459 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-11-21T21:49:37.471 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-11-21T21:49:37.504 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-11-21T21:49:37.505 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-11-21T21:49:37.547 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-11-21T21:49:37.550 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-11-21T21:49:37.559 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-11-21T21:49:37.561 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-11-21T21:49:37.581 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-11-21T21:49:37.583 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-11-21T21:49:37.603 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-11-21T21:49:37.657 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-11-21T21:49:37.668 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-11-21T21:49:37.690 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-11-21T21:49:37.712 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-11-21T21:49:37.755 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-11-21T21:49:37.766 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-11-21T21:49:37.788 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-11-21T21:49:37.810 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-11-21T21:49:37.864 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-11-21T21:49:37.865 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-11-21T21:49:37.887 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-11-21T21:49:37.898 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-11-21T21:49:37.910 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:37.995 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:37.996 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-11-21T21:49:38.009 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:38.011 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-11-21T21:49:38.095 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+ 2021-11-21T21:49:38.097 INFO:teuthology.task.ansible.out:/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-11-21T21:49:38.107 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-11-21T21:49:38.107 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-11-21T21:49:38.203 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-11-21T21:49:38.205 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/h 2021-11-21T21:49:38.207 INFO:teuthology.task.ansible.out:ome/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-11-21T21:49:38.216 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-11-21T21:49:38.302 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-11-21T21:49:38.303 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-11-21T21:49:38.317 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-11-21T21:49:38.330 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-11-21T21:49:38.406 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-11-21T21:49:38.408 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-11-21T21:49:38.409 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-11-21T21:49:38.442 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-11-21T21:49:38.495 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-11-21T21:49:38.506 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-11-21T21:49:38.518 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-11-21T21:49:38.540 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-11-21T21:49:38.593 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-11-21T21:49:38.604 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-21T21:49:38.616 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-11-21T21:49:38.638 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-11-21T21:49:38.691 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-11-21T21:49:38.693 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-11-21T21:49:38.745 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-11-21T21:49:38.747 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-11-21T21:49:38.799 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-11-21T21:49:38.801 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-11-21T21:49:38.844 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-11-21T21:49:38.845 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-11-21T21:49:38.899 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-11-21T21:49:38.910 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-11-21T21:49:38.933 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-11-21T21:49:38.934 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-11-21T21:49:39.011 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-11-21T21:49:39.013 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-11-21T21:49:39.035 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-11-21T21:49:39.068 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-11-21T21:49:39.111 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-11-21T21:49:39.112 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-11-21T21:49:39.154 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-11-21T21:49:39.166 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-11-21T21:49:39.219 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-11-21T21:49:39.242 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-11-21T21:49:39.263 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-11-21T21:49:39.296 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-11-21T21:49:39.318 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-11-21T21:49:39.340 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-11-21T21:49:39.362 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-11-21T21:49:39.383 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-11-21T21:49:39.405 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-11-21T21:49:39.459 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-11-21T21:49:39.470 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-11-21T21:49:39.481 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-11-21T21:49:39.504 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-11-21T21:49:39.558 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-11-21T21:49:39.570 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-11-21T21:49:39.583 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-11-21T21:49:39.605 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-11-21T21:49:39.659 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-11-21T21:49:39.661 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-11-21T21:49:39.681 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-11-21T21:49:39.693 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-11-21T21:49:39.704 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-11-21T21:49:39.706 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-11-21T21:49:39.757 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-11-21T21:49:39.780 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-11-21T21:49:39.782 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-11-21T21:49:39.801 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-11-21T21:49:39.823 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-11-21T21:49:39.856 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-11-21T21:49:39.889 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-11-21T21:49:39.900 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-11-21T21:49:39.933 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-11-21T21:49:39.955 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-11-21T21:49:39.987 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-11-21T21:49:39.988 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-11-21T21:49:40.031 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-11-21T21:49:40.063 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-11-21T21:49:40.065 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-11-21T21:49:40.085 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-11-21T21:49:40.087 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-11-21T21:49:40.088 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-11-21T21:49:40.128 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-11-21T21:49:40.151 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-11-21T21:49:40.183 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-11-21T21:49:40.185 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CR 2021-11-21T21:49:40.187 INFO:teuthology.task.ansible.out:BT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-11-21T21:49:40.216 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-11-21T21:49:40.249 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-11-21T21:49:40.282 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-11-21T21:49:40.293 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-11-21T21:49:40.315 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-11-21T21:49:40.348 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-11-21T21:49:40.390 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-11-21T21:49:40.412 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-11-21T21:49:40.423 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-11-21T21:49:40.425 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-11-21T21:49:40.446 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-11-21T21:49:40.488 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-11-21T21:49:40.514 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-11-21T21:49:40.522 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-11-21T21:49:40.544 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-11-21T21:49:40.587 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-11-21T21:49:40.609 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-11-21T21:49:40.620 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-11-21T21:49:40.653 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-11-21T21:49:40.676 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-11-21T21:49:40.708 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-11-21T21:49:40.720 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-11-21T21:49:40.753 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-11-21T21:49:40.774 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-11-21T21:49:40.798 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-11-21T21:49:40.822 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-11-21T21:49:40.823 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-11-21T21:49:40.854 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-11-21T21:49:40.856 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-11-21T21:49:40.876 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-11-21T21:49:40.877 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-11-21T21:49:40.898 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-11-21T21:49:40.920 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-11-21T21:49:40.941 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-11-21T21:49:40.974 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-11-21T21:49:40.996 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-11-21T21:49:40.997 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-11-21T21:49:41.039 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-11-21T21:49:41.050 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-11-21T21:49:41.072 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-11-21T21:49:41.096 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-11-21T21:49:41.128 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-11-21T21:49:41.150 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-11-21T21:49:41.182 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-11-21T21:49:41.183 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-11-21T21:49:41.227 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-11-21T21:49:41.239 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-11-21T21:49:41.282 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-11-21T21:49:41.314 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-11-21T21:49:41.315 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-11-21T21:49:41.379 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-11-21T21:49:41.381 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-11-21T21:49:41.412 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-11-21T21:49:41.423 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-11-21T21:49:41.477 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-11-21T21:49:41.489 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-11-21T21:49:41.522 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-11-21T21:49:41.524 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:41.544 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-11-21T21:49:41.587 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-11-21T21:49:41.588 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-11-21T21:49:41.609 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-11-21T21:49:41.642 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-11-21T21:49:41.644 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:41.695 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-11-21T21:49:41.697 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-11-21T21:49:41.707 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-11-21T21:49:41.718 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-11-21T21:49:41.720 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:41.730 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-11-21T21:49:41.796 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-11-21T21:49:41.808 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-11-21T21:49:41.820 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-11-21T21:49:41.831 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-11-21T21:49:41.885 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-11-21T21:49:41.896 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-11-21T21:49:41.918 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-11-21T21:49:41.931 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-11-21T21:49:41.984 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-11-21T21:49:41.996 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-11-21T21:49:42.018 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-11-21T21:49:42.062 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-11-21T21:49:42.084 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-11-21T21:49:42.085 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-11-21T21:49:42.118 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-11-21T21:49:42.162 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-11-21T21:49:42.184 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-11-21T21:49:42.195 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-11-21T21:49:42.218 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-11-21T21:49:42.250 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-11-21T21:49:42.293 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-11-21T21:49:42.295 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-11-21T21:49:42.326 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-11-21T21:49:42.349 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-11-21T21:49:42.393 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-11-21T21:49:42.394 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-11-21T21:49:42.396 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-21T21:49:42.416 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-11-21T21:49:42.459 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-11-21T21:49:42.481 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-11-21T21:49:42.493 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-11-21T21:49:42.525 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-11-21T21:49:42.548 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-11-21T21:49:42.590 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-11-21T21:49:42.614 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-11-21T21:49:42.615 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-11-21T21:49:42.658 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-11-21T21:49:42.701 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-11-21T21:49:42.713 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-11-21T21:49:42.725 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-11-21T21:49:42.758 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-11-21T21:49:42.791 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-11-21T21:49:42.814 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-11-21T21:49:42.857 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-11-21T21:49:42.859 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-11-21T21:49:42.922 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-11-21T21:49:42.924 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-11-21T21:49:42.945 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-11-21T21:49:42.957 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-11-21T21:49:43.021 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-11-21T21:49:43.043 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-11-21T21:49:43.045 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-11-21T21:49:43.054 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-11-21T21:49:43.107 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-11-21T21:49:43.129 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-11-21T21:49:43.140 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-11-21T21:49:43.141 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-11-21T21:49:43.204 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-11-21T21:49:43.226 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-11-21T21:49:43.237 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-11-21T21:49:43.249 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-11-21T21:49:43.302 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-11-21T21:49:43.335 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-11-21T21:49:43.337 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-11-21T21:49:43.347 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-11-21T21:49:43.411 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-11-21T21:49:43.413 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-11-21T21:49:43.433 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-11-21T21:49:43.444 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-11-21T21:49:43.446 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-11-21T21:49:43.520 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-11-21T21:49:43.522 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-11-21T21:49:43.532 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-11-21T21:49:43.586 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-11-21T21:49:43.588 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-11-21T21:49:43.640 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-11-21T21:49:43.642 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-11-21T21:49:43.644 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-11-21T21:49:43.663 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-11-21T21:49:43.684 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-11-21T21:49:43.738 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-11-21T21:49:43.760 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-11-21T21:49:43.782 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-11-21T21:49:43.793 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-11-21T21:49:43.826 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-11-21T21:49:43.870 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-11-21T21:49:43.881 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-11-21T21:49:43.924 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-11-21T21:49:43.926 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-11-21T21:49:43.957 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-11-21T21:49:43.979 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-11-21T21:49:44.022 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-11-21T21:49:44.045 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-11-21T21:49:44.067 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-11-21T21:49:44.078 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-11-21T21:49:44.121 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-11-21T21:49:44.133 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-11-21T21:49:44.165 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-11-21T21:49:44.177 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-11-21T21:49:44.220 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-11-21T21:49:44.232 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-11-21T21:49:44.274 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-11-21T21:49:44.276 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-11-21T21:49:44.329 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-11-21T21:49:44.341 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-11-21T21:49:44.342 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-11-21T21:49:44.363 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-11-21T21:49:44.375 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-11-21T21:49:44.460 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-11-21T21:49:44.462 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-11-21T21:49:44.472 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-11-21T21:49:44.472 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-11-21T21:49:44.557 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-11-21T21:49:44.568 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-11-21T21:49:44.570 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs" 2021-11-21T21:49:44.572 INFO:teuthology.task.ansible.out:: true} 2021-11-21T21:49:44.656 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-11-21T21:49:44.658 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-11-21T21:49:44.670 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-11-21T21:49:44.672 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-11-21T21:49:44.756 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-11-21T21:49:44.758 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-11-21T21:49:44.779 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4m0LAcC7XiFiXFtYcCvZqX1CtyV9JU+VUEawZJpWrGTOr8c+vIexq7+WKg+eIUMot4+dtfwG8lByS1VofglIe5/quv17Xj7dO/hF2WkV8O7RYt7NhzmrKg54yB73pbRpvqzGceWWDvGRsWQGzbY7ARqOp4FPIDJvgXABCkak+hmjXyXpvXN418CBDbDmxHWwFb9QtB/MpQFiFrfl1EMDj8DaXZNWgo5J/qytic6LgiyPHhHfhSylk4cl9TGftNatobpMjqBT0O2/plk/qf+7pHPuvGSgihjijdpEPzVfctXE2OmsQAIZgkzXeCqts+s6ZYdUKq1FvpV5rMM8ZLqfCkDjEzFENJS68YYE2EjxAEgMzj7h/sWc4B5I5pxkAqVAQSGjYRu6rCCbFz8PXUdTyW+Vs1YeylnlBMiCa4xE4gwvW3HlWEnumtyURd1PRENrGJsJ/0uX6bpltbSg4sGACRYpLPJMm3OhqI3Jm/J/e2NLjfCw+svU2447xnfJOh68= pdhange@vaquita\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCoeyVaMjhcGrWQB+4Rq+SSIi8+OY/kx2s1vofiGt1gu3RNtFmG0Df1VITJJ32dbzaW7kTNxaRvhldNd/n7yEMssG+Wg+a5WKvSudlbVMmscbSXdFuVbcCW1jNrTGCYrxL7Sn5e7NQEQfAsCbfXwTcYBw33NdBVovC+gs2V3sZGvX3Y0akZONNuVg1mhu0L4m1/5IegJh005KeilwetHA2A5Dr10RG6oyIgT7t4z5gZXsrOiG3UQBXaekILYV9jxMWoFXGkVLc8TpL9SUAdxE3Owc0u1XiPqYYlEN+zDvE0554Zj8Oc2ZO6HRXCZXQFvzXeCD12YswcOfkIDmJRF7GT prashant@mauli", "key_options": null, "ke 2021-11-21T21:49:44.781 INFO:teuthology.task.ansible.out:yfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-11-21T21:49:44.812 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-11-21T21:49:44.855 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-11-21T21:49:44.878 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-11-21T21:49:44.879 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4m0LAcC7XiFiXFtYcCvZqX1CtyV9JU+VUEawZJpWrGTOr8c+vIexq7+WKg+eIUMot4+dtfwG8lByS1VofglIe5/quv17Xj7dO/hF2WkV8O7RYt7NhzmrKg54yB73pbRpvqzGceWWDvGRsWQGzbY7ARqOp4FPIDJvgXABCkak+hmjXyXpvXN418CBDbDmxHWwFb9QtB/MpQFiFrfl1EMDj8DaXZNWgo5J/qytic6LgiyPHhHfhSylk4cl9TGftNatobpMjqBT0O2/plk/qf+7pHPuvGSgihjijdpEPzVfctXE2OmsQAIZgkzXeCqts+s6ZYdUKq1FvpV5rMM8ZLqfCkDjEzFENJS68YYE2EjxAEgMzj7h/sWc4B5I5pxkAqVAQSGjYRu6rCCbFz8PXUdTyW+Vs1YeylnlBMiCa4xE4gwvW3HlWEnumtyURd1PRENrGJsJ/0uX6bpltbSg4sGACRYpLPJMm3OhqI3Jm/J/e2NLjfCw+svU2447xnfJOh68= pdhange@vaquita\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCoeyVaMjhcGrWQB+4Rq+SSIi8+OY/kx2s1vofiGt1gu3RNtFmG0Df1VITJJ32dbzaW7kTNxaRvhldNd/n7yEMssG+Wg+a5WKvSudlbVMmscbSXdFuVbcCW1jNrTGCYrxL7Sn5e7NQEQfAsCbfXwTcYBw33NdBVovC+gs2V3sZGvX3Y0akZONNuVg1mhu0L4m1/5IegJh005KeilwetHA2A5Dr10RG6oyIgT7t4z5gZXsrOiG3UQBXaekILYV9jxMWoFXGkVLc8TpL9SUAdxE3Owc0u1XiPqYYlEN+zDvE0554Zj8Oc2ZO6HRXCZXQFvzXeCD12YswcOfkIDmJRF7GT prashant@mauli", "key_options": null, "ke 2021-11-21T21:49:44.881 INFO:teuthology.task.ansible.out:yfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-11-21T21:49:44.943 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-11-21T21:49:44.956 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-11-21T21:49:44.987 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-11-21T21:49:44.989 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-11-21T21:49:45.041 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-11-21T21:49:45.074 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4m0LAcC7XiFiXFtYcCvZqX1CtyV9JU+VUEawZJpWrGTOr8c+vIexq7+WKg+eIUMot4+dtfwG8lByS1VofglIe5/quv17Xj7dO/hF2WkV8O7RYt7NhzmrKg54yB73pbRpvqzGceWWDvGRsWQGzbY7ARqOp4FPIDJvgXABCkak+hmjXyXpvXN418CBDbDmxHWwFb9QtB/MpQFiFrfl1EMDj8DaXZNWgo5J/qytic6LgiyPHhHfhSylk4cl9TGftNatobpMjqBT0O2/plk/qf+7pHPuvGSgihjijdpEPzVfctXE2OmsQAIZgkzXeCqts+s6ZYdUKq1FvpV5rMM8ZLqfCkDjEzFENJS68YYE2EjxAEgMzj7h/sWc4B5I5pxkAqVAQSGjYRu6rCCbFz8PXUdTyW+Vs1YeylnlBMiCa4xE4gwvW3HlWEnumtyURd1PRENrGJsJ/0uX6bpltbSg4sGACRYpLPJMm3OhqI3Jm/J/e2NLjfCw+svU2447xnfJOh68= pdhange@vaquita\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCoeyVaMjhcGrWQB+4Rq+SSIi8+OY/kx2s1vofiGt1gu3RNtFmG0Df1VITJJ32dbzaW7kTNxaRvhldNd/n7yEMssG+Wg+a5WKvSudlbVMmscbSXdFuVbcCW1jNrTGCYrxL7Sn5e7NQEQfAsCbfXwTcYBw33NdBVovC+gs2V3sZGvX3Y0akZONNuVg1mhu0L4m1/5IegJh005KeilwetHA2A5Dr10RG6oyIgT7t4z5gZXsrOiG3UQBXaekILYV9jxMWoFXGkVLc8TpL9SUAdxE3Owc0u1XiPqYYlEN+zDvE0554Zj8Oc2ZO6HRXCZXQFvzXeCD12YswcOfkIDmJRF7GT prashant@mauli", "key_options": null, "ke 2021-11-21T21:49:45.076 INFO:teuthology.task.ansible.out:yfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-11-21T21:49:45.096 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-11-21T21:49:45.097 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_o 2021-11-21T21:49:45.099 INFO:teuthology.task.ansible.out:ptions": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-11-21T21:49:45.140 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-11-21T21:49:45.206 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} changed: [gibba017.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt4sQTrDDg562IWY9Bw2fxFR7Y7b5lfo9egHa1qNFR5K4gt8lUwQXgzcxZK5dgSTZXHDGxrbYKWsdb8NbpgU/YjSqDUIptMZrhTI6pdHKZ2usY+nieaJj0NIZP/S7M93BbJeoNBYACjvDUr+vXEg6ZL3uAj+zFXr/4te+p4NqJBl4bSpvjcRK12O+F6HE3Q93j0Vlfs8IbXP4vcPcqOW+2usXw7JPdfjwuACJxrLTx/0mBsv8qXJDfOpPIHAMA/9SdTAzhs6XMo+fOwVOKkV+ki2fO3sl8pE16aFOkI3cYX36nkYYpZ2bJLnIPcLv37i3Nypl8eNLA4gzjkDf1o/KLO5vXOs76H189WGLqjh0SHAx/3KO6mrIhTq8Bv6YWoT3ntg22fXaxeajNhHcQ0bKZtGOAb0W1FcMRbSTkgEFAassYxlpTtuaCDIuA4fUVKW97Zu62eDDsSUtTWak/Yf940qAbptPPAw4ljmrvZRcjM9y 2021-11-21T21:49:45.208 INFO:teuthology.task.ansible.out:6mGU8lKi5H4QoEOfbPok= nia@fedora", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-11-21T21:49:45.240 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_o 2021-11-21T21:49:45.242 INFO:teuthology.task.ansible.out:ptions": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-11-21T21:49:45.305 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-11-21T21:49:45.348 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-11-21T21:49:45.350 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt4sQTrDDg562IWY9Bw2fxFR7Y7b5lfo9egHa1qNFR5K4gt8lUwQXgzcxZK5dgSTZXHDGxrbYKWsdb8NbpgU/YjSqDUIptMZrhTI6pdHKZ2usY+nieaJj0NIZP/S7M93BbJeoNBYACjvDUr+vXEg6ZL3uAj+zFXr/4te+p4NqJBl4bSpvjcRK12O+F6HE3Q93j0V 2021-11-21T21:49:45.352 INFO:teuthology.task.ansible.out:lfs8IbXP4vcPcqOW+2usXw7JPdfjwuACJxrLTx/0mBsv8qXJDfOpPIHAMA/9SdTAzhs6XMo+fOwVOKkV+ki2fO3sl8pE16aFOkI3cYX36nkYYpZ2bJLnIPcLv37i3Nypl8eNLA4gzjkDf1o/KLO5vXOs76H189WGLqjh0SHAx/3KO6mrIhTq8Bv6YWoT3ntg22fXaxeajNhHcQ0bKZtGOAb0W1FcMRbSTkgEFAassYxlpTtuaCDIuA4fUVKW97Zu62eDDsSUtTWak/Yf940qAbptPPAw4ljmrvZRcjM9y6mGU8lKi5H4QoEOfbPok= nia@fedora", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-11-21T21:49:45.425 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_o 2021-11-21T21:49:45.427 INFO:teuthology.task.ansible.out:ptions": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-11-21T21:49:45.436 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-11-21T21:49:45.469 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-11-21T21:49:45.491 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-11-21T21:49:45.545 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-11-21T21:49:45.558 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt4sQTrDDg562IWY9Bw2fxFR7Y7b5lfo9egHa1qNFR5K4gt8lUwQXgzcxZK5dgSTZXHDGxrbYKWsdb8NbpgU/YjSqDUIptMZrhTI6pdHKZ2usY+nieaJj0NIZP/S7M93BbJeoNBYACjvDUr+vXEg6ZL3uAj+zFXr/4te+p4NqJBl4bSpvjcRK12O+F6HE3Q93j0Vlfs8IbXP4vcPcqOW+2usXw7JPdfjwuACJxrLTx/0mBsv8qXJDfOpPIHAMA/9SdTAzhs6XMo+fOwVOKkV+ki2fO3sl8pE16aFOkI3cYX36nkYYpZ2bJLnIPcLv37i3Nypl8eNLA4gzjkDf1o/KLO5vXOs76H189WGLqjh0SHAx/3KO6mrIhTq8Bv6YWoT3ntg22fXaxeajNhHcQ0bKZtGOAb0W1FcMRbSTkgEFAassYxlpTtuaCDIuA4fUVKW97Zu62eDDsSUtTWak/Yf940qAbptPPAw4ljmrvZRcjM9y6mGU8lKi5H4QoEOfbPok= nia@fedora", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-11-21T21:49:45.591 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-11-21T21:49:45.602 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-11-21T21:49:45.635 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-11-21T21:49:45.678 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-11-21T21:49:45.679 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-11-21T21:49:45.723 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-11-21T21:49:45.766 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4m0LAcC7XiFiXFtYcCvZqX1CtyV9JU+VUEawZJpWrGTOr8c+vIexq7+WKg+eIUMot4+dtfwG8lByS1VofglIe5/quv17Xj7dO/hF2WkV8O7RYt7NhzmrKg54yB73pbRpvqzGceWWDvGRsWQGzbY7ARqOp4FPIDJvgXABCkak+hmjXyXpvXN418CBDbDmxHWwFb9QtB/MpQFiFrfl1EMDj8DaXZNWgo5J/qytic6LgiyPHhHfhSylk4cl9TGftNatobpMjqBT0O2/plk/qf+7pHPuvGSgihjijdpEPzVfctXE2OmsQAIZgkzXeCqts+s6ZYdUKq1FvpV5rMM8ZLqfCkDjEzFENJS68YYE2EjxAEgMzj7h/sWc4B5I5pxkAqVAQSGjYRu6rCCbFz8PXUdTyW+Vs1YeylnlBMiCa4xE4gwvW3HlWEnumtyURd1PRENrGJsJ/0uX6bpltbSg4sGACRYpLPJMm3OhqI3Jm/J/e2NLjfCw+svU2447xnfJOh68= pdhange@vaquita\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCoeyVaMjhcGrWQB+4Rq+SSIi8+OY/kx2s1vofiGt1gu3RNtFmG0Df1VITJJ32dbzaW7kTNxaRvhldNd/n7yEMssG+Wg+a5WKvSudlbVMmscbSXdFuVbcCW1jNrTGCYrxL7Sn5e7NQEQfAsCbfXwTcYBw33NdBVovC+gs2V3sZGvX3Y0akZONNuVg1mhu0L4m1/5IegJh005KeilwetHA2A5Dr10RG6oyIgT7t4z5gZXsrOiG3UQBXaekILYV9jxMWoFXGkVLc8TpL9SUAdxE3Owc0u1XiPqYYlEN+zDvE0554Zj8Oc2ZO6HRXCZXQFvzXeCD12YswcOfkIDmJRF7GT prashant@mauli", "key_options": null, "ke 2021-11-21T21:49:45.768 INFO:teuthology.task.ansible.out:yfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-11-21T21:49:45.778 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-11-21T21:49:45.789 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-11-21T21:49:45.832 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-11-21T21:49:45.876 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-11-21T21:49:45.878 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-11-21T21:49:45.922 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-11-21T21:49:45.945 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-11-21T21:49:45.978 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-11-21T21:49:45.991 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-11-21T21:49:46.024 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-11-21T21:49:46.057 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-11-21T21:49:46.089 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-11-21T21:49:46.101 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_o 2021-11-21T21:49:46.102 INFO:teuthology.task.ansible.out:ptions": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-11-21T21:49:46.123 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-11-21T21:49:46.165 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-11-21T21:49:46.200 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-11-21T21:49:46.212 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt4sQTrDDg562IWY9Bw2fxFR7Y7b5lfo9egHa1qNFR5K4gt8lUwQXgzcxZK5dgSTZXHDGxrbYKWsdb8NbpgU/YjSqDUIptMZrhTI6pdHKZ2usY+nieaJj0NIZP/S7M93BbJeoNBYACjvDUr+vXEg6ZL3uAj+zFXr/4te+p4NqJBl4bSpvjcRK12O+F6HE3Q93j0Vlfs8IbXP4vcPcqOW+2usXw7JPdfjwuACJxrLTx/0mBsv8qXJDfOpPIHAMA/9SdTAzhs6XMo+fOwVOKkV+ki2fO3sl8pE16aFOkI3cYX36nkYYpZ2bJLnIPcLv37i3Nypl8eNLA4gzjkDf1o/KLO5vXOs76H189WGLqjh0SHAx/3KO6mrIhTq8Bv6YWoT3ntg22fXaxeajNhHcQ0bKZtGOAb0W1FcMRbSTkgEFAassYxlpTtuaCDIuA4fUVKW97Zu62eDDsSUtTWak/Yf940qAbptPPAw4ljmrvZRcjM9y6mGU8lKi5H4QoEOfbPok= nia@fedora", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-11-21T21:49:46.224 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-11-21T21:49:46.268 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-11-21T21:49:46.312 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-11-21T21:49:46.323 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-11-21T21:49:46.324 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-11-21T21:49:46.367 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-11-21T21:49:46.410 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-11-21T21:49:46.422 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-11-21T21:49:46.433 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-11-21T21:49:46.487 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKjxTgyhL1yFriyNvvuhUc6SE8+6f6lT6FTF8cZL9uiz matan.brz@gmail.com\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8l7qvSQR7fnZdQ1yB1F1jgde0OmLcwcDFzc/o2dYBb matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-11-21T21:49:46.508 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKjxTgyhL1yFriyNvvuhUc6SE8+6f6lT6FTF8cZL9uiz matan.brz@gmail.com\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8l7qvSQR7fnZdQ1yB1F1jgde0OmLcwcDFzc/o2dYBb matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-11-21T21:49:46.530 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-11-21T21:49:46.552 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-11-21T21:49:46.584 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkFTEUxQkQ1cplOLOf86Teu4qTYGkMqWtjIwzsU/AcimjFolSI5fJ3B72PH7bhQKkSnOlTjiOos2LrD5i/XgxkJD6icVWwHO11lczsq/8K5x0ZMJbJaiT3QphBB4yItoge2Y4/7g4o/78Ne+U7eZ1L5fgeO1fExtIaEXK1lS8pVlpZQXyproTIzp1WLHoaXb38Mxc5jAtuvEKfEycdX3c8k/9ti2qBCZnmjPH05WOxkqCRyNBT+bSJZyFMYt9RDUuycn3BYVSXTAzPW0qwgvKY5vij+qiFoxij7nKNMH7w6C0arolCcJxhiMyEF3UbTuOxgVONWyqFk/vScGGSYj9ytQYjv0L1cFyOVhlpbu+tnptulfQ3faMpjROE/eYxTIoY7OPDgWbFqEUJ4eEhKwAC7aYIYUhvoI/8k+4u64nsyEJQdHXmB+OY3WE5TegYUCuqSsdPY74eO1o8dDgxF363eggDSBmYqSSs+Tpu9TNhJG7iJFX1tiAPwmKvjVrJOac= ozeneva@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-11-21T21:49:46.596 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkFTEUxQkQ1cplOLOf86Teu4qTYGkMqWtjIwzsU/AcimjFolSI5fJ3B72PH7bhQKkSnOlTjiOos2LrD5i/XgxkJD6icVWwHO11lczsq/8K5x0ZMJbJaiT3QphBB4yItoge2Y4/7g4o/78Ne+U7eZ1L5fgeO1fExtIaEXK1lS8pVlpZQXyproTIzp1WLHoaXb38Mxc5jAtuvEKfEycdX3c8k/9ti2qBCZnmjPH05WOxkqCRyNBT+bSJZyFMYt9RDUuycn3BYVSXTAzPW0qwgvKY5vij+qiFoxij7nKNMH7w6C0arolCcJxhiMyEF3UbTuOxgVONWyqFk/vScGGSYj9ytQYjv0L1cFyOVhlpbu+tnptulfQ3faMpjROE/eYxTIoY7OPDgWbFqEUJ4eEhKwAC7aYIYUhvoI/8k+4u64nsyEJQdHXmB+OY3WE5TegYUCuqSsdPY74eO1o8dDgxF363eggDSBmYqSSs+Tpu9TNhJG7iJFX1tiAPwmKvjVrJOac= ozeneva@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-11-21T21:49:46.639 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-11-21T21:49:46.662 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-11-21T21:49:46.696 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-11-21T21:49:46.697 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-11-21T21:49:46.750 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-11-21T21:49:46.784 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKjxTgyhL1yFriyNvvuhUc6SE8+6f6lT6FTF8cZL9uiz matan.brz@gmail.com\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8l7qvSQR7fnZdQ1yB1F1jgde0OmLcwcDFzc/o2dYBb matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-11-21T21:49:46.795 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-11-21T21:49:46.796 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-11-21T21:49:46.850 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-11-21T21:49:46.894 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} 2021-11-21T21:49:46.906 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} 2021-11-21T21:49:46.929 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkFTEUxQkQ1cplOLOf86Teu4qTYGkMqWtjIwzsU/AcimjFolSI5fJ3B72PH7bhQKkSnOlTjiOos2LrD5i/XgxkJD6icVWwHO11lczsq/8K5x0ZMJbJaiT3QphBB4yItoge2Y4/7g4o/78Ne+U7eZ1L5fgeO1fExtIaEXK1lS8pVlpZQXyproTIzp1WLHoaXb38Mxc5jAtuvEKfEycdX3c8k/9ti2qBCZnmjPH05WOxkqCRyNBT+bSJZyFMYt9RDUuycn3BYVSXTAzPW0qwgvKY5vij+qiFoxij7nKNMH7w6C0arolCcJxhiMyEF3UbTuOxgVONWyqFk/vScGGSYj9ytQYjv0L1cFyOVhlpbu+tnptulfQ3faMpjROE/eYxTIoY7OPDgWbFqEUJ4eEhKwAC7aYIYUhvoI/8k+4u64nsyEJQdHXmB+OY3WE5TegYUCuqSsdPY74eO1o8dDgxF363eggDSBmYqSSs+Tpu9TNhJG7iJFX1tiAPwmKvjVrJOac= ozeneva@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-11-21T21:49:46.962 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-11-21T21:49:46.994 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFG1CoPZEh4KBANdJZIzLsutHsF6tAYZTdDdEW+5E+8YyNHOy42D8iBLg+eCAbIl4SFmJTsPNtyO4L8/Ut79tkG3usA9nuGoAwGYuctZUTgOiTiV61940Y+NTvwan9vFNoknVi/3qIGNSXdm90I1aWRTKLCXRbGA1qvu+qyJv93O8LFaPni2TJmtAebY/xf0fHhW2fURUoeoCsoobrTkPA8SojT0A3SM+31iOD7sxbmsqLBR790wz/FHh152PiqpupIijS9TaZgGuVpejM3rgAC7H2kVmKJlrmDlmfcbKLS6BBi+uUDi6ta8whnDRq03nP/4LJE7ZDIxX3I4mHztZeJggURlsfpJdo99+VHkeWlGDBIP41QgD4UX1neuMcIO3nuEnZmLeJyvo9JOR/I7AudbQCDPjSMNbrF+el70mWLhvPe33NA3x6eOBRPc3r+qrSwMhXEPatZlbywJRY+O6yC2TYqdlUKsUEVA/D0Iw7tO+pf3yH3Y/iUDJJLvViRZs= Arjun@localhost.localdomain", "key_options": null, "keyfile": "/home/arjsharm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "arjsharm", "validate_certs": true} 2021-11-21T21:49:47.006 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFG1CoPZEh4KBANdJZIzLsutHsF6tAYZTdDdEW+5E+8YyNHOy42D8iBLg+eCAbIl4SFmJTsPNtyO4L8/Ut79tkG3usA9nuGoAwGYuctZUTgOiTiV61940Y+NTvwan9vFNoknVi/3qIGNSXdm90I1aWRTKLCXRbGA1qvu+qyJv93O8LFaPni2TJmtAebY/xf0fHhW2fURUoeoCsoobrTkPA8SojT0A3SM+31iOD7sxbmsqLBR790wz/FHh152PiqpupIijS9TaZgGuVpejM3rgAC7H2kVmKJlrmDlmfcbKLS6BBi+uUDi6ta8whnDRq03nP/4LJE7ZDIxX3I4mHztZeJggURlsfpJdo99+VHkeWlGDBIP41QgD4UX1neuMcIO3nuEnZmLeJyvo9JOR/I7AudbQCDPjSMNbrF+el70mWLhvPe33NA3x6eOBRPc3r+qrSwMhXEPatZlbywJRY+O6yC2TYqdlUKsUEVA/D0Iw7tO+pf3yH3Y/iUDJJLvViRZs= Arjun@localhost.localdomain", "key_options": null, "keyfile": "/home/arjsharm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "arjsharm", "validate_certs": true} 2021-11-21T21:49:47.017 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-11-21T21:49:47.060 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-11-21T21:49:47.112 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXZDAUNEK4+/6Y+ywEtFnabnfvZmzzHhO6y4PbRHgx8SoQ8+7au/5ElB0vjjiJJSChnDymdqP0UzwcShzhWA8is6K9c6tXUHg6NcgV5tN2xYYmGIIWwm66Z1kYH6u1x2Jkv9LO04rXgebkzXPPYK401agCWjkjUv5acZE12+M/B6d4h2lpFjzBqZ/mmRNp0tHdCapdG/nS1i6GRi3Yi3fQ89CbiZArG9268DSEnLe+npeyBMVdDkWpVIytQ6pU1+g0prrePIVaMvrK3yEooduD4HeLr4liqkINj6rCs90xedPcFi0++UaLFRcgWh3OOnrI952Oe19dx5hlbmcnMPGjaXnkng2keWFyZMOQRRti5l8C/TXPUTaYGWEoqFEEabIKnCura0NWuZHpeF9xEK0tnvNVeauuUfkPxWQ7PH8QQ/9XjOkKbmy0+2q9tug8RC5i5NwcRE4W3ijM7pLwhWMdxkmIOe3w1taPq8HU95hMQchl2749oj8DRyVePVK9aH0= melissali@melissali-ubuntu", "key_options": null, "keyfile": "/home/melissali/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "melissali", "validate_certs": true} 2021-11-21T21:49:47.114 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXZDAUNEK4+/6Y+ywEtFnabnfvZmzzHhO6y4PbRHgx8SoQ8+7au/5ElB0vjjiJJSChnDymdqP0UzwcShzhWA8is6K9c6tXUHg6NcgV5tN2xYYmGIIWwm66Z1kYH6u1x2Jkv9LO04rXgebkzXPPYK401agCWjkjUv5acZE12+M/B6d4h2lpFjzBqZ/mmRNp0tHdCapdG/nS1i6GRi3Yi3fQ89CbiZArG9268DSEnLe+npeyBMVdDkWpVIytQ6pU1+g0prrePIVaMvrK3yEooduD4HeLr4liqkINj6rCs90xedPcFi0++UaLFRcgWh3OOnrI952Oe19dx5hlbmcnMPGjaXnkng2keWFyZMOQRRti5l8C/TXPUTaYGWEoqFEEabIKnCura0NWuZHpeF9xEK0tnvNVeauuUfkPxWQ7PH8QQ/9XjOkKbmy0+2q9tug8RC5i5NwcRE4W3ijM7pLwhWMdxkmIOe3w1taPq8HU95hMQchl2749oj8DRyVePVK9aH0= melissali@melissali-ubuntu", "key_options": null, "keyfile": "/home/melissali/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "melissali", "validate_certs": true} 2021-11-21T21:49:47.124 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-11-21T21:49:47.169 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-11-21T21:49:47.224 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} 2021-11-21T21:49:47.235 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYdmNB3np8OLc49B4cAh59CdNYIb4fAq2XBQ1sbOZMMZmhpFobyy+i6s4lBnovMNWAB6HmpsECHrKRU4IFl9lQ7vV+++J08SLoAkl6G0Gk9tmbgEs+/ovCZg+qUfZA1ZccSAPHmtqfZOiJcnlIlhQq2BoFnLUwmpeIQ4H5D+Y2q7DMdyGkdCenH80aD5KuiwLIyiBL9q7RYz0U5cdCO3wDFU4+5XuZiT8VYDUgp3mWPpAiPdGsJmCn41tpt3PViPDQlKh4vLW0ieQUJI/v2f6oHbM9YkxSoVlwLcrjTv+Nl+dVymqVTF4UaT1kz+ZnoH6khKZxSHvKm9YHc4LNZKYoVdyCxdZvBfxbEbXpT702vLqDNGEfZSlzJs5kVqP30m8fI/yzTQeD7tgjzzrKWdLZacyo+9cWeH6nFfZAQuZ9ug1lv3UAs5x6MsidEH5u6Iymi8BE+mUsOXnSpSfUDDo5n6+CMVe93brdW21iSX9Hbuz5RKB6xile1ROHpvpi1u7VQ9Hp3FIZFFiix9S5jniyB2OqjGKTO6RULfPsJo8sXCLq9YKq9zrkpMv5wURMQKnjtS5OHUWh0cde3U/VikPIfLkzEsfLV8ICAksyuWHQJGVUMxhPmSTKtRsPScbklekT+n65rexDt4Qr73YOy9BUIPV4Mo+kkl4ugQd6VCKJ6w== ibalutoiu@cloudbasesolutions.com", "key_options": null, "keyfile": "/home/ionutbalutoiu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ionutbalutoiu", "validate_certs": true} 2021-11-21T21:49:47.247 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYdmNB3np8OLc49B4cAh59CdNYIb4fAq2XBQ1sbOZMMZmhpFobyy+i6s4lBnovMNWAB6HmpsECHrKRU4IFl9lQ7vV+++J08SLoAkl6G0Gk9tmbgEs+/ovCZg+qUfZA1ZccSAPHmtqfZOiJcnlIlhQq2BoFnLUwmpeIQ4H5D+Y2q7DMdyGkdCenH80aD5KuiwLIyiBL9q7RYz0U5cdCO3wDFU4+5XuZiT8VYDUgp3mWPpAiPdGsJmCn41tpt3PViPDQlKh4vLW0ieQUJI/v2f6oHbM9YkxSoVlwLcrjTv+Nl+dVymqVTF4UaT1kz+ZnoH6khKZxSHvKm9YHc4LNZKYoVdyCxdZvBfxbEbXpT702vLqDNGEfZSlzJs5kVqP30m8fI/yzTQeD7tgjzzrKWdLZacyo+9cWeH6nFfZAQuZ9ug1lv3UAs5x6MsidEH5u6Iymi8BE+mUsOXnSpSfUDDo5n6+CMVe93brdW21iSX9Hbuz5RKB6xile1ROHpvpi1u7VQ9Hp3FIZFFiix9S5jniyB2OqjGKTO6RULfPsJo8sXCLq9YKq9zrkpMv5wURMQKnjtS5OHUWh0cde3U/VikPIfLkzEsfLV8ICAksyuWHQJGVUMxhPmSTKtRsPScbklekT+n65rexDt4Qr73YOy9BUIPV4Mo+kkl4ugQd6VCKJ6w== ibalutoiu@cloudbasesolutions.com", "key_options": null, "keyfile": "/home/ionutbalutoiu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ionutbalutoiu", "validate_certs": true} 2021-11-21T21:49:47.302 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-11-21T21:49:47.345 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuyLB5rnyjlglcHVCoCSG7pDVJQcTYq/I2yGr3ohK9p6boWxk+zW/a2Q7+qNdKk2DtUC4WvvZ76BU56FC6/LPIUKSJLbN/Xv9/dnZcqKf+ZVlCQgMrE8yGoWt8WLtejJ1iBj0EdoDfYwDiL09ygUim1L1hNxFDOVq9RA2Lo6Tqk5lcAgjoCnU4NZY2YzTXOUSZTzsv/1bd6WVudKHJuWF6ZPb+mbbMoWqwnUZCHwjW8PgTXaPhk1929P0ftWT4ncfKnCc46LXf8OkvzQgCrLyh8964tGKPWppp3wNg2AJ/zFvRY8kkJZSxOawp9Gu8+kCNwWUmLvE9bqYJKT7EMDrU+8TMUEWLFl5fGoxoLWD113PPBcFrpUhvRAJa625eec8a/inv4cqPf6dY6qyXAReXURJm7LnHT4fDq+F6op8+tmZT6Tzkw4+YMUd9h0XDJ4pnPQpk5tjWER/HuWm73+ialXo+ERQWnnVCB1IXq1FEOeF8YAslKC7aSYIZjG9uUkM= peristocles@localhost.localdomain", "key_options": null, "keyfile": "/home/pdiazbou/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdiazbou", "validate_certs": true} 2021-11-21T21:49:47.359 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFG1CoPZEh4KBANdJZIzLsutHsF6tAYZTdDdEW+5E+8YyNHOy42D8iBLg+eCAbIl4SFmJTsPNtyO4L8/Ut79tkG3usA9nuGoAwGYuctZUTgOiTiV61940Y+NTvwan9vFNoknVi/3qIGNSXdm90I1aWRTKLCXRbGA1qvu+qyJv93O8LFaPni2TJmtAebY/xf0fHhW2fURUoeoCsoobrTkPA8SojT0A3SM+31iOD7sxbmsqLBR790wz/FHh152PiqpupIijS9TaZgGuVpejM3rgAC7H2kVmKJlrmDlmfcbKLS6BBi+uUDi6ta8whnDRq03nP/4LJE7ZDIxX3I4mHztZeJggURlsfpJdo99+VHkeWlGDBIP41QgD4UX1neuMcIO3nuEnZmLeJyvo9JOR/I7AudbQCDPjSMNbrF+el70mWLhvPe33NA3x6eOBRPc3r+qrSwMhXEPatZlbywJRY+O6yC2TYqdlUKsUEVA/D0Iw7tO+pf3yH3Y/iUDJJLvViRZs= Arjun@localhost.localdomain", "key_options": null, "keyfile": "/home/arjsharm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "arjsharm", "validate_certs": true} 2021-11-21T21:49:47.360 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuyLB5rnyjlglcHVCoCSG7pDVJQcTYq/I2yGr3ohK9p6boWxk+zW/a2Q7+qNdKk2DtUC4WvvZ76BU56FC6/LPIUKSJLbN/Xv9/dnZcqKf+ZVlCQgMrE8yGoWt8WLtejJ1iBj0EdoDfYwDiL09ygUim1L1hNxFDOVq9RA2Lo6Tqk5lcAgjoCnU4NZY2YzTXOUSZTzsv/1bd6WVudKHJuWF6ZPb+mbbMoWqwnUZCHwjW8PgTXaPhk1929P0ftWT4ncfKnCc46LXf8OkvzQgCrLyh8964tGKPWppp3wNg2AJ/zFvRY8kkJZSxOawp9Gu8+kCNwWUmLvE9bqYJKT7EMDrU+8TMUEWLFl5fGoxoLWD113PPBcFrpUhvRAJa625eec8a/inv4cqPf6dY6qyXAReXURJm7LnHT4fDq+F6op8+tmZT6Tzkw4+YMUd9h0XDJ4pnPQpk5tjWER/HuWm73+ialXo+ERQWnnVCB1IXq1FEOeF8YAslKC7aSYIZjG9uUkM= peristocles@localhost.localdomain", "key_options": null, "keyfile": "/home/pdiazbou/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdiazbou", "validate_certs": true} 2021-11-21T21:49:47.412 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKjxTgyhL1yFriyNvvuhUc6SE8+6f6lT6FTF8cZL9uiz matan.brz@gmail.com\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8l7qvSQR7fnZdQ1yB1F1jgde0OmLcwcDFzc/o2dYBb matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-11-21T21:49:47.434 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2iBYL614KRYvFhow408yzMljYRdfyMmbnkUVN8waja4RCsmxTpvIU1aBQ02bMTsY/0n+WMWAbI1D6C1HbuG+JbFYBLSgPCTSG8FIpukFU8y1by8piPu2/ZBD1XdFcPO6u1Nr/tZC08EXhfqvnDAVJUyg6rLLs5+TFKAiPSuv5GyrMRWvMI8eoPk+4u2r/ThxXZTDaIfJhPXlULMLL5ZxR4KZU+WtWbCGSFDGjSugCqdTzpZyX5jUq9C9JB83RxojCtI5n9rCoXLcGOvW9HBnVMxhS56zE8BwxLxLhGy6S1lOoyBPzu7w6XIc160JjCK5iAmlXRqG93nWEVrhZkxCz wangxu298@foxmail.com", "key_options": null, "keyfile": "/home/xuwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuwang", "validate_certs": true} 2021-11-21T21:49:47.467 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXZDAUNEK4+/6Y+ywEtFnabnfvZmzzHhO6y4PbRHgx8SoQ8+7au/5ElB0vjjiJJSChnDymdqP0UzwcShzhWA8is6K9c6tXUHg6NcgV5tN2xYYmGIIWwm66Z1kYH6u1x2Jkv9LO04rXgebkzXPPYK401agCWjkjUv5acZE12+M/B6d4h2lpFjzBqZ/mmRNp0tHdCapdG/nS1i6GRi3Yi3fQ89CbiZArG9268DSEnLe+npeyBMVdDkWpVIytQ6pU1+g0prrePIVaMvrK3yEooduD4HeLr4liqkINj6rCs90xedPcFi0++UaLFRcgWh3OOnrI952Oe19dx5hlbmcnMPGjaXnkng2keWFyZMOQRRti5l8C/TXPUTaYGWEoqFEEabIKnCura0NWuZHpeF9xEK0tnvNVeauuUfkPxWQ7PH8QQ/9XjOkKbmy0+2q9tug8RC5i5NwcRE4W3ijM7pLwhWMdxkmIOe3w1taPq8HU95hMQchl2749oj8DRyVePVK9aH0= melissali@melissali-ubuntu", "key_options": null, "keyfile": "/home/melissali/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "melissali", "validate_certs": true} 2021-11-21T21:49:47.478 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2iBYL614KRYvFhow408yzMljYRdfyMmbnkUVN8waja4RCsmxTpvIU1aBQ02bMTsY/0n+WMWAbI1D6C1HbuG+JbFYBLSgPCTSG8FIpukFU8y1by8piPu2/ZBD1XdFcPO6u1Nr/tZC08EXhfqvnDAVJUyg6rLLs5+TFKAiPSuv5GyrMRWvMI8eoPk+4u2r/ThxXZTDaIfJhPXlULMLL5ZxR4KZU+WtWbCGSFDGjSugCqdTzpZyX5jUq9C9JB83RxojCtI5n9rCoXLcGOvW9HBnVMxhS56zE8BwxLxLhGy6S1lOoyBPzu7w6XIc160JjCK5iAmlXRqG93nWEVrhZkxCz wangxu298@foxmail.com", "key_options": null, "keyfile": "/home/xuwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuwang", "validate_certs": true} 2021-11-21T21:49:47.500 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkFTEUxQkQ1cplOLOf86Teu4qTYGkMqWtjIwzsU/AcimjFolSI5fJ3B72PH7bhQKkSnOlTjiOos2LrD5i/XgxkJD6icVWwHO11lczsq/8K5x0ZMJbJaiT3QphBB4yItoge2Y4/7g4o/78Ne+U7eZ1L5fgeO1fExtIaEXK1lS8pVlpZQXyproTIzp1WLHoaXb38Mxc5jAtuvEKfEycdX3c8k/9ti2qBCZnmjPH05WOxkqCRyNBT+bSJZyFMYt9RDUuycn3BYVSXTAzPW0qwgvKY5vij+qiFoxij7nKNMH7w6C0arolCcJxhiMyEF3UbTuOxgVONWyqFk/vScGGSYj9ytQYjv0L1cFyOVhlpbu+tnptulfQ3faMpjROE/eYxTIoY7OPDgWbFqEUJ4eEhKwAC7aYIYUhvoI/8k+4u64nsyEJQdHXmB+OY3WE5TegYUCuqSsdPY74eO1o8dDgxF363eggDSBmYqSSs+Tpu9TNhJG7iJFX1tiAPwmKvjVrJOac= ozeneva@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-11-21T21:49:47.532 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKSdu3rI8o3qId+GU5KYsxB9CLPTODJFiwy+ww6SgrNKgPfiaSUsZAvcQwoxHfuaEXj6VGaOS6HyFFmvHKhzbecSQCzPIucIuKJJDA7VTaA9Jet0aT2b8Vk4p0Ud9j4WoausmxEdBQC+ZpJxSuqajplejYK1Yi3YEqXSCvM6Md76HCustS8efPvzLsLYttDd34Lw0/zp7ALE8hrepLYJgzQS5Z6btvQEjL7lvtHk80/Y+M6ZIQAl6nPcjb8u+wfs/lxcRA5i94GsZhEoYAeMQptJNvOaA5vYK4QtNyV6lyjGVJl4RUUAtLKKp4JE/kdx7sa0Gy99L+u4MjzBoP5B7SkdJnJx7FioXEen5l5DpiJ6xRixw+SjbqEffW4HHI5+1k6jP6kuHPbWfcrw693+aiFVDQA+myzfWo7iDTjY2H5CtnCh+wr1t3VktXhoyPob/XfMPzMNg4W8nA9KVePwzEXSy2Ro0t71GiFQqBviUuqfktqz4a52PqGyFHtskhFAH3H/h6JYM88vfc7Bw7P/u4eVaoCNgNIbLA/kmf689Q1H2KENj2SX7CDwNL9GCOy/pg89qai63+PFT8258zTi0TywWduP1D0DXXtIB3e5K1yK4Zt6VCinXZAze4bsT+z8y4f9W2Xs/FhhtUmo3mOiDJuaGmuWRzHQQTjL//v/Rimw== blaine@brgardne-mac", "key_options": null, "keyfile": "/home/blaine/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "blaine", "validate_certs": true} 2021-11-21T21:49:47.586 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYdmNB3np8OLc49B4cAh59CdNYIb4fAq2XBQ1sbOZMMZmhpFobyy+i6s4lBnovMNWAB6HmpsECHrKRU4IFl9lQ7vV+++J08SLoAkl6G0Gk9tmbgEs+/ovCZg+qUfZA1ZccSAPHmtqfZOiJcnlIlhQq2BoFnLUwmpeIQ4H5D+Y2q7DMdyGkdCenH80aD5KuiwLIyiBL9q7RYz0U5cdCO3wDFU4+5XuZiT8VYDUgp3mWPpAiPdGsJmCn41tpt3PViPDQlKh4vLW0ieQUJI/v2f6oHbM9YkxSoVlwLcrjTv+Nl+dVymqVTF4UaT1kz+ZnoH6khKZxSHvKm9YHc4LNZKYoVdyCxdZvBfxbEbXpT702vLqDNGEfZSlzJs5kVqP30m8fI/yzTQeD7tgjzzrKWdLZacyo+9cWeH6nFfZAQuZ9ug1lv3UAs5x6MsidEH5u6Iymi8BE+mUsOXnSpSfUDDo5n6+CMVe93brdW21iSX9Hbuz5RKB6xile1ROHpvpi1u7VQ9Hp3FIZFFiix9S5jniyB2OqjGKTO6RULfPsJo8sXCLq9YKq9zrkpMv5wURMQKnjtS5OHUWh0cde3U/VikPIfLkzEsfLV8ICAksyuWHQJGVUMxhPmSTKtRsPScbklekT+n65rexDt4Qr73YOy9BUIPV4Mo+kkl4ugQd6VCKJ6w== ibalutoiu@cloudbasesolutions.com", "key_options": null, "keyfile": "/home/ionutbalutoiu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ionutbalutoiu", "validate_certs": true} 2021-11-21T21:49:47.588 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKSdu3rI8o3qId+GU5KYsxB9CLPTODJFiwy+ww6SgrNKgPfiaSUsZAvcQwoxHfuaEXj6VGaOS6HyFFmvHKhzbecSQCzPIucIuKJJDA7VTaA9Jet0aT2b8Vk4p0Ud9j4WoausmxEdBQC+ZpJxSuqajplejYK1Yi3YEqXSCvM6Md76HCustS8efPvzLsLYttDd34Lw0/zp7ALE8hrepLYJgzQS5Z6btvQEjL7lvtHk80/Y+M6ZIQAl6nPcjb8u+wfs/lxcRA5i94GsZhEoYAeMQptJNvOaA5vYK4QtNyV6lyjGVJl4RUUAtLKKp4JE/kdx7sa0Gy99L+u4MjzBoP5B7SkdJnJx7FioXEen5l5DpiJ6xRixw+SjbqEffW4HHI5+1k6jP6kuHPbWfcrw693+aiFVDQA+myzfWo7iDTjY2H5CtnCh+wr1t3VktXhoyPob/XfMPzMNg4W8nA9KVePwzEXSy2Ro0t71GiFQqBviUuqfktqz4a52PqGyFHtskhFAH3H/h6JYM88vfc7Bw7P/u4eVaoCNgNIbLA/kmf689Q1H2KENj2SX7CDwNL9GCOy/pg89qai63+PFT8258zTi0TywWduP1D0DXXtIB3e5K1yK4Zt6VCinXZAze4bsT+z8y4f9W2Xs/FhhtUmo3mOiDJuaGmuWRzHQQTjL//v/Rimw== blaine@brgardne-mac", "key_options": null, "keyfile": "/home/blaine/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "blaine", "validate_certs": true} 2021-11-21T21:49:47.598 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-11-21T21:49:47.632 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM7LPHiaPQXdE+yt9cn7UMuRr8k/LjSUmMrnrAqp8ySQtX6OA3/NcOsxg6FJ45chZqeoXwhsYEW+7pTInDqFlOlmVynR26lWnZfk9DjBJZcloUCn8y5MR9QitNYsOMs5KwCEFmBH6sP+2vCFocXBnKQcVZLRfip3MEG4LoWYfI4ir7fLmK1Hr5G+GhtMstXvVSG4VE4IgbU9/LRIOv7P9qFn4bRJoUSiS2dDbUxVIuneA9bIEnB9ZEVGahEzlUhx6qDSs0FvrnneCQ1ayOFpxClzjT5GuKk/J3LJvWAw8CAwDXCf3FzsWQsvb5ru5VSazUI9uA3o9wWkbLW2oZPgtzI8lmwbL+MT58T92tqaJOPID3IZDGjzNrRqU5BQYpvFYHpHRRiqV625og+ktYfXgO+dTbgAxScomqutydI4cyJusDtnWtljWnEocS+2hUSZpmB6B3Wj70TuuzsbYrcrJ/NgRpQNX4dwHCJl6vPT+xkoLh1imYAC7klf7cnrOyGcs= choffman@choffman.net", "key_options": null, "keyfile": "/home/choffman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "choffman", "validate_certs": true} 2021-11-21T21:49:47.715 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM7LPHiaPQXdE+yt9cn7UMuRr8k/LjSUmMrnrAqp8ySQtX6OA3/NcOsxg6FJ45chZqeoXwhsYEW+7pTInDqFlOlmVynR26lWnZfk9DjBJZcloUCn8y5MR9QitNYsOMs5KwCEFmBH6sP+2vCFocXBnKQcVZLRfip3MEG4LoWYfI4ir7fLmK1Hr5G+GhtMstXvVSG4VE4IgbU9/LRIOv7P9qFn4bRJoUSiS2dDbUxVIuneA9bIEnB9ZEVGahEzlUhx6qDSs0FvrnneCQ1ayOFpxClzjT5GuKk/J3LJvWAw8CAwDXCf3FzsWQsvb5ru5VSazUI9uA3o9wWkbLW2oZPgtzI8lmwbL+MT58T92tqaJOPID3IZDGjzNrRqU5BQYpvFYHpHRRiqV625og+ktYfXgO+dTbgAxScomqutydI4cyJusDtnWtljWnEocS+2hUSZpmB6B3Wj70TuuzsbYrcrJ/NgRpQNX4dwHCJl6vPT+xkoLh1imYAC7klf7cnrOyGcs= choffman@choffman.net", "key_options": null, "keyfile": "/home/choffman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "choffman", "validate_certs": true} 2021-11-21T21:49:47.722 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuyLB5rnyjlglcHVCoCSG7pDVJQcTYq/I2yGr3ohK9p6boWxk+zW/a2Q7+qNdKk2DtUC4WvvZ76BU56FC6/LPIUKSJLbN/Xv9/dnZcqKf+ZVlCQgMrE8yGoWt8WLtejJ1iBj0EdoDfYwDiL09ygUim1L1hNxFDOVq9RA2Lo6Tqk5lcAgjoCnU4NZY2YzTXOUSZTzsv/1bd6WVudKHJuWF6ZPb+mbbMoWqwnUZCHwjW8PgTXaPhk1929P0ftWT4ncfKnCc46LXf8OkvzQgCrLyh8964tGKPWppp3wNg2AJ/zFvRY8kkJZSxOawp9Gu8+kCNwWUmLvE9bqYJKT7EMDrU+8TMUEWLFl5fGoxoLWD113PPBcFrpUhvRAJa625eec8a/inv4cqPf6dY6qyXAReXURJm7LnHT4fDq+F6op8+tmZT6Tzkw4+YMUd9h0XDJ4pnPQpk5tjWER/HuWm73+ialXo+ERQWnnVCB1IXq1FEOeF8YAslKC7aSYIZjG9uUkM= peristocles@localhost.localdomain", "key_options": null, "keyfile": "/home/pdiazbou/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdiazbou", "validate_certs": true} 2021-11-21T21:49:47.746 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:47 +0000 (0:00:15.864) 0:00:41.681 ******* 2021-11-21T21:49:47.810 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-11-21T21:49:47.839 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:47 +0000 (0:00:00.093) 0:00:41.774 ******* 2021-11-21T21:49:47.848 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} 2021-11-21T21:49:47.849 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2iBYL614KRYvFhow408yzMljYRdfyMmbnkUVN8waja4RCsmxTpvIU1aBQ02bMTsY/0n+WMWAbI1D6C1HbuG+JbFYBLSgPCTSG8FIpukFU8y1by8piPu2/ZBD1XdFcPO6u1Nr/tZC08EXhfqvnDAVJUyg6rLLs5+TFKAiPSuv5GyrMRWvMI8eoPk+4u2r/ThxXZTDaIfJhPXlULMLL5ZxR4KZU+WtWbCGSFDGjSugCqdTzpZyX5jUq9C9JB83RxojCtI5n9rCoXLcGOvW9HBnVMxhS56zE8BwxLxLhGy6S1lOoyBPzu7w6XIc160JjCK5iAmlXRqG93nWEVrhZkxCz wangxu298@foxmail.com", "key_options": null, "keyfile": "/home/xuwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuwang", "validate_certs": true} 2021-11-21T21:49:47.851 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.852 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.866 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.868 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.879 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.890 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.901 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:47.903 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKSdu3rI8o3qId+GU5KYsxB9CLPTODJFiwy+ww6SgrNKgPfiaSUsZAvcQwoxHfuaEXj6VGaOS6HyFFmvHKhzbecSQCzPIucIuKJJDA7VTaA9Jet0aT2b8Vk4p0Ud9j4WoausmxEdBQC+ZpJxSuqajplejYK1Yi3YEqXSCvM6Md76HCustS8efPvzLsLYttDd34Lw0/zp7ALE8hrepLYJgzQS5Z6btvQEjL7lvtHk80/Y+M6ZIQAl6nPcjb8u+wfs/lxcRA5i94GsZhEoYAeMQptJNvOaA5vYK4QtNyV6lyjGVJl4RUUAtLKKp4JE/kdx7sa0Gy99L+u4MjzBoP5B7SkdJnJx7FioXEen5l5DpiJ6xRixw+SjbqEffW4HHI5+1k6jP6kuHPbWfcrw693+aiFVDQA+myzfWo7iDTjY2H5CtnCh+wr1t3VktXhoyPob/XfMPzMNg4W8nA9KVePwzEXSy2Ro0t71GiFQqBviUuqfktqz4a52PqGyFHtskhFAH3H/h6JYM88vfc7Bw7P/u4eVaoCNgNIbLA/kmf689Q1H2KENj2SX7CDwNL9GCOy/pg89qai63+PFT8258zTi0TywWduP1D0DXXtIB3e5K1yK4Zt6VCinXZAze4bsT+z8y4f9W2Xs/FhhtUmo3mOiDJuaGmuWRzHQQTjL//v/Rimw== blaine@brgardne-mac", "key_options": null, "keyfile": "/home/blaine/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "blaine", "validate_certs": true} 2021-11-21T21:49:47.915 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFG1CoPZEh4KBANdJZIzLsutHsF6tAYZTdDdEW+5E+8YyNHOy42D8iBLg+eCAbIl4SFmJTsPNtyO4L8/Ut79tkG3usA9nuGoAwGYuctZUTgOiTiV61940Y+NTvwan9vFNoknVi/3qIGNSXdm90I1aWRTKLCXRbGA1qvu+qyJv93O8LFaPni2TJmtAebY/xf0fHhW2fURUoeoCsoobrTkPA8SojT0A3SM+31iOD7sxbmsqLBR790wz/FHh152PiqpupIijS9TaZgGuVpejM3rgAC7H2kVmKJlrmDlmfcbKLS6BBi+uUDi6ta8whnDRq03nP/4LJE7ZDIxX3I4mHztZeJggURlsfpJdo99+VHkeWlGDBIP41QgD4UX1neuMcIO3nuEnZmLeJyvo9JOR/I7AudbQCDPjSMNbrF+el70mWLhvPe33NA3x6eOBRPc3r+qrSwMhXEPatZlbywJRY+O6yC2TYqdlUKsUEVA/D0Iw7tO+pf3yH3Y/iUDJJLvViRZs= Arjun@localhost.localdomain", "key_options": null, "keyfile": "/home/arjsharm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "arjsharm", "validate_certs": true} 2021-11-21T21:49:47.929 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.940 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.941 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.951 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.953 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.962 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.964 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.974 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.985 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:47.996 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-11-21T21:49:47.997 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.008 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:48.010 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM7LPHiaPQXdE+yt9cn7UMuRr8k/LjSUmMrnrAqp8ySQtX6OA3/NcOsxg6FJ45chZqeoXwhsYEW+7pTInDqFlOlmVynR26lWnZfk9DjBJZcloUCn8y5MR9QitNYsOMs5KwCEFmBH6sP+2vCFocXBnKQcVZLRfip3MEG4LoWYfI4ir7fLmK1Hr5G+GhtMstXvVSG4VE4IgbU9/LRIOv7P9qFn4bRJoUSiS2dDbUxVIuneA9bIEnB9ZEVGahEzlUhx6qDSs0FvrnneCQ1ayOFpxClzjT5GuKk/J3LJvWAw8CAwDXCf3FzsWQsvb5ru5VSazUI9uA3o9wWkbLW2oZPgtzI8lmwbL+MT58T92tqaJOPID3IZDGjzNrRqU5BQYpvFYHpHRRiqV625og+ktYfXgO+dTbgAxScomqutydI4cyJusDtnWtljWnEocS+2hUSZpmB6B3Wj70TuuzsbYrcrJ/NgRpQNX4dwHCJl6vPT+xkoLh1imYAC7klf7cnrOyGcs= choffman@choffman.net", "key_options": null, "keyfile": "/home/choffman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "choffman", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.039 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:48.074 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXZDAUNEK4+/6Y+ywEtFnabnfvZmzzHhO6y4PbRHgx8SoQ8+7au/5ElB0vjjiJJSChnDymdqP0UzwcShzhWA8is6K9c6tXUHg6NcgV5tN2xYYmGIIWwm66Z1kYH6u1x2Jkv9LO04rXgebkzXPPYK401agCWjkjUv5acZE12+M/B6d4h2lpFjzBqZ/mmRNp0tHdCapdG/nS1i6GRi3Yi3fQ89CbiZArG9268DSEnLe+npeyBMVdDkWpVIytQ6pU1+g0prrePIVaMvrK3yEooduD4HeLr4liqkINj6rCs90xedPcFi0++UaLFRcgWh3OOnrI952Oe19dx5hlbmcnMPGjaXnkng2keWFyZMOQRRti5l8C/TXPUTaYGWEoqFEEabIKnCura0NWuZHpeF9xEK0tnvNVeauuUfkPxWQ7PH8QQ/9XjOkKbmy0+2q9tug8RC5i5NwcRE4W3ijM7pLwhWMdxkmIOe3w1taPq8HU95hMQchl2749oj8DRyVePVK9aH0= melissali@melissali-ubuntu", "key_options": null, "keyfile": "/home/melissali/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "melissali", "validate_certs": true} 2021-11-21T21:49:48.098 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:48 +0000 (0:00:00.259) 0:00:42.034 ******* 2021-11-21T21:49:48.108 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:48.109 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e19 2021-11-21T21:49:48.111 INFO:teuthology.task.ansible.out:02a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b 2021-11-21T21:49:48.112 INFO:teuthology.task.ansible.out:9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-11-21T21:49:48.178 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:48.180 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYdmNB3np8OLc49B4cAh59CdNYIb4fAq2XBQ1sbOZMMZmhpFobyy+i6s4lBnovMNWAB6HmpsECHrKRU4IFl9lQ7vV+++J08SLoAkl6G0Gk9tmbgEs+/ovCZg+qUfZA1ZccSAPHmtqfZOiJcnlIlhQq2BoFnLUwmpeIQ4H5D+Y2q7DMdyGkdCenH80aD5KuiwLIyiBL9q7RYz0U5cdCO3wDFU4+5XuZiT8VYDUgp3mWPpAiPdGsJmCn41tpt3PViPDQlKh4vLW0ieQUJI/v2f6oHbM9YkxSoVlwLcrjTv+Nl+dVymqVTF4UaT1kz+ZnoH6khKZxSHvKm9YHc4LNZKYoVdyCxdZvBfxbEbXpT702vLqDNGEfZSlzJs5kVqP30m8fI/yzTQeD7tgjzzrKWdLZacyo+9cWeH6nFfZAQuZ9ug1lv3UAs5x6MsidEH5u6Iymi8BE+mUsOXnSpSfUDDo5n6+CMVe93brdW21iSX9Hbuz5RKB6xile1ROHpvpi1u7VQ9Hp3FIZFFiix9S5jniyB2OqjGKTO6RULfPsJo8sXCLq9YKq9zrkpMv5wURMQKnjtS5OHUWh0cde3U/VikPIfLkzEsfLV8ICAksyuWHQJGVUMxhPmSTKtRsPScbklekT+n65rexDt4Qr73YOy9BUIPV4Mo+kkl4ugQd6VCKJ6w== ibalutoiu@cloudbasesolutions.com", "key_options": null, "keyfile": "/home/ionutbalutoiu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ionutbalutoiu", "validate_certs": true} 2021-11-21T21:49:48.189 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:48.191 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2021-11-21T21:49:48.193 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.202 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.204 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.214 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.215 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} ok: [gibba017.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTU 2021-11-21T21:49:48.217 INFO:teuthology.task.ansible.out:yWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-11-21T21:49:48.226 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.228 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.238 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.240 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.250 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.252 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.261 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.272 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.274 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuyLB5rnyjlglcHVCoCSG7pDVJQcTYq/I2yGr3ohK9p6boWxk+zW/a2Q7+qNdKk2DtUC4WvvZ76BU56FC6/LPIUKSJLbN/Xv9/dnZcqKf+ZVlCQgMrE8yGoWt8WLtejJ1iBj0EdoDfYwDiL09ygUim1L1hNxFDOVq9RA2Lo6Tqk5lcAgjoCnU4NZY2YzTXOUSZTzsv/1bd6WVudKHJuWF6ZPb+mbbMoWqwnUZCHwjW8PgTXaPhk1929P0ftWT4ncfKnCc46LXf8OkvzQgCrLyh8964tGKPWppp3wNg2AJ/zFvRY8kkJZSxOawp9Gu8+kCNwWUmLvE9bqYJKT7EMDrU+8TMUEWLFl5fGoxoLWD113PPBcFrpUhvRAJa625eec8a/inv4cqPf6dY6qyXAReXURJm7LnHT4fDq+F6op8+tmZT6Tzkw4+YMUd9h0XDJ4pnPQpk5tjWER/HuWm73+ialXo+ERQWnnVCB1IXq1FEOeF8YAslKC7aSYIZjG9uUkM= peristocles@localhost.localdomain", "key_options": null, "keyfile": "/home/pdiazbou/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdiazbou", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJa 2021-11-21T21:49:48.275 INFO:teuthology.task.ansible.out:oA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.287 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.289 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.299 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-11-21T21:49:48.301 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.313 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.314 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.325 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.327 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.337 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.338 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linu 2021-11-21T21:49:48.340 INFO:teuthology.task.ansible.out:xkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pa 2021-11-21T21:49:48.342 INFO:teuthology.task.ansible.out:ntograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.352 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.353 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.355 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.357 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.367 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.369 INFO:teuthology.task.ansible.out: skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:48.369 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2iBYL614KRYvFhow408yzMljYRdfyMmbnkUVN8waja4RCsmxTpvIU1aBQ02bMTsY/0n+WMWAbI1D6C1HbuG+JbFYBLSgPCTSG8FIpukFU8y1by8piPu2/ZBD1XdFcPO6u1Nr/tZC08EXhfqvnDAVJUyg6rLLs5+TFKAiPSuv5GyrMRWvMI8eoPk+4u2r/ThxXZTDaIfJhPXlULMLL5ZxR4KZU+WtWbCGSFDGjSugCqdTzpZyX5jUq9C9JB83RxojCtI5n9rCoXLcGOvW9HBnVMxhS56zE8BwxLxLhGy6S1lOoyBPzu7w6XIc160JjCK5iAmlXRqG93nWEVrhZkxCz wangxu298@foxmail.com", "key_options": null, "keyfile": "/home/xuwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuwang", "validate_certs": true} 2021-11-21T21:49:48.370 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result 2021-11-21T21:49:48.373 INFO:teuthology.task.ansible.out: was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.382 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.383 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.394 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.396 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.407 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.408 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.410 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.410 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.421 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.423 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.433 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.435 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.446 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.447 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33J 2021-11-21T21:49:48.449 INFO:teuthology.task.ansible.out:iv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.459 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.460 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.462 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.462 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:48.462 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKSdu3rI8o3qId+GU5KYsxB9CLPTODJFiwy+ww6SgrNKgPfiaSUsZAvcQwoxHfuaEXj6VGaOS6HyFFmvHKhzbecSQCzPIucIuKJJDA7VTaA9Jet0aT2b8Vk4p0Ud9j4WoausmxEdBQC+ZpJxSuqajplejYK1Yi3YEqXSCvM6Md76HCu 2021-11-21T21:49:48.464 INFO:teuthology.task.ansible.out:stS8efPvzLsLYttDd34Lw0/zp7ALE8hrepLYJgzQS5Z6btvQEjL7lvtHk80/Y+M6ZIQAl6nPcjb8u+wfs/lxcRA5i94GsZhEoYAeMQptJNvOaA5vYK4QtNyV6lyjGVJl4RUUAtLKKp4JE/kdx7sa0Gy99L+u4MjzBoP5B7SkdJnJx7FioXEen5l5DpiJ6xRixw+SjbqEffW4HHI5+1k6jP6kuHPbWfcrw693+aiFVDQA+myzfWo7iDTjY2H5CtnCh+wr1t3VktXhoyPob/XfMPzMNg4W8nA9KVePwzEXSy2Ro0t71GiFQqBviUuqfktqz4a52PqGyFHtskhFAH3H/h6JYM88vfc7Bw7P/u4eVaoCNgNIbLA/kmf689Q1H2KENj2SX7CDwNL9GCOy/pg89qai63+PFT8258zTi0TywWduP1D0DXXtIB3e5K1yK4Zt6VCinXZAze4bsT+z8y4f9W2Xs/FhhtUmo3mOiDJuaGmuWRzHQQTjL//v/Rimw== blaine@brgardne-mac", "key_options": null, "keyfile": "/home/blaine/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "blaine", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.474 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.475 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.477 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.488 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.489 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.491 INFO:teuthology.task.ansible.out: skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.501 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.503 INFO:teuthology.task.ansible.out: skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.505 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.506 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:48.507 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.518 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.519 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.521 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.522 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.533 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.545 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-11-21T21:49:48.546 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.548 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.549 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.549 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.551 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.553 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.564 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.566 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.568 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba036.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM7LPHiaPQXdE+yt9cn7UMuRr8k/LjSUmMrnrAqp8ySQtX6OA3/NcOsxg6FJ45chZqeoXwhsYEW+7pTInDqFlOlmVynR26lWnZfk9DjBJZcloUCn8y5MR9QitNYsOMs5KwCEFmBH6sP+2vCFocXBnKQcVZLRfip3MEG4LoWYfI4ir7fLmK1Hr5G+GhtMstXvVSG4VE4IgbU9/LRIOv7P9qFn4bRJoUSiS2dDbUxVIuneA9bIEnB9ZEVGahEzlUhx6qDSs0FvrnneCQ1ayOFpxClzjT5GuKk/J3LJvWAw8CAwDXCf3FzsWQsvb5ru5VSazUI9uA3o9wWkbLW2oZPgtzI8lmwbL+MT58T92tqaJOPID3IZDGjzNrRqU5BQYpvFYHpHRRiqV625og+ktYfXgO+dTbgAxScomqutydI4cyJusDtnWtljWnEocS+2hUSZpmB6B3Wj70TuuzsbYrcrJ/NgRpQNX4dwHCJl6vPT+xkoLh1imYAC7klf7cnrOyGcs= choffman@choffman.net", "key_options": null, "keyfile": "/home/choffman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "choffman", "validate_certs": true} 2021-11-21T21:49:48.570 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.570 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.583 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.585 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.605 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-21T21:49:48.650 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:48.652 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"n 2021-11-21T21:49:48.654 INFO:teuthology.task.ansible.out:ame": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.654 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.655 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8ed 2021-11-21T21:49:48.656 INFO:teuthology.task.ansible.out:b80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.657 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d23 2021-11-21T21:49:48.658 INFO:teuthology.task.ansible.out:9d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12 2021-11-21T21:49:48.660 INFO:teuthology.task.ansible.out:a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.661 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.662 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.663 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.667 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eK 2021-11-21T21:49:48.668 INFO:teuthology.task.ansible.out:TfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was 2021-11-21T21:49:48.670 INFO:teuthology.task.ansible.out: False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.670 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, 2021-11-21T21:49:48.672 INFO:teuthology.task.ansible.out:"item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.672 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.672 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'n 2021-11-21T21:49:48.674 INFO:teuthology.task.ansible.out:ame': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.701 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:48 +0000 (0:00:00.601) 0:00:42.636 ******* 2021-11-21T21:49:48.707 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.711 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.712 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.713 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.714 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.714 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.716 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.716 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.716 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.717 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.718 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.719 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.720 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.722 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.736 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.739 INFO:teuthology.task.ansible.out: skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.741 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.743 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.744 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.745 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": fal 2021-11-21T21:49:48.746 INFO:teuthology.task.ansible.out:se, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.761 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.762 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@ 2021-11-21T21:49:48.764 INFO:teuthology.task.ansible.out:t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.776 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.778 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.778 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.779 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ov 2021-11-21T21:49:48.780 INFO:teuthology.task.ansible.out:pn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.790 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.792 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.793 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.804 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.806 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.806 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.808 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.809 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.811 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.821 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.823 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f 2021-11-21T21:49:48.825 INFO:teuthology.task.ansible.out:18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.838 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399 2021-11-21T21:49:48.840 INFO:teuthology.task.ansible.out:636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.842 INFO:teuthology.task.ansible.out: skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.852 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.853 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.863 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.865 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.876 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.877 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.878 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgo 2021-11-21T21:49:48.879 INFO:teuthology.task.ansible.out:lub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.890 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.892 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.892 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.894 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.905 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.907 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.907 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.908 INFO:teuthology.task.ansible.out: skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.909 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuert 2021-11-21T21:49:48.910 INFO:teuthology.task.ansible.out:at", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.911 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.912 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.913 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.914 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.926 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.928 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.928 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.929 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h 2021-11-21T21:49:48.930 INFO:teuthology.task.ansible.out:34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.940 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.942 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.953 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.955 INFO:teuthology.task.ansible.out: skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.957 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.958 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.969 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.971 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.972 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.972 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": nu 2021-11-21T21:49:48.974 INFO:teuthology.task.ansible.out:ll, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e 2021-11-21T21:49:48.975 INFO:teuthology.task.ansible.out:4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was Fal 2021-11-21T21:49:48.977 INFO:teuthology.task.ansible.out:se"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.988 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:48.989 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.001 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.003 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.003 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.004 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.005 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b 2021-11-21T21:49:49.007 INFO:teuthology.task.ansible.out:690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.016 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.018 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.019 INFO:teuthology.task.ansible.out: skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.021 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.023 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.024 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.034 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.036 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.037 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.039 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.051 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.051 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.052 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.052 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.053 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.054 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.055 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.066 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.067 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.068 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.069 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e703121 2021-11-21T21:49:49.071 INFO:teuthology.task.ansible.out:89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.080 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.082 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.093 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.094 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.096 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.106 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.108 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.108 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.109 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.109 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.121 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.123 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.133 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.135 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.135 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.136 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", 2021-11-21T21:49:49.137 INFO:teuthology.task.ansible.out:"changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.149 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.151 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.151 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.152 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.153 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var 2021-11-21T21:49:49.155 INFO:teuthology.task.ansible.out:": "item", "changed": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.164 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.165 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.166 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.166 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.168 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "skip_reason": "Conditional result was False"} ok: [gibba036.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin 2021-11-21T21:49:49.170 INFO:teuthology.task.ansible.out:@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.180 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.182 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.203 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.205 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfb 2021-11-21T21:49:49.206 INFO:teuthology.task.ansible.out:Iz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.231 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:49 +0000 (0:00:00.530) 0:00:43.167 ******* 2021-11-21T21:49:49.239 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.241 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.241 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.242 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.243 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.245 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.245 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.247 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.247 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.248 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": f 2021-11-21T21:49:49.249 INFO:teuthology.task.ansible.out:alse, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.249 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.251 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.251 INFO:teuthology.task.ansible.out:skipping 2021-11-21T21:49:49.253 INFO:teuthology.task.ansible.out:: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.253 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG 2021-11-21T21:49:49.255 INFO:teuthology.task.ansible.out:3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.265 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.267 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.267 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.268 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "choffman", "ovpn": "choff 2021-11-21T21:49:49.269 INFO:teuthology.task.ansible.out:man@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.290 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.292 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa11 2021-11-21T21:49:49.293 INFO:teuthology.task.ansible.out:0d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.317 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:49 +0000 (0:00:00.085) 0:00:43.252 ******* 2021-11-21T21:49:49.324 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.326 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.327 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1G 2021-11-21T21:49:49.329 INFO:teuthology.task.ansible.out:eh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} TASK [Filter the revoked_users list] ******************************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:49.329 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.329 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.330 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.330 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao V 2021-11-21T21:49:49.332 INFO:teuthology.task.ansible.out:aMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.332 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INw 2021-11-21T21:49:49.334 INFO:teuthology.task.ansible.out:K6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.362 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:49 +0000 (0:00:00.045) 0:00:43.297 ******* 2021-11-21T21:49:49.370 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.372 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.372 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.385 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.386 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.387 INFO:teuthology.task.ansible.out:skipping: 2021-11-21T21:49:49.387 INFO:teuthology.task.ansible.out:[gibba016.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.388 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.388 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.389 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False 2021-11-21T21:49:49.390 INFO:teuthology.task.ansible.out:"} 2021-11-21T21:49:49.415 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:49 +0000 (0:00:00.053) 0:00:43.351 ******* 2021-11-21T21:49:49.424 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.424 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.426 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.428 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.428 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.430 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.430 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.432 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.432 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.444 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.445 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.456 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.458 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.458 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.469 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.470 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.470 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.471 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.482 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.483 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.483 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.494 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.496 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.507 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.508 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.519 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.521 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.521 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.533 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.535 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:49.535 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-11-21T21:49:49.537 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.538 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.551 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.553 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.572 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.574 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.599 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:49 +0000 (0:00:00.183) 0:00:43.534 ******* 2021-11-21T21:49:49.608 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.609 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.609 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.610 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.611 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:49.612 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-11-21T21:49:49.612 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:49.613 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.625 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.627 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba017.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.627 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.638 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.640 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.649 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.660 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.662 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.672 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-11-21T21:49:49.673 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.673 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.675 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.697 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:49 +0000 (0:00:00.098) 0:00:43.633 ******* 2021-11-21T21:49:49.705 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.706 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:49.706 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-11-21T21:49:49.708 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.708 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.709 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.709 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.720 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.731 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:49.733 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.734 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.744 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.746 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.756 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.757 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.768 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.779 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:49.781 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.791 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.793 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.803 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.804 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.814 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:49.816 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.826 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.828 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.837 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.839 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.849 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.860 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.861 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba016.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-11-21T21:49:49.863 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-11-21T21:49:49.873 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:49.875 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.885 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.896 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.897 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.907 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:49.909 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.919 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.921 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.931 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.942 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.944 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.954 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.956 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.966 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.967 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba016.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-11-21T21:49:49.978 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:49.980 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.990 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:49.992 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba019.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-11-21T21:49:50.002 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-21T21:49:50.004 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba017.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.015 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.017 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.026 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.037 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.039 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.050 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.061 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.063 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba016.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-11-21T21:49:50.064 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.075 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.086 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.088 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba019.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.099 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:50.101 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.112 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.123 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.125 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.135 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'name': 'choffman', 'ovpn': 'choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "choffman", "ovpn": "choffman@choffman.net 56rg/lHPXPRyVivND7+0Uw ca591258cda719a5a0bf5d5f4fa87976773625249d29ff6012ad6efeac33da2c"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.180 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:50 +0000 (0:00:00.482) 0:00:44.115 ******* 2021-11-21T21:49:50.189 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba016.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-11-21T21:49:50.191 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} ok: [gibba019.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-11-21T21:49:50.245 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-11-21T21:49:50.256 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-11-21T21:49:50.258 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:50.281 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:50 +0000 (0:00:00.101) 0:00:44.216 ******* 2021-11-21T21:49:50.288 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:50.289 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} ok: [gibba017.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-11-21T21:49:50.343 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-11-21T21:49:50.376 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-11-21T21:49:50.377 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-11-21T21:49:50.440 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-11-21T21:49:50.462 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-11-21T21:49:50.463 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-11-21T21:49:50.474 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-11-21T21:49:50.538 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-11-21T21:49:50.550 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-11-21T21:49:50.561 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-11-21T21:49:50.562 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-11-21T21:49:50.626 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-11-21T21:49:50.638 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-11-21T21:49:50.650 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-11-21T21:49:50.652 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-11-21T21:49:50.725 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-11-21T21:49:50.727 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-11-21T21:49:50.747 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-11-21T21:49:50.749 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-11-21T21:49:50.814 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-11-21T21:49:50.825 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-11-21T21:49:50.826 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-11-21T21:49:50.838 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-11-21T21:49:50.902 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-11-21T21:49:50.914 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-11-21T21:49:50.926 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-11-21T21:49:50.937 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-11-21T21:49:51.002 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-11-21T21:49:51.015 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-11-21T21:49:51.015 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-11-21T21:49:51.027 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-11-21T21:49:51.101 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-11-21T21:49:51.103 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} ok: [gibba036.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-11-21T21:49:51.114 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-11-21T21:49:51.188 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-11-21T21:49:51.202 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} ok: [gibba036.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-11-21T21:49:51.204 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-11-21T21:49:51.277 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-11-21T21:49:51.289 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-11-21T21:49:51.289 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-11-21T21:49:51.291 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-11-21T21:49:51.375 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-11-21T21:49:51.377 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-11-21T21:49:51.387 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-11-21T21:49:51.412 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-11-21T21:49:51.465 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-11-21T21:49:51.476 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-11-21T21:49:51.498 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-11-21T21:49:51.541 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-11-21T21:49:51.563 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-11-21T21:49:51.564 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-11-21T21:49:51.586 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-11-21T21:49:51.633 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-11-21T21:49:51.656 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-11-21T21:49:51.657 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-11-21T21:49:51.678 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-11-21T21:49:51.722 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-11-21T21:49:51.744 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-11-21T21:49:51.775 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-11-21T21:49:51.807 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-11-21T21:49:51.819 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-11-21T21:49:51.830 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-11-21T21:49:51.863 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-11-21T21:49:51.906 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-11-21T21:49:51.907 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-11-21T21:49:51.919 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-11-21T21:49:51.952 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-11-21T21:49:51.995 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-11-21T21:49:51.997 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-11-21T21:49:52.007 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-11-21T21:49:52.050 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-11-21T21:49:52.083 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-11-21T21:49:52.084 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-11-21T21:49:52.105 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-11-21T21:49:52.142 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-11-21T21:49:52.174 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-11-21T21:49:52.185 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-11-21T21:49:52.208 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-11-21T21:49:52.230 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-11-21T21:49:52.273 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-11-21T21:49:52.296 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-11-21T21:49:52.308 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-11-21T21:49:52.320 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-11-21T21:49:52.363 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-11-21T21:49:52.396 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-11-21T21:49:52.407 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-11-21T21:49:52.408 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-11-21T21:49:52.451 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-11-21T21:49:52.484 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-11-21T21:49:52.486 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-11-21T21:49:52.507 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-11-21T21:49:52.541 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-11-21T21:49:52.574 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-11-21T21:49:52.576 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-11-21T21:49:52.596 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-11-21T21:49:52.630 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-11-21T21:49:52.662 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-11-21T21:49:52.674 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-11-21T21:49:52.686 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-11-21T21:49:52.719 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-11-21T21:49:52.752 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-11-21T21:49:52.763 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-11-21T21:49:52.786 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-11-21T21:49:52.818 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-11-21T21:49:52.851 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-11-21T21:49:52.862 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-11-21T21:49:52.887 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-11-21T21:49:52.920 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-11-21T21:49:52.932 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-11-21T21:49:52.953 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-11-21T21:49:52.996 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-11-21T21:49:53.029 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-11-21T21:49:53.040 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-11-21T21:49:53.052 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-11-21T21:49:53.085 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": false, "item": "jdillaman", "name": "jdillaman", "state": "absent"} 2021-11-21T21:49:53.118 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-11-21T21:49:53.130 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-11-21T21:49:53.152 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-11-21T21:49:53.174 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=davidz) => {"ansible_loop_var": "item", "changed": false, "item": "davidz", "name": "davidz", "state": "absent"} 2021-11-21T21:49:53.218 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": false, "item": "jdillaman", "name": "jdillaman", "state": "absent"} 2021-11-21T21:49:53.230 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-11-21T21:49:53.252 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-11-21T21:49:53.274 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=wusui) => {"ansible_loop_var": "item", "changed": false, "item": "wusui", "name": "wusui", "state": "absent"} 2021-11-21T21:49:53.307 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=davidz) => {"ansible_loop_var": "item", "changed": false, "item": "davidz", "name": "davidz", "state": "absent"} 2021-11-21T21:49:53.318 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-11-21T21:49:53.342 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-11-21T21:49:53.364 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=nwatkins) => {"ansible_loop_var": "item", "changed": false, "item": "nwatkins", "name": "nwatkins", "state": "absent"} 2021-11-21T21:49:53.397 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=wusui) => {"ansible_loop_var": "item", "changed": false, "item": "wusui", "name": "wusui", "state": "absent"} 2021-11-21T21:49:53.409 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-11-21T21:49:53.432 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-11-21T21:49:53.463 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=sidharthanup) => {"ansible_loop_var": "item", "changed": false, "item": "sidharthanup", "name": "sidharthanup", "state": "absent"} 2021-11-21T21:49:53.488 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nwatkins) => {"ansible_loop_var": "item", "changed": false, "item": "nwatkins", "name": "nwatkins", "state": "absent"} 2021-11-21T21:49:53.510 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-11-21T21:49:53.565 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": false, "item": "jdillaman", "name": "jdillaman", "state": "absent"} 2021-11-21T21:49:53.577 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sidharthanup) => {"ansible_loop_var": "item", "changed": false, "item": "sidharthanup", "name": "sidharthanup", "state": "absent"} 2021-11-21T21:49:53.598 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=varsha) => {"ansible_loop_var": "item", "changed": false, "item": "varsha", "name": "varsha", "state": "absent"} 2021-11-21T21:49:53.610 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-11-21T21:49:53.666 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=varsha) => {"ansible_loop_var": "item", "changed": false, "item": "varsha", "name": "varsha", "state": "absent"} 2021-11-21T21:49:53.699 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-11-21T21:49:53.701 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=davidz) => {"ansible_loop_var": "item", "changed": false, "item": "davidz", "name": "davidz", "state": "absent"} 2021-11-21T21:49:53.722 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=hmunjulu) => {"ansible_loop_var": "item", "changed": false, "item": "hmunjulu", "name": "hmunjulu", "state": "absent"} 2021-11-21T21:49:53.754 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=hmunjulu) => {"ansible_loop_var": "item", "changed": false, "item": "hmunjulu", "name": "hmunjulu", "state": "absent"} 2021-11-21T21:49:53.787 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=wusui) => {"ansible_loop_var": "item", "changed": false, "item": "wusui", "name": "wusui", "state": "absent"} 2021-11-21T21:49:53.798 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-11-21T21:49:53.821 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-11-21T21:49:53.854 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-11-21T21:49:53.888 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=nwatkins) => {"ansible_loop_var": "item", "changed": false, "item": "nwatkins", "name": "nwatkins", "state": "absent"} 2021-11-21T21:49:53.889 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-11-21T21:49:53.911 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-11-21T21:49:53.955 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-11-21T21:49:53.988 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=sidharthanup) => {"ansible_loop_var": "item", "changed": false, "item": "sidharthanup", "name": "sidharthanup", "state": "absent"} 2021-11-21T21:49:54.000 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-11-21T21:49:54.011 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": false, "item": "jdillaman", "name": "jdillaman", "state": "absent"} 2021-11-21T21:49:54.055 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-11-21T21:49:54.077 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=varsha) => {"ansible_loop_var": "item", "changed": false, "item": "varsha", "name": "varsha", "state": "absent"} 2021-11-21T21:49:54.099 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=davidz) => {"ansible_loop_var": "item", "changed": false, "item": "davidz", "name": "davidz", "state": "absent"} 2021-11-21T21:49:54.111 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-11-21T21:49:54.143 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-11-21T21:49:54.165 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=hmunjulu) => {"ansible_loop_var": "item", "changed": false, "item": "hmunjulu", "name": "hmunjulu", "state": "absent"} 2021-11-21T21:49:54.198 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=wusui) => {"ansible_loop_var": "item", "changed": false, "item": "wusui", "name": "wusui", "state": "absent"} 2021-11-21T21:49:54.199 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-11-21T21:49:54.232 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-11-21T21:49:54.256 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-11-21T21:49:54.289 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-11-21T21:49:54.301 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=nwatkins) => {"ansible_loop_var": "item", "changed": false, "item": "nwatkins", "name": "nwatkins", "state": "absent"} 2021-11-21T21:49:54.322 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-11-21T21:49:54.344 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-11-21T21:49:54.398 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=sidharthanup) => {"ansible_loop_var": "item", "changed": false, "item": "sidharthanup", "name": "sidharthanup", "state": "absent"} 2021-11-21T21:49:54.399 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-11-21T21:49:54.420 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-11-21T21:49:54.442 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-11-21T21:49:54.485 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-11-21T21:49:54.496 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=varsha) => {"ansible_loop_var": "item", "changed": false, "item": "varsha", "name": "varsha", "state": "absent"} 2021-11-21T21:49:54.508 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-11-21T21:49:54.529 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-11-21T21:49:54.584 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=hmunjulu) => {"ansible_loop_var": "item", "changed": false, "item": "hmunjulu", "name": "hmunjulu", "state": "absent"} 2021-11-21T21:49:54.595 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-11-21T21:49:54.629 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} ok: [gibba016.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-11-21T21:49:54.672 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-11-21T21:49:54.684 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-11-21T21:49:54.732 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:54 +0000 (0:00:04.450) 0:00:48.667 ******* 2021-11-21T21:49:54.740 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-11-21T21:49:54.743 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-11-21T21:49:54.796 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:54 +0000 (0:00:00.064) 0:00:48.732 ******* 2021-11-21T21:49:54.810 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} ok: [gibba016.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-11-21T21:49:54.863 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-11-21T21:49:54.914 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-11-21T21:49:54.956 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-11-21T21:49:55.010 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-11-21T21:49:55.042 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-11-21T21:49:55.097 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-11-21T21:49:55.156 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.359) 0:00:49.092 ******* 2021-11-21T21:49:55.165 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-11-21T21:49:55.243 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-11-21T21:49:55.318 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-11-21T21:49:55.427 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "checksum": "b3b89f3dabc35f78585fcbe6afcac330df9308f6", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "69c3df528743245bb36ff3b8fc484a02", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1637531394.894124-3689-156912623320164/source", "state": "file", "uid": 0} Sunday 21 November 2021 21:49:55 +0000 (0:00:00.199) 0:00:49.291 ******* TASK [Remove revoked users] **************************************************** ok: [gibba036.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "checksum": "b3b89f3dabc35f78585fcbe6afcac330df9308f6", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "69c3df528743245bb36ff3b8fc484a02", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1637531394.8159869-3681-73883899135704/source", "state": "file", "uid": 0} Sunday 21 November 2021 21:49:55 +0000 (0:00:00.046) 0:00:49.338 ******* TASK [common : Log the OS name, version and release] *************************** ok: [gibba017.front.sepia.ceph.com] => { "msg": "Host gibba017.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-11-21T21:49:55.453 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.051) 0:00:49.389 ******* 2021-11-21T21:49:55.481 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => { "msg": "Host gibba019.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-11-21T21:49:55.483 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba036.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-11-21T21:49:55.510 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.056) 0:00:49.445 ******* 2021-11-21T21:49:55.527 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-11-21T21:49:55.529 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-11-21T21:49:55.553 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.043) 0:00:49.489 ******* 2021-11-21T21:49:55.592 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-11-21T21:49:55.593 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-11-21T21:49:55.594 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "checksum": "b3b89f3dabc35f78585fcbe6afcac330df9308f6", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "69c3df528743245bb36ff3b8fc484a02", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1637531395.2434425-3721-84275323099905/source", "state": "file", "uid": 0} 2021-11-21T21:49:55.594 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-21T21:49:55.595 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-11-21T21:49:55.619 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.065) 0:00:49.554 ******* 2021-11-21T21:49:55.641 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.022) 0:00:49.577 ******* 2021-11-21T21:49:55.675 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-11-21T21:49:55.677 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:55.702 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.060) 0:00:49.637 ******* 2021-11-21T21:49:55.725 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.023) 0:00:49.661 ******* 2021-11-21T21:49:55.733 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-11-21T21:49:55.734 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => { "msg": "Host gibba016.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:55.762 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.036) 0:00:49.697 ******* 2021-11-21T21:49:55.785 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.023) 0:00:49.720 ******* 2021-11-21T21:49:55.793 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-11-21T21:49:55.794 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:55.820 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.034) 0:00:49.755 ******* 2021-11-21T21:49:55.839 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-11-21T21:49:55.840 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-11-21T21:49:55.870 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.050) 0:00:49.806 ******* 2021-11-21T21:49:55.878 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-11-21T21:49:55.880 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:55.905 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.035) 0:00:49.841 ******* 2021-11-21T21:49:55.912 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-11-21T21:49:55.914 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:55.939 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.033) 0:00:49.874 ******* 2021-11-21T21:49:55.958 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-11-21T21:49:55.959 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:55.983 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:55 +0000 (0:00:00.044) 0:00:49.919 ******* 2021-11-21T21:49:55.990 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-11-21T21:49:55.992 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.023 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.039) 0:00:49.958 ******* 2021-11-21T21:49:56.031 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-11-21T21:49:56.032 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.060 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.036) 0:00:49.995 ******* 2021-11-21T21:49:56.067 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-11-21T21:49:56.069 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.098 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.038) 0:00:50.034 ******* 2021-11-21T21:49:56.106 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-11-21T21:49:56.108 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "checksum": "b3b89f3dabc35f78585fcbe6afcac330df9308f6", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "69c3df528743245bb36ff3b8fc484a02", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1637531395.8079007-3822-228294196970824/source", "state": "file", "uid": 0} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.137 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.038) 0:00:50.073 ******* 2021-11-21T21:49:56.163 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.026) 0:00:50.099 ******* 2021-11-21T21:49:56.171 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-11-21T21:49:56.173 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.200 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.036) 0:00:50.135 ******* 2021-11-21T21:49:56.224 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.024) 0:00:50.160 ******* 2021-11-21T21:49:56.232 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-11-21T21:49:56.234 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Log the OS name, version and release] *************************** ok: [gibba036.front.sepia.ceph.com] => { "msg": "Host gibba036.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-11-21T21:49:56.260 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.036) 0:00:50.196 ******* 2021-11-21T21:49:56.284 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.023) 0:00:50.219 ******* 2021-11-21T21:49:56.292 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-11-21T21:49:56.294 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.318 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.034) 0:00:50.254 ******* 2021-11-21T21:49:56.326 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-11-21T21:49:56.328 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.351 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.032) 0:00:50.287 ******* 2021-11-21T21:49:56.359 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.384 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.032) 0:00:50.320 ******* 2021-11-21T21:49:56.392 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-11-21T21:49:56.394 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-11-21T21:49:56.427 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.042) 0:00:50.363 ******* 2021-11-21T21:49:56.435 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-11-21T21:49:56.436 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.464 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.036) 0:00:50.399 ******* 2021-11-21T21:49:56.490 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.026) 0:00:50.426 ******* 2021-11-21T21:49:56.499 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.500 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-11-21T21:49:56.501 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.531 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.040) 0:00:50.467 ******* 2021-11-21T21:49:56.558 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.026) 0:00:50.494 ******* 2021-11-21T21:49:56.566 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-11-21T21:49:56.568 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.600 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.041) 0:00:50.535 ******* 2021-11-21T21:49:56.624 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.024) 0:00:50.559 ******* 2021-11-21T21:49:56.633 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.671 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.046) 0:00:50.606 ******* 2021-11-21T21:49:56.697 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.025) 0:00:50.632 ******* 2021-11-21T21:49:56.709 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-11-21T21:49:56.711 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.740 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.042) 0:00:50.675 ******* 2021-11-21T21:49:56.749 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-11-21T21:49:56.751 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba017.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:56.786 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.046) 0:00:50.722 ******* 2021-11-21T21:49:56.798 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:56.800 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.827 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.041) 0:00:50.763 ******* 2021-11-21T21:49:56.867 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.039) 0:00:50.802 ******* 2021-11-21T21:49:56.875 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-11-21T21:49:56.877 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.903 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.036) 0:00:50.839 ******* 2021-11-21T21:49:56.935 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.031) 0:00:50.870 ******* 2021-11-21T21:49:56.943 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:56.970 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:56 +0000 (0:00:00.035) 0:00:50.906 ******* 2021-11-21T21:49:56.979 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-11-21T21:49:56.981 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.008 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.037) 0:00:50.943 ******* 2021-11-21T21:49:57.015 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-11-21T21:49:57.016 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba016.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.045 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.036) 0:00:50.980 ******* 2021-11-21T21:49:57.068 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.023) 0:00:51.003 ******* 2021-11-21T21:49:57.083 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.117 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.048) 0:00:51.052 ******* 2021-11-21T21:49:57.129 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-11-21T21:49:57.132 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.157 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.040) 0:00:51.093 ******* 2021-11-21T21:49:57.180 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.023) 0:00:51.116 ******* 2021-11-21T21:49:57.189 INFO:teuthology.task.ansible.out: TASK [common : Set the releasever] ********************************************* 2021-11-21T21:49:57.191 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.217 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.036) 0:00:51.153 ******* 2021-11-21T21:49:57.347 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.372 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.154) 0:00:51.308 ******* 2021-11-21T21:49:57.391 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-11-21T21:49:57.393 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.426 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.053) 0:00:51.361 ******* 2021-11-21T21:49:57.427 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:57.434 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-11-21T21:49:57.436 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.462 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.036) 0:00:51.397 ******* 2021-11-21T21:49:57.470 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-11-21T21:49:57.471 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:57.498 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.035) 0:00:51.433 ******* 2021-11-21T21:49:57.505 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:57.507 INFO:teuthology.task.ansible.out: TASK [common : Set the releasever] ********************************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.533 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.035) 0:00:51.469 ******* 2021-11-21T21:49:57.556 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.022) 0:00:51.491 ******* 2021-11-21T21:49:57.563 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-11-21T21:49:57.565 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.590 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.034) 0:00:51.526 ******* 2021-11-21T21:49:57.599 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-11-21T21:49:57.601 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.633 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.042) 0:00:51.569 ******* 2021-11-21T21:49:57.641 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-11-21T21:49:57.642 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.668 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.035) 0:00:51.604 ******* 2021-11-21T21:49:57.691 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.022) 0:00:51.627 ******* 2021-11-21T21:49:57.699 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-11-21T21:49:57.701 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.726 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.034) 0:00:51.661 ******* 2021-11-21T21:49:57.733 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-11-21T21:49:57.735 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:57.761 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.034) 0:00:51.696 ******* 2021-11-21T21:49:57.899 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-11-21T21:49:57.901 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:57.924 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.163) 0:00:51.860 ******* 2021-11-21T21:49:57.943 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-11-21T21:49:57.945 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:57.973 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:57 +0000 (0:00:00.049) 0:00:51.909 ******* 2021-11-21T21:49:57.982 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-11-21T21:49:57.984 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:58.141 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.167) 0:00:52.077 ******* 2021-11-21T21:49:58.148 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-11-21T21:49:58.150 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.176 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.033) 0:00:52.110 ******* 2021-11-21T21:49:58.183 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-11-21T21:49:58.185 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:58.210 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.035) 0:00:52.146 ******* 2021-11-21T21:49:58.218 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-11-21T21:49:58.220 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.246 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.035) 0:00:52.182 ******* 2021-11-21T21:49:58.270 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.023) 0:00:52.205 ******* 2021-11-21T21:49:58.278 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-11-21T21:49:58.279 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.306 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.035) 0:00:52.241 ******* 2021-11-21T21:49:58.329 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.023) 0:00:52.265 ******* 2021-11-21T21:49:58.337 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.365 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.035) 0:00:52.300 ******* 2021-11-21T21:49:58.372 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-11-21T21:49:58.374 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.406 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.036) 0:00:52.337 ******* 2021-11-21T21:49:58.409 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-11-21T21:49:58.411 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.437 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.035) 0:00:52.373 ******* 2021-11-21T21:49:58.445 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-11-21T21:49:58.447 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.472 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.034) 0:00:52.408 ******* 2021-11-21T21:49:58.480 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.508 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.035) 0:00:52.443 ******* 2021-11-21T21:49:58.515 INFO:teuthology.task.ansible.out: TASK [common : Set the releasever] ********************************************* 2021-11-21T21:49:58.517 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.549 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.040) 0:00:52.484 ******* 2021-11-21T21:49:58.573 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.024) 0:00:52.509 ******* 2021-11-21T21:49:58.582 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-11-21T21:49:58.583 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.584 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.612 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.038) 0:00:52.548 ******* 2021-11-21T21:49:58.647 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.034) 0:00:52.582 ******* 2021-11-21T21:49:58.656 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.658 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [gibba036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:58.685 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.038) 0:00:52.621 ******* 2021-11-21T21:49:58.711 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.026) 0:00:52.647 ******* 2021-11-21T21:49:58.718 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-11-21T21:49:58.720 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:58.746 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.034) 0:00:52.682 ******* 2021-11-21T21:49:58.771 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.025) 0:00:52.707 ******* 2021-11-21T21:49:58.781 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:58.809 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.037) 0:00:52.744 ******* 2021-11-21T21:49:58.817 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-11-21T21:49:58.819 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.847 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.037) 0:00:52.782 ******* 2021-11-21T21:49:58.854 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-11-21T21:49:58.856 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.881 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.034) 0:00:52.817 ******* 2021-11-21T21:49:58.906 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.024) 0:00:52.841 ******* 2021-11-21T21:49:58.914 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:58.916 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-11-21T21:49:58.916 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:58.962 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.055) 0:00:52.897 ******* 2021-11-21T21:49:58.988 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:58 +0000 (0:00:00.026) 0:00:52.924 ******* 2021-11-21T21:49:58.996 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-11-21T21:49:58.998 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.024 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.036) 0:00:52.960 ******* 2021-11-21T21:49:59.058 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.033) 0:00:52.993 ******* 2021-11-21T21:49:59.068 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.069 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-11-21T21:49:59.069 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.101 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.043) 0:00:53.037 ******* 2021-11-21T21:49:59.126 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.024) 0:00:53.061 ******* 2021-11-21T21:49:59.134 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-11-21T21:49:59.136 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.162 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.036) 0:00:53.098 ******* 2021-11-21T21:49:59.185 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.022) 0:00:53.120 ******* 2021-11-21T21:49:59.192 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.194 INFO:teuthology.task.ansible.out: 2021-11-21T21:49:59.219 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.034) 0:00:53.154 ******* 2021-11-21T21:49:59.226 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-11-21T21:49:59.228 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.253 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.034) 0:00:53.189 ******* 2021-11-21T21:49:59.260 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-11-21T21:49:59.261 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.287 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.033) 0:00:53.222 ******* 2021-11-21T21:49:59.308 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.021) 0:00:53.244 ******* 2021-11-21T21:49:59.315 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.345 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.037) 0:00:53.281 ******* 2021-11-21T21:49:59.353 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-11-21T21:49:59.355 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.380 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.035) 0:00:53.316 ******* 2021-11-21T21:49:59.389 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-11-21T21:49:59.390 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.418 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.037) 0:00:53.353 ******* 2021-11-21T21:49:59.442 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.023) 0:00:53.377 ******* 2021-11-21T21:49:59.449 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.476 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.034) 0:00:53.411 ******* 2021-11-21T21:49:59.483 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-11-21T21:49:59.485 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.513 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.037) 0:00:53.448 ******* 2021-11-21T21:49:59.521 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-11-21T21:49:59.523 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.552 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.039) 0:00:53.488 ******* 2021-11-21T21:49:59.579 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.026) 0:00:53.514 ******* 2021-11-21T21:49:59.588 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.590 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [gibba036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:59.615 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.036) 0:00:53.551 ******* 2021-11-21T21:49:59.639 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.024) 0:00:53.575 ******* 2021-11-21T21:49:59.657 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-11-21T21:49:59.659 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:49:59.685 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.045) 0:00:53.621 ******* 2021-11-21T21:49:59.715 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.029) 0:00:53.650 ******* 2021-11-21T21:49:59.723 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Clean yum cache] ************************************************ skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.770 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.055) 0:00:53.706 ******* 2021-11-21T21:49:59.802 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.031) 0:00:53.737 ******* 2021-11-21T21:49:59.810 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-11-21T21:49:59.812 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.840 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.038) 0:00:53.776 ******* 2021-11-21T21:49:59.865 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.023) 0:00:53.800 ******* 2021-11-21T21:49:59.872 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.900 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.035) 0:00:53.835 ******* 2021-11-21T21:49:59.908 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.938 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.037) 0:00:53.873 ******* 2021-11-21T21:49:59.944 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-11-21T21:49:59.946 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:49:59.982 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:49:59 +0000 (0:00:00.044) 0:00:53.917 ******* 2021-11-21T21:50:00.004 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.022) 0:00:53.940 ******* 2021-11-21T21:50:00.012 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.038 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.034) 0:00:53.974 ******* 2021-11-21T21:50:00.046 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-11-21T21:50:00.048 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.073 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.035) 0:00:54.009 ******* 2021-11-21T21:50:00.081 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-11-21T21:50:00.082 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.083 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.110 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.035) 0:00:54.045 ******* 2021-11-21T21:50:00.140 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.030) 0:00:54.075 ******* 2021-11-21T21:50:00.161 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-11-21T21:50:00.163 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.195 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.055) 0:00:54.131 ******* 2021-11-21T21:50:00.221 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.025) 0:00:54.156 ******* 2021-11-21T21:50:00.230 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-11-21T21:50:00.232 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.234 INFO:teuthology.task.ansible.out: skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.235 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.237 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.266 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.045) 0:00:54.202 ******* 2021-11-21T21:50:00.293 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.026) 0:00:54.228 ******* 2021-11-21T21:50:00.301 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.333 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.040) 0:00:54.269 ******* 2021-11-21T21:50:00.342 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-11-21T21:50:00.343 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.370 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.036) 0:00:54.305 ******* 2021-11-21T21:50:00.377 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-11-21T21:50:00.379 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.407 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.037) 0:00:54.343 ******* 2021-11-21T21:50:00.433 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.025) 0:00:54.368 ******* 2021-11-21T21:50:00.440 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.442 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.444 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.476 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.043) 0:00:54.412 ******* 2021-11-21T21:50:00.507 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.030) 0:00:54.443 ******* 2021-11-21T21:50:00.517 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-11-21T21:50:00.519 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.556 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.049) 0:00:54.492 ******* 2021-11-21T21:50:00.590 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.033) 0:00:54.525 ******* 2021-11-21T21:50:00.598 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.632 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.041) 0:00:54.567 ******* 2021-11-21T21:50:00.640 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-11-21T21:50:00.642 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:00.668 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.035) 0:00:54.603 ******* 2021-11-21T21:50:00.743 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-11-21T21:50:00.744 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.771 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.103) 0:00:54.707 ******* 2021-11-21T21:50:00.780 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-11-21T21:50:00.781 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.010159", "end": "2021-11-21 21:50:01.124328", "rc": 0, "start": "2021-11-21 21:50:01.114169", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.011447", "end": "2021-11-21 21:50:00.515268", "rc": 0, "start": "2021-11-21 21:50:00.503821", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-11-21T21:50:00.809 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.037) 0:00:54.745 ******* 2021-11-21T21:50:00.832 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.022) 0:00:54.767 ******* 2021-11-21T21:50:00.840 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.007506", "end": "2021-11-21 21:50:00.540125", "rc": 0, "start": "2021-11-21 21:50:00.532619", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-11-21T21:50:00.841 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-11-21T21:50:00.843 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.869 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.036) 0:00:54.804 ******* 2021-11-21T21:50:00.893 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.023) 0:00:54.828 ******* 2021-11-21T21:50:00.900 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-11-21T21:50:00.902 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.932 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.039) 0:00:54.867 ******* 2021-11-21T21:50:00.952 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:00.982 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:00 +0000 (0:00:00.049) 0:00:54.917 ******* 2021-11-21T21:50:00.993 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.019 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.037) 0:00:54.955 ******* 2021-11-21T21:50:01.043 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.023) 0:00:54.978 ******* 2021-11-21T21:50:01.050 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-11-21T21:50:01.051 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.081 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.038) 0:00:55.016 ******* 2021-11-21T21:50:01.088 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.120 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.038) 0:00:55.055 ******* 2021-11-21T21:50:01.130 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-11-21T21:50:01.132 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.157 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.037) 0:00:55.093 ******* 2021-11-21T21:50:01.193 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.035) 0:00:55.129 ******* 2021-11-21T21:50:01.277 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-11-21T21:50:01.279 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.304 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.110) 0:00:55.240 ******* 2021-11-21T21:50:01.377 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-11-21T21:50:01.379 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.414 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.109) 0:00:55.349 ******* 2021-11-21T21:50:01.580 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-11-21T21:50:01.582 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.006856", "end": "2021-11-21 21:50:01.781250", "rc": 0, "start": "2021-11-21 21:50:01.774394", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-11-21T21:50:01.606 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.192) 0:00:55.541 ******* 2021-11-21T21:50:01.615 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-11-21T21:50:01.616 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "grub-common.service grub-initrd-fallback.service", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/sleep.target", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked", "Wants": "grub-initrd-fallback.service grub-common.service"}} 2021-11-21T21:50:01.617 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "grub-common.service grub-initrd-fallback.service", "CanIsolate": "no", "CanRe 2021-11-21T21:50:01.618 INFO:teuthology.task.ansible.out:load": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/sleep.target", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked", "Wants": "grub-initrd-fallback.service grub-common.service"}} 2021-11-21T21:50:01.630 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "grub-initrd-fallback.service grub-common.service", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/sleep.target", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked", "Wants": "grub-common.service grub-initrd-fallback.service"}} 2021-11-21T21:50:01.633 INFO:teuthology.task.ansible.out: 2021-11-21T21:50:01.690 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.728 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.120) 0:00:55.662 ******* 2021-11-21T21:50:01.751 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-11-21T21:50:01.753 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/suspend.target", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} ok: [gibba019.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "ye 2021-11-21T21:50:01.755 INFO:teuthology.task.ansible.out:s", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/suspend.target", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:01.787 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/suspend.target", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:01.810 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-11-21T21:50:01.812 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:01.834 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:01 +0000 (0:00:00.107) 0:00:55.769 ******* 2021-11-21T21:50:01.872 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-11-21T21:50:01.874 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hibernate.target", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:01.907 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hibernate.target", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:01.908 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hibernate.target", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:01.995 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hybrid-sleep.target", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:02.025 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.190) 0:00:55.960 ******* 2021-11-21T21:50:02.034 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hybrid-sleep.target", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:02.040 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hybrid-sleep.target", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:02.073 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.048) 0:00:56.008 ******* 2021-11-21T21:50:02.098 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.024) 0:00:56.033 ******* 2021-11-21T21:50:02.107 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "grub-initrd-fallback.service grub-common.service", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/sleep.target", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked", "Wants": "grub-common.service grub-initrd-fallback.service"}} 2021-11-21T21:50:02.108 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.140 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.042) 0:00:56.076 ******* 2021-11-21T21:50:02.157 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.184 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.043) 0:00:56.120 ******* 2021-11-21T21:50:02.191 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.193 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-11-21T21:50:02.193 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/suspend.target", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:02.220 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.036) 0:00:56.156 ******* 2021-11-21T21:50:02.228 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-11-21T21:50:02.230 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.256 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.035) 0:00:56.191 ******* 2021-11-21T21:50:02.263 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.291 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.034) 0:00:56.226 ******* 2021-11-21T21:50:02.298 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.300 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-11-21T21:50:02.301 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hibernate.target", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:02.336 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.045) 0:00:56.272 ******* 2021-11-21T21:50:02.344 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-11-21T21:50:02.346 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.371 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.035) 0:00:56.307 ******* 2021-11-21T21:50:02.380 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.408 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.036) 0:00:56.343 ******* 2021-11-21T21:50:02.415 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.442 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.033) 0:00:56.376 ******* 2021-11-21T21:50:02.449 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-11-21T21:50:02.450 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/etc/systemd/system/hybrid-sleep.target", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFilePreset": "enabled", "UnitFileState": "masked"}} 2021-11-21T21:50:02.485 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.044) 0:00:56.421 ******* 2021-11-21T21:50:02.494 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-11-21T21:50:02.495 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.522 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.036) 0:00:56.458 ******* 2021-11-21T21:50:02.546 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.023) 0:00:56.481 ******* 2021-11-21T21:50:02.555 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.557 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.585 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.039) 0:00:56.521 ******* 2021-11-21T21:50:02.613 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.028) 0:00:56.549 ******* 2021-11-21T21:50:02.622 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-11-21T21:50:02.624 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.653 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.039) 0:00:56.588 ******* 2021-11-21T21:50:02.675 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.022) 0:00:56.610 ******* 2021-11-21T21:50:02.683 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.709 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.034) 0:00:56.645 ******* 2021-11-21T21:50:02.716 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-11-21T21:50:02.718 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.745 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.035) 0:00:56.680 ******* 2021-11-21T21:50:02.835 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-11-21T21:50:02.837 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.860 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.115) 0:00:56.796 ******* 2021-11-21T21:50:02.930 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-11-21T21:50:02.932 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:02.955 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:02 +0000 (0:00:00.095) 0:00:56.891 ******* 2021-11-21T21:50:03.034 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-11-21T21:50:03.036 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:03.059 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.103) 0:00:56.994 ******* 2021-11-21T21:50:03.118 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-11-21T21:50:03.120 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.350128", "end": "2021-11-21 21:50:03.500915", "rc": 0, "start": "2021-11-21 21:50:03.150787", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:03.157 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.098) 0:00:57.093 ******* 2021-11-21T21:50:03.229 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-11-21T21:50:03.231 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:03.254 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.096) 0:00:57.189 ******* 2021-11-21T21:50:03.303 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-11-21T21:50:03.305 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.142029", "end": "2021-11-21 21:50:03.500841", "rc": 0, "start": "2021-11-21 21:50:03.358812", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:03.330 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.076) 0:00:57.266 ******* 2021-11-21T21:50:03.401 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-11-21T21:50:03.403 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:03.425 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.094) 0:00:57.361 ******* 2021-11-21T21:50:03.538 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-11-21T21:50:03.540 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.116336", "end": "2021-11-21 21:50:03.901653", "rc": 0, "start": "2021-11-21 21:50:03.785317", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:03.565 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.139) 0:00:57.501 ******* 2021-11-21T21:50:03.710 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.119215", "end": "2021-11-21 21:50:03.890803", "rc": 0, "start": "2021-11-21 21:50:03.771588", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:03.735 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.169) 0:00:57.670 ******* 2021-11-21T21:50:03.752 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-11-21T21:50:03.754 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.992006", "end": "2021-11-21 21:50:03.500903", "rc": 0, "start": "2021-11-21 21:50:02.508897", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.944136", "end": "2021-11-21 21:50:03.500916", "rc": 0, "start": "2021-11-21 21:50:02.556780", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:03.780 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.045) 0:00:57.716 ******* 2021-11-21T21:50:03.804 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.023) 0:00:57.739 ******* 2021-11-21T21:50:03.854 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-11-21T21:50:03.856 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:03.880 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.076) 0:00:57.815 ******* 2021-11-21T21:50:03.887 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:03.913 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:03 +0000 (0:00:00.033) 0:00:57.849 ******* 2021-11-21T21:50:04.204 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-11-21T21:50:04.206 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.152785", "end": "2021-11-21 21:50:03.941682", "rc": 0, "start": "2021-11-21 21:50:03.788897", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:04.230 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:04 +0000 (0:00:00.316) 0:00:58.166 ******* 2021-11-21T21:50:04.396 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.310552", "end": "2021-11-21 21:50:04.128051", "rc": 0, "start": "2021-11-21 21:50:03.817499", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:04.429 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:04 +0000 (0:00:00.198) 0:00:58.364 ******* 2021-11-21T21:50:06.874 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-11-21T21:50:06.876 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:06.899 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:06 +0000 (0:00:02.470) 0:01:00.835 ******* 2021-11-21T21:50:06.991 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:07.016 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.117) 0:01:00.952 ******* 2021-11-21T21:50:07.425 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:07.451 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.434) 0:01:01.386 ******* 2021-11-21T21:50:07.458 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-11-21T21:50:07.460 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:07.491 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.039) 0:01:01.426 ******* 2021-11-21T21:50:07.502 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:07.531 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.040) 0:01:01.467 ******* 2021-11-21T21:50:07.540 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-11-21T21:50:07.542 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (Debian)] ***************************** ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:07.570 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.038) 0:01:01.505 ******* 2021-11-21T21:50:07.598 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.027) 0:01:01.533 ******* 2021-11-21T21:50:07.637 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-11-21T21:50:07.639 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:07.667 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.069) 0:01:01.603 ******* 2021-11-21T21:50:07.897 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-11-21T21:50:07.899 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-11-21T21:50:07.924 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.256) 0:01:01.859 ******* 2021-11-21T21:50:07.974 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-11-21T21:50:07.999 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:07 +0000 (0:00:00.075) 0:01:01.935 ******* 2021-11-21T21:50:08.017 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-11-21T21:50:08.018 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:08.043 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.043) 0:01:01.978 ******* 2021-11-21T21:50:08.060 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:08.088 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.045) 0:01:02.024 ******* 2021-11-21T21:50:08.116 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-11-21T21:50:08.117 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:08.144 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.055) 0:01:02.080 ******* 2021-11-21T21:50:08.162 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:08.189 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.044) 0:01:02.124 ******* 2021-11-21T21:50:08.427 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:08.428 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-11-21T21:50:08.459 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.270) 0:01:02.394 ******* 2021-11-21T21:50:08.482 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.023) 0:01:02.418 ******* 2021-11-21T21:50:08.491 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-11-21T21:50:08.492 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:08.519 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.036) 0:01:02.454 ******* 2021-11-21T21:50:08.542 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.023) 0:01:02.478 ******* 2021-11-21T21:50:08.792 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-11-21T21:50:08.794 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-11-21T21:50:08.868 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-11-21T21:50:08.932 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-11-21T21:50:08.933 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:08.957 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:08 +0000 (0:00:00.414) 0:01:02.893 ******* 2021-11-21T21:50:09.040 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:09.067 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.109) 0:01:03.003 ******* 2021-11-21T21:50:09.095 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-11-21T21:50:09.097 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-11-21T21:50:09.212 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-11-21T21:50:09.319 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-11-21T21:50:09.321 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-11-21T21:50:09.348 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.280) 0:01:03.284 ******* 2021-11-21T21:50:09.397 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-11-21T21:50:09.399 INFO:teuthology.task.ansible.out: 2021-11-21T21:50:09.431 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-11-21T21:50:09.461 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.112) 0:01:03.396 ******* 2021-11-21T21:50:09.490 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-11-21T21:50:09.492 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:50:09.516 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.055) 0:01:03.452 ******* 2021-11-21T21:50:09.576 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-11-21T21:50:09.578 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-11-21T21:50:09.588 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-11-21T21:50:09.590 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:50:09.618 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.101) 0:01:03.554 ******* 2021-11-21T21:50:09.711 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-11-21T21:50:09.713 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-11-21T21:50:09.787 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-11-21T21:50:09.814 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.195) 0:01:03.750 ******* 2021-11-21T21:50:09.822 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-21T21:50:09.824 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-11-21T21:50:09.898 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-11-21T21:50:09.921 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-11-21T21:50:09.922 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-11-21T21:50:09.951 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.136) 0:01:03.886 ******* 2021-11-21T21:50:09.959 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-11-21T21:50:09.961 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:50:09.990 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:09 +0000 (0:00:00.038) 0:01:03.925 ******* 2021-11-21T21:50:10.052 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:50:10.054 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [gibba036.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-11-21T21:50:10.079 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.089) 0:01:04.015 ******* 2021-11-21T21:50:10.121 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-11-21T21:50:10.282 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3612, "state": "file", "uid": 0} 2021-11-21T21:50:10.286 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-11-21T21:50:10.357 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.277) 0:01:04.293 ******* 2021-11-21T21:50:10.365 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3612, "state": "file", "uid": 0} 2021-11-21T21:50:10.395 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.037) 0:01:04.331 ******* 2021-11-21T21:50:10.403 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-11-21T21:50:10.406 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-11-21T21:50:10.407 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-11-21T21:50:10.433 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.037) 0:01:04.368 ******* 2021-11-21T21:50:10.441 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-11-21T21:50:10.466 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.033) 0:01:04.401 ******* 2021-11-21T21:50:10.473 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-11-21T21:50:10.475 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:10.499 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.033) 0:01:04.435 ******* 2021-11-21T21:50:10.506 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:10.507 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-21T21:50:10.508 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-11-21T21:50:10.534 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.035) 0:01:04.470 ******* 2021-11-21T21:50:10.557 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-11-21T21:50:10.559 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.569 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.570 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.598 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.063) 0:01:04.534 ******* 2021-11-21T21:50:10.606 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-21T21:50:10.607 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-11-21T21:50:10.608 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [gibba017.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.608 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.609 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.637 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.038) 0:01:04.572 ******* 2021-11-21T21:50:10.644 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-11-21T21:50:10.646 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.673 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.036) 0:01:04.609 ******* 2021-11-21T21:50:10.681 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:10.720 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.046) 0:01:04.656 ******* 2021-11-21T21:50:10.740 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-21T21:50:10.741 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3612, "state": "file", "uid": 0} 2021-11-21T21:50:10.778 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.057) 0:01:04.713 ******* 2021-11-21T21:50:10.821 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-11-21T21:50:10.823 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [gibba016.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3612, "state": "file", "uid": 0} 2021-11-21T21:50:10.857 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.079) 0:01:04.792 ******* 2021-11-21T21:50:10.882 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.024) 0:01:04.817 ******* 2021-11-21T21:50:10.899 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-11-21T21:50:10.901 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-11-21T21:50:10.925 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.043) 0:01:04.860 ******* 2021-11-21T21:50:10.932 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-11-21T21:50:10.933 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:10.958 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.033) 0:01:04.894 ******* 2021-11-21T21:50:10.965 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:10.996 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:10 +0000 (0:00:00.037) 0:01:04.932 ******* 2021-11-21T21:50:11.007 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-11-21T21:50:11.009 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.020 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.031 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.061 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.065) 0:01:04.997 ******* 2021-11-21T21:50:11.072 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.073 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.074 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.075 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.101 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.039) 0:01:05.037 ******* 2021-11-21T21:50:11.110 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-11-21T21:50:11.112 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.137 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.036) 0:01:05.073 ******* 2021-11-21T21:50:11.145 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (non-RHEL/CentOS)] ******** 2021-11-21T21:50:11.147 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:11.147 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:11.174 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.035) 0:01:05.109 ******* 2021-11-21T21:50:11.199 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.025) 0:01:05.135 ******* 2021-11-21T21:50:11.207 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (non-RHEL/CentOS)] ******** 2021-11-21T21:50:11.209 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:11.237 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.037) 0:01:05.173 ******* 2021-11-21T21:50:11.456 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-11-21T21:50:11.457 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-11-21T21:50:11.481 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.243) 0:01:05.417 ******* 2021-11-21T21:50:11.498 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-11-21T21:50:11.531 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.050) 0:01:05.467 ******* 2021-11-21T21:50:11.580 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (non-RHEL/CentOS)] ******** 2021-11-21T21:50:11.584 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:11.609 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.078) 0:01:05.545 ******* 2021-11-21T21:50:11.639 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:11.664 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.054) 0:01:05.600 ******* 2021-11-21T21:50:11.758 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-11-21T21:50:11.760 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-11-21T21:50:11.796 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.131) 0:01:05.731 ******* 2021-11-21T21:50:11.860 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-11-21T21:50:11.862 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-11-21T21:50:11.887 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.091) 0:01:05.823 ******* 2021-11-21T21:50:11.895 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-11-21T21:50:11.897 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-11-21T21:50:11.927 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.039) 0:01:05.862 ******* 2021-11-21T21:50:11.935 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-11-21T21:50:11.936 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-11-21T21:50:11.938 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-11-21T21:50:11.970 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:11 +0000 (0:00:00.043) 0:01:05.905 ******* 2021-11-21T21:50:12.023 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-11-21T21:50:12.025 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-11-21T21:50:12.054 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.081) 0:01:05.987 ******* 2021-11-21T21:50:12.070 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-11-21T21:50:12.082 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-11-21T21:50:12.083 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.107 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.055) 0:01:06.043 ******* 2021-11-21T21:50:12.136 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-11-21T21:50:12.138 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-11-21T21:50:12.139 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [gibba036.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:12.165 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.057) 0:01:06.101 ******* 2021-11-21T21:50:12.187 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.021) 0:01:06.123 ******* 2021-11-21T21:50:12.195 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-11-21T21:50:12.197 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.223 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.035) 0:01:06.158 ******* 2021-11-21T21:50:12.253 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.030) 0:01:06.188 ******* 2021-11-21T21:50:12.261 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-11-21T21:50:12.263 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Check firewalld status] ***************************************** skipping: [gibba017.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:12.264 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-11-21T21:50:12.294 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.041) 0:01:06.230 ******* 2021-11-21T21:50:12.323 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.028) 0:01:06.258 ******* 2021-11-21T21:50:12.353 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.030) 0:01:06.288 ******* 2021-11-21T21:50:12.361 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-11-21T21:50:12.362 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.389 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.035) 0:01:06.324 ******* 2021-11-21T21:50:12.396 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-11-21T21:50:12.397 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-11-21T21:50:12.419 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-11-21T21:50:12.494 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-11-21T21:50:12.522 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.132) 0:01:06.457 ******* 2021-11-21T21:50:12.530 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-11-21T21:50:12.560 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.038) 0:01:06.495 ******* 2021-11-21T21:50:12.568 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-11-21T21:50:12.570 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.600 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.039) 0:01:06.535 ******* 2021-11-21T21:50:12.608 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.609 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-11-21T21:50:12.634 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.034) 0:01:06.570 ******* 2021-11-21T21:50:12.658 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.023) 0:01:06.593 ******* 2021-11-21T21:50:12.665 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-11-21T21:50:12.667 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Upload nagios nrpe config.] ************************************* ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-11-21T21:50:12.668 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-21T21:50:12.694 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.035) 0:01:06.629 ******* 2021-11-21T21:50:12.715 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.021) 0:01:06.650 ******* 2021-11-21T21:50:12.745 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.030) 0:01:06.681 ******* 2021-11-21T21:50:12.754 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-11-21T21:50:12.755 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.781 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.035) 0:01:06.717 ******* 2021-11-21T21:50:12.789 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.816 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.034) 0:01:06.752 ******* 2021-11-21T21:50:12.833 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-11-21T21:50:12.835 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:48 UTC", "ActiveEnterTimestampMonotonic": "15393051", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket -.mount systemd-tmpfiles-setup.service local-fs.target var-run.mount remote-fs.target basic.target nss-lookup.target system.slice network.target sysinit.target time-sync.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:48 UTC", "AssertTimestampMonotonic": "15376586", "Before": "plymouth-quit.service shutdown.target xdm.service getty@tty1.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:48 UTC", "Con 2021-11-21T21:50:12.836 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "15376585", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1019", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:48 UTC", "ExecMainStartTimestampMonotonic": "15392938", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-11-21 21:48:48 UTC] ; stop_time=[n/a] ; pid=1019 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-11-21 21:48:48 UTC] ; stop_time=[n/a] ; pid=1019 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", " 2021-11-21T21:50:12.838 INFO:teuthology.task.ansible.out:GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:48 UTC", "InactiveExitTimestampMonotonic": "15393051", "InvocationID": "cbc1dfff82454c5da87829ca79cbeee3", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127822", "LimitNPROCSoft": "127822", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127822", "LimitSIGPENDINGSoft": "127822", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1019", "MemoryAccounting": "yes", "MemoryCurrent": "1269760", "Memo 2021-11-21T21:50:12.840 INFO:teuthology.task.ansible.out:ryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:48 UTC", "StateChangeTimestampMonotonic": "15393051", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": " 2021-11-21T21:50:12.841 INFO:teuthology.task.ansible.out:no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38346", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:50:12.863 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.047) 0:01:06.799 ******* 2021-11-21T21:50:12.885 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:47 UTC", "ActiveEnterTimestampMonotonic": "14531827", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service system.slice basic.target -.mount systemd-journald.socket time-sync.target var-run.mount nss-lookup.target remote-fs.target sysinit.target local-fs.target network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:47 UTC", "AssertTimestampMonotonic": "14482934", "Before": "xdm.service plymouth-quit.service shutdown.target multi-user.target getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:47 UTC", "Con 2021-11-21T21:50:12.887 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "14482933", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "989", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:47 UTC", "ExecMainStartTimestampMonotonic": "14531724", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-11-21 21:48:47 UTC] ; stop_time=[n/a] ; pid=989 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-11-21 21:48:47 UTC] ; stop_time=[n/a] ; pid=989 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID 2021-11-21T21:50:12.888 INFO:teuthology.task.ansible.out:": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:47 UTC", "InactiveExitTimestampMonotonic": "14531827", "InvocationID": "33dee04fac2a4793810c3baa0de42432", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "989", "MemoryAccounting": "yes", "MemoryCurrent": "1269760", "MemoryDe 2021-11-21T21:50:12.890 INFO:teuthology.task.ansible.out:nyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:47 UTC", "StateChangeTimestampMonotonic": "14531827", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", 2021-11-21T21:50:12.892 INFO:teuthology.task.ansible.out: "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38346", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:50:12.911 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.048) 0:01:06.847 ******* 2021-11-21T21:50:12.920 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-11-21T21:50:12.922 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.946 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.035) 0:01:06.882 ******* 2021-11-21T21:50:12.953 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:12.979 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:12 +0000 (0:00:00.032) 0:01:06.914 ******* 2021-11-21T21:50:12.985 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-11-21T21:50:12.987 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.012 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.032) 0:01:06.947 ******* 2021-11-21T21:50:13.019 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.049 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.037) 0:01:06.984 ******* 2021-11-21T21:50:13.056 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-11-21T21:50:13.058 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-11-21T21:50:13.058 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.059 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-11-21T21:50:13.059 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-11-21T21:50:13.083 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.034) 0:01:07.019 ******* 2021-11-21T21:50:13.106 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.022) 0:01:07.041 ******* 2021-11-21T21:50:13.136 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.030) 0:01:07.072 ******* 2021-11-21T21:50:13.147 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-11-21T21:50:13.148 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.176 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.039) 0:01:07.112 ******* 2021-11-21T21:50:13.183 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-11-21T21:50:13.185 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.209 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.033) 0:01:07.145 ******* 2021-11-21T21:50:13.216 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.242 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.032) 0:01:07.177 ******* 2021-11-21T21:50:13.248 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-11-21T21:50:13.249 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.275 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.033) 0:01:07.211 ******* 2021-11-21T21:50:13.282 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-11-21T21:50:13.284 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:49 UTC", "ActiveEnterTimestampMonotonic": "15254491", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target remote-fs.target time-sync.target basic.target local-fs.target -.mount system.slice nss-lookup.target network.target systemd-tmpfiles-setup.service var-run.mount systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:49 UTC", "AssertTimestampMonotonic": "15214829", "Before": "xdm.service multi-user.target plymouth-quit.service shutdown.target getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:49 UTC", "Con 2021-11-21T21:50:13.286 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "15214828", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1018", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:49 UTC", "ExecMainStartTimestampMonotonic": "15254390", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-11-21 21:48:49 UTC] ; stop_time=[n/a] ; pid=1018 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-11-21 21:48:49 UTC] ; stop_time=[n/a] ; pid=1018 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", " 2021-11-21T21:50:13.287 INFO:teuthology.task.ansible.out:GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:49 UTC", "InactiveExitTimestampMonotonic": "15254491", "InvocationID": "9e4d5a09306042c29b5a1b749b8900b6", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1018", "MemoryAccounting": "yes", "MemoryCurrent": "1335296", "Memo 2021-11-21T21:50:13.289 INFO:teuthology.task.ansible.out:ryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:49 UTC", "StateChangeTimestampMonotonic": "15254491", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": " 2021-11-21T21:50:13.291 INFO:teuthology.task.ansible.out:no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38346", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:48 UTC", "ActiveEnterTimestampMonotonic": "15288204", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount nss-lookup.target network.target systemd-journald.socket basic.target remote-fs.target local-fs.target sysinit.target time-sync.target -.mount systemd-tmpfiles-setup.service system.slice", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:48 UTC", "AssertTimestampMonotonic": "15262891", "Before": "getty@tty1.service multi-user.target plymouth-quit.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuo 2021-11-21T21:50:13.292 INFO:teuthology.task.ansible.out:taPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:48 UTC", "ConditionTimestampMonotonic": "15262890", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1017", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:48 UTC", "ExecMainStartTimestampMonotonic": "15288092", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bi 2021-11-21T21:50:13.294 INFO:teuthology.task.ansible.out:n/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-11-21 21:48:48 UTC] ; stop_time=[n/a] ; pid=1017 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-11-21 21:48:48 UTC] ; stop_time=[n/a] ; pid=1017 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:48 UTC", "InactiveExitTimestampMonotonic": "15288204", "InvocationID": "67c403f4012b460abf5880d7d90aaa91", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "i 2021-11-21T21:50:13.296 INFO:teuthology.task.ansible.out:nfinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1017", "MemoryAccounting": "yes", "MemoryCurrent": "1351680", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveI 2021-11-21T21:50:13.297 INFO:teuthology.task.ansible.out:PC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:48 UTC", "StateChangeTimestampMonotonic": "15288204", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "n 2021-11-21T21:50:13.298 INFO:teuthology.task.ansible.out:o", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38346", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:50:13.299 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.323 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.047) 0:01:07.259 ******* 2021-11-21T21:50:13.345 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.021) 0:01:07.280 ******* 2021-11-21T21:50:13.367 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.021) 0:01:07.302 ******* 2021-11-21T21:50:13.390 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.022) 0:01:07.324 ******* 2021-11-21T21:50:13.395 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-11-21T21:50:13.397 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get SELinux status] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.424 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.035) 0:01:07.360 ******* 2021-11-21T21:50:13.447 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.023) 0:01:07.383 ******* 2021-11-21T21:50:13.474 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.026) 0:01:07.410 ******* 2021-11-21T21:50:13.481 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-11-21T21:50:13.483 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.507 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.033) 0:01:07.443 ******* 2021-11-21T21:50:13.529 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.022) 0:01:07.465 ******* 2021-11-21T21:50:13.552 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.022) 0:01:07.487 ******* 2021-11-21T21:50:13.558 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-11-21T21:50:13.560 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.586 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.033) 0:01:07.521 ******* 2021-11-21T21:50:13.610 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.022) 0:01:07.543 ******* 2021-11-21T21:50:13.631 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.023) 0:01:07.567 ******* 2021-11-21T21:50:13.639 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.640 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.665 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.033) 0:01:07.600 ******* 2021-11-21T21:50:13.687 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.022) 0:01:07.623 ******* 2021-11-21T21:50:13.694 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-11-21T21:50:13.696 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.721 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.033) 0:01:07.656 ******* 2021-11-21T21:50:13.742 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.021) 0:01:07.678 ******* 2021-11-21T21:50:13.765 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.021) 0:01:07.699 ******* 2021-11-21T21:50:13.770 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-11-21T21:50:13.772 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove temporary files] ********************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.801 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.036) 0:01:07.736 ******* 2021-11-21T21:50:13.939 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.138) 0:01:07.875 ******* 2021-11-21T21:50:13.962 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.022) 0:01:07.897 ******* 2021-11-21T21:50:13.970 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-11-21T21:50:13.972 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:13.997 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:13 +0000 (0:00:00.034) 0:01:07.932 ******* 2021-11-21T21:50:14.019 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.022) 0:01:07.955 ******* 2021-11-21T21:50:14.025 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-11-21T21:50:14.027 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} TASK [common : nrpe - Remove temporary files] ********************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:14.052 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.033) 0:01:07.988 ******* 2021-11-21T21:50:14.075 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.022) 0:01:08.011 ******* 2021-11-21T21:50:14.097 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.022) 0:01:08.033 ******* 2021-11-21T21:50:14.106 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:14.107 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:14.133 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.035) 0:01:08.069 ******* 2021-11-21T21:50:14.157 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.024) 0:01:08.093 ******* 2021-11-21T21:50:14.165 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-11-21T21:50:14.166 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common", "libgnutls30"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-11-21T21:50:14.169 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-11-21T21:50:14.170 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} TASK [common : nrpe - Remove temporary files] ********************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:14.194 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.036) 0:01:08.129 ******* 2021-11-21T21:50:14.217 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.023) 0:01:08.153 ******* 2021-11-21T21:50:14.241 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.023) 0:01:08.176 ******* 2021-11-21T21:50:14.247 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-11-21T21:50:14.249 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include package type specific vars.] ************************** ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-11-21T21:50:14.283 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.042) 0:01:08.218 ******* 2021-11-21T21:50:14.306 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.023) 0:01:08.241 ******* 2021-11-21T21:50:14.317 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-11-21T21:50:14.319 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-venv", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-11-21T21:50:14.319 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-11-21T21:50:14.320 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common", "libgnutls30"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_ 2021-11-21T21:50:14.321 INFO:teuthology.task.ansible.out:files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-11-21T21:50:14.345 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.039) 0:01:08.281 ******* 2021-11-21T21:50:14.369 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.023) 0:01:08.304 ******* 2021-11-21T21:50:14.378 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common", "libgnutls30"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-11-21T21:50:14.380 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-11-21T21:50:14.382 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Including version specific variables.] ************************ ok: [gibba017.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-21T21:50:14.407 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.038) 0:01:08.343 ******* 2021-11-21T21:50:14.430 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.022) 0:01:08.365 ******* 2021-11-21T21:50:14.454 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.023) 0:01:08.389 ******* 2021-11-21T21:50:14.461 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-11-21T21:50:14.463 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-venv", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-11-21T21:50:14.465 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-venv", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-11-21T21:50:14.505 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.051) 0:01:08.440 ******* 2021-11-21T21:50:14.533 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.027) 0:01:08.468 ******* 2021-11-21T21:50:14.539 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-11-21T21:50:14.541 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:50:14.543 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [gibba016.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-21T21:50:14.569 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.036) 0:01:08.505 ******* 2021-11-21T21:50:14.593 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.023) 0:01:08.529 ******* 2021-11-21T21:50:14.616 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.022) 0:01:08.552 ******* 2021-11-21T21:50:14.624 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-21T21:50:14.659 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.043) 0:01:08.595 ******* 2021-11-21T21:50:14.667 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-11-21T21:50:14.668 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:14.670 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:50:14.698 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.036) 0:01:08.631 ******* 2021-11-21T21:50:14.722 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.025) 0:01:08.657 ******* 2021-11-21T21:50:14.740 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-11-21T21:50:14.742 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-11-21T21:50:14.766 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.044) 0:01:08.701 ******* 2021-11-21T21:50:14.773 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-11-21T21:50:14.774 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-11-21T21:50:14.774 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:50:14.800 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.034) 0:01:08.735 ******* 2021-11-21T21:50:14.824 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.023) 0:01:08.759 ******* 2021-11-21T21:50:14.831 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-11-21T21:50:14.833 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-11-21T21:50:14.833 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-11-21T21:50:14.860 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.035) 0:01:08.795 ******* 2021-11-21T21:50:14.882 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.021) 0:01:08.817 ******* 2021-11-21T21:50:14.909 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.027) 0:01:08.845 ******* 2021-11-21T21:50:14.918 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-11-21T21:50:14.919 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:14.920 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:14.959 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.049) 0:01:08.895 ******* 2021-11-21T21:50:14.984 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:14 +0000 (0:00:00.024) 0:01:08.919 ******* 2021-11-21T21:50:14.992 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-11-21T21:50:14.993 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} TASK [testnode : Including distro specific variables.] ************************* ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common", "libgnutls30"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-11-21T21:50:14.995 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-11-21T21:50:14.997 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:15.022 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.037) 0:01:08.957 ******* 2021-11-21T21:50:15.063 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.041) 0:01:08.999 ******* 2021-11-21T21:50:15.088 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.024) 0:01:09.024 ******* 2021-11-21T21:50:15.108 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-11-21T21:50:15.110 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-venv", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [gibba017.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-11-21T21:50:15.111 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-11-21T21:50:15.134 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.046) 0:01:09.070 ******* 2021-11-21T21:50:15.159 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.024) 0:01:09.095 ******* 2021-11-21T21:50:15.167 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-11-21T21:50:15.169 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [gibba016.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-11-21T21:50:15.197 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.037) 0:01:09.132 ******* 2021-11-21T21:50:15.223 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.025) 0:01:09.158 ******* 2021-11-21T21:50:15.231 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-11-21T21:50:15.233 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-21T21:50:15.261 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.038) 0:01:09.197 ******* 2021-11-21T21:50:15.269 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-11-21T21:50:15.270 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:15.297 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.035) 0:01:09.233 ******* 2021-11-21T21:50:15.368 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-11-21T21:50:15.369 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-11-21T21:50:15.400 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.102) 0:01:09.335 ******* 2021-11-21T21:50:15.427 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-11-21T21:50:15.429 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-11-21T21:50:15.455 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.055) 0:01:09.391 ******* 2021-11-21T21:50:15.487 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-11-21T21:50:15.489 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-11-21T21:50:15.516 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.060) 0:01:09.451 ******* 2021-11-21T21:50:15.533 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-11-21T21:50:15.535 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-11-21T21:50:15.562 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.046) 0:01:09.498 ******* 2021-11-21T21:50:15.595 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.032) 0:01:09.530 ******* 2021-11-21T21:50:15.603 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-11-21T21:50:15.606 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba017"}, "changed": false} ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba016"}, "changed": false} 2021-11-21T21:50:15.631 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.036) 0:01:09.566 ******* 2021-11-21T21:50:15.655 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.024) 0:01:09.591 ******* 2021-11-21T21:50:15.663 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-11-21T21:50:15.665 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:15.691 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.035) 0:01:09.626 ******* 2021-11-21T21:50:15.731 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-11-21T21:50:15.732 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:15.757 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.064) 0:01:09.691 ******* 2021-11-21T21:50:15.862 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-11-21T21:50:15.864 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-11-21T21:50:15.888 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.132) 0:01:09.824 ******* 2021-11-21T21:50:15.898 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-11-21T21:50:15.900 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-11-21T21:50:15.927 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.038) 0:01:09.863 ******* 2021-11-21T21:50:15.937 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-11-21T21:50:15.939 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba036"}, "changed": false} 2021-11-21T21:50:15.968 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:15 +0000 (0:00:00.040) 0:01:09.903 ******* 2021-11-21T21:50:16.225 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba016.front.sepia.ceph.com", "ansible_hostname": "gibba016", "ansible_nodename": "gibba016"}, "changed": true, "name": "gibba016"} 2021-11-21T21:50:16.249 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.280) 0:01:10.184 ******* 2021-11-21T21:50:16.257 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba017.front.sepia.ceph.com", "ansible_hostname": "gibba017", "ansible_nodename": "gibba017"}, "changed": true, "name": "gibba017"} 2021-11-21T21:50:16.258 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba036.front.sepia.ceph.com", "ansible_hostname": "gibba036", "ansible_nodename": "gibba036"}, "changed": true, "name": "gibba036"} 2021-11-21T21:50:16.285 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.036) 0:01:10.221 ******* 2021-11-21T21:50:16.311 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.024) 0:01:10.245 ******* 2021-11-21T21:50:16.317 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-11-21T21:50:16.319 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-11-21T21:50:16.354 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.045) 0:01:10.290 ******* 2021-11-21T21:50:16.395 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-11-21T21:50:16.397 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba019"}, "changed": false} 2021-11-21T21:50:16.422 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.066) 0:01:10.357 ******* 2021-11-21T21:50:16.513 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-11-21T21:50:16.515 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-11-21T21:50:16.540 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.118) 0:01:10.475 ******* 2021-11-21T21:50:16.560 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-11-21T21:50:16.590 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.049) 0:01:10.525 ******* 2021-11-21T21:50:16.600 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-11-21T21:50:16.630 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.040) 0:01:10.565 ******* 2021-11-21T21:50:16.702 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-11-21T21:50:16.704 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba019.front.sepia.ceph.com", "ansible_hostname": "gibba019", "ansible_nodename": "gibba019"}, "changed": true, "name": "gibba019"} 2021-11-21T21:50:16.729 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.099) 0:01:10.665 ******* 2021-11-21T21:50:16.812 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-11-21T21:50:16.813 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-11-21T21:50:16.840 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.110) 0:01:10.775 ******* 2021-11-21T21:50:16.856 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-11-21T21:50:16.887 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.047) 0:01:10.823 ******* 2021-11-21T21:50:16.905 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-11-21T21:50:16.932 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:16 +0000 (0:00:00.045) 0:01:10.868 ******* 2021-11-21T21:50:17.005 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-11-21T21:50:17.007 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-11-21T21:50:17.039 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.106) 0:01:10.974 ******* 2021-11-21T21:50:17.324 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-11-21T21:50:17.350 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.310) 0:01:11.285 ******* 2021-11-21T21:50:17.357 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-11-21T21:50:17.359 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} changed: [gibba017.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-11-21T21:50:17.393 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.043) 0:01:11.329 ******* 2021-11-21T21:50:17.416 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.023) 0:01:11.352 ******* 2021-11-21T21:50:17.434 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-11-21T21:50:17.436 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.461 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.044) 0:01:11.396 ******* 2021-11-21T21:50:17.485 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.024) 0:01:11.421 ******* 2021-11-21T21:50:17.492 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.518 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.032) 0:01:11.453 ******* 2021-11-21T21:50:17.525 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:50:17.527 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create remote.conf] ******************************************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.552 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.034) 0:01:11.488 ******* 2021-11-21T21:50:17.575 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.022) 0:01:11.510 ******* 2021-11-21T21:50:17.581 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:50:17.581 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.582 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.608 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.032) 0:01:11.543 ******* 2021-11-21T21:50:17.630 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.021) 0:01:11.565 ******* 2021-11-21T21:50:17.636 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:50:17.638 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.662 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.032) 0:01:11.598 ******* 2021-11-21T21:50:17.669 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:50:17.671 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.695 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.032) 0:01:11.631 ******* 2021-11-21T21:50:17.717 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.021) 0:01:11.652 ******* 2021-11-21T21:50:17.724 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.725 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-21T21:50:17.726 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.754 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.037) 0:01:11.689 ******* 2021-11-21T21:50:17.778 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.023) 0:01:11.713 ******* 2021-11-21T21:50:17.785 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:50:17.787 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.787 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-11-21T21:50:17.788 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-11-21T21:50:17.789 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.812 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.034) 0:01:11.748 ******* 2021-11-21T21:50:17.834 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.021) 0:01:11.770 ******* 2021-11-21T21:50:17.855 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.021) 0:01:11.791 ******* 2021-11-21T21:50:17.863 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:50:17.867 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.894 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.038) 0:01:11.830 ******* 2021-11-21T21:50:17.925 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.030) 0:01:11.861 ******* 2021-11-21T21:50:17.947 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.021) 0:01:11.882 ******* 2021-11-21T21:50:17.953 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-11-21T21:50:17.954 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Clean up Stream distro-provided repos] ************************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:17.980 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.033) 0:01:11.915 ******* 2021-11-21T21:50:18.002 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:17 +0000 (0:00:00.022) 0:01:11.938 ******* 2021-11-21T21:50:18.027 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.025) 0:01:11.963 ******* 2021-11-21T21:50:18.035 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.037 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.060 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.032) 0:01:11.996 ******* 2021-11-21T21:50:18.091 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.030) 0:01:12.026 ******* 2021-11-21T21:50:18.117 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.026) 0:01:12.052 ******* 2021-11-21T21:50:18.124 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:50:18.126 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include CentOS Stream specific variables] ********************* skipping: [gibba016.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.126 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.152 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.035) 0:01:12.088 ******* 2021-11-21T21:50:18.175 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.023) 0:01:12.111 ******* 2021-11-21T21:50:18.199 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.023) 0:01:12.135 ******* 2021-11-21T21:50:18.207 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-11-21T21:50:18.209 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure local mirrorlists] ********************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.237 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.037) 0:01:12.172 ******* 2021-11-21T21:50:18.259 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.022) 0:01:12.195 ******* 2021-11-21T21:50:18.283 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.024) 0:01:12.219 ******* 2021-11-21T21:50:18.291 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-11-21T21:50:18.293 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-11-21T21:50:18.318 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.034) 0:01:12.253 ******* 2021-11-21T21:50:18.343 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.024) 0:01:12.278 ******* 2021-11-21T21:50:18.350 INFO:teuthology.task.ansible.out: TASK [testnode : Include CentOS Stream specific variables] ********************* 2021-11-21T21:50:18.352 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml", "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.352 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-11-21T21:50:18.377 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.034) 0:01:12.312 ******* 2021-11-21T21:50:18.399 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.021) 0:01:12.334 ******* 2021-11-21T21:50:18.427 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.027) 0:01:12.362 ******* 2021-11-21T21:50:18.433 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:50:18.435 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable copr repos] ******************************************** TASK [testnode : Configure local mirrorlists] ********************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.460 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.033) 0:01:12.396 ******* 2021-11-21T21:50:18.484 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.023) 0:01:12.419 ******* 2021-11-21T21:50:18.509 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.025) 0:01:12.445 ******* 2021-11-21T21:50:18.516 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG] ************************************************** 2021-11-21T21:50:18.518 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-11-21T21:50:18.543 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.033) 0:01:12.478 ******* 2021-11-21T21:50:18.567 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.024) 0:01:12.502 ******* 2021-11-21T21:50:18.593 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.026) 0:01:12.529 ******* 2021-11-21T21:50:18.600 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-21T21:50:18.601 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG keys] ********************************************* skipping: [gibba016.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.603 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.604 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=https://dist.apache.org/repos/dist/dev/arrow/KEYS) => {"ansible_loop_var": "item", "changed": false, "item": "https://dist.apache.org/repos/dist/dev/arrow/KEYS", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.605 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-11-21T21:50:18.630 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.036) 0:01:12.565 ******* 2021-11-21T21:50:18.654 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.023) 0:01:12.589 ******* 2021-11-21T21:50:18.679 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.025) 0:01:12.615 ******* 2021-11-21T21:50:18.817 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Clean yum cache] ********************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:50:18 +0000 (0:00:00.040) 0:01:12.655 ******* Sunday 21 November 2021 21:50:18 +0000 (0:00:00.031) 0:01:12.687 ******* TASK [testnode : Enable copr repos] ******************************************** TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:50:18 +0000 (0:00:00.051) 0:01:12.738 ******* 2021-11-21T21:50:18.834 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.030) 0:01:12.768 ******* 2021-11-21T21:50:18.862 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.028) 0:01:12.797 ******* 2021-11-21T21:50:18.872 INFO:teuthology.task.ansible.out: TASK [testnode : Include CentOS Stream specific variables] ********************* 2021-11-21T21:50:18.875 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml", "skip_reason": "Conditional result was False"} TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG] ************************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.902 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.040) 0:01:12.838 ******* 2021-11-21T21:50:18.929 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.026) 0:01:12.865 ******* 2021-11-21T21:50:18.956 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.027) 0:01:12.892 ******* 2021-11-21T21:50:18.968 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:18.991 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:18 +0000 (0:00:00.035) 0:01:12.927 ******* 2021-11-21T21:50:19.014 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:12.949 ******* 2021-11-21T21:50:19.021 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-11-21T21:50:19.023 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item=https://dist.apache.org/repos/dist/dev/arrow/KEYS) => {"ansible_loop_var": "item", "changed": false, "item": "https://dist.apache.org/repos/dist/dev/arrow/KEYS", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.025 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-11-21T21:50:19.025 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.050 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.035) 0:01:12.985 ******* 2021-11-21T21:50:19.072 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.007 ******* 2021-11-21T21:50:19.094 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.021) 0:01:13.029 ******* 2021-11-21T21:50:19.100 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ 2021-11-21T21:50:19.102 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Clean yum cache] ********************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.133 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.039) 0:01:13.069 ******* 2021-11-21T21:50:19.160 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.026) 0:01:13.096 ******* 2021-11-21T21:50:19.183 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.023) 0:01:13.119 ******* 2021-11-21T21:50:19.191 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-11-21T21:50:19.193 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.218 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.034) 0:01:13.153 ******* 2021-11-21T21:50:19.240 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.176 ******* 2021-11-21T21:50:19.262 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.198 ******* 2021-11-21T21:50:19.268 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:50:19.270 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.295 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.033) 0:01:13.231 ******* 2021-11-21T21:50:19.318 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.254 ******* 2021-11-21T21:50:19.343 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.024) 0:01:13.279 ******* 2021-11-21T21:50:19.351 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-11-21T21:50:19.353 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.378 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.034) 0:01:13.313 ******* 2021-11-21T21:50:19.401 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.336 ******* 2021-11-21T21:50:19.407 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-11-21T21:50:19.407 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.409 INFO:teuthology.task.ansible.out: TASK [testnode : Include CentOS Stream specific variables] ********************* skipping: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.410 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.434 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.032) 0:01:13.369 ******* 2021-11-21T21:50:19.455 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.021) 0:01:13.390 ******* 2021-11-21T21:50:19.484 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.029) 0:01:13.420 ******* 2021-11-21T21:50:19.492 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-11-21T21:50:19.494 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.520 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.036) 0:01:13.456 ******* 2021-11-21T21:50:19.545 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.024) 0:01:13.480 ******* 2021-11-21T21:50:19.569 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.024) 0:01:13.505 ******* 2021-11-21T21:50:19.576 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-11-21T21:50:19.578 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.604 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.034) 0:01:13.540 ******* 2021-11-21T21:50:19.627 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.562 ******* 2021-11-21T21:50:19.650 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.023) 0:01:13.586 ******* 2021-11-21T21:50:19.656 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-11-21T21:50:19.658 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable firewalld] *********************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.683 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.033) 0:01:13.619 ******* 2021-11-21T21:50:19.706 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.642 ******* 2021-11-21T21:50:19.729 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.664 ******* 2021-11-21T21:50:19.734 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-11-21T21:50:19.735 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-11-21T21:50:19.736 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.762 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.033) 0:01:13.698 ******* 2021-11-21T21:50:19.786 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.023) 0:01:13.721 ******* 2021-11-21T21:50:19.793 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-11-21T21:50:19.794 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG keys] ********************************************* skipping: [gibba036.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.795 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.796 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=https://dist.apache.org/repos/dist/dev/arrow/KEYS) => {"ansible_loop_var": "item", "changed": false, "item": "https://dist.apache.org/repos/dist/dev/arrow/KEYS", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.797 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.827 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.041) 0:01:13.762 ******* 2021-11-21T21:50:19.850 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.023) 0:01:13.786 ******* 2021-11-21T21:50:19.875 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.024) 0:01:13.810 ******* 2021-11-21T21:50:19.882 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-11-21T21:50:19.884 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.910 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.035) 0:01:13.846 ******* 2021-11-21T21:50:19.933 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.869 ******* 2021-11-21T21:50:19.955 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.022) 0:01:13.891 ******* 2021-11-21T21:50:19.962 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-11-21T21:50:19.964 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install epel packages] **************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:19.989 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:19 +0000 (0:00:00.033) 0:01:13.924 ******* 2021-11-21T21:50:20.012 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.023) 0:01:13.947 ******* 2021-11-21T21:50:20.034 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.022) 0:01:13.970 ******* 2021-11-21T21:50:20.041 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-11-21T21:50:20.042 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.068 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.033) 0:01:14.003 ******* 2021-11-21T21:50:20.090 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.021) 0:01:14.025 ******* 2021-11-21T21:50:20.110 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.020) 0:01:14.046 ******* 2021-11-21T21:50:20.119 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:50:20.120 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.122 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.123 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.152 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.041) 0:01:14.087 ******* 2021-11-21T21:50:20.175 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.023) 0:01:14.111 ******* 2021-11-21T21:50:20.200 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.024) 0:01:14.135 ******* 2021-11-21T21:50:20.207 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-11-21T21:50:20.209 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable firewalld] *********************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.236 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.036) 0:01:14.171 ******* 2021-11-21T21:50:20.260 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.023) 0:01:14.195 ******* 2021-11-21T21:50:20.283 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.023) 0:01:14.219 ******* 2021-11-21T21:50:20.290 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-11-21T21:50:20.292 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable iptables] ************************************ skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.318 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.034) 0:01:14.254 ******* 2021-11-21T21:50:20.346 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.027) 0:01:14.281 ******* 2021-11-21T21:50:20.353 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-11-21T21:50:20.355 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.356 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=https://dist.apache.org/repos/dist/dev/arrow/KEYS) => {"ansible_loop_var": "item", "changed": false, "item": "https://dist.apache.org/repos/dist/dev/arrow/KEYS", "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.357 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001317", "end": "2021-11-21 21:50:20.098429", "rc": 0, "start": "2021-11-21 21:50:20.097112", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:20.359 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.386 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.040) 0:01:14.321 ******* 2021-11-21T21:50:20.409 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.022) 0:01:14.344 ******* 2021-11-21T21:50:20.430 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.021) 0:01:14.365 ******* 2021-11-21T21:50:20.436 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-11-21T21:50:20.438 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** TASK [testnode : Install abrt] ************************************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.462 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.032) 0:01:14.398 ******* 2021-11-21T21:50:20.495 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.032) 0:01:14.430 ******* 2021-11-21T21:50:20.497 INFO:teuthology.task.ansible.out: 2021-11-21T21:50:20.519 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.024) 0:01:14.455 ******* 2021-11-21T21:50:20.526 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-11-21T21:50:20.528 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.528 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.554 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.034) 0:01:14.490 ******* 2021-11-21T21:50:20.580 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.026) 0:01:14.516 ******* 2021-11-21T21:50:20.589 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-11-21T21:50:20.590 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.590 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.616 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.035) 0:01:14.552 ******* 2021-11-21T21:50:20.643 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.026) 0:01:14.578 ******* 2021-11-21T21:50:20.649 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:50:20.651 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.677 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.033) 0:01:14.612 ******* 2021-11-21T21:50:20.701 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.024) 0:01:14.637 ******* 2021-11-21T21:50:20.709 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-11-21T21:50:20.710 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.741 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.040) 0:01:14.677 ******* 2021-11-21T21:50:20.753 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-11-21T21:50:20.754 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.755 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-11-21T21:50:20.755 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-11-21T21:50:20.757 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-11-21T21:50:20.757 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.782 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.040) 0:01:14.717 ******* 2021-11-21T21:50:20.807 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.024) 0:01:14.742 ******* 2021-11-21T21:50:20.832 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.025) 0:01:14.768 ******* 2021-11-21T21:50:20.842 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-21T21:50:20.844 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add sources list] ********************************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.880 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.047) 0:01:14.816 ******* 2021-11-21T21:50:20.914 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.033) 0:01:14.849 ******* 2021-11-21T21:50:20.928 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-11-21T21:50:20.930 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:20.967 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:20 +0000 (0:00:00.052) 0:01:14.902 ******* 2021-11-21T21:50:20.977 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-11-21T21:50:20.979 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001341", "end": "2021-11-21 21:50:21.308426", "rc": 0, "start": "2021-11-21 21:50:21.307085", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.007 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.040) 0:01:14.942 ******* 2021-11-21T21:50:21.032 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.025) 0:01:14.968 ******* 2021-11-21T21:50:21.039 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-11-21T21:50:21.041 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** 2021-11-21T21:50:21.074 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.041) 0:01:15.009 ******* 2021-11-21T21:50:21.100 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.026) 0:01:15.036 ******* 2021-11-21T21:50:21.109 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:50:21.111 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.142 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.041) 0:01:15.077 ******* 2021-11-21T21:50:21.149 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-11-21T21:50:21.151 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.178 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.036) 0:01:15.113 ******* 2021-11-21T21:50:21.185 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-11-21T21:50:21.187 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.216 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.038) 0:01:15.151 ******* 2021-11-21T21:50:21.224 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-21T21:50:21.226 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.253 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.037) 0:01:15.189 ******* 2021-11-21T21:50:21.262 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-21T21:50:21.264 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.288 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.035) 0:01:15.224 ******* 2021-11-21T21:50:21.295 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-11-21T21:50:21.297 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.329 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.040) 0:01:15.264 ******* 2021-11-21T21:50:21.338 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:50:21.340 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set apt preferences] ****************************************** ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-11-21T21:50:21.368 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.037) 0:01:15.301 ******* 2021-11-21T21:50:21.391 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.025) 0:01:15.327 ******* 2021-11-21T21:50:21.399 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:50:21.401 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add sources list] ********************************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.427 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.035) 0:01:15.362 ******* 2021-11-21T21:50:21.452 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.024) 0:01:15.387 ******* 2021-11-21T21:50:21.460 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-11-21T21:50:21.462 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.493 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.041) 0:01:15.429 ******* 2021-11-21T21:50:21.502 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-11-21T21:50:21.503 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.531 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.038) 0:01:15.467 ******* 2021-11-21T21:50:21.540 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-11-21T21:50:21.542 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.571 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.039) 0:01:15.507 ******* 2021-11-21T21:50:21.579 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-21T21:50:21.581 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.609 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.037) 0:01:15.545 ******* 2021-11-21T21:50:21.618 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-11-21T21:50:21.620 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.646 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.036) 0:01:15.581 ******* 2021-11-21T21:50:21.653 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:50:21.655 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.680 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.034) 0:01:15.616 ******* 2021-11-21T21:50:21.688 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-11-21T21:50:21.690 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.715 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.034) 0:01:15.650 ******* 2021-11-21T21:50:21.722 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-11-21T21:50:21.724 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.755 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.040) 0:01:15.691 ******* 2021-11-21T21:50:21.763 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-11-21T21:50:21.765 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.791 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.035) 0:01:15.726 ******* 2021-11-21T21:50:21.798 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-11-21T21:50:21.799 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.825 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.034) 0:01:15.761 ******* 2021-11-21T21:50:21.833 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-11-21T21:50:21.835 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.859 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.034) 0:01:15.795 ******* 2021-11-21T21:50:21.866 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-11-21T21:50:21.868 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.894 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.034) 0:01:15.830 ******* 2021-11-21T21:50:21.902 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [gibba016.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-11-21T21:50:21.904 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.929 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.034) 0:01:15.864 ******* 2021-11-21T21:50:21.935 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-11-21T21:50:21.936 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.961 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.031) 0:01:15.896 ******* 2021-11-21T21:50:21.967 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-11-21T21:50:21.968 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:21.992 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:21 +0000 (0:00:00.031) 0:01:15.928 ******* 2021-11-21T21:50:21.999 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-11-21T21:50:22.000 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:22.025 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.032) 0:01:15.961 ******* 2021-11-21T21:50:22.053 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-11-21T21:50:22.055 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:22.078 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.053) 0:01:16.014 ******* 2021-11-21T21:50:22.097 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-11-21T21:50:22.099 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001419", "end": "2021-11-21 21:50:22.295824", "rc": 0, "start": "2021-11-21 21:50:22.294405", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:22.122 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.044) 0:01:16.058 ******* 2021-11-21T21:50:22.130 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-11-21T21:50:22.132 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:22.158 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.035) 0:01:16.093 ******* 2021-11-21T21:50:22.164 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-11-21T21:50:22.194 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.036) 0:01:16.129 ******* 2021-11-21T21:50:22.201 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-11-21T21:50:22.203 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:22.228 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.034) 0:01:16.164 ******* 2021-11-21T21:50:22.247 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-21T21:50:22.249 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-11-21T21:50:22.333 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-11-21T21:50:22.334 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001354", "end": "2021-11-21 21:50:22.081277", "rc": 0, "start": "2021-11-21 21:50:22.079923", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:22.358 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.129) 0:01:16.293 ******* 2021-11-21T21:50:22.386 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-11-21T21:50:22.410 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.052) 0:01:16.346 ******* 2021-11-21T21:50:22.428 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-11-21T21:50:22.455 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.044) 0:01:16.390 ******* 2021-11-21T21:50:22.483 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:22.510 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.055) 0:01:16.446 ******* 2021-11-21T21:50:22.579 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-21T21:50:22.581 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-11-21T21:50:22.607 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.097) 0:01:16.543 ******* 2021-11-21T21:50:22.668 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-11-21T21:50:22.670 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-11-21T21:50:22.695 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.087) 0:01:16.631 ******* 2021-11-21T21:50:22.723 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-11-21T21:50:22.724 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:22.747 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.052) 0:01:16.683 ******* 2021-11-21T21:50:22.889 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-21T21:50:22.891 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-11-21T21:50:22.916 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:22 +0000 (0:00:00.168) 0:01:16.851 ******* 2021-11-21T21:50:23.060 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-11-21T21:50:23.062 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:23.086 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:23 +0000 (0:00:00.170) 0:01:17.022 ******* 2021-11-21T21:50:23.116 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-11-21T21:50:23.141 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:23 +0000 (0:00:00.054) 0:01:17.077 ******* 2021-11-21T21:50:23.208 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-21T21:50:23.209 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-11-21T21:50:23.272 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-11-21T21:50:23.274 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.010901", "end": "2021-11-21 21:50:23.030243", "rc": 0, "start": "2021-11-21 21:50:23.019342", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:23.306 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:23 +0000 (0:00:00.164) 0:01:17.242 ******* 2021-11-21T21:50:23.411 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-11-21T21:50:23.414 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:23.442 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:23 +0000 (0:00:00.135) 0:01:17.378 ******* 2021-11-21T21:50:23.471 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-11-21T21:50:23.496 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:23 +0000 (0:00:00.053) 0:01:17.432 ******* 2021-11-21T21:50:23.592 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-21T21:50:23.594 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-11-21T21:50:23.667 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-11-21T21:50:23.669 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.011358", "end": "2021-11-21 21:50:24.041695", "rc": 0, "start": "2021-11-21 21:50:24.030337", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:23.693 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:23 +0000 (0:00:00.196) 0:01:17.628 ******* 2021-11-21T21:50:23.936 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-21T21:50:23.938 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-11-21T21:50:23.963 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:23 +0000 (0:00:00.270) 0:01:17.899 ******* 2021-11-21T21:50:24.290 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-11-21T21:50:24.317 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.353) 0:01:18.252 ******* 2021-11-21T21:50:24.443 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-11-21T21:50:24.445 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:24.480 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.163) 0:01:18.416 ******* 2021-11-21T21:50:24.510 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-11-21T21:50:24.536 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.055) 0:01:18.471 ******* 2021-11-21T21:50:24.661 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-11-21T21:50:24.662 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.010706", "end": "2021-11-21 21:50:24.856361", "rc": 0, "start": "2021-11-21 21:50:24.845655", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:24.695 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.158) 0:01:18.630 ******* 2021-11-21T21:50:24.767 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:24.793 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.098) 0:01:18.729 ******* 2021-11-21T21:50:24.822 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-11-21T21:50:24.852 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.059) 0:01:18.788 ******* 2021-11-21T21:50:24.880 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:24.910 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.057) 0:01:18.846 ******* 2021-11-21T21:50:24.975 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-11-21T21:50:24.977 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.010458", "end": "2021-11-21 21:50:24.724196", "rc": 0, "start": "2021-11-21 21:50:24.713738", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:50:25.002 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:24 +0000 (0:00:00.091) 0:01:18.938 ******* 2021-11-21T21:50:25.187 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-11-21T21:50:25.189 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:25.214 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:25 +0000 (0:00:00.211) 0:01:19.149 ******* 2021-11-21T21:50:26.289 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:26.317 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:26 +0000 (0:00:01.102) 0:01:20.252 ******* 2021-11-21T21:50:26.550 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:26.578 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:26 +0000 (0:00:00.260) 0:01:20.513 ******* 2021-11-21T21:50:32.343 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading dat 2021-11-21T21:50:32.345 INFO:teuthology.task.ansible.out:abase ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nRemoving qemu-utils (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...\r\nRemoving librbd1 (15.2.13-0ubuntu0.20.04.2) ...\r\nRemoving librados2 (15.2.13-0ubuntu0.20.04.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Read 2021-11-21T21:50:32.347 INFO:teuthology.task.ansible.out:ing database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.17) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...", "Removing librbd1 (15.2.13-0ubuntu0.20.04.2) ...", "Removing librados2 (15.2.13-0ubuntu0.20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-11-21T21:50:32.371 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:32 +0000 (0:00:05.793) 0:01:26.306 ******* 2021-11-21T21:50:32.531 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading databa 2021-11-21T21:50:32.532 INFO:teuthology.task.ansible.out:se ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nRemoving qemu-utils (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...\r\nRemoving librbd1 (15.2.13-0ubuntu0.20.04.2) ...\r\nRemoving librados2 (15.2.13-0ubuntu0.20.04.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Readi 2021-11-21T21:50:32.534 INFO:teuthology.task.ansible.out:ng database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.17) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...", "Removing librbd1 (15.2.13-0ubuntu0.20.04.2) ...", "Removing librados2 (15.2.13-0ubuntu0.20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-11-21T21:50:32.557 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:32 +0000 (0:00:00.186) 0:01:26.493 ******* 2021-11-21T21:50:33.719 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading databa 2021-11-21T21:50:33.720 INFO:teuthology.task.ansible.out:se ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nRemoving qemu-utils (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...\r\nRemoving librbd1 (15.2.13-0ubuntu0.20.04.2) ...\r\nRemoving librados2 (15.2.13-0ubuntu0.20.04.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Readi 2021-11-21T21:50:33.722 INFO:teuthology.task.ansible.out:ng database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.17) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...", "Removing librbd1 (15.2.13-0ubuntu0.20.04.2) ...", "Removing librados2 (15.2.13-0ubuntu0.20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-11-21T21:50:33.746 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:33 +0000 (0:00:01.188) 0:01:27.682 ******* 2021-11-21T21:50:33.976 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading databa 2021-11-21T21:50:33.978 INFO:teuthology.task.ansible.out:se ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nRemoving qemu-utils (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.17) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...\r\nRemoving librbd1 (15.2.13-0ubuntu0.20.04.2) ...\r\nRemoving librados2 (15.2.13-0ubuntu0.20.04.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 59 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Readi 2021-11-21T21:50:33.980 INFO:teuthology.task.ansible.out:ng database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.17) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.17) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.17) ...", "Removing librbd1 (15.2.13-0ubuntu0.20.04.2) ...", "Removing librados2 (15.2.13-0ubuntu0.20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-11-21T21:50:34.004 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:34 +0000 (0:00:00.257) 0:01:27.940 ******* 2021-11-21T21:50:46.245 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* changed: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils\n4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.\nNeed to get 1519 kB of archives.\nAfter this operation, 3072 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]\nGet:2 http://archive 2021-11-21T21:50:46.247 INFO:teuthology.task.ansible.out:.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [1181 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]\nFetched 1519 kB in 1s (1399 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.8) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.13) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Readin 2021-11-21T21:50:46.249 INFO:teuthology.task.ansible.out:g state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils", "4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.", "Need to get 1519 kB of archives.", "After this operation, 3072 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [1181 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]", "Fetched 1519 kB in 1s (1399 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database . 2021-11-21T21:50:46.250 INFO:teuthology.task.ansible.out:.. 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.8) ...", "Setting up apache2-data (2.4.41-4ubuntu3.8) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.8) ...", "Setting up apache2 (2.4.41-4ubuntu3.8) ...", "Processing triggers for systemd (245.4-4ubuntu3.13) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for ufw (0.36-6) ..."]} 2021-11-21T21:50:46.270 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:46 +0000 (0:00:12.265) 0:01:40.206 ******* 2021-11-21T21:50:46.299 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-11-21T21:50:46.301 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:46.323 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:46 +0000 (0:00:00.053) 0:01:40.259 ******* 2021-11-21T21:50:46.500 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:50:46.502 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils\n4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.\nNeed to get 1519 kB of archives.\nAfter this operation, 3072 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [118 2021-11-21T21:50:46.504 INFO:teuthology.task.ansible.out:1 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]\nFetched 1519 kB in 1s (1399 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.8) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.13) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and ar 2021-11-21T21:50:46.505 INFO:teuthology.task.ansible.out:e no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils", "4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.", "Need to get 1519 kB of archives.", "After this operation, 3072 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [1181 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]", "Fetched 1519 kB in 1s (1399 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading databas 2021-11-21T21:50:46.507 INFO:teuthology.task.ansible.out:e ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.8) ...", "Setting up apache2-data (2.4.41-4ubuntu3.8) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.8) ...", "Setting up apache2 (2.4.41-4ubuntu3.8) ...", "Processing triggers for systemd (245.4-4ubuntu3.13) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for ufw (0.36-6) ..."]} 2021-11-21T21:50:46.527 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:46 +0000 (0:00:00.203) 0:01:40.463 ******* 2021-11-21T21:50:46.557 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-11-21T21:50:46.559 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:46.594 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:46 +0000 (0:00:00.066) 0:01:40.529 ******* 2021-11-21T21:50:47.497 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:50:47.500 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils\n4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.\nNeed to get 1519 kB of archives.\nAfter this operation, 3072 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [118 2021-11-21T21:50:47.502 INFO:teuthology.task.ansible.out:1 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]\nFetched 1519 kB in 1s (1501 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.8) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.13) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and ar 2021-11-21T21:50:47.504 INFO:teuthology.task.ansible.out:e no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils", "4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.", "Need to get 1519 kB of archives.", "After this operation, 3072 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [1181 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]", "Fetched 1519 kB in 1s (1501 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading databas 2021-11-21T21:50:47.505 INFO:teuthology.task.ansible.out:e ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.8) ...", "Setting up apache2-data (2.4.41-4ubuntu3.8) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.8) ...", "Setting up apache2 (2.4.41-4ubuntu3.8) ...", "Processing triggers for systemd (245.4-4ubuntu3.13) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for ufw (0.36-6) ..."]} 2021-11-21T21:50:47.525 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:47 +0000 (0:00:00.930) 0:01:41.460 ******* 2021-11-21T21:50:47.555 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-11-21T21:50:47.556 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:47.580 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:47 +0000 (0:00:00.055) 0:01:41.515 ******* 2021-11-21T21:50:47.830 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:50:47.832 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11\n liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5\n libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0\n libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common\n libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data\n qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils\n4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.\nNeed to get 1519 kB of archives.\nAfter this operation, 3072 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [118 2021-11-21T21:50:47.834 INFO:teuthology.task.ansible.out:1 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]\nFetched 1519 kB in 1s (1407 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.8) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.8) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.13) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and ar 2021-11-21T21:50:47.835 INFO:teuthology.task.ansible.out:e no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 libllvm11", " liborc-0.4-0 libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5", " libtag1v5-vanilla libusbredirparser1 libv4l-0 libv4lconvert0", " libvirglrenderer1 libvisual-0.4-0 libvte-2.91-0 libvte-2.91-common", " libwayland-server0 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic msr-tools ovmf qemu-system-data", " qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils", "4 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.", "Need to get 1519 kB of archives.", "After this operation, 3072 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.8 [95.5 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.8 [1181 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.8 [159 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.8 [84.5 kB]", "Fetched 1519 kB in 1s (1407 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading databas 2021-11-21T21:50:47.837 INFO:teuthology.task.ansible.out:e ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.8_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.8_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.8) over (2.4.41-4ubuntu3.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.8) ...", "Setting up apache2-data (2.4.41-4ubuntu3.8) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.8) ...", "Setting up apache2 (2.4.41-4ubuntu3.8) ...", "Processing triggers for systemd (245.4-4ubuntu3.13) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for ufw (0.36-6) ..."]} 2021-11-21T21:50:47.859 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:47 +0000 (0:00:00.279) 0:01:41.795 ******* 2021-11-21T21:50:47.901 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:47.926 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:47 +0000 (0:00:00.067) 0:01:41.862 ******* 2021-11-21T21:50:56.407 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:50:56.409 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3.2).\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n qemu-system-data qemu-system-gui\n2 upgraded, 6 newly installed, 0 to remove and 53 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qe 2021-11-21T21:50:56.410 INFO:teuthology.task.ansible.out:mu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]\nFetched 14.3 MB in 1s (10.4 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nPreparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...\r\nPreparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 (1:4.2-3ub 2021-11-21T21:50:56.412 INFO:teuthology.task.ansible.out:untu6.18) over (1:4.2-3ubuntu6.17) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.18) ...\r\nSetting up librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.18) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.25.1-1ubuntu3.2).", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " qemu-system-data qemu-system-gui", "2 upgrade 2021-11-21T21:50:56.414 INFO:teuthology.task.ansible.out:d, 6 newly installed, 0 to remove and 53 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]", "Fetched 14.3 MB in 1s (10.4 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librados2 (15.2.14-0ubuntu0.20.04.1) ..." 2021-11-21T21:50:56.415 INFO:teuthology.task.ansible.out:, "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.18) ...", "Preparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Preparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.18) ...", "Setting up librados2 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.18) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.18) ...", "Proce 2021-11-21T21:50:56.416 INFO:teuthology.task.ansible.out:ssing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for mime-support (3.64ubuntu1) ..."]} 2021-11-21T21:50:56.444 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:56 +0000 (0:00:08.517) 0:01:50.379 ******* 2021-11-21T21:50:56.526 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3.2).\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n qemu-system-data qemu-system-gui\n2 upgraded, 6 newly installed, 0 to remove and 53 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qe 2021-11-21T21:50:56.528 INFO:teuthology.task.ansible.out:mu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]\nFetched 14.3 MB in 1s (10.9 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nPreparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...\r\nPreparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 (1:4.2-3ub 2021-11-21T21:50:56.530 INFO:teuthology.task.ansible.out:untu6.18) over (1:4.2-3ubuntu6.17) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.18) ...\r\nSetting up librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.18) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.25.1-1ubuntu3.2).", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " qemu-system-data qemu-system-gui", "2 upgrade 2021-11-21T21:50:56.531 INFO:teuthology.task.ansible.out:d, 6 newly installed, 0 to remove and 53 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]", "Fetched 14.3 MB in 1s (10.9 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librados2 (15.2.14-0ubuntu0.20.04.1) ..." 2021-11-21T21:50:56.533 INFO:teuthology.task.ansible.out:, "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.18) ...", "Preparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Preparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.18) ...", "Setting up librados2 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.18) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.18) ...", "Processing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for mime-support (3.64ubuntu1) ..."]} 2021-11-21T21:50:56.555 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:56 +0000 (0:00:00.111) 0:01:50.491 ******* 2021-11-21T21:50:56.887 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-11-21T21:50:56.889 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:56.914 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:56 +0000 (0:00:00.358) 0:01:50.850 ******* 2021-11-21T21:50:57.029 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:57.055 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.141) 0:01:50.991 ******* 2021-11-21T21:50:57.376 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-11-21T21:50:57.378 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:57.406 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.350) 0:01:51.341 ******* 2021-11-21T21:50:57.509 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:57.538 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.132) 0:01:51.474 ******* 2021-11-21T21:50:57.559 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-11-21T21:50:57.561 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-11-21T21:50:57.587 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.048) 0:01:51.523 ******* 2021-11-21T21:50:57.616 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-11-21T21:50:57.618 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:57.640 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.053) 0:01:51.576 ******* 2021-11-21T21:50:57.671 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-11-21T21:50:57.673 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-11-21T21:50:57.698 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.056) 0:01:51.633 ******* 2021-11-21T21:50:57.736 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-11-21T21:50:57.738 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:57.761 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.063) 0:01:51.697 ******* 2021-11-21T21:50:57.769 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-11-21T21:50:57.771 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-11-21T21:50:57.797 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.035) 0:01:51.732 ******* 2021-11-21T21:50:57.868 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-11-21T21:50:57.900 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:57 +0000 (0:00:00.103) 0:01:51.836 ******* 2021-11-21T21:50:58.078 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:50:58.080 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3.2).\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n qemu-system-data qemu-system-gui\n2 upgraded, 6 newly installed, 0 to remove and 53 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qe 2021-11-21T21:50:58.081 INFO:teuthology.task.ansible.out:mu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]\nFetched 14.3 MB in 1s (11.8 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nPreparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...\r\nPreparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 (1:4.2-3ub 2021-11-21T21:50:58.083 INFO:teuthology.task.ansible.out:untu6.18) over (1:4.2-3ubuntu6.17) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.18) ...\r\nSetting up librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.18) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.25.1-1ubuntu3.2).", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " qemu-system-data qemu-system-gui", "2 upgrade 2021-11-21T21:50:58.084 INFO:teuthology.task.ansible.out:d, 6 newly installed, 0 to remove and 53 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]", "Fetched 14.3 MB in 1s (11.8 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librados2 (15.2.14-0ubuntu0.20.04.1) ..." 2021-11-21T21:50:58.086 INFO:teuthology.task.ansible.out:, "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.18) ...", "Preparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Preparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.18) ...", "Setting up librados2 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.18) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.18) ...", "Proce 2021-11-21T21:50:58.086 INFO:teuthology.task.ansible.out:ssing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for mime-support (3.64ubuntu1) ..."]} 2021-11-21T21:50:58.088 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-11-21T21:50:58.090 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-11-21T21:50:58.114 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.213) 0:01:52.050 ******* 2021-11-21T21:50:58.138 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.023) 0:01:52.073 ******* 2021-11-21T21:50:58.180 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-11-21T21:50:58.207 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.069) 0:01:52.142 ******* 2021-11-21T21:50:58.238 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [gibba016.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-11-21T21:50:58.273 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.065) 0:01:52.208 ******* 2021-11-21T21:50:58.346 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-11-21T21:50:58.372 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.098) 0:01:52.307 ******* 2021-11-21T21:50:58.392 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-11-21T21:50:58.394 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:50:58.418 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.046) 0:01:52.354 ******* 2021-11-21T21:50:58.509 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:50:58.535 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.116) 0:01:52.470 ******* 2021-11-21T21:50:58.563 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-11-21T21:50:58.565 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:58.589 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.054) 0:01:52.525 ******* 2021-11-21T21:50:58.629 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-11-21T21:50:58.630 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:50:44 UTC", "ActiveEnterTimestampMonotonic": "130102858", "ActiveExitTimestamp": "Sun 2021-11-21 21:50:44 UTC", "ActiveExitTimestampMonotonic": "130022788", "ActiveState": "active", "After": "system.slice -.mount network.target sysinit.target basic.target remote-fs.target nss-lookup.target systemd-tmpfiles-setup.service systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:50:44 UTC", "AssertTimestampMonotonic": "130058008", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:50:44 UTC", "ConditionTimestampMonotonic": "130058007", "ConfigurationDirect 2021-11-21T21:50:58.632 INFO:teuthology.task.ansible.out:oryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6652", "ExecMainStartTimestamp": "Sun 2021-11-21 21:50:44 UTC", "ExecMainStartTimestampMonotonic": "130102840", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriori 2021-11-21T21:50:58.634 INFO:teuthology.task.ansible.out:ty": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:50:44 UTC", "InactiveEnterTimestampMonotonic": "130056084", "InactiveExitTimestamp": "Sun 2021-11-21 21:50:44 UTC", "InactiveExitTimestampMonotonic": "130061349", "InvocationID": "f079295261024a7c8d5109a954ee9262", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6652", "MemoryAccounting": "yes", "MemoryCurrent": "7184384", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySw 2021-11-21T21:50:58.635 INFO:teuthology.task.ansible.out:apMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:50:44 UTC", "StateChangeTimestampMonotonic": "130102858", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "T 2021-11-21T21:50:58.637 INFO:teuthology.task.ansible.out:TYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:50:58.658 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.068) 0:01:52.593 ******* 2021-11-21T21:50:58.666 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:50:58.667 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3.2).\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n qemu-system-data qemu-system-gui\n2 upgraded, 6 newly installed, 0 to remove and 53 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qe 2021-11-21T21:50:58.669 INFO:teuthology.task.ansible.out:mu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]\nFetched 14.3 MB in 1s (11.5 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174615 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nPreparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...\r\nPreparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 (1:4.2-3ub 2021-11-21T21:50:58.671 INFO:teuthology.task.ansible.out:untu6.18) over (1:4.2-3ubuntu6.17) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.18) ...\r\nSetting up librados2 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.18) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.25.1-1ubuntu3.2).", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " qemu-system-data qemu-system-gui", "2 upgrade 2021-11-21T21:50:58.672 INFO:teuthology.task.ansible.out:d, 6 newly installed, 0 to remove and 53 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.1 [3215 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.1 [1620 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.18 [54.8 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.18 [1056 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.18 [563 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.18 [40.8 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.18 [6748 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.18 [973 kB]", "Fetched 14.3 MB in 1s (11.5 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174615 files and directories currently installed.)", "Preparing to unpack .../0-librados2_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librados2 (15.2.14-0ubuntu0.20.04.1) ..." 2021-11-21T21:50:58.678 INFO:teuthology.task.ansible.out:, "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_15.2.14-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Selecting 2021-11-21T21:50:58.679 INFO:teuthology.task.ansible.out: previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.18) ...", "Preparing to unpack .../4-qemu-system-data_1%3a4.2-3ubuntu6.18_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Preparing to unpack .../5-qemu-system-gui_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.17) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../6-qemu-system-x86_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../7-qemu-utils_1%3a4.2-3ubuntu6.18_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.18) ...", "Setting up librados2 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up librbd1 (15.2.14-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.18) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.18) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.18) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.18) ...", "Processing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for mime-support (3.64ubuntu1) ..."]} 2021-11-21T21:50:58.706 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.047) 0:01:52.641 ******* 2021-11-21T21:50:58.712 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:50:58.714 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:58.740 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.034) 0:01:52.676 ******* 2021-11-21T21:50:58.758 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-11-21T21:50:58.760 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ActiveEnterTimestampMonotonic": "132575369", "ActiveExitTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ActiveExitTimestampMonotonic": "132532844", "ActiveState": "active", "After": "network.target basic.target systemd-tmpfiles-setup.service remote-fs.target system.slice nss-lookup.target -.mount systemd-journald.socket sysinit.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:50:45 UTC", "AssertTimestampMonotonic": "132563473", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ConditionTimestampMonotonic": "132563473", "ConfigurationDirect 2021-11-21T21:50:58.762 INFO:teuthology.task.ansible.out:oryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6602", "ExecMainStartTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ExecMainStartTimestampMonotonic": "132575359", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriori 2021-11-21T21:50:58.764 INFO:teuthology.task.ansible.out:ty": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:50:45 UTC", "InactiveEnterTimestampMonotonic": "132563052", "InactiveExitTimestamp": "Sun 2021-11-21 21:50:45 UTC", "InactiveExitTimestampMonotonic": "132564152", "InvocationID": "734a09566fbe47cd8ae5303ecc32cb53", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6602", "MemoryAccounting": "yes", "MemoryCurrent": "7331840", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySw 2021-11-21T21:50:58.765 INFO:teuthology.task.ansible.out:apMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:50:45 UTC", "StateChangeTimestampMonotonic": "132575369", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "T 2021-11-21T21:50:58.767 INFO:teuthology.task.ansible.out:TYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:50:58.787 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.046) 0:01:52.722 ******* 2021-11-21T21:50:58.796 INFO:teuthology.task.ansible.out: 2021-11-21T21:50:58.798 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:58.830 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.043) 0:01:52.766 ******* 2021-11-21T21:50:58.839 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:50:58.841 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:58.865 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.034) 0:01:52.801 ******* 2021-11-21T21:50:58.871 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:50:58.873 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:58.897 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.031) 0:01:52.832 ******* 2021-11-21T21:50:58.903 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-11-21T21:50:58.905 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:58.929 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.031) 0:01:52.864 ******* 2021-11-21T21:50:58.935 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-21T21:50:58.937 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:58.961 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.032) 0:01:52.897 ******* 2021-11-21T21:50:58.969 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:50:58.971 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:58.996 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:58 +0000 (0:00:00.034) 0:01:52.931 ******* 2021-11-21T21:50:59.004 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-11-21T21:50:59.006 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.030 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.033) 0:01:52.965 ******* 2021-11-21T21:50:59.037 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-21T21:50:59.039 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages with --no-install-recommends] **************** ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:59.064 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.033) 0:01:52.999 ******* 2021-11-21T21:50:59.086 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.022) 0:01:53.022 ******* 2021-11-21T21:50:59.094 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-11-21T21:50:59.096 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.122 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.035) 0:01:53.058 ******* 2021-11-21T21:50:59.145 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.023) 0:01:53.081 ******* 2021-11-21T21:50:59.153 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-11-21T21:50:59.155 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.180 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.034) 0:01:53.116 ******* 2021-11-21T21:50:59.210 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.030) 0:01:53.146 ******* 2021-11-21T21:50:59.217 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-11-21T21:50:59.218 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-11-21T21:50:59.219 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-11-21T21:50:59.219 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.248 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.037) 0:01:53.183 ******* 2021-11-21T21:50:59.272 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.024) 0:01:53.207 ******* 2021-11-21T21:50:59.300 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.027) 0:01:53.235 ******* 2021-11-21T21:50:59.307 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:50:59.308 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.335 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.035) 0:01:53.271 ******* 2021-11-21T21:50:59.343 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-21T21:50:59.345 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.371 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.035) 0:01:53.307 ******* 2021-11-21T21:50:59.396 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.024) 0:01:53.331 ******* 2021-11-21T21:50:59.404 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-21T21:50:59.405 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.433 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.037) 0:01:53.369 ******* 2021-11-21T21:50:59.467 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.033) 0:01:53.402 ******* 2021-11-21T21:50:59.475 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:50:59.476 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.508 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.041) 0:01:53.444 ******* 2021-11-21T21:50:59.518 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:50:59.520 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure fuse group exists.] ************************************ ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-11-21T21:50:59.546 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.037) 0:01:53.482 ******* 2021-11-21T21:50:59.575 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.028) 0:01:53.510 ******* 2021-11-21T21:50:59.582 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:50:59.584 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.622 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.047) 0:01:53.558 ******* 2021-11-21T21:50:59.651 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.028) 0:01:53.587 ******* 2021-11-21T21:50:59.660 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-21T21:50:59.662 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.663 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:50:59.691 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.039) 0:01:53.626 ******* 2021-11-21T21:50:59.716 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.025) 0:01:53.652 ******* 2021-11-21T21:50:59.729 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:50:59.730 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.732 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.763 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.047) 0:01:53.699 ******* 2021-11-21T21:50:59.807 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.043) 0:01:53.742 ******* 2021-11-21T21:50:59.815 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-21T21:50:59.817 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.845 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.038) 0:01:53.780 ******* 2021-11-21T21:50:59.855 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:50:59.857 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.857 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-11-21T21:50:59.884 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.039) 0:01:53.820 ******* 2021-11-21T21:50:59.911 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.026) 0:01:53.847 ******* 2021-11-21T21:50:59.919 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-11-21T21:50:59.921 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-11-21T21:50:59.922 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:50:59.949 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.037) 0:01:53.884 ******* 2021-11-21T21:50:59.975 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.026) 0:01:53.911 ******* 2021-11-21T21:51:00.004 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:50:59 +0000 (0:00:00.028) 0:01:53.939 ******* 2021-11-21T21:51:00.013 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-11-21T21:51:00.015 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:00.057 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.053) 0:01:53.992 ******* 2021-11-21T21:51:00.086 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.028) 0:01:54.021 ******* 2021-11-21T21:51:00.116 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.030) 0:01:54.051 ******* 2021-11-21T21:51:00.502 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:51:00 +0000 (0:00:00.058) 0:01:54.110 ******* Sunday 21 November 2021 21:51:00 +0000 (0:00:00.024) 0:01:54.135 ******* TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [gibba019.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} Sunday 21 November 2021 21:51:00 +0000 (0:00:00.036) 0:01:54.171 ******* TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:51:00 +0000 (0:00:00.034) 0:01:54.206 ******* Sunday 21 November 2021 21:51:00 +0000 (0:00:00.023) 0:01:54.229 ******* TASK [testnode : Ensure fuse group exists.] ************************************ ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:51:00 +0000 (0:00:00.033) 0:01:54 2021-11-21T21:51:00.504 INFO:teuthology.task.ansible.out:.262 ******* Sunday 21 November 2021 21:51:00 +0000 (0:00:00.021) 0:01:54.284 ******* TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:51:00 +0000 (0:00:00.033) 0:01:54.317 ******* Sunday 21 November 2021 21:51:00 +0000 (0:00:00.022) 0:01:54.340 ******* TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} Sunday 21 November 2021 21:51:00 +0000 (0:00:00.043) 0:01:54.383 ******* TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:51:00 +0000 (0:00:00.041) 0:01:54.424 ******* 2021-11-21T21:51:00.517 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.028) 0:01:54.452 ******* 2021-11-21T21:51:00.525 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:00.527 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:00.553 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.035) 0:01:54.488 ******* 2021-11-21T21:51:00.560 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-21T21:51:00.562 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:00.588 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.035) 0:01:54.524 ******* 2021-11-21T21:51:00.596 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-21T21:51:00.598 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:00.624 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.035) 0:01:54.559 ******* 2021-11-21T21:51:00.635 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-11-21T21:51:00.660 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.036) 0:01:54.595 ******* 2021-11-21T21:51:00.683 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.023) 0:01:54.619 ******* 2021-11-21T21:51:00.690 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-11-21T21:51:00.691 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ActiveEnterTimestampMonotonic": "132441328", "ActiveExitTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ActiveExitTimestampMonotonic": "132398885", "ActiveState": "active", "After": "network.target system.slice nss-lookup.target -.mount sysinit.target basic.target systemd-tmpfiles-setup.service remote-fs.target systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:50:45 UTC", "AssertTimestampMonotonic": "132429600", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ConditionTimestampMonotonic": "132429600", "ConfigurationDirect 2021-11-21T21:51:00.693 INFO:teuthology.task.ansible.out:oryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6643", "ExecMainStartTimestamp": "Sun 2021-11-21 21:50:45 UTC", "ExecMainStartTimestampMonotonic": "132441318", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriori 2021-11-21T21:51:00.695 INFO:teuthology.task.ansible.out:ty": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:50:45 UTC", "InactiveEnterTimestampMonotonic": "132429176", "InactiveExitTimestamp": "Sun 2021-11-21 21:50:45 UTC", "InactiveExitTimestampMonotonic": "132430233", "InvocationID": "84faa86181944d9faea30bac9120350f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6643", "MemoryAccounting": "yes", "MemoryCurrent": "7192576", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySw 2021-11-21T21:51:00.696 INFO:teuthology.task.ansible.out:apMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:50:45 UTC", "StateChangeTimestampMonotonic": "132441328", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "T 2021-11-21T21:51:00.698 INFO:teuthology.task.ansible.out:TYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:00.721 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.037) 0:01:54.656 ******* 2021-11-21T21:51:00.743 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.022) 0:01:54.679 ******* 2021-11-21T21:51:00.767 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.023) 0:01:54.702 ******* 2021-11-21T21:51:00.774 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-11-21T21:51:00.776 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:00.807 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.040) 0:01:54.743 ******* 2021-11-21T21:51:00.849 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.041) 0:01:54.784 ******* 2021-11-21T21:51:00.857 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:51:00.859 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [gibba036.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-11-21T21:51:00.884 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.035) 0:01:54.819 ******* 2021-11-21T21:51:00.907 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.023) 0:01:54.843 ******* 2021-11-21T21:51:00.914 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:00.916 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:00.916 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-11-21T21:51:00.917 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-11-21T21:51:00.944 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.036) 0:01:54.879 ******* 2021-11-21T21:51:00.966 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.022) 0:01:54.902 ******* 2021-11-21T21:51:00.989 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:00 +0000 (0:00:00.023) 0:01:54.925 ******* 2021-11-21T21:51:00.997 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-21T21:51:00.999 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.025 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.035) 0:01:54.960 ******* 2021-11-21T21:51:01.033 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:51:01.035 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.061 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.035) 0:01:54.996 ******* 2021-11-21T21:51:01.067 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-21T21:51:01.069 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.095 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.033) 0:01:55.030 ******* 2021-11-21T21:51:01.103 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-11-21T21:51:01.105 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.130 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.035) 0:01:55.066 ******* 2021-11-21T21:51:01.156 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.025) 0:01:55.091 ******* 2021-11-21T21:51:01.162 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-11-21T21:51:01.164 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.190 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.034) 0:01:55.126 ******* 2021-11-21T21:51:01.199 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-11-21T21:51:01.201 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.239 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.048) 0:01:55.175 ******* 2021-11-21T21:51:01.248 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-21T21:51:01.249 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-11-21T21:51:01.274 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.035) 0:01:55.210 ******* 2021-11-21T21:51:01.284 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-11-21T21:51:01.285 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.312 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.037) 0:01:55.248 ******* 2021-11-21T21:51:01.319 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-11-21T21:51:01.321 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.346 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.033) 0:01:55.281 ******* 2021-11-21T21:51:01.368 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.022) 0:01:55.304 ******* 2021-11-21T21:51:01.376 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-21T21:51:01.377 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.378 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-11-21T21:51:01.380 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:50:46 UTC", "ActiveEnterTimestampMonotonic": "133183840", "ActiveExitTimestamp": "Sun 2021-11-21 21:50:46 UTC", "ActiveExitTimestampMonotonic": "133142179", "ActiveState": "active", "After": "nss-lookup.target systemd-tmpfiles-setup.service basic.target system.slice sysinit.target -.mount systemd-journald.socket network.target remote-fs.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:50:46 UTC", "AssertTimestampMonotonic": "133172580", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:50:46 UTC", "ConditionTimestampMonotonic": "133172579", "ConfigurationDirect 2021-11-21T21:51:01.381 INFO:teuthology.task.ansible.out:oryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6615", "ExecMainStartTimestamp": "Sun 2021-11-21 21:50:46 UTC", "ExecMainStartTimestampMonotonic": "133183831", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriori 2021-11-21T21:51:01.383 INFO:teuthology.task.ansible.out:ty": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:50:46 UTC", "InactiveEnterTimestampMonotonic": "133172158", "InactiveExitTimestamp": "Sun 2021-11-21 21:50:46 UTC", "InactiveExitTimestampMonotonic": "133173322", "InvocationID": "5ce7dcc342a84dca87dbb7b5a11040f2", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127822", "LimitNPROCSoft": "127822", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127822", "LimitSIGPENDINGSoft": "127822", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6615", "MemoryAccounting": "yes", "MemoryCurrent": "7135232", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySw 2021-11-21T21:51:01.384 INFO:teuthology.task.ansible.out:apMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:50:46 UTC", "StateChangeTimestampMonotonic": "133183840", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "T 2021-11-21T21:51:01.386 INFO:teuthology.task.ansible.out:TYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:01.407 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.038) 0:01:55.342 ******* 2021-11-21T21:51:01.431 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.023) 0:01:55.366 ******* 2021-11-21T21:51:01.437 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-11-21T21:51:01.438 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.467 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.036) 0:01:55.403 ******* 2021-11-21T21:51:01.475 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-21T21:51:01.476 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.502 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.034) 0:01:55.438 ******* 2021-11-21T21:51:01.509 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.535 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.032) 0:01:55.470 ******* 2021-11-21T21:51:01.543 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-11-21T21:51:01.545 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.569 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.034) 0:01:55.505 ******* 2021-11-21T21:51:01.575 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-21T21:51:01.576 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.608 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.039) 0:01:55.544 ******* 2021-11-21T21:51:01.616 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-21T21:51:01.618 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.642 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.033) 0:01:55.578 ******* 2021-11-21T21:51:01.649 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-11-21T21:51:01.651 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} TASK [testnode : Install packages] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.651 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-21T21:51:01.652 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-11-21T21:51:01.676 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.034) 0:01:55.612 ******* 2021-11-21T21:51:01.700 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.023) 0:01:55.635 ******* 2021-11-21T21:51:01.724 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.024) 0:01:55.660 ******* 2021-11-21T21:51:01.731 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.733 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.758 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.034) 0:01:55.694 ******* 2021-11-21T21:51:01.783 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.024) 0:01:55.719 ******* 2021-11-21T21:51:01.791 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:51:01.793 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.821 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.037) 0:01:55.756 ******* 2021-11-21T21:51:01.831 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-11-21T21:51:01.833 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.859 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.038) 0:01:55.795 ******* 2021-11-21T21:51:01.865 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-21T21:51:01.867 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.894 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.034) 0:01:55.829 ******* 2021-11-21T21:51:01.903 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-11-21T21:51:01.905 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.934 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.039) 0:01:55.869 ******* 2021-11-21T21:51:01.941 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:01.943 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:01.968 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:01 +0000 (0:00:00.035) 0:01:55.904 ******* 2021-11-21T21:51:01.974 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-21T21:51:01.976 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-11-21T21:51:02.007 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.038) 0:01:55.943 ******* 2021-11-21T21:51:02.032 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.024) 0:01:55.967 ******* 2021-11-21T21:51:02.039 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-21T21:51:02.040 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-11-21T21:51:02.068 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.035) 0:01:56.003 ******* 2021-11-21T21:51:02.090 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.022) 0:01:56.026 ******* 2021-11-21T21:51:02.099 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:02.101 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.128 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.038) 0:01:56.064 ******* 2021-11-21T21:51:02.136 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-21T21:51:02.137 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.165 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.036) 0:01:56.101 ******* 2021-11-21T21:51:02.173 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:02.175 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.206 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.040) 0:01:56.141 ******* 2021-11-21T21:51:02.213 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-21T21:51:02.214 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.241 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.035) 0:01:56.177 ******* 2021-11-21T21:51:02.249 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-21T21:51:02.251 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.277 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.036) 0:01:56.213 ******* 2021-11-21T21:51:02.285 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-11-21T21:51:02.286 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-11-21T21:51:02.313 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.035) 0:01:56.248 ******* 2021-11-21T21:51:02.337 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.024) 0:01:56.273 ******* 2021-11-21T21:51:02.346 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-21T21:51:02.347 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-11-21T21:51:02.374 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.037) 0:01:56.310 ******* 2021-11-21T21:51:02.410 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.036) 0:01:56.346 ******* 2021-11-21T21:51:02.419 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-21T21:51:02.421 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [gibba016.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:51:02.448 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.037) 0:01:56.383 ******* 2021-11-21T21:51:02.472 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.024) 0:01:56.408 ******* 2021-11-21T21:51:02.479 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-21T21:51:02.481 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.508 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.035) 0:01:56.444 ******* 2021-11-21T21:51:02.533 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.025) 0:01:56.469 ******* 2021-11-21T21:51:02.541 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:02.543 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.568 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.034) 0:01:56.503 ******* 2021-11-21T21:51:02.575 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-11-21T21:51:02.577 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.602 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.033) 0:01:56.537 ******* 2021-11-21T21:51:02.627 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.024) 0:01:56.562 ******* 2021-11-21T21:51:02.636 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-21T21:51:02.637 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.638 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-11-21T21:51:02.665 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.038) 0:01:56.600 ******* 2021-11-21T21:51:02.688 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.022) 0:01:56.623 ******* 2021-11-21T21:51:02.709 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.021) 0:01:56.645 ******* 2021-11-21T21:51:02.717 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:02.717 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-11-21T21:51:02.718 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.744 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.034) 0:01:56.680 ******* 2021-11-21T21:51:02.775 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.030) 0:01:56.711 ******* 2021-11-21T21:51:02.783 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-21T21:51:02.785 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.810 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.034) 0:01:56.745 ******* 2021-11-21T21:51:02.833 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.022) 0:01:56.768 ******* 2021-11-21T21:51:02.841 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-11-21T21:51:02.843 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.843 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-11-21T21:51:02.869 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.036) 0:01:56.805 ******* 2021-11-21T21:51:02.891 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.021) 0:01:56.827 ******* 2021-11-21T21:51:02.915 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.023) 0:01:56.851 ******* 2021-11-21T21:51:02.922 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:02.925 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba016.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.925 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.925 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.926 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba016.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.951 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.036) 0:01:56.887 ******* 2021-11-21T21:51:02.977 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:02 +0000 (0:00:00.025) 0:01:56.912 ******* 2021-11-21T21:51:02.984 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.985 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.986 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-11-21T21:51:02.986 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-11-21T21:51:02.987 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.987 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.987 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.988 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.989 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba016.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.990 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:02.990 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.014 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.037) 0:01:56.950 ******* 2021-11-21T21:51:03.037 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.022) 0:01:56.973 ******* 2021-11-21T21:51:03.045 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.048 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.049 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.076 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.039) 0:01:57.012 ******* 2021-11-21T21:51:03.100 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.022) 0:01:57.035 ******* 2021-11-21T21:51:03.133 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.033) 0:01:57.068 ******* 2021-11-21T21:51:03.141 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-21T21:51:03.143 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.169 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.036) 0:01:57.105 ******* 2021-11-21T21:51:03.194 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.024) 0:01:57.130 ******* 2021-11-21T21:51:03.202 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-21T21:51:03.204 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.230 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.036) 0:01:57.166 ******* 2021-11-21T21:51:03.254 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.023) 0:01:57.189 ******* 2021-11-21T21:51:03.262 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-11-21T21:51:03.264 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-11-21T21:51:03.289 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.035) 0:01:57.225 ******* 2021-11-21T21:51:03.314 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.024) 0:01:57.250 ******* 2021-11-21T21:51:03.322 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-11-21T21:51:03.324 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004190", "end": "2021-11-21 21:51:03.252140", "rc": 0, "start": "2021-11-21 21:51:03.247950", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock"]} TASK [testnode : Set grub config.] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.354 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.040) 0:01:57.290 ******* 2021-11-21T21:51:03.380 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.025) 0:01:57.316 ******* 2021-11-21T21:51:03.388 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-11-21T21:51:03.390 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-21T21:51:03.390 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** 2021-11-21T21:51:03.391 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.391 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.391 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.392 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.393 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [gibba017.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.394 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.426 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.045) 0:01:57.361 ******* 2021-11-21T21:51:03.451 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.024) 0:01:57.386 ******* 2021-11-21T21:51:03.458 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-11-21T21:51:03.460 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.460 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.460 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.462 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [gibba017.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.464 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-11-21T21:51:03.465 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.490 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.039) 0:01:57.426 ******* 2021-11-21T21:51:03.522 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.031) 0:01:57.458 ******* 2021-11-21T21:51:03.529 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:03.530 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove /etc/ceph] ********************************************* ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-11-21T21:51:03.531 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.560 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.037) 0:01:57.495 ******* 2021-11-21T21:51:03.586 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.026) 0:01:57.522 ******* 2021-11-21T21:51:03.614 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.027) 0:01:57.549 ******* 2021-11-21T21:51:03.642 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-21T21:51:03.644 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.669 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.055) 0:01:57.605 ******* 2021-11-21T21:51:03.698 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-21T21:51:03.699 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.724 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.054) 0:01:57.659 ******* 2021-11-21T21:51:03.731 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-11-21T21:51:03.733 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003194", "end": "2021-11-21 21:51:03.703021", "rc": 0, "start": "2021-11-21 21:51:03.699827", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock"]} 2021-11-21T21:51:03.761 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.037) 0:01:57.697 ******* 2021-11-21T21:51:03.769 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-21T21:51:03.770 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.796 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.035) 0:01:57.732 ******* 2021-11-21T21:51:03.803 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-11-21T21:51:03.805 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.833 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.037) 0:01:57.769 ******* 2021-11-21T21:51:03.842 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-21T21:51:03.844 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.870 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.036) 0:01:57.806 ******* 2021-11-21T21:51:03.878 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-21T21:51:03.879 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.906 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.035) 0:01:57.842 ******* 2021-11-21T21:51:03.929 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.022) 0:01:57.865 ******* 2021-11-21T21:51:03.936 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-21T21:51:03.938 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:03.969 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.039) 0:01:57.904 ******* 2021-11-21T21:51:03.994 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:03 +0000 (0:00:00.025) 0:01:57.929 ******* 2021-11-21T21:51:04.003 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-11-21T21:51:04.004 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:04.031 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.036) 0:01:57.966 ******* 2021-11-21T21:51:04.054 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.023) 0:01:57.990 ******* 2021-11-21T21:51:04.063 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-11-21T21:51:04.064 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:04.092 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.037) 0:01:58.027 ******* 2021-11-21T21:51:04.123 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.031) 0:01:58.058 ******* 2021-11-21T21:51:04.142 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-21T21:51:04.144 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item={'block_used': 14049675, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045433, 'size_available': 925882093568, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187090, 'block_size': 4096, 'inode_available': 60867886}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045433, "block_size": 4096, "block_total": 240095108, "block_used": 14049675, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867886, "inode_total": 61054976, "inode_used": 187090, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882093568, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-11-21T21:51:04.171 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.048) 0:01:58.107 ******* 2021-11-21T21:51:04.232 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-11-21T21:51:04.234 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-11-21T21:51:04.259 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.087) 0:01:58.194 ******* 2021-11-21T21:51:04.287 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-11-21T21:51:04.289 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002440", "end": "2021-11-21 21:51:04.040727", "rc": 0, "start": "2021-11-21 21:51:04.038287", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:04.317 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.058) 0:01:58.253 ******* 2021-11-21T21:51:04.327 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-11-21T21:51:04.328 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-11-21T21:51:04.354 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.037) 0:01:58.290 ******* 2021-11-21T21:51:04.362 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-11-21T21:51:04.363 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'block_used': 14049675, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045433, 'size_available': 925882093568, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187090, 'block_size': 4096, 'inode_available': 60867886}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045433, "block_size": 4096, "block_total": 240095108, "block_used": 14049675, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867886, "inode_total": 61054976, "inode_used": 187090, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882093568, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:04.390 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.035) 0:01:58.325 ******* 2021-11-21T21:51:04.408 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:04.409 INFO:teuthology.task.ansible.out:TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-11-21T21:51:04.410 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:04.434 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.044) 0:01:58.370 ******* 2021-11-21T21:51:04.442 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-11-21T21:51:04.477 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.043) 0:01:58.413 ******* 2021-11-21T21:51:04.485 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-11-21T21:51:04.487 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:04.514 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.036) 0:01:58.449 ******* 2021-11-21T21:51:04.521 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-21T21:51:04.522 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-11-21T21:51:04.524 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:04.551 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.037) 0:01:58.487 ******* 2021-11-21T21:51:04.561 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-21T21:51:04.562 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item={'block_used': 14049682, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045426, 'size_available': 925882064896, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045426, "block_size": 4096, "block_total": 240095108, "block_used": 14049682, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882064896, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-11-21T21:51:04.600 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.048) 0:01:58.535 ******* 2021-11-21T21:51:04.607 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-11-21T21:51:04.609 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:04.610 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-11-21T21:51:04.610 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-11-21T21:51:04.638 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.037) 0:01:58.573 ******* 2021-11-21T21:51:04.661 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.023) 0:01:58.597 ******* 2021-11-21T21:51:04.721 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-11-21T21:51:04.723 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002384", "end": "2021-11-21 21:51:05.093840", "rc": 0, "start": "2021-11-21 21:51:05.091456", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:04.749 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.087) 0:01:58.684 ******* 2021-11-21T21:51:04.787 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-11-21T21:51:04.788 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'block_used': 14049682, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045426, 'size_available': 925882064896, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045426, "block_size": 4096, "block_total": 240095108, "block_used": 14049682, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882064896, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:04.813 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.064) 0:01:58.749 ******* 2021-11-21T21:51:04.842 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-11-21T21:51:04.869 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.056) 0:01:58.805 ******* 2021-11-21T21:51:04.899 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-11-21T21:51:04.901 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-11-21T21:51:04.926 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.057) 0:01:58.862 ******* 2021-11-21T21:51:04.935 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-11-21T21:51:04.937 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:04.964 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:04 +0000 (0:00:00.037) 0:01:58.900 ******* 2021-11-21T21:51:05.025 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-11-21T21:51:05.027 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:51:05.058 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.092) 0:01:58.993 ******* 2021-11-21T21:51:05.085 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-11-21T21:51:05.087 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.118 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.061) 0:01:59.054 ******* 2021-11-21T21:51:05.147 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-11-21T21:51:05.149 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.174 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.055) 0:01:59.110 ******* 2021-11-21T21:51:05.204 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-11-21T21:51:05.206 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.229 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.055) 0:01:59.165 ******* 2021-11-21T21:51:05.238 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-11-21T21:51:05.238 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-11-21T21:51:05.265 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.035) 0:01:59.201 ******* 2021-11-21T21:51:05.272 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-11-21T21:51:05.274 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.300 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.034) 0:01:59.236 ******* 2021-11-21T21:51:05.329 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-11-21T21:51:05.330 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.355 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.054) 0:01:59.290 ******* 2021-11-21T21:51:05.383 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:05.385 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-11-21T21:51:05.409 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.054) 0:01:59.344 ******* 2021-11-21T21:51:05.460 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-11-21T21:51:05.462 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.472 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.488 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.500 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.512 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [gibba019.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.523 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.536 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.537 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:05.549 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.551 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.564 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.566 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.601 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.191) 0:01:59.536 ******* 2021-11-21T21:51:05.607 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-11-21T21:51:05.609 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-11-21T21:51:05.635 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.034) 0:01:59.571 ******* 2021-11-21T21:51:05.765 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-11-21T21:51:05.767 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:51:05.790 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.154) 0:01:59.725 ******* 2021-11-21T21:51:05.830 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-11-21T21:51:05.832 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004053", "end": "2021-11-21 21:51:05.798471", "rc": 0, "start": "2021-11-21 21:51:05.794418", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock"]} TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-11-21T21:51:05.832 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.870 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.080) 0:01:59.806 ******* 2021-11-21T21:51:05.897 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.026) 0:01:59.832 ******* 2021-11-21T21:51:05.904 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-11-21T21:51:05.906 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.942 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.045) 0:01:59.878 ******* 2021-11-21T21:51:05.951 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-11-21T21:51:05.954 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:05.983 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:05 +0000 (0:00:00.040) 0:01:59.918 ******* 2021-11-21T21:51:05.991 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-11-21T21:51:05.993 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.027 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.043) 0:01:59.962 ******* 2021-11-21T21:51:06.035 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-11-21T21:51:06.037 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.039 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0FFyM8b6OgYiGnifpfE5Ec2uqo6HcfJKI'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.380841", "end": "2021-11-21 21:51:05.761139", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0FFyM8b6OgYiGnifpfE5Ec2uqo6HcfJKI"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:04.380298", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:06.042 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.043 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.073 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.045) 0:02:00.008 ******* 2021-11-21T21:51:06.079 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-11-21T21:51:06.080 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.111 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.038) 0:02:00.046 ******* 2021-11-21T21:51:06.120 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-11-21T21:51:06.122 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.156 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.045) 0:02:00.091 ******* 2021-11-21T21:51:06.190 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-11-21T21:51:06.191 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.215 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.058) 0:02:00.150 ******* 2021-11-21T21:51:06.256 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-11-21T21:51:06.283 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.067) 0:02:00.218 ******* 2021-11-21T21:51:06.333 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba036.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.344 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.345 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.355 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.366 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.368 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.378 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-11-21T21:51:06.399 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.401 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.410 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.412 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.422 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.423 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.433 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.435 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.463 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.179) 0:02:00.398 ******* 2021-11-21T21:51:06.510 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:06.512 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCmdlvul08UrhqUq0WvUWhZhVie9WLHDM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.469655", "end": "2021-11-21 21:51:06.874897", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCmdlvul08UrhqUq0WvUWhZhVie9WLHDM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:05.405242", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:06.512 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-dea 2021-11-21T21:51:06.514 INFO:teuthology.task.ansible.out:dline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.524 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.536 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-11-21T21:51:06.538 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:06.564 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.101) 0:02:00.500 ******* 2021-11-21T21:51:06.593 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.617 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.052) 0:02:00.552 ******* 2021-11-21T21:51:06.666 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-21T21:51:06.668 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'block_used': 14049683, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045425, 'size_available': 925882060800, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045425, "block_size": 4096, "block_total": 240095108, "block_used": 14049683, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882060800, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-11-21T21:51:06.693 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.076) 0:02:00.629 ******* 2021-11-21T21:51:06.703 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-11-21T21:51:06.705 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003739", "end": "2021-11-21 21:51:06.650669", "rc": 0, "start": "2021-11-21 21:51:06.646930", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology-lock"]} 2021-11-21T21:51:06.747 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.053) 0:02:00.682 ******* 2021-11-21T21:51:06.787 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-11-21T21:51:06.789 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.816 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.068) 0:02:00.751 ******* 2021-11-21T21:51:06.846 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-11-21T21:51:06.847 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002325", "end": "2021-11-21 21:51:06.589423", "rc": 0, "start": "2021-11-21 21:51:06.587098", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:06.882 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.066) 0:02:00.818 ******* 2021-11-21T21:51:06.891 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-11-21T21:51:06.893 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.920 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.037) 0:02:00.856 ******* 2021-11-21T21:51:06.927 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-11-21T21:51:06.929 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'block_used': 14049683, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045425, 'size_available': 925882060800, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045425, "block_size": 4096, "block_total": 240095108, "block_used": 14049683, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882060800, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.956 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.035) 0:02:00.892 ******* 2021-11-21T21:51:06.963 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-11-21T21:51:06.965 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:06.989 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:06 +0000 (0:00:00.032) 0:02:00.924 ******* 2021-11-21T21:51:06.996 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-11-21T21:51:07.025 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:07 +0000 (0:00:00.035) 0:02:00.960 ******* 2021-11-21T21:51:07.074 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:07.075 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:07.232 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0Sx3jf7JeuRRbcu0q57J7PeBOVugDQxLt'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.085727", "end": "2021-11-21 21:51:06.969889", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0Sx3jf7JeuRRbcu0q57J7PeBOVugDQxLt"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:05.884162", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:07.234 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF', 'wwn-0x50000399aba81667'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF-part1', 'wwn-0x50000399aba81667-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399aba81667', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I8K38HF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF", "wwn-0x50000399aba81667"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF-part1", "wwn-0x50000399aba81667-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I8K38HF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399aba81667"}}, "skip_reason": "Cond 2021-11-21T21:51:07.236 INFO:teuthology.task.ansible.out:itional result was False"} 2021-11-21T21:51:07.422 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-11-21T21:51:07.424 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:07.448 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:07 +0000 (0:00:00.423) 0:02:01.383 ******* 2021-11-21T21:51:07.475 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-11-21T21:51:07.477 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:07.505 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:07 +0000 (0:00:00.051) 0:02:01.435 ******* 2021-11-21T21:51:07.554 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-21T21:51:07.556 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item={'block_used': 14049687, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045421, 'size_available': 925882044416, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045421, "block_size": 4096, "block_total": 240095108, "block_used": 14049687, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882044416, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-11-21T21:51:07.579 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:07 +0000 (0:00:00.079) 0:02:01.515 ******* 2021-11-21T21:51:07.691 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-11-21T21:51:07.692 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002457", "end": "2021-11-21 21:51:07.891856", "rc": 0, "start": "2021-11-21 21:51:07.889399", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:07.716 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:07 +0000 (0:00:00.136) 0:02:01.651 ******* 2021-11-21T21:51:07.735 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:07.737 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjKkTOGwIHgEMgn4x0DuAybUaenepGDvMN'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.115143", "end": "2021-11-21 21:51:08.100069", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjKkTOGwIHgEMgn4x0DuAybUaenepGDvMN"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:06.984926", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:07.737 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadl 2021-11-21T21:51:07.739 INFO:teuthology.task.ansible.out:ine', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF', 'wwn-0x50000399ab9813d2'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF-part1', 'wwn-0x50000399ab9813d2-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813d2', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2K4F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF", "wwn-0x50000399ab9813d2"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF-part1", "wwn-0x50000399ab9813d2-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2K4F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813d2"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:07.769 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-11-21T21:51:07.771 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'block_used': 14049687, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045421, 'size_available': 925882044416, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045421, "block_size": 4096, "block_total": 240095108, "block_used": 14049687, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882044416, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:07.803 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:07 +0000 (0:00:00.087) 0:02:01.738 ******* 2021-11-21T21:51:07.831 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-11-21T21:51:07.856 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:07 +0000 (0:00:00.053) 0:02:01.792 ******* 2021-11-21T21:51:07.906 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:07.908 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:08.475 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-tOycDn-c6jk-Ns0S-XeET-Nk2w-qXLY-b4JCwY', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200001H375A', 'nvme-nvme.8086-50484b4d393230303030314833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.119211", "end": "2021-11-21 21:51:08.217747", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-tOycDn-c6jk-Ns0S-XeET-Nk2w-qXLY-b4JCwY", "nvme-INTEL_SSDPEL1K375GA_PHKM9200001H375A", "nvme-nvme.8086-50484b4d393230303030314833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:07.098536", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-11-21T21:51:08.477 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:08.487 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:08.668 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtI8Y4cG2l4G30InEbyYq0xXXCPGdQl0d1v'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.490056", "end": "2021-11-21 21:51:08.401442", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtI8Y4cG2l4G30InEbyYq0xXXCPGdQl0d1v"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:06.911386", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:08.680 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:08.681 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:08.980 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-x5VSHB-5qik-RxJs-HlC2-GhNt-hge8-LT3LOH', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400KY375A', 'nvme-nvme.8086-50484b4d3931333430304b5933373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.145463", "end": "2021-11-21 21:51:09.345084", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-x5VSHB-5qik-RxJs-HlC2-GhNt-hge8-LT3LOH", "nvme-INTEL_SSDPEL1K375GA_PHKM913400KY375A", "nvme-nvme.8086-50484b4d3931333430304b5933373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:08.199621", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-11-21T21:51:08.981 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:08.994 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:08.995 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:09.575 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDL7ofSXsCVIyrK9oHWwJDmj0EDxOVjIzNh'], 'uuids': ['68e9ee92-99bf-4bf6-b51d-7b72de446531']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.555589", "end": "2021-11-21 21:51:09.762972", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDL7ofSXsCVIyrK9oHWwJDmj0EDxOVjIzNh"], "labels": [], "masters": [], "uuids": ["68e9ee92-99bf-4bf6-b51d-7b72de446531"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:08.207383", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:09.577 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:09.588 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:09.673 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0M8dV7FfkMgCpRIbsaILLsPQDhuRgvlZU'], 'uuids': ['a2e94586-6a63-4e1c-8d31-2db0bbeb7c49']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.097071", "end": "2021-11-21 21:51:09.421191", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0M8dV7FfkMgCpRIbsaILLsPQDhuRgvlZU"], "labels": [], "masters": [], "uuids": ["a2e94586-6a63-4e1c-8d31-2db0bbeb7c49"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:08.324120", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:09.685 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:09.686 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:09.864 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIik434KYmJdTReNxKytruS5cZVVlQbfOM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.093081", "end": "2021-11-21 21:51:09.600262", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIik434KYmJdTReNxKytruS5cZVVlQbfOM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:08.507181", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:09.876 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF', 'wwn-0x50000399ab9813cb'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1', 'wwn-0x50000399ab9813cb-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813cb', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JXF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF", "wwn-0x50000399ab9813cb"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1", "wwn-0x50000399ab9813cb-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JXF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813cb"}}, "skip_reason": "Cond 2021-11-21T21:51:09.878 INFO:teuthology.task.ansible.out:itional result was False"} 2021-11-21T21:51:10.191 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjl7XsOQ9tMlhJRItPgq3mZhJvAkwaSxGA'], 'uuids': ['98c8bceb-dc89-4ea6-9874-1f5648ea425a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.111412", "end": "2021-11-21 21:51:10.563521", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjl7XsOQ9tMlhJRItPgq3mZhJvAkwaSxGA"], "labels": [], "masters": [], "uuids": ["98c8bceb-dc89-4ea6-9874-1f5648ea425a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:09.452109", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:10.203 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:10.215 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:10.788 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLiTI2VN9wXrChSrjnpJIcrmrY6tPyNl6P'], 'uuids': ['fc761545-f38a-4cb1-afc6-43b44c8a4db6']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.105904", "end": "2021-11-21 21:51:10.976595", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLiTI2VN9wXrChSrjnpJIcrmrY6tPyNl6P"], "labels": [], "masters": [], "uuids": ["fc761545-f38a-4cb1-afc6-43b44c8a4db6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:09.870691", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:10.790 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF', 'wwn-0x50000399ab9813f6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF-part1', 'wwn-0x50000399ab9813f6-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813f6', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2L4F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF", "wwn-0x50000399ab9813f6"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF-part1", "wwn-0x50000399ab9813f6-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2L4F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813f6"}}, "skip_reason": "Cond 2021-11-21T21:51:10.791 INFO:teuthology.task.ansible.out:itional result was False"} 2021-11-21T21:51:10.876 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB02NQojcIOETWoerYJAWJeBzgTFIMltqwL'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.090616", "end": "2021-11-21 21:51:10.620924", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB02NQojcIOETWoerYJAWJeBzgTFIMltqwL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:09.530308", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:10.877 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:11.122 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-DLxZKY-pW76-21Z8-WiOD-Eew2-oNuC-ODGl0u', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A', 'nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.129175", "end": "2021-11-21 21:51:10.861619", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-DLxZKY-pW76-21Z8-WiOD-Eew2-oNuC-ODGl0u", "nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A", "nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:09.732444", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-11-21T21:51:11.124 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:11.135 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:11.148 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:11.415 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCYzI42yQlvUlD2Sk0M9H1mZL1Ra6OROK'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.115836", "end": "2021-11-21 21:51:11.789123", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCYzI42yQlvUlD2Sk0M9H1mZL1Ra6OROK"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:10.673287", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:11.427 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:11.999 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-NFpNyH-P5HQ-wI8j-ursL-XUBa-MwAG-Q3Vkxs', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200000S375A', 'nvme-nvme.8086-50484b4d393230303030305333373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.120731", "end": "2021-11-21 21:51:12.193294", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-NFpNyH-P5HQ-wI8j-ursL-XUBa-MwAG-Q3Vkxs", "nvme-INTEL_SSDPEL1K375GA_PHKM9200000S375A", "nvme-nvme.8086-50484b4d393230303030305333373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:11.072563", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-11-21T21:51:12.001 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:12.011 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.023 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.087 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0r7oVKeAWxeVBDzUUvpkhI2NEyFBscbtN'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.114644", "end": "2021-11-21 21:51:11.829138", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0r7oVKeAWxeVBDzUUvpkhI2NEyFBscbtN"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:10.714494", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:12.133 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:12 +0000 (0:00:04.276) 0:02:06.068 ******* 2021-11-21T21:51:12.188 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:12.189 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.351 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:12.353 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIRPwOOOqMrtGNlcUx4BiEXxKz1tXoTy1E'], 'uuids': ['5db406f1-d12e-4b71-a330-ade9a38bd973']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.118295", "end": "2021-11-21 21:51:12.089024", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIRPwOOOqMrtGNlcUx4BiEXxKz1tXoTy1E"], "labels": [], "masters": [], "uuids": ["5db406f1-d12e-4b71-a330-ade9a38bd973"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:10.970729", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:12.369 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.370 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.382 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:12.384 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0FFyM8b6OgYiGnifpfE5Ec2uqo6HcfJKI'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.077966", "end": "2021-11-21 21:51:12.120879", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0FFyM8b6OgYiGnifpfE5Ec2uqo6HcfJKI"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:12.042913", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0766434 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0766434 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, " 2021-11-21T21:51:12.385 INFO:teuthology.task.ansible.out:item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.565 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0Sx3jf7JeuRRbcu0q57J7PeBOVugDQxLt'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.078846", "end": "2021-11-21 21:51:12.314207", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0Sx3jf7JeuRRbcu0q57J7PeBOVugDQxLt"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:12.235361", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0776226 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0776226 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:12.578 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF', 'wwn-0x50000399aba81667'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF-part1', 'wwn-0x50000399aba81667-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399aba81667', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I8K38HF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF", "wwn-0x50000399aba81667"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38HF6XF-part1", "wwn-0x50000399aba81667-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I8K38HF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399aba81667"}}, "skip_reason": "Cond 2021-11-21T21:51:12.579 INFO:teuthology.task.ansible.out:itional result was False"} 2021-11-21T21:51:12.632 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:12.634 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjENGJqpb2cgeb9jXjBlIeHezACfwECsTr'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.112668", "end": "2021-11-21 21:51:12.997203", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjENGJqpb2cgeb9jXjBlIeHezACfwECsTr"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:11.884535", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:12.669 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:12 +0000 (0:00:00.535) 0:02:06.604 ******* 2021-11-21T21:51:12.709 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:12.711 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-tOycDn-c6jk-Ns0S-XeET-Nk2w-qXLY-b4JCwY', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200001H375A', 'nvme-nvme.8086-50484b4d393230303030314833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.025431", "end": "2021-11-21 21:51:12.457232", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-tOycDn-c6jk-Ns0S-XeET-Nk2w-qXLY-b4JCwY", "nvme-INTEL_SSDPEL1K375GA_PHKM9200001H375A", "nvme-nvme.8086-50484b4d393230303030314833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:12.431801", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0241796 s, 4.8 GB/s", "stderr_lines": ["1 2021-11-21T21:51:12.713 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0241796 s, 4.8 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:12.722 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.724 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:12.894 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.054 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCmdlvul08UrhqUq0WvUWhZhVie9WLHDM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.078014", "end": "2021-11-21 21:51:13.430742", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCmdlvul08UrhqUq0WvUWhZhVie9WLHDM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:13.352728", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0767111 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0767111 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.068 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.070 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.084 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0M8dV7FfkMgCpRIbsaILLsPQDhuRgvlZU'], 'uuids': ['a2e94586-6a63-4e1c-8d31-2db0bbeb7c49']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.078536", "end": "2021-11-21 21:51:12.833377", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0M8dV7FfkMgCpRIbsaILLsPQDhuRgvlZU"], "labels": [], "masters": [], "uuids": ["a2e94586-6a63-4e1c-8d31-2db0bbeb7c49"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:12.754841", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0772776 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0772776 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.097 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.100 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.206 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:13.208 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLhOC9sDVxXlx1f4Odaoe2EiuMBchAQG8u'], 'uuids': ['1868690b-a862-42d8-89ed-f639e19841d1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.093240", "end": "2021-11-21 21:51:13.392998", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLhOC9sDVxXlx1f4Odaoe2EiuMBchAQG8u"], "labels": [], "masters": [], "uuids": ["1868690b-a862-42d8-89ed-f639e19841d1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:12.299758", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:13.219 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.220 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.243 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:13.244 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjKkTOGwIHgEMgn4x0DuAybUaenepGDvMN'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.078052", "end": "2021-11-21 21:51:13.614440", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjKkTOGwIHgEMgn4x0DuAybUaenepGDvMN"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:13.536388", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0768184 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0768184 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.246 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF', 'wwn-0x50000399ab9813d2'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF-part1', 'wwn-0x50000399ab9813d2-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813d2', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2K4F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF", "wwn-0x50000399ab9813d2"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K4F6XF-part1", "wwn-0x50000399ab9813d2-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2K4F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813d2"}}, "skip_reason": "Cond 2021-11-21T21:51:13.247 INFO:teuthology.task.ansible.out:itional result was False"} 2021-11-21T21:51:13.289 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB02NQojcIOETWoerYJAWJeBzgTFIMltqwL'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.079149", "end": "2021-11-21 21:51:13.036509", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB02NQojcIOETWoerYJAWJeBzgTFIMltqwL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:12.957360", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0778927 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0778927 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.301 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.419 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-x5VSHB-5qik-RxJs-HlC2-GhNt-hge8-LT3LOH', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400KY375A', 'nvme-nvme.8086-50484b4d3931333430304b5933373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.025894", "end": "2021-11-21 21:51:13.783616", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-x5VSHB-5qik-RxJs-HlC2-GhNt-hge8-LT3LOH", "nvme-INTEL_SSDPEL1K375GA_PHKM913400KY375A", "nvme-nvme.8086-50484b4d3931333430304b5933373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:13.757722", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0246717 s, 4.7 GB/s", "stderr_lines": ["1 2021-11-21T21:51:13.421 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0246717 s, 4.7 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.432 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.486 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0r7oVKeAWxeVBDzUUvpkhI2NEyFBscbtN'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.078754", "end": "2021-11-21 21:51:13.231812", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-BlUg1l5VQCu3h6HuqiGNOaejgOEvmJB0r7oVKeAWxeVBDzUUvpkhI2NEyFBscbtN"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:13.153058", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0775524 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0775524 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.523 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:13 +0000 (0:00:00.854) 0:02:07.458 ******* 2021-11-21T21:51:13.566 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:13.568 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIACPNU6iAZd3z1H3eFshEZijMwWqhWAEc'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.107944", "end": "2021-11-21 21:51:13.301727", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIACPNU6iAZd3z1H3eFshEZijMwWqhWAEc"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:12.193783", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:13.580 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.603 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:13.605 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjl7XsOQ9tMlhJRItPgq3mZhJvAkwaSxGA'], 'uuids': ['98c8bceb-dc89-4ea6-9874-1f5648ea425a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.078475", "end": "2021-11-21 21:51:13.976226", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjl7XsOQ9tMlhJRItPgq3mZhJvAkwaSxGA"], "labels": [], "masters": [], "uuids": ["98c8bceb-dc89-4ea6-9874-1f5648ea425a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:13.897751", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0772228 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0772228 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.627 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.638 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.853 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCYzI42yQlvUlD2Sk0M9H1mZL1Ra6OROK'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.078755", "end": "2021-11-21 21:51:14.226223", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjCYzI42yQlvUlD2Sk0M9H1mZL1Ra6OROK"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:14.147468", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0775389 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0775389 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.864 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:13.865 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [gibba016.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.209332", "end": "2021-11-21 21:51:13.597499", "rc": 0, "start": "2021-11-21 21:51:13.388167", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:13.889 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:13 +0000 (0:00:00.366) 0:02:07.825 ******* 2021-11-21T21:51:14.028 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:14.030 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjENGJqpb2cgeb9jXjBlIeHezACfwECsTr'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.078684", "end": "2021-11-21 21:51:14.400548", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Ym4agObcfyCvc2238vh9MeGXTSZvBhfjENGJqpb2cgeb9jXjBlIeHezACfwECsTr"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:14.321864", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0774561 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0774561 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:14.068 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.178) 0:02:08.004 ******* 2021-11-21T21:51:14.087 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-11-21T21:51:14.089 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.090400", "end": "2021-11-21 21:51:13.837516", "rc": 0, "start": "2021-11-21 21:51:13.747116", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:14.114 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.045) 0:02:08.050 ******* 2021-11-21T21:51:14.144 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-11-21T21:51:14.173 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.059) 0:02:08.109 ******* 2021-11-21T21:51:14.202 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-11-21T21:51:14.204 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.233 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.059) 0:02:08.168 ******* 2021-11-21T21:51:14.262 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-11-21T21:51:14.264 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.292 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.059) 0:02:08.228 ******* 2021-11-21T21:51:14.321 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-21T21:51:14.323 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.350 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.057) 0:02:08.286 ******* 2021-11-21T21:51:14.358 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-11-21T21:51:14.360 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.157367", "end": "2021-11-21 21:51:14.708872", "rc": 0, "start": "2021-11-21 21:51:14.551505", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:14.386 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.036) 0:02:08.322 ******* 2021-11-21T21:51:14.394 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-11-21T21:51:14.396 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.423 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.036) 0:02:08.359 ******* 2021-11-21T21:51:14.432 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:14.433 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLFWUw3flsUlfrVjXnyCqZCoAZYGE6rcE9'], 'uuids': ['ddfac024-3a7f-40ee-b0a3-d887ec35a31a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.100463", "end": "2021-11-21 21:51:14.609994", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLFWUw3flsUlfrVjXnyCqZCoAZYGE6rcE9"], "labels": [], "masters": [], "uuids": ["ddfac024-3a7f-40ee-b0a3-d887ec35a31a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:13.509531", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:14.435 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.531 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-11-21T21:51:14.533 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:51:14.558 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.134) 0:02:08.493 ******* 2021-11-21T21:51:14.576 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-11-21T21:51:14.578 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.073763", "end": "2021-11-21 21:51:14.944480", "rc": 0, "start": "2021-11-21 21:51:14.870717", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:14.604 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.046) 0:02:08.540 ******* 2021-11-21T21:51:14.613 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-21T21:51:14.615 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'block_used': 14049675, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045433, 'size_available': 925882093568, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187090, 'block_size': 4096, 'inode_available': 60867886}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045433, "block_size": 4096, "block_total": 240095108, "block_used": 14049675, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867886, "inode_total": 61054976, "inode_used": 187090, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882093568, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.639 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.035) 0:02:08.575 ******* 2021-11-21T21:51:14.647 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-11-21T21:51:14.676 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.036) 0:02:08.611 ******* 2021-11-21T21:51:14.683 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-11-21T21:51:14.685 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.709 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.033) 0:02:08.644 ******* 2021-11-21T21:51:14.716 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-11-21T21:51:14.718 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.746 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.036) 0:02:08.681 ******* 2021-11-21T21:51:14.753 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:14.755 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.779 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.033) 0:02:08.715 ******* 2021-11-21T21:51:14.796 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:14.797 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtINkeUnpxi0SLPIKjaYNaOBFDSzc0cxamm'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.090583", "end": "2021-11-21 21:51:14.509731", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtINkeUnpxi0SLPIKjaYNaOBFDSzc0cxamm"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:13.419148", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:14.799 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:14.804 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:14.983 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.204) 0:02:08.919 ******* 2021-11-21T21:51:15.006 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:14 +0000 (0:00:00.022) 0:02:08.941 ******* 2021-11-21T21:51:15.014 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:15.016 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.043 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.035) 0:02:08.977 ******* 2021-11-21T21:51:15.050 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-21T21:51:15.051 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.079 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.037) 0:02:09.015 ******* 2021-11-21T21:51:15.086 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:15.088 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Create volume_groups dict] ************************************ skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.114 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.034) 0:02:09.049 ******* 2021-11-21T21:51:15.121 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-11-21T21:51:15.123 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.148 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.034) 0:02:09.084 ******* 2021-11-21T21:51:15.154 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-11-21T21:51:15.156 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.182 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.033) 0:02:09.117 ******* 2021-11-21T21:51:15.221 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-11-21T21:51:15.222 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.245 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.063) 0:02:09.181 ******* 2021-11-21T21:51:15.253 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:15.254 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtI8Y4cG2l4G30InEbyYq0xXXCPGdQl0d1v'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.077864", "end": "2021-11-21 21:51:14.971080", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtI8Y4cG2l4G30InEbyYq0xXXCPGdQl0d1v"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:14.893216", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0765782 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0765782 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:15.255 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.257 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-11-21T21:51:15.257 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:51:15.283 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.038) 0:02:09.219 ******* 2021-11-21T21:51:15.333 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-21T21:51:15.335 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'block_used': 14049682, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045426, 'size_available': 925882064896, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045426, "block_size": 4096, "block_total": 240095108, "block_used": 14049682, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882064896, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.358 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.074) 0:02:09.293 ******* 2021-11-21T21:51:15.396 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-11-21T21:51:15.398 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.420 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.062) 0:02:09.356 ******* 2021-11-21T21:51:15.429 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:15.429 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIik434KYmJdTReNxKytruS5cZVVlQbfOM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.078749", "end": "2021-11-21 21:51:15.161101", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIik434KYmJdTReNxKytruS5cZVVlQbfOM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:15.082352", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0775231 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0775231 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:15.431 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF', 'wwn-0x50000399ab9813cb'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1', 'wwn-0x50000399ab9813cb-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813cb', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JXF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF", "wwn-0x50000399ab9813cb"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1", "wwn-0x50000399ab9813cb-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JXF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813cb"}}, "skip_reason": "Cond 2021-11-21T21:51:15.433 INFO:teuthology.task.ansible.out:itional result was False"} 2021-11-21T21:51:15.454 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.479 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.058) 0:02:09.414 ******* 2021-11-21T21:51:15.517 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:15.519 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.547 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.068) 0:02:09.483 ******* 2021-11-21T21:51:15.555 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:15.556 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-DLxZKY-pW76-21Z8-WiOD-Eew2-oNuC-ODGl0u', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A', 'nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.025977", "end": "2021-11-21 21:51:15.282133", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-DLxZKY-pW76-21Z8-WiOD-Eew2-oNuC-ODGl0u", "nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A", "nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:15.256156", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0247232 s, 4.7 GB/s", "stderr_lines": ["1 2021-11-21T21:51:15.558 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0247232 s, 4.7 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.559 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.582 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.608 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.060) 0:02:09.543 ******* 2021-11-21T21:51:15.625 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-21T21:51:15.627 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLkIw5H4FvxffEuWWGDRyEeUMvtoxqO50e'], 'uuids': ['1bc86c78-2413-49b7-80c4-3d878565580d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.100647", "end": "2021-11-21 21:51:15.810897", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLkIw5H4FvxffEuWWGDRyEeUMvtoxqO50e"], "labels": [], "masters": [], "uuids": ["1bc86c78-2413-49b7-80c4-3d878565580d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:14.710250", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-21T21:51:15.638 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-11-21T21:51:15.640 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.663 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.054) 0:02:09.598 ******* 2021-11-21T21:51:15.688 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.025) 0:02:09.623 ******* 2021-11-21T21:51:15.707 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-11-21T21:51:15.709 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.735 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.046) 0:02:09.670 ******* 2021-11-21T21:51:15.743 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:15.744 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIRPwOOOqMrtGNlcUx4BiEXxKz1tXoTy1E'], 'uuids': ['5db406f1-d12e-4b71-a330-ade9a38bd973']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.078997", "end": "2021-11-21 21:51:15.469390", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIRPwOOOqMrtGNlcUx4BiEXxKz1tXoTy1E"], "labels": [], "masters": [], "uuids": ["5db406f1-d12e-4b71-a330-ade9a38bd973"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:15.390393", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0777862 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0777862 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:15.746 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.758 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.759 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.866 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-11-21T21:51:15.893 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:15 +0000 (0:00:00.158) 0:02:09.829 ******* 2021-11-21T21:51:15.923 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:15.924 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDL7ofSXsCVIyrK9oHWwJDmj0EDxOVjIzNh'], 'uuids': ['68e9ee92-99bf-4bf6-b51d-7b72de446531']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.078890", "end": "2021-11-21 21:51:16.116121", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDL7ofSXsCVIyrK9oHWwJDmj0EDxOVjIzNh"], "labels": [], "masters": [], "uuids": ["68e9ee92-99bf-4bf6-b51d-7b72de446531"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:16.037231", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0776228 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0776228 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:15.926 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIACPNU6iAZd3z1H3eFshEZijMwWqhWAEc'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.078305", "end": "2021-11-21 21:51:15.671612", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtIACPNU6iAZd3z1H3eFshEZijMwWqhWAEc"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:15.593307", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0770794 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0770794 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:15.938 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.939 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:15.940 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.114 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtINkeUnpxi0SLPIKjaYNaOBFDSzc0cxamm'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.079134", "end": "2021-11-21 21:51:15.856436", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Vmbi5TyrUrIpWdsEUJ3xX2tdmbOOwrtINkeUnpxi0SLPIKjaYNaOBFDSzc0cxamm"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:15.777302", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0779238 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0779238 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:16.131 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLiTI2VN9wXrChSrjnpJIcrmrY6tPyNl6P'], 'uuids': ['fc761545-f38a-4cb1-afc6-43b44c8a4db6']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.078845", "end": "2021-11-21 21:51:16.317697", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLiTI2VN9wXrChSrjnpJIcrmrY6tPyNl6P"], "labels": [], "masters": [], "uuids": ["fc761545-f38a-4cb1-afc6-43b44c8a4db6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:16.238852", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0775473 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0775473 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:16.156 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.262) 0:02:10.092 ******* 2021-11-21T21:51:16.165 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF', 'wwn-0x50000399ab9813f6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF-part1', 'wwn-0x50000399ab9813f6-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813f6', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2L4F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF", "wwn-0x50000399ab9813f6"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2L4F6XF-part1", "wwn-0x50000399ab9813f6-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2L4F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813f6"}}, "skip_reason": "Cond 2021-11-21T21:51:16.167 INFO:teuthology.task.ansible.out:itional result was False"} 2021-11-21T21:51:16.252 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-NFpNyH-P5HQ-wI8j-ursL-XUBa-MwAG-Q3Vkxs', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200000S375A', 'nvme-nvme.8086-50484b4d393230303030305333373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.026414", "end": "2021-11-21 21:51:16.444828", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-NFpNyH-P5HQ-wI8j-ursL-XUBa-MwAG-Q3Vkxs", "nvme-INTEL_SSDPEL1K375GA_PHKM9200000S375A", "nvme-nvme.8086-50484b4d393230303030305333373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:16.418414", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0251862 s, 4.6 GB/s", "stderr_lines": ["1 2021-11-21T21:51:16.254 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0251862 s, 4.6 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.266 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.288 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-11-21T21:51:16.314 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.158) 0:02:10.250 ******* 2021-11-21T21:51:16.430 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:16.432 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLhOC9sDVxXlx1f4Odaoe2EiuMBchAQG8u'], 'uuids': ['1868690b-a862-42d8-89ed-f639e19841d1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.079143", "end": "2021-11-21 21:51:16.623678", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLhOC9sDVxXlx1f4Odaoe2EiuMBchAQG8u"], "labels": [], "masters": [], "uuids": ["1868690b-a862-42d8-89ed-f639e19841d1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:16.544535", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.077755 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.077755 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:16.445 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.446 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.468 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-11-21T21:51:16.469 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.177761", "end": "2021-11-21 21:51:16.185419", "rc": 0, "start": "2021-11-21 21:51:16.007658", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:16.496 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.180) 0:02:10.431 ******* 2021-11-21T21:51:16.663 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:16.665 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLFWUw3flsUlfrVjXnyCqZCoAZYGE6rcE9'], 'uuids': ['ddfac024-3a7f-40ee-b0a3-d887ec35a31a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.078983", "end": "2021-11-21 21:51:16.854525", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLFWUw3flsUlfrVjXnyCqZCoAZYGE6rcE9"], "labels": [], "masters": [], "uuids": ["ddfac024-3a7f-40ee-b0a3-d887ec35a31a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:16.775542", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0777419 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0777419 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:16.666 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 2021-11-21T21:51:16.667 INFO:teuthology.task.ansible.out:'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.677 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-11-21T21:51:16.679 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.068243", "end": "2021-11-21 21:51:16.421133", "rc": 0, "start": "2021-11-21 21:51:16.352890", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:16.703 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.207) 0:02:10.639 ******* 2021-11-21T21:51:16.723 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:16.725 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:16.735 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-11-21T21:51:16.763 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.059) 0:02:10.698 ******* 2021-11-21T21:51:16.781 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:16.784 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:16.794 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.822 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.058) 0:02:10.757 ******* 2021-11-21T21:51:16.850 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-11-21T21:51:16.851 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.878 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.056) 0:02:10.813 ******* 2021-11-21T21:51:16.886 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-21T21:51:16.887 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLkIw5H4FvxffEuWWGDRyEeUMvtoxqO50e'], 'uuids': ['1bc86c78-2413-49b7-80c4-3d878565580d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.078187", "end": "2021-11-21 21:51:17.069077", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-h4e62fuZvCCQ2uM1QNkuGxJmQLnm0CDLkIw5H4FvxffEuWWGDRyEeUMvtoxqO50e"], "labels": [], "masters": [], "uuids": ["1bc86c78-2413-49b7-80c4-3d878565580d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-21 21:51:16.990890", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0769377 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0769377 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:16.926 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.048) 0:02:10.862 ******* 2021-11-21T21:51:16.934 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-21T21:51:16.935 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:16.975 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:16 +0000 (0:00:00.049) 0:02:10.911 ******* 2021-11-21T21:51:17.006 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-11-21T21:51:17.008 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.042 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.067) 0:02:10.978 ******* 2021-11-21T21:51:17.094 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:17.096 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:17.107 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:17.151 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-11-21T21:51:17.153 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:51:17.176 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.133) 0:02:11.112 ******* 2021-11-21T21:51:17.227 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-21T21:51:17.228 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'block_used': 14049683, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045425, 'size_available': 925882060800, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045425, "block_size": 4096, "block_total": 240095108, "block_used": 14049683, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882060800, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.252 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.076) 0:02:11.188 ******* 2021-11-21T21:51:17.271 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-11-21T21:51:17.273 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.206901", "end": "2021-11-21 21:51:17.445146", "rc": 0, "start": "2021-11-21 21:51:17.238245", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:17.299 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.046) 0:02:11.234 ******* 2021-11-21T21:51:17.306 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-11-21T21:51:17.308 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.334 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.035) 0:02:11.269 ******* 2021-11-21T21:51:17.364 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:17.366 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.393 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.059) 0:02:11.329 ******* 2021-11-21T21:51:17.424 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:17.426 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : set_fact] ***************************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.450 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.057) 0:02:11.386 ******* 2021-11-21T21:51:17.468 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:17.470 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:17.481 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-11-21T21:51:17.483 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.509 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.058) 0:02:11.444 ******* 2021-11-21T21:51:17.517 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-11-21T21:51:17.519 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.087300", "end": "2021-11-21 21:51:17.696950", "rc": 0, "start": "2021-11-21 21:51:17.609650", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:17.545 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.036) 0:02:11.480 ******* 2021-11-21T21:51:17.564 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-11-21T21:51:17.566 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.593 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.048) 0:02:11.529 ******* 2021-11-21T21:51:17.601 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-11-21T21:51:17.630 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.037) 0:02:11.566 ******* 2021-11-21T21:51:17.638 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-11-21T21:51:17.640 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.665 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.034) 0:02:11.600 ******* 2021-11-21T21:51:17.672 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-11-21T21:51:17.674 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.713 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.047) 0:02:11.648 ******* 2021-11-21T21:51:17.741 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-11-21T21:51:17.744 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.768 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.055) 0:02:11.703 ******* 2021-11-21T21:51:17.797 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-21T21:51:17.799 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.830 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.061) 0:02:11.765 ******* 2021-11-21T21:51:17.842 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:17.866 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.036) 0:02:11.801 ******* 2021-11-21T21:51:17.876 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:17.877 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:17.908 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:17 +0000 (0:00:00.041) 0:02:11.843 ******* 2021-11-21T21:51:17.981 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:17.983 INFO:teuthology.task.ansible.out:TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-11-21T21:51:18.010 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.101) 0:02:11.945 ******* 2021-11-21T21:51:18.028 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-11-21T21:51:18.030 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-21T21:51:18.056 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.046) 0:02:11.992 ******* 2021-11-21T21:51:18.106 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [gibba036.front.sepia.ceph.com] => (item={'block_used': 14049687, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226045421, 'size_available': 925882044416, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 187096, 'block_size': 4096, 'inode_available': 60867880}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226045421, "block_size": 4096, "block_total": 240095108, "block_used": 14049687, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60867880, "inode_total": 61054976, "inode_used": 187096, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 925882044416, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.131 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.074) 0:02:12.066 ******* 2021-11-21T21:51:18.133 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:18.160 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-11-21T21:51:18.162 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.186 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.054) 0:02:12.121 ******* 2021-11-21T21:51:18.192 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:18.194 INFO:teuthology.task.ansible.out:TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : Create volume group(s)] *************************************** 2021-11-21T21:51:18.195 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-11-21T21:51:18.224 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.038) 0:02:12.160 ******* 2021-11-21T21:51:18.232 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.260 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.036) 0:02:12.196 ******* 2021-11-21T21:51:18.270 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:18.272 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:18.274 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:18.301 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.040) 0:02:12.237 ******* 2021-11-21T21:51:18.310 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:18.312 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.348 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.047) 0:02:12.284 ******* 2021-11-21T21:51:18.358 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:18.360 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:18.383 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.406 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.058) 0:02:12.342 ******* 2021-11-21T21:51:18.425 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-21T21:51:18.426 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-11-21T21:51:18.449 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.042) 0:02:12.385 ******* 2021-11-21T21:51:18.456 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-11-21T21:51:18.457 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.484 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.034) 0:02:12.419 ******* 2021-11-21T21:51:18.491 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:18.494 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} skipping: [gibba016.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.525 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.040) 0:02:12.460 ******* 2021-11-21T21:51:18.533 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-11-21T21:51:18.535 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.560 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.035) 0:02:12.495 ******* 2021-11-21T21:51:18.570 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-11-21T21:51:18.572 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:18.601 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.040) 0:02:12.536 ******* 2021-11-21T21:51:18.642 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-11-21T21:51:18.645 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:18.645 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:18.668 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.067) 0:02:12.603 ******* 2021-11-21T21:51:18.675 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:18.676 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:18.742 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:18.786 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-11-21T21:51:18.787 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:18.812 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.144) 0:02:12.747 ******* 2021-11-21T21:51:18.821 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:18.833 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.862 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.050) 0:02:12.798 ******* 2021-11-21T21:51:18.871 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-11-21T21:51:18.873 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-11-21T21:51:18.902 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.039) 0:02:12.838 ******* 2021-11-21T21:51:18.909 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-11-21T21:51:18.911 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:18.944 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.041) 0:02:12.880 ******* 2021-11-21T21:51:18.963 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:18.965 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:18.975 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-11-21T21:51:18.977 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.000 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:18 +0000 (0:00:00.056) 0:02:12.936 ******* 2021-11-21T21:51:19.104 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-11-21T21:51:19.106 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-11-21T21:51:19.131 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.130) 0:02:13.067 ******* 2021-11-21T21:51:19.140 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-11-21T21:51:19.141 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:19.193 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.062) 0:02:13.129 ******* 2021-11-21T21:51:19.201 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-11-21T21:51:19.203 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true} 2021-11-21T21:51:19.233 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.039) 0:02:13.169 ******* 2021-11-21T21:51:19.243 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-11-21T21:51:19.244 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-11-21T21:51:19.273 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.040) 0:02:13.209 ******* 2021-11-21T21:51:19.336 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:19.338 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:19.423 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [gibba017.front.sepia.ceph.com] => {"changed": true} 2021-11-21T21:51:19.452 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.178) 0:02:13.388 ******* 2021-11-21T21:51:19.482 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-11-21T21:51:19.515 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.062) 0:02:13.450 ******* 2021-11-21T21:51:19.545 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.547 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:19.572 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.057) 0:02:13.508 ******* 2021-11-21T21:51:19.608 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-11-21T21:51:19.610 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.642 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.069) 0:02:13.577 ******* 2021-11-21T21:51:19.677 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.035) 0:02:13.612 ******* 2021-11-21T21:51:19.686 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:19.688 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Restart cachefilesd] ****************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.713 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.036) 0:02:13.649 ******* 2021-11-21T21:51:19.721 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-11-21T21:51:19.723 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.757 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.043) 0:02:13.693 ******* 2021-11-21T21:51:19.767 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-21T21:51:19.769 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.798 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.040) 0:02:13.733 ******* 2021-11-21T21:51:19.805 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-11-21T21:51:19.807 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.845 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.047) 0:02:13.781 ******* 2021-11-21T21:51:19.875 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-21T21:51:19.876 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.902 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.056) 0:02:13.837 ******* 2021-11-21T21:51:19.921 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:19.923 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:19.933 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-21T21:51:19.934 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:19.960 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:19 +0000 (0:00:00.058) 0:02:13.896 ******* 2021-11-21T21:51:19.979 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-11-21T21:51:19.981 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:20.010 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.049) 0:02:13.945 ******* 2021-11-21T21:51:20.041 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:20.043 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:20.085 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.075) 0:02:14.021 ******* 2021-11-21T21:51:20.094 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-11-21T21:51:20.096 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:20.128 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.042) 0:02:14.063 ******* 2021-11-21T21:51:20.137 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-11-21T21:51:20.139 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:20.164 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.036) 0:02:14.099 ******* 2021-11-21T21:51:20.203 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-21T21:51:20.205 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-11-21T21:51:20.229 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.065) 0:02:14.165 ******* 2021-11-21T21:51:20.247 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-11-21T21:51:20.249 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:20.273 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.044) 0:02:14.209 ******* 2021-11-21T21:51:20.281 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:20.283 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:20.368 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:20.370 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:20.422 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-11-21T21:51:20.424 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-11-21T21:51:20.455 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.181) 0:02:14.391 ******* 2021-11-21T21:51:20.464 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:20.465 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:20.552 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:20.564 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-11-21T21:51:20.566 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-11-21T21:51:20.598 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.143) 0:02:14.534 ******* 2021-11-21T21:51:20.650 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:20.652 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:20.663 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:20.665 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:20.678 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:20.680 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:20.709 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.110) 0:02:14.644 ******* 2021-11-21T21:51:20.749 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-11-21T21:51:20.751 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:20.775 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.066) 0:02:14.711 ******* 2021-11-21T21:51:20.783 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-11-21T21:51:20.784 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-11-21T21:51:20.811 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.035) 0:02:14.746 ******* 2021-11-21T21:51:20.818 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-11-21T21:51:20.820 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:20.845 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.034) 0:02:14.781 ******* 2021-11-21T21:51:20.928 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-11-21T21:51:20.930 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-11-21T21:51:20.957 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.110) 0:02:14.892 ******* 2021-11-21T21:51:20.963 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-11-21T21:51:20.964 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:20.993 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:20 +0000 (0:00:00.036) 0:02:14.929 ******* 2021-11-21T21:51:21.033 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-21T21:51:21.035 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-21T21:51:21.037 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-11-21T21:51:21.061 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.068) 0:02:14.997 ******* 2021-11-21T21:51:21.244 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.026) 0:02:15.023 ******* TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-11-21T21:51:21.263 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.174) 0:02:15.198 ******* 2021-11-21T21:51:21.272 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-11-21T21:51:21.274 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [gibba019.front.sepia.ceph.com] => {"changed": true} 2021-11-21T21:51:21.302 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.039) 0:02:15.237 ******* 2021-11-21T21:51:21.327 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.024) 0:02:15.262 ******* 2021-11-21T21:51:21.388 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:21.390 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:21.401 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-11-21T21:51:21.403 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:21.436 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.109) 0:02:15.371 ******* 2021-11-21T21:51:21.476 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:21.478 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-21T21:51:21.489 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-11-21T21:51:21.522 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.086) 0:02:15.458 ******* 2021-11-21T21:51:21.542 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-11-21T21:51:21.544 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.130702", "end": "2021-11-21 21:51:21.296692", "rc": 0, "start": "2021-11-21 21:51:21.165990", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-11-21T21:51:21.569 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.046) 0:02:15.504 ******* 2021-11-21T21:51:21.578 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:21.580 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : Install cachefilesd] ****************************************** 2021-11-21T21:51:21.581 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.607 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.038) 0:02:15.543 ******* 2021-11-21T21:51:21.621 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.650 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.041) 0:02:15.584 ******* 2021-11-21T21:51:21.656 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-11-21T21:51:21.658 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.682 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.033) 0:02:15.617 ******* 2021-11-21T21:51:21.690 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-21T21:51:21.691 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} skipping: [gibba036.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.695 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.697 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.125352", "end": "2021-11-21 21:51:22.058274", "rc": 0, "start": "2021-11-21 21:51:21.932922", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-11-21T21:51:21.722 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.039) 0:02:15.657 ******* 2021-11-21T21:51:21.746 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.024) 0:02:15.682 ******* 2021-11-21T21:51:21.775 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.026) 0:02:15.708 ******* 2021-11-21T21:51:21.780 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-21T21:51:21.782 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.808 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.035) 0:02:15.743 ******* 2021-11-21T21:51:21.816 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-11-21T21:51:21.818 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create filesystems] ******************************************* skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.843 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.034) 0:02:15.778 ******* 2021-11-21T21:51:21.873 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.029) 0:02:15.808 ******* 2021-11-21T21:51:21.881 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-21T21:51:21.882 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.907 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.034) 0:02:15.843 ******* 2021-11-21T21:51:21.932 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.024) 0:02:15.868 ******* 2021-11-21T21:51:21.940 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-11-21T21:51:21.941 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:21.969 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:21 +0000 (0:00:00.036) 0:02:15.904 ******* 2021-11-21T21:51:22.052 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-11-21T21:51:22.054 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:22.081 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.111) 0:02:16.016 ******* 2021-11-21T21:51:22.088 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-11-21T21:51:22.090 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:22.116 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.035) 0:02:16.052 ******* 2021-11-21T21:51:22.145 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-11-21T21:51:22.146 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:22.171 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.054) 0:02:16.106 ******* 2021-11-21T21:51:22.180 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-11-21T21:51:22.181 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-11-21T21:51:22.209 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.037) 0:02:16.144 ******* 2021-11-21T21:51:22.224 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-11-21T21:51:22.226 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add CPAN config directory for the root user.] ***************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:22.254 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.045) 0:02:16.190 ******* 2021-11-21T21:51:22.281 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.026) 0:02:16.217 ******* 2021-11-21T21:51:22.341 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-11-21T21:51:22.343 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:22.369 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.087) 0:02:16.305 ******* 2021-11-21T21:51:22.397 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-11-21T21:51:22.399 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true} TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:22.422 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.052) 0:02:16.357 ******* 2021-11-21T21:51:22.455 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.033) 0:02:16.391 ******* 2021-11-21T21:51:22.552 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-11-21T21:51:22.555 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-11-21T21:51:22.589 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.133) 0:02:16.524 ******* 2021-11-21T21:51:22.597 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-11-21T21:51:22.599 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-11-21T21:51:22.626 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.036) 0:02:16.561 ******* 2021-11-21T21:51:22.634 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-11-21T21:51:22.636 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:22.661 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.035) 0:02:16.597 ******* 2021-11-21T21:51:22.669 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-11-21T21:51:22.670 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-11-21T21:51:22.700 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.038) 0:02:16.636 ******* 2021-11-21T21:51:22.707 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-11-21T21:51:22.709 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-11-21T21:51:22.736 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.035) 0:02:16.671 ******* 2021-11-21T21:51:22.743 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-11-21T21:51:22.744 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:22.771 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.035) 0:02:16.707 ******* 2021-11-21T21:51:22.778 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-11-21T21:51:22.780 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:22.808 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.032) 0:02:16.740 ******* 2021-11-21T21:51:22.812 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:22.839 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.034) 0:02:16.775 ******* 2021-11-21T21:51:22.858 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-11-21T21:51:22.859 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ActiveEnterTimestampMonotonic": "12314874", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service network.target systemd-journald.socket basic.target sysinit.target -.mount system.slice", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:46 UTC", "AssertTimestampMonotonic": "12170751", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ConditionTimestampMonotonic": "12170751", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": 2021-11-21T21:51:22.861 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "950", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ExecMainStartTimestampMonotonic": "12314865", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:46 UTC", "InactiveExitTimestampMonotonic": "12171596", "InvocationID": "a747afd441de4e0e881392363a6fc813", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-11-21T21:51:22.862 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "950", "MemoryAccounting": "yes", "MemoryCurrent": "3534848", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "Requir 2021-11-21T21:51:22.864 INFO:teuthology.task.ansible.out:esMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:46 UTC", "StateChangeTimestampMonotonic": "12314874", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:22.888 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.048) 0:02:16.824 ******* 2021-11-21T21:51:22.896 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-21T21:51:22.898 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:22.924 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.035) 0:02:16.859 ******* 2021-11-21T21:51:22.930 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:22.932 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-11-21T21:51:22.963 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:22 +0000 (0:00:00.039) 0:02:16.899 ******* 2021-11-21T21:51:22.973 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-11-21T21:51:22.975 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-21T21:51:22.976 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:22.980 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-11-21T21:51:22.983 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ActiveEnterTimestampMonotonic": "13398936", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount system.slice sysinit.target network.target basic.target systemd-journald.socket systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:46 UTC", "AssertTimestampMonotonic": "13311448", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ConditionTimestampMonotonic": "13311446", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": 2021-11-21T21:51:22.985 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "946", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ExecMainStartTimestampMonotonic": "13398926", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:46 UTC", "InactiveExitTimestampMonotonic": "13312460", "InvocationID": "5097dcf08e7a4e4d9148242c5c1eb783", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-11-21T21:51:22.987 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "946", "MemoryAccounting": "yes", "MemoryCurrent": "4100096", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "Requir 2021-11-21T21:51:22.989 INFO:teuthology.task.ansible.out:esMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:46 UTC", "StateChangeTimestampMonotonic": "13398936", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:23.011 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.047) 0:02:16.947 ******* 2021-11-21T21:51:23.037 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.026) 0:02:16.973 ******* 2021-11-21T21:51:23.063 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.024) 0:02:16.998 ******* 2021-11-21T21:51:23.072 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:23.073 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} TASK [testnode : set_fact] ***************************************************** ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-11-21T21:51:23.107 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.040) 0:02:17.038 ******* 2021-11-21T21:51:23.130 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.027) 0:02:17.065 ******* 2021-11-21T21:51:23.151 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-11-21T21:51:23.153 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:23.179 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.048) 0:02:17.114 ******* 2021-11-21T21:51:23.187 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:23.191 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-11-21T21:51:23.215 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.035) 0:02:17.150 ******* 2021-11-21T21:51:23.223 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-11-21T21:51:23.224 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:23.252 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.036) 0:02:17.187 ******* 2021-11-21T21:51:23.258 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-11-21T21:51:23.259 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:23.287 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.035) 0:02:17.222 ******* 2021-11-21T21:51:23.404 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-11-21T21:51:23.406 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:23.433 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.145) 0:02:17.368 ******* 2021-11-21T21:51:23.474 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-11-21T21:51:23.476 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:23.501 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.068) 0:02:17.437 ******* 2021-11-21T21:51:23.763 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:23.764 INFO:teuthology.task.ansible.out:TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-11-21T21:51:23.765 INFO:teuthology.task.ansible.out: ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-11-21T21:51:23.792 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.290) 0:02:17.728 ******* 2021-11-21T21:51:23.803 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-11-21T21:51:23.805 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.152250", "end": "2021-11-21 21:51:23.513183", "rc": 0, "start": "2021-11-21 21:51:23.360933", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-11-21T21:51:23.837 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.045) 0:02:17.773 ******* 2021-11-21T21:51:23.876 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-11-21T21:51:23.878 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:23.901 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.063) 0:02:17.837 ******* 2021-11-21T21:51:23.930 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-11-21T21:51:23.932 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:23.969 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:23 +0000 (0:00:00.068) 0:02:17.905 ******* 2021-11-21T21:51:24.076 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-11-21T21:51:24.078 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-11-21T21:51:24.105 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.135) 0:02:18.041 ******* 2021-11-21T21:51:24.145 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-11-21T21:51:24.147 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:24.173 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.068) 0:02:18.109 ******* 2021-11-21T21:51:24.205 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-11-21T21:51:24.207 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:24.231 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.057) 0:02:18.166 ******* 2021-11-21T21:51:24.281 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-11-21T21:51:24.283 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:24.309 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.077) 0:02:18.243 ******* 2021-11-21T21:51:24.338 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-11-21T21:51:24.340 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:24.363 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.055) 0:02:18.298 ******* 2021-11-21T21:51:24.370 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-11-21T21:51:24.372 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:24.400 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.037) 0:02:18.336 ******* 2021-11-21T21:51:24.409 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-11-21T21:51:24.410 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:24.441 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.040) 0:02:18.376 ******* 2021-11-21T21:51:24.469 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-11-21T21:51:24.471 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:24.502 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.061) 0:02:18.438 ******* 2021-11-21T21:51:24.510 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-11-21T21:51:24.511 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:24.538 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.035) 0:02:18.474 ******* 2021-11-21T21:51:24.557 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-11-21T21:51:24.558 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:24.588 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.048) 0:02:18.523 ******* 2021-11-21T21:51:24.792 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-11-21T21:51:24.794 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-11-21T21:51:24.821 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.233) 0:02:18.756 ******* 2021-11-21T21:51:24.829 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-11-21T21:51:24.831 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-11-21T21:51:24.857 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.036) 0:02:18.793 ******* 2021-11-21T21:51:24.866 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-11-21T21:51:24.869 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.132207", "end": "2021-11-21 21:51:25.025311", "rc": 0, "start": "2021-11-21 21:51:24.893104", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} TASK [testnode : Create the chrony.conf file] ********************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:24.899 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.041) 0:02:18.834 ******* 2021-11-21T21:51:24.928 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.028) 0:02:18.863 ******* 2021-11-21T21:51:24.936 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-11-21T21:51:24.939 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-11-21T21:51:24.966 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:24 +0000 (0:00:00.038) 0:02:18.902 ******* 2021-11-21T21:51:24.974 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:24.976 INFO:teuthology.task.ansible.out:TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:25.016 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:25 +0000 (0:00:00.050) 0:02:18.952 ******* 2021-11-21T21:51:25.046 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-11-21T21:51:25.192 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 21 November 2021 21:51:25 +0000 (0:00:00.054) 0:02:19.007 ******* TASK [testnode : Make sure ntpd is running.] *********************************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:45 UTC", "ActiveEnterTimestampMonotonic": "12631072", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service systemd-journald.socket basic.target system.slice -.mount sysinit.target network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:45 UTC", "AssertTimestampMonotonic": "12534560", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanRes 2021-11-21T21:51:25.194 INFO:teuthology.task.ansible.out:ult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:45 UTC", "ConditionTimestampMonotonic": "12534558", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "946", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:45 UTC", "ExecMainStartTimestampMonotonic": "12631063", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:45 UTC", "InactiveExitTimestampMonotonic": "12535277", "InvocationID": "63507f3 2021-11-21T21:51:25.196 INFO:teuthology.task.ansible.out:588644235bb743fe0dea21584", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127820", "LimitNPROCSoft": "127820", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "946", "MemoryAccounting": "yes", "MemoryCurrent": "3919872", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelL 2021-11-21T21:51:25.198 INFO:teuthology.task.ansible.out:ogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:45 UTC", "StateChangeTimestampMonotonic": "12631072", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} Sunday 21 November 2021 21:51:25 +0000 (0:00:00.039) 2021-11-21T21:51:25.199 INFO:teuthology.task.ansible.out: 0:02:19.047 ******* TASK [testnode : set_fact] ***************************************************** ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} Sunday 21 November 2021 21:51:25 +0000 (0:00:00.064) 0:02:19.111 ******* 2021-11-21T21:51:25.215 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:25.217 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-11-21T21:51:25.240 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:25 +0000 (0:00:00.065) 0:02:19.176 ******* 2021-11-21T21:51:25.268 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-11-21T21:51:25.270 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:25.292 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:25 +0000 (0:00:00.051) 0:02:19.228 ******* 2021-11-21T21:51:25.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-11-21T21:51:25.522 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:25.548 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:25 +0000 (0:00:00.255) 0:02:19.483 ******* 2021-11-21T21:51:25.576 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-11-21T21:51:25.578 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:25.601 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:25 +0000 (0:00:00.053) 0:02:19.537 ******* 2021-11-21T21:51:25.844 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-11-21T21:51:25.846 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-11-21T21:51:25.882 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:25 +0000 (0:00:00.281) 0:02:19.818 ******* 2021-11-21T21:51:25.913 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-11-21T21:51:25.915 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:25.937 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:25 +0000 (0:00:00.054) 0:02:19.873 ******* 2021-11-21T21:51:26.096 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-11-21T21:51:26.098 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ActiveEnterTimestampMonotonic": "12923099", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target basic.target systemd-journald.socket -.mount network.target system.slice systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:48:46 UTC", "AssertTimestampMonotonic": "12776442", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ConditionTimestampMonotonic": "12776442", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": 2021-11-21T21:51:26.098 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "957", "ExecMainStartTimestamp": "Sun 2021-11-21 21:48:46 UTC", "ExecMainStartTimestampMonotonic": "12923089", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-11-21 21:48:46 UTC", "InactiveExitTimestampMonotonic": "12777097", "InvocationID": "ab400d41b0cf4f3884a12cefa8643f29", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-11-21T21:51:26.100 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127822", "LimitNPROCSoft": "127822", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127822", "LimitSIGPENDINGSoft": "127822", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "957", "MemoryAccounting": "yes", "MemoryCurrent": "3969024", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "Requir 2021-11-21T21:51:26.102 INFO:teuthology.task.ansible.out:esMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:48:46 UTC", "StateChangeTimestampMonotonic": "12923099", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38346", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:26.125 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.187) 0:02:20.060 ******* 2021-11-21T21:51:26.153 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:26.155 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-11-21T21:51:26.177 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.052) 0:02:20.113 ******* 2021-11-21T21:51:26.215 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:26.217 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-11-21T21:51:26.245 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.067) 0:02:20.181 ******* 2021-11-21T21:51:26.272 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-11-21T21:51:26.274 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:26.297 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.052) 0:02:20.233 ******* 2021-11-21T21:51:26.325 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-11-21T21:51:26.327 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:26.352 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.055) 0:02:20.288 ******* 2021-11-21T21:51:26.381 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-11-21T21:51:26.383 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:26.406 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.053) 0:02:20.342 ******* 2021-11-21T21:51:26.562 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-11-21T21:51:26.564 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:26.587 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.180) 0:02:20.523 ******* 2021-11-21T21:51:26.754 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-11-21T21:51:26.780 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.193) 0:02:20.716 ******* 2021-11-21T21:51:26.841 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-11-21T21:51:26.843 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:26.869 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.088) 0:02:20.805 ******* 2021-11-21T21:51:26.902 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-11-21T21:51:26.929 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:26 +0000 (0:00:00.059) 0:02:20.864 ******* 2021-11-21T21:51:27.128 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-11-21T21:51:27.130 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false} 2021-11-21T21:51:27.157 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.227) 0:02:21.092 ******* 2021-11-21T21:51:27.186 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-11-21T21:51:27.188 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} TASK [testnode : Set interface] ************************************************ ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-11-21T21:51:27.211 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.054) 0:02:21.147 ******* 2021-11-21T21:51:27.243 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.031) 0:02:21.178 ******* 2021-11-21T21:51:27.261 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-11-21T21:51:27.263 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false} 2021-11-21T21:51:27.286 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.043) 0:02:21.221 ******* 2021-11-21T21:51:27.294 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-11-21T21:51:27.296 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:27.329 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.042) 0:02:21.264 ******* 2021-11-21T21:51:27.337 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-11-21T21:51:27.338 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-11-21T21:51:27.365 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.036) 0:02:21.301 ******* 2021-11-21T21:51:27.376 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-11-21T21:51:27.378 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531085.9068608, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1637531085.9028609, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1637531085.9028609, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-11-21T21:51:27.405 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.039) 0:02:21.341 ******* 2021-11-21T21:51:27.467 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-11-21T21:51:27.469 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-11-21T21:51:27.495 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.089) 0:02:21.430 ******* 2021-11-21T21:51:27.502 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-11-21T21:51:27.504 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531082.0529516, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1637531082.0489516, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1637531082.0489516, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-11-21T21:51:27.542 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.047) 0:02:21.478 ******* 2021-11-21T21:51:27.809 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-11-21T21:51:27.836 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.293) 0:02:21.771 ******* 2021-11-21T21:51:27.853 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-11-21T21:51:27.855 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-11-21T21:51:27.878 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.042) 0:02:21.814 ******* 2021-11-21T21:51:27.885 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-11-21T21:51:27.887 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-11-21T21:51:27.919 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.040) 0:02:21.854 ******* 2021-11-21T21:51:27.947 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.028) 0:02:21.883 ******* 2021-11-21T21:51:27.955 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-11-21T21:51:27.956 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:27.984 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:27 +0000 (0:00:00.036) 0:02:21.920 ******* 2021-11-21T21:51:27.993 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-11-21T21:51:27.995 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-11-21T21:51:28.021 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.036) 0:02:21.956 ******* 2021-11-21T21:51:28.050 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-11-21T21:51:28.052 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.088 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.067) 0:02:22.024 ******* 2021-11-21T21:51:28.117 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-11-21T21:51:28.119 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-11-21T21:51:28.144 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.055) 0:02:22.079 ******* 2021-11-21T21:51:28.216 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-11-21T21:51:28.243 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.098) 0:02:22.178 ******* 2021-11-21T21:51:28.302 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-11-21T21:51:28.304 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-11-21T21:51:28.331 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.088) 0:02:22.267 ******* 2021-11-21T21:51:28.360 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-11-21T21:51:28.362 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.384 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.053) 0:02:22.320 ******* 2021-11-21T21:51:28.392 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-11-21T21:51:28.394 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-11-21T21:51:28.420 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.035) 0:02:22.355 ******* 2021-11-21T21:51:28.428 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-11-21T21:51:28.429 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=docker0) => {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.431 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.442 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.444 INFO:teuthology.task.ansible.out: skipping: [gibba016.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.469 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.049) 0:02:22.404 ******* 2021-11-21T21:51:28.476 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-11-21T21:51:28.478 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.505 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.036) 0:02:22.441 ******* 2021-11-21T21:51:28.513 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-11-21T21:51:28.515 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'docker0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.526 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.527 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba016.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.555 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.049) 0:02:22.491 ******* 2021-11-21T21:51:28.563 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-11-21T21:51:28.565 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=docker0) => {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.565 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.566 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.566 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.567 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.567 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.592 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.036) 0:02:22.527 ******* 2021-11-21T21:51:28.599 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:28.601 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.627 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.035) 0:02:22.563 ******* 2021-11-21T21:51:28.635 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:28.637 INFO:teuthology.task.ansible.out:TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba017.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'docker0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.638 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.638 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.649 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.651 INFO:teuthology.task.ansible.out: skipping: [gibba017.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages via pip] ************************************* 2021-11-21T21:51:28.652 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} TASK [testnode : Define net_to_configure var] ********************************** 2021-11-21T21:51:28.652 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.654 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.677 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.049) 0:02:22.613 ******* 2021-11-21T21:51:28.706 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.028) 0:02:22.642 ******* 2021-11-21T21:51:28.731 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.024) 0:02:22.666 ******* 2021-11-21T21:51:28.738 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-11-21T21:51:28.740 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.768 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.037) 0:02:22.704 ******* 2021-11-21T21:51:28.793 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.024) 0:02:22.728 ******* 2021-11-21T21:51:28.800 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-11-21T21:51:28.802 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.827 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.033) 0:02:22.762 ******* 2021-11-21T21:51:28.834 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-11-21T21:51:28.836 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.861 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.034) 0:02:22.797 ******* 2021-11-21T21:51:28.868 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-11-21T21:51:28.870 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.896 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.035) 0:02:22.832 ******* 2021-11-21T21:51:28.903 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-11-21T21:51:28.905 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.931 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.034) 0:02:22.867 ******* 2021-11-21T21:51:28.938 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-21T21:51:28.940 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:28.968 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.036) 0:02:22.904 ******* 2021-11-21T21:51:28.975 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-11-21T21:51:28.977 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.002 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:28 +0000 (0:00:00.034) 0:02:22.938 ******* 2021-11-21T21:51:29.010 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.037 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.034) 0:02:22.972 ******* 2021-11-21T21:51:29.044 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-11-21T21:51:29.045 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.072 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.034) 0:02:23.007 ******* 2021-11-21T21:51:29.079 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-21T21:51:29.081 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.110 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.038) 0:02:23.046 ******* 2021-11-21T21:51:29.118 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-21T21:51:29.120 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.148 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.037) 0:02:23.083 ******* 2021-11-21T21:51:29.156 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-11-21T21:51:29.158 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false} 2021-11-21T21:51:29.185 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.036) 0:02:23.120 ******* 2021-11-21T21:51:29.193 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-11-21T21:51:29.195 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.222 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.037) 0:02:23.158 ******* 2021-11-21T21:51:29.230 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-11-21T21:51:29.231 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-11-21T21:51:29.257 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.035) 0:02:23.193 ******* 2021-11-21T21:51:29.264 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:29.266 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002491", "end": "2021-11-21 21:51:28.994896", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-11-21 21:51:28.992405", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.293 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.035) 0:02:23.229 ******* 2021-11-21T21:51:29.355 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:29.357 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001743", "end": "2021-11-21 21:51:29.102840", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-11-21 21:51:29.101097", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:29.388 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-11-21T21:51:29.390 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531080.9053764, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1637531080.9013765, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1637531080.9013765, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-11-21T21:51:29.415 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.121) 0:02:23.351 ******* 2021-11-21T21:51:29.443 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:29.445 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002497", "end": "2021-11-21 21:51:29.808404", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-11-21 21:51:29.805907", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:29.466 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001604", "end": "2021-11-21 21:51:29.207631", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-11-21 21:51:29.206027", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:29.509 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-11-21T21:51:29.511 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-11-21T21:51:29.538 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.122) 0:02:23.473 ******* 2021-11-21T21:51:29.559 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:29.561 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001772", "end": "2021-11-21 21:51:29.920536", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-11-21 21:51:29.918764", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:29.571 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-11-21T21:51:29.572 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-11-21T21:51:29.597 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.059) 0:02:23.533 ******* 2021-11-21T21:51:29.606 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:29.608 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.026421", "end": "2021-11-21 21:51:29.335590", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-11-21 21:51:29.309169", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:29.637 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.039) 0:02:23.572 ******* 2021-11-21T21:51:29.645 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-11-21T21:51:29.646 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-11-21T21:51:29.673 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.035) 0:02:23.608 ******* 2021-11-21T21:51:29.685 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:29.687 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001664", "end": "2021-11-21 21:51:30.025496", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-11-21 21:51:30.023832", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-11-21T21:51:29.713 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.040) 0:02:23.648 ******* 2021-11-21T21:51:29.720 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-11-21T21:51:29.722 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.753 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.040) 0:02:23.689 ******* 2021-11-21T21:51:29.761 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-11-21T21:51:29.763 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:29.791 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:29.793 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.027668", "end": "2021-11-21 21:51:30.155818", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-11-21 21:51:30.128150", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:29.828 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.074) 0:02:23.763 ******* 2021-11-21T21:51:29.856 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-11-21T21:51:29.881 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.053) 0:02:23.817 ******* 2021-11-21T21:51:29.890 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-11-21T21:51:29.892 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-11-21T21:51:29.917 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.035) 0:02:23.853 ******* 2021-11-21T21:51:29.927 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-11-21T21:51:29.928 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Purge resolvconf] ********************************************* ok: [gibba036.front.sepia.ceph.com] => {"changed": false} 2021-11-21T21:51:29.968 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:29 +0000 (0:00:00.050) 0:02:23.904 ******* 2021-11-21T21:51:30.002 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-11-21T21:51:30.004 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-11-21T21:51:30.025 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.056) 0:02:23.960 ******* 2021-11-21T21:51:30.075 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-11-21T21:51:30.076 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-11-21T21:51:30.101 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.076) 0:02:24.037 ******* 2021-11-21T21:51:30.129 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-11-21T21:51:30.131 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.154 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.052) 0:02:24.089 ******* 2021-11-21T21:51:30.160 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-11-21T21:51:30.161 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531093.4479356, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1637531093.4439354, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1637531093.4439354, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-11-21T21:51:30.195 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.041) 0:02:24.131 ******* 2021-11-21T21:51:30.203 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-11-21T21:51:30.205 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=docker0) => {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.218 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.220 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.246 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.051) 0:02:24.182 ******* 2021-11-21T21:51:30.309 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-11-21T21:51:30.311 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'docker0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.320 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.322 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.333 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-11-21T21:51:30.334 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "msg": ""} TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.359 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.112) 0:02:24.294 ******* 2021-11-21T21:51:30.367 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.395 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.036) 0:02:24.331 ******* 2021-11-21T21:51:30.402 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-11-21T21:51:30.404 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-11-21T21:51:30.429 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.033) 0:02:24.365 ******* 2021-11-21T21:51:30.436 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-21T21:51:30.437 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.467 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.037) 0:02:24.402 ******* 2021-11-21T21:51:30.474 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-11-21T21:51:30.476 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.502 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.035) 0:02:24.438 ******* 2021-11-21T21:51:30.509 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-11-21T21:51:30.511 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.549 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.046) 0:02:24.485 ******* 2021-11-21T21:51:30.589 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-11-21T21:51:30.590 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.619 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.069) 0:02:24.554 ******* 2021-11-21T21:51:30.648 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-11-21T21:51:30.650 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.672 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.053) 0:02:24.607 ******* 2021-11-21T21:51:30.679 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-11-21T21:51:30.681 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-11-21T21:51:30.707 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.035) 0:02:24.643 ******* 2021-11-21T21:51:30.715 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-11-21T21:51:30.717 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.747 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.039) 0:02:24.682 ******* 2021-11-21T21:51:30.776 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-21T21:51:30.778 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.826 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.079) 0:02:24.761 ******* 2021-11-21T21:51:30.860 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.892 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.066) 0:02:24.828 ******* 2021-11-21T21:51:30.899 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-11-21T21:51:30.901 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-11-21T21:51:30.926 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.033) 0:02:24.861 ******* 2021-11-21T21:51:30.933 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-21T21:51:30.935 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.959 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.033) 0:02:24.895 ******* 2021-11-21T21:51:30.966 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-11-21T21:51:30.967 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:30.994 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:30 +0000 (0:00:00.033) 0:02:24.929 ******* 2021-11-21T21:51:31.030 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:31.032 INFO:teuthology.task.ansible.out:TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [gibba036.front.sepia.ceph.com] => (item=docker0) => {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.043 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.045 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.055 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.081 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.087) 0:02:25.017 ******* 2021-11-21T21:51:31.121 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:31.122 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002583", "end": "2021-11-21 21:51:30.841101", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-11-21 21:51:30.838518", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'docker0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "docker0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.123 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.134 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.145 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.147 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.176 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.091) 0:02:25.109 ******* 2021-11-21T21:51:31.202 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.229 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.056) 0:02:25.165 ******* 2021-11-21T21:51:31.238 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:31.239 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001749", "end": "2021-11-21 21:51:30.950686", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-11-21 21:51:30.948937", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:31.261 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.286 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.056) 0:02:25.222 ******* 2021-11-21T21:51:31.314 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-11-21T21:51:31.316 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [gibba019.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001678", "end": "2021-11-21 21:51:31.053114", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-11-21 21:51:31.051436", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:31.356 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.069) 0:02:25.292 ******* 2021-11-21T21:51:31.384 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-11-21T21:51:31.386 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.411 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.053) 0:02:25.346 ******* 2021-11-21T21:51:31.438 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-11-21T21:51:31.440 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.466 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.055) 0:02:25.401 ******* 2021-11-21T21:51:31.474 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:31.476 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.053392", "end": "2021-11-21 21:51:31.209542", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-11-21 21:51:31.156150", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:31.510 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.044) 0:02:25.446 ******* 2021-11-21T21:51:31.518 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-21T21:51:31.520 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.547 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.036) 0:02:25.483 ******* 2021-11-21T21:51:31.555 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-11-21T21:51:31.581 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.034) 0:02:25.517 ******* 2021-11-21T21:51:31.589 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-11-21T21:51:31.591 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.618 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.036) 0:02:25.553 ******* 2021-11-21T21:51:31.625 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-11-21T21:51:31.626 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.653 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:31.704 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:31 +0000 (0:00:00.086) 0:02:25.640 ******* 2021-11-21T21:51:31.848 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-21T21:51:31.850 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002586", "end": "2021-11-21 21:51:32.040830", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-11-21 21:51:32.038244", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:31.955 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001863", "end": "2021-11-21 21:51:32.147753", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-11-21 21:51:32.145890", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:32.061 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001718", "end": "2021-11-21 21:51:32.252224", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-11-21 21:51:32.250506", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:32.198 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.020942", "end": "2021-11-21 21:51:32.374038", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-11-21 21:51:32.353096", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:32.231 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:32 +0000 (0:00:00.519) 0:02:26.159 ******* 2021-11-21T21:51:32.258 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-11-21T21:51:32.282 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:32 +0000 (0:00:00.057) 0:02:26.217 ******* 2021-11-21T21:51:32.309 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-11-21T21:51:32.311 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:32.384 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-11-21T21:51:32.549 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:32 +0000 (0:00:00.267) 0:02:26.484 ******* 2021-11-21T21:51:32.570 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:32 +0000 (0:00:00.021) 0:02:26.506 ******* 2021-11-21T21:51:32.594 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:32 +0000 (0:00:00.023) 0:02:26.529 ******* 2021-11-21T21:51:32.618 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:32 +0000 (0:00:00.024) 0:02:26.554 ******* 2021-11-21T21:51:33.328 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba017.front.sepia.ceph.com] 2021-11-21T21:51:33.384 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.766) 0:02:27.320 ******* 2021-11-21T21:51:33.424 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:33.426 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-11-21T21:51:33.455 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.071) 0:02:27.391 ******* 2021-11-21T21:51:33.475 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba036.front.sepia.ceph.com] 2021-11-21T21:51:33.489 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:33.490 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:33.493 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba016.front.sepia.ceph.com] 2021-11-21T21:51:33.536 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.080) 0:02:27.471 ******* 2021-11-21T21:51:33.571 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] 2021-11-21T21:51:33.601 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.065) 0:02:27.537 ******* 2021-11-21T21:51:33.644 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.042) 0:02:27.580 ******* 2021-11-21T21:51:33.654 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-11-21T21:51:33.656 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io", "python3-setuptools", "python3-pip"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:33.656 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-11-21T21:51:33.681 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.037) 0:02:27.617 ******* 2021-11-21T21:51:33.704 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.022) 0:02:27.639 ******* 2021-11-21T21:51:33.732 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.027) 0:02:27.667 ******* 2021-11-21T21:51:33.739 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-11-21T21:51:33.741 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:33.769 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.036) 0:02:27.704 ******* 2021-11-21T21:51:33.795 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.026) 0:02:27.730 ******* 2021-11-21T21:51:33.801 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:33.804 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-11-21T21:51:33.831 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.036) 0:02:27.766 ******* 2021-11-21T21:51:33.837 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:33.839 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [container-host : Including distro specific variables] ******************** ok: [gibba016.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io", "python3-setuptools", "python3-pip"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-11-21T21:51:33.865 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.034) 0:02:27.801 ******* 2021-11-21T21:51:33.873 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:33.874 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:33.909 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.043) 0:02:27.845 ******* 2021-11-21T21:51:33.935 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.025) 0:02:27.870 ******* 2021-11-21T21:51:33.944 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-11-21T21:51:33.946 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io", "python3-setuptools", "python3-pip"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-11-21T21:51:33.973 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:33 +0000 (0:00:00.037) 0:02:27.908 ******* 2021-11-21T21:51:33.991 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io", "python3-setuptools", "python3-pip"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-11-21T21:51:34.021 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:34 +0000 (0:00:00.048) 0:02:27.956 ******* 2021-11-21T21:51:46.365 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-11-21T21:51:46.367 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 4096 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 2s (17.4 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...\r\nSetting up docker.io (20.10.7-0ubuntu5~20.04.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", " 2021-11-21T21:51:46.369 INFO:teuthology.task.ansible.out:Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "Suggested packages:", " aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 4096 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 2s (17.4 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...", "Setting up docker.io (20.10.7-0ubuntu5~20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-11-21T21:51:46.392 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:12.370) 0:02:40.327 ******* 2021-11-21T21:51:46.421 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:46.423 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:46.445 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.053) 0:02:40.381 ******* 2021-11-21T21:51:46.483 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:46.485 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-11-21T21:51:46.508 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.062) 0:02:40.443 ******* 2021-11-21T21:51:46.578 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-11-21T21:51:46.580 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 4096 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 2s (16.2 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...\r\nSetting up docker.io (20.10.7-0ubuntu5~20.04.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", " 2021-11-21T21:51:46.582 INFO:teuthology.task.ansible.out:Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "Suggested packages:", " aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 4096 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 2s (16.2 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...", "Setting up docker.io (20.10.7-0ubuntu5~20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-11-21T21:51:46.605 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.097) 0:02:40.541 ******* 2021-11-21T21:51:46.634 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:46.637 INFO:teuthology.task.ansible.out:skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:46.659 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.053) 0:02:40.594 ******* 2021-11-21T21:51:46.698 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:46.700 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-11-21T21:51:46.732 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.073) 0:02:40.668 ******* 2021-11-21T21:51:46.744 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-11-21T21:51:46.745 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:46.783 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:46.785 INFO:teuthology.task.ansible.out:TASK [container-host : Install container packages] ***************************** changed: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 4096 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 2s (16.9 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...\r\nSetting up docker.io (20.10.7-0ubuntu5~20.04.2) ...\r\nProcessing trig 2021-11-21T21:51:46.787 INFO:teuthology.task.ansible.out:gers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "Suggested packages:", " aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 4096 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 2s (16.9 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...", "Setting up docker.io (20.10.7-0ubuntu5~20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-11-21T21:51:46.820 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.086) 0:02:40.755 ******* 2021-11-21T21:51:46.831 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-11-21T21:51:46.833 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:46.868 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.048) 0:02:40.804 ******* 2021-11-21T21:51:46.876 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:46.878 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:46.903 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.034) 0:02:40.839 ******* 2021-11-21T21:51:46.914 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* ok: [gibba036.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} TASK [container-host : Install container packages] ***************************** changed: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 4096 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 2s (16.4 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Readin 2021-11-21T21:51:46.916 INFO:teuthology.task.ansible.out:g database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 174687 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...\r\nSetting up docker.io (20.10.7-0ubuntu5~20.04.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic", " linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic", " linux-modules-extra-5.4.0-66-generic", "Use 'sudo apt autoremove' to remove them.", "Suggested packages:", " aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 4096 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.7-0ubuntu5~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 2s (16.4 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database . 2021-11-21T21:51:46.917 INFO:teuthology.task.ansible.out:.. 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 174687 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.7-0ubuntu5~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.7-0ubuntu5~20.04.2) over (20.10.7-0ubuntu1~20.04.1) ...", "Setting up docker.io (20.10.7-0ubuntu5~20.04.2) ...", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-11-21T21:51:46.949 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.046) 0:02:40.885 ******* 2021-11-21T21:51:46.959 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:46.961 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-11-21T21:51:46.989 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:46 +0000 (0:00:00.039) 0:02:40.925 ******* 2021-11-21T21:51:47.000 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:47.002 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:47.032 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:47 +0000 (0:00:00.042) 0:02:40.967 ******* 2021-11-21T21:51:47.041 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-11-21T21:51:47.043 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:47.084 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:47 +0000 (0:00:00.050) 0:02:41.018 ******* 2021-11-21T21:51:47.093 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-21T21:51:47.096 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-11-21T21:51:47.125 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:47 +0000 (0:00:00.041) 0:02:41.060 ******* 2021-11-21T21:51:47.178 INFO:teuthology.task.ansible.out: 2021-11-21T21:51:47.180 INFO:teuthology.task.ansible.out:TASK [container-host : Create container_mirror_cert_paths] ********************* ok: [gibba019.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:47.203 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-11-21T21:51:47.205 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:47.300 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-11-21T21:51:47.302 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:47.343 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:47 +0000 (0:00:00.218) 0:02:41.279 ******* 2021-11-21T21:51:47.354 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:47.419 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-11-21T21:51:47.421 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:47.452 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-11-21T21:51:47.454 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-11-21T21:51:47.482 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:47 +0000 (0:00:00.138) 0:02:41.418 ******* 2021-11-21T21:51:47.501 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-11-21T21:51:47.503 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:47.532 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:47 +0000 (0:00:00.049) 0:02:41.467 ******* 2021-11-21T21:51:47.681 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:47.744 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:47.770 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:47 +0000 (0:00:00.238) 0:02:41.705 ******* 2021-11-21T21:51:47.834 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:47.975 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:48.008 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:48 +0000 (0:00:00.238) 0:02:41.944 ******* 2021-11-21T21:51:48.038 INFO:teuthology.task.ansible.out: TASK [container-host : Ensure git is installed] ******************************** 2021-11-21T21:51:48.039 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:48.072 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:48 +0000 (0:00:00.063) 0:02:42.008 ******* 2021-11-21T21:51:48.154 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-11-21T21:51:48.156 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "size": 2334, "state": "file", "uid": 0} 2021-11-21T21:51:48.184 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:48 +0000 (0:00:00.111) 0:02:42.120 ******* 2021-11-21T21:51:48.280 INFO:teuthology.task.ansible.out: TASK [container-host : Ensure git is installed] ******************************** ok: [gibba036.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:48.306 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:48 +0000 (0:00:00.121) 0:02:42.241 ******* 2021-11-21T21:51:48.509 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:48.535 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:48 +0000 (0:00:00.229) 0:02:42.471 ******* 2021-11-21T21:51:48.725 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"cache_update_time": 1637531146, "cache_updated": false, "changed": false} 2021-11-21T21:51:48.751 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:48 +0000 (0:00:00.216) 0:02:42.687 ******* 2021-11-21T21:51:51.389 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-11-21T21:51:51.391 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-fizbx411\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-fizbx411"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-fizbx411\nRequirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)\nRequirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)\nBuilding wheels for collected packages: registries-conf-ctl\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2\n Stored in directory: /tmp/pip-ephem-wheel-cache-jg07s9im/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\nSuccessfully built registries-conf-ctl\nInstalling collected packages: registries-conf-ctl\n Attempting uninstall: registries-conf-ctl\n Found existing installation: registries-conf-ctl 1.0\n Uninstalling registries-conf-ctl-1.0:\n Successfully uninstalled registries-conf-ctl-1.0\nSuccessfully installed registries-conf-ctl-1.0\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/regi 2021-11-21T21:51:51.393 INFO:teuthology.task.ansible.out:stries-conf-ctl to ./pip-req-build-fizbx411", "Requirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)", "Requirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)", "Building wheels for collected packages: registries-conf-ctl", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2", " Stored in directory: /tmp/pip-ephem-wheel-cache-jg07s9im/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", "Successfully built registries-conf-ctl", "Installing collected packages: registries-conf-ctl", " Attempting uninstall: registries-conf-ctl", " Found existing installation: registries-conf-ctl 1.0", " Uninstalling registries-conf-ctl-1.0:", " Successfully uninstalled registries-conf-ctl-1.0", "Successfully installed registries-conf-ctl-1.0"], "version": null, "virtualenv": null} 2021-11-21T21:51:51.418 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:02.665) 0:02:45.352 ******* 2021-11-21T21:51:51.503 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-_u56xmpc\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-_u56xmpc"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-_u56xmpc\nRequirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)\nRequirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)\nBuilding wheels for collected packages: registries-conf-ctl\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2\n Stored in directory: /tmp/pip-ephem-wheel-cache-nhikwezh/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\nSuccessfully built registries-conf-ctl\nInstalling collected packages: registries-conf-ctl\n Attempting uninstall: registries-conf-ctl\n Found existing installation: registries-conf-ctl 1.0\n Uninstalling registries-conf-ctl-1.0:\n Successfully uninstalled registries-conf-ctl-1.0\nSuccessfully installed registries-conf-ctl-1.0\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/regi 2021-11-21T21:51:51.505 INFO:teuthology.task.ansible.out:stries-conf-ctl to ./pip-req-build-_u56xmpc", "Requirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)", "Requirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)", "Building wheels for collected packages: registries-conf-ctl", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2", " Stored in directory: /tmp/pip-ephem-wheel-cache-nhikwezh/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", "Successfully built registries-conf-ctl", "Installing collected packages: registries-conf-ctl", " Attempting uninstall: registries-conf-ctl", " Found existing installation: registries-conf-ctl 1.0", " Uninstalling registries-conf-ctl-1.0:", " Successfully uninstalled registries-conf-ctl-1.0", "Successfully installed registries-conf-ctl-1.0"], "version": null, "virtualenv": null} 2021-11-21T21:51:51.531 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:00.114) 0:02:45.466 ******* 2021-11-21T21:51:51.572 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-11-21T21:51:51.575 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531070.1129172, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "13ed9dbcd1f35408627b14794589868021e7d546", "ctime": 1633111259.4300582, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 97715, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1633111259.4300582, "nlink": 1, "path": "/etc/docker/daemon.json", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 151, "uid": 0, "version": "539352039", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-11-21T21:51:51.606 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:00.073) 0:02:45.540 ******* 2021-11-21T21:51:51.645 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-11-21T21:51:51.647 INFO:teuthology.task.ansible.out:skipping: [gibba017.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:51.670 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:00.065) 0:02:45.606 ******* 2021-11-21T21:51:51.710 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-11-21T21:51:51.712 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531081.3399005, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "13ed9dbcd1f35408627b14794589868021e7d546", "ctime": 1633111259.4300582, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 97715, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1633111259.4300582, "nlink": 1, "path": "/etc/docker/daemon.json", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 151, "uid": 0, "version": "539352039", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-11-21T21:51:51.740 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:00.069) 0:02:45.675 ******* 2021-11-21T21:51:51.780 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ skipping: [gibba036.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:51.807 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:00.066) 0:02:45.742 ******* 2021-11-21T21:51:51.815 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-11-21T21:51:51.815 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-6iz64e1i\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-6iz64e1i"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-6iz64e1i\nRequirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)\nRequirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)\nBuilding wheels for collected packages: registries-conf-ctl\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2\n Stored in directory: /tmp/pip-ephem-wheel-cache-ahfa0gdz/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\nSuccessfully built registries-conf-ctl\nInstalling collected packages: registries-conf-ctl\n Attempting uninstall: registries-conf-ctl\n Found existing installation: registries-conf-ctl 1.0\n Uninstalling registries-conf-ctl-1.0:\n Successfully uninstalled registries-conf-ctl-1.0\nSuccessfully installed registries-conf-ctl-1.0\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/regi 2021-11-21T21:51:51.818 INFO:teuthology.task.ansible.out:stries-conf-ctl to ./pip-req-build-6iz64e1i", "Requirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)", "Requirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)", "Building wheels for collected packages: registries-conf-ctl", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2", " Stored in directory: /tmp/pip-ephem-wheel-cache-ahfa0gdz/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", "Successfully built registries-conf-ctl", "Installing collected packages: registries-conf-ctl", " Attempting uninstall: registries-conf-ctl", " Found existing installation: registries-conf-ctl 1.0", " Uninstalling registries-conf-ctl-1.0:", " Successfully uninstalled registries-conf-ctl-1.0", "Successfully installed registries-conf-ctl-1.0"], "version": null, "virtualenv": null} 2021-11-21T21:51:51.847 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:00.040) 0:02:45.782 ******* 2021-11-21T21:51:51.878 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.066241", "end": "2021-11-21 21:51:52.252938", "rc": 0, "start": "2021-11-21 21:51:52.186697", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:51.910 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:51 +0000 (0:00:00.063) 0:02:45.846 ******* 2021-11-21T21:51:52.010 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-11-21T21:51:52.011 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531068.4213405, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "13ed9dbcd1f35408627b14794589868021e7d546", "ctime": 1633111259.4300582, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 97715, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1633111259.4300582, "nlink": 1, "path": "/etc/docker/daemon.json", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 151, "uid": 0, "version": "539352039", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} TASK [container-host : Add local docker.io registry mirror] ******************** changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.039234", "end": "2021-11-21 21:51:52.212626", "rc": 0, "start": "2021-11-21 21:51:52.173392", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:52.040 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.129) 0:02:45.975 ******* 2021-11-21T21:51:52.065 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.025) 0:02:46.001 ******* 2021-11-21T21:51:52.073 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-11-21T21:51:52.075 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-uwiz9gdv\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-uwiz9gdv"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-uwiz9gdv\nRequirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)\nRequirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)\nBuilding wheels for collected packages: registries-conf-ctl\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2\n Stored in directory: /tmp/pip-ephem-wheel-cache-vwa6aq3v/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\nSuccessfully built registries-conf-ctl\nInstalling collected packages: registries-conf-ctl\n Attempting uninstall: registries-conf-ctl\n Found existing installation: registries-conf-ctl 1.0\n Uninstalling registries-conf-ctl-1.0:\n Successfully uninstalled registries-conf-ctl-1.0\nSuccessfully installed registries-conf-ctl-1.0\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/regi 2021-11-21T21:51:52.077 INFO:teuthology.task.ansible.out:stries-conf-ctl to ./pip-req-build-uwiz9gdv", "Requirement already satisfied, skipping upgrade: docopt in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.6.2)", "Requirement already satisfied, skipping upgrade: toml in /usr/local/lib/python3.8/dist-packages (from registries-conf-ctl==1.0) (0.10.2)", "Building wheels for collected packages: registries-conf-ctl", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=8180 sha256=a89bbffc9bc2f927f495c7466f15f9493e97fb7d169efb095f63fe7675713de2", " Stored in directory: /tmp/pip-ephem-wheel-cache-vwa6aq3v/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", "Successfully built registries-conf-ctl", "Installing collected packages: registries-conf-ctl", " Attempting uninstall: registries-conf-ctl", " Found existing installation: registries-conf-ctl 1.0", " Uninstalling registries-conf-ctl-1.0:", " Successfully uninstalled registries-conf-ctl-1.0", "Successfully installed registries-conf-ctl-1.0"], "version": null, "virtualenv": null} 2021-11-21T21:51:52.101 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.035) 0:02:46.036 ******* 2021-11-21T21:51:52.110 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:52.146 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.044) 0:02:46.081 ******* 2021-11-21T21:51:52.258 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-11-21T21:51:52.261 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1637531074.3068273, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "13ed9dbcd1f35408627b14794589868021e7d546", "ctime": 1633111259.4300582, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 97715, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1633111259.4300582, "nlink": 1, "path": "/etc/docker/daemon.json", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 151, "uid": 0, "version": "539352039", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-11-21T21:51:52.287 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.141) 0:02:46.223 ******* 2021-11-21T21:51:52.328 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-11-21T21:51:52.329 INFO:teuthology.task.ansible.out:skipping: [gibba016.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-21T21:51:52.354 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.066) 0:02:46.289 ******* 2021-11-21T21:51:52.375 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.064855", "end": "2021-11-21 21:51:52.102193", "rc": 0, "start": "2021-11-21 21:51:52.037338", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:52.405 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.051) 0:02:46.341 ******* 2021-11-21T21:51:52.564 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.058803", "end": "2021-11-21 21:51:52.311713", "rc": 0, "start": "2021-11-21 21:51:52.252910", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-21T21:51:52.592 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:52 +0000 (0:00:00.187) 0:02:46.528 ******* 2021-11-21T21:51:53.610 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-11-21T21:51:53.611 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:51:44 UTC", "ActiveEnterTimestampMonotonic": "191656103", "ActiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ActiveExitTimestampMonotonic": "190756104", "ActiveState": "active", "After": "system.slice systemd-journald.socket containerd.service basic.target sysinit.target firewalld.service network-online.target docker.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:51:43 UTC", "AssertTimestampMonotonic": "190760720", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ConditionTimestampMonotonic": "190760720", "ConfigurationDirectoryMode": "0755", 2021-11-21T21:51:53.613 INFO:teuthology.task.ansible.out: "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8445", "ExecMainStartTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ExecMainStartTimestampMonotonic": "190761077", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IP 2021-11-21T21:51:53.614 INFO:teuthology.task.ansible.out:IngressPackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveEnterTimestampMonotonic": "190758546", "InactiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveExitTimestampMonotonic": "190761369", "InvocationID": "6a4843f9fbdd49c78cc017bb6352e063", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "8445", "MemoryAccounting": "yes", "MemoryCurrent": "48271360", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": 2021-11-21T21:51:53.616 INFO:teuthology.task.ansible.out: "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continue", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target docker.socket", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:51:44 UTC", "StateChangeTimestampMonotonic": "191656103", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "23", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec 2021-11-21T21:51:53.618 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "containerd.service network-online.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:53.640 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:51:44 UTC", "ActiveEnterTimestampMonotonic": "191190328", "ActiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ActiveExitTimestampMonotonic": "190336995", "ActiveState": "active", "After": "sysinit.target docker.socket basic.target system.slice systemd-journald.socket firewalld.service containerd.service network-online.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:51:43 UTC", "AssertTimestampMonotonic": "190342133", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ConditionTimestampMonotonic": "190342133", "ConfigurationDirectoryMode": "0755", 2021-11-21T21:51:53.642 INFO:teuthology.task.ansible.out: "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8474", "ExecMainStartTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ExecMainStartTimestampMonotonic": "190342520", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IP 2021-11-21T21:51:53.643 INFO:teuthology.task.ansible.out:IngressPackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveEnterTimestampMonotonic": "190339714", "InactiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveExitTimestampMonotonic": "190342828", "InvocationID": "aebd3bdfa46c4959a6ae54342e89d859", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127822", "LimitSIGPENDINGSoft": "127822", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "8474", "MemoryAccounting": "yes", "MemoryCurrent": "47505408", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": 2021-11-21T21:51:53.645 INFO:teuthology.task.ansible.out: "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continue", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket sysinit.target system.slice", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:51:44 UTC", "StateChangeTimestampMonotonic": "191190328", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "24", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec 2021-11-21T21:51:53.646 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "containerd.service network-online.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:54.073 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ActiveEnterTimestampMonotonic": "190691135", "ActiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ActiveExitTimestampMonotonic": "189753623", "ActiveState": "active", "After": "system.slice basic.target docker.socket firewalld.service containerd.service systemd-journald.socket sysinit.target network-online.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:51:43 UTC", "AssertTimestampMonotonic": "189758443", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ConditionTimestampMonotonic": "189758443", "ConfigurationDirectoryMode": "0755", 2021-11-21T21:51:54.075 INFO:teuthology.task.ansible.out: "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8458", "ExecMainStartTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ExecMainStartTimestampMonotonic": "189758849", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IP 2021-11-21T21:51:54.076 INFO:teuthology.task.ansible.out:IngressPackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveEnterTimestampMonotonic": "189756220", "InactiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveExitTimestampMonotonic": "189759108", "InvocationID": "ea2789e8b6c4457695a0c4fecce6d05f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "8458", "MemoryAccounting": "yes", "MemoryCurrent": "48906240", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": 2021-11-21T21:51:54.078 INFO:teuthology.task.ansible.out: "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continue", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target docker.socket", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:51:43 UTC", "StateChangeTimestampMonotonic": "190691135", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "22", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec 2021-11-21T21:51:54.079 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init 2021-11-21T21:51:54.080 INFO:teuthology.task.ansible.out:", "WantedBy": "multi-user.target", "Wants": "network-online.target containerd.service", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:54.295 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-11-21 21:51:44 UTC", "ActiveEnterTimestampMonotonic": "189903161", "ActiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ActiveExitTimestampMonotonic": "188901397", "ActiveState": "active", "After": "docker.socket sysinit.target basic.target network-online.target firewalld.service system.slice containerd.service systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-11-21 21:51:43 UTC", "AssertTimestampMonotonic": "188906230", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ConditionTimestampMonotonic": "188906230", "ConfigurationDirectoryMode": "0755", 2021-11-21T21:51:54.297 INFO:teuthology.task.ansible.out: "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8458", "ExecMainStartTimestamp": "Sun 2021-11-21 21:51:43 UTC", "ExecMainStartTimestampMonotonic": "188906602", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IP 2021-11-21T21:51:54.298 INFO:teuthology.task.ansible.out:IngressPackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveEnterTimestampMonotonic": "188903849", "InactiveExitTimestamp": "Sun 2021-11-21 21:51:43 UTC", "InactiveExitTimestampMonotonic": "188906724", "InvocationID": "9a55bd0637d243fe93b8c130e48621a2", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127820", "LimitSIGPENDINGSoft": "127820", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "8458", "MemoryAccounting": "yes", "MemoryCurrent": "46813184", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": 2021-11-21T21:51:54.300 INFO:teuthology.task.ansible.out: "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continue", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket sysinit.target system.slice", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-11-21 21:51:44 UTC", "StateChangeTimestampMonotonic": "189903161", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "23", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartU 2021-11-21T21:51:54.300 INFO:teuthology.task.ansible.out:Sec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec 2021-11-21T21:51:54.302 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "containerd.service network-online.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-21T21:51:54.377 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-11-21T21:51:54.378 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-11-21T21:51:54.380 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-11-21T21:51:54.381 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-11-21T21:51:54.398 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-11-21T21:51:54.399 INFO:teuthology.task.ansible.out:Sunday 21 November 2021 21:51:54 +0000 (0:00:01.805) 0:02:48.333 ******* 2021-11-21T21:51:55.228 INFO:teuthology.task.ansible.out:ok: [gibba016.front.sepia.ceph.com] 2021-11-21T21:51:55.261 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] 2021-11-21T21:51:55.375 INFO:teuthology.task.ansible.out:ok: [gibba036.front.sepia.ceph.com] 2021-11-21T21:51:56.191 INFO:teuthology.task.ansible.out:ok: [gibba017.front.sepia.ceph.com] 2021-11-21T21:51:56.211 INFO:teuthology.task.ansible.out: TASK [Touch /ceph-qa-ready] **************************************************** Sunday 21 November 2021 21:51:56 +0000 (0:00:01.813) 0:02:50.147 ******* 2021-11-21T21:51:56.350 INFO:teuthology.task.ansible.out:changed: [gibba016.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-11-21T21:51:56.383 INFO:teuthology.task.ansible.out:changed: [gibba017.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-11-21T21:51:56.406 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-11-21T21:51:56.428 INFO:teuthology.task.ansible.out:changed: [gibba036.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-11-21T21:51:56.432 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* 2021-11-21T21:51:56.433 INFO:teuthology.task.ansible.out:gibba016.front.sepia.ceph.com : ok=131 changed=32 unreachable=0 failed=0 skipped=188 rescued=0 ignored=0 gibba017.front.sepia.ceph.com : ok=131 changed=32 unreachable=0 failed=0 skipped=188 rescued=0 ignored=0 gibba019.front.sepia.ceph.com : ok=131 changed=32 unreachable=0 failed=0 skipped=188 rescued=0 ignored=0 gibba036.front.sepia.ceph.com : ok=131 changed=32 unreachable=0 failed=0 skipped=188 rescued=0 ignored=0 Sunday 21 November 2021 21:51:56 +0000 (0:00:00.220) 0:02:50.368 ******* =============================================================================== users : Update authorized_keys using the keys repo --------------------- 15.86s Create all admin users with sudo access. ------------------------------- 15.44s container-host : Install container packages ---------------------------- 12.37s testnode : Upgrade packages -------------------------------------------- 12.27s testnode : Install packages --------------------------------------------- 8.52s testnode : Ensure packages are not present. ----------------------------- 5.79s Remove revoked users ---------------------------------------------------- 4.45s testnode : Zap all non-root disks --------------------------------------- 4.28s container-host : Install registries-conf-ctl ---------------------------- 2.66s common : Update apt cache ----------------------------------------------- 2.47s Gathering Facts --------------------------------------------------------- 1.95s Gathering Facts --------------------------------------------------------- 1.81s container-host : Restart docker service --------------------------------- 1.81s Gathering Facts --------------------------------------------------------- 0.77s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.76s Gathering Facts --------------------------------------------------------- 0.63s ansible-managed : Add authorized keys for the ansi 2021-11-21T21:51:56.435 INFO:teuthology.task.ansible.out:ble user. ------------- 0.59s Gathering Facts --------------------------------------------------------- 0.56s users : Update authorized_keys for each user with literal keys ---------- 0.53s users : Check keys_repo HEAD sha1 --------------------------------------- 0.53s 2021-11-21T21:51:56.835 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@gibba016.front.sepia.ceph.com'), Remote(name='ubuntu@gibba017.front.sepia.ceph.com'), Remote(name='ubuntu@gibba019.front.sepia.ceph.com'), Remote(name='ubuntu@gibba036.front.sepia.ceph.com')] 2021-11-21T21:51:56.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba016.front.sepia.ceph.com' 2021-11-21T21:51:56.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:51:56.876 DEBUG:teuthology.orchestra.run.gibba016:> true 2021-11-21T21:51:56.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba017.front.sepia.ceph.com' 2021-11-21T21:51:56.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:51:57.022 DEBUG:teuthology.orchestra.run.gibba017:> true 2021-11-21T21:51:57.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba019.front.sepia.ceph.com' 2021-11-21T21:51:57.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:51:57.146 DEBUG:teuthology.orchestra.run.gibba019:> true 2021-11-21T21:51:57.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba036.front.sepia.ceph.com' 2021-11-21T21:51:57.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-21T21:51:57.276 DEBUG:teuthology.orchestra.run.gibba036:> true 2021-11-21T21:51:57.362 INFO:teuthology.run_tasks:Running task clock... 2021-11-21T21:51:57.380 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-11-21T21:51:57.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-11-21T21:51:57.382 DEBUG:teuthology.orchestra.run.gibba016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T21:51:57.386 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-11-21T21:51:57.386 DEBUG:teuthology.orchestra.run.gibba017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T21:51:57.389 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-11-21T21:51:57.390 DEBUG:teuthology.orchestra.run.gibba019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T21:51:57.393 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-11-21T21:51:57.393 DEBUG:teuthology.orchestra.run.gibba036:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T21:51:57.403 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-11-21T21:51:57.403 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Command line: ntpd -gq 2021-11-21T21:51:57.404 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: proto: precision = 0.052 usec (-24) 2021-11-21T21:51:57.404 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.404 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.405 INFO:teuthology.orchestra.run.gibba016.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.405 INFO:teuthology.orchestra.run.gibba016.stderr:restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.406 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Listen and drop on 0 v6wildcard [::]:123 2021-11-21T21:51:57.407 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-11-21T21:51:57.407 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Listen normally on 2 lo 127.0.0.1:123 2021-11-21T21:51:57.407 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Listen normally on 3 enp1s0f0 172.21.2.116:123 2021-11-21T21:51:57.408 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Listen normally on 4 lo [::1]:123 2021-11-21T21:51:57.408 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8de6%4]:123 2021-11-21T21:51:57.409 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:51:57 ntpd[9919]: Listening on routing socket on fd #22 for interface updates 2021-11-21T21:51:57.410 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-11-21T21:51:57.411 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Command line: ntpd -gq 2021-11-21T21:51:57.411 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: proto: precision = 0.036 usec (-25) 2021-11-21T21:51:57.412 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.412 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.413 INFO:teuthology.orchestra.run.gibba017.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.413 INFO:teuthology.orchestra.run.gibba017.stderr:restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.415 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Listen and drop on 0 v6wildcard [::]:123 2021-11-21T21:51:57.415 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-11-21T21:51:57.416 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Listen normally on 2 lo 127.0.0.1:123 2021-11-21T21:51:57.416 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Listen normally on 3 enp1s0f0 172.21.2.117:123 2021-11-21T21:51:57.417 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Listen normally on 4 lo [::1]:123 2021-11-21T21:51:57.417 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:9232%4]:123 2021-11-21T21:51:57.417 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:51:57 ntpd[9876]: Listening on routing socket on fd #22 for interface updates 2021-11-21T21:51:57.422 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-11-21T21:51:57.422 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Command line: ntpd -gq 2021-11-21T21:51:57.423 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: proto: precision = 0.047 usec (-24) 2021-11-21T21:51:57.423 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.424 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.424 INFO:teuthology.orchestra.run.gibba019.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.425 INFO:teuthology.orchestra.run.gibba019.stderr:restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.427 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Listen and drop on 0 v6wildcard [::]:123 2021-11-21T21:51:57.427 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-11-21T21:51:57.428 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Listen normally on 2 lo 127.0.0.1:123 2021-11-21T21:51:57.428 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Listen normally on 3 enp1s0f0 172.21.2.119:123 2021-11-21T21:51:57.429 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Listen normally on 4 lo [::1]:123 2021-11-21T21:51:57.429 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8c52%4]:123 2021-11-21T21:51:57.429 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:51:57 ntpd[9877]: Listening on routing socket on fd #22 for interface updates 2021-11-21T21:51:57.431 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-11-21T21:51:57.431 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Command line: ntpd -gq 2021-11-21T21:51:57.432 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: proto: precision = 0.034 usec (-25) 2021-11-21T21:51:57.432 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.433 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.433 INFO:teuthology.orchestra.run.gibba036.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-11-21T21:51:57.434 INFO:teuthology.orchestra.run.gibba036.stderr:restrict ::: KOD does nothing without LIMITED. 2021-11-21T21:51:57.435 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Listen and drop on 0 v6wildcard [::]:123 2021-11-21T21:51:57.436 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-11-21T21:51:57.436 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Listen normally on 2 lo 127.0.0.1:123 2021-11-21T21:51:57.436 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Listen normally on 3 enp1s0f0 172.21.2.136:123 2021-11-21T21:51:57.437 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Listen normally on 4 lo [::1]:123 2021-11-21T21:51:57.437 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8ddc%4]:123 2021-11-21T21:51:57.437 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:51:57 ntpd[9902]: Listening on routing socket on fd #22 for interface updates 2021-11-21T21:51:58.407 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:58 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:58.407 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:58 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:58.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:58 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:58.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:58 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:58.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:58 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:51:58.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:58 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:51:58.423 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:51:58 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:58.424 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:51:58 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:58.424 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:51:58 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:58.424 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:51:58 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:58.424 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:51:58 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:51:58.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:51:58 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:51:59.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:51:59 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:51:59 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:51:59 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:51:59 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:51:59 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:51:59.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:51:59 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:51:59.407 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:59 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.407 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:59 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.407 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:59 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:59 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:59 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:51:59.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:51:59 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:51:59.430 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:51:59.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:51:59.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:51:59.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:51:59.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:51:59.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:51:59 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:00 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:00 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:00 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:00 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:00 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:00 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:00 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.429 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.429 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.431 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:00 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:00.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:00 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:00 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:00 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:00.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:00 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:00.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:00 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:00.435 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:00 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:01.403 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:01 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.544 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:01 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.545 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:01 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.545 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:01 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.545 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:01 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:01.545 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:01 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:01.549 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.550 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.550 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.550 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.550 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.551 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.551 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.551 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.551 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:01.551 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:01.552 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.552 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.552 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:01.552 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:01.553 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.553 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.553 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:01.553 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:01 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:01.554 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.554 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.554 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.554 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.555 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.555 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.555 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.555 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.555 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:01.556 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:01.556 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.556 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.556 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:01.557 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:01.557 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:01.557 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:01.557 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:01.557 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:01 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.409 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:02 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:02 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:02 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:02 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:02 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:02 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:02 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.427 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.430 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.431 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.431 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.431 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.431 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.431 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.432 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.432 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:02 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:02.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:02 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:02 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:02 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:02.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:02 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:02.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:02 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:02.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:02 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:03.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:03 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:03 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:03 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:03 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:03 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:03.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:03 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:03.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.408 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:03.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:03.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:03.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:03.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.411 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:03.412 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:03 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:03.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:03.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:03.433 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:03.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:03.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:03.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:03.435 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:03.435 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:03 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:04.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:04.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:04.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:04.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:04.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:04.408 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:04 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:04.409 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:04.410 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:04.411 INFO:teuthology.orchestra.run.gibba017.stdout:21 Nov 21:52:04 ntpd[9876]: ntpd: time set -0.389788 s 2021-11-21T21:52:04.411 INFO:teuthology.orchestra.run.gibba017.stdout:ntpd: time set -0.389788s 2021-11-21T21:52:04.412 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-11-21T21:52:04.412 INFO:teuthology.orchestra.run.gibba017.stderr:21 Nov 21:52:04 ntpd[9876]: can't open /var/log/ntpstats/loopstats.20211121: Permission denied 2021-11-21T21:52:04.424 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.424 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.424 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.425 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:04.426 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:04.427 INFO:teuthology.orchestra.run.gibba019.stdout:21 Nov 21:52:04 ntpd[9877]: ntpd: time set +0.239891 s 2021-11-21T21:52:04.427 INFO:teuthology.orchestra.run.gibba019.stdout:ntpd: time set +0.239891s 2021-11-21T21:52:04.428 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-11-21T21:52:04.429 INFO:teuthology.orchestra.run.gibba019.stderr:21 Nov 21:52:04 ntpd[9877]: can't open /var/log/ntpstats/loopstats.20211121: Permission denied 2021-11-21T21:52:04.429 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:04 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.430 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:04 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.430 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:04 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:04.430 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:04 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:04.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:04 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:04.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:04 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:04.474 INFO:teuthology.orchestra.run.gibba019.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T21:52:04.474 INFO:teuthology.orchestra.run.gibba019.stdout:============================================================================== 2021-11-21T21:52:04.475 INFO:teuthology.orchestra.run.gibba019.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:04.475 INFO:teuthology.orchestra.run.gibba019.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:04.475 INFO:teuthology.orchestra.run.gibba019.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:04.475 INFO:teuthology.orchestra.run.gibba019.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:04.523 INFO:teuthology.orchestra.run.gibba017.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T21:52:04.523 INFO:teuthology.orchestra.run.gibba017.stdout:============================================================================== 2021-11-21T21:52:04.523 INFO:teuthology.orchestra.run.gibba017.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:04.524 INFO:teuthology.orchestra.run.gibba017.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:04.524 INFO:teuthology.orchestra.run.gibba017.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:04.524 INFO:teuthology.orchestra.run.gibba017.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:05.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:05.404 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:05.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:05.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:05.405 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:05.406 INFO:teuthology.orchestra.run.gibba016.stdout:21 Nov 21:52:05 ntpd[9919]: ntpd: time set +0.233477 s 2021-11-21T21:52:05.406 INFO:teuthology.orchestra.run.gibba016.stdout:ntpd: time set +0.233477s 2021-11-21T21:52:05.406 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-11-21T21:52:05.407 INFO:teuthology.orchestra.run.gibba016.stderr:21 Nov 21:52:05 ntpd[9919]: can't open /var/log/ntpstats/loopstats.20211121: Permission denied 2021-11-21T21:52:05.430 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:05.430 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:05.430 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:05.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:05.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:05.431 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:05.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-11-21T21:52:05.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: can't open /var/log/ntpstats/rawstats.20211121: Permission denied 2021-11-21T21:52:05.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-11-21T21:52:05.432 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: can't open /var/log/ntpstats/peerstats.20211121: Permission denied 2021-11-21T21:52:05.433 INFO:teuthology.orchestra.run.gibba036.stdout:21 Nov 21:52:05 ntpd[9902]: ntpd: time set -0.212718 s 2021-11-21T21:52:05.433 INFO:teuthology.orchestra.run.gibba036.stdout:ntpd: time set -0.212718s 2021-11-21T21:52:05.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-11-21T21:52:05.434 INFO:teuthology.orchestra.run.gibba036.stderr:21 Nov 21:52:05 ntpd[9902]: can't open /var/log/ntpstats/loopstats.20211121: Permission denied 2021-11-21T21:52:05.449 INFO:teuthology.orchestra.run.gibba016.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T21:52:05.449 INFO:teuthology.orchestra.run.gibba016.stdout:============================================================================== 2021-11-21T21:52:05.449 INFO:teuthology.orchestra.run.gibba016.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.450 INFO:teuthology.orchestra.run.gibba016.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.450 INFO:teuthology.orchestra.run.gibba016.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.450 INFO:teuthology.orchestra.run.gibba016.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.528 INFO:teuthology.orchestra.run.gibba036.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T21:52:05.528 INFO:teuthology.orchestra.run.gibba036.stdout:============================================================================== 2021-11-21T21:52:05.528 INFO:teuthology.orchestra.run.gibba036.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.528 INFO:teuthology.orchestra.run.gibba036.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.529 INFO:teuthology.orchestra.run.gibba036.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.529 INFO:teuthology.orchestra.run.gibba036.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-11-21T21:52:05.529 INFO:teuthology.run_tasks:Running task install... 2021-11-21T21:52:05.541 DEBUG:teuthology.task.install:project ceph 2021-11-21T21:52:05.542 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332'}} 2021-11-21T21:52:05.542 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332'} 2021-11-21T21:52:05.542 INFO:teuthology.task.install:Using flavor: default 2021-11-21T21:52:05.549 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-11-21T21:52:05.550 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2021-11-21T21:52:05.550 DEBUG:teuthology.orchestra.run.gibba016:> sudo lsb_release -is 2021-11-21T21:52:05.577 INFO:teuthology.orchestra.run.gibba016.stdout:Ubuntu 2021-11-21T21:52:05.580 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.580 DEBUG:teuthology.orchestra.run.gibba017:> sudo lsb_release -is 2021-11-21T21:52:05.581 DEBUG:teuthology.orchestra.run.gibba016:> sudo apt-key list | grep Ceph 2021-11-21T21:52:05.607 INFO:teuthology.orchestra.run.gibba017.stdout:Ubuntu 2021-11-21T21:52:05.610 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.611 DEBUG:teuthology.orchestra.run.gibba019:> sudo lsb_release -is 2021-11-21T21:52:05.612 DEBUG:teuthology.orchestra.run.gibba017:> sudo apt-key list | grep Ceph 2021-11-21T21:52:05.615 INFO:teuthology.orchestra.run.gibba016.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-11-21T21:52:05.639 INFO:teuthology.orchestra.run.gibba019.stdout:Ubuntu 2021-11-21T21:52:05.642 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.643 DEBUG:teuthology.orchestra.run.gibba036:> sudo lsb_release -is 2021-11-21T21:52:05.644 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-key list | grep Ceph 2021-11-21T21:52:05.646 INFO:teuthology.orchestra.run.gibba016.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-11-21T21:52:05.647 INFO:teuthology.orchestra.run.gibba016.stdout:uid [ unknown] Ceph.com (release key) 2021-11-21T21:52:05.649 DEBUG:teuthology.orchestra.run.gibba016:> sudo lsb_release -is 2021-11-21T21:52:05.667 INFO:teuthology.orchestra.run.gibba017.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-11-21T21:52:05.676 INFO:teuthology.orchestra.run.gibba036.stdout:Ubuntu 2021-11-21T21:52:05.678 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.679 DEBUG:teuthology.orchestra.run.gibba036:> sudo apt-key list | grep Ceph 2021-11-21T21:52:05.705 INFO:teuthology.orchestra.run.gibba017.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-11-21T21:52:05.706 INFO:teuthology.orchestra.run.gibba017.stdout:uid [ unknown] Ceph.com (release key) 2021-11-21T21:52:05.706 DEBUG:teuthology.orchestra.run.gibba017:> sudo lsb_release -is 2021-11-21T21:52:05.742 INFO:teuthology.orchestra.run.gibba016.stdout:Ubuntu 2021-11-21T21:52:05.746 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.747 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-11-21T21:52:05.747 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-11-21T21:52:05.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:52:05.756 INFO:teuthology.orchestra.run.gibba019.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-11-21T21:52:05.758 INFO:teuthology.orchestra.run.gibba017.stdout:Ubuntu 2021-11-21T21:52:05.758 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.759 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-11-21T21:52:05.759 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-11-21T21:52:05.760 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:52:05.772 INFO:teuthology.orchestra.run.gibba019.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-11-21T21:52:05.773 INFO:teuthology.orchestra.run.gibba019.stdout:uid [ unknown] Ceph.com (release key) 2021-11-21T21:52:05.773 DEBUG:teuthology.orchestra.run.gibba019:> sudo lsb_release -is 2021-11-21T21:52:05.803 INFO:teuthology.orchestra.run.gibba036.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-11-21T21:52:05.813 INFO:teuthology.orchestra.run.gibba019.stdout:Ubuntu 2021-11-21T21:52:05.814 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.814 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-11-21T21:52:05.814 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-11-21T21:52:05.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:52:05.833 INFO:teuthology.orchestra.run.gibba036.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-11-21T21:52:05.834 INFO:teuthology.orchestra.run.gibba036.stdout:uid [ unknown] Ceph.com (release key) 2021-11-21T21:52:05.835 DEBUG:teuthology.orchestra.run.gibba036:> sudo lsb_release -is 2021-11-21T21:52:05.921 INFO:teuthology.orchestra.run.gibba036.stdout:Ubuntu 2021-11-21T21:52:05.923 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:52:05.924 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-11-21T21:52:05.924 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-11-21T21:52:05.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:52:05.994 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default/ 2021-11-21T21:52:05.995 INFO:teuthology.task.install.deb:Package version is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:06.019 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default/ 2021-11-21T21:52:06.020 INFO:teuthology.task.install.deb:Package version is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:06.034 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default/ 2021-11-21T21:52:06.034 INFO:teuthology.task.install.deb:Package version is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:06.163 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default/ 2021-11-21T21:52:06.163 INFO:teuthology.task.install.deb:Package version is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:06.184 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:52:06.184 DEBUG:teuthology.orchestra.run.gibba016:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-11-21T21:52:06.189 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:52:06.190 DEBUG:teuthology.orchestra.run.gibba017:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-11-21T21:52:06.204 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:52:06.205 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-11-21T21:52:06.222 DEBUG:teuthology.orchestra.run.gibba016:> sudo apt-get update 2021-11-21T21:52:06.223 DEBUG:teuthology.orchestra.run.gibba017:> sudo apt-get update 2021-11-21T21:52:06.235 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get update 2021-11-21T21:52:06.329 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:52:06.345 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:52:06.346 DEBUG:teuthology.orchestra.run.gibba036:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-11-21T21:52:06.360 DEBUG:teuthology.orchestra.run.gibba036:> sudo apt-get update 2021-11-21T21:52:06.432 INFO:teuthology.orchestra.run.gibba016.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal InRelease 2021-11-21T21:52:06.445 INFO:teuthology.orchestra.run.gibba019.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal InRelease 2021-11-21T21:52:06.446 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:52:06.446 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:52:06.449 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:52:06.465 INFO:teuthology.orchestra.run.gibba017.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal InRelease 2021-11-21T21:52:06.465 INFO:teuthology.orchestra.run.gibba016.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-11-21T21:52:06.482 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-11-21T21:52:06.494 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:52:06.494 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:52:06.499 INFO:teuthology.orchestra.run.gibba017.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-11-21T21:52:06.499 INFO:teuthology.orchestra.run.gibba016.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release.gpg 2021-11-21T21:52:06.519 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:52:06.520 INFO:teuthology.orchestra.run.gibba019.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release.gpg 2021-11-21T21:52:06.531 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:52:06.533 INFO:teuthology.orchestra.run.gibba016.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main i386 Packages [3,065 B] 2021-11-21T21:52:06.533 INFO:teuthology.orchestra.run.gibba017.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release.gpg 2021-11-21T21:52:06.547 INFO:teuthology.orchestra.run.gibba036.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal InRelease 2021-11-21T21:52:06.557 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main i386 Packages [3,065 B] 2021-11-21T21:52:06.574 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-11-21T21:52:06.574 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-11-21T21:52:06.575 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:52:06.576 INFO:teuthology.orchestra.run.gibba017.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-11-21T21:52:06.581 INFO:teuthology.orchestra.run.gibba036.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-11-21T21:52:06.584 INFO:teuthology.orchestra.run.gibba016.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-11-21T21:52:06.600 INFO:teuthology.orchestra.run.gibba016.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:52:06.613 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:52:06.616 INFO:teuthology.orchestra.run.gibba036.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal Release.gpg 2021-11-21T21:52:06.618 INFO:teuthology.orchestra.run.gibba017.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main i386 Packages [3,065 B] 2021-11-21T21:52:06.622 INFO:teuthology.orchestra.run.gibba019.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-11-21T21:52:06.650 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-11-21T21:52:06.650 INFO:teuthology.orchestra.run.gibba036.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main i386 Packages [3,065 B] 2021-11-21T21:52:06.651 INFO:teuthology.orchestra.run.gibba017.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:52:06.705 INFO:teuthology.orchestra.run.gibba036.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-11-21T21:52:06.731 INFO:teuthology.orchestra.run.gibba036.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-11-21T21:52:06.817 INFO:teuthology.orchestra.run.gibba016.stdout:Fetched 26.5 kB in 1s (53.0 kB/s) 2021-11-21T21:52:06.922 INFO:teuthology.orchestra.run.gibba017.stdout:Fetched 26.5 kB in 1s (52.4 kB/s) 2021-11-21T21:52:06.993 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 26.5 kB in 1s (42.8 kB/s) 2021-11-21T21:52:07.034 INFO:teuthology.orchestra.run.gibba036.stdout:Fetched 26.5 kB in 1s (49.1 kB/s) 2021-11-21T21:52:07.340 INFO:teuthology.orchestra.run.gibba016.stdout:Reading package lists... 2021-11-21T21:52:07.350 DEBUG:teuthology.orchestra.run.gibba016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.15-36-g911bc85f-1focal cephadm=15.2.15-36-g911bc85f-1focal ceph-mds=15.2.15-36-g911bc85f-1focal ceph-mgr=15.2.15-36-g911bc85f-1focal ceph-common=15.2.15-36-g911bc85f-1focal ceph-fuse=15.2.15-36-g911bc85f-1focal ceph-test=15.2.15-36-g911bc85f-1focal radosgw=15.2.15-36-g911bc85f-1focal python3-rados=15.2.15-36-g911bc85f-1focal python3-rgw=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal python3-rbd=15.2.15-36-g911bc85f-1focal libcephfs2=15.2.15-36-g911bc85f-1focal libcephfs-dev=15.2.15-36-g911bc85f-1focal librados2=15.2.15-36-g911bc85f-1focal librbd1=15.2.15-36-g911bc85f-1focal rbd-fuse=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal cephfs-shell=15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:07.386 INFO:teuthology.orchestra.run.gibba016.stdout:Reading package lists... 2021-11-21T21:52:07.438 INFO:teuthology.orchestra.run.gibba017.stdout:Reading package lists... 2021-11-21T21:52:07.449 DEBUG:teuthology.orchestra.run.gibba017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.15-36-g911bc85f-1focal cephadm=15.2.15-36-g911bc85f-1focal ceph-mds=15.2.15-36-g911bc85f-1focal ceph-mgr=15.2.15-36-g911bc85f-1focal ceph-common=15.2.15-36-g911bc85f-1focal ceph-fuse=15.2.15-36-g911bc85f-1focal ceph-test=15.2.15-36-g911bc85f-1focal radosgw=15.2.15-36-g911bc85f-1focal python3-rados=15.2.15-36-g911bc85f-1focal python3-rgw=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal python3-rbd=15.2.15-36-g911bc85f-1focal libcephfs2=15.2.15-36-g911bc85f-1focal libcephfs-dev=15.2.15-36-g911bc85f-1focal librados2=15.2.15-36-g911bc85f-1focal librbd1=15.2.15-36-g911bc85f-1focal rbd-fuse=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal cephfs-shell=15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:07.485 INFO:teuthology.orchestra.run.gibba017.stdout:Reading package lists... 2021-11-21T21:52:07.486 INFO:teuthology.orchestra.run.gibba016.stdout:Building dependency tree... 2021-11-21T21:52:07.487 INFO:teuthology.orchestra.run.gibba016.stdout:Reading state information... 2021-11-21T21:52:07.513 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-11-21T21:52:07.525 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.15-36-g911bc85f-1focal cephadm=15.2.15-36-g911bc85f-1focal ceph-mds=15.2.15-36-g911bc85f-1focal ceph-mgr=15.2.15-36-g911bc85f-1focal ceph-common=15.2.15-36-g911bc85f-1focal ceph-fuse=15.2.15-36-g911bc85f-1focal ceph-test=15.2.15-36-g911bc85f-1focal radosgw=15.2.15-36-g911bc85f-1focal python3-rados=15.2.15-36-g911bc85f-1focal python3-rgw=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal python3-rbd=15.2.15-36-g911bc85f-1focal libcephfs2=15.2.15-36-g911bc85f-1focal libcephfs-dev=15.2.15-36-g911bc85f-1focal librados2=15.2.15-36-g911bc85f-1focal librbd1=15.2.15-36-g911bc85f-1focal rbd-fuse=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal cephfs-shell=15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:07.561 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-11-21T21:52:07.563 INFO:teuthology.orchestra.run.gibba036.stdout:Reading package lists... 2021-11-21T21:52:07.565 INFO:teuthology.orchestra.run.gibba016.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:52:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:52:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:52:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:52:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:52:07.567 INFO:teuthology.orchestra.run.gibba016.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:52:07.567 INFO:teuthology.orchestra.run.gibba016.stdout:The following additional packages will be installed: 2021-11-21T21:52:07.567 INFO:teuthology.orchestra.run.gibba016.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.567 INFO:teuthology.orchestra.run.gibba016.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.568 INFO:teuthology.orchestra.run.gibba016.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-11-21T21:52:07.568 INFO:teuthology.orchestra.run.gibba016.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-11-21T21:52:07.568 INFO:teuthology.orchestra.run.gibba016.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-11-21T21:52:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-11-21T21:52:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-11-21T21:52:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-11-21T21:52:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-11-21T21:52:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-11-21T21:52:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-11-21T21:52:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-11-21T21:52:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-11-21T21:52:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-11-21T21:52:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-11-21T21:52:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-11-21T21:52:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-11-21T21:52:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-11-21T21:52:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-11-21T21:52:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: xmlstarlet 2021-11-21T21:52:07.572 INFO:teuthology.orchestra.run.gibba016.stdout:Suggested packages: 2021-11-21T21:52:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-11-21T21:52:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-11-21T21:52:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-11-21T21:52:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-11-21T21:52:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-11-21T21:52:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-11-21T21:52:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-11-21T21:52:07.574 INFO:teuthology.orchestra.run.gibba016.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-11-21T21:52:07.574 INFO:teuthology.orchestra.run.gibba016.stdout:Recommended packages: 2021-11-21T21:52:07.574 INFO:teuthology.orchestra.run.gibba016.stdout: btrfs-tools 2021-11-21T21:52:07.575 DEBUG:teuthology.orchestra.run.gibba036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.15-36-g911bc85f-1focal cephadm=15.2.15-36-g911bc85f-1focal ceph-mds=15.2.15-36-g911bc85f-1focal ceph-mgr=15.2.15-36-g911bc85f-1focal ceph-common=15.2.15-36-g911bc85f-1focal ceph-fuse=15.2.15-36-g911bc85f-1focal ceph-test=15.2.15-36-g911bc85f-1focal radosgw=15.2.15-36-g911bc85f-1focal python3-rados=15.2.15-36-g911bc85f-1focal python3-rgw=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal python3-rbd=15.2.15-36-g911bc85f-1focal libcephfs2=15.2.15-36-g911bc85f-1focal libcephfs-dev=15.2.15-36-g911bc85f-1focal librados2=15.2.15-36-g911bc85f-1focal librbd1=15.2.15-36-g911bc85f-1focal rbd-fuse=15.2.15-36-g911bc85f-1focal python3-cephfs=15.2.15-36-g911bc85f-1focal cephfs-shell=15.2.15-36-g911bc85f-1focal 2021-11-21T21:52:07.585 INFO:teuthology.orchestra.run.gibba017.stdout:Building dependency tree... 2021-11-21T21:52:07.586 INFO:teuthology.orchestra.run.gibba017.stdout:Reading state information... 2021-11-21T21:52:07.612 INFO:teuthology.orchestra.run.gibba036.stdout:Reading package lists... 2021-11-21T21:52:07.616 INFO:teuthology.orchestra.run.gibba016.stdout:The following NEW packages will be installed: 2021-11-21T21:52:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-11-21T21:52:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-11-21T21:52:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-11-21T21:52:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-11-21T21:52:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-11-21T21:52:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-11-21T21:52:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-11-21T21:52:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-11-21T21:52:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-11-21T21:52:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-11-21T21:52:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-11-21T21:52:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-11-21T21:52:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-11-21T21:52:07.620 INFO:teuthology.orchestra.run.gibba016.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-11-21T21:52:07.620 INFO:teuthology.orchestra.run.gibba016.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-11-21T21:52:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-11-21T21:52:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-11-21T21:52:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-11-21T21:52:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-11-21T21:52:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-11-21T21:52:07.622 INFO:teuthology.orchestra.run.gibba016.stdout:The following packages will be upgraded: 2021-11-21T21:52:07.622 INFO:teuthology.orchestra.run.gibba016.stdout: librados2 librbd1 2021-11-21T21:52:07.663 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-11-21T21:52:07.664 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-11-21T21:52:07.665 INFO:teuthology.orchestra.run.gibba017.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:52:07.666 INFO:teuthology.orchestra.run.gibba017.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:52:07.666 INFO:teuthology.orchestra.run.gibba017.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:52:07.667 INFO:teuthology.orchestra.run.gibba017.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:52:07.667 INFO:teuthology.orchestra.run.gibba017.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:52:07.667 INFO:teuthology.orchestra.run.gibba017.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:52:07.667 INFO:teuthology.orchestra.run.gibba017.stdout:The following additional packages will be installed: 2021-11-21T21:52:07.667 INFO:teuthology.orchestra.run.gibba017.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.668 INFO:teuthology.orchestra.run.gibba017.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.668 INFO:teuthology.orchestra.run.gibba017.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-11-21T21:52:07.668 INFO:teuthology.orchestra.run.gibba017.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-11-21T21:52:07.668 INFO:teuthology.orchestra.run.gibba017.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-11-21T21:52:07.669 INFO:teuthology.orchestra.run.gibba017.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-11-21T21:52:07.669 INFO:teuthology.orchestra.run.gibba017.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-11-21T21:52:07.669 INFO:teuthology.orchestra.run.gibba017.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-11-21T21:52:07.669 INFO:teuthology.orchestra.run.gibba017.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-11-21T21:52:07.669 INFO:teuthology.orchestra.run.gibba017.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-11-21T21:52:07.670 INFO:teuthology.orchestra.run.gibba017.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-11-21T21:52:07.670 INFO:teuthology.orchestra.run.gibba017.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-11-21T21:52:07.670 INFO:teuthology.orchestra.run.gibba017.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-11-21T21:52:07.670 INFO:teuthology.orchestra.run.gibba017.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-11-21T21:52:07.671 INFO:teuthology.orchestra.run.gibba017.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-11-21T21:52:07.671 INFO:teuthology.orchestra.run.gibba017.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-11-21T21:52:07.671 INFO:teuthology.orchestra.run.gibba017.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-11-21T21:52:07.672 INFO:teuthology.orchestra.run.gibba017.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-11-21T21:52:07.672 INFO:teuthology.orchestra.run.gibba017.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-11-21T21:52:07.673 INFO:teuthology.orchestra.run.gibba017.stdout: xmlstarlet 2021-11-21T21:52:07.673 INFO:teuthology.orchestra.run.gibba017.stdout:Suggested packages: 2021-11-21T21:52:07.673 INFO:teuthology.orchestra.run.gibba017.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-11-21T21:52:07.673 INFO:teuthology.orchestra.run.gibba017.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-11-21T21:52:07.674 INFO:teuthology.orchestra.run.gibba017.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-11-21T21:52:07.674 INFO:teuthology.orchestra.run.gibba017.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-11-21T21:52:07.674 INFO:teuthology.orchestra.run.gibba017.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-11-21T21:52:07.674 INFO:teuthology.orchestra.run.gibba017.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-11-21T21:52:07.674 INFO:teuthology.orchestra.run.gibba017.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-11-21T21:52:07.675 INFO:teuthology.orchestra.run.gibba017.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-11-21T21:52:07.675 INFO:teuthology.orchestra.run.gibba017.stdout:Recommended packages: 2021-11-21T21:52:07.675 INFO:teuthology.orchestra.run.gibba017.stdout: btrfs-tools 2021-11-21T21:52:07.717 INFO:teuthology.orchestra.run.gibba017.stdout:The following NEW packages will be installed: 2021-11-21T21:52:07.717 INFO:teuthology.orchestra.run.gibba017.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-11-21T21:52:07.717 INFO:teuthology.orchestra.run.gibba017.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.718 INFO:teuthology.orchestra.run.gibba017.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.718 INFO:teuthology.orchestra.run.gibba017.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-11-21T21:52:07.718 INFO:teuthology.orchestra.run.gibba017.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-11-21T21:52:07.718 INFO:teuthology.orchestra.run.gibba017.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-11-21T21:52:07.719 INFO:teuthology.orchestra.run.gibba017.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-11-21T21:52:07.719 INFO:teuthology.orchestra.run.gibba017.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-11-21T21:52:07.719 INFO:teuthology.orchestra.run.gibba017.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-11-21T21:52:07.719 INFO:teuthology.orchestra.run.gibba017.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-11-21T21:52:07.720 INFO:teuthology.orchestra.run.gibba017.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-11-21T21:52:07.720 INFO:teuthology.orchestra.run.gibba017.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-11-21T21:52:07.720 INFO:teuthology.orchestra.run.gibba017.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-11-21T21:52:07.720 INFO:teuthology.orchestra.run.gibba017.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-11-21T21:52:07.720 INFO:teuthology.orchestra.run.gibba017.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-11-21T21:52:07.721 INFO:teuthology.orchestra.run.gibba017.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-11-21T21:52:07.721 INFO:teuthology.orchestra.run.gibba017.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-11-21T21:52:07.721 INFO:teuthology.orchestra.run.gibba017.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-11-21T21:52:07.721 INFO:teuthology.orchestra.run.gibba017.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-11-21T21:52:07.722 INFO:teuthology.orchestra.run.gibba017.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-11-21T21:52:07.722 INFO:teuthology.orchestra.run.gibba017.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-11-21T21:52:07.722 INFO:teuthology.orchestra.run.gibba017.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-11-21T21:52:07.722 INFO:teuthology.orchestra.run.gibba017.stdout:The following packages will be upgraded: 2021-11-21T21:52:07.723 INFO:teuthology.orchestra.run.gibba017.stdout: librados2 librbd1 2021-11-21T21:52:07.723 INFO:teuthology.orchestra.run.gibba036.stdout:Building dependency tree... 2021-11-21T21:52:07.724 INFO:teuthology.orchestra.run.gibba036.stdout:Reading state information... 2021-11-21T21:52:07.744 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:52:07.744 INFO:teuthology.orchestra.run.gibba019.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:52:07.745 INFO:teuthology.orchestra.run.gibba019.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:52:07.745 INFO:teuthology.orchestra.run.gibba019.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:52:07.746 INFO:teuthology.orchestra.run.gibba019.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:52:07.746 INFO:teuthology.orchestra.run.gibba019.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:52:07.746 INFO:teuthology.orchestra.run.gibba019.stdout:The following additional packages will be installed: 2021-11-21T21:52:07.746 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.747 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.747 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-11-21T21:52:07.748 INFO:teuthology.orchestra.run.gibba019.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-11-21T21:52:07.748 INFO:teuthology.orchestra.run.gibba019.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-11-21T21:52:07.748 INFO:teuthology.orchestra.run.gibba019.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-11-21T21:52:07.749 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-11-21T21:52:07.749 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-11-21T21:52:07.749 INFO:teuthology.orchestra.run.gibba019.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-11-21T21:52:07.749 INFO:teuthology.orchestra.run.gibba019.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-11-21T21:52:07.750 INFO:teuthology.orchestra.run.gibba019.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-11-21T21:52:07.750 INFO:teuthology.orchestra.run.gibba019.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-11-21T21:52:07.750 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-11-21T21:52:07.750 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-11-21T21:52:07.751 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-11-21T21:52:07.751 INFO:teuthology.orchestra.run.gibba019.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-11-21T21:52:07.751 INFO:teuthology.orchestra.run.gibba019.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-11-21T21:52:07.751 INFO:teuthology.orchestra.run.gibba019.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-11-21T21:52:07.752 INFO:teuthology.orchestra.run.gibba019.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-11-21T21:52:07.752 INFO:teuthology.orchestra.run.gibba019.stdout: xmlstarlet 2021-11-21T21:52:07.752 INFO:teuthology.orchestra.run.gibba019.stdout:Suggested packages: 2021-11-21T21:52:07.753 INFO:teuthology.orchestra.run.gibba019.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-11-21T21:52:07.753 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-11-21T21:52:07.753 INFO:teuthology.orchestra.run.gibba019.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-11-21T21:52:07.753 INFO:teuthology.orchestra.run.gibba019.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-11-21T21:52:07.754 INFO:teuthology.orchestra.run.gibba019.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-11-21T21:52:07.754 INFO:teuthology.orchestra.run.gibba019.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-11-21T21:52:07.754 INFO:teuthology.orchestra.run.gibba019.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-11-21T21:52:07.754 INFO:teuthology.orchestra.run.gibba019.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-11-21T21:52:07.754 INFO:teuthology.orchestra.run.gibba019.stdout:Recommended packages: 2021-11-21T21:52:07.755 INFO:teuthology.orchestra.run.gibba019.stdout: btrfs-tools 2021-11-21T21:52:07.792 INFO:teuthology.orchestra.run.gibba016.stdout:2 upgraded, 101 newly installed, 0 to remove and 52 not upgraded. 2021-11-21T21:52:07.792 INFO:teuthology.orchestra.run.gibba016.stdout:Need to get 153 MB of archives. 2021-11-21T21:52:07.792 INFO:teuthology.orchestra.run.gibba016.stdout:After this operation, 698 MB of additional disk space will be used. 2021-11-21T21:52:07.793 INFO:teuthology.orchestra.run.gibba016.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-11-21T21:52:07.797 INFO:teuthology.orchestra.run.gibba019.stdout:The following NEW packages will be installed: 2021-11-21T21:52:07.797 INFO:teuthology.orchestra.run.gibba019.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-11-21T21:52:07.797 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.797 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.798 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-11-21T21:52:07.798 INFO:teuthology.orchestra.run.gibba019.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-11-21T21:52:07.798 INFO:teuthology.orchestra.run.gibba019.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-11-21T21:52:07.798 INFO:teuthology.orchestra.run.gibba019.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-11-21T21:52:07.799 INFO:teuthology.orchestra.run.gibba019.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-11-21T21:52:07.799 INFO:teuthology.orchestra.run.gibba019.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-11-21T21:52:07.799 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-11-21T21:52:07.799 INFO:teuthology.orchestra.run.gibba019.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-11-21T21:52:07.799 INFO:teuthology.orchestra.run.gibba019.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-11-21T21:52:07.800 INFO:teuthology.orchestra.run.gibba019.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-11-21T21:52:07.800 INFO:teuthology.orchestra.run.gibba019.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-11-21T21:52:07.800 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-11-21T21:52:07.800 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-11-21T21:52:07.801 INFO:teuthology.orchestra.run.gibba019.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-11-21T21:52:07.801 INFO:teuthology.orchestra.run.gibba019.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-11-21T21:52:07.801 INFO:teuthology.orchestra.run.gibba019.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-11-21T21:52:07.801 INFO:teuthology.orchestra.run.gibba019.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-11-21T21:52:07.801 INFO:teuthology.orchestra.run.gibba019.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-11-21T21:52:07.802 INFO:teuthology.orchestra.run.gibba019.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-11-21T21:52:07.802 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages will be upgraded: 2021-11-21T21:52:07.803 INFO:teuthology.orchestra.run.gibba019.stdout: librados2 librbd1 2021-11-21T21:52:07.805 INFO:teuthology.orchestra.run.gibba036.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:52:07.805 INFO:teuthology.orchestra.run.gibba036.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:52:07.806 INFO:teuthology.orchestra.run.gibba036.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:52:07.806 INFO:teuthology.orchestra.run.gibba036.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:52:07.806 INFO:teuthology.orchestra.run.gibba036.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:52:07.806 INFO:teuthology.orchestra.run.gibba036.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:52:07.807 INFO:teuthology.orchestra.run.gibba036.stdout:The following additional packages will be installed: 2021-11-21T21:52:07.807 INFO:teuthology.orchestra.run.gibba036.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.807 INFO:teuthology.orchestra.run.gibba036.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.807 INFO:teuthology.orchestra.run.gibba036.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-11-21T21:52:07.808 INFO:teuthology.orchestra.run.gibba036.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-11-21T21:52:07.808 INFO:teuthology.orchestra.run.gibba036.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-11-21T21:52:07.808 INFO:teuthology.orchestra.run.gibba036.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-11-21T21:52:07.808 INFO:teuthology.orchestra.run.gibba036.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-11-21T21:52:07.808 INFO:teuthology.orchestra.run.gibba036.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-11-21T21:52:07.809 INFO:teuthology.orchestra.run.gibba036.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-11-21T21:52:07.809 INFO:teuthology.orchestra.run.gibba036.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-11-21T21:52:07.809 INFO:teuthology.orchestra.run.gibba036.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-11-21T21:52:07.809 INFO:teuthology.orchestra.run.gibba036.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-11-21T21:52:07.810 INFO:teuthology.orchestra.run.gibba036.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-11-21T21:52:07.810 INFO:teuthology.orchestra.run.gibba036.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-11-21T21:52:07.810 INFO:teuthology.orchestra.run.gibba036.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-11-21T21:52:07.810 INFO:teuthology.orchestra.run.gibba036.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-11-21T21:52:07.810 INFO:teuthology.orchestra.run.gibba036.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-11-21T21:52:07.811 INFO:teuthology.orchestra.run.gibba036.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-11-21T21:52:07.811 INFO:teuthology.orchestra.run.gibba036.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-11-21T21:52:07.811 INFO:teuthology.orchestra.run.gibba036.stdout: xmlstarlet 2021-11-21T21:52:07.811 INFO:teuthology.orchestra.run.gibba036.stdout:Suggested packages: 2021-11-21T21:52:07.812 INFO:teuthology.orchestra.run.gibba036.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-11-21T21:52:07.812 INFO:teuthology.orchestra.run.gibba036.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-11-21T21:52:07.812 INFO:teuthology.orchestra.run.gibba036.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-11-21T21:52:07.812 INFO:teuthology.orchestra.run.gibba036.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-11-21T21:52:07.812 INFO:teuthology.orchestra.run.gibba036.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-11-21T21:52:07.813 INFO:teuthology.orchestra.run.gibba036.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-11-21T21:52:07.813 INFO:teuthology.orchestra.run.gibba036.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-11-21T21:52:07.813 INFO:teuthology.orchestra.run.gibba036.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-11-21T21:52:07.813 INFO:teuthology.orchestra.run.gibba036.stdout:Recommended packages: 2021-11-21T21:52:07.814 INFO:teuthology.orchestra.run.gibba036.stdout: btrfs-tools 2021-11-21T21:52:07.814 INFO:teuthology.orchestra.run.gibba016.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.15-36-g911bc85f-1focal [1,694 kB] 2021-11-21T21:52:07.851 INFO:teuthology.orchestra.run.gibba036.stdout:The following NEW packages will be installed: 2021-11-21T21:52:07.852 INFO:teuthology.orchestra.run.gibba036.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-11-21T21:52:07.852 INFO:teuthology.orchestra.run.gibba036.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-11-21T21:52:07.852 INFO:teuthology.orchestra.run.gibba036.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-11-21T21:52:07.853 INFO:teuthology.orchestra.run.gibba036.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-11-21T21:52:07.853 INFO:teuthology.orchestra.run.gibba036.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-11-21T21:52:07.853 INFO:teuthology.orchestra.run.gibba036.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-11-21T21:52:07.854 INFO:teuthology.orchestra.run.gibba036.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-11-21T21:52:07.854 INFO:teuthology.orchestra.run.gibba036.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-11-21T21:52:07.854 INFO:teuthology.orchestra.run.gibba036.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-11-21T21:52:07.854 INFO:teuthology.orchestra.run.gibba036.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-11-21T21:52:07.855 INFO:teuthology.orchestra.run.gibba036.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-11-21T21:52:07.855 INFO:teuthology.orchestra.run.gibba036.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-11-21T21:52:07.855 INFO:teuthology.orchestra.run.gibba036.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-11-21T21:52:07.855 INFO:teuthology.orchestra.run.gibba036.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-11-21T21:52:07.855 INFO:teuthology.orchestra.run.gibba036.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-11-21T21:52:07.856 INFO:teuthology.orchestra.run.gibba036.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-11-21T21:52:07.856 INFO:teuthology.orchestra.run.gibba036.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-11-21T21:52:07.856 INFO:teuthology.orchestra.run.gibba036.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-11-21T21:52:07.856 INFO:teuthology.orchestra.run.gibba036.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-11-21T21:52:07.857 INFO:teuthology.orchestra.run.gibba036.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-11-21T21:52:07.857 INFO:teuthology.orchestra.run.gibba036.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-11-21T21:52:07.857 INFO:teuthology.orchestra.run.gibba036.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-11-21T21:52:07.857 INFO:teuthology.orchestra.run.gibba036.stdout:The following packages will be upgraded: 2021-11-21T21:52:07.857 INFO:teuthology.orchestra.run.gibba036.stdout: librados2 librbd1 2021-11-21T21:52:07.876 INFO:teuthology.orchestra.run.gibba016.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-11-21T21:52:07.883 INFO:teuthology.orchestra.run.gibba016.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-11-21T21:52:07.892 INFO:teuthology.orchestra.run.gibba017.stdout:2 upgraded, 101 newly installed, 0 to remove and 52 not upgraded. 2021-11-21T21:52:07.892 INFO:teuthology.orchestra.run.gibba017.stdout:Need to get 153 MB of archives. 2021-11-21T21:52:07.893 INFO:teuthology.orchestra.run.gibba017.stdout:After this operation, 698 MB of additional disk space will be used. 2021-11-21T21:52:07.893 INFO:teuthology.orchestra.run.gibba017.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-11-21T21:52:07.904 INFO:teuthology.orchestra.run.gibba017.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.15-36-g911bc85f-1focal [1,694 kB] 2021-11-21T21:52:07.973 INFO:teuthology.orchestra.run.gibba019.stdout:2 upgraded, 101 newly installed, 0 to remove and 52 not upgraded. 2021-11-21T21:52:07.974 INFO:teuthology.orchestra.run.gibba019.stdout:Need to get 153 MB of archives. 2021-11-21T21:52:07.974 INFO:teuthology.orchestra.run.gibba019.stdout:After this operation, 698 MB of additional disk space will be used. 2021-11-21T21:52:07.974 INFO:teuthology.orchestra.run.gibba019.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-11-21T21:52:07.976 INFO:teuthology.orchestra.run.gibba017.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-11-21T21:52:07.991 INFO:teuthology.orchestra.run.gibba016.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-11-21T21:52:08.002 INFO:teuthology.orchestra.run.gibba019.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.15-36-g911bc85f-1focal [1,694 kB] 2021-11-21T21:52:08.007 INFO:teuthology.orchestra.run.gibba017.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-11-21T21:52:08.027 INFO:teuthology.orchestra.run.gibba036.stdout:2 upgraded, 101 newly installed, 0 to remove and 52 not upgraded. 2021-11-21T21:52:08.027 INFO:teuthology.orchestra.run.gibba036.stdout:Need to get 153 MB of archives. 2021-11-21T21:52:08.028 INFO:teuthology.orchestra.run.gibba036.stdout:After this operation, 698 MB of additional disk space will be used. 2021-11-21T21:52:08.028 INFO:teuthology.orchestra.run.gibba036.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-11-21T21:52:08.042 INFO:teuthology.orchestra.run.gibba036.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.15-36-g911bc85f-1focal [1,694 kB] 2021-11-21T21:52:08.058 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-11-21T21:52:08.059 INFO:teuthology.orchestra.run.gibba016.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.15-36-g911bc85f-1focal [3,247 kB] 2021-11-21T21:52:08.086 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-11-21T21:52:08.112 INFO:teuthology.orchestra.run.gibba036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-11-21T21:52:08.118 INFO:teuthology.orchestra.run.gibba017.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-11-21T21:52:08.141 INFO:teuthology.orchestra.run.gibba036.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-11-21T21:52:08.148 INFO:teuthology.orchestra.run.gibba016.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-11-21T21:52:08.149 INFO:teuthology.orchestra.run.gibba016.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-11-21T21:52:08.151 INFO:teuthology.orchestra.run.gibba016.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-11-21T21:52:08.189 INFO:teuthology.orchestra.run.gibba017.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.15-36-g911bc85f-1focal [3,247 kB] 2021-11-21T21:52:08.202 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-11-21T21:52:08.237 INFO:teuthology.orchestra.run.gibba016.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.15-36-g911bc85f-1focal [463 kB] 2021-11-21T21:52:08.251 INFO:teuthology.orchestra.run.gibba017.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-11-21T21:52:08.252 INFO:teuthology.orchestra.run.gibba016.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.15-36-g911bc85f-1focal [365 kB] 2021-11-21T21:52:08.258 INFO:teuthology.orchestra.run.gibba036.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-11-21T21:52:08.260 INFO:teuthology.orchestra.run.gibba017.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-11-21T21:52:08.269 INFO:teuthology.orchestra.run.gibba017.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-11-21T21:52:08.269 INFO:teuthology.orchestra.run.gibba016.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-11-21T21:52:08.299 INFO:teuthology.orchestra.run.gibba016.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-11-21T21:52:08.302 INFO:teuthology.orchestra.run.gibba016.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.15-36-g911bc85f-1focal [20.8 kB] 2021-11-21T21:52:08.303 INFO:teuthology.orchestra.run.gibba016.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.15-36-g911bc85f-1focal [156 kB] 2021-11-21T21:52:08.306 INFO:teuthology.orchestra.run.gibba016.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-11-21T21:52:08.309 INFO:teuthology.orchestra.run.gibba016.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.15-36-g911bc85f-1focal [29.4 kB] 2021-11-21T21:52:08.311 INFO:teuthology.orchestra.run.gibba016.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.15-36-g911bc85f-1focal [362 kB] 2021-11-21T21:52:08.322 INFO:teuthology.orchestra.run.gibba016.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.15-36-g911bc85f-1focal [3,138 kB] 2021-11-21T21:52:08.329 INFO:teuthology.orchestra.run.gibba016.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-11-21T21:52:08.332 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-11-21T21:52:08.333 INFO:teuthology.orchestra.run.gibba016.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-11-21T21:52:08.336 INFO:teuthology.orchestra.run.gibba016.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-11-21T21:52:08.344 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-11-21T21:52:08.353 INFO:teuthology.orchestra.run.gibba016.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-11-21T21:52:08.354 INFO:teuthology.orchestra.run.gibba016.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-11-21T21:52:08.356 INFO:teuthology.orchestra.run.gibba016.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-11-21T21:52:08.359 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-11-21T21:52:08.377 INFO:teuthology.orchestra.run.gibba017.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-11-21T21:52:08.393 INFO:teuthology.orchestra.run.gibba036.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-11-21T21:52:08.400 INFO:teuthology.orchestra.run.gibba016.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-11-21T21:52:08.403 INFO:teuthology.orchestra.run.gibba017.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-11-21T21:52:08.404 INFO:teuthology.orchestra.run.gibba036.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-11-21T21:52:08.404 INFO:teuthology.orchestra.run.gibba016.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-11-21T21:52:08.405 INFO:teuthology.orchestra.run.gibba016.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-11-21T21:52:08.408 INFO:teuthology.orchestra.run.gibba017.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-11-21T21:52:08.416 INFO:teuthology.orchestra.run.gibba036.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-11-21T21:52:08.431 INFO:teuthology.orchestra.run.gibba017.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-11-21T21:52:08.433 INFO:teuthology.orchestra.run.gibba017.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.15-36-g911bc85f-1focal [463 kB] 2021-11-21T21:52:08.436 INFO:teuthology.orchestra.run.gibba017.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-11-21T21:52:08.438 INFO:teuthology.orchestra.run.gibba017.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-11-21T21:52:08.448 INFO:teuthology.orchestra.run.gibba017.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.15-36-g911bc85f-1focal [365 kB] 2021-11-21T21:52:08.453 INFO:teuthology.orchestra.run.gibba017.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-11-21T21:52:08.454 INFO:teuthology.orchestra.run.gibba017.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-11-21T21:52:08.456 INFO:teuthology.orchestra.run.gibba017.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-11-21T21:52:08.463 INFO:teuthology.orchestra.run.gibba019.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-11-21T21:52:08.482 INFO:teuthology.orchestra.run.gibba016.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-11-21T21:52:08.483 INFO:teuthology.orchestra.run.gibba019.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-11-21T21:52:08.485 INFO:teuthology.orchestra.run.gibba016.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-11-21T21:52:08.487 INFO:teuthology.orchestra.run.gibba016.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-11-21T21:52:08.489 INFO:teuthology.orchestra.run.gibba019.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-11-21T21:52:08.491 INFO:teuthology.orchestra.run.gibba016.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-11-21T21:52:08.494 INFO:teuthology.orchestra.run.gibba016.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-11-21T21:52:08.495 INFO:teuthology.orchestra.run.gibba016.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-11-21T21:52:08.501 INFO:teuthology.orchestra.run.gibba017.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-11-21T21:52:08.505 INFO:teuthology.orchestra.run.gibba017.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-11-21T21:52:08.505 INFO:teuthology.orchestra.run.gibba017.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-11-21T21:52:08.511 INFO:teuthology.orchestra.run.gibba017.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.15-36-g911bc85f-1focal [20.8 kB] 2021-11-21T21:52:08.512 INFO:teuthology.orchestra.run.gibba017.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.15-36-g911bc85f-1focal [156 kB] 2021-11-21T21:52:08.513 INFO:teuthology.orchestra.run.gibba019.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-11-21T21:52:08.515 INFO:teuthology.orchestra.run.gibba016.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-11-21T21:52:08.517 INFO:teuthology.orchestra.run.gibba017.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.15-36-g911bc85f-1focal [29.4 kB] 2021-11-21T21:52:08.518 INFO:teuthology.orchestra.run.gibba017.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.15-36-g911bc85f-1focal [362 kB] 2021-11-21T21:52:08.519 INFO:teuthology.orchestra.run.gibba019.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-11-21T21:52:08.520 INFO:teuthology.orchestra.run.gibba036.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-11-21T21:52:08.522 INFO:teuthology.orchestra.run.gibba019.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-11-21T21:52:08.530 INFO:teuthology.orchestra.run.gibba017.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.15-36-g911bc85f-1focal [3,138 kB] 2021-11-21T21:52:08.537 INFO:teuthology.orchestra.run.gibba019.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-11-21T21:52:08.539 INFO:teuthology.orchestra.run.gibba019.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-11-21T21:52:08.541 INFO:teuthology.orchestra.run.gibba036.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-11-21T21:52:08.542 INFO:teuthology.orchestra.run.gibba019.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-11-21T21:52:08.546 INFO:teuthology.orchestra.run.gibba036.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-11-21T21:52:08.555 INFO:teuthology.orchestra.run.gibba016.stdout:Get:36 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.15-36-g911bc85f-1focal [116 kB] 2021-11-21T21:52:08.560 INFO:teuthology.orchestra.run.gibba016.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.15-36-g911bc85f-1focal [361 kB] 2021-11-21T21:52:08.564 INFO:teuthology.orchestra.run.gibba016.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-11-21T21:52:08.565 INFO:teuthology.orchestra.run.gibba016.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-11-21T21:52:08.567 INFO:teuthology.orchestra.run.gibba016.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-11-21T21:52:08.571 INFO:teuthology.orchestra.run.gibba036.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-11-21T21:52:08.575 INFO:teuthology.orchestra.run.gibba016.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.15-36-g911bc85f-1focal [20.0 MB] 2021-11-21T21:52:08.576 INFO:teuthology.orchestra.run.gibba036.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-11-21T21:52:08.580 INFO:teuthology.orchestra.run.gibba036.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-11-21T21:52:08.582 INFO:teuthology.orchestra.run.gibba019.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-11-21T21:52:08.586 INFO:teuthology.orchestra.run.gibba019.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-11-21T21:52:08.587 INFO:teuthology.orchestra.run.gibba019.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-11-21T21:52:08.587 INFO:teuthology.orchestra.run.gibba017.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-11-21T21:52:08.587 INFO:teuthology.orchestra.run.gibba017.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-11-21T21:52:08.588 INFO:teuthology.orchestra.run.gibba017.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-11-21T21:52:08.591 INFO:teuthology.orchestra.run.gibba017.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-11-21T21:52:08.594 INFO:teuthology.orchestra.run.gibba036.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-11-21T21:52:08.595 INFO:teuthology.orchestra.run.gibba017.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-11-21T21:52:08.596 INFO:teuthology.orchestra.run.gibba036.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-11-21T21:52:08.596 INFO:teuthology.orchestra.run.gibba017.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-11-21T21:52:08.598 INFO:teuthology.orchestra.run.gibba036.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-11-21T21:52:08.614 INFO:teuthology.orchestra.run.gibba017.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-11-21T21:52:08.643 INFO:teuthology.orchestra.run.gibba036.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-11-21T21:52:08.646 INFO:teuthology.orchestra.run.gibba036.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-11-21T21:52:08.647 INFO:teuthology.orchestra.run.gibba016.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-11-21T21:52:08.647 INFO:teuthology.orchestra.run.gibba036.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-11-21T21:52:08.652 INFO:teuthology.orchestra.run.gibba016.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-11-21T21:52:08.655 INFO:teuthology.orchestra.run.gibba016.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-11-21T21:52:08.657 INFO:teuthology.orchestra.run.gibba016.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-11-21T21:52:08.660 INFO:teuthology.orchestra.run.gibba016.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-11-21T21:52:08.663 INFO:teuthology.orchestra.run.gibba016.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-11-21T21:52:08.664 INFO:teuthology.orchestra.run.gibba019.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-11-21T21:52:08.665 INFO:teuthology.orchestra.run.gibba016.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-11-21T21:52:08.666 INFO:teuthology.orchestra.run.gibba017.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-11-21T21:52:08.667 INFO:teuthology.orchestra.run.gibba017.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-11-21T21:52:08.667 INFO:teuthology.orchestra.run.gibba019.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-11-21T21:52:08.668 INFO:teuthology.orchestra.run.gibba017.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-11-21T21:52:08.669 INFO:teuthology.orchestra.run.gibba019.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-11-21T21:52:08.672 INFO:teuthology.orchestra.run.gibba019.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-11-21T21:52:08.675 INFO:teuthology.orchestra.run.gibba019.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-11-21T21:52:08.677 INFO:teuthology.orchestra.run.gibba019.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-11-21T21:52:08.695 INFO:teuthology.orchestra.run.gibba019.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-11-21T21:52:08.724 INFO:teuthology.orchestra.run.gibba036.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-11-21T21:52:08.727 INFO:teuthology.orchestra.run.gibba036.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-11-21T21:52:08.729 INFO:teuthology.orchestra.run.gibba016.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-11-21T21:52:08.729 INFO:teuthology.orchestra.run.gibba036.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-11-21T21:52:08.730 INFO:teuthology.orchestra.run.gibba016.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-11-21T21:52:08.733 INFO:teuthology.orchestra.run.gibba036.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-11-21T21:52:08.737 INFO:teuthology.orchestra.run.gibba036.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-11-21T21:52:08.738 INFO:teuthology.orchestra.run.gibba036.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-11-21T21:52:08.747 INFO:teuthology.orchestra.run.gibba019.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-11-21T21:52:08.748 INFO:teuthology.orchestra.run.gibba019.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-11-21T21:52:08.749 INFO:teuthology.orchestra.run.gibba017.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-11-21T21:52:08.750 INFO:teuthology.orchestra.run.gibba019.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-11-21T21:52:08.756 INFO:teuthology.orchestra.run.gibba017.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-11-21T21:52:08.759 INFO:teuthology.orchestra.run.gibba017.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-11-21T21:52:08.759 INFO:teuthology.orchestra.run.gibba017.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-11-21T21:52:08.760 INFO:teuthology.orchestra.run.gibba036.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-11-21T21:52:08.761 INFO:teuthology.orchestra.run.gibba017.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-11-21T21:52:08.763 INFO:teuthology.orchestra.run.gibba017.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-11-21T21:52:08.765 INFO:teuthology.orchestra.run.gibba017.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-11-21T21:52:08.807 INFO:teuthology.orchestra.run.gibba036.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-11-21T21:52:08.808 INFO:teuthology.orchestra.run.gibba036.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-11-21T21:52:08.809 INFO:teuthology.orchestra.run.gibba036.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-11-21T21:52:08.829 INFO:teuthology.orchestra.run.gibba019.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-11-21T21:52:08.832 INFO:teuthology.orchestra.run.gibba017.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-11-21T21:52:08.833 INFO:teuthology.orchestra.run.gibba017.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-11-21T21:52:08.835 INFO:teuthology.orchestra.run.gibba019.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-11-21T21:52:08.839 INFO:teuthology.orchestra.run.gibba019.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-11-21T21:52:08.841 INFO:teuthology.orchestra.run.gibba019.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-11-21T21:52:08.845 INFO:teuthology.orchestra.run.gibba019.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-11-21T21:52:08.848 INFO:teuthology.orchestra.run.gibba019.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-11-21T21:52:08.851 INFO:teuthology.orchestra.run.gibba019.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-11-21T21:52:08.879 INFO:teuthology.orchestra.run.gibba017.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.15-36-g911bc85f-1focal [116 kB] 2021-11-21T21:52:08.880 INFO:teuthology.orchestra.run.gibba017.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.15-36-g911bc85f-1focal [361 kB] 2021-11-21T21:52:08.882 INFO:teuthology.orchestra.run.gibba017.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.15-36-g911bc85f-1focal [20.0 MB] 2021-11-21T21:52:08.889 INFO:teuthology.orchestra.run.gibba036.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-11-21T21:52:08.896 INFO:teuthology.orchestra.run.gibba036.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-11-21T21:52:08.898 INFO:teuthology.orchestra.run.gibba036.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-11-21T21:52:08.900 INFO:teuthology.orchestra.run.gibba036.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-11-21T21:52:08.903 INFO:teuthology.orchestra.run.gibba036.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-11-21T21:52:08.906 INFO:teuthology.orchestra.run.gibba036.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-11-21T21:52:08.909 INFO:teuthology.orchestra.run.gibba036.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-11-21T21:52:08.911 INFO:teuthology.orchestra.run.gibba019.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-11-21T21:52:08.912 INFO:teuthology.orchestra.run.gibba019.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-11-21T21:52:08.924 INFO:teuthology.orchestra.run.gibba019.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.15-36-g911bc85f-1focal [3,247 kB] 2021-11-21T21:52:08.985 INFO:teuthology.orchestra.run.gibba036.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-11-21T21:52:08.987 INFO:teuthology.orchestra.run.gibba036.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-11-21T21:52:09.120 INFO:teuthology.orchestra.run.gibba016.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-11-21T21:52:09.195 INFO:teuthology.orchestra.run.gibba016.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-11-21T21:52:09.235 INFO:teuthology.orchestra.run.gibba016.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-11-21T21:52:09.331 INFO:teuthology.orchestra.run.gibba016.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-11-21T21:52:09.332 INFO:teuthology.orchestra.run.gibba016.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-11-21T21:52:09.334 INFO:teuthology.orchestra.run.gibba016.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-11-21T21:52:09.334 INFO:teuthology.orchestra.run.gibba016.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-11-21T21:52:09.335 INFO:teuthology.orchestra.run.gibba016.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-11-21T21:52:09.335 INFO:teuthology.orchestra.run.gibba016.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-11-21T21:52:09.336 INFO:teuthology.orchestra.run.gibba016.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-11-21T21:52:09.336 INFO:teuthology.orchestra.run.gibba016.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-11-21T21:52:09.336 INFO:teuthology.orchestra.run.gibba016.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-11-21T21:52:09.375 INFO:teuthology.orchestra.run.gibba036.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.15-36-g911bc85f-1focal [3,247 kB] 2021-11-21T21:52:09.404 INFO:teuthology.orchestra.run.gibba017.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-11-21T21:52:09.414 INFO:teuthology.orchestra.run.gibba016.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-11-21T21:52:09.421 INFO:teuthology.orchestra.run.gibba016.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-11-21T21:52:09.426 INFO:teuthology.orchestra.run.gibba016.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-11-21T21:52:09.432 INFO:teuthology.orchestra.run.gibba016.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-11-21T21:52:09.434 INFO:teuthology.orchestra.run.gibba016.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-11-21T21:52:09.442 INFO:teuthology.orchestra.run.gibba016.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-11-21T21:52:09.446 INFO:teuthology.orchestra.run.gibba016.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-11-21T21:52:09.450 INFO:teuthology.orchestra.run.gibba016.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-11-21T21:52:09.451 INFO:teuthology.orchestra.run.gibba016.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-11-21T21:52:09.519 INFO:teuthology.orchestra.run.gibba017.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-11-21T21:52:09.523 INFO:teuthology.orchestra.run.gibba017.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-11-21T21:52:09.524 INFO:teuthology.orchestra.run.gibba016.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-11-21T21:52:09.542 INFO:teuthology.orchestra.run.gibba019.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-11-21T21:52:09.560 INFO:teuthology.orchestra.run.gibba016.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-11-21T21:52:09.561 INFO:teuthology.orchestra.run.gibba016.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-11-21T21:52:09.562 INFO:teuthology.orchestra.run.gibba016.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-11-21T21:52:09.565 INFO:teuthology.orchestra.run.gibba016.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-11-21T21:52:09.595 INFO:teuthology.orchestra.run.gibba036.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-11-21T21:52:09.603 INFO:teuthology.orchestra.run.gibba016.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-11-21T21:52:09.604 INFO:teuthology.orchestra.run.gibba016.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-11-21T21:52:09.605 INFO:teuthology.orchestra.run.gibba017.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-11-21T21:52:09.605 INFO:teuthology.orchestra.run.gibba016.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-11-21T21:52:09.606 INFO:teuthology.orchestra.run.gibba016.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-11-21T21:52:09.606 INFO:teuthology.orchestra.run.gibba017.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-11-21T21:52:09.607 INFO:teuthology.orchestra.run.gibba017.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-11-21T21:52:09.608 INFO:teuthology.orchestra.run.gibba019.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-11-21T21:52:09.611 INFO:teuthology.orchestra.run.gibba019.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-11-21T21:52:09.614 INFO:teuthology.orchestra.run.gibba017.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-11-21T21:52:09.615 INFO:teuthology.orchestra.run.gibba017.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-11-21T21:52:09.616 INFO:teuthology.orchestra.run.gibba017.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-11-21T21:52:09.616 INFO:teuthology.orchestra.run.gibba017.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-11-21T21:52:09.627 INFO:teuthology.orchestra.run.gibba016.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2021-11-21T21:52:09.652 INFO:teuthology.orchestra.run.gibba036.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-11-21T21:52:09.656 INFO:teuthology.orchestra.run.gibba036.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-11-21T21:52:09.664 INFO:teuthology.orchestra.run.gibba019.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-11-21T21:52:09.665 INFO:teuthology.orchestra.run.gibba019.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-11-21T21:52:09.665 INFO:teuthology.orchestra.run.gibba019.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-11-21T21:52:09.666 INFO:teuthology.orchestra.run.gibba019.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-11-21T21:52:09.669 INFO:teuthology.orchestra.run.gibba019.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-11-21T21:52:09.669 INFO:teuthology.orchestra.run.gibba019.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-11-21T21:52:09.670 INFO:teuthology.orchestra.run.gibba019.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-11-21T21:52:09.698 INFO:teuthology.orchestra.run.gibba017.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-11-21T21:52:09.698 INFO:teuthology.orchestra.run.gibba017.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-11-21T21:52:09.699 INFO:teuthology.orchestra.run.gibba017.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-11-21T21:52:09.704 INFO:teuthology.orchestra.run.gibba017.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-11-21T21:52:09.707 INFO:teuthology.orchestra.run.gibba017.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-11-21T21:52:09.709 INFO:teuthology.orchestra.run.gibba017.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-11-21T21:52:09.711 INFO:teuthology.orchestra.run.gibba017.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-11-21T21:52:09.711 INFO:teuthology.orchestra.run.gibba016.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-11-21T21:52:09.712 INFO:teuthology.orchestra.run.gibba016.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-11-21T21:52:09.717 INFO:teuthology.orchestra.run.gibba016.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-11-21T21:52:09.718 INFO:teuthology.orchestra.run.gibba017.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-11-21T21:52:09.730 INFO:teuthology.orchestra.run.gibba036.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-11-21T21:52:09.730 INFO:teuthology.orchestra.run.gibba036.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-11-21T21:52:09.731 INFO:teuthology.orchestra.run.gibba019.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-11-21T21:52:09.732 INFO:teuthology.orchestra.run.gibba036.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-11-21T21:52:09.733 INFO:teuthology.orchestra.run.gibba017.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-11-21T21:52:09.733 INFO:teuthology.orchestra.run.gibba019.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-11-21T21:52:09.734 INFO:teuthology.orchestra.run.gibba036.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-11-21T21:52:09.734 INFO:teuthology.orchestra.run.gibba036.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-11-21T21:52:09.734 INFO:teuthology.orchestra.run.gibba036.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-11-21T21:52:09.734 INFO:teuthology.orchestra.run.gibba036.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-11-21T21:52:09.739 INFO:teuthology.orchestra.run.gibba017.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-11-21T21:52:09.762 INFO:teuthology.orchestra.run.gibba036.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-11-21T21:52:09.762 INFO:teuthology.orchestra.run.gibba036.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-11-21T21:52:09.814 INFO:teuthology.orchestra.run.gibba019.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-11-21T21:52:09.815 INFO:teuthology.orchestra.run.gibba017.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-11-21T21:52:09.822 INFO:teuthology.orchestra.run.gibba019.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-11-21T21:52:09.825 INFO:teuthology.orchestra.run.gibba017.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-11-21T21:52:09.826 INFO:teuthology.orchestra.run.gibba017.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-11-21T21:52:09.827 INFO:teuthology.orchestra.run.gibba019.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-11-21T21:52:09.828 INFO:teuthology.orchestra.run.gibba017.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-11-21T21:52:09.830 INFO:teuthology.orchestra.run.gibba017.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-11-21T21:52:09.832 INFO:teuthology.orchestra.run.gibba017.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-11-21T21:52:09.834 INFO:teuthology.orchestra.run.gibba019.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-11-21T21:52:09.837 INFO:teuthology.orchestra.run.gibba019.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-11-21T21:52:09.845 INFO:teuthology.orchestra.run.gibba019.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-11-21T21:52:09.849 INFO:teuthology.orchestra.run.gibba019.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-11-21T21:52:09.849 INFO:teuthology.orchestra.run.gibba036.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-11-21T21:52:09.852 INFO:teuthology.orchestra.run.gibba036.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-11-21T21:52:09.853 INFO:teuthology.orchestra.run.gibba019.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-11-21T21:52:09.854 INFO:teuthology.orchestra.run.gibba036.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-11-21T21:52:09.856 INFO:teuthology.orchestra.run.gibba036.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-11-21T21:52:09.858 INFO:teuthology.orchestra.run.gibba036.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-11-21T21:52:09.861 INFO:teuthology.orchestra.run.gibba036.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-11-21T21:52:09.863 INFO:teuthology.orchestra.run.gibba036.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-11-21T21:52:09.864 INFO:teuthology.orchestra.run.gibba017.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-11-21T21:52:09.864 INFO:teuthology.orchestra.run.gibba017.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-11-21T21:52:09.866 INFO:teuthology.orchestra.run.gibba036.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-11-21T21:52:09.895 INFO:teuthology.orchestra.run.gibba019.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-11-21T21:52:09.897 INFO:teuthology.orchestra.run.gibba017.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-11-21T21:52:09.898 INFO:teuthology.orchestra.run.gibba017.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-11-21T21:52:09.899 INFO:teuthology.orchestra.run.gibba019.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-11-21T21:52:09.932 INFO:teuthology.orchestra.run.gibba036.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-11-21T21:52:09.944 INFO:teuthology.orchestra.run.gibba036.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-11-21T21:52:09.977 INFO:teuthology.orchestra.run.gibba019.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-11-21T21:52:09.978 INFO:teuthology.orchestra.run.gibba019.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-11-21T21:52:09.980 INFO:teuthology.orchestra.run.gibba017.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2021-11-21T21:52:09.980 INFO:teuthology.orchestra.run.gibba019.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-11-21T21:52:09.983 INFO:teuthology.orchestra.run.gibba017.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-11-21T21:52:09.983 INFO:teuthology.orchestra.run.gibba019.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-11-21T21:52:09.984 INFO:teuthology.orchestra.run.gibba017.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-11-21T21:52:09.989 INFO:teuthology.orchestra.run.gibba017.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-11-21T21:52:10.014 INFO:teuthology.orchestra.run.gibba036.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-11-21T21:52:10.015 INFO:teuthology.orchestra.run.gibba036.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-11-21T21:52:10.017 INFO:teuthology.orchestra.run.gibba036.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-11-21T21:52:10.019 INFO:teuthology.orchestra.run.gibba019.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-11-21T21:52:10.020 INFO:teuthology.orchestra.run.gibba019.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-11-21T21:52:10.021 INFO:teuthology.orchestra.run.gibba036.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-11-21T21:52:10.021 INFO:teuthology.orchestra.run.gibba019.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-11-21T21:52:10.021 INFO:teuthology.orchestra.run.gibba019.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-11-21T21:52:10.057 INFO:teuthology.orchestra.run.gibba036.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-11-21T21:52:10.057 INFO:teuthology.orchestra.run.gibba036.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-11-21T21:52:10.058 INFO:teuthology.orchestra.run.gibba036.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-11-21T21:52:10.060 INFO:teuthology.orchestra.run.gibba036.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-11-21T21:52:10.060 INFO:teuthology.orchestra.run.gibba019.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2021-11-21T21:52:10.061 INFO:teuthology.orchestra.run.gibba019.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-11-21T21:52:10.096 INFO:teuthology.orchestra.run.gibba036.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2021-11-21T21:52:10.096 INFO:teuthology.orchestra.run.gibba036.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-11-21T21:52:10.130 INFO:teuthology.orchestra.run.gibba016.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.15-36-g911bc85f-1focal [5,334 kB] 2021-11-21T21:52:10.143 INFO:teuthology.orchestra.run.gibba019.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-11-21T21:52:10.148 INFO:teuthology.orchestra.run.gibba019.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-11-21T21:52:10.176 INFO:teuthology.orchestra.run.gibba019.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.15-36-g911bc85f-1focal [463 kB] 2021-11-21T21:52:10.178 INFO:teuthology.orchestra.run.gibba036.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-11-21T21:52:10.183 INFO:teuthology.orchestra.run.gibba036.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-11-21T21:52:10.324 INFO:teuthology.orchestra.run.gibba019.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.15-36-g911bc85f-1focal [365 kB] 2021-11-21T21:52:10.468 INFO:teuthology.orchestra.run.gibba019.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.15-36-g911bc85f-1focal [20.8 kB] 2021-11-21T21:52:10.473 INFO:teuthology.orchestra.run.gibba019.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.15-36-g911bc85f-1focal [156 kB] 2021-11-21T21:52:10.487 INFO:teuthology.orchestra.run.gibba016.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.15-36-g911bc85f-1focal [162 kB] 2021-11-21T21:52:10.497 INFO:teuthology.orchestra.run.gibba019.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.15-36-g911bc85f-1focal [29.4 kB] 2021-11-21T21:52:10.499 INFO:teuthology.orchestra.run.gibba016.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.15-36-g911bc85f-1focal [1,402 kB] 2021-11-21T21:52:10.515 INFO:teuthology.orchestra.run.gibba019.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.15-36-g911bc85f-1focal [362 kB] 2021-11-21T21:52:10.522 INFO:teuthology.orchestra.run.gibba017.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.15-36-g911bc85f-1focal [5,334 kB] 2021-11-21T21:52:10.603 INFO:teuthology.orchestra.run.gibba016.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.15-36-g911bc85f-1focal [6,173 kB] 2021-11-21T21:52:10.634 INFO:teuthology.orchestra.run.gibba019.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.15-36-g911bc85f-1focal [3,138 kB] 2021-11-21T21:52:10.881 INFO:teuthology.orchestra.run.gibba017.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.15-36-g911bc85f-1focal [162 kB] 2021-11-21T21:52:10.886 INFO:teuthology.orchestra.run.gibba017.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.15-36-g911bc85f-1focal [1,402 kB] 2021-11-21T21:52:11.002 INFO:teuthology.orchestra.run.gibba017.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.15-36-g911bc85f-1focal [6,173 kB] 2021-11-21T21:52:11.033 INFO:teuthology.orchestra.run.gibba036.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.15-36-g911bc85f-1focal [463 kB] 2021-11-21T21:52:11.080 INFO:teuthology.orchestra.run.gibba016.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.15-36-g911bc85f-1focal [23.9 MB] 2021-11-21T21:52:11.278 INFO:teuthology.orchestra.run.gibba036.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.15-36-g911bc85f-1focal [365 kB] 2021-11-21T21:52:11.464 INFO:teuthology.orchestra.run.gibba036.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.15-36-g911bc85f-1focal [20.8 kB] 2021-11-21T21:52:11.466 INFO:teuthology.orchestra.run.gibba036.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.15-36-g911bc85f-1focal [156 kB] 2021-11-21T21:52:11.500 INFO:teuthology.orchestra.run.gibba017.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.15-36-g911bc85f-1focal [23.9 MB] 2021-11-21T21:52:11.584 INFO:teuthology.orchestra.run.gibba036.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.15-36-g911bc85f-1focal [29.4 kB] 2021-11-21T21:52:11.587 INFO:teuthology.orchestra.run.gibba036.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.15-36-g911bc85f-1focal [362 kB] 2021-11-21T21:52:11.697 INFO:teuthology.orchestra.run.gibba019.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.15-36-g911bc85f-1focal [116 kB] 2021-11-21T21:52:11.730 INFO:teuthology.orchestra.run.gibba019.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.15-36-g911bc85f-1focal [361 kB] 2021-11-21T21:52:11.774 INFO:teuthology.orchestra.run.gibba036.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.15-36-g911bc85f-1focal [3,138 kB] 2021-11-21T21:52:11.855 INFO:teuthology.orchestra.run.gibba019.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.15-36-g911bc85f-1focal [20.0 MB] 2021-11-21T21:52:12.946 INFO:teuthology.orchestra.run.gibba016.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.15-36-g911bc85f-1focal [4,016 B] 2021-11-21T21:52:12.946 INFO:teuthology.orchestra.run.gibba016.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.15-36-g911bc85f-1focal [538 kB] 2021-11-21T21:52:12.953 INFO:teuthology.orchestra.run.gibba016.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.15-36-g911bc85f-1focal [2,007 kB] 2021-11-21T21:52:13.076 INFO:teuthology.orchestra.run.gibba016.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.15-36-g911bc85f-1focal [53.2 kB] 2021-11-21T21:52:13.091 INFO:teuthology.orchestra.run.gibba016.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.15-36-g911bc85f-1focal [58.2 kB] 2021-11-21T21:52:13.093 INFO:teuthology.orchestra.run.gibba016.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.15-36-g911bc85f-1focal [3,208 kB] 2021-11-21T21:52:13.301 INFO:teuthology.orchestra.run.gibba036.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.15-36-g911bc85f-1focal [116 kB] 2021-11-21T21:52:13.334 INFO:teuthology.orchestra.run.gibba016.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.15-36-g911bc85f-1focal [34.4 kB] 2021-11-21T21:52:13.335 INFO:teuthology.orchestra.run.gibba016.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.15-36-g911bc85f-1focal [7,855 kB] 2021-11-21T21:52:13.364 INFO:teuthology.orchestra.run.gibba036.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.15-36-g911bc85f-1focal [361 kB] 2021-11-21T21:52:13.499 INFO:teuthology.orchestra.run.gibba036.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.15-36-g911bc85f-1focal [20.0 MB] 2021-11-21T21:52:13.502 INFO:teuthology.orchestra.run.gibba017.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.15-36-g911bc85f-1focal [4,016 B] 2021-11-21T21:52:13.503 INFO:teuthology.orchestra.run.gibba017.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.15-36-g911bc85f-1focal [538 kB] 2021-11-21T21:52:13.508 INFO:teuthology.orchestra.run.gibba017.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.15-36-g911bc85f-1focal [2,007 kB] 2021-11-21T21:52:13.663 INFO:teuthology.orchestra.run.gibba017.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.15-36-g911bc85f-1focal [53.2 kB] 2021-11-21T21:52:13.665 INFO:teuthology.orchestra.run.gibba017.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.15-36-g911bc85f-1focal [58.2 kB] 2021-11-21T21:52:13.668 INFO:teuthology.orchestra.run.gibba017.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.15-36-g911bc85f-1focal [3,208 kB] 2021-11-21T21:52:13.918 INFO:teuthology.orchestra.run.gibba016.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.15-36-g911bc85f-1focal [4,128 B] 2021-11-21T21:52:13.919 INFO:teuthology.orchestra.run.gibba016.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.15-36-g911bc85f-1focal [39.8 MB] 2021-11-21T21:52:13.930 INFO:teuthology.orchestra.run.gibba017.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.15-36-g911bc85f-1focal [34.4 kB] 2021-11-21T21:52:13.931 INFO:teuthology.orchestra.run.gibba017.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.15-36-g911bc85f-1focal [7,855 kB] 2021-11-21T21:52:14.587 INFO:teuthology.orchestra.run.gibba017.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.15-36-g911bc85f-1focal [4,128 B] 2021-11-21T21:52:14.588 INFO:teuthology.orchestra.run.gibba017.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.15-36-g911bc85f-1focal [39.8 MB] 2021-11-21T21:52:17.190 INFO:teuthology.orchestra.run.gibba016.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.15-36-g911bc85f-1focal [15.9 kB] 2021-11-21T21:52:17.190 INFO:teuthology.orchestra.run.gibba016.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.15-36-g911bc85f-1focal [18.1 kB] 2021-11-21T21:52:17.191 INFO:teuthology.orchestra.run.gibba016.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.15-36-g911bc85f-1focal [9,716 kB] 2021-11-21T21:52:17.798 INFO:teuthology.orchestra.run.gibba016.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.15-36-g911bc85f-1focal [61.0 kB] 2021-11-21T21:52:17.994 INFO:teuthology.orchestra.run.gibba017.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.15-36-g911bc85f-1focal [15.9 kB] 2021-11-21T21:52:17.994 INFO:teuthology.orchestra.run.gibba017.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.15-36-g911bc85f-1focal [18.1 kB] 2021-11-21T21:52:17.994 INFO:teuthology.orchestra.run.gibba017.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.15-36-g911bc85f-1focal [9,716 kB] 2021-11-21T21:52:18.117 INFO:teuthology.orchestra.run.gibba016.stdout:Fetched 153 MB in 10s (15.0 MB/s) 2021-11-21T21:52:18.209 INFO:teuthology.orchestra.run.gibba019.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.15-36-g911bc85f-1focal [5,334 kB] 2021-11-21T21:52:18.241 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pyperclip. 2021-11-21T21:52:18.312 INFO:teuthology.orchestra.run.gibba016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174687 files and directories currently installed.) 2021-11-21T21:52:18.315 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-11-21T21:52:18.355 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-11-21T21:52:18.380 INFO:teuthology.orchestra.run.gibba017.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.15-36-g911bc85f-1focal [61.0 kB] 2021-11-21T21:52:18.683 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-wcwidth. 2021-11-21T21:52:18.697 INFO:teuthology.orchestra.run.gibba017.stdout:Fetched 153 MB in 11s (14.3 MB/s) 2021-11-21T21:52:18.715 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-11-21T21:52:18.758 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:52:18.797 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pyperclip. 2021-11-21T21:52:18.879 INFO:teuthology.orchestra.run.gibba017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174687 files and directories currently installed.) 2021-11-21T21:52:18.879 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-11-21T21:52:18.911 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-11-21T21:52:18.935 INFO:teuthology.orchestra.run.gibba019.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.15-36-g911bc85f-1focal [162 kB] 2021-11-21T21:52:18.965 INFO:teuthology.orchestra.run.gibba019.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.15-36-g911bc85f-1focal [1,402 kB] 2021-11-21T21:52:19.111 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-cmd2. 2021-11-21T21:52:19.120 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-11-21T21:52:19.151 INFO:teuthology.orchestra.run.gibba019.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.15-36-g911bc85f-1focal [6,173 kB] 2021-11-21T21:52:19.160 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:52:19.248 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-wcwidth. 2021-11-21T21:52:19.280 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-11-21T21:52:19.323 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:52:19.418 INFO:teuthology.orchestra.run.gibba036.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.15-36-g911bc85f-1focal [5,334 kB] 2021-11-21T21:52:19.505 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-11-21T21:52:19.537 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-11-21T21:52:19.622 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:52:19.667 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-cmd2. 2021-11-21T21:52:19.677 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-11-21T21:52:19.716 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:52:19.876 INFO:teuthology.orchestra.run.gibba019.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.15-36-g911bc85f-1focal [23.9 MB] 2021-11-21T21:52:19.961 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../004-librbd1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:20.027 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-11-21T21:52:20.036 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-11-21T21:52:20.037 INFO:teuthology.orchestra.run.gibba036.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.15-36-g911bc85f-1focal [162 kB] 2021-11-21T21:52:20.061 INFO:teuthology.orchestra.run.gibba036.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.15-36-g911bc85f-1focal [1,402 kB] 2021-11-21T21:52:20.084 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking librbd1 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:20.103 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:52:20.231 INFO:teuthology.orchestra.run.gibba036.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.15-36-g911bc85f-1focal [6,173 kB] 2021-11-21T21:52:20.408 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../004-librbd1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:20.516 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking librbd1 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:20.627 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../005-librados2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:20.754 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking librados2 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:20.946 INFO:teuthology.orchestra.run.gibba036.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.15-36-g911bc85f-1focal [23.9 MB] 2021-11-21T21:52:21.036 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../005-librados2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:21.186 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking librados2 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:21.375 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libcephfs2. 2021-11-21T21:52:21.407 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../006-libcephfs2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:21.441 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:21.710 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-rados. 2021-11-21T21:52:21.742 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../007-python3-rados_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:21.802 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:21.832 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libcephfs2. 2021-11-21T21:52:21.841 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../006-libcephfs2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:21.881 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:22.110 INFO:teuthology.orchestra.run.gibba019.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.15-36-g911bc85f-1focal [4,016 B] 2021-11-21T21:52:22.110 INFO:teuthology.orchestra.run.gibba019.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.15-36-g911bc85f-1focal [538 kB] 2021-11-21T21:52:22.112 INFO:teuthology.orchestra.run.gibba019.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.15-36-g911bc85f-1focal [2,007 kB] 2021-11-21T21:52:22.151 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-rados. 2021-11-21T21:52:22.182 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../007-python3-rados_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:22.187 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-11-21T21:52:22.197 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:22.216 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:22.237 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:22.248 INFO:teuthology.orchestra.run.gibba019.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.15-36-g911bc85f-1focal [53.2 kB] 2021-11-21T21:52:22.250 INFO:teuthology.orchestra.run.gibba019.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.15-36-g911bc85f-1focal [58.2 kB] 2021-11-21T21:52:22.252 INFO:teuthology.orchestra.run.gibba019.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.15-36-g911bc85f-1focal [3,208 kB] 2021-11-21T21:52:22.470 INFO:teuthology.orchestra.run.gibba019.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.15-36-g911bc85f-1focal [34.4 kB] 2021-11-21T21:52:22.471 INFO:teuthology.orchestra.run.gibba019.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.15-36-g911bc85f-1focal [7,855 kB] 2021-11-21T21:52:22.572 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-cephfs. 2021-11-21T21:52:22.582 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../009-python3-cephfs_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:22.621 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:22.621 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-11-21T21:52:22.629 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:22.669 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:22.949 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-ceph-common. 2021-11-21T21:52:22.972 INFO:teuthology.orchestra.run.gibba019.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.15-36-g911bc85f-1focal [4,128 B] 2021-11-21T21:52:22.973 INFO:teuthology.orchestra.run.gibba019.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.15-36-g911bc85f-1focal [39.8 MB] 2021-11-21T21:52:22.981 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:22.996 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-cephfs. 2021-11-21T21:52:23.023 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:23.028 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../009-python3-cephfs_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:23.071 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:23.377 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-prettytable. 2021-11-21T21:52:23.409 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-11-21T21:52:23.415 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-ceph-common. 2021-11-21T21:52:23.425 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:23.451 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-11-21T21:52:23.465 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:23.536 INFO:teuthology.orchestra.run.gibba036.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.15-36-g911bc85f-1focal [4,016 B] 2021-11-21T21:52:23.536 INFO:teuthology.orchestra.run.gibba036.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.15-36-g911bc85f-1focal [538 kB] 2021-11-21T21:52:23.539 INFO:teuthology.orchestra.run.gibba036.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.15-36-g911bc85f-1focal [2,007 kB] 2021-11-21T21:52:23.721 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-rbd. 2021-11-21T21:52:23.737 INFO:teuthology.orchestra.run.gibba036.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.15-36-g911bc85f-1focal [53.2 kB] 2021-11-21T21:52:23.738 INFO:teuthology.orchestra.run.gibba036.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.15-36-g911bc85f-1focal [58.2 kB] 2021-11-21T21:52:23.738 INFO:teuthology.orchestra.run.gibba036.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.15-36-g911bc85f-1focal [3,208 kB] 2021-11-21T21:52:23.753 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../012-python3-rbd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:23.760 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-prettytable. 2021-11-21T21:52:23.791 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-11-21T21:52:23.812 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:23.826 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-11-21T21:52:24.052 INFO:teuthology.orchestra.run.gibba036.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.15-36-g911bc85f-1focal [34.4 kB] 2021-11-21T21:52:24.054 INFO:teuthology.orchestra.run.gibba036.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.15-36-g911bc85f-1focal [7,855 kB] 2021-11-21T21:52:24.096 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-rbd. 2021-11-21T21:52:24.127 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../012-python3-rbd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:24.171 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:24.182 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-11-21T21:52:24.192 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-11-21T21:52:24.231 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:52:24.517 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-11-21T21:52:24.553 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-11-21T21:52:24.584 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:52:24.615 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-11-21T21:52:24.627 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-11-21T21:52:24.666 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:52:24.881 INFO:teuthology.orchestra.run.gibba036.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.15-36-g911bc85f-1focal [4,128 B] 2021-11-21T21:52:24.882 INFO:teuthology.orchestra.run.gibba036.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.15-36-g911bc85f-1focal [39.8 MB] 2021-11-21T21:52:25.027 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-11-21T21:52:25.058 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-11-21T21:52:25.084 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package librgw2. 2021-11-21T21:52:25.093 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../015-librgw2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:25.101 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:52:25.102 INFO:teuthology.orchestra.run.gibba019.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.15-36-g911bc85f-1focal [15.9 kB] 2021-11-21T21:52:25.102 INFO:teuthology.orchestra.run.gibba019.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.15-36-g911bc85f-1focal [18.1 kB] 2021-11-21T21:52:25.102 INFO:teuthology.orchestra.run.gibba019.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.15-36-g911bc85f-1focal [9,716 kB] 2021-11-21T21:52:25.158 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:25.438 INFO:teuthology.orchestra.run.gibba019.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.15-36-g911bc85f-1focal [61.0 kB] 2021-11-21T21:52:25.458 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package librgw2. 2021-11-21T21:52:25.467 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../015-librgw2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:25.507 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:25.636 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-rgw. 2021-11-21T21:52:25.668 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../016-python3-rgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:25.711 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:25.756 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 153 MB in 18s (8,665 kB/s) 2021-11-21T21:52:25.888 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyperclip. 2021-11-21T21:52:25.960 INFO:teuthology.orchestra.run.gibba019.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174687 files and directories currently installed.) 2021-11-21T21:52:25.963 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-11-21T21:52:26.002 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-rgw. 2021-11-21T21:52:26.003 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-11-21T21:52:26.033 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../016-python3-rgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:26.066 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-11-21T21:52:26.068 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:26.073 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-11-21T21:52:26.113 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:52:26.288 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-wcwidth. 2021-11-21T21:52:26.320 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-11-21T21:52:26.355 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:52:26.363 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-11-21T21:52:26.373 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-11-21T21:52:26.412 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:52:26.479 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package liboath0. 2021-11-21T21:52:26.511 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-11-21T21:52:26.557 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-11-21T21:52:26.691 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cmd2. 2021-11-21T21:52:26.701 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-11-21T21:52:26.740 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:52:26.782 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package liboath0. 2021-11-21T21:52:26.814 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-11-21T21:52:26.827 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libradosstriper1. 2021-11-21T21:52:26.856 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-11-21T21:52:26.859 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../019-libradosstriper1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:26.893 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:27.085 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-11-21T21:52:27.117 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-11-21T21:52:27.185 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libradosstriper1. 2021-11-21T21:52:27.194 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../019-libradosstriper1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:27.202 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:52:27.234 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:27.397 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-common. 2021-11-21T21:52:27.407 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../020-ceph-common_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:27.446 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:27.574 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../004-librbd1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:27.621 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-common. 2021-11-21T21:52:27.653 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../020-ceph-common_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:27.687 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:27.697 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librbd1 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:28.097 INFO:teuthology.orchestra.run.gibba036.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.15-36-g911bc85f-1focal [15.9 kB] 2021-11-21T21:52:28.097 INFO:teuthology.orchestra.run.gibba036.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.15-36-g911bc85f-1focal [18.1 kB] 2021-11-21T21:52:28.098 INFO:teuthology.orchestra.run.gibba036.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.15-36-g911bc85f-1focal [9,716 kB] 2021-11-21T21:52:28.185 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../005-librados2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:28.309 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librados2 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:28.587 INFO:teuthology.orchestra.run.gibba036.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/911bc85f89d5c4ff92555237a37b5b774d7b1332/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.15-36-g911bc85f-1focal [61.0 kB] 2021-11-21T21:52:28.897 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libcephfs2. 2021-11-21T21:52:28.906 INFO:teuthology.orchestra.run.gibba036.stdout:Fetched 153 MB in 21s (7,372 kB/s) 2021-11-21T21:52:28.906 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../006-libcephfs2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:28.946 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:29.011 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pyperclip. 2021-11-21T21:52:29.081 INFO:teuthology.orchestra.run.gibba036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174687 files and directories currently installed.) 2021-11-21T21:52:29.084 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-11-21T21:52:29.150 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-11-21T21:52:29.212 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-base. 2021-11-21T21:52:29.222 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../021-ceph-base_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:29.265 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:29.274 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rados. 2021-11-21T21:52:29.305 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../007-python3-rados_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:29.348 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:29.454 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-wcwidth. 2021-11-21T21:52:29.486 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-11-21T21:52:29.529 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:52:29.621 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-base. 2021-11-21T21:52:29.631 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../021-ceph-base_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:29.651 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-11-21T21:52:29.674 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:29.682 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:29.717 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:29.848 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-cmd2. 2021-11-21T21:52:29.858 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-11-21T21:52:29.897 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:52:30.044 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cephfs. 2021-11-21T21:52:30.052 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../009-python3-cephfs_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:30.075 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-cryptography. 2021-11-21T21:52:30.094 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:30.106 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-11-21T21:52:30.149 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:52:30.267 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-11-21T21:52:30.299 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-11-21T21:52:30.384 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:52:30.446 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-ceph-common. 2021-11-21T21:52:30.459 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-cryptography. 2021-11-21T21:52:30.477 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:30.491 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-11-21T21:52:30.520 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:30.525 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:52:30.561 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-openssl. 2021-11-21T21:52:30.571 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-11-21T21:52:30.610 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-11-21T21:52:30.681 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../004-librbd1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:30.813 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking librbd1 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:30.840 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-prettytable. 2021-11-21T21:52:30.850 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-11-21T21:52:30.855 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-11-21T21:52:30.887 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:30.890 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-11-21T21:52:30.928 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-openssl. 2021-11-21T21:52:30.929 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:30.938 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-11-21T21:52:30.978 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-11-21T21:52:31.176 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rbd. 2021-11-21T21:52:31.207 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../012-python3-rbd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:31.240 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-11-21T21:52:31.250 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:31.271 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:31.301 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-bcrypt. 2021-11-21T21:52:31.309 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../005-librados2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:31.311 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-11-21T21:52:31.314 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:31.350 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:52:31.433 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking librados2 (15.2.15-36-g911bc85f-1focal) over (15.2.14-0ubuntu0.20.04.1) ... 2021-11-21T21:52:31.587 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-11-21T21:52:31.596 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-11-21T21:52:31.636 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:52:31.652 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-cherrypy3. 2021-11-21T21:52:31.684 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-11-21T21:52:31.727 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:52:31.777 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-bcrypt. 2021-11-21T21:52:31.787 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-11-21T21:52:31.835 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:52:32.006 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-11-21T21:52:32.037 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libcephfs2. 2021-11-21T21:52:32.037 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-11-21T21:52:32.048 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../006-libcephfs2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:32.080 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-jwt. 2021-11-21T21:52:32.081 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:52:32.086 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:32.090 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-11-21T21:52:32.130 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:52:32.179 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-cherrypy3. 2021-11-21T21:52:32.211 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-11-21T21:52:32.254 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:52:32.339 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-rados. 2021-11-21T21:52:32.371 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../007-python3-rados_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:32.414 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:32.425 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-logutils. 2021-11-21T21:52:32.456 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-11-21T21:52:32.459 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package librgw2. 2021-11-21T21:52:32.469 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../015-librgw2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:32.499 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-11-21T21:52:32.508 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:32.615 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-jwt. 2021-11-21T21:52:32.626 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-11-21T21:52:32.665 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:52:32.783 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-11-21T21:52:32.793 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:32.810 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-markupsafe. 2021-11-21T21:52:32.819 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-11-21T21:52:32.832 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:32.859 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:52:33.010 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-logutils. 2021-11-21T21:52:33.011 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rgw. 2021-11-21T21:52:33.042 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-11-21T21:52:33.043 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../016-python3-rgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:33.084 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-11-21T21:52:33.085 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:33.143 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-cephfs. 2021-11-21T21:52:33.172 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-mako. 2021-11-21T21:52:33.175 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../009-python3-cephfs_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:33.203 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-11-21T21:52:33.219 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:33.246 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:52:33.363 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-11-21T21:52:33.395 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-11-21T21:52:33.421 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-markupsafe. 2021-11-21T21:52:33.431 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:52:33.431 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-11-21T21:52:33.471 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:52:33.562 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-ceph-common. 2021-11-21T21:52:33.566 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-simplegeneric. 2021-11-21T21:52:33.573 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:33.577 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-11-21T21:52:33.611 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:33.615 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:52:33.740 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liboath0. 2021-11-21T21:52:33.751 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-11-21T21:52:33.790 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-11-21T21:52:33.802 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-mako. 2021-11-21T21:52:33.834 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-11-21T21:52:33.865 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-prettytable. 2021-11-21T21:52:33.882 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:52:33.896 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-11-21T21:52:33.931 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-11-21T21:52:33.935 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-singledispatch. 2021-11-21T21:52:33.967 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-11-21T21:52:34.010 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:52:34.077 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libradosstriper1. 2021-11-21T21:52:34.109 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../019-libradosstriper1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:34.185 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:34.201 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-rbd. 2021-11-21T21:52:34.227 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-simplegeneric. 2021-11-21T21:52:34.233 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../012-python3-rbd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:34.236 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-11-21T21:52:34.275 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:34.284 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:52:34.338 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-webob. 2021-11-21T21:52:34.347 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-11-21T21:52:34.387 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-11-21T21:52:34.613 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-common. 2021-11-21T21:52:34.623 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../020-ceph-common_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:34.651 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-singledispatch. 2021-11-21T21:52:34.662 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:34.682 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-11-21T21:52:34.686 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-11-21T21:52:34.697 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-11-21T21:52:34.725 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:52:34.736 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:52:34.749 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-soupsieve. 2021-11-21T21:52:34.781 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-11-21T21:52:34.823 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:52:35.070 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-webob. 2021-11-21T21:52:35.080 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-11-21T21:52:35.089 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-11-21T21:52:35.119 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-11-21T21:52:35.121 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-11-21T21:52:35.127 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-bs4. 2021-11-21T21:52:35.159 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-11-21T21:52:35.164 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:52:35.213 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-11-21T21:52:35.470 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-soupsieve. 2021-11-21T21:52:35.496 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package librgw2. 2021-11-21T21:52:35.502 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-11-21T21:52:35.505 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../015-librgw2_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:35.539 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:52:35.545 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:35.555 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-waitress. 2021-11-21T21:52:35.564 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-11-21T21:52:35.605 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-11-21T21:52:35.881 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-bs4. 2021-11-21T21:52:35.882 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-tempita. 2021-11-21T21:52:35.891 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-11-21T21:52:35.914 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-11-21T21:52:35.933 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-11-21T21:52:35.957 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-11-21T21:52:36.081 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-rgw. 2021-11-21T21:52:36.113 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../016-python3-rgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:36.147 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:36.252 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-paste. 2021-11-21T21:52:36.283 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-11-21T21:52:36.300 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-waitress. 2021-11-21T21:52:36.327 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:52:36.332 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-11-21T21:52:36.382 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-11-21T21:52:36.394 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-base. 2021-11-21T21:52:36.402 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../021-ceph-base_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:36.409 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-11-21T21:52:36.441 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-11-21T21:52:36.447 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:36.475 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:52:36.664 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-11-21T21:52:36.674 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-11-21T21:52:36.713 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:52:36.731 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-tempita. 2021-11-21T21:52:36.741 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-11-21T21:52:36.780 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-11-21T21:52:36.837 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package liboath0. 2021-11-21T21:52:36.847 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-11-21T21:52:36.886 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-11-21T21:52:37.000 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pastedeploy. 2021-11-21T21:52:37.032 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-11-21T21:52:37.066 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:52:37.092 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-paste. 2021-11-21T21:52:37.124 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-11-21T21:52:37.175 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:52:37.197 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libradosstriper1. 2021-11-21T21:52:37.227 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cryptography. 2021-11-21T21:52:37.229 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../019-libradosstriper1_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:37.259 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-11-21T21:52:37.272 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:37.304 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:52:37.403 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-webtest. 2021-11-21T21:52:37.412 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-11-21T21:52:37.452 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-11-21T21:52:37.475 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-11-21T21:52:37.507 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-11-21T21:52:37.566 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:52:37.638 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-openssl. 2021-11-21T21:52:37.651 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-common. 2021-11-21T21:52:37.661 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../020-ceph-common_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:37.670 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-11-21T21:52:37.700 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:37.704 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-11-21T21:52:37.764 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pecan. 2021-11-21T21:52:37.796 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-11-21T21:52:37.838 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:52:37.860 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pastedeploy. 2021-11-21T21:52:37.871 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-11-21T21:52:37.910 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:52:38.016 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-11-21T21:52:38.026 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:38.065 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:38.193 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-werkzeug. 2021-11-21T21:52:38.203 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-11-21T21:52:38.243 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:52:38.247 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-webtest. 2021-11-21T21:52:38.279 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-11-21T21:52:38.322 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-11-21T21:52:38.386 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-bcrypt. 2021-11-21T21:52:38.418 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-11-21T21:52:38.483 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:52:38.621 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mgr. 2021-11-21T21:52:38.653 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../044-ceph-mgr_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:38.667 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pecan. 2021-11-21T21:52:38.676 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-11-21T21:52:38.696 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:38.715 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:52:38.780 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cherrypy3. 2021-11-21T21:52:38.790 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-11-21T21:52:38.830 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:52:39.062 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-werkzeug. 2021-11-21T21:52:39.094 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-11-21T21:52:39.136 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:52:39.148 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mon. 2021-11-21T21:52:39.158 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../045-ceph-mon_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:39.189 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-jwt. 2021-11-21T21:52:39.198 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:39.221 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-11-21T21:52:39.264 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:52:39.592 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-logutils. 2021-11-21T21:52:39.601 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-11-21T21:52:39.641 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-11-21T21:52:40.046 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mgr. 2021-11-21T21:52:40.056 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../044-ceph-mgr_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:40.111 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-osd. 2021-11-21T21:52:40.120 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../046-ceph-osd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:40.137 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-base. 2021-11-21T21:52:40.147 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../021-ceph-base_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:40.191 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:40.277 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:40.324 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:40.678 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-markupsafe. 2021-11-21T21:52:40.688 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-11-21T21:52:40.727 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:52:40.802 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mon. 2021-11-21T21:52:40.811 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../045-ceph-mon_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:40.851 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:40.925 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-cryptography. 2021-11-21T21:52:40.957 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-11-21T21:52:40.999 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:52:41.039 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-mako. 2021-11-21T21:52:41.071 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-11-21T21:52:41.114 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:52:41.311 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-openssl. 2021-11-21T21:52:41.344 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-11-21T21:52:41.377 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-11-21T21:52:41.459 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-simplegeneric. 2021-11-21T21:52:41.468 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-11-21T21:52:41.516 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:52:41.646 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-11-21T21:52:41.679 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:41.729 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:41.797 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-osd. 2021-11-21T21:52:41.829 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../046-ceph-osd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:41.844 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-singledispatch. 2021-11-21T21:52:41.876 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-11-21T21:52:41.897 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:41.911 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:52:42.025 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-bcrypt. 2021-11-21T21:52:42.057 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-11-21T21:52:42.100 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:52:42.197 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-webob. 2021-11-21T21:52:42.229 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-11-21T21:52:42.271 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-11-21T21:52:42.415 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-cherrypy3. 2021-11-21T21:52:42.426 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-11-21T21:52:42.464 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:52:42.532 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph. 2021-11-21T21:52:42.542 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../047-ceph_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:42.581 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:42.591 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-soupsieve. 2021-11-21T21:52:42.601 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-11-21T21:52:42.641 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:52:42.801 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-jwt. 2021-11-21T21:52:42.834 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-11-21T21:52:42.876 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:52:42.926 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-fuse. 2021-11-21T21:52:42.958 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../048-ceph-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:42.961 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-bs4. 2021-11-21T21:52:42.992 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-11-21T21:52:43.000 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:43.060 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-11-21T21:52:43.188 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-logutils. 2021-11-21T21:52:43.197 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-11-21T21:52:43.237 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-11-21T21:52:43.421 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mds. 2021-11-21T21:52:43.431 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../049-ceph-mds_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:43.455 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-waitress. 2021-11-21T21:52:43.465 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-11-21T21:52:43.470 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:43.506 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-11-21T21:52:43.582 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-markupsafe. 2021-11-21T21:52:43.615 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-11-21T21:52:43.657 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:52:43.893 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-tempita. 2021-11-21T21:52:43.910 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph. 2021-11-21T21:52:43.923 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package cephadm. 2021-11-21T21:52:43.924 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-11-21T21:52:43.933 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../050-cephadm_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:43.943 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../047-ceph_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:43.967 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-11-21T21:52:43.972 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:43.976 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-mako. 2021-11-21T21:52:43.986 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-11-21T21:52:44.004 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:44.034 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:52:44.309 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-jinja2. 2021-11-21T21:52:44.320 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-paste. 2021-11-21T21:52:44.328 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-11-21T21:52:44.329 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-fuse. 2021-11-21T21:52:44.339 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../048-ceph-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:44.341 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-11-21T21:52:44.369 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:52:44.379 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:44.380 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-simplegeneric. 2021-11-21T21:52:44.383 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-11-21T21:52:44.413 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-11-21T21:52:44.455 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:52:44.672 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-11-21T21:52:44.704 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-11-21T21:52:44.723 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-11-21T21:52:44.733 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:44.747 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:52:44.758 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mds. 2021-11-21T21:52:44.770 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-singledispatch. 2021-11-21T21:52:44.778 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:44.781 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-11-21T21:52:44.790 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../049-ceph-mds_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:44.828 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:52:44.857 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:45.072 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-repoze.lru. 2021-11-21T21:52:45.086 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pastedeploy. 2021-11-21T21:52:45.097 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-11-21T21:52:45.104 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-11-21T21:52:45.135 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:52:45.147 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-11-21T21:52:45.156 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-webob. 2021-11-21T21:52:45.188 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-11-21T21:52:45.222 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-11-21T21:52:45.311 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package cephadm. 2021-11-21T21:52:45.343 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../050-cephadm_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:45.376 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:45.433 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libjs-underscore. 2021-11-21T21:52:45.447 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-webtest. 2021-11-21T21:52:45.465 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-11-21T21:52:45.479 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-11-21T21:52:45.507 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:52:45.521 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-11-21T21:52:45.592 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-soupsieve. 2021-11-21T21:52:45.603 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-11-21T21:52:45.630 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-jinja2. 2021-11-21T21:52:45.649 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:52:45.662 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-11-21T21:52:45.704 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-11-21T21:52:45.785 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-11-21T21:52:45.816 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-11-21T21:52:45.841 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pecan. 2021-11-21T21:52:45.850 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-11-21T21:52:45.859 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:52:45.898 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:52:45.957 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-11-21T21:52:45.962 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-bs4. 2021-11-21T21:52:45.990 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:45.994 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-11-21T21:52:46.024 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:46.028 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-11-21T21:52:46.204 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-routes. 2021-11-21T21:52:46.214 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-11-21T21:52:46.254 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-11-21T21:52:46.254 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-werkzeug. 2021-11-21T21:52:46.286 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-11-21T21:52:46.319 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-repoze.lru. 2021-11-21T21:52:46.328 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:52:46.329 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-11-21T21:52:46.368 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-11-21T21:52:46.415 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-waitress. 2021-11-21T21:52:46.425 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-11-21T21:52:46.465 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-11-21T21:52:46.582 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-11-21T21:52:46.614 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:46.648 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:46.654 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libjs-underscore. 2021-11-21T21:52:46.686 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-11-21T21:52:46.690 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr. 2021-11-21T21:52:46.699 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../044-ceph-mgr_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:46.729 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:52:46.739 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:46.767 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-tempita. 2021-11-21T21:52:46.800 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-11-21T21:52:46.842 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-11-21T21:52:47.023 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-11-21T21:52:47.055 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-11-21T21:52:47.098 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:52:47.154 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-paste. 2021-11-21T21:52:47.165 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-11-21T21:52:47.193 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mon. 2021-11-21T21:52:47.202 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../045-ceph-mon_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:47.203 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:52:47.250 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:47.371 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-11-21T21:52:47.380 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:47.392 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-routes. 2021-11-21T21:52:47.402 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-11-21T21:52:47.433 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-11-21T21:52:47.446 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:47.488 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-11-21T21:52:47.520 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-11-21T21:52:47.563 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:52:47.746 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-11-21T21:52:47.778 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:47.790 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-decorator. 2021-11-21T21:52:47.820 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:47.823 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-11-21T21:52:47.865 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:52:47.882 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pastedeploy. 2021-11-21T21:52:47.893 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-11-21T21:52:47.940 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:52:48.138 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-osd. 2021-11-21T21:52:48.148 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../046-ceph-osd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:48.187 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:48.201 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-11-21T21:52:48.210 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-11-21T21:52:48.250 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:52:48.277 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-webtest. 2021-11-21T21:52:48.310 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-11-21T21:52:48.369 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-11-21T21:52:48.527 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-11-21T21:52:48.538 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:48.554 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-scipy. 2021-11-21T21:52:48.585 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-11-21T21:52:48.601 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:48.628 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-11-21T21:52:48.680 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pecan. 2021-11-21T21:52:48.690 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-11-21T21:52:48.729 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:52:48.904 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-decorator. 2021-11-21T21:52:48.937 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-11-21T21:52:49.003 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:52:49.050 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-werkzeug. 2021-11-21T21:52:49.083 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-11-21T21:52:49.125 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:52:49.324 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-11-21T21:52:49.334 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-11-21T21:52:49.381 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:52:49.521 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mgr. 2021-11-21T21:52:49.531 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../044-ceph-mgr_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:49.570 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:49.718 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-scipy. 2021-11-21T21:52:49.728 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-11-21T21:52:49.751 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-11-21T21:52:49.761 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-11-21T21:52:49.793 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-11-21T21:52:49.803 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:52:49.940 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mon. 2021-11-21T21:52:49.973 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../045-ceph-mon_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:50.022 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:50.034 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph. 2021-11-21T21:52:50.065 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../047-ceph_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:50.100 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:50.315 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-joblib. 2021-11-21T21:52:50.325 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-11-21T21:52:50.373 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-11-21T21:52:50.429 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-fuse. 2021-11-21T21:52:50.439 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../048-ceph-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:50.495 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:50.701 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-sklearn. 2021-11-21T21:52:50.734 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-11-21T21:52:50.768 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-11-21T21:52:50.778 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-11-21T21:52:50.801 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:52:50.817 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:52:50.848 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mds. 2021-11-21T21:52:50.880 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../049-ceph-mds_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:50.914 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:50.935 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-osd. 2021-11-21T21:52:50.945 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../046-ceph-osd_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:50.992 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:51.305 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-joblib. 2021-11-21T21:52:51.315 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-11-21T21:52:51.354 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-11-21T21:52:51.375 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package cephadm. 2021-11-21T21:52:51.385 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../050-cephadm_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:51.400 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-11-21T21:52:51.424 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:51.433 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:51.475 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:51.711 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-jinja2. 2021-11-21T21:52:51.724 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-sklearn. 2021-11-21T21:52:51.743 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-11-21T21:52:51.757 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-11-21T21:52:51.794 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-11-21T21:52:51.799 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:52:52.114 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-11-21T21:52:52.123 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:52.163 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:52.357 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-11-21T21:52:52.368 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:52.406 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:52.445 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-cachetools. 2021-11-21T21:52:52.478 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-11-21T21:52:52.508 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-repoze.lru. 2021-11-21T21:52:52.519 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-11-21T21:52:52.541 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-11-21T21:52:52.583 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-11-21T21:52:52.856 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pyasn1. 2021-11-21T21:52:52.866 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-11-21T21:52:52.869 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libjs-underscore. 2021-11-21T21:52:52.901 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-11-21T21:52:52.913 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:52:52.923 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph. 2021-11-21T21:52:52.933 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../047-ceph_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:52.943 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:52:52.972 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:53.250 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-11-21T21:52:53.271 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-11-21T21:52:53.275 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-fuse. 2021-11-21T21:52:53.283 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-11-21T21:52:53.303 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-11-21T21:52:53.307 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../048-ceph-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:53.325 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:52:53.345 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:52:53.365 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:53.376 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-cachetools. 2021-11-21T21:52:53.386 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-11-21T21:52:53.426 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-11-21T21:52:53.661 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-rsa. 2021-11-21T21:52:53.671 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-11-21T21:52:53.673 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-routes. 2021-11-21T21:52:53.683 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-11-21T21:52:53.720 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-rsa (4.0-3) ... 2021-11-21T21:52:53.723 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-11-21T21:52:53.754 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mds. 2021-11-21T21:52:53.765 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../049-ceph-mds_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:53.771 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pyasn1. 2021-11-21T21:52:53.803 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:53.804 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-11-21T21:52:53.845 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:52:54.043 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-11-21T21:52:54.047 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-google-auth. 2021-11-21T21:52:54.075 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:54.079 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-11-21T21:52:54.117 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:54.146 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-11-21T21:52:54.199 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-11-21T21:52:54.209 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-11-21T21:52:54.210 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package cephadm. 2021-11-21T21:52:54.243 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../050-cephadm_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:54.248 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:52:54.281 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:54.450 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-blinker. 2021-11-21T21:52:54.460 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-11-21T21:52:54.499 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:52:54.567 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-jinja2. 2021-11-21T21:52:54.578 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-11-21T21:52:54.593 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-rsa. 2021-11-21T21:52:54.616 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-11-21T21:52:54.626 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-11-21T21:52:54.667 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-rsa (4.0-3) ... 2021-11-21T21:52:54.844 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-oauthlib. 2021-11-21T21:52:54.849 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-11-21T21:52:54.877 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-11-21T21:52:54.882 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:54.919 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:52:54.932 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:54.936 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-11-21T21:52:54.969 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:55.012 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:55.014 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-google-auth. 2021-11-21T21:52:55.024 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-11-21T21:52:55.070 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-11-21T21:52:55.247 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-11-21T21:52:55.257 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-11-21T21:52:55.260 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-decorator. 2021-11-21T21:52:55.292 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-11-21T21:52:55.296 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:52:55.335 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:52:55.398 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-repoze.lru. 2021-11-21T21:52:55.407 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-blinker. 2021-11-21T21:52:55.408 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-11-21T21:52:55.442 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-11-21T21:52:55.544 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:52:55.632 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-websocket. 2021-11-21T21:52:55.665 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-11-21T21:52:55.687 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-11-21T21:52:55.697 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-11-21T21:52:55.707 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:52:55.744 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:52:55.758 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-11-21T21:52:56.019 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-kubernetes. 2021-11-21T21:52:56.029 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-11-21T21:52:56.058 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libjs-underscore. 2021-11-21T21:52:56.069 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-11-21T21:52:56.076 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:52:56.116 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:52:56.184 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-oauthlib. 2021-11-21T21:52:56.217 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-11-21T21:52:56.259 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:52:56.394 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-11-21T21:52:56.425 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-11-21T21:52:56.469 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:52:56.523 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-scipy. 2021-11-21T21:52:56.532 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-11-21T21:52:56.572 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-11-21T21:52:56.613 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-11-21T21:52:56.624 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-11-21T21:52:56.662 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:52:56.797 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-routes. 2021-11-21T21:52:56.806 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-11-21T21:52:56.846 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-11-21T21:52:56.999 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-websocket. 2021-11-21T21:52:57.032 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-11-21T21:52:57.056 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-11-21T21:52:57.073 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:52:57.089 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:57.131 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:57.183 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-11-21T21:52:57.216 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:57.257 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:57.408 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libonig5:amd64. 2021-11-21T21:52:57.418 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-kubernetes. 2021-11-21T21:52:57.426 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-11-21T21:52:57.441 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-11-21T21:52:57.467 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:52:57.481 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-11-21T21:52:57.491 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-11-21T21:52:57.500 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:52:57.564 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:52:57.844 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libjq1:amd64. 2021-11-21T21:52:57.878 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:52:57.919 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:52:57.948 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-11-21T21:52:57.973 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-11-21T21:52:57.981 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:57.983 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:58.010 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-joblib. 2021-11-21T21:52:58.030 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:58.031 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:58.042 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-11-21T21:52:58.076 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-11-21T21:52:58.213 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package jq. 2021-11-21T21:52:58.246 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:52:58.288 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:52:58.325 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libonig5:amd64. 2021-11-21T21:52:58.335 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-11-21T21:52:58.374 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:52:58.375 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-decorator. 2021-11-21T21:52:58.409 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-11-21T21:52:58.421 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-sklearn. 2021-11-21T21:52:58.454 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-11-21T21:52:58.467 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:52:58.495 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:52:58.533 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package socat. 2021-11-21T21:52:58.566 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-11-21T21:52:58.598 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking socat (1.7.3.3-2) ... 2021-11-21T21:52:58.748 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libjq1:amd64. 2021-11-21T21:52:58.777 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:52:58.777 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-11-21T21:52:58.790 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-11-21T21:52:58.827 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:52:58.835 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:52:59.070 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-11-21T21:52:59.081 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:52:59.091 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-11-21T21:52:59.099 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-11-21T21:52:59.114 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package jq. 2021-11-21T21:52:59.123 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:52:59.128 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:52:59.130 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-scipy. 2021-11-21T21:52:59.138 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:52:59.163 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:52:59.163 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-11-21T21:52:59.205 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-11-21T21:52:59.391 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package xmlstarlet. 2021-11-21T21:52:59.424 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-11-21T21:52:59.449 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package socat. 2021-11-21T21:52:59.465 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:52:59.481 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-11-21T21:52:59.524 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking socat (1.7.3.3-2) ... 2021-11-21T21:52:59.787 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package ceph-test. 2021-11-21T21:52:59.820 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../083-ceph-test_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:52:59.861 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:00.012 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-11-21T21:53:00.019 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-11-21T21:53:00.062 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:53:00.080 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cachetools. 2021-11-21T21:53:00.087 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-11-21T21:53:00.122 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-11-21T21:53:00.147 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-11-21T21:53:00.157 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-11-21T21:53:00.196 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:00.366 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package xmlstarlet. 2021-11-21T21:53:00.399 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-11-21T21:53:00.416 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyasn1. 2021-11-21T21:53:00.440 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:53:00.448 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-11-21T21:53:00.491 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:53:00.687 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-joblib. 2021-11-21T21:53:00.693 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-11-21T21:53:00.780 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package ceph-test. 2021-11-21T21:53:00.809 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-11-21T21:53:00.812 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../083-ceph-test_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:00.828 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-11-21T21:53:00.839 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-11-21T21:53:00.853 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:00.878 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:53:01.170 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-sklearn. 2021-11-21T21:53:01.203 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-11-21T21:53:01.222 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rsa. 2021-11-21T21:53:01.245 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:01.255 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-11-21T21:53:01.296 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rsa (4.0-3) ... 2021-11-21T21:53:01.616 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-google-auth. 2021-11-21T21:53:01.625 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-11-21T21:53:01.665 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-11-21T21:53:01.786 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-11-21T21:53:01.797 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:53:01.844 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:02.002 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-blinker. 2021-11-21T21:53:02.035 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-11-21T21:53:02.077 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:53:02.405 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-oauthlib. 2021-11-21T21:53:02.416 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-11-21T21:53:02.454 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:53:02.574 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-colorama. 2021-11-21T21:53:02.584 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-11-21T21:53:02.623 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:02.808 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-11-21T21:53:02.818 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package cephfs-shell. 2021-11-21T21:53:02.840 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-11-21T21:53:02.847 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-cachetools. 2021-11-21T21:53:02.850 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../085-cephfs-shell_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:53:02.880 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-11-21T21:53:02.882 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:53:02.893 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:02.922 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-11-21T21:53:03.171 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libcephfs-dev. 2021-11-21T21:53:03.202 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-websocket. 2021-11-21T21:53:03.204 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:03.211 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-11-21T21:53:03.233 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pyasn1. 2021-11-21T21:53:03.246 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:03.250 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:53:03.265 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-11-21T21:53:03.307 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:53:03.563 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-kubernetes. 2021-11-21T21:53:03.572 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-11-21T21:53:03.612 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:53:03.657 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package nvme-cli. 2021-11-21T21:53:03.661 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-11-21T21:53:03.667 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-11-21T21:53:03.672 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-11-21T21:53:03.675 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-colorama. 2021-11-21T21:53:03.707 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-11-21T21:53:03.710 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:53:03.713 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:03.751 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:03.987 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-atomicwrites. 2021-11-21T21:53:04.020 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package cephfs-shell. 2021-11-21T21:53:04.021 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-11-21T21:53:04.022 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-rsa. 2021-11-21T21:53:04.034 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-11-21T21:53:04.051 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../085-cephfs-shell_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:53:04.053 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:04.054 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-11-21T21:53:04.066 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:53:04.086 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:04.096 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-rsa (4.0-3) ... 2021-11-21T21:53:04.108 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:04.313 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libcephfs-dev. 2021-11-21T21:53:04.340 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-attr. 2021-11-21T21:53:04.344 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:04.372 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-11-21T21:53:04.414 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-11-21T21:53:04.415 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:04.449 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-google-auth. 2021-11-21T21:53:04.463 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-11-21T21:53:04.469 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libonig5:amd64. 2021-11-21T21:53:04.479 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-11-21T21:53:04.507 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-11-21T21:53:04.527 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:53:04.717 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-webencodings. 2021-11-21T21:53:04.728 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-11-21T21:53:04.766 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:04.833 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package nvme-cli. 2021-11-21T21:53:04.836 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-blinker. 2021-11-21T21:53:04.843 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-11-21T21:53:04.872 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-11-21T21:53:04.881 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libjq1:amd64. 2021-11-21T21:53:04.890 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:04.911 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:53:04.914 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:53:04.955 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:05.121 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-html5lib. 2021-11-21T21:53:05.152 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-11-21T21:53:05.194 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:05.222 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-oauthlib. 2021-11-21T21:53:05.232 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-11-21T21:53:05.249 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package jq. 2021-11-21T21:53:05.255 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-atomicwrites. 2021-11-21T21:53:05.271 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:53:05.281 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:53:05.288 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-11-21T21:53:05.324 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:05.329 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:05.489 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-11-21T21:53:05.523 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-11-21T21:53:05.588 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:05.591 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-attr. 2021-11-21T21:53:05.591 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-11-21T21:53:05.601 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package socat. 2021-11-21T21:53:05.624 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-11-21T21:53:05.625 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-11-21T21:53:05.635 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-11-21T21:53:05.666 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:53:05.691 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-11-21T21:53:05.692 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking socat (1.7.3.3-2) ... 2021-11-21T21:53:06.002 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-webencodings. 2021-11-21T21:53:06.008 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-packaging. 2021-11-21T21:53:06.012 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-11-21T21:53:06.019 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-11-21T21:53:06.028 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-websocket. 2021-11-21T21:53:06.037 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-11-21T21:53:06.051 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:06.058 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-packaging (20.3-1) ... 2021-11-21T21:53:06.077 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:53:06.157 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-11-21T21:53:06.168 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-11-21T21:53:06.207 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:53:06.319 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pastescript. 2021-11-21T21:53:06.352 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-11-21T21:53:06.385 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:06.388 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-kubernetes. 2021-11-21T21:53:06.396 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-html5lib. 2021-11-21T21:53:06.422 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-11-21T21:53:06.429 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-11-21T21:53:06.463 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:53:06.471 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:06.493 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package xmlstarlet. 2021-11-21T21:53:06.526 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-11-21T21:53:06.568 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:53:06.648 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pluggy. 2021-11-21T21:53:06.681 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-11-21T21:53:06.722 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:06.824 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-11-21T21:53:06.834 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-11-21T21:53:06.873 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:06.924 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-test. 2021-11-21T21:53:06.934 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../083-ceph-test_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:06.943 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-11-21T21:53:06.973 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:06.976 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:53:07.017 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:07.059 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-psutil. 2021-11-21T21:53:07.069 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-11-21T21:53:07.108 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:07.277 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-packaging. 2021-11-21T21:53:07.311 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-11-21T21:53:07.345 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libonig5:amd64. 2021-11-21T21:53:07.356 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-11-21T21:53:07.377 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-packaging (20.3-1) ... 2021-11-21T21:53:07.395 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:53:07.428 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-py. 2021-11-21T21:53:07.461 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2021-11-21T21:53:07.502 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:07.688 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pastescript. 2021-11-21T21:53:07.698 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-11-21T21:53:07.764 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libjq1:amd64. 2021-11-21T21:53:07.771 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:07.798 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:53:07.839 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:07.840 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pyinotify. 2021-11-21T21:53:07.850 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-11-21T21:53:07.888 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:08.100 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package jq. 2021-11-21T21:53:08.101 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pluggy. 2021-11-21T21:53:08.133 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-11-21T21:53:08.268 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:08.268 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-pytest. 2021-11-21T21:53:08.268 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-11-21T21:53:08.269 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-11-21T21:53:08.269 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:08.283 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-11-21T21:53:08.444 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package socat. 2021-11-21T21:53:08.477 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-11-21T21:53:08.518 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking socat (1.7.3.3-2) ... 2021-11-21T21:53:08.519 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-psutil. 2021-11-21T21:53:08.530 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-11-21T21:53:08.569 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:08.603 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package python3-simplejson. 2021-11-21T21:53:08.612 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-11-21T21:53:08.652 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:08.889 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-py. 2021-11-21T21:53:08.921 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2021-11-21T21:53:08.963 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:08.975 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-11-21T21:53:08.986 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-11-21T21:53:09.024 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:53:09.090 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package radosgw. 2021-11-21T21:53:09.099 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../101-radosgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:09.139 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:09.309 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pyinotify. 2021-11-21T21:53:09.311 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package xmlstarlet. 2021-11-21T21:53:09.320 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-11-21T21:53:09.344 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-11-21T21:53:09.358 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:09.385 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:53:09.636 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-colorama. 2021-11-21T21:53:09.670 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-11-21T21:53:09.703 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-pytest. 2021-11-21T21:53:09.708 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package ceph-test. 2021-11-21T21:53:09.710 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:09.736 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-11-21T21:53:09.741 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../083-ceph-test_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:09.777 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-11-21T21:53:09.783 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:09.996 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package cephfs-shell. 2021-11-21T21:53:10.011 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../085-cephfs-shell_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:53:10.025 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package rbd-fuse. 2021-11-21T21:53:10.034 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../102-rbd-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:10.057 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:10.075 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:10.124 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package python3-simplejson. 2021-11-21T21:53:10.132 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-11-21T21:53:10.172 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:10.283 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libcephfs-dev. 2021-11-21T21:53:10.316 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:10.357 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:10.385 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-attr (19.3.0-2) ... 2021-11-21T21:53:10.610 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package radosgw. 2021-11-21T21:53:10.621 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../101-radosgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:10.630 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:53:10.667 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:10.731 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:53:10.769 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package nvme-cli. 2021-11-21T21:53:10.789 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-11-21T21:53:10.825 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:10.833 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:11.042 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:11.156 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-atomicwrites. 2021-11-21T21:53:11.190 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-11-21T21:53:11.231 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:11.501 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-attr. 2021-11-21T21:53:11.534 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-11-21T21:53:11.554 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package rbd-fuse. 2021-11-21T21:53:11.567 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-11-21T21:53:11.592 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../102-rbd-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:11.629 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:11.671 INFO:teuthology.orchestra.run.gibba016.stdout:Adding system user cephadm....done 2021-11-21T21:53:11.831 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-11-21T21:53:11.845 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-webencodings. 2021-11-21T21:53:11.879 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-11-21T21:53:11.919 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:11.948 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-attr (19.3.0-2) ... 2021-11-21T21:53:12.063 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-11-21T21:53:12.216 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:53:12.222 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-html5lib. 2021-11-21T21:53:12.233 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-11-21T21:53:12.242 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-11-21T21:53:12.271 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:12.335 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:53:12.392 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:12.478 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:12.537 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-colorama. 2021-11-21T21:53:12.548 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-11-21T21:53:12.602 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:12.617 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-11-21T21:53:12.618 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-11-21T21:53:12.651 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-11-21T21:53:12.687 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:12.692 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:12.865 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package cephfs-shell. 2021-11-21T21:53:12.896 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../085-cephfs-shell_15.2.15-36-g911bc85f-1focal_all.deb ... 2021-11-21T21:53:12.940 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:12.962 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:53:13.146 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-packaging. 2021-11-21T21:53:13.156 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-11-21T21:53:13.194 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-packaging (20.3-1) ... 2021-11-21T21:53:13.204 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-11-21T21:53:13.209 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libcephfs-dev. 2021-11-21T21:53:13.239 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:13.283 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:13.367 INFO:teuthology.orchestra.run.gibba017.stdout:Adding system user cephadm....done 2021-11-21T21:53:13.389 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:13.527 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-11-21T21:53:13.533 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pastescript. 2021-11-21T21:53:13.563 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-11-21T21:53:13.574 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:53:13.605 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:13.678 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package nvme-cli. 2021-11-21T21:53:13.688 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-11-21T21:53:13.743 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:13.769 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-11-21T21:53:13.833 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:13.951 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pluggy. 2021-11-21T21:53:13.962 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-11-21T21:53:13.979 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-11-21T21:53:14.001 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:14.084 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:14.092 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-atomicwrites. 2021-11-21T21:53:14.124 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-11-21T21:53:14.131 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:14.167 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:14.335 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:14.346 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-psutil. 2021-11-21T21:53:14.365 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-11-21T21:53:14.379 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-11-21T21:53:14.420 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:14.462 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-11-21T21:53:14.495 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-attr. 2021-11-21T21:53:14.505 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-11-21T21:53:14.547 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-11-21T21:53:14.662 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:53:14.708 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:53:14.764 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-py. 2021-11-21T21:53:14.776 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2021-11-21T21:53:14.814 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:14.855 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-webencodings. 2021-11-21T21:53:14.889 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-11-21T21:53:14.897 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:14.932 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:14.968 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-11-21T21:53:15.117 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:15.159 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyinotify. 2021-11-21T21:53:15.192 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-11-21T21:53:15.211 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:15.234 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:15.275 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-html5lib. 2021-11-21T21:53:15.286 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-11-21T21:53:15.324 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:15.399 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:15.454 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:53:15.562 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pytest. 2021-11-21T21:53:15.571 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-11-21T21:53:15.645 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-11-21T21:53:15.650 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:15.661 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-11-21T21:53:15.695 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-11-21T21:53:15.696 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:15.736 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:15.953 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:53:15.957 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-simplejson. 2021-11-21T21:53:15.981 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:15.990 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-11-21T21:53:16.031 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:16.194 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-11-21T21:53:16.205 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-packaging. 2021-11-21T21:53:16.215 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-11-21T21:53:16.215 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:16.255 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-packaging (20.3-1) ... 2021-11-21T21:53:16.341 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-11-21T21:53:16.437 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-11-21T21:53:16.493 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package radosgw. 2021-11-21T21:53:16.503 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../101-radosgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:16.550 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:16.551 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:53:16.567 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pastescript. 2021-11-21T21:53:16.602 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-11-21T21:53:16.641 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:16.646 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-packaging (20.3-1) ... 2021-11-21T21:53:16.810 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:16.874 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:53:16.991 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:53:16.995 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pluggy. 2021-11-21T21:53:17.005 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-11-21T21:53:17.044 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:17.053 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:17.175 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:53:17.313 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:17.357 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-psutil. 2021-11-21T21:53:17.389 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-11-21T21:53:17.431 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:17.435 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:53:17.472 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package rbd-fuse. 2021-11-21T21:53:17.504 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../102-rbd-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:17.546 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:17.580 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:17.694 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:53:17.818 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-py. 2021-11-21T21:53:17.829 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2021-11-21T21:53:17.855 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-attr (19.3.0-2) ... 2021-11-21T21:53:17.875 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:17.899 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:53:17.954 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:53:18.108 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-11-21T21:53:18.175 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:53:18.204 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pyinotify. 2021-11-21T21:53:18.237 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-11-21T21:53:18.255 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up socat (1.7.3.3-2) ... 2021-11-21T21:53:18.287 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:18.309 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:53:18.390 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:18.392 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-11-21T21:53:18.435 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:18.584 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:18.623 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-pytest. 2021-11-21T21:53:18.626 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-packaging (20.3-1) ... 2021-11-21T21:53:18.632 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-11-21T21:53:18.637 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:18.672 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-11-21T21:53:18.843 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:53:18.895 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:53:18.988 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:18.993 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package python3-simplejson. 2021-11-21T21:53:19.021 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:53:19.027 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-11-21T21:53:19.068 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:19.187 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:53:19.247 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:53:19.313 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:53:19.348 INFO:teuthology.orchestra.run.gibba019.stdout:Adding system user cephadm....done 2021-11-21T21:53:19.517 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-11-21T21:53:19.530 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package radosgw. 2021-11-21T21:53:19.532 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:53:19.540 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../101-radosgw_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:19.579 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:19.580 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:53:19.744 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-11-21T21:53:19.792 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:53:19.823 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-11-21T21:53:20.008 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:53:20.021 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-11-21T21:53:20.093 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:53:20.109 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:53:20.147 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:20.368 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-11-21T21:53:20.373 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-11-21T21:53:20.387 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up socat (1.7.3.3-2) ... 2021-11-21T21:53:20.466 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package rbd-fuse. 2021-11-21T21:53:20.500 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../102-rbd-fuse_15.2.15-36-g911bc85f-1focal_amd64.deb ... 2021-11-21T21:53:20.513 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:20.541 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:20.619 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:53:20.666 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:53:20.739 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:20.829 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:53:20.842 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-attr (19.3.0-2) ... 2021-11-21T21:53:20.917 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-11-21T21:53:20.982 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:53:21.104 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-11-21T21:53:21.108 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:21.168 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:21.205 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-11-21T21:53:21.278 INFO:teuthology.orchestra.run.gibba016.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-11-21T21:53:21.305 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-11-21T21:53:21.376 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:53:21.394 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:53:21.423 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:53:21.473 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:21.502 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:53:21.524 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:53:21.637 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:21.767 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:53:21.779 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:53:21.888 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:22.013 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-11-21T21:53:22.046 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:53:22.079 INFO:teuthology.orchestra.run.gibba036.stdout:Adding system user cephadm....done 2021-11-21T21:53:22.139 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:22.172 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:53:22.223 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:53:22.230 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-11-21T21:53:22.266 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-11-21T21:53:22.299 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-11-21T21:53:22.349 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:53:22.456 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-11-21T21:53:22.491 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:53:22.555 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:53:22.574 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-11-21T21:53:22.682 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-11-21T21:53:22.700 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:22.764 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up librados2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:22.783 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2021-11-21T21:53:22.784 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:53:22.882 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-11-21T21:53:22.951 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:23.010 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-11-21T21:53:23.027 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:53:23.236 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:23.336 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-11-21T21:53:23.479 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:23.485 INFO:teuthology.orchestra.run.gibba017.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-11-21T21:53:23.562 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-11-21T21:53:23.665 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:53:23.747 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-11-21T21:53:23.790 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:53:23.798 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:53:23.973 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-11-21T21:53:23.997 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:24.017 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:53:24.040 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-11-21T21:53:24.122 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:24.199 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-11-21T21:53:24.249 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-11-21T21:53:24.300 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:53:24.341 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-11-21T21:53:24.416 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-ceph-argparse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:24.426 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:53:24.516 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-11-21T21:53:24.577 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-11-21T21:53:24.592 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-packaging (20.3-1) ... 2021-11-21T21:53:24.660 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:24.711 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:53:24.786 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-11-21T21:53:24.801 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:53:24.803 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:53:24.935 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-11-21T21:53:24.961 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:53:24.987 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-11-21T21:53:25.029 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up librados2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:25.156 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-11-21T21:53:25.203 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:25.211 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:53:25.255 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-11-21T21:53:25.329 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:53:25.456 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-11-21T21:53:25.522 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:53:25.555 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:25.682 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:53:25.723 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-11-21T21:53:25.807 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:53:25.966 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-11-21T21:53:25.982 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:26.108 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:53:26.244 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:26.276 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-11-21T21:53:26.354 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:26.443 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up socat (1.7.3.3-2) ... 2021-11-21T21:53:26.569 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:26.609 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-11-21T21:53:26.755 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up librbd1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:26.827 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-11-21T21:53:26.845 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:26.881 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:53:26.998 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:53:27.112 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-11-21T21:53:27.132 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:27.306 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-11-21T21:53:27.321 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-packaging (20.3-1) ... 2021-11-21T21:53:27.335 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:27.435 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-rsa (4.0-3) ... 2021-11-21T21:53:27.543 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:53:27.566 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:53:27.606 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-11-21T21:53:27.722 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:27.732 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-11-21T21:53:27.809 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-11-21T21:53:27.927 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:53:27.941 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-11-21T21:53:28.028 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-11-21T21:53:28.050 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:53:28.126 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:28.226 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-11-21T21:53:28.237 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:28.255 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:53:28.343 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:53:28.363 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:28.485 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-11-21T21:53:28.495 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:28.521 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:28.521 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:28.553 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-11-21T21:53:28.622 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:53:28.787 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:53:28.797 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-11-21T21:53:28.831 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:28.890 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:28.921 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:53:28.940 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:53:29.089 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up socat (1.7.3.3-2) ... 2021-11-21T21:53:29.121 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-11-21T21:53:29.125 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:29.219 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:29.392 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:53:29.393 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:53:29.458 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-11-21T21:53:29.649 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:53:29.670 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up librbd1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:29.670 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:29.701 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-11-21T21:53:29.796 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:53:29.797 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:29.827 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-11-21T21:53:29.922 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:53:29.923 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:30.073 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:30.081 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-routes (2.4.1-2) ... 2021-11-21T21:53:30.095 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-11-21T21:53:30.156 INFO:teuthology.orchestra.run.gibba019.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-11-21T21:53:30.213 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-11-21T21:53:30.282 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:30.302 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:53:30.342 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-11-21T21:53:30.428 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:53:30.434 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-rsa (4.0-3) ... 2021-11-21T21:53:30.439 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-11-21T21:53:30.576 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:30.677 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:30.678 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:53:30.682 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-11-21T21:53:30.684 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-11-21T21:53:30.902 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:30.903 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:30.921 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:53:30.941 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-11-21T21:53:31.028 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:31.030 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:53:31.048 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:53:31.068 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-11-21T21:53:31.174 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-11-21T21:53:31.186 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:31.187 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:31.276 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-11-21T21:53:31.442 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:53:31.505 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:31.528 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-11-21T21:53:31.692 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:53:31.693 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librados2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:31.717 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:31.788 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-11-21T21:53:31.819 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-11-21T21:53:31.920 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:32.174 INFO:teuthology.orchestra.run.gibba016.stdout:Adding group ceph....done 2021-11-21T21:53:32.227 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:53:32.230 INFO:teuthology.orchestra.run.gibba036.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-11-21T21:53:32.382 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-11-21T21:53:32.509 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-11-21T21:53:32.563 INFO:teuthology.orchestra.run.gibba016.stdout:Adding system user ceph....done 2021-11-21T21:53:32.728 INFO:teuthology.orchestra.run.gibba016.stdout:Setting system user ceph properties....done 2021-11-21T21:53:32.745 INFO:teuthology.orchestra.run.gibba016.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-11-21T21:53:32.750 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-11-21T21:53:32.770 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:32.821 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-11-21T21:53:32.896 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:32.899 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:33.013 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-11-21T21:53:33.025 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:33.043 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-11-21T21:53:33.047 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:33.170 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-11-21T21:53:33.172 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-routes (2.4.1-2) ... 2021-11-21T21:53:33.175 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-11-21T21:53:33.296 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-11-21T21:53:33.350 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:33.446 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-11-21T21:53:33.450 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-11-21T21:53:33.485 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:33.522 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-11-21T21:53:33.659 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:33.660 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:33.702 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:33.703 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:53:33.765 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up librados2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:33.844 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:33.891 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-11-21T21:53:33.930 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-11-21T21:53:34.054 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:34.103 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:34.197 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:53:34.206 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:34.332 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:53:34.340 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-11-21T21:53:34.591 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:34.663 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:34.718 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:53:34.849 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:34.850 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:34.883 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:34.979 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up librgw2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:34.994 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:35.123 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:35.140 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:35.249 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-11-21T21:53:35.300 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:35.301 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:35.447 INFO:teuthology.orchestra.run.gibba017.stdout:Adding group ceph....done 2021-11-21T21:53:35.516 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-11-21T21:53:35.599 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:35.750 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:35.767 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-11-21T21:53:35.783 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librbd1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:35.789 INFO:teuthology.orchestra.run.gibba017.stdout:Adding system user ceph....done 2021-11-21T21:53:35.909 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:53:35.961 INFO:teuthology.orchestra.run.gibba017.stdout:Setting system user ceph properties....done 2021-11-21T21:53:35.970 INFO:teuthology.orchestra.run.gibba017.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-11-21T21:53:36.019 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-11-21T21:53:36.035 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:53:36.062 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-11-21T21:53:36.136 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:36.136 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:36.160 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:36.260 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-11-21T21:53:36.345 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libcephfs2 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:36.387 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:36.419 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:36.471 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-11-21T21:53:36.510 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:36.513 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rsa (4.0-3) ... 2021-11-21T21:53:36.528 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:36.619 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:36.662 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:36.707 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libradosstriper1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:36.731 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-11-21T21:53:36.846 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:36.846 INFO:teuthology.orchestra.run.gibba016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:36.851 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:36.852 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:36.858 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-11-21T21:53:36.982 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.108 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.130 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.133 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-11-21T21:53:37.262 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.265 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.308 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:37.309 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:37.383 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.480 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-11-21T21:53:37.518 INFO:teuthology.orchestra.run.gibba016.stdout:Processing triggers for systemd (245.4-4ubuntu3.13) ... 2021-11-21T21:53:37.600 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.718 INFO:teuthology.orchestra.run.gibba016.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:37.727 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:53:37.772 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.930 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up librbd1 (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:37.936 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:37.960 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:37.961 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:38.048 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-11-21T21:53:38.175 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-11-21T21:53:38.205 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:53:38.207 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:38.301 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-11-21T21:53:38.395 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:38.396 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:38.456 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:38.560 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-11-21T21:53:38.583 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:38.684 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:38.686 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-rsa (4.0-3) ... 2021-11-21T21:53:38.708 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:38.852 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:38.860 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-routes (2.4.1-2) ... 2021-11-21T21:53:38.937 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-11-21T21:53:39.161 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-11-21T21:53:39.196 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mgr-modules-core (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.256 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:39.256 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:39.356 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.422 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.506 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:39.506 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-11-21T21:53:39.513 INFO:teuthology.orchestra.run.gibba016.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:39.546 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.548 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.697 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.757 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.774 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libcephfs-dev (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.823 INFO:teuthology.orchestra.run.gibba016.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:53:39.825 DEBUG:teuthology.orchestra.run.gibba016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-11-21T21:53:39.831 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:39.861 INFO:teuthology.orchestra.run.gibba016.stdout:Reading package lists... 2021-11-21T21:53:39.883 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:53:39.902 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-11-21T21:53:39.968 INFO:teuthology.orchestra.run.gibba016.stdout:Building dependency tree... 2021-11-21T21:53:39.969 INFO:teuthology.orchestra.run.gibba016.stdout:Reading state information... 2021-11-21T21:53:39.989 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:39.990 INFO:teuthology.orchestra.run.gibba017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:40.051 INFO:teuthology.orchestra.run.gibba016.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-11-21T21:53:40.052 INFO:teuthology.orchestra.run.gibba016.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-11-21T21:53:40.052 INFO:teuthology.orchestra.run.gibba016.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-11-21T21:53:40.053 INFO:teuthology.orchestra.run.gibba016.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:53:40.053 INFO:teuthology.orchestra.run.gibba016.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:53:40.054 INFO:teuthology.orchestra.run.gibba016.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:53:40.054 INFO:teuthology.orchestra.run.gibba016.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:53:40.055 INFO:teuthology.orchestra.run.gibba016.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:53:40.055 INFO:teuthology.orchestra.run.gibba016.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:53:40.055 INFO:teuthology.orchestra.run.gibba016.stdout:The following additional packages will be installed: 2021-11-21T21:53:40.055 INFO:teuthology.orchestra.run.gibba016.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-11-21T21:53:40.056 INFO:teuthology.orchestra.run.gibba016.stdout:Suggested packages: 2021-11-21T21:53:40.056 INFO:teuthology.orchestra.run.gibba016.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-11-21T21:53:40.063 INFO:teuthology.orchestra.run.gibba016.stdout:The following NEW packages will be installed: 2021-11-21T21:53:40.064 INFO:teuthology.orchestra.run.gibba016.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-11-21T21:53:40.064 INFO:teuthology.orchestra.run.gibba016.stdout: libssl-dev 2021-11-21T21:53:40.065 INFO:teuthology.orchestra.run.gibba016.stdout:The following packages will be upgraded: 2021-11-21T21:53:40.065 INFO:teuthology.orchestra.run.gibba016.stdout: libssl1.1 2021-11-21T21:53:40.110 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-11-21T21:53:40.282 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:40.285 INFO:teuthology.orchestra.run.gibba016.stdout:1 upgraded, 10 newly installed, 0 to remove and 51 not upgraded. 2021-11-21T21:53:40.285 INFO:teuthology.orchestra.run.gibba016.stdout:Need to get 4,233 kB of archives. 2021-11-21T21:53:40.285 INFO:teuthology.orchestra.run.gibba016.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-11-21T21:53:40.285 INFO:teuthology.orchestra.run.gibba016.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-11-21T21:53:40.336 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-11-21T21:53:40.409 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:40.535 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:40.586 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:40.629 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-rados (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:40.653 INFO:teuthology.orchestra.run.gibba017.stdout:Processing triggers for systemd (245.4-4ubuntu3.13) ... 2021-11-21T21:53:40.756 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-rbd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:40.767 INFO:teuthology.orchestra.run.gibba016.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.9 [1,321 kB] 2021-11-21T21:53:40.862 INFO:teuthology.orchestra.run.gibba017.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:40.915 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up rbd-fuse (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:41.021 INFO:teuthology.orchestra.run.gibba016.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-11-21T21:53:41.041 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-routes (2.4.1-2) ... 2021-11-21T21:53:41.067 INFO:teuthology.orchestra.run.gibba016.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-11-21T21:53:41.069 INFO:teuthology.orchestra.run.gibba016.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-11-21T21:53:41.077 INFO:teuthology.orchestra.run.gibba016.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-11-21T21:53:41.083 INFO:teuthology.orchestra.run.gibba016.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-11-21T21:53:41.086 INFO:teuthology.orchestra.run.gibba016.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-11-21T21:53:41.094 INFO:teuthology.orchestra.run.gibba016.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-11-21T21:53:41.094 INFO:teuthology.orchestra.run.gibba016.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-11-21T21:53:41.095 INFO:teuthology.orchestra.run.gibba016.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.9 [1,583 kB] 2021-11-21T21:53:41.095 INFO:teuthology.orchestra.run.gibba019.stdout:Adding group ceph....done 2021-11-21T21:53:41.326 INFO:teuthology.orchestra.run.gibba016.stdout:Preconfiguring packages ... 2021-11-21T21:53:41.335 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-11-21T21:53:41.409 INFO:teuthology.orchestra.run.gibba016.stdout:Fetched 4,233 kB in 1s (3,868 kB/s) 2021-11-21T21:53:41.475 INFO:teuthology.orchestra.run.gibba019.stdout:Adding system user ceph....done 2021-11-21T21:53:41.536 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package flex. 2021-11-21T21:53:41.610 INFO:teuthology.orchestra.run.gibba016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 182110 files and directories currently installed.) 2021-11-21T21:53:41.614 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:41.628 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-rgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:41.643 INFO:teuthology.orchestra.run.gibba019.stdout:Setting system user ceph properties....done 2021-11-21T21:53:41.650 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking flex (2.6.4-6.2) ... 2021-11-21T21:53:41.651 INFO:teuthology.orchestra.run.gibba019.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-11-21T21:53:41.724 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-11-21T21:53:41.780 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-cephfs (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:41.905 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-11-21T21:53:41.980 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up cephfs-shell (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:42.082 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:42.107 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-11-21T21:53:42.212 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:42.247 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.9) over (1.1.1f-1ubuntu2.8) ... 2021-11-21T21:53:42.338 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:42.357 INFO:teuthology.orchestra.run.gibba017.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:42.548 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:42.548 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:42.768 INFO:teuthology.orchestra.run.gibba017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:53:42.769 DEBUG:teuthology.orchestra.run.gibba017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-11-21T21:53:42.777 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package bison. 2021-11-21T21:53:42.785 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-11-21T21:53:42.793 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-common (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:42.849 INFO:teuthology.orchestra.run.gibba017.stdout:Reading package lists... 2021-11-21T21:53:42.899 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:42.951 INFO:teuthology.orchestra.run.gibba017.stdout:Building dependency tree... 2021-11-21T21:53:42.952 INFO:teuthology.orchestra.run.gibba017.stdout:Reading state information... 2021-11-21T21:53:42.965 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:43.031 INFO:teuthology.orchestra.run.gibba017.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-11-21T21:53:43.031 INFO:teuthology.orchestra.run.gibba017.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-11-21T21:53:43.032 INFO:teuthology.orchestra.run.gibba017.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-11-21T21:53:43.032 INFO:teuthology.orchestra.run.gibba017.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:53:43.032 INFO:teuthology.orchestra.run.gibba017.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:53:43.032 INFO:teuthology.orchestra.run.gibba017.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:53:43.033 INFO:teuthology.orchestra.run.gibba017.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:53:43.033 INFO:teuthology.orchestra.run.gibba017.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:53:43.033 INFO:teuthology.orchestra.run.gibba017.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:53:43.033 INFO:teuthology.orchestra.run.gibba017.stdout:The following additional packages will be installed: 2021-11-21T21:53:43.034 INFO:teuthology.orchestra.run.gibba017.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-11-21T21:53:43.034 INFO:teuthology.orchestra.run.gibba017.stdout:Suggested packages: 2021-11-21T21:53:43.034 INFO:teuthology.orchestra.run.gibba017.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-11-21T21:53:43.044 INFO:teuthology.orchestra.run.gibba017.stdout:The following NEW packages will be installed: 2021-11-21T21:53:43.044 INFO:teuthology.orchestra.run.gibba017.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-11-21T21:53:43.045 INFO:teuthology.orchestra.run.gibba017.stdout: libssl-dev 2021-11-21T21:53:43.045 INFO:teuthology.orchestra.run.gibba017.stdout:The following packages will be upgraded: 2021-11-21T21:53:43.046 INFO:teuthology.orchestra.run.gibba017.stdout: libssl1.1 2021-11-21T21:53:43.166 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-11-21T21:53:43.225 INFO:teuthology.orchestra.run.gibba017.stdout:1 upgraded, 10 newly installed, 0 to remove and 51 not upgraded. 2021-11-21T21:53:43.225 INFO:teuthology.orchestra.run.gibba017.stdout:Need to get 4,233 kB of archives. 2021-11-21T21:53:43.225 INFO:teuthology.orchestra.run.gibba017.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-11-21T21:53:43.226 INFO:teuthology.orchestra.run.gibba017.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-11-21T21:53:43.323 INFO:teuthology.orchestra.run.gibba036.stdout:Adding group ceph....done 2021-11-21T21:53:43.459 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:43.498 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-11-21T21:53:43.508 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-11-21T21:53:43.547 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:43.625 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:43.626 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:43.682 INFO:teuthology.orchestra.run.gibba036.stdout:Adding system user ceph....done 2021-11-21T21:53:43.722 INFO:teuthology.orchestra.run.gibba017.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.9 [1,321 kB] 2021-11-21T21:53:43.801 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package dump. 2021-11-21T21:53:43.809 INFO:teuthology.orchestra.run.gibba036.stdout:Setting system user ceph properties....done 2021-11-21T21:53:43.823 INFO:teuthology.orchestra.run.gibba036.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-11-21T21:53:43.833 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-11-21T21:53:43.867 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking dump (0.4b46-6) ... 2021-11-21T21:53:43.903 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-11-21T21:53:43.929 INFO:teuthology.orchestra.run.gibba017.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-11-21T21:53:43.935 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:43.974 INFO:teuthology.orchestra.run.gibba017.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-11-21T21:53:43.976 INFO:teuthology.orchestra.run.gibba017.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-11-21T21:53:43.980 INFO:teuthology.orchestra.run.gibba017.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-11-21T21:53:43.983 INFO:teuthology.orchestra.run.gibba017.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-11-21T21:53:43.984 INFO:teuthology.orchestra.run.gibba017.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-11-21T21:53:43.987 INFO:teuthology.orchestra.run.gibba017.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-11-21T21:53:43.987 INFO:teuthology.orchestra.run.gibba017.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-11-21T21:53:43.988 INFO:teuthology.orchestra.run.gibba017.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.9 [1,583 kB] 2021-11-21T21:53:44.059 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-11-21T21:53:44.159 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:44.159 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:44.170 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package indent. 2021-11-21T21:53:44.203 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-11-21T21:53:44.218 INFO:teuthology.orchestra.run.gibba017.stdout:Preconfiguring packages ... 2021-11-21T21:53:44.245 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking indent (2.2.12-1) ... 2021-11-21T21:53:44.320 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-test (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:44.343 INFO:teuthology.orchestra.run.gibba017.stdout:Fetched 4,233 kB in 1s (4,218 kB/s) 2021-11-21T21:53:44.446 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up radosgw (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:44.447 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package flex. 2021-11-21T21:53:44.514 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-11-21T21:53:44.520 INFO:teuthology.orchestra.run.gibba017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 182110 files and directories currently installed.) 2021-11-21T21:53:44.524 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:44.525 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-11-21T21:53:44.562 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking flex (2.6.4-6.2) ... 2021-11-21T21:53:44.563 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:44.620 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:44.649 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:44.650 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-11-21T21:53:44.755 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:44.926 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-11-21T21:53:44.959 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-11-21T21:53:45.001 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:45.022 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-base (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:45.034 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:45.145 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:45.145 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:45.225 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.9) over (1.1.1f-1ubuntu2.8) ... 2021-11-21T21:53:45.292 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-11-21T21:53:45.362 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libfl2:amd64. 2021-11-21T21:53:45.372 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:45.411 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:45.424 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:45.549 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mds (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:45.550 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:45.689 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-11-21T21:53:45.700 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:45.717 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:45.717 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-11-21T21:53:45.723 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:45.764 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:45.780 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package bison. 2021-11-21T21:53:45.791 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-11-21T21:53:45.831 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:45.843 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:45.843 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:45.959 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mgr (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:46.042 INFO:teuthology.orchestra.run.gibba016.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-11-21T21:53:46.075 INFO:teuthology.orchestra.run.gibba016.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:46.116 INFO:teuthology.orchestra.run.gibba016.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:46.135 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:46.153 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:46.154 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-11-21T21:53:46.259 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-11-21T21:53:46.292 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-11-21T21:53:46.294 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:46.351 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:46.395 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:46.395 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:46.512 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-osd (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:46.530 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for systemd (245.4-4ubuntu3.13) ... 2021-11-21T21:53:46.556 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up flex (2.6.4-6.2) ... 2021-11-21T21:53:46.620 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package dump. 2021-11-21T21:53:46.655 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-11-21T21:53:46.679 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:46.695 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking dump (0.4b46-6) ... 2021-11-21T21:53:46.739 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:46.875 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:46.875 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-11-21T21:53:46.955 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:46.974 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package indent. 2021-11-21T21:53:47.007 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-11-21T21:53:47.048 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking indent (2.2.12-1) ... 2021-11-21T21:53:47.081 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:47.139 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mgr-k8sevents (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:47.207 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:47.265 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mgr-diskprediction-local (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:47.333 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:47.335 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-11-21T21:53:47.370 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-11-21T21:53:47.407 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mon (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:47.409 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:47.443 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:47.585 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up dump (0.4b46-6) ... 2021-11-21T21:53:47.608 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:47.608 INFO:teuthology.orchestra.run.gibba036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-11-21T21:53:47.683 INFO:teuthology.orchestra.run.gibba016.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-11-21T21:53:47.753 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:47.807 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-11-21T21:53:47.817 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-11-21T21:53:47.844 INFO:teuthology.orchestra.run.gibba016.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-11-21T21:53:47.879 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:47.883 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mgr-cephadm (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:47.904 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up indent (2.2.12-1) ... 2021-11-21T21:53:48.031 INFO:teuthology.orchestra.run.gibba016.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:48.035 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:48.161 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up ceph-mgr-dashboard (15.2.15-36-g911bc85f-1focal) ... 2021-11-21T21:53:48.180 INFO:teuthology.orchestra.run.gibba016.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:48.216 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libfl2:amd64. 2021-11-21T21:53:48.250 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:48.291 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:48.296 INFO:teuthology.orchestra.run.gibba036.stdout:Processing triggers for systemd (245.4-4ubuntu3.13) ... 2021-11-21T21:53:48.459 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:48.493 INFO:teuthology.orchestra.run.gibba016.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:48.505 INFO:teuthology.orchestra.run.gibba036.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:48.577 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-11-21T21:53:48.611 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:48.652 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:48.826 INFO:teuthology.orchestra.run.gibba019.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:53:48.827 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-11-21T21:53:48.863 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-11-21T21:53:48.938 INFO:teuthology.orchestra.run.gibba017.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-11-21T21:53:48.949 INFO:teuthology.orchestra.run.gibba017.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:48.969 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-11-21T21:53:48.969 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-11-21T21:53:48.987 INFO:teuthology.orchestra.run.gibba017.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:49.047 INFO:teuthology.orchestra.run.gibba019.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-11-21T21:53:49.048 INFO:teuthology.orchestra.run.gibba019.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-11-21T21:53:49.048 INFO:teuthology.orchestra.run.gibba019.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-11-21T21:53:49.048 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:53:49.048 INFO:teuthology.orchestra.run.gibba019.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:53:49.049 INFO:teuthology.orchestra.run.gibba019.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:53:49.049 INFO:teuthology.orchestra.run.gibba019.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:53:49.049 INFO:teuthology.orchestra.run.gibba019.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:53:49.049 INFO:teuthology.orchestra.run.gibba019.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:53:49.050 INFO:teuthology.orchestra.run.gibba019.stdout:The following additional packages will be installed: 2021-11-21T21:53:49.050 INFO:teuthology.orchestra.run.gibba019.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-11-21T21:53:49.051 INFO:teuthology.orchestra.run.gibba019.stdout:Suggested packages: 2021-11-21T21:53:49.051 INFO:teuthology.orchestra.run.gibba019.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-11-21T21:53:49.060 INFO:teuthology.orchestra.run.gibba019.stdout:The following NEW packages will be installed: 2021-11-21T21:53:49.061 INFO:teuthology.orchestra.run.gibba019.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-11-21T21:53:49.061 INFO:teuthology.orchestra.run.gibba019.stdout: libssl-dev 2021-11-21T21:53:49.062 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages will be upgraded: 2021-11-21T21:53:49.062 INFO:teuthology.orchestra.run.gibba019.stdout: libssl1.1 2021-11-21T21:53:49.234 INFO:teuthology.orchestra.run.gibba019.stdout:1 upgraded, 10 newly installed, 0 to remove and 51 not upgraded. 2021-11-21T21:53:49.235 INFO:teuthology.orchestra.run.gibba019.stdout:Need to get 4,233 kB of archives. 2021-11-21T21:53:49.235 INFO:teuthology.orchestra.run.gibba019.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-11-21T21:53:49.235 INFO:teuthology.orchestra.run.gibba019.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-11-21T21:53:49.428 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up flex (2.6.4-6.2) ... 2021-11-21T21:53:49.559 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:49.679 INFO:teuthology.orchestra.run.gibba016.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-11-21T21:53:49.829 INFO:teuthology.orchestra.run.gibba019.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.9 [1,321 kB] 2021-11-21T21:53:49.835 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:49.945 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:49.983 INFO:teuthology.orchestra.run.gibba036.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:50.104 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:50.122 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-11-21T21:53:50.177 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-11-21T21:53:50.190 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-11-21T21:53:50.203 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-11-21T21:53:50.206 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-11-21T21:53:50.207 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-11-21T21:53:50.208 INFO:teuthology.orchestra.run.gibba019.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-11-21T21:53:50.209 INFO:teuthology.orchestra.run.gibba019.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-11-21T21:53:50.209 INFO:teuthology.orchestra.run.gibba019.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.9 [1,583 kB] 2021-11-21T21:53:50.222 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:50.303 INFO:teuthology.orchestra.run.gibba036.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:53:50.304 DEBUG:teuthology.orchestra.run.gibba036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-11-21T21:53:50.323 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:50.341 INFO:teuthology.orchestra.run.gibba036.stdout:Reading package lists... 2021-11-21T21:53:50.406 INFO:teuthology.orchestra.run.gibba016.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:53:50.409 DEBUG:teuthology.parallel:result is None 2021-11-21T21:53:50.424 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up dump (0.4b46-6) ... 2021-11-21T21:53:50.439 INFO:teuthology.orchestra.run.gibba019.stdout:Preconfiguring packages ... 2021-11-21T21:53:50.448 INFO:teuthology.orchestra.run.gibba036.stdout:Building dependency tree... 2021-11-21T21:53:50.449 INFO:teuthology.orchestra.run.gibba036.stdout:Reading state information... 2021-11-21T21:53:50.527 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 4,233 kB in 1s (3,497 kB/s) 2021-11-21T21:53:50.528 INFO:teuthology.orchestra.run.gibba036.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-11-21T21:53:50.529 INFO:teuthology.orchestra.run.gibba036.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-11-21T21:53:50.529 INFO:teuthology.orchestra.run.gibba036.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-11-21T21:53:50.529 INFO:teuthology.orchestra.run.gibba036.stdout:The following packages were automatically installed and are no longer required: 2021-11-21T21:53:50.529 INFO:teuthology.orchestra.run.gibba036.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 libllvm11 2021-11-21T21:53:50.530 INFO:teuthology.orchestra.run.gibba036.stdout: linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic 2021-11-21T21:53:50.530 INFO:teuthology.orchestra.run.gibba036.stdout: linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic 2021-11-21T21:53:50.530 INFO:teuthology.orchestra.run.gibba036.stdout: linux-modules-extra-5.4.0-66-generic 2021-11-21T21:53:50.530 INFO:teuthology.orchestra.run.gibba036.stdout:Use 'sudo apt autoremove' to remove them. 2021-11-21T21:53:50.531 INFO:teuthology.orchestra.run.gibba036.stdout:The following additional packages will be installed: 2021-11-21T21:53:50.531 INFO:teuthology.orchestra.run.gibba036.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-11-21T21:53:50.531 INFO:teuthology.orchestra.run.gibba036.stdout:Suggested packages: 2021-11-21T21:53:50.532 INFO:teuthology.orchestra.run.gibba036.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-11-21T21:53:50.539 INFO:teuthology.orchestra.run.gibba017.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-11-21T21:53:50.541 INFO:teuthology.orchestra.run.gibba036.stdout:The following NEW packages will be installed: 2021-11-21T21:53:50.542 INFO:teuthology.orchestra.run.gibba036.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-11-21T21:53:50.543 INFO:teuthology.orchestra.run.gibba036.stdout: libssl-dev 2021-11-21T21:53:50.543 INFO:teuthology.orchestra.run.gibba036.stdout:The following packages will be upgraded: 2021-11-21T21:53:50.543 INFO:teuthology.orchestra.run.gibba036.stdout: libssl1.1 2021-11-21T21:53:50.600 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:50.640 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package flex. 2021-11-21T21:53:50.691 INFO:teuthology.orchestra.run.gibba017.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-11-21T21:53:50.714 INFO:teuthology.orchestra.run.gibba019.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 182110 files and directories currently installed.) 2021-11-21T21:53:50.717 INFO:teuthology.orchestra.run.gibba036.stdout:1 upgraded, 10 newly installed, 0 to remove and 51 not upgraded. 2021-11-21T21:53:50.717 INFO:teuthology.orchestra.run.gibba036.stdout:Need to get 4,233 kB of archives. 2021-11-21T21:53:50.717 INFO:teuthology.orchestra.run.gibba036.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-11-21T21:53:50.717 INFO:teuthology.orchestra.run.gibba036.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-11-21T21:53:50.718 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:50.751 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up indent (2.2.12-1) ... 2021-11-21T21:53:50.754 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking flex (2.6.4-6.2) ... 2021-11-21T21:53:50.911 INFO:teuthology.orchestra.run.gibba017.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:51.038 INFO:teuthology.orchestra.run.gibba017.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:51.171 INFO:teuthology.orchestra.run.gibba036.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.9 [1,321 kB] 2021-11-21T21:53:51.173 INFO:teuthology.orchestra.run.gibba017.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:51.221 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:51.377 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.9) over (1.1.1f-1ubuntu2.8) ... 2021-11-21T21:53:51.391 INFO:teuthology.orchestra.run.gibba036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-11-21T21:53:51.445 INFO:teuthology.orchestra.run.gibba036.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-11-21T21:53:51.446 INFO:teuthology.orchestra.run.gibba036.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-11-21T21:53:51.452 INFO:teuthology.orchestra.run.gibba036.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-11-21T21:53:51.456 INFO:teuthology.orchestra.run.gibba036.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-11-21T21:53:51.456 INFO:teuthology.orchestra.run.gibba036.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-11-21T21:53:51.459 INFO:teuthology.orchestra.run.gibba036.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-11-21T21:53:51.459 INFO:teuthology.orchestra.run.gibba036.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-11-21T21:53:51.460 INFO:teuthology.orchestra.run.gibba036.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.9 [1,583 kB] 2021-11-21T21:53:51.689 INFO:teuthology.orchestra.run.gibba036.stdout:Preconfiguring packages ... 2021-11-21T21:53:51.771 INFO:teuthology.orchestra.run.gibba036.stdout:Fetched 4,233 kB in 1s (4,331 kB/s) 2021-11-21T21:53:51.856 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package flex. 2021-11-21T21:53:51.881 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package bison. 2021-11-21T21:53:51.891 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-11-21T21:53:51.929 INFO:teuthology.orchestra.run.gibba036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 182110 files and directories currently installed.) 2021-11-21T21:53:51.932 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:51.933 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:51.970 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking flex (2.6.4-6.2) ... 2021-11-21T21:53:52.301 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-11-21T21:53:52.335 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-11-21T21:53:52.376 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:52.595 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:52.730 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package dump. 2021-11-21T21:53:52.739 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-11-21T21:53:52.768 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.9) over (1.1.1f-1ubuntu2.8) ... 2021-11-21T21:53:52.779 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking dump (0.4b46-6) ... 2021-11-21T21:53:52.792 INFO:teuthology.orchestra.run.gibba017.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-11-21T21:53:53.066 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package indent. 2021-11-21T21:53:53.100 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-11-21T21:53:53.172 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking indent (2.2.12-1) ... 2021-11-21T21:53:53.281 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package bison. 2021-11-21T21:53:53.292 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-11-21T21:53:53.332 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:53.510 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-11-21T21:53:53.521 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-11-21T21:53:53.568 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:53.623 INFO:teuthology.orchestra.run.gibba017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:53:53.625 DEBUG:teuthology.parallel:result is None 2021-11-21T21:53:53.735 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-11-21T21:53:53.768 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-11-21T21:53:53.809 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:53.888 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-11-21T21:53:53.922 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-11-21T21:53:53.963 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:54.162 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package dump. 2021-11-21T21:53:54.173 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-11-21T21:53:54.245 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking dump (0.4b46-6) ... 2021-11-21T21:53:54.349 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libfl2:amd64. 2021-11-21T21:53:54.361 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:54.407 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:54.533 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package indent. 2021-11-21T21:53:54.565 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-11-21T21:53:54.608 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking indent (2.2.12-1) ... 2021-11-21T21:53:54.668 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-11-21T21:53:54.702 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:54.743 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:54.885 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-11-21T21:53:54.919 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-11-21T21:53:54.960 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:55.038 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-11-21T21:53:55.072 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:55.113 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:55.263 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-11-21T21:53:55.296 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-11-21T21:53:55.338 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:55.561 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up flex (2.6.4-6.2) ... 2021-11-21T21:53:55.684 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:55.700 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libfl2:amd64. 2021-11-21T21:53:55.710 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:55.758 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:55.926 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:56.052 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:56.069 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-11-21T21:53:56.102 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-11-21T21:53:56.143 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:56.195 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:56.321 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:56.405 INFO:teuthology.orchestra.run.gibba036.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-11-21T21:53:56.437 INFO:teuthology.orchestra.run.gibba036.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.9_amd64.deb ... 2021-11-21T21:53:56.439 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:56.479 INFO:teuthology.orchestra.run.gibba036.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:56.565 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up dump (0.4b46-6) ... 2021-11-21T21:53:56.663 INFO:teuthology.orchestra.run.gibba019.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-11-21T21:53:56.724 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:56.849 INFO:teuthology.orchestra.run.gibba019.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-11-21T21:53:56.918 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up indent (2.2.12-1) ... 2021-11-21T21:53:56.937 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up flex (2.6.4-6.2) ... 2021-11-21T21:53:57.043 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:57.060 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:57.170 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:57.294 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-11-21T21:53:57.310 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:57.396 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-11-21T21:53:57.496 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.9) ... 2021-11-21T21:53:57.964 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-11-21T21:53:58.333 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:58.484 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up dump (0.4b46-6) ... 2021-11-21T21:53:58.582 INFO:teuthology.orchestra.run.gibba036.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-11-21T21:53:58.651 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-11-21T21:53:58.726 INFO:teuthology.orchestra.run.gibba036.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-11-21T21:53:58.778 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up indent (2.2.12-1) ... 2021-11-21T21:53:58.826 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-11-21T21:53:58.879 INFO:teuthology.orchestra.run.gibba036.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-11-21T21:53:58.981 INFO:teuthology.orchestra.run.gibba036.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-11-21T21:53:59.132 INFO:teuthology.orchestra.run.gibba036.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-11-21T21:53:59.543 INFO:teuthology.orchestra.run.gibba019.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:53:59.544 DEBUG:teuthology.parallel:result is None 2021-11-21T21:54:00.286 INFO:teuthology.orchestra.run.gibba036.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-11-21T21:54:01.029 INFO:teuthology.orchestra.run.gibba036.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-11-21T21:54:01.031 DEBUG:teuthology.parallel:result is None 2021-11-21T21:54:01.032 DEBUG:teuthology.orchestra.run.gibba016:> sudo lsb_release -is 2021-11-21T21:54:01.059 INFO:teuthology.orchestra.run.gibba016.stdout:Ubuntu 2021-11-21T21:54:01.062 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:54:01.062 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:54:01.373 DEBUG:teuthology.orchestra.run.gibba016:> dpkg-query -W -f '${Version}' ceph 2021-11-21T21:54:01.389 INFO:teuthology.orchestra.run.gibba016.stdout:15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:01.390 INFO:teuthology.packaging:The installed version of ceph is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:01.391 INFO:teuthology.task.install:The correct ceph version 15.2.15-36-g911bc85f-1focal is installed. 2021-11-21T21:54:01.391 DEBUG:teuthology.orchestra.run.gibba017:> sudo lsb_release -is 2021-11-21T21:54:01.418 INFO:teuthology.orchestra.run.gibba017.stdout:Ubuntu 2021-11-21T21:54:01.421 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:54:01.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:54:01.707 DEBUG:teuthology.orchestra.run.gibba017:> dpkg-query -W -f '${Version}' ceph 2021-11-21T21:54:01.752 INFO:teuthology.orchestra.run.gibba017.stdout:15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:01.753 INFO:teuthology.packaging:The installed version of ceph is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:01.753 INFO:teuthology.task.install:The correct ceph version 15.2.15-36-g911bc85f-1focal is installed. 2021-11-21T21:54:01.754 DEBUG:teuthology.orchestra.run.gibba019:> sudo lsb_release -is 2021-11-21T21:54:01.782 INFO:teuthology.orchestra.run.gibba019.stdout:Ubuntu 2021-11-21T21:54:01.784 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:54:01.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:54:02.121 DEBUG:teuthology.orchestra.run.gibba019:> dpkg-query -W -f '${Version}' ceph 2021-11-21T21:54:02.137 INFO:teuthology.orchestra.run.gibba019.stdout:15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:02.137 INFO:teuthology.packaging:The installed version of ceph is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:02.138 INFO:teuthology.task.install:The correct ceph version 15.2.15-36-g911bc85f-1focal is installed. 2021-11-21T21:54:02.138 DEBUG:teuthology.orchestra.run.gibba036:> sudo lsb_release -is 2021-11-21T21:54:02.167 INFO:teuthology.orchestra.run.gibba036.stdout:Ubuntu 2021-11-21T21:54:02.169 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-11-21T21:54:02.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=911bc85f89d5c4ff92555237a37b5b774d7b1332 2021-11-21T21:54:02.424 DEBUG:teuthology.orchestra.run.gibba036:> dpkg-query -W -f '${Version}' ceph 2021-11-21T21:54:02.471 INFO:teuthology.orchestra.run.gibba036.stdout:15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:02.472 INFO:teuthology.packaging:The installed version of ceph is 15.2.15-36-g911bc85f-1focal 2021-11-21T21:54:02.473 INFO:teuthology.task.install:The correct ceph version 15.2.15-36-g911bc85f-1focal is installed. 2021-11-21T21:54:02.473 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-11-21T21:54:02.474 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:54:02.474 DEBUG:teuthology.orchestra.run.gibba016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-11-21T21:54:02.487 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:54:02.487 DEBUG:teuthology.orchestra.run.gibba017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-11-21T21:54:02.501 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:54:02.502 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-11-21T21:54:02.514 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:54:02.515 DEBUG:teuthology.orchestra.run.gibba036:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-11-21T21:54:02.537 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-11-21T21:54:02.538 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:54:02.538 DEBUG:teuthology.orchestra.run.gibba016:> sudo dd of=/usr/bin/daemon-helper 2021-11-21T21:54:02.550 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-11-21T21:54:02.568 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:54:02.569 DEBUG:teuthology.orchestra.run.gibba017:> sudo dd of=/usr/bin/daemon-helper 2021-11-21T21:54:02.588 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-11-21T21:54:02.654 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:54:02.655 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/usr/bin/daemon-helper 2021-11-21T21:54:02.668 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-11-21T21:54:02.737 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:54:02.738 DEBUG:teuthology.orchestra.run.gibba036:> sudo dd of=/usr/bin/daemon-helper 2021-11-21T21:54:02.753 DEBUG:teuthology.orchestra.run.gibba036:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-11-21T21:54:02.807 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-11-21T21:54:02.808 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:54:02.808 DEBUG:teuthology.orchestra.run.gibba016:> sudo dd of=/usr/bin/adjust-ulimits 2021-11-21T21:54:02.834 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-11-21T21:54:02.843 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:54:02.847 DEBUG:teuthology.orchestra.run.gibba017:> sudo dd of=/usr/bin/adjust-ulimits 2021-11-21T21:54:02.872 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-11-21T21:54:02.936 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:54:02.936 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/usr/bin/adjust-ulimits 2021-11-21T21:54:02.963 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-11-21T21:54:02.984 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:54:02.985 DEBUG:teuthology.orchestra.run.gibba036:> sudo dd of=/usr/bin/adjust-ulimits 2021-11-21T21:54:03.007 DEBUG:teuthology.orchestra.run.gibba036:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-11-21T21:54:03.029 INFO:teuthology.run_tasks:Running task ceph... 2021-11-21T21:54:03.100 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-11-21T21:54:03.101 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 777 /var/log/ceph 2021-11-21T21:54:03.106 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 777 /var/log/ceph 2021-11-21T21:54:03.110 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 777 /var/log/ceph 2021-11-21T21:54:03.114 DEBUG:teuthology.orchestra.run.gibba036:> sudo chmod 777 /var/log/ceph 2021-11-21T21:54:03.136 INFO:tasks.ceph:Disabling ceph logrotate... 2021-11-21T21:54:03.137 DEBUG:teuthology.orchestra.run.gibba016:> sudo rm -f -- /etc/logrotate.d/ceph 2021-11-21T21:54:03.140 DEBUG:teuthology.orchestra.run.gibba017:> sudo rm -f -- /etc/logrotate.d/ceph 2021-11-21T21:54:03.174 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /etc/logrotate.d/ceph 2021-11-21T21:54:03.179 DEBUG:teuthology.orchestra.run.gibba036:> sudo rm -f -- /etc/logrotate.d/ceph 2021-11-21T21:54:03.200 INFO:tasks.ceph:Creating extra log directories... 2021-11-21T21:54:03.201 DEBUG:teuthology.orchestra.run.gibba016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-11-21T21:54:03.204 DEBUG:teuthology.orchestra.run.gibba017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-11-21T21:54:03.207 DEBUG:teuthology.orchestra.run.gibba019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-11-21T21:54:03.241 DEBUG:teuthology.orchestra.run.gibba036:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-11-21T21:54:03.263 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2021-11-21T21:54:03.264 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2021-11-21T21:54:03.264 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2021-11-21T21:54:03.265 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:54:03.265 DEBUG:teuthology.orchestra.run.gibba016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-11-21T21:54:03.274 DEBUG:teuthology.orchestra.run.gibba016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:03.294 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:54:03.295 DEBUG:teuthology.orchestra.run.gibba017:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-11-21T21:54:03.304 DEBUG:teuthology.orchestra.run.gibba017:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:03.324 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:54:03.324 DEBUG:teuthology.orchestra.run.gibba019:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-11-21T21:54:03.333 DEBUG:teuthology.orchestra.run.gibba019:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:03.345 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:54:03.345 DEBUG:teuthology.orchestra.run.gibba036:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-11-21T21:54:03.355 DEBUG:teuthology.orchestra.run.gibba036:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:03.374 INFO:tasks.ceph:Creating ceph cluster ceph... 2021-11-21T21:54:03.375 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'filestore', 'osd op complaint time': 180, 'osd sloppy crc': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-11-21T21:54:03.375 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819', 'branch': 'octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '6512819', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'gibba', 'name': 'teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'filestore', 'osd op complaint time': 180, 'osd sloppy crc': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332'}}, 'workunit': {'branch': 'octopus', 'sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1']], 'sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332', 'targets': {'gibba016.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbXkiIf+eb7zGtRPK3lWcFKzUuKOrgtMaTJTd36n1GegE0Yr1xXyD47mFwAh994XjmyywD0J7U6QI4B6makRyLIF4Pr60tZ7K6mnPzUDYvTcJHYJB2d6yOBgc7FqYzfYnWDWvFm1hrJ2QYPtqFoormplI9Xa9mNs12w3bdQH9E8NoSeLHk06/UeQwwWlkEY5qYgQeqqb0Dafaang2AMDep8v5MpQV0rxnaLKrv7krm2JexHuJIndmhjeRLFM9inpjZ1oSKLru4cC0a1EPrFPKfcQx07tIMI+7QvBoE+GSMKU3Tj0S3TsV6/nTlaSryFtKhikFLOWHF6IS8Lw7nwUhDYSEZ1X2oac3AiqPdTuaW+bFkXMQxVicb2/EwpjGZMx30B9ereV+Z8iUQl4Kz/Tmv2NsJKNan9A3sSF+b4LDxvEKbSliaKjkmVpEDl4/TMhpdPRly7llyvDtwgrNGgOC1O6xnob9QLe8KZTx2muLuAYCLPct/6gaGV4sdIl/jI08=', 'gibba017.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgM4jU37Z4i0t4d5zVA+fpbIZqWv4wZaq6viFXwkYqq8kBhca49H2EbhNKK0AYgvci0cdE9Zi5AmEYIjt7zDocjDrlAaAOnHJ9fGEv1eQUmLpJjj0cr43A2ziIdTmAy5mXk+/yZ5Ewj3lchN/NcszAxd0ahsIX5Pf6ZmjZO+W5mPvuHUzp8+kgx7FjfTB87Pfs4xRNK5oVDdaCKOFWJv9KqSUUjzzQlzMXpeNxru6xt4yLXvh7MzJw5IYF7PJDn8StkXu+mZSuD9I6vQ5Jfkw15UDfhXymT8I3dDgaSb7Kkf1JMGG34aMIiA5ooKwGEK4QYM4DOKXaY1OIWpEhMr3V6d3zKIey8Lqw8rAQn6qRAReyUNnOhBUARb0h5qKLMlBoTfq8GGLQ60gtlVZIORR2Arc2HJKlRblMpqzBkTsdfaremYKNzn+OXAzdl0Zo4j2pbis1zHREIhj0x/Tv91iQE/9xo0XMV3+EseNvTgIT+qOVCEpIOPYSA3llev1YAKs=', 'gibba019.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIe+TTxY3Zgr8xdckrjIs412kMfHS2r40ciQq47T2ubn2KrPOng709Xa5yi2A6JfSKXxFq0vHq/nGE7NDoOGSxq9H6co/fgSMhNvzzwimt/YFcTf9BLnXlPOrsbAxl3P282zN48iSk9Mp7kE7gwXbAJRnsGydqWxm1ViOeUeSocmeohGziYAWytq+nTIH1Os+c1WFBmgPm97LzM2Vn/IkU5DXvPmhIfqN4wMq8zM1i7Vz6LVNW+ZhjmnzOJn9KkqAL4pyCaWoaUtr0nWSFlVQ39kROW9IlMgFTnCnvxO9QracTMV6LTjLqtVR49/4dedABRMzzbx3tJ4ERs/jjqOYFQJ/LhfxwdCW5/NgmMMG2wTHamzIeqB0qW/6v0qeSpSVBQi/EcmXKyhiO9UHtMT594c/6OkplG6sPgenEx9+nNhVB67kfUtIjz1oRUqXsJU1GxNuvdsMicAZjIITL73gMkq8csydPC6zf2jW2caFF2iqotjaSp6uwYZeVmXZXjb0=', 'gibba036.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKcULwJOz5ZQZp/E5QF+Cdla5PaLDPEvKPeItzDNuAxJYhjtDOr33hj34nuOJj1QTv4QoUmjwoqeJf+vB/PaFKOQb1JfjkusBLbVlGQuvXkphmlaRgpIYNXjWhBbvU29aL4cLlSWFOj6gOBfIKDn8UPlX1l6xl9DixM9rcZX6HI5rm73b358jNBx3qAxo+QH/j1rCvfQhyvUf6KU+47wQyWBAfm+YJcwgs5xxfqt8evozoU0/KoBiNy4885gnoDAdSrfF/z1qXSJaagKWvwdUnjkhm6VPNpxlJBQdI3sQF4ga+Tmgeqt618d5lFtLE35/HpofjvAh3N9s9AhdH8cxvCXwku0gS50NVfm5GVB8Obanh8XqFwXzWnKwZgO/aBC8+Z6zMWpXNoqj2Jl2/UYddtBuaSEjL2ClwYkPehPzbpvxySpwHqGgZ9smStMC4DkHD/79S8LYCN2oOqIUGqdLd5JhZwneSmCQq4s7ZDYClcx3jksrQAmP3Fr4C5LURTEU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '911bc85f89d5c4ff92555237a37b5b774d7b1332'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'master', 'teuthology_sha1': '6cfddd5e1c7d5ede1367cf3087c21af44eb88356', 'timestamp': '2021-11-19_04:10:03', 'tube': 'gibba', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.gibba.19943'} 2021-11-21T21:54:03.376 DEBUG:teuthology.orchestra.run.gibba016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-11-21T21:54:03.380 DEBUG:teuthology.orchestra.run.gibba017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-11-21T21:54:03.383 DEBUG:teuthology.orchestra.run.gibba019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-11-21T21:54:03.389 DEBUG:teuthology.orchestra.run.gibba036:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-11-21T21:54:03.420 DEBUG:teuthology.orchestra.run.gibba016:> sudo install -d -m0777 -- /var/run/ceph 2021-11-21T21:54:03.424 DEBUG:teuthology.orchestra.run.gibba017:> sudo install -d -m0777 -- /var/run/ceph 2021-11-21T21:54:03.437 DEBUG:teuthology.orchestra.run.gibba019:> sudo install -d -m0777 -- /var/run/ceph 2021-11-21T21:54:03.441 DEBUG:teuthology.orchestra.run.gibba036:> sudo install -d -m0777 -- /var/run/ceph 2021-11-21T21:54:03.460 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:54:03.461 DEBUG:teuthology.orchestra.run.gibba016:> dd if=/scratch_devs of=/dev/stdout 2021-11-21T21:54:03.496 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-11-21T21:54:03.496 DEBUG:teuthology.orchestra.run.gibba016:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-11-21T21:54:03.506 INFO:teuthology.orchestra.run.gibba016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-11-21T21:54:03.507 INFO:teuthology.orchestra.run.gibba016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.507 INFO:teuthology.orchestra.run.gibba016.stdout:Device: 6h/6d Inode: 369 Links: 1 2021-11-21T21:54:03.508 INFO:teuthology.orchestra.run.gibba016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.508 INFO:teuthology.orchestra.run.gibba016.stdout:Access: 2021-11-21 21:51:16.287184717 +0000 2021-11-21T21:54:03.509 INFO:teuthology.orchestra.run.gibba016.stdout:Modify: 2021-11-21 21:51:16.139184290 +0000 2021-11-21T21:54:03.509 INFO:teuthology.orchestra.run.gibba016.stdout:Change: 2021-11-21 21:51:16.139184290 +0000 2021-11-21T21:54:03.509 INFO:teuthology.orchestra.run.gibba016.stdout: Birth: - 2021-11-21T21:54:03.519 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records in 2021-11-21T21:54:03.520 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records out 2021-11-21T21:54:03.520 INFO:teuthology.orchestra.run.gibba016.stderr:512 bytes copied, 0.000444797 s, 1.2 MB/s 2021-11-21T21:54:03.526 DEBUG:teuthology.orchestra.run.gibba016:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-11-21T21:54:03.535 INFO:teuthology.orchestra.run.gibba016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-11-21T21:54:03.535 INFO:teuthology.orchestra.run.gibba016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.536 INFO:teuthology.orchestra.run.gibba016.stdout:Device: 6h/6d Inode: 373 Links: 1 2021-11-21T21:54:03.536 INFO:teuthology.orchestra.run.gibba016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.536 INFO:teuthology.orchestra.run.gibba016.stdout:Access: 2021-11-21 21:51:16.627185697 +0000 2021-11-21T21:54:03.536 INFO:teuthology.orchestra.run.gibba016.stdout:Modify: 2021-11-21 21:51:16.487185294 +0000 2021-11-21T21:54:03.536 INFO:teuthology.orchestra.run.gibba016.stdout:Change: 2021-11-21 21:51:16.487185294 +0000 2021-11-21T21:54:03.537 INFO:teuthology.orchestra.run.gibba016.stdout: Birth: - 2021-11-21T21:54:03.548 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records in 2021-11-21T21:54:03.548 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records out 2021-11-21T21:54:03.549 INFO:teuthology.orchestra.run.gibba016.stderr:512 bytes copied, 0.000421121 s, 1.2 MB/s 2021-11-21T21:54:03.554 DEBUG:teuthology.orchestra.run.gibba016:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-11-21T21:54:03.564 INFO:teuthology.orchestra.run.gibba016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-11-21T21:54:03.564 INFO:teuthology.orchestra.run.gibba016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.565 INFO:teuthology.orchestra.run.gibba016.stdout:Device: 6h/6d Inode: 381 Links: 1 2021-11-21T21:54:03.565 INFO:teuthology.orchestra.run.gibba016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.565 INFO:teuthology.orchestra.run.gibba016.stdout:Access: 2021-11-21 21:51:16.947186619 +0000 2021-11-21T21:54:03.565 INFO:teuthology.orchestra.run.gibba016.stdout:Modify: 2021-11-21 21:51:16.851186343 +0000 2021-11-21T21:54:03.565 INFO:teuthology.orchestra.run.gibba016.stdout:Change: 2021-11-21 21:51:16.851186343 +0000 2021-11-21T21:54:03.566 INFO:teuthology.orchestra.run.gibba016.stdout: Birth: - 2021-11-21T21:54:03.577 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records in 2021-11-21T21:54:03.578 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records out 2021-11-21T21:54:03.578 INFO:teuthology.orchestra.run.gibba016.stderr:512 bytes copied, 0.000485787 s, 1.1 MB/s 2021-11-21T21:54:03.584 DEBUG:teuthology.orchestra.run.gibba016:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-11-21T21:54:03.636 INFO:teuthology.orchestra.run.gibba016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-11-21T21:54:03.636 INFO:teuthology.orchestra.run.gibba016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.637 INFO:teuthology.orchestra.run.gibba016.stdout:Device: 6h/6d Inode: 388 Links: 1 2021-11-21T21:54:03.637 INFO:teuthology.orchestra.run.gibba016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.637 INFO:teuthology.orchestra.run.gibba016.stdout:Access: 2021-11-21 21:51:17.315187681 +0000 2021-11-21T21:54:03.637 INFO:teuthology.orchestra.run.gibba016.stdout:Modify: 2021-11-21 21:51:17.183187300 +0000 2021-11-21T21:54:03.638 INFO:teuthology.orchestra.run.gibba016.stdout:Change: 2021-11-21 21:51:17.183187300 +0000 2021-11-21T21:54:03.638 INFO:teuthology.orchestra.run.gibba016.stdout: Birth: - 2021-11-21T21:54:03.643 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records in 2021-11-21T21:54:03.643 INFO:teuthology.orchestra.run.gibba016.stderr:1+0 records out 2021-11-21T21:54:03.644 INFO:teuthology.orchestra.run.gibba016.stderr:512 bytes copied, 0.000148115 s, 3.5 MB/s 2021-11-21T21:54:03.645 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:03.645 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:54:03.645 DEBUG:teuthology.orchestra.run.gibba017:> dd if=/scratch_devs of=/dev/stdout 2021-11-21T21:54:03.654 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-11-21T21:54:03.655 DEBUG:teuthology.orchestra.run.gibba017:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-11-21T21:54:03.702 INFO:teuthology.orchestra.run.gibba017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-11-21T21:54:03.703 INFO:teuthology.orchestra.run.gibba017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.703 INFO:teuthology.orchestra.run.gibba017.stdout:Device: 6h/6d Inode: 347 Links: 1 2021-11-21T21:54:03.703 INFO:teuthology.orchestra.run.gibba017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.703 INFO:teuthology.orchestra.run.gibba017.stdout:Access: 2021-11-21 21:51:17.174877918 +0000 2021-11-21T21:54:03.704 INFO:teuthology.orchestra.run.gibba017.stdout:Modify: 2021-11-21 21:51:17.078877641 +0000 2021-11-21T21:54:03.704 INFO:teuthology.orchestra.run.gibba017.stdout:Change: 2021-11-21 21:51:17.078877641 +0000 2021-11-21T21:54:03.704 INFO:teuthology.orchestra.run.gibba017.stdout: Birth: - 2021-11-21T21:54:03.706 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records in 2021-11-21T21:54:03.706 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records out 2021-11-21T21:54:03.707 INFO:teuthology.orchestra.run.gibba017.stderr:512 bytes copied, 0.000218299 s, 2.3 MB/s 2021-11-21T21:54:03.708 DEBUG:teuthology.orchestra.run.gibba017:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-11-21T21:54:03.713 INFO:teuthology.orchestra.run.gibba017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-11-21T21:54:03.713 INFO:teuthology.orchestra.run.gibba017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.713 INFO:teuthology.orchestra.run.gibba017.stdout:Device: 6h/6d Inode: 354 Links: 1 2021-11-21T21:54:03.714 INFO:teuthology.orchestra.run.gibba017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.714 INFO:teuthology.orchestra.run.gibba017.stdout:Access: 2021-11-21 21:51:17.562879036 +0000 2021-11-21T21:54:03.714 INFO:teuthology.orchestra.run.gibba017.stdout:Modify: 2021-11-21 21:51:17.426878644 +0000 2021-11-21T21:54:03.714 INFO:teuthology.orchestra.run.gibba017.stdout:Change: 2021-11-21 21:51:17.426878644 +0000 2021-11-21T21:54:03.714 INFO:teuthology.orchestra.run.gibba017.stdout: Birth: - 2021-11-21T21:54:03.717 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records in 2021-11-21T21:54:03.718 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records out 2021-11-21T21:54:03.718 INFO:teuthology.orchestra.run.gibba017.stderr:512 bytes copied, 0.000261145 s, 2.0 MB/s 2021-11-21T21:54:03.720 DEBUG:teuthology.orchestra.run.gibba017:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-11-21T21:54:03.727 INFO:teuthology.orchestra.run.gibba017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-11-21T21:54:03.727 INFO:teuthology.orchestra.run.gibba017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.728 INFO:teuthology.orchestra.run.gibba017.stdout:Device: 6h/6d Inode: 358 Links: 1 2021-11-21T21:54:03.728 INFO:teuthology.orchestra.run.gibba017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.729 INFO:teuthology.orchestra.run.gibba017.stdout:Access: 2021-11-21 21:51:17.898880005 +0000 2021-11-21T21:54:03.729 INFO:teuthology.orchestra.run.gibba017.stdout:Modify: 2021-11-21 21:51:17.770879636 +0000 2021-11-21T21:54:03.730 INFO:teuthology.orchestra.run.gibba017.stdout:Change: 2021-11-21 21:51:17.770879636 +0000 2021-11-21T21:54:03.731 INFO:teuthology.orchestra.run.gibba017.stdout: Birth: - 2021-11-21T21:54:03.735 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records in 2021-11-21T21:54:03.736 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records out 2021-11-21T21:54:03.736 INFO:teuthology.orchestra.run.gibba017.stderr:512 bytes copied, 0.000330273 s, 1.6 MB/s 2021-11-21T21:54:03.738 DEBUG:teuthology.orchestra.run.gibba017:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-11-21T21:54:03.749 INFO:teuthology.orchestra.run.gibba017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-11-21T21:54:03.750 INFO:teuthology.orchestra.run.gibba017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-21T21:54:03.750 INFO:teuthology.orchestra.run.gibba017.stdout:Device: 6h/6d Inode: 366 Links: 1 2021-11-21T21:54:03.750 INFO:teuthology.orchestra.run.gibba017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-21T21:54:03.751 INFO:teuthology.orchestra.run.gibba017.stdout:Access: 2021-11-21 21:51:18.326881239 +0000 2021-11-21T21:54:03.752 INFO:teuthology.orchestra.run.gibba017.stdout:Modify: 2021-11-21 21:51:18.122880651 +0000 2021-11-21T21:54:03.752 INFO:teuthology.orchestra.run.gibba017.stdout:Change: 2021-11-21 21:51:18.122880651 +0000 2021-11-21T21:54:03.753 INFO:teuthology.orchestra.run.gibba017.stdout: Birth: - 2021-11-21T21:54:03.757 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records in 2021-11-21T21:54:03.757 INFO:teuthology.orchestra.run.gibba017.stderr:1+0 records out 2021-11-21T21:54:03.757 INFO:teuthology.orchestra.run.gibba017.stderr:512 bytes copied, 0.000226121 s, 2.3 MB/s 2021-11-21T21:54:03.758 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:03.759 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@gibba016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba017.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-11-21T21:54:03.759 INFO:tasks.ceph:Generating config... 2021-11-21T21:54:03.761 INFO:tasks.ceph:[client] client mount timeout = 600 2021-11-21T21:54:03.761 INFO:tasks.ceph:[client] debug client = 20 2021-11-21T21:54:03.761 INFO:tasks.ceph:[client] debug ms = 1 2021-11-21T21:54:03.762 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2021-11-21T21:54:03.762 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2021-11-21T21:54:03.762 INFO:tasks.ceph:[mds] debug mds = 20 2021-11-21T21:54:03.762 INFO:tasks.ceph:[mds] debug ms = 1 2021-11-21T21:54:03.763 INFO:tasks.ceph:[mds] mds bal frag = True 2021-11-21T21:54:03.763 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2021-11-21T21:54:03.763 INFO:tasks.ceph:[mds] mds bal merge size = 5 2021-11-21T21:54:03.764 INFO:tasks.ceph:[mds] mds bal split bits = 3 2021-11-21T21:54:03.764 INFO:tasks.ceph:[mds] mds bal split size = 100 2021-11-21T21:54:03.764 INFO:tasks.ceph:[mds] mds debug frag = True 2021-11-21T21:54:03.764 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2021-11-21T21:54:03.765 INFO:tasks.ceph:[mds] mds op complaint time = 180 2021-11-21T21:54:03.765 INFO:tasks.ceph:[mds] mds verify scatter = True 2021-11-21T21:54:03.765 INFO:tasks.ceph:[mds] osd op complaint time = 180 2021-11-21T21:54:03.766 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2021-11-21T21:54:03.766 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2021-11-21T21:54:03.766 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-11-21T21:54:03.767 INFO:tasks.ceph:[mgr] debug ms = 1 2021-11-21T21:54:03.767 INFO:tasks.ceph:[mon] debug mon = 20 2021-11-21T21:54:03.767 INFO:tasks.ceph:[mon] debug ms = 1 2021-11-21T21:54:03.768 INFO:tasks.ceph:[mon] debug paxos = 20 2021-11-21T21:54:03.768 INFO:tasks.ceph:[mon] mon op complaint time = 120 2021-11-21T21:54:03.768 INFO:tasks.ceph:[osd] debug ms = 1 2021-11-21T21:54:03.768 INFO:tasks.ceph:[osd] debug osd = 20 2021-11-21T21:54:03.769 INFO:tasks.ceph:[osd] osd objectstore = filestore 2021-11-21T21:54:03.769 INFO:tasks.ceph:[osd] osd op complaint time = 180 2021-11-21T21:54:03.769 INFO:tasks.ceph:[osd] osd sloppy crc = True 2021-11-21T21:54:03.770 INFO:tasks.ceph:Setting up mon.a... 2021-11-21T21:54:03.770 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2021-11-21T21:54:03.800 INFO:teuthology.orchestra.run.gibba016.stdout:creating /etc/ceph/ceph.keyring 2021-11-21T21:54:03.802 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2021-11-21T21:54:03.819 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-11-21T21:54:03.872 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.2.116'), ('mon.b', '172.21.2.117'), ('mon.c', '[v2:172.21.2.117:3301,v1:172.21.2.117:6790]')] 2021-11-21T21:54:03.873 DEBUG:teuthology.orchestra.run.gibba016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.2.116 --add b 172.21.2.117 --addv c '[v2:172.21.2.117:3301,v1:172.21.2.117:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:03.914 INFO:teuthology.orchestra.run.gibba016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:03.914 INFO:teuthology.orchestra.run.gibba016.stdout:monmaptool: generated fsid fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T21:54:03.914 INFO:teuthology.orchestra.run.gibba016.stdout:epoch 0 2021-11-21T21:54:03.915 INFO:teuthology.orchestra.run.gibba016.stdout:fsid fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T21:54:03.915 INFO:teuthology.orchestra.run.gibba016.stdout:last_changed 2021-11-21T21:54:03.915639+0000 2021-11-21T21:54:03.916 INFO:teuthology.orchestra.run.gibba016.stdout:created 2021-11-21T21:54:03.915639+0000 2021-11-21T21:54:03.916 INFO:teuthology.orchestra.run.gibba016.stdout:min_mon_release 0 (unknown) 2021-11-21T21:54:03.916 INFO:teuthology.orchestra.run.gibba016.stdout:0: [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] mon.a 2021-11-21T21:54:03.916 INFO:teuthology.orchestra.run.gibba016.stdout:1: [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] mon.b 2021-11-21T21:54:03.917 INFO:teuthology.orchestra.run.gibba016.stdout:2: [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] mon.c 2021-11-21T21:54:03.917 INFO:teuthology.orchestra.run.gibba016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2021-11-21T21:54:03.918 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fe3c4e1d-e687-4491-9a93-95ac7299e2b3... 2021-11-21T21:54:03.919 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T21:54:03.923 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T21:54:03.926 DEBUG:teuthology.orchestra.run.gibba019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T21:54:03.929 DEBUG:teuthology.orchestra.run.gibba036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T21:54:03.934 INFO:teuthology.orchestra.run.gibba016.stdout:[global] 2021-11-21T21:54:03.934 INFO:teuthology.orchestra.run.gibba016.stdout: chdir = "" 2021-11-21T21:54:03.935 INFO:teuthology.orchestra.run.gibba016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T21:54:03.935 INFO:teuthology.orchestra.run.gibba016.stdout: auth supported = cephx 2021-11-21T21:54:03.935 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.935 INFO:teuthology.orchestra.run.gibba016.stdout: filestore xattr use omap = true 2021-11-21T21:54:03.936 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.936 INFO:teuthology.orchestra.run.gibba016.stdout: mon clock drift allowed = 1.000 2021-11-21T21:54:03.936 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.936 INFO:teuthology.orchestra.run.gibba016.stdout: osd crush chooseleaf type = 0 2021-11-21T21:54:03.936 INFO:teuthology.orchestra.run.gibba016.stdout: auth debug = true 2021-11-21T21:54:03.937 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.937 INFO:teuthology.orchestra.run.gibba016.stdout: ms die on old message = true 2021-11-21T21:54:03.937 INFO:teuthology.orchestra.run.gibba016.stdout: ms die on bug = true 2021-11-21T21:54:03.937 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.938 INFO:teuthology.orchestra.run.gibba016.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T21:54:03.938 INFO:teuthology.orchestra.run.gibba016.stdout: mon pg warn max object skew = 0 2021-11-21T21:54:03.938 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.938 INFO:teuthology.orchestra.run.gibba016.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T21:54:03.938 INFO:teuthology.orchestra.run.gibba016.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T21:54:03.939 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.939 INFO:teuthology.orchestra.run.gibba016.stdout: osd pool default size = 2 2021-11-21T21:54:03.939 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.939 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd allow primary affinity = true 2021-11-21T21:54:03.940 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd allow pg remap = true 2021-11-21T21:54:03.940 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on legacy crush tunables = false 2021-11-21T21:54:03.940 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on crush straw calc version zero = false 2021-11-21T21:54:03.940 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on no sortbitwise = false 2021-11-21T21:54:03.940 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on osd down out interval zero = false 2021-11-21T21:54:03.941 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on too few osds = false 2021-11-21T21:54:03.941 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T21:54:03.941 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T21:54:03.942 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.942 INFO:teuthology.orchestra.run.gibba016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T21:54:03.942 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.942 INFO:teuthology.orchestra.run.gibba016.stdout: osd default data pool replay window = 5 2021-11-21T21:54:03.943 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.943 INFO:teuthology.orchestra.run.gibba016.stdout: mon allow pool delete = true 2021-11-21T21:54:03.943 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.943 INFO:teuthology.orchestra.run.gibba016.stdout: mon cluster log file level = debug 2021-11-21T21:54:03.943 INFO:teuthology.orchestra.run.gibba016.stdout: debug asserts on shutdown = true 2021-11-21T21:54:03.944 INFO:teuthology.orchestra.run.gibba016.stdout: mon health detail to clog = false 2021-11-21T21:54:03.944 INFO:teuthology.orchestra.run.gibba016.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T21:54:03.944 INFO:teuthology.orchestra.run.gibba016.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T21:54:03.944 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.945 INFO:teuthology.orchestra.run.gibba016.stdout:[osd] 2021-11-21T21:54:03.945 INFO:teuthology.orchestra.run.gibba016.stdout: osd journal size = 100 2021-11-21T21:54:03.945 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.945 INFO:teuthology.orchestra.run.gibba016.stdout: osd scrub load threshold = 5.0 2021-11-21T21:54:03.946 INFO:teuthology.orchestra.run.gibba016.stdout: osd scrub max interval = 600 2021-11-21T21:54:03.946 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.946 INFO:teuthology.orchestra.run.gibba016.stdout: osd recover clone overlap = true 2021-11-21T21:54:03.946 INFO:teuthology.orchestra.run.gibba016.stdout: osd recovery max chunk = 1048576 2021-11-21T21:54:03.946 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.947 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug shutdown = true 2021-11-21T21:54:03.947 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug op order = true 2021-11-21T21:54:03.947 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug verify stray on activate = true 2021-11-21T21:54:03.947 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.948 INFO:teuthology.orchestra.run.gibba016.stdout: osd open classes on start = true 2021-11-21T21:54:03.948 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug pg log writeout = true 2021-11-21T21:54:03.948 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.948 INFO:teuthology.orchestra.run.gibba016.stdout: osd deep scrub update digest min age = 30 2021-11-21T21:54:03.949 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.949 INFO:teuthology.orchestra.run.gibba016.stdout: osd map max advance = 10 2021-11-21T21:54:03.949 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.949 INFO:teuthology.orchestra.run.gibba016.stdout: journal zero on create = true 2021-11-21T21:54:03.949 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.950 INFO:teuthology.orchestra.run.gibba016.stdout: filestore ondisk finisher threads = 3 2021-11-21T21:54:03.950 INFO:teuthology.orchestra.run.gibba016.stdout: filestore apply finisher threads = 3 2021-11-21T21:54:03.950 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.950 INFO:teuthology.orchestra.run.gibba016.stdout: bdev debug aio = true 2021-11-21T21:54:03.950 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug misdirected ops = true 2021-11-21T21:54:03.951 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T21:54:03.951 INFO:teuthology.orchestra.run.gibba016.stdout: debug osd = 20 2021-11-21T21:54:03.951 INFO:teuthology.orchestra.run.gibba016.stdout: osd objectstore = filestore 2021-11-21T21:54:03.951 INFO:teuthology.orchestra.run.gibba016.stdout: osd op complaint time = 180 2021-11-21T21:54:03.952 INFO:teuthology.orchestra.run.gibba016.stdout: osd sloppy crc = True 2021-11-21T21:54:03.952 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.952 INFO:teuthology.orchestra.run.gibba016.stdout:[mgr] 2021-11-21T21:54:03.953 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T21:54:03.953 INFO:teuthology.orchestra.run.gibba016.stdout: debug mgr = 20 2021-11-21T21:54:03.953 INFO:teuthology.orchestra.run.gibba016.stdout: debug mon = 20 2021-11-21T21:54:03.953 INFO:teuthology.orchestra.run.gibba016.stdout: debug auth = 20 2021-11-21T21:54:03.953 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:03.954 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:03.954 INFO:teuthology.orchestra.run.gibba016.stdout: mgr/telemetry/nag = false 2021-11-21T21:54:03.954 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.954 INFO:teuthology.orchestra.run.gibba016.stdout:[mon] 2021-11-21T21:54:03.955 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T21:54:03.955 INFO:teuthology.orchestra.run.gibba016.stdout: debug mon = 20 2021-11-21T21:54:03.955 INFO:teuthology.orchestra.run.gibba016.stdout: debug paxos = 20 2021-11-21T21:54:03.955 INFO:teuthology.orchestra.run.gibba016.stdout: debug auth = 20 2021-11-21T21:54:03.955 INFO:teuthology.orchestra.run.gibba016.stdout: mon data avail warn = 5 2021-11-21T21:54:03.956 INFO:teuthology.orchestra.run.gibba016.stdout: mon mgr mkfs grace = 240 2021-11-21T21:54:03.956 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:03.956 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd reporter subtree level = osd 2021-11-21T21:54:03.956 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd prime pg temp = true 2021-11-21T21:54:03.957 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:03.957 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.957 INFO:teuthology.orchestra.run.gibba016.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T21:54:03.957 INFO:teuthology.orchestra.run.gibba016.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T21:54:03.957 INFO:teuthology.orchestra.run.gibba016.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T21:54:03.958 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.958 INFO:teuthology.orchestra.run.gibba016.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T21:54:03.958 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T21:54:03.958 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T21:54:03.959 INFO:teuthology.orchestra.run.gibba016.stdout: mon op complaint time = 120 2021-11-21T21:54:03.959 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:03.959 INFO:teuthology.orchestra.run.gibba016.stdout:[client] 2021-11-21T21:54:03.959 INFO:teuthology.orchestra.run.gibba016.stdout: rgw cache enabled = true 2021-11-21T21:54:03.959 INFO:teuthology.orchestra.run.gibba016.stdout: rgw enable ops log = true 2021-11-21T21:54:03.960 INFO:teuthology.orchestra.run.gibba016.stdout: rgw enable usage log = true 2021-11-21T21:54:03.960 INFO:teuthology.orchestra.run.gibba016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T21:54:03.960 INFO:teuthology.orchestra.run.gibba016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T21:54:03.960 INFO:teuthology.orchestra.run.gibba016.stdout: client mount timeout = 600 2021-11-21T21:54:03.961 INFO:teuthology.orchestra.run.gibba016.stdout: debug client = 20 2021-11-21T21:54:03.961 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T21:54:03.961 INFO:teuthology.orchestra.run.gibba016.stdout: rados mon op timeout = 15m 2021-11-21T21:54:03.961 INFO:teuthology.orchestra.run.gibba016.stdout: rados osd op timeout = 15m 2021-11-21T21:54:03.961 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.a] 2021-11-21T21:54:03.962 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.b] 2021-11-21T21:54:03.962 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.c] 2021-11-21T21:54:03.962 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.a] 2021-11-21T21:54:03.963 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.c] 2021-11-21T21:54:03.963 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.b] 2021-11-21T21:54:03.963 INFO:teuthology.orchestra.run.gibba016.stdout:[mds] 2021-11-21T21:54:03.963 INFO:teuthology.orchestra.run.gibba016.stdout: debug mds = 20 2021-11-21T21:54:03.964 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T21:54:03.964 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal frag = True 2021-11-21T21:54:03.964 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal fragment size max = 10000 2021-11-21T21:54:03.964 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal merge size = 5 2021-11-21T21:54:03.964 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal split bits = 3 2021-11-21T21:54:03.965 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal split size = 100 2021-11-21T21:54:03.965 INFO:teuthology.orchestra.run.gibba016.stdout: mds debug frag = True 2021-11-21T21:54:03.965 INFO:teuthology.orchestra.run.gibba016.stdout: mds debug scatterstat = True 2021-11-21T21:54:03.965 INFO:teuthology.orchestra.run.gibba016.stdout: mds op complaint time = 180 2021-11-21T21:54:03.966 INFO:teuthology.orchestra.run.gibba016.stdout: mds verify scatter = True 2021-11-21T21:54:03.966 INFO:teuthology.orchestra.run.gibba016.stdout: osd op complaint time = 180 2021-11-21T21:54:03.966 INFO:teuthology.orchestra.run.gibba016.stdout: rados mon op timeout = 15m 2021-11-21T21:54:03.966 INFO:teuthology.orchestra.run.gibba016.stdout: rados osd op timeout = 15m 2021-11-21T21:54:03.969 INFO:teuthology.orchestra.run.gibba036.stdout:[global] 2021-11-21T21:54:03.969 INFO:teuthology.orchestra.run.gibba036.stdout: chdir = "" 2021-11-21T21:54:03.970 INFO:teuthology.orchestra.run.gibba036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T21:54:03.970 INFO:teuthology.orchestra.run.gibba036.stdout: auth supported = cephx 2021-11-21T21:54:03.970 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.970 INFO:teuthology.orchestra.run.gibba036.stdout: filestore xattr use omap = true 2021-11-21T21:54:03.971 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.971 INFO:teuthology.orchestra.run.gibba036.stdout: mon clock drift allowed = 1.000 2021-11-21T21:54:03.971 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.971 INFO:teuthology.orchestra.run.gibba036.stdout: osd crush chooseleaf type = 0 2021-11-21T21:54:03.971 INFO:teuthology.orchestra.run.gibba036.stdout: auth debug = true 2021-11-21T21:54:03.972 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.972 INFO:teuthology.orchestra.run.gibba036.stdout: ms die on old message = true 2021-11-21T21:54:03.972 INFO:teuthology.orchestra.run.gibba036.stdout: ms die on bug = true 2021-11-21T21:54:03.973 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.973 INFO:teuthology.orchestra.run.gibba036.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T21:54:03.973 INFO:teuthology.orchestra.run.gibba036.stdout: mon pg warn max object skew = 0 2021-11-21T21:54:03.974 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.974 INFO:teuthology.orchestra.run.gibba036.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T21:54:03.974 INFO:teuthology.orchestra.run.gibba036.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T21:54:03.974 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.974 INFO:teuthology.orchestra.run.gibba036.stdout: osd pool default size = 2 2021-11-21T21:54:03.975 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.975 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd allow primary affinity = true 2021-11-21T21:54:03.975 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd allow pg remap = true 2021-11-21T21:54:03.975 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on legacy crush tunables = false 2021-11-21T21:54:03.975 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on crush straw calc version zero = false 2021-11-21T21:54:03.976 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on no sortbitwise = false 2021-11-21T21:54:03.976 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on osd down out interval zero = false 2021-11-21T21:54:03.976 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on too few osds = false 2021-11-21T21:54:03.976 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T21:54:03.977 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T21:54:03.977 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.977 INFO:teuthology.orchestra.run.gibba036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T21:54:03.978 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.978 INFO:teuthology.orchestra.run.gibba036.stdout: osd default data pool replay window = 5 2021-11-21T21:54:03.978 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.978 INFO:teuthology.orchestra.run.gibba036.stdout: mon allow pool delete = true 2021-11-21T21:54:03.978 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.979 INFO:teuthology.orchestra.run.gibba036.stdout: mon cluster log file level = debug 2021-11-21T21:54:03.979 INFO:teuthology.orchestra.run.gibba036.stdout: debug asserts on shutdown = true 2021-11-21T21:54:03.979 INFO:teuthology.orchestra.run.gibba036.stdout: mon health detail to clog = false 2021-11-21T21:54:03.979 INFO:teuthology.orchestra.run.gibba036.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T21:54:03.980 INFO:teuthology.orchestra.run.gibba036.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T21:54:03.980 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.980 INFO:teuthology.orchestra.run.gibba036.stdout:[osd] 2021-11-21T21:54:03.980 INFO:teuthology.orchestra.run.gibba036.stdout: osd journal size = 100 2021-11-21T21:54:03.980 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.981 INFO:teuthology.orchestra.run.gibba036.stdout: osd scrub load threshold = 5.0 2021-11-21T21:54:03.981 INFO:teuthology.orchestra.run.gibba036.stdout: osd scrub max interval = 600 2021-11-21T21:54:03.981 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.981 INFO:teuthology.orchestra.run.gibba036.stdout: osd recover clone overlap = true 2021-11-21T21:54:03.981 INFO:teuthology.orchestra.run.gibba036.stdout: osd recovery max chunk = 1048576 2021-11-21T21:54:03.982 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.982 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug shutdown = true 2021-11-21T21:54:03.982 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug op order = true 2021-11-21T21:54:03.982 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug verify stray on activate = true 2021-11-21T21:54:03.983 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.983 INFO:teuthology.orchestra.run.gibba036.stdout: osd open classes on start = true 2021-11-21T21:54:03.984 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug pg log writeout = true 2021-11-21T21:54:03.984 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.984 INFO:teuthology.orchestra.run.gibba036.stdout: osd deep scrub update digest min age = 30 2021-11-21T21:54:03.984 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.985 INFO:teuthology.orchestra.run.gibba036.stdout: osd map max advance = 10 2021-11-21T21:54:03.985 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.985 INFO:teuthology.orchestra.run.gibba036.stdout: journal zero on create = true 2021-11-21T21:54:03.985 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.985 INFO:teuthology.orchestra.run.gibba036.stdout: filestore ondisk finisher threads = 3 2021-11-21T21:54:03.986 INFO:teuthology.orchestra.run.gibba036.stdout: filestore apply finisher threads = 3 2021-11-21T21:54:03.986 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.986 INFO:teuthology.orchestra.run.gibba036.stdout: bdev debug aio = true 2021-11-21T21:54:03.986 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug misdirected ops = true 2021-11-21T21:54:03.987 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T21:54:03.987 INFO:teuthology.orchestra.run.gibba036.stdout: debug osd = 20 2021-11-21T21:54:03.987 INFO:teuthology.orchestra.run.gibba036.stdout: osd objectstore = filestore 2021-11-21T21:54:03.987 INFO:teuthology.orchestra.run.gibba036.stdout: osd op complaint time = 180 2021-11-21T21:54:03.987 INFO:teuthology.orchestra.run.gibba036.stdout: osd sloppy crc = True 2021-11-21T21:54:03.988 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.988 INFO:teuthology.orchestra.run.gibba036.stdout:[mgr] 2021-11-21T21:54:03.988 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T21:54:03.988 INFO:teuthology.orchestra.run.gibba036.stdout: debug mgr = 20 2021-11-21T21:54:03.988 INFO:teuthology.orchestra.run.gibba036.stdout: debug mon = 20 2021-11-21T21:54:03.989 INFO:teuthology.orchestra.run.gibba036.stdout: debug auth = 20 2021-11-21T21:54:03.989 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:03.989 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:03.990 INFO:teuthology.orchestra.run.gibba036.stdout: mgr/telemetry/nag = false 2021-11-21T21:54:03.990 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.990 INFO:teuthology.orchestra.run.gibba036.stdout:[mon] 2021-11-21T21:54:03.990 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T21:54:03.991 INFO:teuthology.orchestra.run.gibba036.stdout: debug mon = 20 2021-11-21T21:54:03.991 INFO:teuthology.orchestra.run.gibba036.stdout: debug paxos = 20 2021-11-21T21:54:03.991 INFO:teuthology.orchestra.run.gibba036.stdout: debug auth = 20 2021-11-21T21:54:03.991 INFO:teuthology.orchestra.run.gibba036.stdout: mon data avail warn = 5 2021-11-21T21:54:03.991 INFO:teuthology.orchestra.run.gibba036.stdout: mon mgr mkfs grace = 240 2021-11-21T21:54:03.992 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:03.992 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd reporter subtree level = osd 2021-11-21T21:54:03.992 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd prime pg temp = true 2021-11-21T21:54:03.992 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:03.992 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.993 INFO:teuthology.orchestra.run.gibba036.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T21:54:03.993 INFO:teuthology.orchestra.run.gibba036.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T21:54:03.993 INFO:teuthology.orchestra.run.gibba036.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T21:54:03.994 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.994 INFO:teuthology.orchestra.run.gibba036.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T21:54:03.994 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T21:54:03.994 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T21:54:03.995 INFO:teuthology.orchestra.run.gibba036.stdout: mon op complaint time = 120 2021-11-21T21:54:03.995 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T21:54:03.995 INFO:teuthology.orchestra.run.gibba036.stdout:[client] 2021-11-21T21:54:03.995 INFO:teuthology.orchestra.run.gibba036.stdout: rgw cache enabled = true 2021-11-21T21:54:03.996 INFO:teuthology.orchestra.run.gibba036.stdout: rgw enable ops log = true 2021-11-21T21:54:03.996 INFO:teuthology.orchestra.run.gibba036.stdout: rgw enable usage log = true 2021-11-21T21:54:03.996 INFO:teuthology.orchestra.run.gibba036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T21:54:03.996 INFO:teuthology.orchestra.run.gibba036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T21:54:03.996 INFO:teuthology.orchestra.run.gibba036.stdout: client mount timeout = 600 2021-11-21T21:54:03.997 INFO:teuthology.orchestra.run.gibba036.stdout: debug client = 20 2021-11-21T21:54:03.997 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T21:54:03.997 INFO:teuthology.orchestra.run.gibba036.stdout: rados mon op timeout = 15m 2021-11-21T21:54:03.997 INFO:teuthology.orchestra.run.gibba036.stdout: rados osd op timeout = 15m 2021-11-21T21:54:03.997 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.a] 2021-11-21T21:54:03.998 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.b] 2021-11-21T21:54:03.998 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.c] 2021-11-21T21:54:03.998 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.a] 2021-11-21T21:54:03.998 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.c] 2021-11-21T21:54:03.998 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.b] 2021-11-21T21:54:03.999 INFO:teuthology.orchestra.run.gibba036.stdout:[mds] 2021-11-21T21:54:03.999 INFO:teuthology.orchestra.run.gibba036.stdout: debug mds = 20 2021-11-21T21:54:03.999 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T21:54:04.000 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal frag = True 2021-11-21T21:54:04.000 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal fragment size max = 10000 2021-11-21T21:54:04.000 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal merge size = 5 2021-11-21T21:54:04.000 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal split bits = 3 2021-11-21T21:54:04.000 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal split size = 100 2021-11-21T21:54:04.001 INFO:teuthology.orchestra.run.gibba036.stdout: mds debug frag = True 2021-11-21T21:54:04.001 INFO:teuthology.orchestra.run.gibba036.stdout: mds debug scatterstat = True 2021-11-21T21:54:04.001 INFO:teuthology.orchestra.run.gibba036.stdout: mds op complaint time = 180 2021-11-21T21:54:04.001 INFO:teuthology.orchestra.run.gibba036.stdout: mds verify scatter = True 2021-11-21T21:54:04.002 INFO:teuthology.orchestra.run.gibba036.stdout: osd op complaint time = 180 2021-11-21T21:54:04.002 INFO:teuthology.orchestra.run.gibba036.stdout: rados mon op timeout = 15m 2021-11-21T21:54:04.002 INFO:teuthology.orchestra.run.gibba036.stdout: rados osd op timeout = 15m 2021-11-21T21:54:04.002 INFO:teuthology.orchestra.run.gibba019.stdout:[global] 2021-11-21T21:54:04.003 INFO:teuthology.orchestra.run.gibba019.stdout: chdir = "" 2021-11-21T21:54:04.003 INFO:teuthology.orchestra.run.gibba019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T21:54:04.003 INFO:teuthology.orchestra.run.gibba019.stdout: auth supported = cephx 2021-11-21T21:54:04.003 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.003 INFO:teuthology.orchestra.run.gibba019.stdout: filestore xattr use omap = true 2021-11-21T21:54:04.004 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.004 INFO:teuthology.orchestra.run.gibba019.stdout: mon clock drift allowed = 1.000 2021-11-21T21:54:04.004 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.005 INFO:teuthology.orchestra.run.gibba019.stdout: osd crush chooseleaf type = 0 2021-11-21T21:54:04.005 INFO:teuthology.orchestra.run.gibba019.stdout: auth debug = true 2021-11-21T21:54:04.005 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.005 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on old message = true 2021-11-21T21:54:04.006 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on bug = true 2021-11-21T21:54:04.006 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.006 INFO:teuthology.orchestra.run.gibba019.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T21:54:04.006 INFO:teuthology.orchestra.run.gibba019.stdout: mon pg warn max object skew = 0 2021-11-21T21:54:04.006 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.007 INFO:teuthology.orchestra.run.gibba019.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T21:54:04.007 INFO:teuthology.orchestra.run.gibba019.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T21:54:04.007 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.007 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default size = 2 2021-11-21T21:54:04.007 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.008 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow primary affinity = true 2021-11-21T21:54:04.008 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow pg remap = true 2021-11-21T21:54:04.008 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on legacy crush tunables = false 2021-11-21T21:54:04.008 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on crush straw calc version zero = false 2021-11-21T21:54:04.008 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on no sortbitwise = false 2021-11-21T21:54:04.009 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on osd down out interval zero = false 2021-11-21T21:54:04.009 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on too few osds = false 2021-11-21T21:54:04.009 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T21:54:04.009 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T21:54:04.010 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.010 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T21:54:04.010 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.010 INFO:teuthology.orchestra.run.gibba019.stdout: osd default data pool replay window = 5 2021-11-21T21:54:04.010 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.011 INFO:teuthology.orchestra.run.gibba019.stdout: mon allow pool delete = true 2021-11-21T21:54:04.011 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.011 INFO:teuthology.orchestra.run.gibba019.stdout: mon cluster log file level = debug 2021-11-21T21:54:04.011 INFO:teuthology.orchestra.run.gibba019.stdout: debug asserts on shutdown = true 2021-11-21T21:54:04.011 INFO:teuthology.orchestra.run.gibba019.stdout: mon health detail to clog = false 2021-11-21T21:54:04.012 INFO:teuthology.orchestra.run.gibba019.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T21:54:04.012 INFO:teuthology.orchestra.run.gibba019.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T21:54:04.012 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.012 INFO:teuthology.orchestra.run.gibba019.stdout:[osd] 2021-11-21T21:54:04.012 INFO:teuthology.orchestra.run.gibba019.stdout: osd journal size = 100 2021-11-21T21:54:04.013 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.013 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub load threshold = 5.0 2021-11-21T21:54:04.013 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub max interval = 600 2021-11-21T21:54:04.013 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.014 INFO:teuthology.orchestra.run.gibba019.stdout: osd recover clone overlap = true 2021-11-21T21:54:04.014 INFO:teuthology.orchestra.run.gibba019.stdout: osd recovery max chunk = 1048576 2021-11-21T21:54:04.014 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.015 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug shutdown = true 2021-11-21T21:54:04.015 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug op order = true 2021-11-21T21:54:04.015 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug verify stray on activate = true 2021-11-21T21:54:04.015 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.015 INFO:teuthology.orchestra.run.gibba019.stdout: osd open classes on start = true 2021-11-21T21:54:04.016 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug pg log writeout = true 2021-11-21T21:54:04.016 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.016 INFO:teuthology.orchestra.run.gibba019.stdout: osd deep scrub update digest min age = 30 2021-11-21T21:54:04.016 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.017 INFO:teuthology.orchestra.run.gibba019.stdout: osd map max advance = 10 2021-11-21T21:54:04.017 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.017 INFO:teuthology.orchestra.run.gibba019.stdout: journal zero on create = true 2021-11-21T21:54:04.017 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.017 INFO:teuthology.orchestra.run.gibba019.stdout: filestore ondisk finisher threads = 3 2021-11-21T21:54:04.018 INFO:teuthology.orchestra.run.gibba019.stdout: filestore apply finisher threads = 3 2021-11-21T21:54:04.018 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.018 INFO:teuthology.orchestra.run.gibba019.stdout: bdev debug aio = true 2021-11-21T21:54:04.018 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug misdirected ops = true 2021-11-21T21:54:04.018 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T21:54:04.019 INFO:teuthology.orchestra.run.gibba019.stdout: debug osd = 20 2021-11-21T21:54:04.019 INFO:teuthology.orchestra.run.gibba019.stdout: osd objectstore = filestore 2021-11-21T21:54:04.019 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-11-21T21:54:04.019 INFO:teuthology.orchestra.run.gibba019.stdout: osd sloppy crc = True 2021-11-21T21:54:04.019 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.020 INFO:teuthology.orchestra.run.gibba019.stdout:[mgr] 2021-11-21T21:54:04.020 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T21:54:04.020 INFO:teuthology.orchestra.run.gibba019.stdout: debug mgr = 20 2021-11-21T21:54:04.020 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-11-21T21:54:04.021 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-11-21T21:54:04.021 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:04.021 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:04.021 INFO:teuthology.orchestra.run.gibba019.stdout: mgr/telemetry/nag = false 2021-11-21T21:54:04.021 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.022 INFO:teuthology.orchestra.run.gibba019.stdout:[mon] 2021-11-21T21:54:04.022 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T21:54:04.022 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-11-21T21:54:04.022 INFO:teuthology.orchestra.run.gibba019.stdout: debug paxos = 20 2021-11-21T21:54:04.022 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-11-21T21:54:04.023 INFO:teuthology.orchestra.run.gibba019.stdout: mon data avail warn = 5 2021-11-21T21:54:04.023 INFO:teuthology.orchestra.run.gibba019.stdout: mon mgr mkfs grace = 240 2021-11-21T21:54:04.023 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:04.024 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd reporter subtree level = osd 2021-11-21T21:54:04.024 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd prime pg temp = true 2021-11-21T21:54:04.024 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:04.024 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.024 INFO:teuthology.orchestra.run.gibba019.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T21:54:04.025 INFO:teuthology.orchestra.run.gibba019.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T21:54:04.025 INFO:teuthology.orchestra.run.gibba019.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T21:54:04.025 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.026 INFO:teuthology.orchestra.run.gibba019.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T21:54:04.026 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T21:54:04.026 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T21:54:04.026 INFO:teuthology.orchestra.run.gibba019.stdout: mon op complaint time = 120 2021-11-21T21:54:04.026 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T21:54:04.027 INFO:teuthology.orchestra.run.gibba019.stdout:[client] 2021-11-21T21:54:04.027 INFO:teuthology.orchestra.run.gibba019.stdout: rgw cache enabled = true 2021-11-21T21:54:04.027 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable ops log = true 2021-11-21T21:54:04.027 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable usage log = true 2021-11-21T21:54:04.028 INFO:teuthology.orchestra.run.gibba019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T21:54:04.028 INFO:teuthology.orchestra.run.gibba019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T21:54:04.028 INFO:teuthology.orchestra.run.gibba019.stdout: client mount timeout = 600 2021-11-21T21:54:04.028 INFO:teuthology.orchestra.run.gibba019.stdout: debug client = 20 2021-11-21T21:54:04.028 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T21:54:04.029 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-11-21T21:54:04.029 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-11-21T21:54:04.029 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.a] 2021-11-21T21:54:04.029 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.b] 2021-11-21T21:54:04.029 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.c] 2021-11-21T21:54:04.030 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.a] 2021-11-21T21:54:04.030 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.c] 2021-11-21T21:54:04.030 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.b] 2021-11-21T21:54:04.030 INFO:teuthology.orchestra.run.gibba019.stdout:[mds] 2021-11-21T21:54:04.030 INFO:teuthology.orchestra.run.gibba019.stdout: debug mds = 20 2021-11-21T21:54:04.031 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T21:54:04.031 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal frag = True 2021-11-21T21:54:04.031 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal fragment size max = 10000 2021-11-21T21:54:04.031 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal merge size = 5 2021-11-21T21:54:04.032 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split bits = 3 2021-11-21T21:54:04.032 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split size = 100 2021-11-21T21:54:04.032 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug frag = True 2021-11-21T21:54:04.032 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug scatterstat = True 2021-11-21T21:54:04.032 INFO:teuthology.orchestra.run.gibba019.stdout: mds op complaint time = 180 2021-11-21T21:54:04.033 INFO:teuthology.orchestra.run.gibba019.stdout: mds verify scatter = True 2021-11-21T21:54:04.033 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-11-21T21:54:04.033 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-11-21T21:54:04.033 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-11-21T21:54:04.033 INFO:teuthology.orchestra.run.gibba017.stdout:[global] 2021-11-21T21:54:04.034 INFO:teuthology.orchestra.run.gibba017.stdout: chdir = "" 2021-11-21T21:54:04.034 INFO:teuthology.orchestra.run.gibba017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T21:54:04.034 INFO:teuthology.orchestra.run.gibba017.stdout: auth supported = cephx 2021-11-21T21:54:04.034 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.035 INFO:teuthology.orchestra.run.gibba017.stdout: filestore xattr use omap = true 2021-11-21T21:54:04.035 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.035 INFO:teuthology.orchestra.run.gibba017.stdout: mon clock drift allowed = 1.000 2021-11-21T21:54:04.036 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.036 INFO:teuthology.orchestra.run.gibba017.stdout: osd crush chooseleaf type = 0 2021-11-21T21:54:04.036 INFO:teuthology.orchestra.run.gibba017.stdout: auth debug = true 2021-11-21T21:54:04.036 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.036 INFO:teuthology.orchestra.run.gibba017.stdout: ms die on old message = true 2021-11-21T21:54:04.037 INFO:teuthology.orchestra.run.gibba017.stdout: ms die on bug = true 2021-11-21T21:54:04.037 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.037 INFO:teuthology.orchestra.run.gibba017.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T21:54:04.037 INFO:teuthology.orchestra.run.gibba017.stdout: mon pg warn max object skew = 0 2021-11-21T21:54:04.037 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.038 INFO:teuthology.orchestra.run.gibba017.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T21:54:04.038 INFO:teuthology.orchestra.run.gibba017.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T21:54:04.038 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.038 INFO:teuthology.orchestra.run.gibba017.stdout: osd pool default size = 2 2021-11-21T21:54:04.039 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd allow primary affinity = true 2021-11-21T21:54:04.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd allow pg remap = true 2021-11-21T21:54:04.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on legacy crush tunables = false 2021-11-21T21:54:04.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on crush straw calc version zero = false 2021-11-21T21:54:04.040 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on no sortbitwise = false 2021-11-21T21:54:04.040 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on osd down out interval zero = false 2021-11-21T21:54:04.040 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on too few osds = false 2021-11-21T21:54:04.040 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T21:54:04.040 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T21:54:04.041 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.041 INFO:teuthology.orchestra.run.gibba017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T21:54:04.041 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.041 INFO:teuthology.orchestra.run.gibba017.stdout: osd default data pool replay window = 5 2021-11-21T21:54:04.041 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.042 INFO:teuthology.orchestra.run.gibba017.stdout: mon allow pool delete = true 2021-11-21T21:54:04.042 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.042 INFO:teuthology.orchestra.run.gibba017.stdout: mon cluster log file level = debug 2021-11-21T21:54:04.042 INFO:teuthology.orchestra.run.gibba017.stdout: debug asserts on shutdown = true 2021-11-21T21:54:04.042 INFO:teuthology.orchestra.run.gibba017.stdout: mon health detail to clog = false 2021-11-21T21:54:04.043 INFO:teuthology.orchestra.run.gibba017.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T21:54:04.043 INFO:teuthology.orchestra.run.gibba017.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T21:54:04.043 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.043 INFO:teuthology.orchestra.run.gibba017.stdout:[osd] 2021-11-21T21:54:04.044 INFO:teuthology.orchestra.run.gibba017.stdout: osd journal size = 100 2021-11-21T21:54:04.044 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.044 INFO:teuthology.orchestra.run.gibba017.stdout: osd scrub load threshold = 5.0 2021-11-21T21:54:04.044 INFO:teuthology.orchestra.run.gibba017.stdout: osd scrub max interval = 600 2021-11-21T21:54:04.044 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.045 INFO:teuthology.orchestra.run.gibba017.stdout: osd recover clone overlap = true 2021-11-21T21:54:04.045 INFO:teuthology.orchestra.run.gibba017.stdout: osd recovery max chunk = 1048576 2021-11-21T21:54:04.045 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.045 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug shutdown = true 2021-11-21T21:54:04.046 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug op order = true 2021-11-21T21:54:04.046 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug verify stray on activate = true 2021-11-21T21:54:04.046 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.046 INFO:teuthology.orchestra.run.gibba017.stdout: osd open classes on start = true 2021-11-21T21:54:04.047 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug pg log writeout = true 2021-11-21T21:54:04.047 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.047 INFO:teuthology.orchestra.run.gibba017.stdout: osd deep scrub update digest min age = 30 2021-11-21T21:54:04.047 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.048 INFO:teuthology.orchestra.run.gibba017.stdout: osd map max advance = 10 2021-11-21T21:54:04.048 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.048 INFO:teuthology.orchestra.run.gibba017.stdout: journal zero on create = true 2021-11-21T21:54:04.048 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.048 INFO:teuthology.orchestra.run.gibba017.stdout: filestore ondisk finisher threads = 3 2021-11-21T21:54:04.049 INFO:teuthology.orchestra.run.gibba017.stdout: filestore apply finisher threads = 3 2021-11-21T21:54:04.049 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.049 INFO:teuthology.orchestra.run.gibba017.stdout: bdev debug aio = true 2021-11-21T21:54:04.049 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug misdirected ops = true 2021-11-21T21:54:04.049 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T21:54:04.050 INFO:teuthology.orchestra.run.gibba017.stdout: debug osd = 20 2021-11-21T21:54:04.050 INFO:teuthology.orchestra.run.gibba017.stdout: osd objectstore = filestore 2021-11-21T21:54:04.050 INFO:teuthology.orchestra.run.gibba017.stdout: osd op complaint time = 180 2021-11-21T21:54:04.050 INFO:teuthology.orchestra.run.gibba017.stdout: osd sloppy crc = True 2021-11-21T21:54:04.050 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.051 INFO:teuthology.orchestra.run.gibba017.stdout:[mgr] 2021-11-21T21:54:04.051 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T21:54:04.051 INFO:teuthology.orchestra.run.gibba017.stdout: debug mgr = 20 2021-11-21T21:54:04.051 INFO:teuthology.orchestra.run.gibba017.stdout: debug mon = 20 2021-11-21T21:54:04.051 INFO:teuthology.orchestra.run.gibba017.stdout: debug auth = 20 2021-11-21T21:54:04.052 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:04.052 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:04.052 INFO:teuthology.orchestra.run.gibba017.stdout: mgr/telemetry/nag = false 2021-11-21T21:54:04.052 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.052 INFO:teuthology.orchestra.run.gibba017.stdout:[mon] 2021-11-21T21:54:04.053 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T21:54:04.053 INFO:teuthology.orchestra.run.gibba017.stdout: debug mon = 20 2021-11-21T21:54:04.053 INFO:teuthology.orchestra.run.gibba017.stdout: debug paxos = 20 2021-11-21T21:54:04.053 INFO:teuthology.orchestra.run.gibba017.stdout: debug auth = 20 2021-11-21T21:54:04.054 INFO:teuthology.orchestra.run.gibba017.stdout: mon data avail warn = 5 2021-11-21T21:54:04.054 INFO:teuthology.orchestra.run.gibba017.stdout: mon mgr mkfs grace = 240 2021-11-21T21:54:04.054 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min pgs per osd = 4 2021-11-21T21:54:04.055 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd reporter subtree level = osd 2021-11-21T21:54:04.055 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd prime pg temp = true 2021-11-21T21:54:04.055 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min bytes per osd = 10 2021-11-21T21:54:04.055 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.055 INFO:teuthology.orchestra.run.gibba017.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T21:54:04.056 INFO:teuthology.orchestra.run.gibba017.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T21:54:04.056 INFO:teuthology.orchestra.run.gibba017.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T21:54:04.056 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.056 INFO:teuthology.orchestra.run.gibba017.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T21:54:04.056 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T21:54:04.057 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T21:54:04.057 INFO:teuthology.orchestra.run.gibba017.stdout: mon op complaint time = 120 2021-11-21T21:54:04.057 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T21:54:04.057 INFO:teuthology.orchestra.run.gibba017.stdout:[client] 2021-11-21T21:54:04.058 INFO:teuthology.orchestra.run.gibba017.stdout: rgw cache enabled = true 2021-11-21T21:54:04.058 INFO:teuthology.orchestra.run.gibba017.stdout: rgw enable ops log = true 2021-11-21T21:54:04.058 INFO:teuthology.orchestra.run.gibba017.stdout: rgw enable usage log = true 2021-11-21T21:54:04.058 INFO:teuthology.orchestra.run.gibba017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T21:54:04.058 INFO:teuthology.orchestra.run.gibba017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T21:54:04.059 INFO:teuthology.orchestra.run.gibba017.stdout: client mount timeout = 600 2021-11-21T21:54:04.059 INFO:teuthology.orchestra.run.gibba017.stdout: debug client = 20 2021-11-21T21:54:04.059 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T21:54:04.059 INFO:teuthology.orchestra.run.gibba017.stdout: rados mon op timeout = 15m 2021-11-21T21:54:04.059 INFO:teuthology.orchestra.run.gibba017.stdout: rados osd op timeout = 15m 2021-11-21T21:54:04.060 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.a] 2021-11-21T21:54:04.060 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.b] 2021-11-21T21:54:04.060 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.c] 2021-11-21T21:54:04.060 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.a] 2021-11-21T21:54:04.060 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.c] 2021-11-21T21:54:04.061 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.b] 2021-11-21T21:54:04.061 INFO:teuthology.orchestra.run.gibba017.stdout:[mds] 2021-11-21T21:54:04.061 INFO:teuthology.orchestra.run.gibba017.stdout: debug mds = 20 2021-11-21T21:54:04.061 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T21:54:04.061 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal frag = True 2021-11-21T21:54:04.062 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal fragment size max = 10000 2021-11-21T21:54:04.062 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal merge size = 5 2021-11-21T21:54:04.062 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal split bits = 3 2021-11-21T21:54:04.062 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal split size = 100 2021-11-21T21:54:04.062 INFO:teuthology.orchestra.run.gibba017.stdout: mds debug frag = True 2021-11-21T21:54:04.063 INFO:teuthology.orchestra.run.gibba017.stdout: mds debug scatterstat = True 2021-11-21T21:54:04.063 INFO:teuthology.orchestra.run.gibba017.stdout: mds op complaint time = 180 2021-11-21T21:54:04.063 INFO:teuthology.orchestra.run.gibba017.stdout: mds verify scatter = True 2021-11-21T21:54:04.063 INFO:teuthology.orchestra.run.gibba017.stdout: osd op complaint time = 180 2021-11-21T21:54:04.063 INFO:teuthology.orchestra.run.gibba017.stdout: rados mon op timeout = 15m 2021-11-21T21:54:04.064 INFO:teuthology.orchestra.run.gibba017.stdout: rados osd op timeout = 15m 2021-11-21T21:54:04.064 INFO:tasks.ceph:Creating admin key on mon.a... 2021-11-21T21:54:04.065 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2021-11-21T21:54:04.105 INFO:tasks.ceph:Copying monmap to all nodes... 2021-11-21T21:54:04.113 DEBUG:teuthology.orchestra.remote:gibba016:/etc/ceph/ceph.keyring is 206B 2021-11-21T21:54:04.171 DEBUG:teuthology.orchestra.remote:gibba016:/home/ubuntu/cephtest/ceph.monmap is 379B 2021-11-21T21:54:04.191 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T21:54:04.191 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:54:04.192 DEBUG:teuthology.orchestra.run.gibba016:> sudo dd of=/etc/ceph/ceph.keyring 2021-11-21T21:54:04.192 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-11-21T21:54:04.227 DEBUG:teuthology.orchestra.run.gibba016:> set -ex 2021-11-21T21:54:04.228 DEBUG:teuthology.orchestra.run.gibba016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:04.280 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T21:54:04.280 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:54:04.281 DEBUG:teuthology.orchestra.run.gibba017:> sudo dd of=/etc/ceph/ceph.keyring 2021-11-21T21:54:04.281 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-11-21T21:54:04.300 DEBUG:teuthology.orchestra.run.gibba017:> set -ex 2021-11-21T21:54:04.301 DEBUG:teuthology.orchestra.run.gibba017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:04.347 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba019.front.sepia.ceph.com 2021-11-21T21:54:04.347 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:54:04.348 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/ceph/ceph.keyring 2021-11-21T21:54:04.348 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-11-21T21:54:04.366 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-11-21T21:54:04.367 DEBUG:teuthology.orchestra.run.gibba019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:04.371 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba036.front.sepia.ceph.com 2021-11-21T21:54:04.371 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:54:04.372 DEBUG:teuthology.orchestra.run.gibba036:> sudo dd of=/etc/ceph/ceph.keyring 2021-11-21T21:54:04.372 DEBUG:teuthology.orchestra.run.gibba036:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-11-21T21:54:04.390 DEBUG:teuthology.orchestra.run.gibba036:> set -ex 2021-11-21T21:54:04.390 DEBUG:teuthology.orchestra.run.gibba036:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:04.436 INFO:tasks.ceph:Setting up mon nodes... 2021-11-21T21:54:04.437 INFO:tasks.ceph:Setting up mgr nodes... 2021-11-21T21:54:04.437 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2021-11-21T21:54:04.461 INFO:teuthology.orchestra.run.gibba016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2021-11-21T21:54:04.463 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2021-11-21T21:54:04.505 INFO:teuthology.orchestra.run.gibba017.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2021-11-21T21:54:04.506 INFO:tasks.ceph:Setting up mds nodes... 2021-11-21T21:54:04.507 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2021-11-21T21:54:04.533 INFO:teuthology.orchestra.run.gibba016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2021-11-21T21:54:04.534 DEBUG:teuthology.orchestra.run.gibba016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2021-11-21T21:54:04.540 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2021-11-21T21:54:04.557 INFO:teuthology.orchestra.run.gibba016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2021-11-21T21:54:04.558 DEBUG:teuthology.orchestra.run.gibba016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2021-11-21T21:54:04.607 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2021-11-21T21:54:04.643 INFO:teuthology.orchestra.run.gibba017.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2021-11-21T21:54:04.644 DEBUG:teuthology.orchestra.run.gibba017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2021-11-21T21:54:04.651 INFO:tasks.ceph_client:Setting up client nodes... 2021-11-21T21:54:04.651 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-11-21T21:54:04.703 INFO:teuthology.orchestra.run.gibba019.stdout:creating /etc/ceph/ceph.client.0.keyring 2021-11-21T21:54:04.710 DEBUG:teuthology.orchestra.run.gibba036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2021-11-21T21:54:04.761 INFO:teuthology.orchestra.run.gibba036.stdout:creating /etc/ceph/ceph.client.1.keyring 2021-11-21T21:54:04.765 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-11-21T21:54:04.766 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@gibba016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba017.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-11-21T21:54:04.766 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2021-11-21T21:54:04.778 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:04.779 INFO:tasks.ceph:role: osd.0 2021-11-21T21:54:04.779 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T21:54:04.780 DEBUG:teuthology.orchestra.run.gibba016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-11-21T21:54:04.819 INFO:teuthology.orchestra.run.gibba016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:04.820 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:04.820 INFO:teuthology.orchestra.run.gibba016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:04.820 INFO:teuthology.orchestra.run.gibba016.stdout: = reflink=1 2021-11-21T21:54:04.820 INFO:teuthology.orchestra.run.gibba016.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:04.820 INFO:teuthology.orchestra.run.gibba016.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:04.821 INFO:teuthology.orchestra.run.gibba016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:04.821 INFO:teuthology.orchestra.run.gibba016.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:04.821 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:04.821 INFO:teuthology.orchestra.run.gibba016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:04.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba016.front.sepia.ceph.com -o noatime 2021-11-21T21:54:04.823 DEBUG:teuthology.orchestra.run.gibba016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2021-11-21T21:54:04.894 DEBUG:teuthology.orchestra.run.gibba016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2021-11-21T21:54:04.910 INFO:teuthology.orchestra.run.gibba016.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:04.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:04.912 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2021-11-21T21:54:04.922 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:04.922 INFO:tasks.ceph:role: osd.1 2021-11-21T21:54:04.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T21:54:04.923 DEBUG:teuthology.orchestra.run.gibba016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-11-21T21:54:05.005 INFO:teuthology.orchestra.run.gibba016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:05.005 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:05.006 INFO:teuthology.orchestra.run.gibba016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:05.006 INFO:teuthology.orchestra.run.gibba016.stdout: = reflink=1 2021-11-21T21:54:05.006 INFO:teuthology.orchestra.run.gibba016.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:05.006 INFO:teuthology.orchestra.run.gibba016.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:05.007 INFO:teuthology.orchestra.run.gibba016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:05.007 INFO:teuthology.orchestra.run.gibba016.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:05.007 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:05.007 INFO:teuthology.orchestra.run.gibba016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:05.008 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba016.front.sepia.ceph.com -o noatime 2021-11-21T21:54:05.009 DEBUG:teuthology.orchestra.run.gibba016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2021-11-21T21:54:05.042 DEBUG:teuthology.orchestra.run.gibba016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2021-11-21T21:54:05.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:05.094 INFO:teuthology.orchestra.run.gibba016.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:05.094 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2021-11-21T21:54:05.103 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:05.103 INFO:tasks.ceph:role: osd.2 2021-11-21T21:54:05.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T21:54:05.104 DEBUG:teuthology.orchestra.run.gibba016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-11-21T21:54:05.191 INFO:teuthology.orchestra.run.gibba016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:05.192 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:05.192 INFO:teuthology.orchestra.run.gibba016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:05.192 INFO:teuthology.orchestra.run.gibba016.stdout: = reflink=1 2021-11-21T21:54:05.193 INFO:teuthology.orchestra.run.gibba016.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:05.193 INFO:teuthology.orchestra.run.gibba016.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:05.193 INFO:teuthology.orchestra.run.gibba016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:05.193 INFO:teuthology.orchestra.run.gibba016.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:05.193 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:05.194 INFO:teuthology.orchestra.run.gibba016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:05.195 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba016.front.sepia.ceph.com -o noatime 2021-11-21T21:54:05.196 DEBUG:teuthology.orchestra.run.gibba016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2021-11-21T21:54:05.227 DEBUG:teuthology.orchestra.run.gibba016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2021-11-21T21:54:05.286 INFO:teuthology.orchestra.run.gibba016.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:05.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:05.288 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2021-11-21T21:54:05.310 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:05.310 INFO:tasks.ceph:role: osd.3 2021-11-21T21:54:05.310 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T21:54:05.311 DEBUG:teuthology.orchestra.run.gibba016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-11-21T21:54:05.410 INFO:teuthology.orchestra.run.gibba016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:05.411 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:05.411 INFO:teuthology.orchestra.run.gibba016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:05.411 INFO:teuthology.orchestra.run.gibba016.stdout: = reflink=1 2021-11-21T21:54:05.411 INFO:teuthology.orchestra.run.gibba016.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:05.412 INFO:teuthology.orchestra.run.gibba016.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:05.412 INFO:teuthology.orchestra.run.gibba016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:05.412 INFO:teuthology.orchestra.run.gibba016.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:05.412 INFO:teuthology.orchestra.run.gibba016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:05.413 INFO:teuthology.orchestra.run.gibba016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:05.414 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba016.front.sepia.ceph.com -o noatime 2021-11-21T21:54:05.414 DEBUG:teuthology.orchestra.run.gibba016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2021-11-21T21:54:05.446 DEBUG:teuthology.orchestra.run.gibba016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2021-11-21T21:54:05.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:05.452 INFO:teuthology.orchestra.run.gibba016.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:05.452 DEBUG:teuthology.orchestra.run.gibba016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:05.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.566+0000 7fd4807b7d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2021-11-21T21:54:05.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.566+0000 7fd4807b7d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2021-11-21T21:54:05.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.574+0000 7fd4807b7d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:05.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.574+0000 7fd4807b7d80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:05.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.642+0000 7fd4807b7d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:05.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.642+0000 7fd4807b7d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:05.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.642+0000 7fd4807b7d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:05.653 DEBUG:teuthology.orchestra.run.gibba016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-11-21T21:54:05.659 DEBUG:teuthology.orchestra.run.gibba016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:05.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.678+0000 7fb821b29d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2021-11-21T21:54:05.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.678+0000 7fb821b29d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2021-11-21T21:54:05.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.686+0000 7fb821b29d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:05.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.690+0000 7fb821b29d80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:05.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.762+0000 7fb821b29d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:05.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.762+0000 7fb821b29d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:05.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.762+0000 7fb821b29d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:05.774 DEBUG:teuthology.orchestra.run.gibba016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-11-21T21:54:05.825 DEBUG:teuthology.orchestra.run.gibba016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:05.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.874+0000 7f0ee1b5dd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2021-11-21T21:54:05.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.874+0000 7f0ee1b5dd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2021-11-21T21:54:05.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.886+0000 7f0ee1b5dd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:05.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.886+0000 7f0ee1b5dd80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:05.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.970+0000 7f0ee1b5dd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:05.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.970+0000 7f0ee1b5dd80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:05.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:05.970+0000 7f0ee1b5dd80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:05.980 DEBUG:teuthology.orchestra.run.gibba016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-11-21T21:54:05.986 DEBUG:teuthology.orchestra.run.gibba016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:06.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:06.002+0000 7f69597a4d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2021-11-21T21:54:06.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:06.002+0000 7f69597a4d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2021-11-21T21:54:06.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:06.010+0000 7f69597a4d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:06.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:06.010+0000 7f69597a4d80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:06.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:06.086+0000 7f69597a4d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:06.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:06.086+0000 7f69597a4d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:06.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:06.086+0000 7f69597a4d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:06.097 DEBUG:teuthology.orchestra.run.gibba016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-11-21T21:54:06.149 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2021-11-21T21:54:06.161 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:06.162 INFO:tasks.ceph:role: osd.4 2021-11-21T21:54:06.162 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T21:54:06.162 DEBUG:teuthology.orchestra.run.gibba017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-11-21T21:54:06.244 INFO:teuthology.orchestra.run.gibba017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:06.245 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:06.245 INFO:teuthology.orchestra.run.gibba017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:06.245 INFO:teuthology.orchestra.run.gibba017.stdout: = reflink=1 2021-11-21T21:54:06.245 INFO:teuthology.orchestra.run.gibba017.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:06.246 INFO:teuthology.orchestra.run.gibba017.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:06.246 INFO:teuthology.orchestra.run.gibba017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:06.246 INFO:teuthology.orchestra.run.gibba017.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:06.246 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:06.247 INFO:teuthology.orchestra.run.gibba017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:06.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba017.front.sepia.ceph.com -o noatime 2021-11-21T21:54:06.249 DEBUG:teuthology.orchestra.run.gibba017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2021-11-21T21:54:06.259 DEBUG:teuthology.orchestra.run.gibba017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2021-11-21T21:54:06.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:06.264 INFO:teuthology.orchestra.run.gibba017.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:06.264 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2021-11-21T21:54:06.318 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:06.319 INFO:tasks.ceph:role: osd.5 2021-11-21T21:54:06.319 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T21:54:06.320 DEBUG:teuthology.orchestra.run.gibba017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-11-21T21:54:06.360 INFO:teuthology.orchestra.run.gibba017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:06.360 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:06.360 INFO:teuthology.orchestra.run.gibba017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:06.360 INFO:teuthology.orchestra.run.gibba017.stdout: = reflink=1 2021-11-21T21:54:06.361 INFO:teuthology.orchestra.run.gibba017.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:06.361 INFO:teuthology.orchestra.run.gibba017.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:06.361 INFO:teuthology.orchestra.run.gibba017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:06.361 INFO:teuthology.orchestra.run.gibba017.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:06.361 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:06.362 INFO:teuthology.orchestra.run.gibba017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:06.363 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba017.front.sepia.ceph.com -o noatime 2021-11-21T21:54:06.364 DEBUG:teuthology.orchestra.run.gibba017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2021-11-21T21:54:06.438 DEBUG:teuthology.orchestra.run.gibba017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2021-11-21T21:54:06.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:06.447 INFO:teuthology.orchestra.run.gibba017.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:06.448 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2021-11-21T21:54:06.453 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:06.453 INFO:tasks.ceph:role: osd.6 2021-11-21T21:54:06.454 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T21:54:06.455 DEBUG:teuthology.orchestra.run.gibba017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-11-21T21:54:06.539 INFO:teuthology.orchestra.run.gibba017.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:06.539 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:06.540 INFO:teuthology.orchestra.run.gibba017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:06.540 INFO:teuthology.orchestra.run.gibba017.stdout: = reflink=1 2021-11-21T21:54:06.540 INFO:teuthology.orchestra.run.gibba017.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:06.540 INFO:teuthology.orchestra.run.gibba017.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:06.541 INFO:teuthology.orchestra.run.gibba017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:06.541 INFO:teuthology.orchestra.run.gibba017.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:06.541 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:06.541 INFO:teuthology.orchestra.run.gibba017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:06.542 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba017.front.sepia.ceph.com -o noatime 2021-11-21T21:54:06.543 DEBUG:teuthology.orchestra.run.gibba017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2021-11-21T21:54:06.559 DEBUG:teuthology.orchestra.run.gibba017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2021-11-21T21:54:06.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:06.564 INFO:teuthology.orchestra.run.gibba017.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:06.565 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2021-11-21T21:54:06.615 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-11-21T21:54:06.615 INFO:tasks.ceph:role: osd.7 2021-11-21T21:54:06.616 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T21:54:06.616 DEBUG:teuthology.orchestra.run.gibba017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-11-21T21:54:06.663 INFO:teuthology.orchestra.run.gibba017.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-11-21T21:54:06.663 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 attr=2, projid32bit=1 2021-11-21T21:54:06.663 INFO:teuthology.orchestra.run.gibba017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-11-21T21:54:06.664 INFO:teuthology.orchestra.run.gibba017.stdout: = reflink=1 2021-11-21T21:54:06.664 INFO:teuthology.orchestra.run.gibba017.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-11-21T21:54:06.664 INFO:teuthology.orchestra.run.gibba017.stdout: = sunit=0 swidth=0 blks 2021-11-21T21:54:06.664 INFO:teuthology.orchestra.run.gibba017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-11-21T21:54:06.664 INFO:teuthology.orchestra.run.gibba017.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-11-21T21:54:06.665 INFO:teuthology.orchestra.run.gibba017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-11-21T21:54:06.665 INFO:teuthology.orchestra.run.gibba017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-11-21T21:54:06.666 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba017.front.sepia.ceph.com -o noatime 2021-11-21T21:54:06.667 DEBUG:teuthology.orchestra.run.gibba017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2021-11-21T21:54:06.694 DEBUG:teuthology.orchestra.run.gibba017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2021-11-21T21:54:06.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:06.748 INFO:teuthology.orchestra.run.gibba017.stderr:sudo: /sbin/restorecon: command not found 2021-11-21T21:54:06.748 DEBUG:teuthology.orchestra.run.gibba017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:06.819 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:06.813+0000 7fad93fb2d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2021-11-21T21:54:06.819 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:06.813+0000 7fad93fb2d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2021-11-21T21:54:06.825 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:06.821+0000 7fad93fb2d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:06.826 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:06.821+0000 7fad93fb2d80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:06.898 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:06.893+0000 7fad93fb2d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:06.898 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:06.893+0000 7fad93fb2d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:06.899 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:06.893+0000 7fad93fb2d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:06.922 DEBUG:teuthology.orchestra.run.gibba017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-11-21T21:54:06.943 DEBUG:teuthology.orchestra.run.gibba017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:07.058 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.053+0000 7f6534bd7d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2021-11-21T21:54:07.059 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.053+0000 7f6534bd7d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2021-11-21T21:54:07.069 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.065+0000 7f6534bd7d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:07.069 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.065+0000 7f6534bd7d80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:07.141 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.137+0000 7f6534bd7d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:07.141 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.137+0000 7f6534bd7d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:07.142 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.137+0000 7f6534bd7d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:07.148 DEBUG:teuthology.orchestra.run.gibba017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-11-21T21:54:07.155 DEBUG:teuthology.orchestra.run.gibba017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:07.218 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.213+0000 7f4a01985d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2021-11-21T21:54:07.218 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.213+0000 7f4a01985d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2021-11-21T21:54:07.225 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.221+0000 7f4a01985d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:07.225 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.221+0000 7f4a01985d80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:07.304 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.301+0000 7f4a01985d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:07.304 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.301+0000 7f4a01985d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:07.304 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.301+0000 7f4a01985d80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:07.326 DEBUG:teuthology.orchestra.run.gibba017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-11-21T21:54:07.347 DEBUG:teuthology.orchestra.run.gibba017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:07.459 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.457+0000 7f1f4df3ad80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2021-11-21T21:54:07.460 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.457+0000 7f1f4df3ad80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2021-11-21T21:54:07.470 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.465+0000 7f1f4df3ad80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:07.470 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.465+0000 7f1f4df3ad80 -1 journal FileJournal::_open_file : zeroing journal 2021-11-21T21:54:07.545 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.541+0000 7f1f4df3ad80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:07.545 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.541+0000 7f1f4df3ad80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:07.546 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:07.541+0000 7f1f4df3ad80 -1 journal do_read_entry(4096): bad header magic 2021-11-21T21:54:07.551 DEBUG:teuthology.orchestra.run.gibba017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-11-21T21:54:07.559 INFO:tasks.ceph:Reading keys from all nodes... 2021-11-21T21:54:07.560 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:07.569 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.b62YfIArWe 2021-11-21T21:54:07.570 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.b62YfIArWe 2021-11-21T21:54:07.576 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.b62YfIArWe 2021-11-21T21:54:07.636 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.b62YfIArWe is 56B 2021-11-21T21:54:07.649 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.b62YfIArWe 2021-11-21T21:54:07.657 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:07.661 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.ZWk29iEJzb 2021-11-21T21:54:07.661 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.ZWk29iEJzb 2021-11-21T21:54:07.711 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.ZWk29iEJzb 2021-11-21T21:54:07.728 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.ZWk29iEJzb is 56B 2021-11-21T21:54:07.741 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.ZWk29iEJzb 2021-11-21T21:54:07.748 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:07.801 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.m7fMycSKo1 2021-11-21T21:54:07.801 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/mds/ceph-c/keyring /tmp/tmp.m7fMycSKo1 2021-11-21T21:54:07.823 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.m7fMycSKo1 2021-11-21T21:54:07.903 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.m7fMycSKo1 is 56B 2021-11-21T21:54:07.918 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.m7fMycSKo1 2021-11-21T21:54:07.922 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:07.967 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.L6RukSTd10 2021-11-21T21:54:07.968 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.L6RukSTd10 2021-11-21T21:54:07.977 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.L6RukSTd10 2021-11-21T21:54:07.997 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.L6RukSTd10 is 56B 2021-11-21T21:54:08.051 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.L6RukSTd10 2021-11-21T21:54:08.061 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:08.112 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.vIgaaHeWiF 2021-11-21T21:54:08.112 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.vIgaaHeWiF 2021-11-21T21:54:08.135 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.vIgaaHeWiF 2021-11-21T21:54:08.215 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.vIgaaHeWiF is 56B 2021-11-21T21:54:08.235 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.vIgaaHeWiF 2021-11-21T21:54:08.245 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:08.291 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.OSJOlT8Nda 2021-11-21T21:54:08.292 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.OSJOlT8Nda 2021-11-21T21:54:08.303 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.OSJOlT8Nda 2021-11-21T21:54:08.383 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.OSJOlT8Nda is 56B 2021-11-21T21:54:08.400 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.OSJOlT8Nda 2021-11-21T21:54:08.404 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:08.451 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.jdZaqmT4Dr 2021-11-21T21:54:08.452 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.jdZaqmT4Dr 2021-11-21T21:54:08.463 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.jdZaqmT4Dr 2021-11-21T21:54:08.539 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.jdZaqmT4Dr is 56B 2021-11-21T21:54:08.561 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.jdZaqmT4Dr 2021-11-21T21:54:08.570 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:08.579 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.kHbE2BqJU8 2021-11-21T21:54:08.579 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.kHbE2BqJU8 2021-11-21T21:54:08.586 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.kHbE2BqJU8 2021-11-21T21:54:08.643 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.kHbE2BqJU8 is 56B 2021-11-21T21:54:08.656 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.kHbE2BqJU8 2021-11-21T21:54:08.665 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:08.715 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.xXTnbmIB3H 2021-11-21T21:54:08.716 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.xXTnbmIB3H 2021-11-21T21:54:08.738 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.xXTnbmIB3H 2021-11-21T21:54:08.817 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.xXTnbmIB3H is 56B 2021-11-21T21:54:08.835 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.xXTnbmIB3H 2021-11-21T21:54:08.839 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:08.883 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.jhhrMNj72A 2021-11-21T21:54:08.883 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.jhhrMNj72A 2021-11-21T21:54:08.892 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.jhhrMNj72A 2021-11-21T21:54:08.950 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.jhhrMNj72A is 56B 2021-11-21T21:54:08.961 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.jhhrMNj72A 2021-11-21T21:54:08.967 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:09.024 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.ONWjbsGrdh 2021-11-21T21:54:09.025 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.ONWjbsGrdh 2021-11-21T21:54:09.088 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.ONWjbsGrdh 2021-11-21T21:54:09.125 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.ONWjbsGrdh is 56B 2021-11-21T21:54:09.144 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.ONWjbsGrdh 2021-11-21T21:54:09.154 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:09.203 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.K1tkLz1hWq 2021-11-21T21:54:09.203 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.K1tkLz1hWq 2021-11-21T21:54:09.212 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.K1tkLz1hWq 2021-11-21T21:54:09.271 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.K1tkLz1hWq is 56B 2021-11-21T21:54:09.285 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.K1tkLz1hWq 2021-11-21T21:54:09.294 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:09.343 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.oMMWMswpsJ 2021-11-21T21:54:09.343 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.oMMWMswpsJ 2021-11-21T21:54:09.366 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.oMMWMswpsJ 2021-11-21T21:54:09.443 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.oMMWMswpsJ is 56B 2021-11-21T21:54:09.462 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.oMMWMswpsJ 2021-11-21T21:54:09.483 DEBUG:teuthology.orchestra.remote:gibba019:/etc/ceph/ceph.client.0.keyring is 59B 2021-11-21T21:54:09.547 DEBUG:teuthology.orchestra.remote:gibba036:/etc/ceph/ceph.client.1.keyring is 59B 2021-11-21T21:54:09.558 INFO:tasks.ceph:Adding keys to all mons... 2021-11-21T21:54:09.558 DEBUG:teuthology.orchestra.run.gibba016:> sudo tee -a /etc/ceph/ceph.keyring 2021-11-21T21:54:09.563 DEBUG:teuthology.orchestra.run.gibba017:> sudo tee -a /etc/ceph/ceph.keyring 2021-11-21T21:54:09.570 INFO:teuthology.orchestra.run.gibba016.stdout:[mgr.y] 2021-11-21T21:54:09.570 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA== 2021-11-21T21:54:09.570 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.a] 2021-11-21T21:54:09.571 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ== 2021-11-21T21:54:09.571 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.c] 2021-11-21T21:54:09.571 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ== 2021-11-21T21:54:09.571 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.0] 2021-11-21T21:54:09.572 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA== 2021-11-21T21:54:09.572 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.1] 2021-11-21T21:54:09.572 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ== 2021-11-21T21:54:09.572 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.2] 2021-11-21T21:54:09.572 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg== 2021-11-21T21:54:09.573 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.3] 2021-11-21T21:54:09.573 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A== 2021-11-21T21:54:09.573 INFO:teuthology.orchestra.run.gibba016.stdout:[mgr.x] 2021-11-21T21:54:09.573 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ== 2021-11-21T21:54:09.574 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.b] 2021-11-21T21:54:09.574 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg== 2021-11-21T21:54:09.574 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.4] 2021-11-21T21:54:09.574 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA== 2021-11-21T21:54:09.575 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.5] 2021-11-21T21:54:09.575 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA== 2021-11-21T21:54:09.575 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.6] 2021-11-21T21:54:09.575 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA== 2021-11-21T21:54:09.576 INFO:teuthology.orchestra.run.gibba016.stdout:[osd.7] 2021-11-21T21:54:09.576 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg== 2021-11-21T21:54:09.576 INFO:teuthology.orchestra.run.gibba016.stdout:[client.0] 2021-11-21T21:54:09.576 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg== 2021-11-21T21:54:09.576 INFO:teuthology.orchestra.run.gibba016.stdout:[client.1] 2021-11-21T21:54:09.577 INFO:teuthology.orchestra.run.gibba016.stdout: key = AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg== 2021-11-21T21:54:09.579 INFO:teuthology.orchestra.run.gibba017.stdout:[mgr.y] 2021-11-21T21:54:09.579 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA== 2021-11-21T21:54:09.579 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.a] 2021-11-21T21:54:09.579 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ== 2021-11-21T21:54:09.580 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.c] 2021-11-21T21:54:09.580 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ== 2021-11-21T21:54:09.580 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.0] 2021-11-21T21:54:09.580 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA== 2021-11-21T21:54:09.580 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.1] 2021-11-21T21:54:09.581 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ== 2021-11-21T21:54:09.581 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.2] 2021-11-21T21:54:09.581 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg== 2021-11-21T21:54:09.581 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.3] 2021-11-21T21:54:09.581 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A== 2021-11-21T21:54:09.582 INFO:teuthology.orchestra.run.gibba017.stdout:[mgr.x] 2021-11-21T21:54:09.582 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ== 2021-11-21T21:54:09.582 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.b] 2021-11-21T21:54:09.582 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg== 2021-11-21T21:54:09.583 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.4] 2021-11-21T21:54:09.583 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA== 2021-11-21T21:54:09.583 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.5] 2021-11-21T21:54:09.583 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA== 2021-11-21T21:54:09.583 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.6] 2021-11-21T21:54:09.584 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA== 2021-11-21T21:54:09.584 INFO:teuthology.orchestra.run.gibba017.stdout:[osd.7] 2021-11-21T21:54:09.584 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg== 2021-11-21T21:54:09.584 INFO:teuthology.orchestra.run.gibba017.stdout:[client.0] 2021-11-21T21:54:09.584 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg== 2021-11-21T21:54:09.585 INFO:teuthology.orchestra.run.gibba017.stdout:[client.1] 2021-11-21T21:54:09.585 INFO:teuthology.orchestra.run.gibba017.stdout: key = AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg== 2021-11-21T21:54:09.585 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-11-21T21:54:09.588 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-11-21T21:54:09.608 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-11-21T21:54:09.649 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-11-21T21:54:09.679 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-11-21T21:54:09.681 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-11-21T21:54:09.711 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.714 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.769 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.777 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.799 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.802 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.880 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.883 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:09.936 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-11-21T21:54:09.938 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-11-21T21:54:09.964 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-11-21T21:54:10.009 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-11-21T21:54:10.075 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.078 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.133 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.136 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.165 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.167 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.230 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.233 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-11-21T21:54:10.256 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-11-21T21:54:10.259 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-11-21T21:54:10.283 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-11-21T21:54:10.286 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-11-21T21:54:10.364 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-11-21T21:54:10.365 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2021-11-21T21:54:10.374 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-11-21T21:54:10.439 DEBUG:teuthology.orchestra.run.gibba016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2021-11-21T21:54:10.452 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2021-11-21T21:54:10.474 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-11-21T21:54:10.525 DEBUG:teuthology.orchestra.run.gibba017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2021-11-21T21:54:10.537 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2021-11-21T21:54:10.544 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-11-21T21:54:10.575 DEBUG:teuthology.orchestra.run.gibba017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2021-11-21T21:54:10.583 DEBUG:teuthology.orchestra.run.gibba016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:10.587 DEBUG:teuthology.orchestra.run.gibba017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-11-21T21:54:10.634 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2021-11-21T21:54:10.635 INFO:tasks.ceph.mon.a:Restarting daemon 2021-11-21T21:54:10.635 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2021-11-21T21:54:10.638 INFO:tasks.ceph.mon.a:Started 2021-11-21T21:54:10.638 INFO:tasks.ceph.mon.b:Restarting daemon 2021-11-21T21:54:10.639 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2021-11-21T21:54:10.642 INFO:tasks.ceph.mon.b:Started 2021-11-21T21:54:10.642 INFO:tasks.ceph.mon.c:Restarting daemon 2021-11-21T21:54:10.643 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2021-11-21T21:54:10.645 INFO:tasks.ceph.mon.c:Started 2021-11-21T21:54:10.645 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2021-11-21T21:54:10.646 INFO:tasks.ceph.mgr.y:Restarting daemon 2021-11-21T21:54:10.646 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2021-11-21T21:54:10.648 INFO:tasks.ceph.mgr.y:Started 2021-11-21T21:54:10.649 INFO:tasks.ceph.mgr.x:Restarting daemon 2021-11-21T21:54:10.649 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2021-11-21T21:54:10.651 INFO:tasks.ceph.mgr.x:Started 2021-11-21T21:54:10.651 INFO:tasks.ceph:Setting crush tunables to default 2021-11-21T21:54:10.652 DEBUG:teuthology.orchestra.run.gibba016:> sudo ceph --cluster ceph osd crush tunables default 2021-11-21T21:54:10.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb644630700 1 Processor -- start 2021-11-21T21:54:10.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb644630700 1 -- start start 2021-11-21T21:54:10.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb644630700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:10.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb644630700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:10.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb644630700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c064260 con 0x7fb63c063e80 2021-11-21T21:54:10.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb644630700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c140550 con 0x7fb63c13d410 2021-11-21T21:54:10.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb644630700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c1407f0 con 0x7fb63c0603b0 2021-11-21T21:54:10.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb6423cc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb63c063e80 0x7fb63c13cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:55892/0 (socket says 172.21.2.116:55892) 2021-11-21T21:54:10.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb6423cc700 1 -- 172.21.2.116:0/1065017644 learned_addr learned my addr 172.21.2.116:0/1065017644 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:10.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:10.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb642bcd700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:10.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb641bcb700 1 -- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 msgr2=0x7fb63c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2021-11-21T21:54:10.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb641bcb700 1 -- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 msgr2=0x7fb63c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-11-21T21:54:10.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-11-21T21:54:10.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-11-21T21:54:10.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb642bcd700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 msgr2=0x7fb63c0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-11-21T21:54:10.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb642bcd700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 msgr2=0x7fb63c0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-11-21T21:54:10.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb642bcd700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-11-21T21:54:10.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.710+0000 7fb642bcd700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-11-21T21:54:10.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb642bcd700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:10.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:10.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb642bcd700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 msgr2=0x7fb63c0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-11-21T21:54:10.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb642bcd700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 msgr2=0x7fb63c0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-11-21T21:54:10.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb642bcd700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-11-21T21:54:10.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb642bcd700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2021-11-21T21:54:10.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb641bcb700 1 -- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 msgr2=0x7fb63c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2021-11-21T21:54:10.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb641bcb700 1 -- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 msgr2=0x7fb63c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-11-21T21:54:10.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-11-21T21:54:10.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:10.910+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2021-11-21T21:54:11.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.010+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744174767 0 0) 0x7fb63c064260 con 0x7fb63c063e80 2021-11-21T21:54:11.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.010+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb624003260 con 0x7fb63c063e80 2021-11-21T21:54:11.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642363999 0 0) 0x7fb624003260 con 0x7fb63c063e80 2021-11-21T21:54:11.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 msgr2=0x7fb63c0af550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-11-21T21:54:11.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb63c0af550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 msgr2=0x7fb63c0af060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-11-21T21:54:11.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 --2- 172.21.2.116:0/1065017644 >> v2:172.21.2.117:3300/0 conn(0x7fb63c13d410 0x7fb63c0af060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0ae920 con 0x7fb63c063e80 2021-11-21T21:54:11.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb638001680 con 0x7fb63c063e80 2021-11-21T21:54:11.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 >> v1:172.21.2.116:6789/0 conn(0x7fb63c063e80 legacy=0x7fb63c13cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:11.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624003750 0x7fb624003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:11.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb624004500 0x7fb624004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:11.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb624004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:11.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c140550 con 0x7fb624004500 2021-11-21T21:54:11.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c1407f0 con 0x7fb624003750 2021-11-21T21:54:11.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62400b7a0 con 0x7fb63c0603b0 2021-11-21T21:54:11.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb624004500 0x7fb624004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:11.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb6423cc700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624003750 0x7fb624003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:11.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb642bcd700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb624004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:11.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb641bcb700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 msgr2=0x7fb624004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:11.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c0603b0 0x7fb624004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb641bcb700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624003750 msgr2=0x7fb624003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:11.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624003750 0x7fb624003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb641bcb700 1 -- 172.21.2.116:0/1065017644 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624003260 con 0x7fb624004500 2021-11-21T21:54:11.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.014+0000 7fb641bcb700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb624004500 0x7fb624004980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fb63000c000 tx=0x7fb630006ed0).ready entity=mon.0 client_cookie=f848bb527ce52e29 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:11.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb630011a20 con 0x7fb624004500 2021-11-21T21:54:11.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630011b80 con 0x7fb624004500 2021-11-21T21:54:11.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb6413ca700 1 -- 172.21.2.116:0/1065017644 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb63001a440 con 0x7fb624004500 2021-11-21T21:54:11.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb644630700 1 -- 172.21.2.116:0/1065017644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb624004500 msgr2=0x7fb624004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:11.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb644630700 1 --2- 172.21.2.116:0/1065017644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb624004500 0x7fb624004980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fb63000c000 tx=0x7fb630006ed0).stop 2021-11-21T21:54:11.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb642bcd700 1 -- 172.21.2.116:0/1065017644 reap_dead start 2021-11-21T21:54:11.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb644630700 1 -- 172.21.2.116:0/1065017644 shutdown_connections 2021-11-21T21:54:11.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb644630700 1 -- 172.21.2.116:0/1065017644 >> 172.21.2.116:0/1065017644 conn(0x7fb63c061d40 msgr2=0x7fb63c062120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:11.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb644630700 1 -- 172.21.2.116:0/1065017644 shutdown_connections 2021-11-21T21:54:11.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.022+0000 7fb644630700 1 -- 172.21.2.116:0/1065017644 wait complete. 2021-11-21T21:54:11.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.026+0000 7fb644630700 1 Processor -- start 2021-11-21T21:54:11.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.026+0000 7fb644630700 1 -- start start 2021-11-21T21:54:11.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb644630700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624004500 0x7fb63c13edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:11.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb644630700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c134b20 0x7fb63c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:11.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb644630700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb63c155cd0 0x7fb63c135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:11.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb644630700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0b2fb0 con 0x7fb63c155cd0 2021-11-21T21:54:11.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb644630700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0b2790 con 0x7fb624004500 2021-11-21T21:54:11.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb644630700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0b23f0 con 0x7fb63c134b20 2021-11-21T21:54:11.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624004500 0x7fb63c13edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:11.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb642bcd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb63c155cd0 0x7fb63c135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:11.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624004500 0x7fb63c13edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:45140/0 (socket says 172.21.2.116:45140) 2021-11-21T21:54:11.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 -- 172.21.2.116:0/2609506191 learned_addr learned my addr 172.21.2.116:0/2609506191 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:11.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb641bcb700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c134b20 0x7fb63c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:11.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 -- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c134b20 msgr2=0x7fb63c1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:11.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c134b20 0x7fb63c1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 -- 172.21.2.116:0/2609506191 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb63c155cd0 msgr2=0x7fb63c135ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:11.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb63c155cd0 0x7fb63c135ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 -- 172.21.2.116:0/2609506191 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0b29f0 con 0x7fb624004500 2021-11-21T21:54:11.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb641bcb700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c134b20 0x7fb63c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:11.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.030+0000 7fb6423cc700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624004500 0x7fb63c13edb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fb63800d000 tx=0x7fb63c13f950).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:11.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.034+0000 7fb62f7fe700 1 -- 172.21.2.116:0/2609506191 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6380116b0 con 0x7fb624004500 2021-11-21T21:54:11.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.034+0000 7fb62f7fe700 1 -- 172.21.2.116:0/2609506191 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb638011810 con 0x7fb624004500 2021-11-21T21:54:11.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.034+0000 7fb62f7fe700 1 -- 172.21.2.116:0/2609506191 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb638011b70 con 0x7fb624004500 2021-11-21T21:54:11.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.034+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c0b1df0 con 0x7fb624004500 2021-11-21T21:54:11.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.034+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c0b1830 con 0x7fb624004500 2021-11-21T21:54:11.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.034+0000 7fb62f7fe700 1 -- 172.21.2.116:0/2609506191 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fb63801b030 con 0x7fb624004500 2021-11-21T21:54:11.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.034+0000 7fb62f7fe700 1 -- 172.21.2.116:0/2609506191 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fb63801e970 con 0x7fb624004500 2021-11-21T21:54:11.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.038+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c1564c0 con 0x7fb624004500 2021-11-21T21:54:11.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.046+0000 7fb62f7fe700 1 -- 172.21.2.116:0/2609506191 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fb638019070 con 0x7fb624004500 2021-11-21T21:54:11.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.126+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb63c0573d0 con 0x7fb624004500 2021-11-21T21:54:11.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb62f7fe700 1 -- 172.21.2.116:0/2609506191 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb6380263a0 con 0x7fb624004500 2021-11-21T21:54:11.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624004500 msgr2=0x7fb63c13edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624004500 0x7fb63c13edb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fb63800d000 tx=0x7fb63c13f950).stop 2021-11-21T21:54:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 shutdown_connections 2021-11-21T21:54:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb63c155cd0 0x7fb63c135ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb63c134b20 0x7fb63c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 --2- 172.21.2.116:0/2609506191 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb624004500 0x7fb63c13edb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 >> 172.21.2.116:0/2609506191 conn(0x7fb63c061d40 msgr2=0x7fb63c137160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 shutdown_connections 2021-11-21T21:54:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:11.134+0000 7fb644630700 1 -- 172.21.2.116:0/2609506191 wait complete. 2021-11-21T21:54:11.141 INFO:teuthology.orchestra.run.gibba016.stderr:adjusted tunables profile to default 2021-11-21T21:54:11.146 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2021-11-21T21:54:11.146 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:11.150 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.fIMuisEqup 2021-11-21T21:54:11.151 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.fIMuisEqup 2021-11-21T21:54:11.156 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.fIMuisEqup 2021-11-21T21:54:11.167 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.fIMuisEqup is 37B 2021-11-21T21:54:11.176 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.fIMuisEqup 2021-11-21T21:54:11.179 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:11.226 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.PKfa9zmKuL 2021-11-21T21:54:11.227 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.PKfa9zmKuL 2021-11-21T21:54:11.232 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.PKfa9zmKuL 2021-11-21T21:54:11.242 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.PKfa9zmKuL is 37B 2021-11-21T21:54:11.293 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.PKfa9zmKuL 2021-11-21T21:54:11.298 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:11.304 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.6Lqpl2l3le 2021-11-21T21:54:11.304 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.6Lqpl2l3le 2021-11-21T21:54:11.355 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.6Lqpl2l3le 2021-11-21T21:54:11.370 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.6Lqpl2l3le is 37B 2021-11-21T21:54:11.381 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.6Lqpl2l3le 2021-11-21T21:54:11.386 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T21:54:11.435 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.uLUcIgE7cV 2021-11-21T21:54:11.435 DEBUG:teuthology.orchestra.run.gibba016:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.uLUcIgE7cV 2021-11-21T21:54:11.454 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.uLUcIgE7cV 2021-11-21T21:54:11.512 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.uLUcIgE7cV is 37B 2021-11-21T21:54:11.523 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.uLUcIgE7cV 2021-11-21T21:54:11.526 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:11.532 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.IuVUG3II2d 2021-11-21T21:54:11.532 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.IuVUG3II2d 2021-11-21T21:54:11.580 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.IuVUG3II2d 2021-11-21T21:54:11.590 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.IuVUG3II2d is 37B 2021-11-21T21:54:11.639 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.IuVUG3II2d 2021-11-21T21:54:11.643 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:11.646 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.t0OrxaJ1oK 2021-11-21T21:54:11.647 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.t0OrxaJ1oK 2021-11-21T21:54:11.699 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.t0OrxaJ1oK 2021-11-21T21:54:11.752 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.t0OrxaJ1oK is 37B 2021-11-21T21:54:11.760 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.t0OrxaJ1oK 2021-11-21T21:54:11.763 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:11.814 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.AlKwgYDXVc 2021-11-21T21:54:11.814 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.AlKwgYDXVc 2021-11-21T21:54:11.821 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.AlKwgYDXVc 2021-11-21T21:54:11.875 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.AlKwgYDXVc is 37B 2021-11-21T21:54:11.883 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.AlKwgYDXVc 2021-11-21T21:54:11.888 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T21:54:11.891 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.QLk48kv47C 2021-11-21T21:54:11.891 DEBUG:teuthology.orchestra.run.gibba017:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.QLk48kv47C 2021-11-21T21:54:11.940 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.QLk48kv47C 2021-11-21T21:54:11.950 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.QLk48kv47C is 37B 2021-11-21T21:54:11.999 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.QLk48kv47C 2021-11-21T21:54:12.004 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new c3baac6c-e283-460c-b082-9b135ec136dc 0 2021-11-21T21:54:12.054 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.049+0000 7f70646cc700 1 Processor -- start 2021-11-21T21:54:12.055 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.049+0000 7f70646cc700 1 -- start start 2021-11-21T21:54:12.059 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.060 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f705c144a10 0x7f705c141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.060 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f705c141510 0x7f705c13a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.060 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c0b3c50 con 0x7f705c144a10 2021-11-21T21:54:12.060 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c0b3ef0 con 0x7f705c141510 2021-11-21T21:54:12.061 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0b4190 con 0x7f705c0adc10 2021-11-21T21:54:12.061 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062c69700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f705c141510 0x7f705c13a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.061 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.061 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43326/0 (socket says 172.21.2.117:43326) 2021-11-21T21:54:12.061 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062c69700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f705c141510 0x7f705c13a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50232/0 (socket says 172.21.2.117:50232) 2021-11-21T21:54:12.062 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 -- 172.21.2.117:0/3285545781 learned_addr learned my addr 172.21.2.117:0/3285545781 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.062 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 --2- 172.21.2.117:0/3285545781 >> v2:172.21.2.116:3300/0 conn(0x7f705c144a10 0x7f705c141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.062 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 -- 172.21.2.117:0/3285545781 >> v2:172.21.2.117:3300/0 conn(0x7f705c141510 msgr2=0x7f705c13a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.062 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 --2- 172.21.2.117:0/3285545781 >> v2:172.21.2.117:3300/0 conn(0x7f705c141510 0x7f705c13a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.062 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 -- 172.21.2.117:0/3285545781 >> v2:172.21.2.116:3300/0 conn(0x7f705c144a10 msgr2=0x7f705c141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.063 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 --2- 172.21.2.117:0/3285545781 >> v2:172.21.2.116:3300/0 conn(0x7f705c144a10 0x7f705c141020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.063 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 -- 172.21.2.117:0/3285545781 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c140b80 con 0x7f705c0adc10 2021-11-21T21:54:12.063 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062c69700 1 --2- 172.21.2.117:0/3285545781 >> v2:172.21.2.117:3300/0 conn(0x7f705c141510 0x7f705c13a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:12.063 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 --2- 172.21.2.117:0/3285545781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c0b2030 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7050002f70 tx=0x7f7050006eb0).ready entity=mon.2 client_cookie=59f05c497eb5ac99 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.064 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061466700 1 -- 172.21.2.117:0/3285545781 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f705000d570 con 0x7f705c0adc10 2021-11-21T21:54:12.064 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061466700 1 -- 172.21.2.117:0/3285545781 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050000e90 con 0x7f705c0adc10 2021-11-21T21:54:12.064 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061466700 1 -- 172.21.2.117:0/3285545781 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f705000d900 con 0x7f705c0adc10 2021-11-21T21:54:12.064 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- 172.21.2.117:0/3285545781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 msgr2=0x7f705c0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.065 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- 172.21.2.117:0/3285545781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c0b2030 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7050002f70 tx=0x7f7050006eb0).stop 2021-11-21T21:54:12.065 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- 172.21.2.117:0/3285545781 shutdown_connections 2021-11-21T21:54:12.065 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- 172.21.2.117:0/3285545781 >> v2:172.21.2.117:3300/0 conn(0x7f705c141510 0x7f705c13a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.065 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- 172.21.2.117:0/3285545781 >> v2:172.21.2.116:3300/0 conn(0x7f705c144a10 0x7f705c141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.065 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- 172.21.2.117:0/3285545781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c0b2030 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.066 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- 172.21.2.117:0/3285545781 >> 172.21.2.117:0/3285545781 conn(0x7f705c05bd20 msgr2=0x7f705c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.066 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- 172.21.2.117:0/3285545781 shutdown_connections 2021-11-21T21:54:12.066 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- 172.21.2.117:0/3285545781 wait complete. 2021-11-21T21:54:12.066 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 Processor -- start 2021-11-21T21:54:12.067 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- start start 2021-11-21T21:54:12.067 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.067 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f705c141510 0x7f705c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.067 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f705c144a10 0x7f705c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.067 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0aed70 con 0x7f705c144a10 2021-11-21T21:54:12.068 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0af010 con 0x7f705c141510 2021-11-21T21:54:12.068 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0af2b0 con 0x7f705c0adc10 2021-11-21T21:54:12.068 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.068 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c13d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43330/0 (socket says 172.21.2.117:43330) 2021-11-21T21:54:12.069 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062468700 1 -- 172.21.2.117:0/2014284942 learned_addr learned my addr 172.21.2.117:0/2014284942 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.069 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f705c141510 0x7f705c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.069 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7062c69700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f705c144a10 0x7f705c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.069 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 -- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 msgr2=0x7f705c13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.069 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.070 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 -- 172.21.2.117:0/2014284942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f705c144a10 msgr2=0x7f705c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.070 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f705c144a10 0x7f705c1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.070 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 -- 172.21.2.117:0/2014284942 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054009470 con 0x7f705c141510 2021-11-21T21:54:12.070 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f7061c67700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f705c141510 0x7f705c0b5520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f705800c000 tx=0x7f7058006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.070 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f704f7fe700 1 -- 172.21.2.117:0/2014284942 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7058010780 con 0x7f705c141510 2021-11-21T21:54:12.071 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f704f7fe700 1 -- 172.21.2.117:0/2014284942 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70580108e0 con 0x7f705c141510 2021-11-21T21:54:12.071 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705000b1e0 con 0x7f705c141510 2021-11-21T21:54:12.071 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f704f7fe700 1 -- 172.21.2.117:0/2014284942 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7058010c40 con 0x7f705c141510 2021-11-21T21:54:12.071 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.053+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c115be0 con 0x7f705c141510 2021-11-21T21:54:12.072 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.057+0000 7f704f7fe700 1 -- 172.21.2.117:0/2014284942 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f705801c460 con 0x7f705c141510 2021-11-21T21:54:12.072 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.057+0000 7f704f7fe700 1 -- 172.21.2.117:0/2014284942 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f7058010da0 con 0x7f705c141510 2021-11-21T21:54:12.072 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.057+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c13d810 con 0x7f705c141510 2021-11-21T21:54:12.072 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.057+0000 7f704f7fe700 1 -- 172.21.2.117:0/2014284942 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f705c13d810 con 0x7f705c141510 2021-11-21T21:54:12.105 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.101+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", "id": 0} v 0) v1 -- 0x7f705c0573d0 con 0x7f705c141510 2021-11-21T21:54:12.109 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f704f7fe700 1 -- 172.21.2.117:0/2014284942 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7058006020 con 0x7f705c141510 2021-11-21T21:54:12.110 INFO:teuthology.orchestra.run.gibba017.stdout:0 2021-11-21T21:54:12.111 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f705c141510 msgr2=0x7f705c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.111 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f705c141510 0x7f705c0b5520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f705800c000 tx=0x7f7058006df0).stop 2021-11-21T21:54:12.111 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 shutdown_connections 2021-11-21T21:54:12.111 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f705c144a10 0x7f705c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.112 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f705c141510 0x7f705c0b5520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.112 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 --2- 172.21.2.117:0/2014284942 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f705c0adc10 0x7f705c13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.112 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 >> 172.21.2.117:0/2014284942 conn(0x7f705c05bd20 msgr2=0x7f705c0b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.112 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 shutdown_connections 2021-11-21T21:54:12.113 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.105+0000 7f70646cc700 1 -- 172.21.2.117:0/2014284942 wait complete. 2021-11-21T21:54:12.116 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new 61210302-784a-4f36-a7bd-7c05c0e31483 1 2021-11-21T21:54:12.168 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 Processor -- start 2021-11-21T21:54:12.169 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- start start 2021-11-21T21:54:12.169 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc17c061060 0x7fc17c0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.169 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c09aa10 0x7fc17c146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.169 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c09b180 con 0x7fc17c05ced0 2021-11-21T21:54:12.170 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17c09aea0 con 0x7fc17c061060 2021-11-21T21:54:12.170 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c09b3e0 con 0x7fc17c09aa10 2021-11-21T21:54:12.173 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc17c061060 0x7fc17c0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.173 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc17c061060 0x7fc17c0acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50244/0 (socket says 172.21.2.117:50244) 2021-11-21T21:54:12.174 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 -- 172.21.2.117:0/1025540144 learned_addr learned my addr 172.21.2.117:0/1025540144 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.174 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17b7fe700 1 --2- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c09aa10 0x7fc17c146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.174 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 -- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c09aa10 msgr2=0x7fc17c146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.174 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 --2- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c09aa10 0x7fc17c146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.175 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 -- 172.21.2.117:0/1025540144 >> v1:172.21.2.117:6789/0 conn(0x7fc17c05ced0 legacy=0x7fc17c09a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.175 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 -- 172.21.2.117:0/1025540144 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c09b680 con 0x7fc17c061060 2021-11-21T21:54:12.175 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17b7fe700 1 --2- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c09aa10 0x7fc17c146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:12.175 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc17a7fc700 1 --2- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c061060 0x7fc17c0acb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fc16c002f70 tx=0x7fc16c006eb0).ready entity=mon.1 client_cookie=2e91fab4a97d88be server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.176 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc179ffb700 1 -- 172.21.2.117:0/1025540144 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc16c00d570 con 0x7fc17c061060 2021-11-21T21:54:12.176 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc179ffb700 1 -- 172.21.2.117:0/1025540144 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c000e90 con 0x7fc17c061060 2021-11-21T21:54:12.176 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc179ffb700 1 -- 172.21.2.117:0/1025540144 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc16c00d900 con 0x7fc17c061060 2021-11-21T21:54:12.176 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c061060 msgr2=0x7fc17c0acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.177 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 --2- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c061060 0x7fc17c0acb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fc16c002f70 tx=0x7fc16c006eb0).stop 2021-11-21T21:54:12.177 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- 172.21.2.117:0/1025540144 shutdown_connections 2021-11-21T21:54:12.177 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 --2- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c09aa10 0x7fc17c146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.177 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 --2- 172.21.2.117:0/1025540144 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c061060 0x7fc17c0acb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.177 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- 172.21.2.117:0/1025540144 >> 172.21.2.117:0/1025540144 conn(0x7fc17c0b5de0 msgr2=0x7fc17c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.178 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- 172.21.2.117:0/1025540144 shutdown_connections 2021-11-21T21:54:12.178 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 -- 172.21.2.117:0/1025540144 wait complete. 2021-11-21T21:54:12.178 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.165+0000 7fc1815eb700 1 Processor -- start 2021-11-21T21:54:12.178 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 -- start start 2021-11-21T21:54:12.179 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc17c05ced0 0x7fc17c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.179 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c061060 0x7fc17c0b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.179 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c09aa10 0x7fc17c0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.179 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c156500 con 0x7fc17c05ced0 2021-11-21T21:54:12.180 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0b1380 con 0x7fc17c09aa10 2021-11-21T21:54:12.180 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0b0f10 con 0x7fc17c061060 2021-11-21T21:54:12.180 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c09aa10 0x7fc17c0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.180 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17a7fc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c061060 0x7fc17c0b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.180 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c09aa10 0x7fc17c0ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50252/0 (socket says 172.21.2.117:50252) 2021-11-21T21:54:12.181 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 -- 172.21.2.117:0/853471939 learned_addr learned my addr 172.21.2.117:0/853471939 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.181 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17affd700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc17c05ced0 0x7fc17c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.181 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 -- 172.21.2.117:0/853471939 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c061060 msgr2=0x7fc17c0b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.181 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c061060 0x7fc17c0b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.182 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 -- 172.21.2.117:0/853471939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc17c05ced0 msgr2=0x7fc17c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.182 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc17c05ced0 0x7fc17c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.182 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 -- 172.21.2.117:0/853471939 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc174009470 con 0x7fc17c09aa10 2021-11-21T21:54:12.182 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17affd700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc17c05ced0 0x7fc17c0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:54:12.183 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc17b7fe700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c09aa10 0x7fc17c0ae0e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fc17c0b0a80 tx=0x7fc1740091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.183 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc173fff700 1 -- 172.21.2.117:0/853471939 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc17400c850 con 0x7fc17c09aa10 2021-11-21T21:54:12.183 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc173fff700 1 -- 172.21.2.117:0/853471939 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400c9b0 con 0x7fc17c09aa10 2021-11-21T21:54:12.183 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c00b1e0 con 0x7fc17c09aa10 2021-11-21T21:54:12.183 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc173fff700 1 -- 172.21.2.117:0/853471939 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc17400cd10 con 0x7fc17c09aa10 2021-11-21T21:54:12.184 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c0af380 con 0x7fc17c09aa10 2021-11-21T21:54:12.184 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc173fff700 1 -- 172.21.2.117:0/853471939 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fc174022070 con 0x7fc17c09aa10 2021-11-21T21:54:12.184 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc173fff700 1 -- 172.21.2.117:0/853471939 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fc17401bb60 con 0x7fc17c09aa10 2021-11-21T21:54:12.184 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c0b10d0 con 0x7fc17c09aa10 2021-11-21T21:54:12.185 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.169+0000 7fc173fff700 1 -- 172.21.2.117:0/853471939 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fc174039870 con 0x7fc17c09aa10 2021-11-21T21:54:12.219 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.217+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", "id": 1} v 0) v1 -- 0x7fc17c09b920 con 0x7fc17c09aa10 2021-11-21T21:54:12.223 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc173fff700 1 -- 172.21.2.117:0/853471939 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc174029030 con 0x7fc17c09aa10 2021-11-21T21:54:12.224 INFO:teuthology.orchestra.run.gibba017.stdout:1 2021-11-21T21:54:12.224 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c09aa10 msgr2=0x7fc17c0ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.225 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c09aa10 0x7fc17c0ae0e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fc17c0b0a80 tx=0x7fc1740091d0).stop 2021-11-21T21:54:12.225 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 shutdown_connections 2021-11-21T21:54:12.225 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc17c09aa10 0x7fc17c0ae0e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.225 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc17c061060 0x7fc17c0b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.226 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 --2- 172.21.2.117:0/853471939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc17c05ced0 0x7fc17c0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.226 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 >> 172.21.2.117:0/853471939 conn(0x7fc17c0b5de0 msgr2=0x7fc17c0601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.226 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 shutdown_connections 2021-11-21T21:54:12.226 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.221+0000 7fc1815eb700 1 -- 172.21.2.117:0/853471939 wait complete. 2021-11-21T21:54:12.230 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56 2 2021-11-21T21:54:12.284 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745efea700 1 Processor -- start 2021-11-21T21:54:12.284 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745efea700 1 -- start start 2021-11-21T21:54:12.285 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745efea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7458142590 0x7f745814b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.285 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745efea700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580b06c0 con 0x7f74580b02e0 2021-11-21T21:54:12.285 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745efea700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580adf20 con 0x7f745805f730 2021-11-21T21:54:12.285 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745efea700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580b2470 con 0x7f7458142590 2021-11-21T21:54:12.286 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f7457fff700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f745805f730 0x7f7458141e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.117:49692/0 (socket says 172.21.2.117:49692) 2021-11-21T21:54:12.286 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f7457fff700 1 -- 172.21.2.117:0/4196366862 learned_addr learned my addr 172.21.2.117:0/4196366862 (peer_addr_for_me v1:172.21.2.117:0/0) 2021-11-21T21:54:12.286 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745d587700 1 --2- 172.21.2.117:0/4196366862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7458142590 0x7f745814b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.286 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f74577fe700 1 -- 172.21.2.117:0/4196366862 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799035800 0 0) 0x7f74580adf20 con 0x7f745805f730 2021-11-21T21:54:12.286 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745d587700 1 -- 172.21.2.117:0/4196366862 >> v1:172.21.2.117:6789/0 conn(0x7f745805f730 legacy=0x7f7458141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.287 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745d587700 1 -- 172.21.2.117:0/4196366862 >> v1:172.21.2.116:6789/0 conn(0x7f74580b02e0 legacy=0x7f74580af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.287 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.277+0000 7f745d587700 1 -- 172.21.2.117:0/4196366862 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580b26d0 con 0x7f7458142590 2021-11-21T21:54:12.287 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745d587700 1 --2- 172.21.2.117:0/4196366862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7458142590 0x7f745814b550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f74500032f0 tx=0x7f74500091d0).ready entity=mon.2 client_cookie=81ea8a8912b27768 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.287 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f74577fe700 1 -- 172.21.2.117:0/4196366862 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745000b870 con 0x7f7458142590 2021-11-21T21:54:12.288 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f74577fe700 1 -- 172.21.2.117:0/4196366862 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745000b9d0 con 0x7f7458142590 2021-11-21T21:54:12.288 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f74577fe700 1 -- 172.21.2.117:0/4196366862 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745000bb40 con 0x7f7458142590 2021-11-21T21:54:12.288 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/4196366862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7458142590 msgr2=0x7f745814b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.288 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 --2- 172.21.2.117:0/4196366862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7458142590 0x7f745814b550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f74500032f0 tx=0x7f74500091d0).stop 2021-11-21T21:54:12.288 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/4196366862 shutdown_connections 2021-11-21T21:54:12.289 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 --2- 172.21.2.117:0/4196366862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7458142590 0x7f745814b550 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.289 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/4196366862 >> 172.21.2.117:0/4196366862 conn(0x7f74580b5ff0 msgr2=0x7f74580b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.289 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/4196366862 shutdown_connections 2021-11-21T21:54:12.289 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/4196366862 wait complete. 2021-11-21T21:54:12.289 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 Processor -- start 2021-11-21T21:54:12.290 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- start start 2021-11-21T21:54:12.292 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f745805f730 0x7f74580a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.292 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74580b02e0 0x7f74580a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.293 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7458142590 0x7f745809f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.293 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745805c430 con 0x7f745805f730 2021-11-21T21:54:12.293 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745805c6d0 con 0x7f7458142590 2021-11-21T21:54:12.293 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745805c970 con 0x7f74580b02e0 2021-11-21T21:54:12.293 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74580b02e0 0x7f74580a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.294 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745d587700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7458142590 0x7f745809f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.294 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74580b02e0 0x7f74580a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43356/0 (socket says 172.21.2.117:43356) 2021-11-21T21:54:12.294 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 -- 172.21.2.117:0/3265964602 learned_addr learned my addr 172.21.2.117:0/3265964602 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.294 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745d587700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7458142590 0x7f745809f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50264/0 (socket says 172.21.2.117:50264) 2021-11-21T21:54:12.295 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745cd86700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f745805f730 0x7f74580a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.295 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 -- 172.21.2.117:0/3265964602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7458142590 msgr2=0x7f745809f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.295 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7458142590 0x7f745809f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.295 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 -- 172.21.2.117:0/3265964602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f745805f730 msgr2=0x7f74580a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.296 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f745805f730 0x7f74580a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.296 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 -- 172.21.2.117:0/3265964602 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450009470 con 0x7f74580b02e0 2021-11-21T21:54:12.296 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745cd86700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f745805f730 0x7f74580a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:12.296 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7457fff700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74580b02e0 0x7f74580a0ec0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f7448000ed0 tx=0x7f7448007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.296 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7455ffb700 1 -- 172.21.2.117:0/3265964602 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74480070d0 con 0x7f74580b02e0 2021-11-21T21:54:12.297 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7455ffb700 1 -- 172.21.2.117:0/3265964602 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7448007230 con 0x7f74580b02e0 2021-11-21T21:54:12.297 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7455ffb700 1 -- 172.21.2.117:0/3265964602 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7448005610 con 0x7f74580b02e0 2021-11-21T21:54:12.297 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580a4ae0 con 0x7f74580b02e0 2021-11-21T21:54:12.297 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745809eee0 con 0x7f74580b02e0 2021-11-21T21:54:12.298 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7455ffb700 1 -- 172.21.2.117:0/3265964602 <== mon.2 v2:172.21.2.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f7448010070 con 0x7f74580b02e0 2021-11-21T21:54:12.298 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f7455ffb700 1 -- 172.21.2.117:0/3265964602 <== mon.2 v2:172.21.2.117:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f7448005e10 con 0x7f74580b02e0 2021-11-21T21:54:12.298 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.281+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74580a1550 con 0x7f74580b02e0 2021-11-21T21:54:12.298 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.285+0000 7f7455ffb700 1 -- 172.21.2.117:0/3265964602 <== mon.2 v2:172.21.2.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f7448013450 con 0x7f74580b02e0 2021-11-21T21:54:12.332 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.329+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", "id": 2} v 0) v1 -- 0x7f745814ff20 con 0x7f74580b02e0 2021-11-21T21:54:12.335 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f7455ffb700 1 -- 172.21.2.117:0/3265964602 <== mon.2 v2:172.21.2.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7448005020 con 0x7f74580b02e0 2021-11-21T21:54:12.336 INFO:teuthology.orchestra.run.gibba017.stdout:2 2021-11-21T21:54:12.337 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74580b02e0 msgr2=0x7f74580a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.337 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74580b02e0 0x7f74580a0ec0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f7448000ed0 tx=0x7f7448007ac0).stop 2021-11-21T21:54:12.337 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 shutdown_connections 2021-11-21T21:54:12.337 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7458142590 0x7f745809f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.338 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74580b02e0 0x7f74580a0ec0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.338 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 --2- 172.21.2.117:0/3265964602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f745805f730 0x7f74580a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.338 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 >> 172.21.2.117:0/3265964602 conn(0x7f74580b5ff0 msgr2=0x7f7458143840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.338 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 shutdown_connections 2021-11-21T21:54:12.339 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.333+0000 7f745efea700 1 -- 172.21.2.117:0/3265964602 wait complete. 2021-11-21T21:54:12.342 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new dcebb53a-56e7-4d2f-9ac5-a61776613ba6 3 2021-11-21T21:54:12.395 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.389+0000 7fa207b63700 1 Processor -- start 2021-11-21T21:54:12.395 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.389+0000 7fa207b63700 1 -- start start 2021-11-21T21:54:12.395 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.389+0000 7fa207b63700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa200061180 0x7fa200099a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.396 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.389+0000 7fa207b63700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa200060860 con 0x7fa20014abb0 2021-11-21T21:54:12.396 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.389+0000 7fa207b63700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20005d450 con 0x7fa20005d070 2021-11-21T21:54:12.396 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.389+0000 7fa207b63700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa200098f60 con 0x7fa200061180 2021-11-21T21:54:12.402 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2058ff700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fa20005d070 0x7fa200099380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.117:49700/0 (socket says 172.21.2.117:49700) 2021-11-21T21:54:12.402 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2058ff700 1 -- 172.21.2.117:0/3099851159 learned_addr learned my addr 172.21.2.117:0/3099851159 (peer_addr_for_me v1:172.21.2.117:0/0) 2021-11-21T21:54:12.402 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 --2- 172.21.2.117:0/3099851159 >> v2:172.21.2.117:3300/0 conn(0x7fa200061180 0x7fa200099a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.407 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2048fd700 1 -- 172.21.2.117:0/3099851159 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104133928 0 0) 0x7fa20005d450 con 0x7fa20005d070 2021-11-21T21:54:12.407 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2048fd700 1 -- 172.21.2.117:0/3099851159 --> v1:172.21.2.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e8003260 con 0x7fa20005d070 2021-11-21T21:54:12.407 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 -- 172.21.2.117:0/3099851159 >> v1:172.21.2.117:6789/0 conn(0x7fa20005d070 legacy=0x7fa200099380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.408 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 -- 172.21.2.117:0/3099851159 >> v1:172.21.2.116:6789/0 conn(0x7fa20014abb0 legacy=0x7fa20014afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.408 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 -- 172.21.2.117:0/3099851159 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000b4bc0 con 0x7fa200061180 2021-11-21T21:54:12.408 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 --2- 172.21.2.117:0/3099851159 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200099a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa200068920 tx=0x7fa1f0006df0).ready entity=mon.1 client_cookie=4fe70abed34a30c5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.408 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2048fd700 1 -- 172.21.2.117:0/3099851159 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1f000d5c0 con 0x7fa200061180 2021-11-21T21:54:12.409 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2048fd700 1 -- 172.21.2.117:0/3099851159 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0000e90 con 0x7fa200061180 2021-11-21T21:54:12.409 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2048fd700 1 -- 172.21.2.117:0/3099851159 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1f000d910 con 0x7fa200061180 2021-11-21T21:54:12.409 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3099851159 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 msgr2=0x7fa200099a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.410 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 --2- 172.21.2.117:0/3099851159 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200099a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa200068920 tx=0x7fa1f0006df0).stop 2021-11-21T21:54:12.410 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3099851159 shutdown_connections 2021-11-21T21:54:12.410 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 --2- 172.21.2.117:0/3099851159 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200099a90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.410 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3099851159 >> 172.21.2.117:0/3099851159 conn(0x7fa2000b5de0 msgr2=0x7fa2000b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.410 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3099851159 shutdown_connections 2021-11-21T21:54:12.411 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3099851159 wait complete. 2021-11-21T21:54:12.411 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 Processor -- start 2021-11-21T21:54:12.411 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- start start 2021-11-21T21:54:12.411 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa20005d070 0x7fa2000afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.412 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.412 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa20014abb0 0x7fa200156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.412 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20014e280 con 0x7fa20014abb0 2021-11-21T21:54:12.412 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20014e500 con 0x7fa200061180 2021-11-21T21:54:12.413 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20014e7a0 con 0x7fa20005d070 2021-11-21T21:54:12.413 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.413 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50272/0 (socket says 172.21.2.117:50272) 2021-11-21T21:54:12.413 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 -- 172.21.2.117:0/3094918282 learned_addr learned my addr 172.21.2.117:0/3094918282 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.414 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2058ff700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa20005d070 0x7fa2000afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.414 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa206100700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa20014abb0 0x7fa200156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.414 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 -- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa20005d070 msgr2=0x7fa2000afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.415 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa20005d070 0x7fa2000afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.415 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 -- 172.21.2.117:0/3094918282 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa20014abb0 msgr2=0x7fa200156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.415 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa20014abb0 0x7fa200156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.415 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 -- 172.21.2.117:0/3094918282 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f000b180 con 0x7fa200061180 2021-11-21T21:54:12.415 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa206100700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa20014abb0 0x7fa200156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:12.416 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa2050fe700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200156790 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa200068920 tx=0x7fa1f0014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.416 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa1eeffd700 1 -- 172.21.2.117:0/3094918282 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1f0013e90 con 0x7fa200061180 2021-11-21T21:54:12.416 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa1eeffd700 1 -- 172.21.2.117:0/3094918282 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0002f70 con 0x7fa200061180 2021-11-21T21:54:12.416 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20014ea40 con 0x7fa200061180 2021-11-21T21:54:12.417 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa1eeffd700 1 -- 172.21.2.117:0/3094918282 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1f001e5f0 con 0x7fa200061180 2021-11-21T21:54:12.417 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2000acb80 con 0x7fa200061180 2021-11-21T21:54:12.417 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa1eeffd700 1 -- 172.21.2.117:0/3094918282 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fa1f0025030 con 0x7fa200061180 2021-11-21T21:54:12.417 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa1eeffd700 1 -- 172.21.2.117:0/3094918282 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7fa1f0011880 con 0x7fa200061180 2021-11-21T21:54:12.418 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.393+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2000573d0 con 0x7fa200061180 2021-11-21T21:54:12.418 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.397+0000 7fa1eeffd700 1 -- 172.21.2.117:0/3094918282 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fa1f0023070 con 0x7fa200061180 2021-11-21T21:54:12.444 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.441+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", "id": 3} v 0) v1 -- 0x7fa200067b60 con 0x7fa200061180 2021-11-21T21:54:12.448 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa1eeffd700 1 -- 172.21.2.117:0/3094918282 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa1f0019030 con 0x7fa200061180 2021-11-21T21:54:12.449 INFO:teuthology.orchestra.run.gibba017.stdout:3 2021-11-21T21:54:12.450 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 msgr2=0x7fa200156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.450 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200156790 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa200068920 tx=0x7fa1f0014000).stop 2021-11-21T21:54:12.451 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 shutdown_connections 2021-11-21T21:54:12.451 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa20014abb0 0x7fa200156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.451 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa200061180 0x7fa200156790 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.451 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 --2- 172.21.2.117:0/3094918282 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa20005d070 0x7fa2000afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.451 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 >> 172.21.2.117:0/3094918282 conn(0x7fa2000b5de0 msgr2=0x7fa2000b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.452 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 shutdown_connections 2021-11-21T21:54:12.452 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.445+0000 7fa207b63700 1 -- 172.21.2.117:0/3094918282 wait complete. 2021-11-21T21:54:12.456 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new a7251112-9008-418e-a7b6-4501e7b9365d 4 2021-11-21T21:54:12.509 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 Processor -- start 2021-11-21T21:54:12.510 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- start start 2021-11-21T21:54:12.510 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f316005ced0 0x7f31600acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.510 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3160061060 0x7f316009a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.510 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316009b180 con 0x7f316009a7f0 2021-11-21T21:54:12.511 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316009b3e0 con 0x7f316005ced0 2021-11-21T21:54:12.511 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316009b680 con 0x7f3160061060 2021-11-21T21:54:12.511 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f31657eb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f316005ced0 0x7f31600acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.511 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3164fea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3160061060 0x7f316009a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.512 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f31657eb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f316005ced0 0x7f31600acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50280/0 (socket says 172.21.2.117:50280) 2021-11-21T21:54:12.512 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3164fea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3160061060 0x7f316009a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43372/0 (socket says 172.21.2.117:43372) 2021-11-21T21:54:12.512 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f31657eb700 1 -- 172.21.2.117:0/2196103102 learned_addr learned my addr 172.21.2.117:0/2196103102 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.512 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3164fea700 1 -- 172.21.2.117:0/2196103102 >> v2:172.21.2.117:3300/0 conn(0x7f316005ced0 msgr2=0x7f31600acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.512 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3164fea700 1 --2- 172.21.2.117:0/2196103102 >> v2:172.21.2.117:3300/0 conn(0x7f316005ced0 0x7f31600acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.513 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3164fea700 1 -- 172.21.2.117:0/2196103102 >> v1:172.21.2.116:6789/0 conn(0x7f316009a7f0 legacy=0x7f3160146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.513 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3164fea700 1 -- 172.21.2.117:0/2196103102 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316009b920 con 0x7f3160061060 2021-11-21T21:54:12.513 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3164fea700 1 --2- 172.21.2.117:0/2196103102 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3160061060 0x7f316009a300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f31600991e0 tx=0x7f3154006df0).ready entity=mon.2 client_cookie=ed5d394c51487b3a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.513 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3153fff700 1 -- 172.21.2.117:0/2196103102 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f315400d570 con 0x7f3160061060 2021-11-21T21:54:12.514 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3153fff700 1 -- 172.21.2.117:0/2196103102 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3154000e90 con 0x7f3160061060 2021-11-21T21:54:12.514 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3153fff700 1 -- 172.21.2.117:0/2196103102 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f315400d8c0 con 0x7f3160061060 2021-11-21T21:54:12.516 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- 172.21.2.117:0/2196103102 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3160061060 msgr2=0x7f316009a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.517 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 --2- 172.21.2.117:0/2196103102 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3160061060 0x7f316009a300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f31600991e0 tx=0x7f3154006df0).stop 2021-11-21T21:54:12.517 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- 172.21.2.117:0/2196103102 shutdown_connections 2021-11-21T21:54:12.517 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 --2- 172.21.2.117:0/2196103102 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3160061060 0x7f316009a300 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.518 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 --2- 172.21.2.117:0/2196103102 >> v2:172.21.2.117:3300/0 conn(0x7f316005ced0 0x7f31600acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.518 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- 172.21.2.117:0/2196103102 >> 172.21.2.117:0/2196103102 conn(0x7f31600b5de0 msgr2=0x7f31600b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.518 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- 172.21.2.117:0/2196103102 shutdown_connections 2021-11-21T21:54:12.518 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 -- 172.21.2.117:0/2196103102 wait complete. 2021-11-21T21:54:12.518 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.505+0000 7f3167a4f700 1 Processor -- start 2021-11-21T21:54:12.519 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 -- start start 2021-11-21T21:54:12.519 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f316005ced0 0x7f316009e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.519 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3160061060 0x7f3160152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.519 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f316009a7f0 0x7f3160152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.520 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600b0630 con 0x7f316005ced0 2021-11-21T21:54:12.520 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600b12c0 con 0x7f3160061060 2021-11-21T21:54:12.520 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31600b0860 con 0x7f316009a7f0 2021-11-21T21:54:12.520 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3165fec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f316009a7f0 0x7f3160152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.520 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3160061060 0x7f3160152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.521 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3165fec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f316009a7f0 0x7f3160152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43378/0 (socket says 172.21.2.117:43378) 2021-11-21T21:54:12.521 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3165fec700 1 -- 172.21.2.117:0/1392779251 learned_addr learned my addr 172.21.2.117:0/1392779251 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.521 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3160061060 0x7f3160152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50288/0 (socket says 172.21.2.117:50288) 2021-11-21T21:54:12.521 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f31657eb700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f316005ced0 0x7f316009e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.521 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 -- 172.21.2.117:0/1392779251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f316009a7f0 msgr2=0x7f3160152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.522 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f316009a7f0 0x7f3160152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.522 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 -- 172.21.2.117:0/1392779251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f316005ced0 msgr2=0x7f316009e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.522 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f316005ced0 0x7f316009e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.522 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 -- 172.21.2.117:0/1392779251 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315400b180 con 0x7f3160061060 2021-11-21T21:54:12.523 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3164fea700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3160061060 0x7f3160152310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f31600991e0 tx=0x7f3154014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.523 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f31527fc700 1 -- 172.21.2.117:0/1392779251 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3154016070 con 0x7f3160061060 2021-11-21T21:54:12.523 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f31527fc700 1 -- 172.21.2.117:0/1392779251 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3154002f70 con 0x7f3160061060 2021-11-21T21:54:12.523 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31600b1020 con 0x7f3160061060 2021-11-21T21:54:12.523 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f31527fc700 1 -- 172.21.2.117:0/1392779251 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3154013e90 con 0x7f3160061060 2021-11-21T21:54:12.524 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31600b4640 con 0x7f3160061060 2021-11-21T21:54:12.524 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f31527fc700 1 -- 172.21.2.117:0/1392779251 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f3154025030 con 0x7f3160061060 2021-11-21T21:54:12.524 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f31527fc700 1 -- 172.21.2.117:0/1392779251 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f315400de40 con 0x7f3160061060 2021-11-21T21:54:12.524 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31600573d0 con 0x7f3160061060 2021-11-21T21:54:12.525 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.509+0000 7f31527fc700 1 -- 172.21.2.117:0/1392779251 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f3154023070 con 0x7f3160061060 2021-11-21T21:54:12.559 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.557+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", "id": 4} v 0) v1 -- 0x7f3160151fd0 con 0x7f3160061060 2021-11-21T21:54:12.563 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.557+0000 7f31527fc700 1 -- 172.21.2.117:0/1392779251 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f315401ea30 con 0x7f3160061060 2021-11-21T21:54:12.563 INFO:teuthology.orchestra.run.gibba017.stdout:4 2021-11-21T21:54:12.564 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3160061060 msgr2=0x7f3160152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.564 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3160061060 0x7f3160152310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f31600991e0 tx=0x7f3154014000).stop 2021-11-21T21:54:12.565 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 shutdown_connections 2021-11-21T21:54:12.565 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f316009a7f0 0x7f3160152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.565 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3160061060 0x7f3160152310 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.565 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 --2- 172.21.2.117:0/1392779251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f316005ced0 0x7f316009e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.566 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 >> 172.21.2.117:0/1392779251 conn(0x7f31600b5de0 msgr2=0x7f316005c470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.566 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 shutdown_connections 2021-11-21T21:54:12.566 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.561+0000 7f3167a4f700 1 -- 172.21.2.117:0/1392779251 wait complete. 2021-11-21T21:54:12.570 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new 0c8cf701-2ace-4542-a985-c4435cd023fb 5 2021-11-21T21:54:12.621 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 Processor -- start 2021-11-21T21:54:12.622 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- start start 2021-11-21T21:54:12.622 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f4858142ac0 0x7f48580adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.622 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48580ae070 con 0x7f4858062e60 2021-11-21T21:54:12.622 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48580afae0 con 0x7f48581426e0 2021-11-21T21:54:12.623 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48580b0f60 con 0x7f4858142ac0 2021-11-21T21:54:12.623 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485e3e9700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f48581426e0 0x7f48580af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.117:49724/0 (socket says 172.21.2.117:49724) 2021-11-21T21:54:12.623 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485e3e9700 1 -- 172.21.2.117:0/623153602 learned_addr learned my addr 172.21.2.117:0/623153602 (peer_addr_for_me v1:172.21.2.117:0/0) 2021-11-21T21:54:12.623 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485ebea700 1 --2- 172.21.2.117:0/623153602 >> v2:172.21.2.117:3300/0 conn(0x7f4858142ac0 0x7f48580adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.627 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485d3e7700 1 -- 172.21.2.117:0/623153602 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671658206 0 0) 0x7f48580afae0 con 0x7f48581426e0 2021-11-21T21:54:12.628 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485d3e7700 1 -- 172.21.2.117:0/623153602 --> v1:172.21.2.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4840003260 con 0x7f48581426e0 2021-11-21T21:54:12.628 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485ebea700 1 -- 172.21.2.117:0/623153602 >> v1:172.21.2.117:6789/0 conn(0x7f48581426e0 legacy=0x7f48580af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.628 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485ebea700 1 -- 172.21.2.117:0/623153602 >> v1:172.21.2.116:6789/0 conn(0x7f4858062e60 legacy=0x7f48580ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.628 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485ebea700 1 -- 172.21.2.117:0/623153602 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858063470 con 0x7f4858142ac0 2021-11-21T21:54:12.629 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485ebea700 1 --2- 172.21.2.117:0/623153602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4858142ac0 0x7f48580adb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f48500032f0 tx=0x7f485000a760).ready entity=mon.1 client_cookie=ebdddee3b7349c56 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.629 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485d3e7700 1 -- 172.21.2.117:0/623153602 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4850005e70 con 0x7f4858142ac0 2021-11-21T21:54:12.629 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485d3e7700 1 -- 172.21.2.117:0/623153602 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485000c990 con 0x7f4858142ac0 2021-11-21T21:54:12.629 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f485d3e7700 1 -- 172.21.2.117:0/623153602 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f485000ccf0 con 0x7f4858142ac0 2021-11-21T21:54:12.629 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- 172.21.2.117:0/623153602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4858142ac0 msgr2=0x7f48580adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.630 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 --2- 172.21.2.117:0/623153602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4858142ac0 0x7f48580adb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f48500032f0 tx=0x7f485000a760).stop 2021-11-21T21:54:12.630 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- 172.21.2.117:0/623153602 shutdown_connections 2021-11-21T21:54:12.630 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 --2- 172.21.2.117:0/623153602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4858142ac0 0x7f48580adb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.630 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- 172.21.2.117:0/623153602 >> 172.21.2.117:0/623153602 conn(0x7f48580b5730 msgr2=0x7f48580b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.631 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- 172.21.2.117:0/623153602 shutdown_connections 2021-11-21T21:54:12.631 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.617+0000 7f486064d700 1 -- 172.21.2.117:0/623153602 wait complete. 2021-11-21T21:54:12.631 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 Processor -- start 2021-11-21T21:54:12.631 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 -- start start 2021-11-21T21:54:12.632 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48581426e0 0x7f48581238c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.632 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4858142ac0 0x7f4858122620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.632 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4858122b10 0x7f485811fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.632 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485813f800 con 0x7f4858142ac0 2021-11-21T21:54:12.632 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4858122090 con 0x7f48581426e0 2021-11-21T21:54:12.633 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485813fa30 con 0x7f4858122b10 2021-11-21T21:54:12.633 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48581426e0 0x7f48581238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.633 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48581426e0 0x7f48581238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50298/0 (socket says 172.21.2.117:50298) 2021-11-21T21:54:12.633 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485ebea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4858122b10 0x7f485811fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.633 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 -- 172.21.2.117:0/1241929475 learned_addr learned my addr 172.21.2.117:0/1241929475 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.634 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485dbe8700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4858142ac0 0x7f4858122620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.634 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 -- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4858122b10 msgr2=0x7f485811fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.634 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4858122b10 0x7f485811fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.634 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 -- 172.21.2.117:0/1241929475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4858142ac0 msgr2=0x7f4858122620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.635 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4858142ac0 0x7f4858122620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.635 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 -- 172.21.2.117:0/1241929475 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485000aa00 con 0x7f48581426e0 2021-11-21T21:54:12.635 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485ebea700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4858122b10 0x7f485811fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:12.635 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485dbe8700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4858142ac0 0x7f4858122620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:54:12.636 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f485e3e9700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48581426e0 0x7f48581238c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f484c001110 tx=0x7f484c007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.636 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f484b7fe700 1 -- 172.21.2.117:0/1241929475 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484c00ce20 con 0x7f48581426e0 2021-11-21T21:54:12.636 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f484b7fe700 1 -- 172.21.2.117:0/1241929475 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c0072e0 con 0x7f48581426e0 2021-11-21T21:54:12.636 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485813fed0 con 0x7f48581426e0 2021-11-21T21:54:12.636 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f484b7fe700 1 -- 172.21.2.117:0/1241929475 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484c005800 con 0x7f48581426e0 2021-11-21T21:54:12.637 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4858155280 con 0x7f48581426e0 2021-11-21T21:54:12.637 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f484b7fe700 1 -- 172.21.2.117:0/1241929475 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f484c010070 con 0x7f48581426e0 2021-11-21T21:54:12.637 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f484b7fe700 1 -- 172.21.2.117:0/1241929475 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f484c005be0 con 0x7f48581426e0 2021-11-21T21:54:12.637 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485813fbf0 con 0x7f48581426e0 2021-11-21T21:54:12.638 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.621+0000 7f484b7fe700 1 -- 172.21.2.117:0/1241929475 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f484c031870 con 0x7f48581426e0 2021-11-21T21:54:12.671 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.669+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", "id": 5} v 0) v1 -- 0x7f4858143050 con 0x7f48581426e0 2021-11-21T21:54:12.675 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f484b7fe700 1 -- 172.21.2.117:0/1241929475 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f484c021070 con 0x7f48581426e0 2021-11-21T21:54:12.675 INFO:teuthology.orchestra.run.gibba017.stdout:5 2021-11-21T21:54:12.676 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48581426e0 msgr2=0x7f48581238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.677 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48581426e0 0x7f48581238c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f484c001110 tx=0x7f484c007ad0).stop 2021-11-21T21:54:12.677 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 shutdown_connections 2021-11-21T21:54:12.677 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4858122b10 0x7f485811fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.677 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4858142ac0 0x7f4858122620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.677 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 --2- 172.21.2.117:0/1241929475 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48581426e0 0x7f48581238c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.678 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 >> 172.21.2.117:0/1241929475 conn(0x7f48580b5730 msgr2=0x7f4858069680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.678 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 shutdown_connections 2021-11-21T21:54:12.678 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.673+0000 7f486064d700 1 -- 172.21.2.117:0/1241929475 wait complete. 2021-11-21T21:54:12.682 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new 66ee78f9-53ac-45a6-a441-8a71e4e06bac 6 2021-11-21T21:54:12.735 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6aa13c1700 1 Processor -- start 2021-11-21T21:54:12.736 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6aa13c1700 1 -- start start 2021-11-21T21:54:12.736 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6aa13c1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0adc10 0x7f6a9c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.736 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6aa13c1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6a9c0b3690 0x7f6a9c13a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.737 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6aa13c1700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c0b3c50 con 0x7f6a9c144a10 2021-11-21T21:54:12.737 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6aa13c1700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9c0b3ef0 con 0x7f6a9c0b3690 2021-11-21T21:54:12.737 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6aa13c1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0b4190 con 0x7f6a9c0adc10 2021-11-21T21:54:12.737 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0adc10 0x7f6a9c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.737 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0adc10 0x7f6a9c0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43396/0 (socket says 172.21.2.117:43396) 2021-11-21T21:54:12.738 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 -- 172.21.2.117:0/4092041754 learned_addr learned my addr 172.21.2.117:0/4092041754 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.738 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9b7fe700 1 --2- 172.21.2.117:0/4092041754 >> v2:172.21.2.116:3300/0 conn(0x7f6a9c0b3690 0x7f6a9c13a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.738 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 -- 172.21.2.117:0/4092041754 >> v2:172.21.2.116:3300/0 conn(0x7f6a9c0b3690 msgr2=0x7f6a9c13a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.738 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 --2- 172.21.2.117:0/4092041754 >> v2:172.21.2.116:3300/0 conn(0x7f6a9c0b3690 0x7f6a9c13a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.739 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 -- 172.21.2.117:0/4092041754 >> v1:172.21.2.116:6789/0 conn(0x7f6a9c144a10 legacy=0x7f6a9c141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.739 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 -- 172.21.2.117:0/4092041754 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c140b80 con 0x7f6a9c0adc10 2021-11-21T21:54:12.739 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.729+0000 7f6a9affd700 1 --2- 172.21.2.117:0/4092041754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0adc10 0x7f6a9c0b2030 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f6a9c0b7000 tx=0x7f6a84006eb0).ready entity=mon.2 client_cookie=977f62f37a779bd4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.739 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a99ffb700 1 -- 172.21.2.117:0/4092041754 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8400d570 con 0x7f6a9c0adc10 2021-11-21T21:54:12.740 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a99ffb700 1 -- 172.21.2.117:0/4092041754 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a84000e90 con 0x7f6a9c0adc10 2021-11-21T21:54:12.740 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a99ffb700 1 -- 172.21.2.117:0/4092041754 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8400d8c0 con 0x7f6a9c0adc10 2021-11-21T21:54:12.740 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/4092041754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0adc10 msgr2=0x7f6a9c0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.740 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 --2- 172.21.2.117:0/4092041754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0adc10 0x7f6a9c0b2030 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f6a9c0b7000 tx=0x7f6a84006eb0).stop 2021-11-21T21:54:12.740 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/4092041754 shutdown_connections 2021-11-21T21:54:12.741 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 --2- 172.21.2.117:0/4092041754 >> v2:172.21.2.116:3300/0 conn(0x7f6a9c0b3690 0x7f6a9c13a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.741 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 --2- 172.21.2.117:0/4092041754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0adc10 0x7f6a9c0b2030 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.741 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/4092041754 >> 172.21.2.117:0/4092041754 conn(0x7f6a9c05bd20 msgr2=0x7f6a9c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.741 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/4092041754 shutdown_connections 2021-11-21T21:54:12.742 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/4092041754 wait complete. 2021-11-21T21:54:12.744 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 Processor -- start 2021-11-21T21:54:12.744 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- start start 2021-11-21T21:54:12.744 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a9c0adc10 0x7f6a9c0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.745 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0b3690 0x7f6a9c0ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.745 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a9c144a10 0x7f6a9c110d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.745 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c153720 con 0x7f6a9c0adc10 2021-11-21T21:54:12.745 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0af3b0 con 0x7f6a9c144a10 2021-11-21T21:54:12.745 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0670b0 con 0x7f6a9c0b3690 2021-11-21T21:54:12.746 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0b3690 0x7f6a9c0ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.746 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0b3690 0x7f6a9c0ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43404/0 (socket says 172.21.2.117:43404) 2021-11-21T21:54:12.746 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 -- 172.21.2.117:0/1931522022 learned_addr learned my addr 172.21.2.117:0/1931522022 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.746 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9affd700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a9c0adc10 0x7f6a9c0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.747 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9b7fe700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a9c144a10 0x7f6a9c110d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.747 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 -- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a9c144a10 msgr2=0x7f6a9c110d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.747 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a9c144a10 0x7f6a9c110d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.747 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 -- 172.21.2.117:0/1931522022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a9c0adc10 msgr2=0x7f6a9c0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.748 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a9c0adc10 0x7f6a9c0afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.748 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 -- 172.21.2.117:0/1931522022 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8400b1e0 con 0x7f6a9c0b3690 2021-11-21T21:54:12.748 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a9a7fc700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0b3690 0x7f6a9c0ae7a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f6a9c147b00 tx=0x7f6a8c006d70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.748 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a93fff700 1 -- 172.21.2.117:0/1931522022 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8c012410 con 0x7f6a9c0b3690 2021-11-21T21:54:12.748 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a93fff700 1 -- 172.21.2.117:0/1931522022 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c012570 con 0x7f6a9c0b3690 2021-11-21T21:54:12.749 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a93fff700 1 -- 172.21.2.117:0/1931522022 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8c0128d0 con 0x7f6a9c0b3690 2021-11-21T21:54:12.749 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c067350 con 0x7f6a9c0b3690 2021-11-21T21:54:12.749 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c0aef80 con 0x7f6a9c0b3690 2021-11-21T21:54:12.749 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a93fff700 1 -- 172.21.2.117:0/1931522022 <== mon.2 v2:172.21.2.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f6a8c00f030 con 0x7f6a9c0b3690 2021-11-21T21:54:12.749 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6a93fff700 1 -- 172.21.2.117:0/1931522022 <== mon.2 v2:172.21.2.117:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f6a8c01ba00 con 0x7f6a9c0b3690 2021-11-21T21:54:12.750 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.733+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c0af5b0 con 0x7f6a9c0b3690 2021-11-21T21:54:12.750 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.737+0000 7f6a93fff700 1 -- 172.21.2.117:0/1931522022 <== mon.2 v2:172.21.2.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f6a8c01bc10 con 0x7f6a9c0b3690 2021-11-21T21:54:12.784 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.781+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", "id": 6} v 0) v1 -- 0x7f6a9c0674e0 con 0x7f6a9c0b3690 2021-11-21T21:54:12.788 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6a93fff700 1 -- 172.21.2.117:0/1931522022 <== mon.2 v2:172.21.2.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6a8c01bdb0 con 0x7f6a9c0b3690 2021-11-21T21:54:12.789 INFO:teuthology.orchestra.run.gibba017.stdout:6 2021-11-21T21:54:12.790 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0b3690 msgr2=0x7f6a9c0ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.790 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0b3690 0x7f6a9c0ae7a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f6a9c147b00 tx=0x7f6a8c006d70).stop 2021-11-21T21:54:12.790 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 shutdown_connections 2021-11-21T21:54:12.790 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a9c144a10 0x7f6a9c110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.791 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a9c0b3690 0x7f6a9c0ae7a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.791 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 --2- 172.21.2.117:0/1931522022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a9c0adc10 0x7f6a9c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.791 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 >> 172.21.2.117:0/1931522022 conn(0x7f6a9c05bd20 msgr2=0x7f6a9c0ac9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.791 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 shutdown_connections 2021-11-21T21:54:12.792 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.785+0000 7f6aa13c1700 1 -- 172.21.2.117:0/1931522022 wait complete. 2021-11-21T21:54:12.795 DEBUG:teuthology.orchestra.run.gibba017:> sudo ceph --cluster ceph osd new 7c1bf6d9-c24a-4373-9eee-5488fd33e157 7 2021-11-21T21:54:12.847 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.841+0000 7f32ba0b4700 1 Processor -- start 2021-11-21T21:54:12.847 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.841+0000 7f32ba0b4700 1 -- start start 2021-11-21T21:54:12.852 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b405ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.853 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f32b40b5620 0x7f32b40b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.853 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f32b40b6020 0x7f32b412ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.853 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b4063000 con 0x7f32b40b6020 2021-11-21T21:54:12.853 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b40b64b0 con 0x7f32b40b5620 2021-11-21T21:54:12.854 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4142dd0 con 0x7f32b4062c20 2021-11-21T21:54:12.854 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b2ffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f32b40b5620 0x7f32b40b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.854 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b2ffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f32b40b5620 0x7f32b40b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.117:50316/0 (socket says 172.21.2.117:50316) 2021-11-21T21:54:12.854 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b2ffd700 1 -- 172.21.2.117:0/3210456182 learned_addr learned my addr 172.21.2.117:0/3210456182 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.855 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- 172.21.2.117:0/3210456182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b405ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.855 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b3fff700 1 --2- 172.21.2.117:0/3210456182 >> v2:172.21.2.116:3300/0 conn(0x7f32b40b6020 0x7f32b412ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.855 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 -- 172.21.2.117:0/3210456182 >> v2:172.21.2.117:3300/0 conn(0x7f32b40b5620 msgr2=0x7f32b40b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.855 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- 172.21.2.117:0/3210456182 >> v2:172.21.2.117:3300/0 conn(0x7f32b40b5620 0x7f32b40b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.856 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 -- 172.21.2.117:0/3210456182 >> v2:172.21.2.116:3300/0 conn(0x7f32b40b6020 msgr2=0x7f32b412ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:12.856 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- 172.21.2.117:0/3210456182 >> v2:172.21.2.116:3300/0 conn(0x7f32b40b6020 0x7f32b412ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.856 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 -- 172.21.2.117:0/3210456182 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4143070 con 0x7f32b4062c20 2021-11-21T21:54:12.856 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b3fff700 1 --2- 172.21.2.117:0/3210456182 >> v2:172.21.2.116:3300/0 conn(0x7f32b40b6020 0x7f32b412ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:12.856 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- 172.21.2.117:0/3210456182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b405ce60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f329c002f70 tx=0x7f329c006df0).ready entity=mon.2 client_cookie=cafade0f0c293e08 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.857 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b27fc700 1 -- 172.21.2.117:0/3210456182 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329c00d570 con 0x7f32b4062c20 2021-11-21T21:54:12.857 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b27fc700 1 -- 172.21.2.117:0/3210456182 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c000e90 con 0x7f32b4062c20 2021-11-21T21:54:12.857 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b27fc700 1 -- 172.21.2.117:0/3210456182 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329c00d8c0 con 0x7f32b4062c20 2021-11-21T21:54:12.858 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- 172.21.2.117:0/3210456182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 msgr2=0x7f32b405ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.858 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/3210456182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b405ce60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f329c002f70 tx=0x7f329c006df0).stop 2021-11-21T21:54:12.858 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- 172.21.2.117:0/3210456182 shutdown_connections 2021-11-21T21:54:12.858 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/3210456182 >> v2:172.21.2.116:3300/0 conn(0x7f32b40b6020 0x7f32b412ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.859 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/3210456182 >> v2:172.21.2.117:3300/0 conn(0x7f32b40b5620 0x7f32b40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.859 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/3210456182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b405ce60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.859 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- 172.21.2.117:0/3210456182 >> 172.21.2.117:0/3210456182 conn(0x7f32b40ac9b0 msgr2=0x7f32b4069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.859 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- 172.21.2.117:0/3210456182 shutdown_connections 2021-11-21T21:54:12.860 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- 172.21.2.117:0/3210456182 wait complete. 2021-11-21T21:54:12.860 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 Processor -- start 2021-11-21T21:54:12.860 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- start start 2021-11-21T21:54:12.860 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b40b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.860 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f32b40b5620 0x7f32b40af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.861 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f32b40b6020 0x7f32b40afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:12.861 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4143d60 con 0x7f32b40b5620 2021-11-21T21:54:12.861 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40b0230 con 0x7f32b40b6020 2021-11-21T21:54:12.861 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40aeb00 con 0x7f32b4062c20 2021-11-21T21:54:12.861 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b40b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.862 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b3fff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f32b40b6020 0x7f32b40afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.862 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b40b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.117:43414/0 (socket says 172.21.2.117:43414) 2021-11-21T21:54:12.862 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 -- 172.21.2.117:0/1987392611 learned_addr learned my addr 172.21.2.117:0/1987392611 (peer_addr_for_me v2:172.21.2.117:0/0) 2021-11-21T21:54:12.862 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b2ffd700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f32b40b5620 0x7f32b40af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:12.863 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 -- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f32b40b6020 msgr2=0x7f32b40afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.863 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f32b40b6020 0x7f32b40afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.863 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 -- 172.21.2.117:0/1987392611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f32b40b5620 msgr2=0x7f32b40af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.863 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f32b40b5620 0x7f32b40af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.864 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 -- 172.21.2.117:0/1987392611 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c00b180 con 0x7f32b4062c20 2021-11-21T21:54:12.864 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b3fff700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f32b40b6020 0x7f32b40afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:12.864 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b37fe700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b40b4570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f329c002f70 tx=0x7f329c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:12.864 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b0ff9700 1 -- 172.21.2.117:0/1987392611 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329c00dea0 con 0x7f32b4062c20 2021-11-21T21:54:12.864 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b0ff9700 1 -- 172.21.2.117:0/1987392611 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c013e90 con 0x7f32b4062c20 2021-11-21T21:54:12.865 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b4145f50 con 0x7f32b4062c20 2021-11-21T21:54:12.865 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b0ff9700 1 -- 172.21.2.117:0/1987392611 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329c01e5f0 con 0x7f32b4062c20 2021-11-21T21:54:12.865 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b4151d50 con 0x7f32b4062c20 2021-11-21T21:54:12.865 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b0ff9700 1 -- 172.21.2.117:0/1987392611 <== mon.2 v2:172.21.2.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f329c025030 con 0x7f32b4062c20 2021-11-21T21:54:12.866 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.845+0000 7f32b0ff9700 1 -- 172.21.2.117:0/1987392611 <== mon.2 v2:172.21.2.117:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f329c011a50 con 0x7f32b4062c20 2021-11-21T21:54:12.866 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.849+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b40af6c0 con 0x7f32b4062c20 2021-11-21T21:54:12.866 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.849+0000 7f32b0ff9700 1 -- 172.21.2.117:0/1987392611 <== mon.2 v2:172.21.2.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f329c023070 con 0x7f32b4062c20 2021-11-21T21:54:12.897 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.893+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", "id": 7} v 0) v1 -- 0x7f32b4144030 con 0x7f32b4062c20 2021-11-21T21:54:12.901 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32b0ff9700 1 -- 172.21.2.117:0/1987392611 <== mon.2 v2:172.21.2.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f329c019030 con 0x7f32b4062c20 2021-11-21T21:54:12.901 INFO:teuthology.orchestra.run.gibba017.stdout:7 2021-11-21T21:54:12.902 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 msgr2=0x7f32b40b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:12.903 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b40b4570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f329c002f70 tx=0x7f329c014000).stop 2021-11-21T21:54:12.903 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 shutdown_connections 2021-11-21T21:54:12.903 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f32b40b6020 0x7f32b40afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.903 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f32b40b5620 0x7f32b40af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.904 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 --2- 172.21.2.117:0/1987392611 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f32b4062c20 0x7f32b40b4570 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:12.904 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 >> 172.21.2.117:0/1987392611 conn(0x7f32b40ac9b0 msgr2=0x7f32b412bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:12.904 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 shutdown_connections 2021-11-21T21:54:12.904 INFO:teuthology.orchestra.run.gibba017.stderr:2021-11-21T21:54:12.897+0000 7f32ba0b4700 1 -- 172.21.2.117:0/1987392611 wait complete. 2021-11-21T21:54:12.908 INFO:tasks.ceph.osd.0:Restarting daemon 2021-11-21T21:54:12.908 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2021-11-21T21:54:12.911 INFO:tasks.ceph.osd.0:Started 2021-11-21T21:54:12.911 INFO:tasks.ceph.osd.1:Restarting daemon 2021-11-21T21:54:12.912 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2021-11-21T21:54:12.913 INFO:tasks.ceph.osd.1:Started 2021-11-21T21:54:12.914 INFO:tasks.ceph.osd.2:Restarting daemon 2021-11-21T21:54:12.914 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2021-11-21T21:54:12.916 INFO:tasks.ceph.osd.2:Started 2021-11-21T21:54:12.917 INFO:tasks.ceph.osd.3:Restarting daemon 2021-11-21T21:54:12.917 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2021-11-21T21:54:12.919 INFO:tasks.ceph.osd.3:Started 2021-11-21T21:54:12.919 INFO:tasks.ceph.osd.4:Restarting daemon 2021-11-21T21:54:12.919 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2021-11-21T21:54:12.922 INFO:tasks.ceph.osd.4:Started 2021-11-21T21:54:12.922 INFO:tasks.ceph.osd.5:Restarting daemon 2021-11-21T21:54:12.922 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2021-11-21T21:54:12.924 INFO:tasks.ceph.osd.5:Started 2021-11-21T21:54:12.924 INFO:tasks.ceph.osd.6:Restarting daemon 2021-11-21T21:54:12.925 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2021-11-21T21:54:12.926 INFO:tasks.ceph.osd.6:Started 2021-11-21T21:54:12.927 INFO:tasks.ceph.osd.7:Restarting daemon 2021-11-21T21:54:12.927 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2021-11-21T21:54:12.929 INFO:tasks.ceph.osd.7:Started 2021-11-21T21:54:12.929 INFO:tasks.ceph:Waiting for OSDs to come up 2021-11-21T21:54:12.945 INFO:tasks.ceph.osd.0.gibba016.stderr:2021-11-21T21:54:12.942+0000 7efd456ecd80 -1 Falling back to public interface 2021-11-21T21:54:12.948 INFO:tasks.ceph.osd.1.gibba016.stderr:2021-11-21T21:54:12.946+0000 7fec5ecfbd80 -1 Falling back to public interface 2021-11-21T21:54:12.950 INFO:tasks.ceph.osd.2.gibba016.stderr:2021-11-21T21:54:12.946+0000 7fbbaf1ded80 -1 Falling back to public interface 2021-11-21T21:54:12.953 INFO:tasks.ceph.osd.3.gibba016.stderr:2021-11-21T21:54:12.950+0000 7f7a0c1c8d80 -1 Falling back to public interface 2021-11-21T21:54:12.954 INFO:tasks.ceph.osd.0.gibba016.stderr:2021-11-21T21:54:12.950+0000 7efd456ecd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.954 INFO:tasks.ceph.osd.0.gibba016.stderr:2021-11-21T21:54:12.950+0000 7efd456ecd80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.954 INFO:tasks.ceph.osd.0.gibba016.stderr:2021-11-21T21:54:12.950+0000 7efd456ecd80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.955 INFO:tasks.ceph.osd.4.gibba017.stderr:2021-11-21T21:54:12.953+0000 7f5994b75d80 -1 Falling back to public interface 2021-11-21T21:54:12.956 INFO:tasks.ceph.osd.0.gibba016.stderr:2021-11-21T21:54:12.954+0000 7efd456ecd80 -1 osd.0 0 log_to_monitors {default=true} 2021-11-21T21:54:12.957 INFO:tasks.ceph.osd.1.gibba016.stderr:2021-11-21T21:54:12.954+0000 7fec5ecfbd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.958 INFO:tasks.ceph.osd.1.gibba016.stderr:2021-11-21T21:54:12.954+0000 7fec5ecfbd80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.958 INFO:tasks.ceph.osd.1.gibba016.stderr:2021-11-21T21:54:12.954+0000 7fec5ecfbd80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.958 INFO:tasks.ceph.osd.5.gibba017.stderr:2021-11-21T21:54:12.953+0000 7fa14bb18d80 -1 Falling back to public interface 2021-11-21T21:54:12.960 INFO:tasks.ceph.osd.1.gibba016.stderr:2021-11-21T21:54:12.958+0000 7fec5ecfbd80 -1 osd.1 0 log_to_monitors {default=true} 2021-11-21T21:54:12.961 INFO:tasks.ceph.osd.2.gibba016.stderr:2021-11-21T21:54:12.958+0000 7fbbaf1ded80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.961 INFO:tasks.ceph.osd.2.gibba016.stderr:2021-11-21T21:54:12.958+0000 7fbbaf1ded80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.962 INFO:tasks.ceph.osd.2.gibba016.stderr:2021-11-21T21:54:12.958+0000 7fbbaf1ded80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.962 INFO:tasks.ceph.osd.6.gibba017.stderr:2021-11-21T21:54:12.957+0000 7fe48f5b0d80 -1 Falling back to public interface 2021-11-21T21:54:12.963 INFO:tasks.ceph.osd.7.gibba017.stderr:2021-11-21T21:54:12.957+0000 7fe3fb9b9d80 -1 Falling back to public interface 2021-11-21T21:54:12.964 INFO:tasks.ceph.osd.3.gibba016.stderr:2021-11-21T21:54:12.962+0000 7f7a0c1c8d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.964 INFO:tasks.ceph.osd.3.gibba016.stderr:2021-11-21T21:54:12.962+0000 7f7a0c1c8d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.965 INFO:tasks.ceph.osd.3.gibba016.stderr:2021-11-21T21:54:12.962+0000 7f7a0c1c8d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.965 INFO:tasks.ceph.osd.4.gibba017.stderr:2021-11-21T21:54:12.961+0000 7f5994b75d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.965 INFO:tasks.ceph.osd.4.gibba017.stderr:2021-11-21T21:54:12.961+0000 7f5994b75d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.966 INFO:tasks.ceph.osd.4.gibba017.stderr:2021-11-21T21:54:12.961+0000 7f5994b75d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.966 INFO:tasks.ceph.osd.2.gibba016.stderr:2021-11-21T21:54:12.962+0000 7fbbaf1ded80 -1 osd.2 0 log_to_monitors {default=true} 2021-11-21T21:54:12.967 INFO:tasks.ceph.osd.5.gibba017.stderr:2021-11-21T21:54:12.961+0000 7fa14bb18d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.967 INFO:tasks.ceph.osd.5.gibba017.stderr:2021-11-21T21:54:12.961+0000 7fa14bb18d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.967 INFO:tasks.ceph.osd.5.gibba017.stderr:2021-11-21T21:54:12.961+0000 7fa14bb18d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.967 INFO:tasks.ceph.osd.4.gibba017.stderr:2021-11-21T21:54:12.961+0000 7f5994b75d80 -1 osd.4 0 log_to_monitors {default=true} 2021-11-21T21:54:12.968 INFO:tasks.ceph.osd.3.gibba016.stderr:2021-11-21T21:54:12.962+0000 7f7a0c1c8d80 -1 osd.3 0 log_to_monitors {default=true} 2021-11-21T21:54:12.969 INFO:tasks.ceph.osd.5.gibba017.stderr:2021-11-21T21:54:12.965+0000 7fa14bb18d80 -1 osd.5 0 log_to_monitors {default=true} 2021-11-21T21:54:12.970 INFO:tasks.ceph.osd.6.gibba017.stderr:2021-11-21T21:54:12.965+0000 7fe48f5b0d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.971 INFO:tasks.ceph.osd.6.gibba017.stderr:2021-11-21T21:54:12.965+0000 7fe48f5b0d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.971 INFO:tasks.ceph.osd.6.gibba017.stderr:2021-11-21T21:54:12.965+0000 7fe48f5b0d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.973 INFO:tasks.ceph.osd.6.gibba017.stderr:2021-11-21T21:54:12.969+0000 7fe48f5b0d80 -1 osd.6 0 log_to_monitors {default=true} 2021-11-21T21:54:12.976 INFO:tasks.ceph.osd.7.gibba017.stderr:2021-11-21T21:54:12.973+0000 7fe3fb9b9d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2021-11-21T21:54:12.976 INFO:tasks.ceph.osd.7.gibba017.stderr:2021-11-21T21:54:12.973+0000 7fe3fb9b9d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.977 INFO:tasks.ceph.osd.7.gibba017.stderr:2021-11-21T21:54:12.973+0000 7fe3fb9b9d80 -1 journal do_read_entry(8192): bad header magic 2021-11-21T21:54:12.978 INFO:tasks.ceph.osd.7.gibba017.stderr:2021-11-21T21:54:12.973+0000 7fe3fb9b9d80 -1 osd.7 0 log_to_monitors {default=true} 2021-11-21T21:54:13.735 DEBUG:teuthology.orchestra.run.gibba016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:13.789 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 Processor -- start 2021-11-21T21:54:13.790 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- start start 2021-11-21T21:54:13.790 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff268067890 0x7ff2680b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:13.791 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2680b2950 0x7ff2680b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:13.791 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff268145290 con 0x7ff2680b1ba0 2021-11-21T21:54:13.791 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff268067c70 con 0x7ff2680b2950 2021-11-21T21:54:13.791 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff268144d50 con 0x7ff268067890 2021-11-21T21:54:13.792 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2680b2950 0x7ff2680b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:13.792 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2680b2950 0x7ff2680b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44028/0 (socket says 172.21.2.116:44028) 2021-11-21T21:54:13.792 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 -- 172.21.2.116:0/4275623777 learned_addr learned my addr 172.21.2.116:0/4275623777 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:13.796 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 -- 172.21.2.116:0/4275623777 >> v2:172.21.2.117:3300/0 conn(0x7ff268067890 msgr2=0x7ff2680b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:13.796 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 --2- 172.21.2.116:0/4275623777 >> v2:172.21.2.117:3300/0 conn(0x7ff268067890 0x7ff2680b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:13.796 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 -- 172.21.2.116:0/4275623777 >> v1:172.21.2.117:6789/0 conn(0x7ff2680b1ba0 legacy=0x7ff2680b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:13.797 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 -- 172.21.2.116:0/4275623777 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268144ff0 con 0x7ff2680b2950 2021-11-21T21:54:13.797 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff267fff700 1 --2- 172.21.2.116:0/4275623777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b2950 0x7ff2680b3030 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff268064190 tx=0x7ff25c006eb0).ready entity=mon.0 client_cookie=188d2e90d7a403c0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:13.798 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff2677fe700 1 -- 172.21.2.116:0/4275623777 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25c00d5c0 con 0x7ff2680b2950 2021-11-21T21:54:13.798 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff2677fe700 1 -- 172.21.2.116:0/4275623777 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c000e90 con 0x7ff2680b2950 2021-11-21T21:54:13.798 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff2677fe700 1 -- 172.21.2.116:0/4275623777 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25c00d910 con 0x7ff2680b2950 2021-11-21T21:54:13.799 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- 172.21.2.116:0/4275623777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b2950 msgr2=0x7ff2680b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:13.799 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 --2- 172.21.2.116:0/4275623777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b2950 0x7ff2680b3030 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff268064190 tx=0x7ff25c006eb0).stop 2021-11-21T21:54:13.799 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- 172.21.2.116:0/4275623777 shutdown_connections 2021-11-21T21:54:13.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 --2- 172.21.2.116:0/4275623777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b2950 0x7ff2680b3030 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:13.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 --2- 172.21.2.116:0/4275623777 >> v2:172.21.2.117:3300/0 conn(0x7ff268067890 0x7ff2680b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:13.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- 172.21.2.116:0/4275623777 >> 172.21.2.116:0/4275623777 conn(0x7ff268064cf0 msgr2=0x7ff268065100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:13.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- 172.21.2.116:0/4275623777 shutdown_connections 2021-11-21T21:54:13.801 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.786+0000 7ff26dfba700 1 -- 172.21.2.116:0/4275623777 wait complete. 2021-11-21T21:54:13.801 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 Processor -- start 2021-11-21T21:54:13.801 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 -- start start 2021-11-21T21:54:13.801 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff268067890 0x7ff2680ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:13.802 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b1ba0 0x7ff2680a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:13.802 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2680b2950 0x7ff2680af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:13.802 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680a75d0 con 0x7ff2680b1ba0 2021-11-21T21:54:13.802 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680ad800 con 0x7ff2680b2950 2021-11-21T21:54:13.803 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680b05d0 con 0x7ff268067890 2021-11-21T21:54:13.803 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b1ba0 0x7ff2680a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:13.804 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b1ba0 0x7ff2680a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44034/0 (socket says 172.21.2.116:44034) 2021-11-21T21:54:13.804 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 -- 172.21.2.116:0/1202563614 learned_addr learned my addr 172.21.2.116:0/1202563614 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:13.804 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 -- 172.21.2.116:0/1202563614 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff268067890 msgr2=0x7ff2680ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:13.805 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 --2- 172.21.2.116:0/1202563614 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff268067890 0x7ff2680ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:13.805 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 -- 172.21.2.116:0/1202563614 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2680b2950 msgr2=0x7ff2680af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:13.806 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 --2- 172.21.2.116:0/1202563614 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2680b2950 0x7ff2680af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:13.806 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 -- 172.21.2.116:0/1202563614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c00b1e0 con 0x7ff2680b1ba0 2021-11-21T21:54:13.806 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff267fff700 1 --2- 172.21.2.116:0/1202563614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b1ba0 0x7ff2680a8950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff268064190 tx=0x7ff25c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:13.806 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff265ffb700 1 -- 172.21.2.116:0/1202563614 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25c002f70 con 0x7ff2680b1ba0 2021-11-21T21:54:13.807 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff265ffb700 1 -- 172.21.2.116:0/1202563614 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c00d940 con 0x7ff2680b1ba0 2021-11-21T21:54:13.807 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2680b0ec0 con 0x7ff2680b1ba0 2021-11-21T21:54:13.807 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff265ffb700 1 -- 172.21.2.116:0/1202563614 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25c011e90 con 0x7ff2680b1ba0 2021-11-21T21:54:13.808 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2680afda0 con 0x7ff2680b1ba0 2021-11-21T21:54:13.808 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff265ffb700 1 -- 172.21.2.116:0/1202563614 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 2) v1 ==== 32742+0+0 (secure 0 0 0) 0x7ff25c025030 con 0x7ff2680b1ba0 2021-11-21T21:54:13.808 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff265ffb700 1 -- 172.21.2.116:0/1202563614 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7ff25c03b500 con 0x7ff2680b1ba0 2021-11-21T21:54:13.808 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2680b0d00 con 0x7ff2680b1ba0 2021-11-21T21:54:13.809 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.790+0000 7ff265ffb700 1 -- 172.21.2.116:0/1202563614 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7ff25c023020 con 0x7ff2680b1ba0 2021-11-21T21:54:13.831 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.826+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2680b1050 con 0x7ff2680b1ba0 2021-11-21T21:54:13.832 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.826+0000 7ff265ffb700 1 -- 172.21.2.116:0/1202563614 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6849 (secure 0 0 0) 0x7ff25c023020 con 0x7ff2680b1ba0 2021-11-21T21:54:13.833 INFO:teuthology.misc.health.gibba016.stdout: 2021-11-21T21:54:13.833 INFO:teuthology.misc.health.gibba016.stdout:{"epoch":10,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:12.899716+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:13.834 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b1ba0 msgr2=0x7ff2680a8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:13.834 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 --2- 172.21.2.116:0/1202563614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b1ba0 0x7ff2680a8950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff268064190 tx=0x7ff25c015000).stop 2021-11-21T21:54:13.834 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 shutdown_connections 2021-11-21T21:54:13.835 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 --2- 172.21.2.116:0/1202563614 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2680b2950 0x7ff2680af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:13.835 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 --2- 172.21.2.116:0/1202563614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2680b1ba0 0x7ff2680a8950 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:13.835 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 --2- 172.21.2.116:0/1202563614 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff268067890 0x7ff2680ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:13.835 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 >> 172.21.2.116:0/1202563614 conn(0x7ff268064cf0 msgr2=0x7ff2681445c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:13.836 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 shutdown_connections 2021-11-21T21:54:13.836 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:13.830+0000 7ff26dfba700 1 -- 172.21.2.116:0/1202563614 wait complete. 2021-11-21T21:54:13.838 DEBUG:teuthology.misc:0 of 8 OSDs are up 2021-11-21T21:54:14.918 INFO:tasks.ceph.osd.1.gibba016.stderr:2021-11-21T21:54:14.914+0000 7fec48dbe700 -1 osd.1 0 waiting for initial osdmap 2021-11-21T21:54:14.919 INFO:tasks.ceph.osd.2.gibba016.stderr:2021-11-21T21:54:14.914+0000 7fbb992a1700 -1 osd.2 0 waiting for initial osdmap 2021-11-21T21:54:14.919 INFO:tasks.ceph.osd.3.gibba016.stderr:2021-11-21T21:54:14.914+0000 7f79f628b700 -1 osd.3 0 waiting for initial osdmap 2021-11-21T21:54:14.920 INFO:tasks.ceph.osd.0.gibba016.stderr:2021-11-21T21:54:14.914+0000 7efd2f7af700 -1 osd.0 0 waiting for initial osdmap 2021-11-21T21:54:14.920 INFO:tasks.ceph.osd.1.gibba016.stderr:2021-11-21T21:54:14.918+0000 7fec58240700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:14.921 INFO:tasks.ceph.osd.3.gibba016.stderr:2021-11-21T21:54:14.918+0000 7f7a0570d700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:14.921 INFO:tasks.ceph.osd.4.gibba017.stderr:2021-11-21T21:54:14.917+0000 7f597ec38700 -1 osd.4 0 waiting for initial osdmap 2021-11-21T21:54:14.923 INFO:tasks.ceph.osd.6.gibba017.stderr:2021-11-21T21:54:14.917+0000 7fe479673700 -1 osd.6 0 waiting for initial osdmap 2021-11-21T21:54:14.923 INFO:tasks.ceph.osd.5.gibba017.stderr:2021-11-21T21:54:14.917+0000 7fa135bdb700 -1 osd.5 0 waiting for initial osdmap 2021-11-21T21:54:14.924 INFO:tasks.ceph.osd.7.gibba017.stderr:2021-11-21T21:54:14.917+0000 7fe3e5a7c700 -1 osd.7 0 waiting for initial osdmap 2021-11-21T21:54:14.924 INFO:tasks.ceph.osd.5.gibba017.stderr:2021-11-21T21:54:14.917+0000 7fa14505d700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:14.925 INFO:tasks.ceph.osd.6.gibba017.stderr:2021-11-21T21:54:14.917+0000 7fe488af5700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:14.925 INFO:tasks.ceph.osd.7.gibba017.stderr:2021-11-21T21:54:14.917+0000 7fe3f4efe700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:14.926 INFO:tasks.ceph.osd.4.gibba017.stderr:2021-11-21T21:54:14.917+0000 7f598e0ba700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:14.945 INFO:tasks.ceph.osd.2.gibba016.stderr:2021-11-21T21:54:14.942+0000 7fbba8723700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:14.956 INFO:tasks.ceph.osd.0.gibba016.stderr:2021-11-21T21:54:14.954+0000 7efd3ec31700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-21T21:54:20.647 DEBUG:teuthology.orchestra.run.gibba016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:20.702 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 Processor -- start 2021-11-21T21:54:20.702 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- start start 2021-11-21T21:54:20.702 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80840b08f0 0x7f808409a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.703 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f808414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.703 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840b0400 con 0x7f80840af8c0 2021-11-21T21:54:20.703 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808409b090 con 0x7f80840b08f0 2021-11-21T21:54:20.703 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80840b21a0 con 0x7f8084098020 2021-11-21T21:54:20.703 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80840b08f0 0x7f808409a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.704 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80840b08f0 0x7f808409a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44234/0 (socket says 172.21.2.116:44234) 2021-11-21T21:54:20.704 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 -- 172.21.2.116:0/2033113462 learned_addr learned my addr 172.21.2.116:0/2033113462 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:20.708 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 -- 172.21.2.116:0/2033113462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 msgr2=0x7f808414a7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:20.709 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 --2- 172.21.2.116:0/2033113462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f808414a7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:20.709 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 -- 172.21.2.116:0/2033113462 >> v1:172.21.2.116:6789/0 conn(0x7f80840af8c0 legacy=0x7f808409a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.710 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 -- 172.21.2.116:0/2033113462 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840b0cd0 con 0x7f80840b08f0 2021-11-21T21:54:20.710 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8089670700 1 --2- 172.21.2.116:0/2033113462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840b08f0 0x7f808409a490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8080002f70 tx=0x7f8080006eb0).ready entity=mon.0 client_cookie=a5502370cb4f86c7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:20.710 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8073fff700 1 -- 172.21.2.116:0/2033113462 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f808000d5c0 con 0x7f80840b08f0 2021-11-21T21:54:20.711 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8073fff700 1 -- 172.21.2.116:0/2033113462 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080000e90 con 0x7f80840b08f0 2021-11-21T21:54:20.711 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f8073fff700 1 -- 172.21.2.116:0/2033113462 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f808000d950 con 0x7f80840b08f0 2021-11-21T21:54:20.711 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- 172.21.2.116:0/2033113462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840b08f0 msgr2=0x7f808409a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.711 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 --2- 172.21.2.116:0/2033113462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840b08f0 0x7f808409a490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8080002f70 tx=0x7f8080006eb0).stop 2021-11-21T21:54:20.712 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- 172.21.2.116:0/2033113462 shutdown_connections 2021-11-21T21:54:20.712 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 --2- 172.21.2.116:0/2033113462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f808414a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:20.712 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 --2- 172.21.2.116:0/2033113462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840b08f0 0x7f808409a490 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.712 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- 172.21.2.116:0/2033113462 >> 172.21.2.116:0/2033113462 conn(0x7f8084069680 msgr2=0x7f8084069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:20.713 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- 172.21.2.116:0/2033113462 shutdown_connections 2021-11-21T21:54:20.713 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.698+0000 7f808b8d4700 1 -- 172.21.2.116:0/2033113462 wait complete. 2021-11-21T21:54:20.713 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 Processor -- start 2021-11-21T21:54:20.713 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 -- start start 2021-11-21T21:54:20.713 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f8084064130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.714 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840af8c0 0x7f8084156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.714 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80840b08f0 0x7f8084157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.714 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084066bd0 con 0x7f80840af8c0 2021-11-21T21:54:20.714 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840aed60 con 0x7f80840b08f0 2021-11-21T21:54:20.714 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80840ae1e0 con 0x7f8084098020 2021-11-21T21:54:20.715 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840af8c0 0x7f8084156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.715 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840af8c0 0x7f8084156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44242/0 (socket says 172.21.2.116:44242) 2021-11-21T21:54:20.715 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 -- 172.21.2.116:0/4203734079 learned_addr learned my addr 172.21.2.116:0/4203734079 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:20.715 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8089670700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f8084064130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.716 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 -- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 msgr2=0x7f8084064130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.716 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8089e71700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80840b08f0 0x7f8084157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.717 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f8084064130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.717 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 -- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80840b08f0 msgr2=0x7f8084157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.717 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80840b08f0 0x7f8084157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.717 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 -- 172.21.2.116:0/4203734079 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808000b1e0 con 0x7f80840af8c0 2021-11-21T21:54:20.718 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8089670700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f8084064130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:20.718 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8089e71700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80840b08f0 0x7f8084157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:20.718 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8088e6f700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840af8c0 0x7f8084156c10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f80840628d0 tx=0x7f8074007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:20.718 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f80727fc700 1 -- 172.21.2.116:0/4203734079 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f807400ce00 con 0x7f80840af8c0 2021-11-21T21:54:20.719 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f80727fc700 1 -- 172.21.2.116:0/4203734079 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074007240 con 0x7f80840af8c0 2021-11-21T21:54:20.719 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80840ae480 con 0x7f80840af8c0 2021-11-21T21:54:20.719 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f80727fc700 1 -- 172.21.2.116:0/4203734079 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8074006700 con 0x7f80840af8c0 2021-11-21T21:54:20.719 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80840aea50 con 0x7f80840af8c0 2021-11-21T21:54:20.720 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80840af250 con 0x7f80840af8c0 2021-11-21T21:54:20.720 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f80727fc700 1 -- 172.21.2.116:0/4203734079 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8074006860 con 0x7f80840af8c0 2021-11-21T21:54:20.721 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f80727fc700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8064053610 0x7f8064055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.721 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f80727fc700 1 -- 172.21.2.116:0/4203734079 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4896+0+0 (secure 0 0 0) 0x7f807406aed0 con 0x7f80840af8c0 2021-11-21T21:54:20.721 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8089670700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8064053610 0x7f8064055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.721 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.702+0000 7f8089670700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8064053610 0x7f8064055ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8080017000 tx=0x7f8080015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:20.722 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.706+0000 7f80727fc700 1 -- 172.21.2.116:0/4203734079 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f80840af250 con 0x7f80840af8c0 2021-11-21T21:54:20.791 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.786+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80840af250 con 0x7f80840af8c0 2021-11-21T21:54:20.792 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f80727fc700 1 -- 172.21.2.116:0/4203734079 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11805 (secure 0 0 0) 0x7f80840af250 con 0x7f80840af8c0 2021-11-21T21:54:20.793 INFO:teuthology.misc.health.gibba016.stdout: 2021-11-21T21:54:20.793 INFO:teuthology.misc.health.gibba016.stdout:{"epoch":15,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:17.933275+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:20.794 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8064053610 msgr2=0x7f8064055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.798 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8064053610 0x7f8064055ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8080017000 tx=0x7f8080015000).stop 2021-11-21T21:54:20.798 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840af8c0 msgr2=0x7f8084156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.799 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840af8c0 0x7f8084156c10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f80840628d0 tx=0x7f8074007be0).stop 2021-11-21T21:54:20.799 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 shutdown_connections 2021-11-21T21:54:20.799 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80840b08f0 0x7f8084157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.799 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8064053610 0x7f8064055ad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80840af8c0 0x7f8084156c10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 --2- 172.21.2.116:0/4203734079 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8084098020 0x7f8084064130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 >> 172.21.2.116:0/4203734079 conn(0x7f8084069680 msgr2=0x7f8084069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:20.800 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 shutdown_connections 2021-11-21T21:54:20.801 INFO:teuthology.misc.health.gibba016.stderr:2021-11-21T21:54:20.790+0000 7f808b8d4700 1 -- 172.21.2.116:0/4203734079 wait complete. 2021-11-21T21:54:20.802 DEBUG:teuthology.misc:8 of 8 OSDs are up 2021-11-21T21:54:20.803 INFO:tasks.ceph:Creating RBD pool 2021-11-21T21:54:20.803 DEBUG:teuthology.orchestra.run.gibba016:> sudo ceph --cluster ceph osd pool create rbd 8 2021-11-21T21:54:20.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 Processor -- start 2021-11-21T21:54:20.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- start start 2021-11-21T21:54:20.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ec40b02e0 0x7f3ec405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3ec40afab0 0x7f3ec4143680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec40b06c0 con 0x7f3ec40af620 2021-11-21T21:54:20.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec40adf20 con 0x7f3ec40b02e0 2021-11-21T21:54:20.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec40b2470 con 0x7f3ec40afab0 2021-11-21T21:54:20.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ec40b02e0 0x7f3ec405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ec40b02e0 0x7f3ec405f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44250/0 (socket says 172.21.2.116:44250) 2021-11-21T21:54:20.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 -- 172.21.2.116:0/2818923586 learned_addr learned my addr 172.21.2.116:0/2818923586 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:20.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 -- 172.21.2.116:0/2818923586 >> v2:172.21.2.117:3300/0 conn(0x7f3ec40afab0 msgr2=0x7f3ec4143680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:20.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 --2- 172.21.2.116:0/2818923586 >> v2:172.21.2.117:3300/0 conn(0x7f3ec40afab0 0x7f3ec4143680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:20.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 -- 172.21.2.116:0/2818923586 >> v1:172.21.2.117:6789/0 conn(0x7f3ec40af620 legacy=0x7f3ec4142f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:20.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 -- 172.21.2.116:0/2818923586 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec40b26d0 con 0x7f3ec40b02e0 2021-11-21T21:54:20.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec259c700 1 --2- 172.21.2.116:0/2818923586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40b02e0 0x7f3ec405f730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3ec4063350 tx=0x7f3eac006f40).ready entity=mon.0 client_cookie=e734d491de0fa5da server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:20.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec159a700 1 -- 172.21.2.116:0/2818923586 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3eac00dc30 con 0x7f3ec40b02e0 2021-11-21T21:54:20.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec159a700 1 -- 172.21.2.116:0/2818923586 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac000e90 con 0x7f3ec40b02e0 2021-11-21T21:54:20.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec159a700 1 -- 172.21.2.116:0/2818923586 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3eac00be90 con 0x7f3ec40b02e0 2021-11-21T21:54:20.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/2818923586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40b02e0 msgr2=0x7f3ec405f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/2818923586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40b02e0 0x7f3ec405f730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3ec4063350 tx=0x7f3eac006f40).stop 2021-11-21T21:54:20.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/2818923586 shutdown_connections 2021-11-21T21:54:20.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/2818923586 >> v2:172.21.2.117:3300/0 conn(0x7f3ec40afab0 0x7f3ec4143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:20.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/2818923586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40b02e0 0x7f3ec405f730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/2818923586 >> 172.21.2.116:0/2818923586 conn(0x7f3ec40b5ff0 msgr2=0x7f3ec40b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:20.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/2818923586 shutdown_connections 2021-11-21T21:54:20.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/2818923586 wait complete. 2021-11-21T21:54:20.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.858+0000 7f3ec8dfd700 1 Processor -- start 2021-11-21T21:54:20.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 -- start start 2021-11-21T21:54:20.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ec40af620 0x7f3ec4065ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40afab0 0x7f3ec414e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ec40b02e0 0x7f3ec4151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec414dfb0 con 0x7f3ec40afab0 2021-11-21T21:54:20.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec40a01c0 con 0x7f3ec40b02e0 2021-11-21T21:54:20.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec414e1a0 con 0x7f3ec40af620 2021-11-21T21:54:20.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40afab0 0x7f3ec414e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40afab0 0x7f3ec414e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44256/0 (socket says 172.21.2.116:44256) 2021-11-21T21:54:20.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 -- 172.21.2.116:0/902559820 learned_addr learned my addr 172.21.2.116:0/902559820 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:20.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec259c700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ec40af620 0x7f3ec4065ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec2d9d700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ec40b02e0 0x7f3ec4151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 -- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ec40af620 msgr2=0x7f3ec4065ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ec40af620 0x7f3ec4065ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 -- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ec40b02e0 msgr2=0x7f3ec4151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:20.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ec40b02e0 0x7f3ec4151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:20.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 -- 172.21.2.116:0/902559820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac00b460 con 0x7f3ec40afab0 2021-11-21T21:54:20.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec2d9d700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ec40b02e0 0x7f3ec4151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:20.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec259c700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ec40af620 0x7f3ec4065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:20.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec1d9b700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40afab0 0x7f3ec414e8e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3eb4002fb0 tx=0x7f3eb4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:20.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ebb7fe700 1 -- 172.21.2.116:0/902559820 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3eb4013460 con 0x7f3ec40afab0 2021-11-21T21:54:20.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ebb7fe700 1 -- 172.21.2.116:0/902559820 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb40135c0 con 0x7f3ec40afab0 2021-11-21T21:54:20.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec414e440 con 0x7f3ec40afab0 2021-11-21T21:54:20.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec40a8ed0 con 0x7f3ec40afab0 2021-11-21T21:54:20.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ebb7fe700 1 -- 172.21.2.116:0/902559820 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3eb4013920 con 0x7f3ec40afab0 2021-11-21T21:54:20.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec409fb60 con 0x7f3ec40afab0 2021-11-21T21:54:20.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ebb7fe700 1 -- 172.21.2.116:0/902559820 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3eb400f070 con 0x7f3ec40afab0 2021-11-21T21:54:20.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ebb7fe700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3ea80536f0 0x7f3ea8055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:20.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ebb7fe700 1 -- 172.21.2.116:0/902559820 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4896+0+0 (secure 0 0 0) 0x7f3eb406af40 con 0x7f3ec40afab0 2021-11-21T21:54:20.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.862+0000 7f3ec259c700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3ea80536f0 0x7f3ea8055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:20.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.866+0000 7f3ec259c700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3ea80536f0 0x7f3ea8055bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3eac019000 tx=0x7f3ec40b7000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:20.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.866+0000 7f3ebb7fe700 1 -- 172.21.2.116:0/902559820 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3eb4042540 con 0x7f3ec40afab0 2021-11-21T21:54:20.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:20.946+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f3ec40b0da0 con 0x7f3ec40afab0 2021-11-21T21:54:21.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.698+0000 7f3ebb7fe700 1 -- 172.21.2.116:0/902559820 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f3eb4045b60 con 0x7f3ec40afab0 2021-11-21T21:54:21.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3ea80536f0 msgr2=0x7f3ea8055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:21.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3ea80536f0 0x7f3ea8055bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3eac019000 tx=0x7f3ec40b7000).stop 2021-11-21T21:54:21.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40afab0 msgr2=0x7f3ec414e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:21.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40afab0 0x7f3ec414e8e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3eb4002fb0 tx=0x7f3eb4006df0).stop 2021-11-21T21:54:21.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 shutdown_connections 2021-11-21T21:54:21.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ec40b02e0 0x7f3ec4151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:21.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3ea80536f0 0x7f3ea8055bb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:21.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ec40afab0 0x7f3ec414e8e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:21.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 --2- 172.21.2.116:0/902559820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ec40af620 0x7f3ec4065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:21.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 >> 172.21.2.116:0/902559820 conn(0x7f3ec40b5ff0 msgr2=0x7f3ec40b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:21.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 shutdown_connections 2021-11-21T21:54:21.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.706+0000 7f3ec8dfd700 1 -- 172.21.2.116:0/902559820 wait complete. 2021-11-21T21:54:21.718 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'rbd' created 2021-11-21T21:54:21.719 DEBUG:teuthology.orchestra.run.gibba016:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2021-11-21T21:54:21.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 Processor -- start 2021-11-21T21:54:21.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 -- start start 2021-11-21T21:54:21.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f21c805c060 0x7f21c8141200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:21.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f21c805f730 0x7f21c81416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:21.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c80b1ae0 con 0x7f21c80aa930 2021-11-21T21:54:21.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c80b2250 con 0x7f21c805c060 2021-11-21T21:54:21.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c805c440 con 0x7f21c805f730 2021-11-21T21:54:21.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f21c805c060 0x7f21c8141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:21.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f21c805c060 0x7f21c8141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44264/0 (socket says 172.21.2.116:44264) 2021-11-21T21:54:21.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 -- 172.21.2.116:0/1034918852 learned_addr learned my addr 172.21.2.116:0/1034918852 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:21.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c759e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f21c80aa930 0x7f21c805fe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56218/0 (socket says 172.21.2.116:56218) 2021-11-21T21:54:21.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 -- 172.21.2.116:0/1034918852 >> v2:172.21.2.117:3300/0 conn(0x7f21c805f730 msgr2=0x7f21c81416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:21.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 --2- 172.21.2.116:0/1034918852 >> v2:172.21.2.117:3300/0 conn(0x7f21c805f730 0x7f21c81416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:21.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 -- 172.21.2.116:0/1034918852 >> v1:172.21.2.116:6789/0 conn(0x7f21c80aa930 legacy=0x7f21c805fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:21.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 -- 172.21.2.116:0/1034918852 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8140da0 con 0x7f21c805c060 2021-11-21T21:54:21.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c6d9d700 1 --2- 172.21.2.116:0/1034918852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805c060 0x7f21c8141200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0006eb0).ready entity=mon.0 client_cookie=830233f4ff3bb5bf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:21.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c5d9b700 1 -- 172.21.2.116:0/1034918852 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b000d610 con 0x7f21c805c060 2021-11-21T21:54:21.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c5d9b700 1 -- 172.21.2.116:0/1034918852 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b0000e90 con 0x7f21c805c060 2021-11-21T21:54:21.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21c5d9b700 1 -- 172.21.2.116:0/1034918852 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b000d9a0 con 0x7f21c805c060 2021-11-21T21:54:21.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 -- 172.21.2.116:0/1034918852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805c060 msgr2=0x7f21c8141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:21.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 --2- 172.21.2.116:0/1034918852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805c060 0x7f21c8141200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0006eb0).stop 2021-11-21T21:54:21.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 -- 172.21.2.116:0/1034918852 shutdown_connections 2021-11-21T21:54:21.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 --2- 172.21.2.116:0/1034918852 >> v2:172.21.2.117:3300/0 conn(0x7f21c805f730 0x7f21c81416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:21.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 --2- 172.21.2.116:0/1034918852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805c060 0x7f21c8141200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:21.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.770+0000 7f21cd10d700 1 -- 172.21.2.116:0/1034918852 >> 172.21.2.116:0/1034918852 conn(0x7f21c80b3be0 msgr2=0x7f21c80b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:21.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- 172.21.2.116:0/1034918852 shutdown_connections 2021-11-21T21:54:21.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- 172.21.2.116:0/1034918852 wait complete. 2021-11-21T21:54:21.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 Processor -- start 2021-11-21T21:54:21.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- start start 2021-11-21T21:54:21.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805f730 0x7f21c813fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:21.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21c80aa930 0x7f21c8140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:21.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21c815a160 0x7f21c80af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:21.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80b05d0 con 0x7f21c805f730 2021-11-21T21:54:21.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80b0870 con 0x7f21c80aa930 2021-11-21T21:54:21.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80b0b10 con 0x7f21c815a160 2021-11-21T21:54:21.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805f730 0x7f21c813fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:21.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805f730 0x7f21c813fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44268/0 (socket says 172.21.2.116:44268) 2021-11-21T21:54:21.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 -- 172.21.2.116:0/739503057 learned_addr learned my addr 172.21.2.116:0/739503057 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:21.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 -- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21c815a160 msgr2=0x7f21c80af550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:21.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21c815a160 0x7f21c80af550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:21.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 -- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21c80aa930 msgr2=0x7f21c8140250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:21.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21c80aa930 0x7f21c8140250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:21.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 -- 172.21.2.116:0/739503057 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b000b1e0 con 0x7f21c805f730 2021-11-21T21:54:21.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c6d9d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805f730 0x7f21c813fd60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:21.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21bffff700 1 -- 172.21.2.116:0/739503057 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b0006be0 con 0x7f21c805f730 2021-11-21T21:54:21.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21bffff700 1 -- 172.21.2.116:0/739503057 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b0011ce0 con 0x7f21c805f730 2021-11-21T21:54:21.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21bffff700 1 -- 172.21.2.116:0/739503057 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b001e650 con 0x7f21c805f730 2021-11-21T21:54:21.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c80b0db0 con 0x7f21c805f730 2021-11-21T21:54:21.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c80b1250 con 0x7f21c805f730 2021-11-21T21:54:21.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c80b1d10 con 0x7f21c805f730 2021-11-21T21:54:21.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21bffff700 1 -- 172.21.2.116:0/739503057 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f21b001a040 con 0x7f21c805f730 2021-11-21T21:54:21.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21bffff700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21ac0536d0 0x7f21ac055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:21.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21bffff700 1 -- 172.21.2.116:0/739503057 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5238+0+0 (secure 0 0 0) 0x7f21b006c5f0 con 0x7f21c805f730 2021-11-21T21:54:21.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.774+0000 7f21c659c700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21ac0536d0 0x7f21ac055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:21.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.778+0000 7f21c659c700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21ac0536d0 0x7f21ac055b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f21b8002f70 tx=0x7f21b8006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:21.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.778+0000 7f21bffff700 1 -- 172.21.2.116:0/739503057 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f21b0043ad0 con 0x7f21c805f730 2021-11-21T21:54:21.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:21.862+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f21c80573d0 con 0x7f21c805f730 2021-11-21T21:54:22.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21bffff700 1 -- 172.21.2.116:0/739503057 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f21b00470b0 con 0x7f21c805f730 2021-11-21T21:54:22.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21ac0536d0 msgr2=0x7f21ac055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21ac0536d0 0x7f21ac055b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f21b8002f70 tx=0x7f21b8006a00).stop 2021-11-21T21:54:22.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805f730 msgr2=0x7f21c813fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805f730 0x7f21c813fd60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0015000).stop 2021-11-21T21:54:22.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 shutdown_connections 2021-11-21T21:54:22.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21c815a160 0x7f21c80af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21c80aa930 0x7f21c8140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21ac0536d0 0x7f21ac055b90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:22.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 --2- 172.21.2.116:0/739503057 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21c805f730 0x7f21c813fd60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:22.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 >> 172.21.2.116:0/739503057 conn(0x7f21c80b3be0 msgr2=0x7f21c8062d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:22.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 shutdown_connections 2021-11-21T21:54:22.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.706+0000 7f21cd10d700 1 -- 172.21.2.116:0/739503057 wait complete. 2021-11-21T21:54:22.714 INFO:teuthology.orchestra.run.gibba016.stderr:enabled application 'rbd' on pool 'rbd' 2021-11-21T21:54:22.716 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2021-11-21T21:54:22.717 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T21:54:22.717 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T21:54:22.719 INFO:tasks.ceph.mds.a:Started 2021-11-21T21:54:22.720 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T21:54:22.720 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T21:54:22.722 INFO:tasks.ceph.mds.c:Started 2021-11-21T21:54:22.724 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T21:54:22.724 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T21:54:22.727 INFO:tasks.ceph.mds.b:Started 2021-11-21T21:54:22.727 INFO:tasks.ceph:Setting up CephFS filesystem... 2021-11-21T21:54:22.728 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:22.753 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T21:54:22.755 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:54:22.766 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T21:54:22.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 Processor -- start 2021-11-21T21:54:22.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- start start 2021-11-21T21:54:22.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcbc80a86b0 0x7fcbc814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcbc8062410 0x7fcbc81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc80a6e80 con 0x7fcbc80a6aa0 2021-11-21T21:54:22.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc80a8bc0 con 0x7fcbc8062410 2021-11-21T21:54:22.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc814a7b0 con 0x7fcbc80a86b0 2021-11-21T21:54:22.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcbc8062410 0x7fcbc81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:22.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcbc8062410 0x7fcbc81443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44304/0 (socket says 172.21.2.116:44304) 2021-11-21T21:54:22.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3993382870 learned_addr learned my addr 172.21.2.116:0/3993382870 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:22.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3993382870 >> v2:172.21.2.117:3300/0 conn(0x7fcbc80a86b0 msgr2=0x7fcbc814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:22.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- 172.21.2.116:0/3993382870 >> v2:172.21.2.117:3300/0 conn(0x7fcbc80a86b0 0x7fcbc814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3993382870 >> v1:172.21.2.117:6789/0 conn(0x7fcbc80a6aa0 legacy=0x7fcbc81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:22.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3993382870 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc814aa10 con 0x7fcbc8062410 2021-11-21T21:54:22.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- 172.21.2.116:0/3993382870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc8062410 0x7fcbc81443f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fcbc8061ab0 tx=0x7fcbc0009290).ready entity=mon.0 client_cookie=e3cd6a6976d87acd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:22.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc67fc700 1 -- 172.21.2.116:0/3993382870 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbc000b830 con 0x7fcbc8062410 2021-11-21T21:54:22.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc67fc700 1 -- 172.21.2.116:0/3993382870 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc000b990 con 0x7fcbc8062410 2021-11-21T21:54:22.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc67fc700 1 -- 172.21.2.116:0/3993382870 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbc000bb00 con 0x7fcbc8062410 2021-11-21T21:54:22.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3993382870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc8062410 msgr2=0x7fcbc81443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3993382870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc8062410 0x7fcbc81443f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fcbc8061ab0 tx=0x7fcbc0009290).stop 2021-11-21T21:54:22.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3993382870 shutdown_connections 2021-11-21T21:54:22.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3993382870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc8062410 0x7fcbc81443f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:22.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3993382870 >> v2:172.21.2.117:3300/0 conn(0x7fcbc80a86b0 0x7fcbc814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3993382870 >> 172.21.2.116:0/3993382870 conn(0x7fcbc805bd20 msgr2=0x7fcbc805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:22.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3993382870 shutdown_connections 2021-11-21T21:54:22.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3993382870 wait complete. 2021-11-21T21:54:22.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 Processor -- start 2021-11-21T21:54:22.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- start start 2021-11-21T21:54:22.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcbc8062410 0x7fcbc81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcbc80a6aa0 0x7fcbc8143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc80a86b0 0x7fcbc8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc8064180 con 0x7fcbc80a86b0 2021-11-21T21:54:22.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc8159840 con 0x7fcbc8062410 2021-11-21T21:54:22.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbcdb4f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc8065700 con 0x7fcbc80a6aa0 2021-11-21T21:54:22.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc80a86b0 0x7fcbc8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:22.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc80a86b0 0x7fcbc8159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44312/0 (socket says 172.21.2.116:44312) 2021-11-21T21:54:22.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3690536733 learned_addr learned my addr 172.21.2.116:0/3690536733 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:22.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcbc80a6aa0 msgr2=0x7fcbc8143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcbc80a6aa0 0x7fcbc8143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcbc8062410 msgr2=0x7fcbc81562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcbc8062410 0x7fcbc81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 -- 172.21.2.116:0/3690536733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0009530 con 0x7fcbc80a86b0 2021-11-21T21:54:22.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.778+0000 7fcbc7fff700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc80a86b0 0x7fcbc8159ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcbc8061ab0 tx=0x7fcbc0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:22.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc4ff9700 1 -- 172.21.2.116:0/3690536733 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbc000b830 con 0x7fcbc80a86b0 2021-11-21T21:54:22.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc4ff9700 1 -- 172.21.2.116:0/3690536733 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc0020620 con 0x7fcbc80a86b0 2021-11-21T21:54:22.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc4ff9700 1 -- 172.21.2.116:0/3690536733 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbc000bee0 con 0x7fcbc80a86b0 2021-11-21T21:54:22.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc80a9640 con 0x7fcbc80a86b0 2021-11-21T21:54:22.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc8159a30 con 0x7fcbc80a86b0 2021-11-21T21:54:22.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc8159bf0 con 0x7fcbc80a86b0 2021-11-21T21:54:22.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc4ff9700 1 -- 172.21.2.116:0/3690536733 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcbc0026020 con 0x7fcbc80a86b0 2021-11-21T21:54:22.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc4ff9700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcbac053610 0x7fcbac055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc4ff9700 1 -- 172.21.2.116:0/3690536733 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcbc006c4b0 con 0x7fcbc80a86b0 2021-11-21T21:54:22.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc77fe700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcbac053610 0x7fcbac055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:22.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc77fe700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcbac053610 0x7fcbac055ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcbb0002f90 tx=0x7fcbb00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:22.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.782+0000 7fcbc4ff9700 1 -- 172.21.2.116:0/3690536733 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcbc00439c0 con 0x7fcbc80a86b0 2021-11-21T21:54:22.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.866+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc80573d0 con 0x7fcbc80a86b0 2021-11-21T21:54:22.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbc4ff9700 1 -- 172.21.2.116:0/3690536733 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fcbc0015020 con 0x7fcbc80a86b0 2021-11-21T21:54:22.872 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:22.873 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:22.698378+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:22.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcbac053610 msgr2=0x7fcbac055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcbac053610 0x7fcbac055ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcbb0002f90 tx=0x7fcbb00069b0).stop 2021-11-21T21:54:22.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc80a86b0 msgr2=0x7fcbc8159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc80a86b0 0x7fcbc8159ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcbc8061ab0 tx=0x7fcbc0016000).stop 2021-11-21T21:54:22.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 shutdown_connections 2021-11-21T21:54:22.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcbac053610 0x7fcbac055ad0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:22.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcbc80a86b0 0x7fcbc8159ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:22.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcbc80a6aa0 0x7fcbc8143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 --2- 172.21.2.116:0/3690536733 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcbc8062410 0x7fcbc81562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 >> 172.21.2.116:0/3690536733 conn(0x7fcbc805bd20 msgr2=0x7fcbc80b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:22.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 shutdown_connections 2021-11-21T21:54:22.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.870+0000 7fcbcdb4f700 1 -- 172.21.2.116:0/3690536733 wait complete. 2021-11-21T21:54:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T21:54:22.881 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:22.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 Processor -- start 2021-11-21T21:54:22.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- start start 2021-11-21T21:54:22.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f58341426e0 0x7f5834142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 0x7f58340ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340afae0 con 0x7f5834062e60 2021-11-21T21:54:22.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340b0f60 con 0x7f58341426e0 2021-11-21T21:54:22.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5834063470 con 0x7f58340ad470 2021-11-21T21:54:22.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f58341426e0 0x7f5834142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:22.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f58341426e0 0x7f5834142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44316/0 (socket says 172.21.2.116:44316) 2021-11-21T21:54:22.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 -- 172.21.2.116:0/2533030061 learned_addr learned my addr 172.21.2.116:0/2533030061 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:22.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 -- 172.21.2.116:0/2533030061 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 msgr2=0x7f58340ad900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:22.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 --2- 172.21.2.116:0/2533030061 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 0x7f58340ad900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 -- 172.21.2.116:0/2533030061 >> v1:172.21.2.116:6789/0 conn(0x7f5834062e60 legacy=0x7f58340af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 -- 172.21.2.116:0/2533030061 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834143050 con 0x7f58341426e0 2021-11-21T21:54:22.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583259c700 1 --2- 172.21.2.116:0/2533030061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58341426e0 0x7f5834142ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5824002f70 tx=0x7f5824006eb0).ready entity=mon.0 client_cookie=83a13fa710312dd6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:22.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583159a700 1 -- 172.21.2.116:0/2533030061 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5824010810 con 0x7f58341426e0 2021-11-21T21:54:22.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583159a700 1 -- 172.21.2.116:0/2533030061 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5824000e90 con 0x7f58341426e0 2021-11-21T21:54:22.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f583159a700 1 -- 172.21.2.116:0/2533030061 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5824010b60 con 0x7f58341426e0 2021-11-21T21:54:22.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- 172.21.2.116:0/2533030061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58341426e0 msgr2=0x7f5834142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 --2- 172.21.2.116:0/2533030061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58341426e0 0x7f5834142ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5824002f70 tx=0x7f5824006eb0).stop 2021-11-21T21:54:22.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- 172.21.2.116:0/2533030061 shutdown_connections 2021-11-21T21:54:22.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 --2- 172.21.2.116:0/2533030061 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 0x7f58340ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 --2- 172.21.2.116:0/2533030061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58341426e0 0x7f5834142ac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:22.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- 172.21.2.116:0/2533030061 >> 172.21.2.116:0/2533030061 conn(0x7f58340b5730 msgr2=0x7f58340b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:22.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- 172.21.2.116:0/2533030061 shutdown_connections 2021-11-21T21:54:22.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.930+0000 7f5838b7e700 1 -- 172.21.2.116:0/2533030061 wait complete. 2021-11-21T21:54:22.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 Processor -- start 2021-11-21T21:54:22.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 -- start start 2021-11-21T21:54:22.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 0x7f5834127e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f58341426e0 0x7f5834128300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f583413fb60 0x7f5834159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583413f340 con 0x7f583413fb60 2021-11-21T21:54:22.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583414a2f0 con 0x7f58341426e0 2021-11-21T21:54:22.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5834063840 con 0x7f58340ad470 2021-11-21T21:54:22.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f583413fb60 0x7f5834159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:22.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f583413fb60 0x7f5834159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44326/0 (socket says 172.21.2.116:44326) 2021-11-21T21:54:22.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 -- 172.21.2.116:0/80594724 learned_addr learned my addr 172.21.2.116:0/80594724 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:22.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 -- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 msgr2=0x7f5834127e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 0x7f5834127e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 -- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f58341426e0 msgr2=0x7f5834128300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:22.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f58341426e0 0x7f5834128300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:22.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 -- 172.21.2.116:0/80594724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582400c2f0 con 0x7f583413fb60 2021-11-21T21:54:22.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5832d9d700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f583413fb60 0x7f5834159c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f582c003330 tx=0x7f582c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:22.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f582b7fe700 1 -- 172.21.2.116:0/80594724 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f582c001520 con 0x7f583413fb60 2021-11-21T21:54:22.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5834063ce0 con 0x7f583413fb60 2021-11-21T21:54:22.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f582b7fe700 1 -- 172.21.2.116:0/80594724 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c001680 con 0x7f583413fb60 2021-11-21T21:54:22.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f582b7fe700 1 -- 172.21.2.116:0/80594724 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f582c0019e0 con 0x7f583413fb60 2021-11-21T21:54:22.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583414a590 con 0x7f583413fb60 2021-11-21T21:54:22.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58340b1e00 con 0x7f583413fb60 2021-11-21T21:54:22.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f582b7fe700 1 -- 172.21.2.116:0/80594724 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f582c018030 con 0x7f583413fb60 2021-11-21T21:54:22.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f582b7fe700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5818053540 0x7f5818055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:22.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.934+0000 7f582b7fe700 1 -- 172.21.2.116:0/80594724 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f582c06b180 con 0x7f583413fb60 2021-11-21T21:54:22.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.938+0000 7f583259c700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5818053540 0x7f5818055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:22.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.938+0000 7f583259c700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5818053540 0x7f5818055a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5824002f70 tx=0x7f5824015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:22.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:22.938+0000 7f582b7fe700 1 -- 172.21.2.116:0/80594724 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f582c0425b0 con 0x7f583413fb60 2021-11-21T21:54:23.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.018+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f58340573d0 con 0x7f583413fb60 2021-11-21T21:54:23.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.018+0000 7f582b7fe700 1 -- 172.21.2.116:0/80594724 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f582c045bd0 con 0x7f583413fb60 2021-11-21T21:54:23.024 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:23.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5818053540 msgr2=0x7f5818055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5818053540 0x7f5818055a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5824002f70 tx=0x7f5824015000).stop 2021-11-21T21:54:23.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f583413fb60 msgr2=0x7f5834159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f583413fb60 0x7f5834159c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f582c003330 tx=0x7f582c0091a0).stop 2021-11-21T21:54:23.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 shutdown_connections 2021-11-21T21:54:23.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5818053540 0x7f5818055a00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f583413fb60 0x7f5834159c60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f58341426e0 0x7f5834128300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 --2- 172.21.2.116:0/80594724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58340ad470 0x7f5834127e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 >> 172.21.2.116:0/80594724 conn(0x7f58340b5730 msgr2=0x7f583414dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 shutdown_connections 2021-11-21T21:54:23.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.022+0000 7f5838b7e700 1 -- 172.21.2.116:0/80594724 wait complete. 2021-11-21T21:54:23.031 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:23.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.078+0000 7fcec66ba700 1 Processor -- start 2021-11-21T21:54:23.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.078+0000 7fcec66ba700 1 -- start start 2021-11-21T21:54:23.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcec01478c0 0x7fcec00acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec0147e80 con 0x7fcec0144810 2021-11-21T21:54:23.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00af5a0 con 0x7fcec00abce0 2021-11-21T21:54:23.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec00b1490 con 0x7fcec01478c0 2021-11-21T21:54:23.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec4c57700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcec01478c0 0x7fcec00acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec4c57700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcec01478c0 0x7fcec00acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44334/0 (socket says 172.21.2.116:44334) 2021-11-21T21:54:23.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec4c57700 1 -- 172.21.2.116:0/430099212 learned_addr learned my addr 172.21.2.116:0/430099212 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:23.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec4c57700 1 -- 172.21.2.116:0/430099212 >> v1:172.21.2.117:6789/0 conn(0x7fcec00abce0 legacy=0x7fcec00ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:23.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec4c57700 1 -- 172.21.2.116:0/430099212 >> v1:172.21.2.116:6789/0 conn(0x7fcec0144810 legacy=0x7fcec00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec4c57700 1 -- 172.21.2.116:0/430099212 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00b1fd0 con 0x7fcec01478c0 2021-11-21T21:54:23.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec4c57700 1 --2- 172.21.2.116:0/430099212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec01478c0 0x7fcec00acd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fcec00b4e00 tx=0x7fceb0006eb0).ready entity=mon.0 client_cookie=883e812e5e2e3897 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebeffd700 1 -- 172.21.2.116:0/430099212 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000d610 con 0x7fcec01478c0 2021-11-21T21:54:23.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebeffd700 1 -- 172.21.2.116:0/430099212 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb0000e90 con 0x7fcec01478c0 2021-11-21T21:54:23.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebeffd700 1 -- 172.21.2.116:0/430099212 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000d960 con 0x7fcec01478c0 2021-11-21T21:54:23.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- 172.21.2.116:0/430099212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec01478c0 msgr2=0x7fcec00acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 --2- 172.21.2.116:0/430099212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec01478c0 0x7fcec00acd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fcec00b4e00 tx=0x7fceb0006eb0).stop 2021-11-21T21:54:23.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- 172.21.2.116:0/430099212 shutdown_connections 2021-11-21T21:54:23.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 --2- 172.21.2.116:0/430099212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec01478c0 0x7fcec00acd10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- 172.21.2.116:0/430099212 >> 172.21.2.116:0/430099212 conn(0x7fcec005bd20 msgr2=0x7fcec005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- 172.21.2.116:0/430099212 shutdown_connections 2021-11-21T21:54:23.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- 172.21.2.116:0/430099212 wait complete. 2021-11-21T21:54:23.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 Processor -- start 2021-11-21T21:54:23.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- start start 2021-11-21T21:54:23.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec00abce0 0x7fcec0142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcec0146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec01478c0 0x7fcec00b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0146790 con 0x7fcec00abce0 2021-11-21T21:54:23.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec013ca60 con 0x7fcec01478c0 2021-11-21T21:54:23.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec013cd00 con 0x7fcec0144810 2021-11-21T21:54:23.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec00abce0 0x7fcec0142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec00abce0 0x7fcec0142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44340/0 (socket says 172.21.2.116:44340) 2021-11-21T21:54:23.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 -- 172.21.2.116:0/2941736163 learned_addr learned my addr 172.21.2.116:0/2941736163 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:23.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 -- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 msgr2=0x7fcec0146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcec0146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 -- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec01478c0 msgr2=0x7fcec00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec01478c0 0x7fcec00b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 -- 172.21.2.116:0/2941736163 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb000b1e0 con 0x7fcec00abce0 2021-11-21T21:54:23.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebffff700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec00abce0 0x7fcec0142de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcec0062500 tx=0x7fceb4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebd7fa700 1 -- 172.21.2.116:0/2941736163 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb400ce30 con 0x7fcec00abce0 2021-11-21T21:54:23.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec01469e0 con 0x7fcec00abce0 2021-11-21T21:54:23.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebd7fa700 1 -- 172.21.2.116:0/2941736163 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb400f470 con 0x7fcec00abce0 2021-11-21T21:54:23.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcebd7fa700 1 -- 172.21.2.116:0/2941736163 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb400f7d0 con 0x7fcec00abce0 2021-11-21T21:54:23.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.082+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec0147370 con 0x7fcec00abce0 2021-11-21T21:54:23.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.086+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec0146b70 con 0x7fcec00abce0 2021-11-21T21:54:23.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.086+0000 7fcebd7fa700 1 -- 172.21.2.116:0/2941736163 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fceb4010070 con 0x7fcec00abce0 2021-11-21T21:54:23.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.086+0000 7fcebd7fa700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea00536a0 0x7fcea0055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.086+0000 7fcebd7fa700 1 -- 172.21.2.116:0/2941736163 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fceb406ac30 con 0x7fcec00abce0 2021-11-21T21:54:23.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.086+0000 7fcebf7fe700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea00536a0 0x7fcea0055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.086+0000 7fcebf7fe700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea00536a0 0x7fcea0055b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fceb8003330 tx=0x7fceb8006d30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.086+0000 7fcebd7fa700 1 -- 172.21.2.116:0/2941736163 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fceb4042020 con 0x7fcec00abce0 2021-11-21T21:54:23.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcec00af780 con 0x7fcec00abce0 2021-11-21T21:54:23.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcebd7fa700 1 -- 172.21.2.116:0/2941736163 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fceb4045640 con 0x7fcec00abce0 2021-11-21T21:54:23.175 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:23.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea00536a0 msgr2=0x7fcea0055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea00536a0 0x7fcea0055b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fceb8003330 tx=0x7fceb8006d30).stop 2021-11-21T21:54:23.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec00abce0 msgr2=0x7fcec0142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec00abce0 0x7fcec0142de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcec0062500 tx=0x7fceb4007c50).stop 2021-11-21T21:54:23.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 shutdown_connections 2021-11-21T21:54:23.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec01478c0 0x7fcec00b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcec0146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea00536a0 0x7fcea0055b60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 --2- 172.21.2.116:0/2941736163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec00abce0 0x7fcec0142de0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 >> 172.21.2.116:0/2941736163 conn(0x7fcec005bd20 msgr2=0x7fcec005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 shutdown_connections 2021-11-21T21:54:23.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.170+0000 7fcec66ba700 1 -- 172.21.2.116:0/2941736163 wait complete. 2021-11-21T21:54:23.187 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T21:54:23.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 Processor -- start 2021-11-21T21:54:23.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- start start 2021-11-21T21:54:23.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1d5c060740 0x7f1d5c060b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d5c0b1a50 0x7f1d5c140e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 0x7f1d5c141310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c141800 con 0x7f1d5c0b1a50 2021-11-21T21:54:23.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c144410 con 0x7f1d5c060740 2021-11-21T21:54:23.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1446b0 con 0x7f1d5c065b70 2021-11-21T21:54:23.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d5c0b1a50 0x7f1d5c140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d5c0b1a50 0x7f1d5c140e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44348/0 (socket says 172.21.2.116:44348) 2021-11-21T21:54:23.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 -- 172.21.2.116:0/2416167195 learned_addr learned my addr 172.21.2.116:0/2416167195 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:23.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 -- 172.21.2.116:0/2416167195 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 msgr2=0x7f1d5c141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 --2- 172.21.2.116:0/2416167195 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 0x7f1d5c141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 -- 172.21.2.116:0/2416167195 >> v2:172.21.2.117:3300/0 conn(0x7f1d5c060740 msgr2=0x7f1d5c060b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 --2- 172.21.2.116:0/2416167195 >> v2:172.21.2.117:3300/0 conn(0x7f1d5c060740 0x7f1d5c060b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 -- 172.21.2.116:0/2416167195 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c144950 con 0x7f1d5c0b1a50 2021-11-21T21:54:23.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5b7fe700 1 --2- 172.21.2.116:0/2416167195 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c0b1a50 0x7f1d5c140e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1d44002f70 tx=0x7f1d44006eb0).ready entity=mon.0 client_cookie=260f8de3f61b5a3f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5affd700 1 -- 172.21.2.116:0/2416167195 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d4400d570 con 0x7f1d5c0b1a50 2021-11-21T21:54:23.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5affd700 1 -- 172.21.2.116:0/2416167195 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44000e90 con 0x7f1d5c0b1a50 2021-11-21T21:54:23.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d5affd700 1 -- 172.21.2.116:0/2416167195 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d4400d900 con 0x7f1d5c0b1a50 2021-11-21T21:54:23.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- 172.21.2.116:0/2416167195 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c0b1a50 msgr2=0x7f1d5c140e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2416167195 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c0b1a50 0x7f1d5c140e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1d44002f70 tx=0x7f1d44006eb0).stop 2021-11-21T21:54:23.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- 172.21.2.116:0/2416167195 shutdown_connections 2021-11-21T21:54:23.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2416167195 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 0x7f1d5c141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2416167195 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c0b1a50 0x7f1d5c140e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2416167195 >> v2:172.21.2.117:3300/0 conn(0x7f1d5c060740 0x7f1d5c060b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- 172.21.2.116:0/2416167195 >> 172.21.2.116:0/2416167195 conn(0x7f1d5c0b7b80 msgr2=0x7f1d5c0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- 172.21.2.116:0/2416167195 shutdown_connections 2021-11-21T21:54:23.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.234+0000 7f1d6260d700 1 -- 172.21.2.116:0/2416167195 wait complete. 2021-11-21T21:54:23.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 Processor -- start 2021-11-21T21:54:23.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 -- start start 2021-11-21T21:54:23.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c060740 0x7f1d5c140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 0x7f1d5c09a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d5c0b1a50 0x7f1d5c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c09acf0 con 0x7f1d5c060740 2021-11-21T21:54:23.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c15a930 con 0x7f1d5c0b1a50 2021-11-21T21:54:23.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c15abd0 con 0x7f1d5c065b70 2021-11-21T21:54:23.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c060740 0x7f1d5c140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c060740 0x7f1d5c140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44350/0 (socket says 172.21.2.116:44350) 2021-11-21T21:54:23.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 -- 172.21.2.116:0/2837712269 learned_addr learned my addr 172.21.2.116:0/2837712269 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:23.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 -- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 msgr2=0x7f1d5c09a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 0x7f1d5c09a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 -- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d5c0b1a50 msgr2=0x7f1d5c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d5c0b1a50 0x7f1d5c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 -- 172.21.2.116:0/2837712269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4400b1e0 con 0x7f1d5c060740 2021-11-21T21:54:23.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5bfff700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c060740 0x7f1d5c140180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1d4c002f70 tx=0x7f1d4c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d597fa700 1 -- 172.21.2.116:0/2837712269 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d4c013460 con 0x7f1d5c060740 2021-11-21T21:54:23.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d597fa700 1 -- 172.21.2.116:0/2837712269 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c0135c0 con 0x7f1d5c060740 2021-11-21T21:54:23.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d597fa700 1 -- 172.21.2.116:0/2837712269 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d4c013920 con 0x7f1d5c060740 2021-11-21T21:54:23.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c09a360 con 0x7f1d5c060740 2021-11-21T21:54:23.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c0b1230 con 0x7f1d5c060740 2021-11-21T21:54:23.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c144bf0 con 0x7f1d5c060740 2021-11-21T21:54:23.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d597fa700 1 -- 172.21.2.116:0/2837712269 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1d4c01a060 con 0x7f1d5c060740 2021-11-21T21:54:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d597fa700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1d400536f0 0x7f1d40055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d597fa700 1 -- 172.21.2.116:0/2837712269 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1d4c06b1d0 con 0x7f1d5c060740 2021-11-21T21:54:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.238+0000 7f1d5b7fe700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1d400536f0 0x7f1d40055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.242+0000 7f1d5b7fe700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1d400536f0 0x7f1d40055bb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1d44002f70 tx=0x7f1d44015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.242+0000 7f1d597fa700 1 -- 172.21.2.116:0/2837712269 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d4c042550 con 0x7f1d5c060740 2021-11-21T21:54:23.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.322+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1d5c0573d0 con 0x7f1d5c060740 2021-11-21T21:54:23.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.322+0000 7f1d597fa700 1 -- 172.21.2.116:0/2837712269 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+143 (secure 0 0 0) 0x7f1d4c018030 con 0x7f1d5c060740 2021-11-21T21:54:23.328 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:23.328 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T21:54:23.328 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:23.329 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T21:54:23.329 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T21:54:23.329 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:23.329 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:23.330 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T21:54:23.330 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T21:54:23.330 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:23.330 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T21:54:23.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1d400536f0 msgr2=0x7f1d40055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1d400536f0 0x7f1d40055bb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1d44002f70 tx=0x7f1d44015000).stop 2021-11-21T21:54:23.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c060740 msgr2=0x7f1d5c140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c060740 0x7f1d5c140180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1d4c002f70 tx=0x7f1d4c006ec0).stop 2021-11-21T21:54:23.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 shutdown_connections 2021-11-21T21:54:23.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d5c0b1a50 0x7f1d5c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d5c065b70 0x7f1d5c09a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1d400536f0 0x7f1d40055bb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 --2- 172.21.2.116:0/2837712269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d5c060740 0x7f1d5c140180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 >> 172.21.2.116:0/2837712269 conn(0x7f1d5c0b7b80 msgr2=0x7f1d5c1421a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 shutdown_connections 2021-11-21T21:54:23.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.326+0000 7f1d6260d700 1 -- 172.21.2.116:0/2837712269 wait complete. 2021-11-21T21:54:23.335 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T21:54:23.336 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T21:54:23.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 Processor -- start 2021-11-21T21:54:23.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- start start 2021-11-21T21:54:23.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00b3690 0x7f72e00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e00b3c50 con 0x7f72e0144a10 2021-11-21T21:54:23.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e00b3ef0 con 0x7f72e00adc10 2021-11-21T21:54:23.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00b4190 con 0x7f72e00b3690 2021-11-21T21:54:23.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72ddd9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f72e0144a10 0x7f72e013a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56318/0 (socket says 172.21.2.116:56318) 2021-11-21T21:54:23.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72ddd9b700 1 -- 172.21.2.116:0/2059861069 learned_addr learned my addr 172.21.2.116:0/2059861069 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:23.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622429661 0 0) 0x7f72e00b3c50 con 0x7f72e0144a10 2021-11-21T21:54:23.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c0003260 con 0x7f72e0144a10 2021-11-21T21:54:23.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1412735159 0 0) 0x7f72c0003260 con 0x7f72e0144a10 2021-11-21T21:54:23.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00b3690 msgr2=0x7f72e00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00b3690 0x7f72e00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 >> v1:172.21.2.117:6789/0 conn(0x7f72e00adc10 legacy=0x7f72e0141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0140b80 con 0x7f72e0144a10 2021-11-21T21:54:23.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f72c8001680 con 0x7f72e0144a10 2021-11-21T21:54:23.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 >> v1:172.21.2.116:6789/0 conn(0x7f72e0144a10 legacy=0x7f72e013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72c0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00b3690 0x7f72c0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72c0004bf0 0x7f72c000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00b4190 con 0x7f72c0003800 2021-11-21T21:54:23.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c00052a0 con 0x7f72c0004bf0 2021-11-21T21:54:23.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c000bc40 con 0x7f72e00b3690 2021-11-21T21:54:23.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72de59c700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72c0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72de59c700 1 -- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00b3690 msgr2=0x7f72c0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72de59c700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00b3690 0x7f72c0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72de59c700 1 -- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72c0004bf0 msgr2=0x7f72c000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72de59c700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72c0004bf0 0x7f72c000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72de59c700 1 -- 172.21.2.116:0/2059861069 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0003260 con 0x7f72c0003800 2021-11-21T21:54:23.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72de59c700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72c0003c00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f72d0000f70 tx=0x7f72d00079e0).ready entity=mon.0 client_cookie=91c789503ef443d9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72d0007cf0 con 0x7f72c0003800 2021-11-21T21:54:23.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0014410 con 0x7f72c0003800 2021-11-21T21:54:23.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72dd59a700 1 -- 172.21.2.116:0/2059861069 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72d0014770 con 0x7f72c0003800 2021-11-21T21:54:23.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- 172.21.2.116:0/2059861069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 msgr2=0x7f72c0003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 --2- 172.21.2.116:0/2059861069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72c0003c00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f72d0000f70 tx=0x7f72d00079e0).stop 2021-11-21T21:54:23.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72ded9d700 1 -- 172.21.2.116:0/2059861069 reap_dead start 2021-11-21T21:54:23.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- 172.21.2.116:0/2059861069 shutdown_connections 2021-11-21T21:54:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- 172.21.2.116:0/2059861069 >> 172.21.2.116:0/2059861069 conn(0x7f72e005bd20 msgr2=0x7f72e005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- 172.21.2.116:0/2059861069 shutdown_connections 2021-11-21T21:54:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- 172.21.2.116:0/2059861069 wait complete. 2021-11-21T21:54:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 Processor -- start 2021-11-21T21:54:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.386+0000 7f72e4974700 1 -- start start 2021-11-21T21:54:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72e0067420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00af3e0 0x7f72e0151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72e0152420 0x7f72e00626f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00ae570 con 0x7f72c0003800 2021-11-21T21:54:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e01560d0 con 0x7f72e0152420 2021-11-21T21:54:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00afc80 con 0x7f72e00af3e0 2021-11-21T21:54:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72e0067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72e0067420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44370/0 (socket says 172.21.2.116:44370) 2021-11-21T21:54:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 -- 172.21.2.116:0/826074128 learned_addr learned my addr 172.21.2.116:0/826074128 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 -- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00af3e0 msgr2=0x7f72e0151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00af3e0 0x7f72e0151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 -- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72e0152420 msgr2=0x7f72e00626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72e0152420 0x7f72e00626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 -- 172.21.2.116:0/826074128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00aff20 con 0x7f72c0003800 2021-11-21T21:54:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72de59c700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72e0067420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f72d0000f70 tx=0x7f72d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72d77fe700 1 -- 172.21.2.116:0/826074128 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72d0013840 con 0x7f72c0003800 2021-11-21T21:54:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72d77fe700 1 -- 172.21.2.116:0/826074128 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d00139a0 con 0x7f72c0003800 2021-11-21T21:54:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72d77fe700 1 -- 172.21.2.116:0/826074128 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72d0013d00 con 0x7f72c0003800 2021-11-21T21:54:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e0155d90 con 0x7f72c0003800 2021-11-21T21:54:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e01528d0 con 0x7f72c0003800 2021-11-21T21:54:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e01562d0 con 0x7f72c0003800 2021-11-21T21:54:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72d77fe700 1 -- 172.21.2.116:0/826074128 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f72d0022470 con 0x7f72c0003800 2021-11-21T21:54:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72d77fe700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f72c40535d0 0x7f72c4055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72d77fe700 1 -- 172.21.2.116:0/826074128 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f72d001c040 con 0x7f72c0003800 2021-11-21T21:54:23.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.390+0000 7f72ddd9b700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f72c40535d0 0x7f72c4055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.394+0000 7f72ddd9b700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f72c40535d0 0x7f72c4055a90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f72e00b7000 tx=0x7f72e0065ed0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.394+0000 7f72d77fe700 1 -- 172.21.2.116:0/826074128 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72d0044030 con 0x7f72c0003800 2021-11-21T21:54:23.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.474+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f72e00b00b0 con 0x7f72c0003800 2021-11-21T21:54:23.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72d77fe700 1 -- 172.21.2.116:0/826074128 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f72d0047650 con 0x7f72c0003800 2021-11-21T21:54:23.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f72c40535d0 msgr2=0x7f72c4055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f72c40535d0 0x7f72c4055a90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f72e00b7000 tx=0x7f72e0065ed0).stop 2021-11-21T21:54:23.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 msgr2=0x7f72e0067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72e0067420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f72d0000f70 tx=0x7f72d0015000).stop 2021-11-21T21:54:23.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 shutdown_connections 2021-11-21T21:54:23.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72e0152420 0x7f72e00626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72e00af3e0 0x7f72e0151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f72c40535d0 0x7f72c4055a90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 --2- 172.21.2.116:0/826074128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f72c0003800 0x7f72e0067420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 >> 172.21.2.116:0/826074128 conn(0x7f72e005bd20 msgr2=0x7f72e013e710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 shutdown_connections 2021-11-21T21:54:23.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.714+0000 7f72e4974700 1 -- 172.21.2.116:0/826074128 wait complete. 2021-11-21T21:54:23.720 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T21:54:23.725 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T21:54:23.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0f48d7700 1 Processor -- start 2021-11-21T21:54:23.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0f48d7700 1 -- start start 2021-11-21T21:54:23.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0f48d7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 0x7fd0f01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0f48d7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd0f00abce0 0x7fd0f00b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0f48d7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f0147e80 con 0x7fd0f00b1db0 2021-11-21T21:54:23.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0f48d7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f00b1490 con 0x7fd0f00abce0 2021-11-21T21:54:23.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0f48d7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00af060 con 0x7fd0f0144810 2021-11-21T21:54:23.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0edd9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd0f00abce0 0x7fd0f00b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0edd9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd0f00abce0 0x7fd0f00b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44382/0 (socket says 172.21.2.116:44382) 2021-11-21T21:54:23.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0edd9b700 1 -- 172.21.2.116:0/2157521296 learned_addr learned my addr 172.21.2.116:0/2157521296 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:23.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0edd9b700 1 -- 172.21.2.116:0/2157521296 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 msgr2=0x7fd0f01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0edd9b700 1 --2- 172.21.2.116:0/2157521296 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 0x7fd0f01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0edd9b700 1 -- 172.21.2.116:0/2157521296 >> v1:172.21.2.117:6789/0 conn(0x7fd0f00b1db0 legacy=0x7fd0f00ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:23.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.782+0000 7fd0edd9b700 1 -- 172.21.2.116:0/2157521296 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00af5a0 con 0x7fd0f00abce0 2021-11-21T21:54:23.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0edd9b700 1 --2- 172.21.2.116:0/2157521296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00abce0 0x7fd0f00b18c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd0f00b2a30 tx=0x7fd0d8006eb0).ready entity=mon.0 client_cookie=1e55d685b196f581 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ed59a700 1 -- 172.21.2.116:0/2157521296 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d800d570 con 0x7fd0f00abce0 2021-11-21T21:54:23.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ed59a700 1 -- 172.21.2.116:0/2157521296 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d8000e90 con 0x7fd0f00abce0 2021-11-21T21:54:23.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ed59a700 1 -- 172.21.2.116:0/2157521296 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d800d8c0 con 0x7fd0f00abce0 2021-11-21T21:54:23.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/2157521296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00abce0 msgr2=0x7fd0f00b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/2157521296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00abce0 0x7fd0f00b18c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd0f00b2a30 tx=0x7fd0d8006eb0).stop 2021-11-21T21:54:23.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/2157521296 shutdown_connections 2021-11-21T21:54:23.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/2157521296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00abce0 0x7fd0f00b18c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/2157521296 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 0x7fd0f01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:23.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/2157521296 >> 172.21.2.116:0/2157521296 conn(0x7fd0f005bd20 msgr2=0x7fd0f005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:23.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/2157521296 shutdown_connections 2021-11-21T21:54:23.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/2157521296 wait complete. 2021-11-21T21:54:23.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 Processor -- start 2021-11-21T21:54:23.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- start start 2021-11-21T21:54:23.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd0f00abce0 0x7fd0f0159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 0x7fd0f0145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00670c0 0x7fd0f013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f014ad20 con 0x7fd0f00670c0 2021-11-21T21:54:23.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00667b0 con 0x7fd0f00abce0 2021-11-21T21:54:23.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00669c0 con 0x7fd0f0144810 2021-11-21T21:54:23.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0eed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00670c0 0x7fd0f013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0eed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00670c0 0x7fd0f013ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44388/0 (socket says 172.21.2.116:44388) 2021-11-21T21:54:23.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0eed9d700 1 -- 172.21.2.116:0/1492645264 learned_addr learned my addr 172.21.2.116:0/1492645264 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:23.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0edd9b700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 0x7fd0f0145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ee59c700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd0f00abce0 0x7fd0f0159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ee59c700 1 -- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 msgr2=0x7fd0f0145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ee59c700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 0x7fd0f0145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ee59c700 1 -- 172.21.2.116:0/1492645264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00670c0 msgr2=0x7fd0f013ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:23.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ee59c700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00670c0 0x7fd0f013ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:23.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ee59c700 1 -- 172.21.2.116:0/1492645264 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d800b1e0 con 0x7fd0f00abce0 2021-11-21T21:54:23.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0eed9d700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00670c0 0x7fd0f013ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:23.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0ee59c700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd0f00abce0 0x7fd0f0159db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fd0f0065c10 tx=0x7fd0e0006d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0e77fe700 1 -- 172.21.2.116:0/1492645264 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e0009ea0 con 0x7fd0f00abce0 2021-11-21T21:54:23.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0e77fe700 1 -- 172.21.2.116:0/1492645264 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e0012410 con 0x7fd0f00abce0 2021-11-21T21:54:23.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0e77fe700 1 -- 172.21.2.116:0/1492645264 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e0012770 con 0x7fd0f00abce0 2021-11-21T21:54:23.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f014af50 con 0x7fd0f00abce0 2021-11-21T21:54:23.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f0149e20 con 0x7fd0f00abce0 2021-11-21T21:54:23.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.786+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00af300 con 0x7fd0f00abce0 2021-11-21T21:54:23.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.790+0000 7fd0e77fe700 1 -- 172.21.2.116:0/1492645264 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd0e000f030 con 0x7fd0f00abce0 2021-11-21T21:54:23.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.790+0000 7fd0e77fe700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0d4053610 0x7fd0d4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:23.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.790+0000 7fd0e77fe700 1 -- 172.21.2.116:0/1492645264 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fd0e006b440 con 0x7fd0f00abce0 2021-11-21T21:54:23.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.790+0000 7fd0edd9b700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0d4053610 0x7fd0d4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:23.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.790+0000 7fd0edd9b700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0d4053610 0x7fd0d4055ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd0d801c000 tx=0x7fd0d8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:23.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.790+0000 7fd0e77fe700 1 -- 172.21.2.116:0/1492645264 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0e0042660 con 0x7fd0f00abce0 2021-11-21T21:54:23.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:23.874+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd0f00573d0 con 0x7fd0f00abce0 2021-11-21T21:54:24.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.718+0000 7fd0e77fe700 1 -- 172.21.2.116:0/1492645264 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v19) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd0e0045c80 con 0x7fd0f00abce0 2021-11-21T21:54:24.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0d4053610 msgr2=0x7fd0d4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0d4053610 0x7fd0d4055ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd0d801c000 tx=0x7fd0d8015000).stop 2021-11-21T21:54:24.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd0f00abce0 msgr2=0x7fd0f0159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd0f00abce0 0x7fd0f0159db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fd0f0065c10 tx=0x7fd0e0006d60).stop 2021-11-21T21:54:24.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 shutdown_connections 2021-11-21T21:54:24.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0d4053610 0x7fd0d4055ad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd0f00670c0 0x7fd0f013ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd0f0144810 0x7fd0f0145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 --2- 172.21.2.116:0/1492645264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd0f00abce0 0x7fd0f0159db0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 >> 172.21.2.116:0/1492645264 conn(0x7fd0f005bd20 msgr2=0x7fd0f00af780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:24.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 shutdown_connections 2021-11-21T21:54:24.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.722+0000 7fd0f48d7700 1 -- 172.21.2.116:0/1492645264 wait complete. 2021-11-21T21:54:24.727 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T21:54:24.730 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T21:54:24.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 Processor -- start 2021-11-21T21:54:24.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- start start 2021-11-21T21:54:24.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc2a80ad780 0x7fc2a805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc2a80af7e0 0x7fc2a813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a8143f70 con 0x7fc2a80affc0 2021-11-21T21:54:24.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a80adb60 con 0x7fc2a80ad780 2021-11-21T21:54:24.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a8143160 con 0x7fc2a80af7e0 2021-11-21T21:54:24.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc2a80ad780 0x7fc2a805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc2a80ad780 0x7fc2a805f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44394/0 (socket says 172.21.2.116:44394) 2021-11-21T21:54:24.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 -- 172.21.2.116:0/2345880047 learned_addr learned my addr 172.21.2.116:0/2345880047 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:24.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 -- 172.21.2.116:0/2345880047 >> v2:172.21.2.117:3300/0 conn(0x7fc2a80af7e0 msgr2=0x7fc2a813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:24.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 --2- 172.21.2.116:0/2345880047 >> v2:172.21.2.117:3300/0 conn(0x7fc2a80af7e0 0x7fc2a813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 -- 172.21.2.116:0/2345880047 >> v1:172.21.2.116:6789/0 conn(0x7fc2a80affc0 legacy=0x7fc2a8143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 -- 172.21.2.116:0/2345880047 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80b0ce0 con 0x7fc2a80ad780 2021-11-21T21:54:24.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a659c700 1 --2- 172.21.2.116:0/2345880047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a805f730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fc298002f70 tx=0x7fc298006eb0).ready entity=mon.0 client_cookie=e263f55f97788676 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:24.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a559a700 1 -- 172.21.2.116:0/2345880047 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc29800d610 con 0x7fc2a80ad780 2021-11-21T21:54:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a559a700 1 -- 172.21.2.116:0/2345880047 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298000e90 con 0x7fc2a80ad780 2021-11-21T21:54:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2a559a700 1 -- 172.21.2.116:0/2345880047 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc29800d9a0 con 0x7fc2a80ad780 2021-11-21T21:54:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- 172.21.2.116:0/2345880047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 msgr2=0x7fc2a805f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 --2- 172.21.2.116:0/2345880047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a805f730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fc298002f70 tx=0x7fc298006eb0).stop 2021-11-21T21:54:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- 172.21.2.116:0/2345880047 shutdown_connections 2021-11-21T21:54:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 --2- 172.21.2.116:0/2345880047 >> v2:172.21.2.117:3300/0 conn(0x7fc2a80af7e0 0x7fc2a813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 --2- 172.21.2.116:0/2345880047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a805f730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- 172.21.2.116:0/2345880047 >> 172.21.2.116:0/2345880047 conn(0x7fc2a80b6000 msgr2=0x7fc2a80b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- 172.21.2.116:0/2345880047 shutdown_connections 2021-11-21T21:54:24.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- 172.21.2.116:0/2345880047 wait complete. 2021-11-21T21:54:24.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 Processor -- start 2021-11-21T21:54:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.782+0000 7fc2ac818700 1 -- start start 2021-11-21T21:54:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a8064ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2a80af7e0 0x7fc2a813f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2a80affc0 0x7fc2a813fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a813d930 con 0x7fc2a80ad780 2021-11-21T21:54:24.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8067d30 con 0x7fc2a80af7e0 2021-11-21T21:54:24.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a813b460 con 0x7fc2a80affc0 2021-11-21T21:54:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a8064ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a8064ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44400/0 (socket says 172.21.2.116:44400) 2021-11-21T21:54:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 -- 172.21.2.116:0/356182026 learned_addr learned my addr 172.21.2.116:0/356182026 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 -- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2a80affc0 msgr2=0x7fc2a813fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2a80affc0 0x7fc2a813fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 -- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2a80af7e0 msgr2=0x7fc2a813f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2a80af7e0 0x7fc2a813f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 -- 172.21.2.116:0/356182026 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29800b1e0 con 0x7fc2a80ad780 2021-11-21T21:54:24.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a659c700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a8064ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc298002f70 tx=0x7fc298015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:24.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2977fe700 1 -- 172.21.2.116:0/356182026 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc29800d290 con 0x7fc2a80ad780 2021-11-21T21:54:24.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2977fe700 1 -- 172.21.2.116:0/356182026 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298006b60 con 0x7fc2a80ad780 2021-11-21T21:54:24.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a813b700 con 0x7fc2a80ad780 2021-11-21T21:54:24.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2977fe700 1 -- 172.21.2.116:0/356182026 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc29801e460 con 0x7fc2a80ad780 2021-11-21T21:54:24.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a8065920 con 0x7fc2a80ad780 2021-11-21T21:54:24.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a80b0a40 con 0x7fc2a80ad780 2021-11-21T21:54:24.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2977fe700 1 -- 172.21.2.116:0/356182026 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc298025030 con 0x7fc2a80ad780 2021-11-21T21:54:24.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2977fe700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc284053690 0x7fc284055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2977fe700 1 -- 172.21.2.116:0/356182026 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fc29806cbf0 con 0x7fc2a80ad780 2021-11-21T21:54:24.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a5d9b700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc284053690 0x7fc284055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.786+0000 7fc2a5d9b700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc284053690 0x7fc284055b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc2a8068820 tx=0x7fc29c007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:24.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.790+0000 7fc2977fe700 1 -- 172.21.2.116:0/356182026 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc298043dc0 con 0x7fc2a80ad780 2021-11-21T21:54:24.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.898+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fc2a80573d0 con 0x7fc2a80ad780 2021-11-21T21:54:24.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2977fe700 1 -- 172.21.2.116:0/356182026 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7fc2980230d0 con 0x7fc2a80ad780 2021-11-21T21:54:24.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc284053690 msgr2=0x7fc284055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc284053690 0x7fc284055b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc2a8068820 tx=0x7fc29c007810).stop 2021-11-21T21:54:24.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 msgr2=0x7fc2a8064ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a8064ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc298002f70 tx=0x7fc298015000).stop 2021-11-21T21:54:24.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 shutdown_connections 2021-11-21T21:54:24.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2a80affc0 0x7fc2a813fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2a80af7e0 0x7fc2a813f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc284053690 0x7fc284055b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 --2- 172.21.2.116:0/356182026 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2a80ad780 0x7fc2a8064ce0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 >> 172.21.2.116:0/356182026 conn(0x7fc2a80b6000 msgr2=0x7fc2a80b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:24.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 shutdown_connections 2021-11-21T21:54:24.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.906+0000 7fc2ac818700 1 -- 172.21.2.116:0/356182026 wait complete. 2021-11-21T21:54:24.913 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 3 and data pool 4 2021-11-21T21:54:24.917 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:24.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 Processor -- start 2021-11-21T21:54:24.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- start start 2021-11-21T21:54:24.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e80a6aa0 0x7f76e814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f76e8062410 0x7f76e81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e80a6e80 con 0x7f76e80a86b0 2021-11-21T21:54:24.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e80a8bc0 con 0x7f76e8062410 2021-11-21T21:54:24.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e814a7b0 con 0x7f76e80a6aa0 2021-11-21T21:54:24.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e67fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f76e80a86b0 0x7f76e81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56374/0 (socket says 172.21.2.116:56374) 2021-11-21T21:54:24.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e67fc700 1 -- 172.21.2.116:0/2917739847 learned_addr learned my addr 172.21.2.116:0/2917739847 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:24.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267037105 0 0) 0x7f76e80a6e80 con 0x7f76e80a86b0 2021-11-21T21:54:24.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c8003260 con 0x7f76e80a86b0 2021-11-21T21:54:24.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e77fe700 1 --2- 172.21.2.116:0/2917739847 >> v2:172.21.2.117:3300/0 conn(0x7f76e8062410 0x7f76e81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e80a6aa0 0x7f76e814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435703200 0 0) 0x7f76c8003260 con 0x7f76e80a86b0 2021-11-21T21:54:24.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e80a6aa0 msgr2=0x7f76e814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:24.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e80a6aa0 0x7f76e814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 >> v2:172.21.2.117:3300/0 conn(0x7f76e8062410 msgr2=0x7f76e81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:24.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 --2- 172.21.2.116:0/2917739847 >> v2:172.21.2.117:3300/0 conn(0x7f76e8062410 0x7f76e81443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e814aa10 con 0x7f76e80a86b0 2021-11-21T21:54:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f76d8001680 con 0x7f76e80a86b0 2021-11-21T21:54:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 >> v1:172.21.2.116:6789/0 conn(0x7f76e80a86b0 legacy=0x7f76e81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76c8003750 0x7f76c8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76c8004500 0x7f76c8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76c8004e70 0x7f76c800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80a8bc0 con 0x7f76c8003750 2021-11-21T21:54:24.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e814a7b0 con 0x7f76c8004500 2021-11-21T21:54:24.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c800bd70 con 0x7f76c8004e70 2021-11-21T21:54:24.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e80a6aa0 0x7f76e814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:24.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76c8003750 0x7f76c8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 -- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76c8004e70 msgr2=0x7f76c800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:24.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76c8004e70 0x7f76c800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 -- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76c8004500 msgr2=0x7f76c8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76c8004500 0x7f76c8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 -- 172.21.2.116:0/2917739847 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8003260 con 0x7f76c8003750 2021-11-21T21:54:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76c8003750 0x7f76c8003b70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f76d0007000 tx=0x7f76d0008040).ready entity=mon.0 client_cookie=3064c5a50e4545a2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d0012430 con 0x7f76c8003750 2021-11-21T21:54:24.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d000c030 con 0x7f76c8003750 2021-11-21T21:54:24.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e5ffb700 1 -- 172.21.2.116:0/2917739847 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d0012780 con 0x7f76c8003750 2021-11-21T21:54:24.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- 172.21.2.116:0/2917739847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76c8003750 msgr2=0x7f76c8003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 --2- 172.21.2.116:0/2917739847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76c8003750 0x7f76c8003b70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f76d0007000 tx=0x7f76d0008040).stop 2021-11-21T21:54:24.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e77fe700 1 -- 172.21.2.116:0/2917739847 reap_dead start 2021-11-21T21:54:24.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- 172.21.2.116:0/2917739847 shutdown_connections 2021-11-21T21:54:24.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- 172.21.2.116:0/2917739847 >> 172.21.2.116:0/2917739847 conn(0x7f76e805bd20 msgr2=0x7f76e805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:24.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- 172.21.2.116:0/2917739847 shutdown_connections 2021-11-21T21:54:24.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- 172.21.2.116:0/2917739847 wait complete. 2021-11-21T21:54:24.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 Processor -- start 2021-11-21T21:54:24.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- start start 2021-11-21T21:54:24.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76c8003750 0x7f76e8145330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76e8145ca0 0x7f76e8066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e8156c00 0x7f76e8067200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80676f0 con 0x7f76e8145ca0 2021-11-21T21:54:24.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8157170 con 0x7f76c8003750 2021-11-21T21:54:24.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76ed279700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8063e60 con 0x7f76e8156c00 2021-11-21T21:54:24.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76e8145ca0 0x7f76e8066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76e8145ca0 0x7f76e8066d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44432/0 (socket says 172.21.2.116:44432) 2021-11-21T21:54:24.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.966+0000 7f76e67fc700 1 -- 172.21.2.116:0/2823627410 learned_addr learned my addr 172.21.2.116:0/2823627410 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:24.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e67fc700 1 -- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e8156c00 msgr2=0x7f76e8067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e67fc700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e8156c00 0x7f76e8067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e67fc700 1 -- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76c8003750 msgr2=0x7f76e8145330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:24.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e67fc700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76c8003750 0x7f76e8145330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:24.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e67fc700 1 -- 172.21.2.116:0/2823627410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8064100 con 0x7f76e8145ca0 2021-11-21T21:54:24.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e67fc700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76e8145ca0 0x7f76e8066d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f76d8001390 tx=0x7f76d8001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:24.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76dffff700 1 -- 172.21.2.116:0/2823627410 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d8007170 con 0x7f76e8145ca0 2021-11-21T21:54:24.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76dffff700 1 -- 172.21.2.116:0/2823627410 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d80072d0 con 0x7f76e8145ca0 2021-11-21T21:54:24.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e80643a0 con 0x7f76e8145ca0 2021-11-21T21:54:24.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e80aab80 con 0x7f76e8145ca0 2021-11-21T21:54:24.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76dffff700 1 -- 172.21.2.116:0/2823627410 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d8004920 con 0x7f76e8145ca0 2021-11-21T21:54:24.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e80573d0 con 0x7f76e8145ca0 2021-11-21T21:54:24.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76dffff700 1 -- 172.21.2.116:0/2823627410 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f76d8004a80 con 0x7f76e8145ca0 2021-11-21T21:54:24.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76dffff700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76cc0535b0 0x7f76cc055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:24.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76dffff700 1 -- 172.21.2.116:0/2823627410 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f76d806b730 con 0x7f76e8145ca0 2021-11-21T21:54:24.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76cc0535b0 0x7f76cc055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:24.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.970+0000 7f76e6ffd700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76cc0535b0 0x7f76cc055a70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f76d0007000 tx=0x7f76d0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:24.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:24.974+0000 7f76dffff700 1 -- 172.21.2.116:0/2823627410 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76d8042720 con 0x7f76e8145ca0 2021-11-21T21:54:25.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76e805db70 con 0x7f76e8145ca0 2021-11-21T21:54:25.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76dffff700 1 -- 172.21.2.116:0/2823627410 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f76d8045d40 con 0x7f76e8145ca0 2021-11-21T21:54:25.061 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:25.061 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":20,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:24.904427+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:25.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76cc0535b0 msgr2=0x7f76cc055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76cc0535b0 0x7f76cc055a70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f76d0007000 tx=0x7f76d0008040).stop 2021-11-21T21:54:25.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76e8145ca0 msgr2=0x7f76e8066d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76e8145ca0 0x7f76e8066d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f76d8001390 tx=0x7f76d8001b50).stop 2021-11-21T21:54:25.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 shutdown_connections 2021-11-21T21:54:25.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76e8156c00 0x7f76e8067200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76cc0535b0 0x7f76cc055a70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76e8145ca0 0x7f76e8066d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 --2- 172.21.2.116:0/2823627410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76c8003750 0x7f76e8145330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 >> 172.21.2.116:0/2823627410 conn(0x7f76e805bd20 msgr2=0x7f76e81441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:25.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 shutdown_connections 2021-11-21T21:54:25.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.058+0000 7f76ed279700 1 -- 172.21.2.116:0/2823627410 wait complete. 2021-11-21T21:54:25.068 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:25.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f26190bc700 1 Processor -- start 2021-11-21T21:54:25.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f26190bc700 1 -- start start 2021-11-21T21:54:25.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f26190bc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f26140b3ca0 0x7f2614142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f26190bc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f26140b5730 0x7f261409b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f26190bc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2614143330 con 0x7f26140b67f0 2021-11-21T21:54:25.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f26190bc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2614143e60 con 0x7f26140b5730 2021-11-21T21:54:25.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f26190bc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140b6bd0 con 0x7f26140b3ca0 2021-11-21T21:54:25.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.114+0000 7f261359e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f26140b5730 0x7f261409b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f26140b5730 0x7f261409b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44436/0 (socket says 172.21.2.116:44436) 2021-11-21T21:54:25.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/3186492106 learned_addr learned my addr 172.21.2.116:0/3186492106 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:25.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/3186492106 >> v2:172.21.2.117:3300/0 conn(0x7f26140b3ca0 msgr2=0x7f2614142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:25.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- 172.21.2.116:0/3186492106 >> v2:172.21.2.117:3300/0 conn(0x7f26140b3ca0 0x7f2614142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/3186492106 >> v1:172.21.2.117:6789/0 conn(0x7f26140b67f0 legacy=0x7f26140b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:25.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/3186492106 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140ac990 con 0x7f26140b5730 2021-11-21T21:54:25.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- 172.21.2.116:0/3186492106 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b5730 0x7f261409b5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f2614068920 tx=0x7f260c009290).ready entity=mon.0 client_cookie=f393d42b0e51216a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f2611d9b700 1 -- 172.21.2.116:0/3186492106 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f260c00b830 con 0x7f26140b5730 2021-11-21T21:54:25.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f2611d9b700 1 -- 172.21.2.116:0/3186492106 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00b990 con 0x7f26140b5730 2021-11-21T21:54:25.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f2611d9b700 1 -- 172.21.2.116:0/3186492106 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f260c00bb40 con 0x7f26140b5730 2021-11-21T21:54:25.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/3186492106 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b5730 msgr2=0x7f261409b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 --2- 172.21.2.116:0/3186492106 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b5730 0x7f261409b5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f2614068920 tx=0x7f260c009290).stop 2021-11-21T21:54:25.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/3186492106 shutdown_connections 2021-11-21T21:54:25.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 --2- 172.21.2.116:0/3186492106 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b5730 0x7f261409b5a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 --2- 172.21.2.116:0/3186492106 >> v2:172.21.2.117:3300/0 conn(0x7f26140b3ca0 0x7f2614142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/3186492106 >> 172.21.2.116:0/3186492106 conn(0x7f26140b9d80 msgr2=0x7f26140b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:25.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/3186492106 shutdown_connections 2021-11-21T21:54:25.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/3186492106 wait complete. 2021-11-21T21:54:25.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 Processor -- start 2021-11-21T21:54:25.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- start start 2021-11-21T21:54:25.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26140b3ca0 0x7f261413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f26140b5730 0x7f26140b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b67f0 0x7f26140b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140af230 con 0x7f26140b67f0 2021-11-21T21:54:25.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140af4d0 con 0x7f26140b3ca0 2021-11-21T21:54:25.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140af770 con 0x7f26140b5730 2021-11-21T21:54:25.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b67f0 0x7f26140b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b67f0 0x7f26140b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44442/0 (socket says 172.21.2.116:44442) 2021-11-21T21:54:25.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/1820175918 learned_addr learned my addr 172.21.2.116:0/1820175918 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:25.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f2612d9d700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26140b3ca0 0x7f261413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f26140b5730 msgr2=0x7f26140b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f26140b5730 0x7f26140b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26140b3ca0 msgr2=0x7f261413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26140b3ca0 0x7f261413d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 -- 172.21.2.116:0/1820175918 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c009530 con 0x7f26140b67f0 2021-11-21T21:54:25.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f261359e700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b67f0 0x7f26140b3250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f2614068920 tx=0x7f260c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f260bfff700 1 -- 172.21.2.116:0/1820175918 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f260c00b830 con 0x7f26140b67f0 2021-11-21T21:54:25.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f260bfff700 1 -- 172.21.2.116:0/1820175918 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c020620 con 0x7f26140b67f0 2021-11-21T21:54:25.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f260bfff700 1 -- 172.21.2.116:0/1820175918 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f260c020980 con 0x7f26140b67f0 2021-11-21T21:54:25.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26140b28c0 con 0x7f26140b67f0 2021-11-21T21:54:25.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26140afb70 con 0x7f26140b67f0 2021-11-21T21:54:25.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26140b2a50 con 0x7f26140b67f0 2021-11-21T21:54:25.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.118+0000 7f260bfff700 1 -- 172.21.2.116:0/1820175918 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f260c026020 con 0x7f26140b67f0 2021-11-21T21:54:25.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.122+0000 7f260bfff700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25f80536a0 0x7f25f8055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.122+0000 7f260bfff700 1 -- 172.21.2.116:0/1820175918 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f260c06c660 con 0x7f26140b67f0 2021-11-21T21:54:25.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.122+0000 7f2612d9d700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25f80536a0 0x7f25f8055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.122+0000 7f2612d9d700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25f80536a0 0x7f25f8055b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f260400b000 tx=0x7f2604009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.122+0000 7f260bfff700 1 -- 172.21.2.116:0/1820175918 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f260c0437c0 con 0x7f26140b67f0 2021-11-21T21:54:25.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2614143330 con 0x7f26140b67f0 2021-11-21T21:54:25.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f260bfff700 1 -- 172.21.2.116:0/1820175918 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f2614143330 con 0x7f26140b67f0 2021-11-21T21:54:25.210 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:25.210 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":20,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:24.904427+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:25.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25f80536a0 msgr2=0x7f25f8055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25f80536a0 0x7f25f8055b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f260400b000 tx=0x7f2604009040).stop 2021-11-21T21:54:25.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b67f0 msgr2=0x7f26140b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b67f0 0x7f26140b3250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f2614068920 tx=0x7f260c016000).stop 2021-11-21T21:54:25.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 shutdown_connections 2021-11-21T21:54:25.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25f80536a0 0x7f25f8055b60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26140b67f0 0x7f26140b3250 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f26140b5730 0x7f26140b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 --2- 172.21.2.116:0/1820175918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26140b3ca0 0x7f261413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 >> 172.21.2.116:0/1820175918 conn(0x7f26140b9d80 msgr2=0x7f261413ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:25.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 shutdown_connections 2021-11-21T21:54:25.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.206+0000 7f26190bc700 1 -- 172.21.2.116:0/1820175918 wait complete. 2021-11-21T21:54:25.218 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T21:54:25.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 Processor -- start 2021-11-21T21:54:25.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- start start 2021-11-21T21:54:25.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1601426e0 0x7fc160142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc160062e60 0x7fc1600af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 0x7fc1600ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1600b0f60 con 0x7fc1601426e0 2021-11-21T21:54:25.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc160063470 con 0x7fc160062e60 2021-11-21T21:54:25.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc160143050 con 0x7fc1600ad470 2021-11-21T21:54:25.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1601426e0 0x7fc160142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1601426e0 0x7fc160142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44452/0 (socket says 172.21.2.116:44452) 2021-11-21T21:54:25.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 -- 172.21.2.116:0/1605455142 learned_addr learned my addr 172.21.2.116:0/1605455142 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:25.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 -- 172.21.2.116:0/1605455142 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 msgr2=0x7fc1600ad900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:25.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 --2- 172.21.2.116:0/1605455142 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 0x7fc1600ad900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 -- 172.21.2.116:0/1605455142 >> v2:172.21.2.117:3300/0 conn(0x7fc160062e60 msgr2=0x7fc1600af3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:25.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 --2- 172.21.2.116:0/1605455142 >> v2:172.21.2.117:3300/0 conn(0x7fc160062e60 0x7fc1600af3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 -- 172.21.2.116:0/1605455142 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1600b1e00 con 0x7fc1601426e0 2021-11-21T21:54:25.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc1656c0700 1 --2- 172.21.2.116:0/1605455142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160142ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc15c002f70 tx=0x7fc15c006e40).ready entity=mon.0 client_cookie=40fa6d0cee32345c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc14ffff700 1 -- 172.21.2.116:0/1605455142 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc15c00bcf0 con 0x7fc1601426e0 2021-11-21T21:54:25.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc14ffff700 1 -- 172.21.2.116:0/1605455142 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c000e90 con 0x7fc1601426e0 2021-11-21T21:54:25.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc14ffff700 1 -- 172.21.2.116:0/1605455142 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc15c0155e0 con 0x7fc1601426e0 2021-11-21T21:54:25.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- 172.21.2.116:0/1605455142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 msgr2=0x7fc160142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- 172.21.2.116:0/1605455142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160142ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc15c002f70 tx=0x7fc15c006e40).stop 2021-11-21T21:54:25.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- 172.21.2.116:0/1605455142 shutdown_connections 2021-11-21T21:54:25.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- 172.21.2.116:0/1605455142 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 0x7fc1600ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- 172.21.2.116:0/1605455142 >> v2:172.21.2.117:3300/0 conn(0x7fc160062e60 0x7fc1600af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- 172.21.2.116:0/1605455142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160142ac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- 172.21.2.116:0/1605455142 >> 172.21.2.116:0/1605455142 conn(0x7fc1600b5730 msgr2=0x7fc1600b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:25.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- 172.21.2.116:0/1605455142 shutdown_connections 2021-11-21T21:54:25.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- 172.21.2.116:0/1605455142 wait complete. 2021-11-21T21:54:25.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 Processor -- start 2021-11-21T21:54:25.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- start start 2021-11-21T21:54:25.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 0x7fc160151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc160066e40 0x7fc1600672d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc160151fe0 con 0x7fc1601426e0 2021-11-21T21:54:25.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc160145d60 con 0x7fc160066e40 2021-11-21T21:54:25.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc167924700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16014db90 con 0x7fc1600ad470 2021-11-21T21:54:25.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc164ebf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc164ebf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44456/0 (socket says 172.21.2.116:44456) 2021-11-21T21:54:25.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.266+0000 7fc164ebf700 1 -- 172.21.2.116:0/562873320 learned_addr learned my addr 172.21.2.116:0/562873320 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:25.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc164ebf700 1 -- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 msgr2=0x7fc160151af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:25.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc164ebf700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 0x7fc160151af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc164ebf700 1 -- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc160066e40 msgr2=0x7fc1600672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc164ebf700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc160066e40 0x7fc1600672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc164ebf700 1 -- 172.21.2.116:0/562873320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c00d4d0 con 0x7fc1601426e0 2021-11-21T21:54:25.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc164ebf700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160141900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc150002fb0 tx=0x7fc150006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc14e7fc700 1 -- 172.21.2.116:0/562873320 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc150013460 con 0x7fc1601426e0 2021-11-21T21:54:25.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc14e7fc700 1 -- 172.21.2.116:0/562873320 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1500135c0 con 0x7fc1601426e0 2021-11-21T21:54:25.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16013f770 con 0x7fc1601426e0 2021-11-21T21:54:25.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16013fcb0 con 0x7fc1601426e0 2021-11-21T21:54:25.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc14e7fc700 1 -- 172.21.2.116:0/562873320 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc150013920 con 0x7fc1601426e0 2021-11-21T21:54:25.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1600b13c0 con 0x7fc1601426e0 2021-11-21T21:54:25.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc14e7fc700 1 -- 172.21.2.116:0/562873320 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc15000f070 con 0x7fc1601426e0 2021-11-21T21:54:25.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc14e7fc700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc140053610 0x7fc140055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc14e7fc700 1 -- 172.21.2.116:0/562873320 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc15006b430 con 0x7fc1601426e0 2021-11-21T21:54:25.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc1656c0700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc140053610 0x7fc140055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.270+0000 7fc1656c0700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc140053610 0x7fc140055ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fc15c006fc0 tx=0x7fc15c018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.274+0000 7fc14e7fc700 1 -- 172.21.2.116:0/562873320 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc150042480 con 0x7fc1601426e0 2021-11-21T21:54:25.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.378+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc1600573d0 con 0x7fc1601426e0 2021-11-21T21:54:25.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc14e7fc700 1 -- 172.21.2.116:0/562873320 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc150045aa0 con 0x7fc1601426e0 2021-11-21T21:54:25.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc140053610 msgr2=0x7fc140055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc140053610 0x7fc140055ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fc15c006fc0 tx=0x7fc15c018000).stop 2021-11-21T21:54:25.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 msgr2=0x7fc160141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160141900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc150002fb0 tx=0x7fc150006df0).stop 2021-11-21T21:54:25.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 shutdown_connections 2021-11-21T21:54:25.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc160066e40 0x7fc1600672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc140053610 0x7fc140055ad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1601426e0 0x7fc160141900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 --2- 172.21.2.116:0/562873320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1600ad470 0x7fc160151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 >> 172.21.2.116:0/562873320 conn(0x7fc1600b5730 msgr2=0x7fc1600b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:25.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 shutdown_connections 2021-11-21T21:54:25.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.918+0000 7fc167924700 1 -- 172.21.2.116:0/562873320 wait complete. 2021-11-21T21:54:25.929 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:25.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bfb66700 1 Processor -- start 2021-11-21T21:54:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bfb66700 1 -- start start 2021-11-21T21:54:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bfb66700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bfb66700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa7b8062410 0x7fa7b81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bfb66700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b80a6e80 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bfb66700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b80a8bc0 con 0x7fa7b8062410 2021-11-21T21:54:25.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bfb66700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b814a7b0 con 0x7fa7b80a86b0 2021-11-21T21:54:25.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7be103700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa7b8062410 0x7fa7b81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:47810/0 (socket says 172.21.2.116:47810) 2021-11-21T21:54:25.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 -- 172.21.2.116:0/3267420521 learned_addr learned my addr 172.21.2.116:0/3267420521 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:25.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 -- 172.21.2.116:0/3267420521 >> v2:172.21.2.117:3300/0 conn(0x7fa7b8062410 msgr2=0x7fa7b81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:25.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 --2- 172.21.2.116:0/3267420521 >> v2:172.21.2.117:3300/0 conn(0x7fa7b8062410 0x7fa7b81443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 -- 172.21.2.116:0/3267420521 >> v1:172.21.2.117:6789/0 conn(0x7fa7b80a6aa0 legacy=0x7fa7b81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 -- 172.21.2.116:0/3267420521 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b814aa10 con 0x7fa7b80a86b0 2021-11-21T21:54:25.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.978+0000 7fa7bd101700 1 --2- 172.21.2.116:0/3267420521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b814b150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa7a8002f70 tx=0x7fa7a8006eb0).ready entity=mon.2 client_cookie=e108c8038ae7547a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bc900700 1 -- 172.21.2.116:0/3267420521 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a800d570 con 0x7fa7b80a86b0 2021-11-21T21:54:25.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bc900700 1 -- 172.21.2.116:0/3267420521 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a8000e90 con 0x7fa7b80a86b0 2021-11-21T21:54:25.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bc900700 1 -- 172.21.2.116:0/3267420521 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a800d900 con 0x7fa7b80a86b0 2021-11-21T21:54:25.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/3267420521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 msgr2=0x7fa7b814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/3267420521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b814b150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa7a8002f70 tx=0x7fa7a8006eb0).stop 2021-11-21T21:54:25.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/3267420521 shutdown_connections 2021-11-21T21:54:25.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/3267420521 >> v2:172.21.2.117:3300/0 conn(0x7fa7b8062410 0x7fa7b81443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/3267420521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b814b150 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:25.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/3267420521 >> 172.21.2.116:0/3267420521 conn(0x7fa7b805bd20 msgr2=0x7fa7b805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:25.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/3267420521 shutdown_connections 2021-11-21T21:54:25.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/3267420521 wait complete. 2021-11-21T21:54:25.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 Processor -- start 2021-11-21T21:54:25.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- start start 2021-11-21T21:54:25.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7b8062410 0x7fa7b8065b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7b80a6aa0 0x7fa7b81566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b8156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8066790 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8064890 con 0x7fa7b8062410 2021-11-21T21:54:25.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8067670 con 0x7fa7b80a86b0 2021-11-21T21:54:25.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7b80a6aa0 0x7fa7b81566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7b80a6aa0 0x7fa7b81566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44472/0 (socket says 172.21.2.116:44472) 2021-11-21T21:54:25.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 -- 172.21.2.116:0/536983369 learned_addr learned my addr 172.21.2.116:0/536983369 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:25.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 -- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 msgr2=0x7fa7b8156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b8156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 -- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7b8062410 msgr2=0x7fa7b8065b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:25.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7b8062410 0x7fa7b8065b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:25.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 -- 172.21.2.116:0/536983369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a800b1e0 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd101700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7b80a6aa0 0x7fa7b81566b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa7a8002f70 tx=0x7fa7a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:25.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7a6ffd700 1 -- 172.21.2.116:0/536983369 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a8011860 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7a6ffd700 1 -- 172.21.2.116:0/536983369 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a80119c0 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7a6ffd700 1 -- 172.21.2.116:0/536983369 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a8011d20 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b8063d70 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b8064a80 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b80573d0 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7a6ffd700 1 -- 172.21.2.116:0/536983369 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa7a801a040 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7a6ffd700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7980535b0 0x7fa798055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:25.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7a6ffd700 1 -- 172.21.2.116:0/536983369 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa7a806c860 con 0x7fa7b80a6aa0 2021-11-21T21:54:25.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.982+0000 7fa7bd902700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7980535b0 0x7fa798055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:25.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.986+0000 7fa7bd902700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7980535b0 0x7fa798055a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa7b8061ab0 tx=0x7fa7b40077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:25.986+0000 7fa7a6ffd700 1 -- 172.21.2.116:0/536983369 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7a80438b0 con 0x7fa7b80a6aa0 2021-11-21T21:54:26.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7b805db70 con 0x7fa7b80a6aa0 2021-11-21T21:54:26.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7a6ffd700 1 -- 172.21.2.116:0/536983369 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2483 (secure 0 0 0) 0x7fa7a8023090 con 0x7fa7b80a6aa0 2021-11-21T21:54:26.097 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:26.098 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/862809391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":862809391},{"type":"v1","addr":"172.21.2.117:6835","nonce":862809391}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2422183040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2422183040},{"type":"v1","addr":"172.21.2.116:6833","nonce":2422183040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:24.904554+0000","modified":"2021-11-21T21:54:25.909207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/2778157441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2778157441},{"type":"v1","addr":"172.21.2.116:6835","nonce":2778157441}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-11-21T21:54:26.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7980535b0 msgr2=0x7fa798055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7980535b0 0x7fa798055a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa7b8061ab0 tx=0x7fa7b40077f0).stop 2021-11-21T21:54:26.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7b80a6aa0 msgr2=0x7fa7b81566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7b80a6aa0 0x7fa7b81566b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa7a8002f70 tx=0x7fa7a8015000).stop 2021-11-21T21:54:26.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 shutdown_connections 2021-11-21T21:54:26.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7b80a86b0 0x7fa7b8156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7980535b0 0x7fa798055a70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7b80a6aa0 0x7fa7b81566b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 --2- 172.21.2.116:0/536983369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7b8062410 0x7fa7b8065b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 >> 172.21.2.116:0/536983369 conn(0x7fa7b805bd20 msgr2=0x7fa7b81426b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 shutdown_connections 2021-11-21T21:54:26.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.094+0000 7fa7bfb66700 1 -- 172.21.2.116:0/536983369 wait complete. 2021-11-21T21:54:26.101 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 5 2021-11-21T21:54:26.104 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:26.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcf723700 1 Processor -- start 2021-11-21T21:54:26.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcf723700 1 -- start start 2021-11-21T21:54:26.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcf723700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 0x7f7fc8144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcf723700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc80a6e80 con 0x7f7fc80a86b0 2021-11-21T21:54:26.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcf723700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc814a7b0 con 0x7f7fc80a6aa0 2021-11-21T21:54:26.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcf723700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80a8bc0 con 0x7f7fc8062410 2021-11-21T21:54:26.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcccbe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7fc80a86b0 0x7f7fc81443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56436/0 (socket says 172.21.2.116:56436) 2021-11-21T21:54:26.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcccbe700 1 -- 172.21.2.116:0/691384237 learned_addr learned my addr 172.21.2.116:0/691384237 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:26.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220198591 0 0) 0x7f7fc80a6e80 con 0x7f7fc80a86b0 2021-11-21T21:54:26.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb0003260 con 0x7f7fc80a86b0 2021-11-21T21:54:26.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fcdcc0700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 0x7f7fc8144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671862910 0 0) 0x7f7fb0003260 con 0x7f7fc80a86b0 2021-11-21T21:54:26.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 msgr2=0x7f7fc8144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.150+0000 7f7fbbfff700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 0x7f7fc8144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 >> v1:172.21.2.117:6789/0 conn(0x7f7fc80a6aa0 legacy=0x7f7fc81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc814aa10 con 0x7f7fc80a86b0 2021-11-21T21:54:26.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f7fbc001680 con 0x7f7fc80a86b0 2021-11-21T21:54:26.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 >> v1:172.21.2.116:6789/0 conn(0x7f7fc80a86b0 legacy=0x7f7fc81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 0x7f7fb0003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fb0003e10 0x7f7fb0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fb0004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80a8bc0 con 0x7f7fb0004b10 2021-11-21T21:54:26.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb000b960 con 0x7f7fb0003e10 2021-11-21T21:54:26.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb000bb20 con 0x7f7fc8062410 2021-11-21T21:54:26.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fb0004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 0x7f7fb0003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcccbe700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fb0003e10 0x7f7fb0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 -- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 msgr2=0x7f7fb0003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 0x7f7fb0003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 -- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fb0003e10 msgr2=0x7f7fb0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fb0003e10 0x7f7fb0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 -- 172.21.2.116:0/691384237 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb0003260 con 0x7f7fb0004b10 2021-11-21T21:54:26.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc8062410 0x7f7fb0003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:26.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcccbe700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fb0003e10 0x7f7fb0004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:26.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fb0004f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f7fc0007000 tx=0x7f7fc0008040).ready entity=mon.0 client_cookie=db5c271290a53458 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fc000c640 con 0x7f7fb0004b10 2021-11-21T21:54:26.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000c7a0 con 0x7f7fb0004b10 2021-11-21T21:54:26.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fbbfff700 1 -- 172.21.2.116:0/691384237 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fc000cb00 con 0x7f7fb0004b10 2021-11-21T21:54:26.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- 172.21.2.116:0/691384237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 msgr2=0x7f7fb0004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 --2- 172.21.2.116:0/691384237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fb0004f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f7fc0007000 tx=0x7f7fc0008040).stop 2021-11-21T21:54:26.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 -- 172.21.2.116:0/691384237 reap_dead start 2021-11-21T21:54:26.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- 172.21.2.116:0/691384237 shutdown_connections 2021-11-21T21:54:26.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- 172.21.2.116:0/691384237 >> 172.21.2.116:0/691384237 conn(0x7f7fc805bd20 msgr2=0x7f7fc805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- 172.21.2.116:0/691384237 shutdown_connections 2021-11-21T21:54:26.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- 172.21.2.116:0/691384237 wait complete. 2021-11-21T21:54:26.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 Processor -- start 2021-11-21T21:54:26.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- start start 2021-11-21T21:54:26.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fc814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc814c690 0x7f7fc8147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fc8147a10 0x7f7fc8159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc814d050 con 0x7f7fb0004b10 2021-11-21T21:54:26.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc8145170 con 0x7f7fc8147a10 2021-11-21T21:54:26.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc814cad0 con 0x7f7fc814c690 2021-11-21T21:54:26.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fc814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fc814c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44490/0 (socket says 172.21.2.116:44490) 2021-11-21T21:54:26.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 -- 172.21.2.116:0/1294715735 learned_addr learned my addr 172.21.2.116:0/1294715735 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcdcc0700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fc8147a10 0x7f7fc8159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 -- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc814c690 msgr2=0x7f7fc8147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc814c690 0x7f7fc8147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 -- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fc8147a10 msgr2=0x7f7fc8159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fc8147a10 0x7f7fc8159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 -- 172.21.2.116:0/1294715735 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8147e90 con 0x7f7fb0004b10 2021-11-21T21:54:26.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcd4bf700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fc814c1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7fc4008000 tx=0x7f7fc4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fba7fc700 1 -- 172.21.2.116:0/1294715735 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fc4007e60 con 0x7f7fb0004b10 2021-11-21T21:54:26.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fba7fc700 1 -- 172.21.2.116:0/1294715735 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc4000e90 con 0x7f7fb0004b10 2021-11-21T21:54:26.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fba7fc700 1 -- 172.21.2.116:0/1294715735 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fc40106d0 con 0x7f7fb0004b10 2021-11-21T21:54:26.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc8159830 con 0x7f7fb0004b10 2021-11-21T21:54:26.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.154+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc814bc20 con 0x7f7fb0004b10 2021-11-21T21:54:26.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.158+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc8145330 con 0x7f7fb0004b10 2021-11-21T21:54:26.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.158+0000 7f7fba7fc700 1 -- 172.21.2.116:0/1294715735 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7fc4019030 con 0x7f7fb0004b10 2021-11-21T21:54:26.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.158+0000 7f7fba7fc700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fa80536d0 0x7f7fa8055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.158+0000 7f7fba7fc700 1 -- 172.21.2.116:0/1294715735 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7fc406aff0 con 0x7f7fb0004b10 2021-11-21T21:54:26.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.158+0000 7f7fcccbe700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fa80536d0 0x7f7fa8055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.158+0000 7f7fcccbe700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fa80536d0 0x7f7fa8055b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7fbc008000 tx=0x7f7fbc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.158+0000 7f7fba7fc700 1 -- 172.21.2.116:0/1294715735 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7fc4042040 con 0x7f7fb0004b10 2021-11-21T21:54:26.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fc80573d0 con 0x7f7fb0004b10 2021-11-21T21:54:26.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fba7fc700 1 -- 172.21.2.116:0/1294715735 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f7fc4045660 con 0x7f7fb0004b10 2021-11-21T21:54:26.246 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:26.247 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":21,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:25.908725+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:26.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fa80536d0 msgr2=0x7f7fa8055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fa80536d0 0x7f7fa8055b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7fbc008000 tx=0x7f7fbc009040).stop 2021-11-21T21:54:26.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 msgr2=0x7f7fc814c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fc814c1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7fc4008000 tx=0x7f7fc4009040).stop 2021-11-21T21:54:26.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 shutdown_connections 2021-11-21T21:54:26.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7fc8147a10 0x7f7fc8159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7fc814c690 0x7f7fc8147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fa80536d0 0x7f7fa8055b90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 --2- 172.21.2.116:0/1294715735 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7fb0004b10 0x7f7fc814c1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 >> 172.21.2.116:0/1294715735 conn(0x7f7fc805bd20 msgr2=0x7f7fc8149c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 shutdown_connections 2021-11-21T21:54:26.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.242+0000 7f7fcf723700 1 -- 172.21.2.116:0/1294715735 wait complete. 2021-11-21T21:54:26.254 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:26.255 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-11-21T21:54:26.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 Processor -- start 2021-11-21T21:54:26.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- start start 2021-11-21T21:54:26.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55a80a6aa0 0x7f55a814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a80a6e80 con 0x7f55a8062410 2021-11-21T21:54:26.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a814a7b0 con 0x7f55a80a86b0 2021-11-21T21:54:26.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a80a8bc0 con 0x7f55a80a6aa0 2021-11-21T21:54:26.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55a80a6aa0 0x7f55a814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55a80a6aa0 0x7f55a814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44498/0 (socket says 172.21.2.116:44498) 2021-11-21T21:54:26.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a77fe700 1 -- 172.21.2.116:0/2244594456 learned_addr learned my addr 172.21.2.116:0/2244594456 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a77fe700 1 -- 172.21.2.116:0/2244594456 >> v1:172.21.2.117:6789/0 conn(0x7f55a80a86b0 legacy=0x7f55a81470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:26.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a77fe700 1 -- 172.21.2.116:0/2244594456 >> v1:172.21.2.116:6789/0 conn(0x7f55a8062410 legacy=0x7f55a81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a77fe700 1 -- 172.21.2.116:0/2244594456 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a814aa10 con 0x7f55a80a6aa0 2021-11-21T21:54:26.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a77fe700 1 --2- 172.21.2.116:0/2244594456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a814b150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f55a8061b50 tx=0x7f559c006eb0).ready entity=mon.0 client_cookie=ed910bee650c4443 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a67fc700 1 -- 172.21.2.116:0/2244594456 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f559c00d610 con 0x7f55a80a6aa0 2021-11-21T21:54:26.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a67fc700 1 -- 172.21.2.116:0/2244594456 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559c000e90 con 0x7f55a80a6aa0 2021-11-21T21:54:26.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55a67fc700 1 -- 172.21.2.116:0/2244594456 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f559c00d960 con 0x7f55a80a6aa0 2021-11-21T21:54:26.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- 172.21.2.116:0/2244594456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 msgr2=0x7f55a814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 --2- 172.21.2.116:0/2244594456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a814b150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f55a8061b50 tx=0x7f559c006eb0).stop 2021-11-21T21:54:26.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- 172.21.2.116:0/2244594456 shutdown_connections 2021-11-21T21:54:26.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 --2- 172.21.2.116:0/2244594456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a814b150 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- 172.21.2.116:0/2244594456 >> 172.21.2.116:0/2244594456 conn(0x7f55a805bd20 msgr2=0x7f55a805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- 172.21.2.116:0/2244594456 shutdown_connections 2021-11-21T21:54:26.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.302+0000 7f55ada92700 1 -- 172.21.2.116:0/2244594456 wait complete. 2021-11-21T21:54:26.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 Processor -- start 2021-11-21T21:54:26.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 -- start start 2021-11-21T21:54:26.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a8062410 0x7f55a80b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a8140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a80a86b0 0x7f55a8140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80aac30 con 0x7f55a80a6aa0 2021-11-21T21:54:26.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80aaed0 con 0x7f55a80a86b0 2021-11-21T21:54:26.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80ab170 con 0x7f55a8062410 2021-11-21T21:54:26.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a8140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a8140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44504/0 (socket says 172.21.2.116:44504) 2021-11-21T21:54:26.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 -- 172.21.2.116:0/3938115815 learned_addr learned my addr 172.21.2.116:0/3938115815 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 -- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a8062410 msgr2=0x7f55a80b3ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:26.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a8062410 0x7f55a80b3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 -- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a80a86b0 msgr2=0x7f55a8140670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:26.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a80a86b0 0x7f55a8140670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 -- 172.21.2.116:0/3938115815 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c00b1e0 con 0x7f55a80a6aa0 2021-11-21T21:54:26.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a6ffd700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a8140180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5598002fb0 tx=0x7f5598006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a4ff9700 1 -- 172.21.2.116:0/3938115815 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5598013460 con 0x7f55a80a6aa0 2021-11-21T21:54:26.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a4ff9700 1 -- 172.21.2.116:0/3938115815 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55980135c0 con 0x7f55a80a6aa0 2021-11-21T21:54:26.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a80ab410 con 0x7f55a80a6aa0 2021-11-21T21:54:26.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a4ff9700 1 -- 172.21.2.116:0/3938115815 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5598013920 con 0x7f55a80a6aa0 2021-11-21T21:54:26.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8153eb0 con 0x7f55a80a6aa0 2021-11-21T21:54:26.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a80ab5a0 con 0x7f55a80a6aa0 2021-11-21T21:54:26.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a4ff9700 1 -- 172.21.2.116:0/3938115815 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f559800f070 con 0x7f55a80a6aa0 2021-11-21T21:54:26.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a4ff9700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053580 0x7f5588055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a4ff9700 1 -- 172.21.2.116:0/3938115815 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f559806b0c0 con 0x7f55a80a6aa0 2021-11-21T21:54:26.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a77fe700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053580 0x7f5588055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.306+0000 7f55a77fe700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053580 0x7f5588055a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f55a8061b50 tx=0x7f559c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.310+0000 7f55a4ff9700 1 -- 172.21.2.116:0/3938115815 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5598042220 con 0x7f55a80a6aa0 2021-11-21T21:54:26.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.390+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55a80573d0 con 0x7f55a80a6aa0 2021-11-21T21:54:26.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55a4ff9700 1 -- 172.21.2.116:0/3938115815 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f5598018030 con 0x7f55a80a6aa0 2021-11-21T21:54:26.397 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:26.397 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":21,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:25.908725+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:26.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053580 msgr2=0x7f5588055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053580 0x7f5588055a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f55a8061b50 tx=0x7f559c015000).stop 2021-11-21T21:54:26.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 msgr2=0x7f55a8140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a8140180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5598002fb0 tx=0x7f5598006df0).stop 2021-11-21T21:54:26.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 shutdown_connections 2021-11-21T21:54:26.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a80a86b0 0x7f55a8140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053580 0x7f5588055a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80a6aa0 0x7f55a8140180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 --2- 172.21.2.116:0/3938115815 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a8062410 0x7f55a80b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 >> 172.21.2.116:0/3938115815 conn(0x7f55a805bd20 msgr2=0x7f55a80b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 shutdown_connections 2021-11-21T21:54:26.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.394+0000 7f55ada92700 1 -- 172.21.2.116:0/3938115815 wait complete. 2021-11-21T21:54:26.407 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:26.408 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:26.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 Processor -- start 2021-11-21T21:54:26.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- start start 2021-11-21T21:54:26.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd930144a10 0x7fd9300b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9300b3c50 con 0x7fd9300b3690 2021-11-21T21:54:26.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9300b3ef0 con 0x7fd9300adc10 2021-11-21T21:54:26.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300b4190 con 0x7fd930144a10 2021-11-21T21:54:26.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936872700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd9300b3690 0x7fd93013a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56470/0 (socket says 172.21.2.116:56470) 2021-11-21T21:54:26.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936872700 1 -- 172.21.2.116:0/978329966 learned_addr learned my addr 172.21.2.116:0/978329966 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:26.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971156075 0 0) 0x7fd9300b3c50 con 0x7fd9300b3690 2021-11-21T21:54:26.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd918003260 con 0x7fd9300b3690 2021-11-21T21:54:26.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119229117 0 0) 0x7fd918003260 con 0x7fd9300b3690 2021-11-21T21:54:26.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd930144a10 msgr2=0x7fd9300b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd930144a10 0x7fd9300b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 >> v1:172.21.2.117:6789/0 conn(0x7fd9300adc10 legacy=0x7fd930141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930140b80 con 0x7fd9300b3690 2021-11-21T21:54:26.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd9280019e0 con 0x7fd9300b3690 2021-11-21T21:54:26.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 >> v1:172.21.2.116:6789/0 conn(0x7fd9300b3690 legacy=0x7fd93013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd918003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd918004500 0x7fd918004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd918004f90 0x7fd91800fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300b4190 con 0x7fd918003800 2021-11-21T21:54:26.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918010260 con 0x7fd918004f90 2021-11-21T21:54:26.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd918010420 con 0x7fd918004500 2021-11-21T21:54:26.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936071700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd918003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936071700 1 -- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd918004500 msgr2=0x7fd918004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936071700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd918004500 0x7fd918004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936071700 1 -- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd918004f90 msgr2=0x7fd91800fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936071700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd918004f90 0x7fd91800fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936071700 1 -- 172.21.2.116:0/978329966 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930140b80 con 0x7fd918003800 2021-11-21T21:54:26.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936071700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd918003c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd92c000f70 tx=0x7fd92c0079e0).ready entity=mon.0 client_cookie=86a1c17bb0c6f93 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd92c014410 con 0x7fd918003800 2021-11-21T21:54:26.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c014570 con 0x7fd918003800 2021-11-21T21:54:26.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd93506f700 1 -- 172.21.2.116:0/978329966 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd92c0148d0 con 0x7fd918003800 2021-11-21T21:54:26.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- 172.21.2.116:0/978329966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 msgr2=0x7fd918003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 --2- 172.21.2.116:0/978329966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd918003c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd92c000f70 tx=0x7fd92c0079e0).stop 2021-11-21T21:54:26.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd936872700 1 -- 172.21.2.116:0/978329966 reap_dead start 2021-11-21T21:54:26.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- 172.21.2.116:0/978329966 shutdown_connections 2021-11-21T21:54:26.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- 172.21.2.116:0/978329966 >> 172.21.2.116:0/978329966 conn(0x7fd93005bd20 msgr2=0x7fd93005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- 172.21.2.116:0/978329966 shutdown_connections 2021-11-21T21:54:26.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- 172.21.2.116:0/978329966 wait complete. 2021-11-21T21:54:26.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 Processor -- start 2021-11-21T21:54:26.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.458+0000 7fd9382d5700 1 -- start start 2021-11-21T21:54:26.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd9300b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd930062a20 0x7fd9300b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd930061760 0x7fd93013bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93005ae20 con 0x7fd918003800 2021-11-21T21:54:26.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930062810 con 0x7fd930062a20 2021-11-21T21:54:26.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9301102a0 con 0x7fd930061760 2021-11-21T21:54:26.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd9300b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd9300b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44524/0 (socket says 172.21.2.116:44524) 2021-11-21T21:54:26.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 -- 172.21.2.116:0/4044245685 learned_addr learned my addr 172.21.2.116:0/4044245685 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 -- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd930061760 msgr2=0x7fd93013bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd930061760 0x7fd93013bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 -- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd930062a20 msgr2=0x7fd9300b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd930062a20 0x7fd9300b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 -- 172.21.2.116:0/4044245685 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930110540 con 0x7fd918003800 2021-11-21T21:54:26.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd936071700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd9300b5ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd92c000f70 tx=0x7fd92c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd91f7fe700 1 -- 172.21.2.116:0/4044245685 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd92c014410 con 0x7fd918003800 2021-11-21T21:54:26.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd91f7fe700 1 -- 172.21.2.116:0/4044245685 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c013e90 con 0x7fd918003800 2021-11-21T21:54:26.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd91f7fe700 1 -- 172.21.2.116:0/4044245685 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd92c022660 con 0x7fd918003800 2021-11-21T21:54:26.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93014e350 con 0x7fd918003800 2021-11-21T21:54:26.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9300b13a0 con 0x7fd918003800 2021-11-21T21:54:26.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd930061db0 con 0x7fd918003800 2021-11-21T21:54:26.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd91f7fe700 1 -- 172.21.2.116:0/4044245685 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd92c0227c0 con 0x7fd918003800 2021-11-21T21:54:26.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd91f7fe700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd910053660 0x7fd910055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd91f7fe700 1 -- 172.21.2.116:0/4044245685 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd92c01c040 con 0x7fd918003800 2021-11-21T21:54:26.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd935870700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd910053660 0x7fd910055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.462+0000 7fd935870700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd910053660 0x7fd910055b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd920000b60 tx=0x7fd920006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.466+0000 7fd91f7fe700 1 -- 172.21.2.116:0/4044245685 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd92c044050 con 0x7fd918003800 2021-11-21T21:54:26.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.546+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd93014e7f0 con 0x7fd918003800 2021-11-21T21:54:26.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd91f7fe700 1 -- 172.21.2.116:0/4044245685 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd93014e7f0 con 0x7fd918003800 2021-11-21T21:54:26.553 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:26.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd910053660 msgr2=0x7fd910055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd910053660 0x7fd910055b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd920000b60 tx=0x7fd920006a00).stop 2021-11-21T21:54:26.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 msgr2=0x7fd9300b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd9300b5ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd92c000f70 tx=0x7fd92c015000).stop 2021-11-21T21:54:26.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 shutdown_connections 2021-11-21T21:54:26.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd930061760 0x7fd93013bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd930062a20 0x7fd9300b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd910053660 0x7fd910055b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 --2- 172.21.2.116:0/4044245685 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd918003800 0x7fd9300b5ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 >> 172.21.2.116:0/4044245685 conn(0x7fd93005bd20 msgr2=0x7fd9300b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 shutdown_connections 2021-11-21T21:54:26.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.550+0000 7fd9382d5700 1 -- 172.21.2.116:0/4044245685 wait complete. 2021-11-21T21:54:26.559 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:26.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 Processor -- start 2021-11-21T21:54:26.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- start start 2021-11-21T21:54:26.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8394144a10 0x7f83940b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f83940b3690 0x7f839413a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83940b3c50 con 0x7f83940adc10 2021-11-21T21:54:26.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83940b3ef0 con 0x7f83940b3690 2021-11-21T21:54:26.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83940b4190 con 0x7f8394144a10 2021-11-21T21:54:26.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f83940b3690 0x7f839413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f83940b3690 0x7f839413a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44536/0 (socket says 172.21.2.116:44536) 2021-11-21T21:54:26.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 -- 172.21.2.116:0/2595066231 learned_addr learned my addr 172.21.2.116:0/2595066231 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 -- 172.21.2.116:0/2595066231 >> v2:172.21.2.117:3300/0 conn(0x7f8394144a10 msgr2=0x7f83940b2030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:26.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 --2- 172.21.2.116:0/2595066231 >> v2:172.21.2.117:3300/0 conn(0x7f8394144a10 0x7f83940b2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 -- 172.21.2.116:0/2595066231 >> v1:172.21.2.117:6789/0 conn(0x7f83940adc10 legacy=0x7f8394141020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:26.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 -- 172.21.2.116:0/2595066231 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8394140b80 con 0x7f83940b3690 2021-11-21T21:54:26.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8392d9d700 1 --2- 172.21.2.116:0/2595066231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f839413a540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f839405f430 tx=0x7f838c009290).ready entity=mon.0 client_cookie=6b7ec56271a82c7b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8391d9b700 1 -- 172.21.2.116:0/2595066231 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f838c00b830 con 0x7f83940b3690 2021-11-21T21:54:26.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8391d9b700 1 -- 172.21.2.116:0/2595066231 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c00b990 con 0x7f83940b3690 2021-11-21T21:54:26.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8391d9b700 1 -- 172.21.2.116:0/2595066231 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f838c00bb40 con 0x7f83940b3690 2021-11-21T21:54:26.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- 172.21.2.116:0/2595066231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 msgr2=0x7f839413a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- 172.21.2.116:0/2595066231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f839413a540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f839405f430 tx=0x7f838c009290).stop 2021-11-21T21:54:26.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- 172.21.2.116:0/2595066231 shutdown_connections 2021-11-21T21:54:26.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- 172.21.2.116:0/2595066231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f839413a540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- 172.21.2.116:0/2595066231 >> v2:172.21.2.117:3300/0 conn(0x7f8394144a10 0x7f83940b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- 172.21.2.116:0/2595066231 >> 172.21.2.116:0/2595066231 conn(0x7f839405bd20 msgr2=0x7f839405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- 172.21.2.116:0/2595066231 shutdown_connections 2021-11-21T21:54:26.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- 172.21.2.116:0/2595066231 wait complete. 2021-11-21T21:54:26.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 Processor -- start 2021-11-21T21:54:26.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- start start 2021-11-21T21:54:26.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83940adc10 0x7f83941506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f83940b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8394144a10 0x7f83940af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83941139a0 con 0x7f83940b3690 2021-11-21T21:54:26.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8394150bc0 con 0x7f83940adc10 2021-11-21T21:54:26.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83941131c0 con 0x7f8394144a10 2021-11-21T21:54:26.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f83940b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f83940b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44540/0 (socket says 172.21.2.116:44540) 2021-11-21T21:54:26.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 -- 172.21.2.116:0/554320702 learned_addr learned my addr 172.21.2.116:0/554320702 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 -- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8394144a10 msgr2=0x7f83940af2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:26.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8394144a10 0x7f83940af2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 -- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83940adc10 msgr2=0x7f83941506d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:26.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83940adc10 0x7f83941506d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 -- 172.21.2.116:0/554320702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c009530 con 0x7f83940b3690 2021-11-21T21:54:26.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838bfff700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f83940b0400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f837c000ed0 tx=0x7f837c006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838b7fe700 1 -- 172.21.2.116:0/554320702 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c009e00 con 0x7f83940b3690 2021-11-21T21:54:26.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838b7fe700 1 -- 172.21.2.116:0/554320702 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c012410 con 0x7f83940b3690 2021-11-21T21:54:26.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f838b7fe700 1 -- 172.21.2.116:0/554320702 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c012770 con 0x7f83940b3690 2021-11-21T21:54:26.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8394113460 con 0x7f83940b3690 2021-11-21T21:54:26.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.614+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839414d8f0 con 0x7f83940b3690 2021-11-21T21:54:26.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.618+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83941102a0 con 0x7f83940b3690 2021-11-21T21:54:26.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.618+0000 7f838b7fe700 1 -- 172.21.2.116:0/554320702 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f837c00f030 con 0x7f83940b3690 2021-11-21T21:54:26.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.618+0000 7f838b7fe700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83780536f0 0x7f8378055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.618+0000 7f838b7fe700 1 -- 172.21.2.116:0/554320702 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f837c06b520 con 0x7f83940b3690 2021-11-21T21:54:26.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.618+0000 7f839259c700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83780536f0 0x7f8378055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.618+0000 7f839259c700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83780536f0 0x7f8378055bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8384002fb0 tx=0x7f8384006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.618+0000 7f838b7fe700 1 -- 172.21.2.116:0/554320702 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f837c042680 con 0x7f83940b3690 2021-11-21T21:54:26.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f83940573d0 con 0x7f83940b3690 2021-11-21T21:54:26.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f838b7fe700 1 -- 172.21.2.116:0/554320702 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f837c045ca0 con 0x7f83940b3690 2021-11-21T21:54:26.707 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:26.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83780536f0 msgr2=0x7f8378055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83780536f0 0x7f8378055bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8384002fb0 tx=0x7f8384006930).stop 2021-11-21T21:54:26.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 msgr2=0x7f83940b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f83940b0400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f837c000ed0 tx=0x7f837c006d60).stop 2021-11-21T21:54:26.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 shutdown_connections 2021-11-21T21:54:26.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8394144a10 0x7f83940af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83780536f0 0x7f8378055bb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83940b3690 0x7f83940b0400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 --2- 172.21.2.116:0/554320702 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83940adc10 0x7f83941506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 >> 172.21.2.116:0/554320702 conn(0x7f839405bd20 msgr2=0x7f8394144010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 shutdown_connections 2021-11-21T21:54:26.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.702+0000 7f8398a5f700 1 -- 172.21.2.116:0/554320702 wait complete. 2021-11-21T21:54:26.713 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:26.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 Processor -- start 2021-11-21T21:54:26.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- start start 2021-11-21T21:54:26.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5f880603b0 0x7f5f880af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f88064260 con 0x7f5f88063e80 2021-11-21T21:54:26.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f88140550 con 0x7f5f8813d410 2021-11-21T21:54:26.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f881407f0 con 0x7f5f880603b0 2021-11-21T21:54:26.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5f88063e80 0x7f5f8813cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56502/0 (socket says 172.21.2.116:56502) 2021-11-21T21:54:26.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 -- 172.21.2.116:0/2333509567 learned_addr learned my addr 172.21.2.116:0/2333509567 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:26.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963466906 0 0) 0x7f5f88064260 con 0x7f5f88063e80 2021-11-21T21:54:26.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8dc59700 1 --2- 172.21.2.116:0/2333509567 >> v2:172.21.2.117:3300/0 conn(0x7f5f880603b0 0x7f5f880af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f70003260 con 0x7f5f88063e80 2021-11-21T21:54:26.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805520662 0 0) 0x7f5f70003260 con 0x7f5f88063e80 2021-11-21T21:54:26.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 >> v2:172.21.2.117:3300/0 conn(0x7f5f880603b0 msgr2=0x7f5f880af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 --2- 172.21.2.116:0/2333509567 >> v2:172.21.2.117:3300/0 conn(0x7f5f880603b0 0x7f5f880af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 >> v1:172.21.2.117:6789/0 conn(0x7f5f8813d410 legacy=0x7f5f880af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880ae920 con 0x7f5f88063e80 2021-11-21T21:54:26.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f5f84001680 con 0x7f5f88063e80 2021-11-21T21:54:26.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 >> v1:172.21.2.116:6789/0 conn(0x7f5f88063e80 legacy=0x7f5f8813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f70003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f70004500 0x7f5f70004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f70004f90 0x7f5f7000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881407f0 con 0x7f5f70003820 2021-11-21T21:54:26.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7000be80 con 0x7f5f70004f90 2021-11-21T21:54:26.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7000c040 con 0x7f5f70004500 2021-11-21T21:54:26.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f70003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8dc59700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f70004f90 0x7f5f7000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 -- 172.21.2.116:0/2333509567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f70004500 msgr2=0x7f5f70004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f70004500 0x7f5f70004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 -- 172.21.2.116:0/2333509567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f70004f90 msgr2=0x7f5f7000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f70004f90 0x7f5f7000b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 -- 172.21.2.116:0/2333509567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f70003260 con 0x7f5f70003820 2021-11-21T21:54:26.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8dc59700 1 -- 172.21.2.116:0/2333509567 reap_dead start 2021-11-21T21:54:26.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8d458700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f70003c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f5f84001390 tx=0x7f5f880ae7a0).ready entity=mon.0 client_cookie=ddf6309024076f70 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f840070d0 con 0x7f5f70003820 2021-11-21T21:54:26.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84007230 con 0x7f5f70003820 2021-11-21T21:54:26.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f7bfff700 1 -- 172.21.2.116:0/2333509567 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f8400c8c0 con 0x7f5f70003820 2021-11-21T21:54:26.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/2333509567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 msgr2=0x7f5f70003c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f70003c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f5f84001390 tx=0x7f5f880ae7a0).stop 2021-11-21T21:54:26.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/2333509567 shutdown_connections 2021-11-21T21:54:26.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/2333509567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f70003c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/2333509567 >> 172.21.2.116:0/2333509567 conn(0x7f5f88061d40 msgr2=0x7f5f88062120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/2333509567 shutdown_connections 2021-11-21T21:54:26.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.762+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/2333509567 wait complete. 2021-11-21T21:54:26.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 Processor -- start 2021-11-21T21:54:26.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 -- start start 2021-11-21T21:54:26.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f88136b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f880b4b70 0x7f5f880b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f88133100 0x7f5f88133580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b2460 con 0x7f5f70003820 2021-11-21T21:54:26.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b3ff0 con 0x7f5f88133100 2021-11-21T21:54:26.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b17b0 con 0x7f5f880b4b70 2021-11-21T21:54:26.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f88136b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f88136b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44558/0 (socket says 172.21.2.116:44558) 2021-11-21T21:54:26.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 -- 172.21.2.116:0/1898692611 learned_addr learned my addr 172.21.2.116:0/1898692611 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 -- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f880b4b70 msgr2=0x7f5f880b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f880b4b70 0x7f5f880b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 -- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f88133100 msgr2=0x7f5f88133580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f88133100 0x7f5f88133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 -- 172.21.2.116:0/1898692611 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880b5900 con 0x7f5f70003820 2021-11-21T21:54:26.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8d458700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f88136b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f5f880b4300 tx=0x7f5f84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f7a7fc700 1 -- 172.21.2.116:0/1898692611 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f840070d0 con 0x7f5f70003820 2021-11-21T21:54:26.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f7a7fc700 1 -- 172.21.2.116:0/1898692611 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84004920 con 0x7f5f70003820 2021-11-21T21:54:26.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f880b54c0 con 0x7f5f70003820 2021-11-21T21:54:26.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f7a7fc700 1 -- 172.21.2.116:0/1898692611 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f84004c80 con 0x7f5f70003820 2021-11-21T21:54:26.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f880b1cc0 con 0x7f5f70003820 2021-11-21T21:54:26.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f880b4130 con 0x7f5f70003820 2021-11-21T21:54:26.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f7a7fc700 1 -- 172.21.2.116:0/1898692611 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5f84020480 con 0x7f5f70003820 2021-11-21T21:54:26.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f7a7fc700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053600 0x7f5f68055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f7a7fc700 1 -- 172.21.2.116:0/1898692611 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5f8406cac0 con 0x7f5f70003820 2021-11-21T21:54:26.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.766+0000 7f5f8cc57700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053600 0x7f5f68055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.770+0000 7f5f8cc57700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053600 0x7f5f68055ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f5f880b06b0 tx=0x7f5f7c007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.770+0000 7f5f7a7fc700 1 -- 172.21.2.116:0/1898692611 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f84043c70 con 0x7f5f70003820 2021-11-21T21:54:26.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f880573d0 con 0x7f5f70003820 2021-11-21T21:54:26.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f7a7fc700 1 -- 172.21.2.116:0/1898692611 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5f84047290 con 0x7f5f70003820 2021-11-21T21:54:26.857 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:26.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053600 msgr2=0x7f5f68055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053600 0x7f5f68055ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f5f880b06b0 tx=0x7f5f7c007810).stop 2021-11-21T21:54:26.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 msgr2=0x7f5f88136b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f88136b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f5f880b4300 tx=0x7f5f84015000).stop 2021-11-21T21:54:26.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 shutdown_connections 2021-11-21T21:54:26.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f88133100 0x7f5f88133580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f880b4b70 0x7f5f880b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053600 0x7f5f68055ac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 --2- 172.21.2.116:0/1898692611 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f70003820 0x7f5f88136b40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 >> 172.21.2.116:0/1898692611 conn(0x7f5f88061d40 msgr2=0x7f5f8813f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 shutdown_connections 2021-11-21T21:54:26.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.854+0000 7f5f8f6bc700 1 -- 172.21.2.116:0/1898692611 wait complete. 2021-11-21T21:54:26.864 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:26.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5463990700 1 Processor -- start 2021-11-21T21:54:26.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5463990700 1 -- start start 2021-11-21T21:54:26.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5463990700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c144a10 0x7f545c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5463990700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f545c0b3690 0x7f545c13a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5463990700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b3c50 con 0x7f545c0adc10 2021-11-21T21:54:26.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5463990700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c0b3ef0 con 0x7f545c0b3690 2021-11-21T21:54:26.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5463990700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0b4190 con 0x7f545c144a10 2021-11-21T21:54:26.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5461f2d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f545c0b3690 0x7f545c13a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5461f2d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f545c0b3690 0x7f545c13a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44570/0 (socket says 172.21.2.116:44570) 2021-11-21T21:54:26.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5461f2d700 1 -- 172.21.2.116:0/3968182810 learned_addr learned my addr 172.21.2.116:0/3968182810 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5461f2d700 1 -- 172.21.2.116:0/3968182810 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c144a10 msgr2=0x7f545c0b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:26.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5461f2d700 1 --2- 172.21.2.116:0/3968182810 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c144a10 0x7f545c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5461f2d700 1 -- 172.21.2.116:0/3968182810 >> v1:172.21.2.116:6789/0 conn(0x7f545c0adc10 legacy=0x7f545c141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f544ffff700 1 -- 172.21.2.116:0/3968182810 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231790188 0 0) 0x7f545c0b3c50 con 0x7f545c0adc10 2021-11-21T21:54:26.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.914+0000 7f5461f2d700 1 -- 172.21.2.116:0/3968182810 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c140b80 con 0x7f545c0b3690 2021-11-21T21:54:26.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 --2- 172.21.2.116:0/3968182810 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c0b3690 0x7f545c13a540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f54540032f0 tx=0x7f5454009290).ready entity=mon.0 client_cookie=7f8d18c01275743a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f544ffff700 1 -- 172.21.2.116:0/3968182810 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f545400b8a0 con 0x7f545c0b3690 2021-11-21T21:54:26.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f544ffff700 1 -- 172.21.2.116:0/3968182810 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545400ba00 con 0x7f545c0b3690 2021-11-21T21:54:26.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f544ffff700 1 -- 172.21.2.116:0/3968182810 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f545400bb70 con 0x7f545c0b3690 2021-11-21T21:54:26.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- 172.21.2.116:0/3968182810 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c0b3690 msgr2=0x7f545c13a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 --2- 172.21.2.116:0/3968182810 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c0b3690 0x7f545c13a540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f54540032f0 tx=0x7f5454009290).stop 2021-11-21T21:54:26.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- 172.21.2.116:0/3968182810 shutdown_connections 2021-11-21T21:54:26.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 --2- 172.21.2.116:0/3968182810 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c0b3690 0x7f545c13a540 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:26.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 --2- 172.21.2.116:0/3968182810 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c144a10 0x7f545c0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- 172.21.2.116:0/3968182810 >> 172.21.2.116:0/3968182810 conn(0x7f545c05bd20 msgr2=0x7f545c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:26.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- 172.21.2.116:0/3968182810 shutdown_connections 2021-11-21T21:54:26.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- 172.21.2.116:0/3968182810 wait complete. 2021-11-21T21:54:26.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 Processor -- start 2021-11-21T21:54:26.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- start start 2021-11-21T21:54:26.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c0adc10 0x7f545c13d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f545c0b3690 0x7f545c0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c144a10 0x7f545c0b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0aec00 con 0x7f545c144a10 2021-11-21T21:54:26.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0b05c0 con 0x7f545c0b3690 2021-11-21T21:54:26.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0af640 con 0x7f545c0adc10 2021-11-21T21:54:26.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c144a10 0x7f545c0b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c144a10 0x7f545c0b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44576/0 (socket says 172.21.2.116:44576) 2021-11-21T21:54:26.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 -- 172.21.2.116:0/1409653443 learned_addr learned my addr 172.21.2.116:0/1409653443 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:26.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 -- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c0adc10 msgr2=0x7f545c13d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c0adc10 0x7f545c13d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 -- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f545c0b3690 msgr2=0x7f545c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:26.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f545c0b3690 0x7f545c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:26.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 -- 172.21.2.116:0/1409653443 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5454009530 con 0x7f545c144a10 2021-11-21T21:54:26.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5461f2d700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c144a10 0x7f545c0b09f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f545c114cc0 tx=0x7f5454016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f544e7fc700 1 -- 172.21.2.116:0/1409653443 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f545400b8a0 con 0x7f545c144a10 2021-11-21T21:54:26.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c0af8e0 con 0x7f545c144a10 2021-11-21T21:54:26.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c115b40 con 0x7f545c144a10 2021-11-21T21:54:26.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f544e7fc700 1 -- 172.21.2.116:0/1409653443 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454011e40 con 0x7f545c144a10 2021-11-21T21:54:26.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.918+0000 7f544e7fc700 1 -- 172.21.2.116:0/1409653443 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5454020620 con 0x7f545c144a10 2021-11-21T21:54:26.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.922+0000 7f544e7fc700 1 -- 172.21.2.116:0/1409653443 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5454026020 con 0x7f545c144a10 2021-11-21T21:54:26.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.922+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5438003410 con 0x7f545c144a10 2021-11-21T21:54:26.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.922+0000 7f544e7fc700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f543c053610 0x7f543c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:26.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.922+0000 7f544e7fc700 1 -- 172.21.2.116:0/1409653443 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f545406c660 con 0x7f545c144a10 2021-11-21T21:54:26.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.922+0000 7f546172c700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f543c053610 0x7f543c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:26.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.922+0000 7f546172c700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f543c053610 0x7f543c055ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f5458000ed0 tx=0x7f5458007680).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:26.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:26.922+0000 7f544e7fc700 1 -- 172.21.2.116:0/1409653443 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5454043870 con 0x7f545c144a10 2021-11-21T21:54:27.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.006+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5438003c40 con 0x7f545c144a10 2021-11-21T21:54:27.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.006+0000 7f544e7fc700 1 -- 172.21.2.116:0/1409653443 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f54540150a0 con 0x7f545c144a10 2021-11-21T21:54:27.012 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:27.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f543c053610 msgr2=0x7f543c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f543c053610 0x7f543c055ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f5458000ed0 tx=0x7f5458007680).stop 2021-11-21T21:54:27.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c144a10 msgr2=0x7f545c0b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c144a10 0x7f545c0b09f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f545c114cc0 tx=0x7f5454016000).stop 2021-11-21T21:54:27.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 shutdown_connections 2021-11-21T21:54:27.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f543c053610 0x7f543c055ad0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f545c144a10 0x7f545c0b09f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f545c0b3690 0x7f545c0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 --2- 172.21.2.116:0/1409653443 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f545c0adc10 0x7f545c13d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 >> 172.21.2.116:0/1409653443 conn(0x7f545c05bd20 msgr2=0x7f545c143f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 shutdown_connections 2021-11-21T21:54:27.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.010+0000 7f5463990700 1 -- 172.21.2.116:0/1409653443 wait complete. 2021-11-21T21:54:27.019 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-11-21T21:54:27.019 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2021-11-21T21:54:27.020 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2021-11-21T21:54:27.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad45c0f700 1 Processor -- start 2021-11-21T21:54:27.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad45c0f700 1 -- start start 2021-11-21T21:54:27.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad45c0f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad40062410 0x7fad40144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad45c0f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad400a6e80 con 0x7fad400a6aa0 2021-11-21T21:54:27.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad45c0f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4014a7b0 con 0x7fad400a86b0 2021-11-21T21:54:27.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad45c0f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad400a8bc0 con 0x7fad40062410 2021-11-21T21:54:27.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad3f7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fad400a6aa0 0x7fad401470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56536/0 (socket says 172.21.2.116:56536) 2021-11-21T21:54:27.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1290815891 learned_addr learned my addr 172.21.2.116:0/1290815891 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:27.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202323393 0 0) 0x7fad400a6e80 con 0x7fad400a6aa0 2021-11-21T21:54:27.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.066+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad20003260 con 0x7fad400a6aa0 2021-11-21T21:54:27.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4150810706 0 0) 0x7fad20003260 con 0x7fad400a6aa0 2021-11-21T21:54:27.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad40062410 msgr2=0x7fad40144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad40062410 0x7fad40144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 >> v1:172.21.2.117:6789/0 conn(0x7fad400a86b0 legacy=0x7fad401443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4014aa10 con 0x7fad400a6aa0 2021-11-21T21:54:27.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fad28001680 con 0x7fad400a6aa0 2021-11-21T21:54:27.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 >> v1:172.21.2.116:6789/0 conn(0x7fad400a6aa0 legacy=0x7fad401470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad20003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad20004500 0x7fad20004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad40062410 0x7fad2000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad400a8bc0 con 0x7fad200037d0 2021-11-21T21:54:27.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200052b0 con 0x7fad20004500 2021-11-21T21:54:27.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2000bc40 con 0x7fad40062410 2021-11-21T21:54:27.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad20003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad40062410 msgr2=0x7fad2000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad40062410 0x7fad2000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad20004500 msgr2=0x7fad20004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad20004500 0x7fad20004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1290815891 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20003260 con 0x7fad200037d0 2021-11-21T21:54:27.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad20003bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fad40061b50 tx=0x7fad40062060).ready entity=mon.0 client_cookie=95a5e3d9a58e8e6a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad28007170 con 0x7fad200037d0 2021-11-21T21:54:27.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad280072d0 con 0x7fad200037d0 2021-11-21T21:54:27.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3e7fc700 1 -- 172.21.2.116:0/1290815891 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2800c8f0 con 0x7fad200037d0 2021-11-21T21:54:27.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- 172.21.2.116:0/1290815891 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 msgr2=0x7fad20003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 --2- 172.21.2.116:0/1290815891 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad20003bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fad40061b50 tx=0x7fad40062060).stop 2021-11-21T21:54:27.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3ffff700 1 -- 172.21.2.116:0/1290815891 reap_dead start 2021-11-21T21:54:27.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- 172.21.2.116:0/1290815891 shutdown_connections 2021-11-21T21:54:27.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- 172.21.2.116:0/1290815891 >> 172.21.2.116:0/1290815891 conn(0x7fad4005bd20 msgr2=0x7fad4005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- 172.21.2.116:0/1290815891 shutdown_connections 2021-11-21T21:54:27.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- 172.21.2.116:0/1290815891 wait complete. 2021-11-21T21:54:27.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 Processor -- start 2021-11-21T21:54:27.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- start start 2021-11-21T21:54:27.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad40146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad40145290 0x7fad4015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad4013b180 0x7fad4013b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad401456f0 con 0x7fad200037d0 2021-11-21T21:54:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad401471f0 con 0x7fad40145290 2021-11-21T21:54:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad40147490 con 0x7fad4013b180 2021-11-21T21:54:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad40146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad40146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44594/0 (socket says 172.21.2.116:44594) 2021-11-21T21:54:27.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1993134062 learned_addr learned my addr 172.21.2.116:0/1993134062 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3effd700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad40145290 0x7fad4015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad4013b180 msgr2=0x7fad4013b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad4013b180 0x7fad4013b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad40145290 msgr2=0x7fad4015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad40145290 0x7fad4015a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 -- 172.21.2.116:0/1993134062 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4013d3a0 con 0x7fad200037d0 2021-11-21T21:54:27.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3f7fe700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad40146bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fad40061b50 tx=0x7fad28015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3cff9700 1 -- 172.21.2.116:0/1993134062 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad28007170 con 0x7fad200037d0 2021-11-21T21:54:27.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3cff9700 1 -- 172.21.2.116:0/1993134062 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad28018070 con 0x7fad200037d0 2021-11-21T21:54:27.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad3cff9700 1 -- 172.21.2.116:0/1993134062 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad28004e00 con 0x7fad200037d0 2021-11-21T21:54:27.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4013baf0 con 0x7fad200037d0 2021-11-21T21:54:27.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.070+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4013d9e0 con 0x7fad200037d0 2021-11-21T21:54:27.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.074+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4013d530 con 0x7fad200037d0 2021-11-21T21:54:27.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.074+0000 7fad3cff9700 1 -- 172.21.2.116:0/1993134062 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fad2801d070 con 0x7fad200037d0 2021-11-21T21:54:27.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.074+0000 7fad3cff9700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fad240534f0 0x7fad240559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.074+0000 7fad3cff9700 1 -- 172.21.2.116:0/1993134062 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fad2806cac0 con 0x7fad200037d0 2021-11-21T21:54:27.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.074+0000 7fad3effd700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fad240534f0 0x7fad240559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.074+0000 7fad3effd700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fad240534f0 0x7fad240559b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fad30008000 tx=0x7fad30009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.074+0000 7fad3cff9700 1 -- 172.21.2.116:0/1993134062 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad2806ee60 con 0x7fad200037d0 2021-11-21T21:54:27.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.178+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fad40147650 con 0x7fad200037d0 2021-11-21T21:54:27.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.178+0000 7fad3cff9700 1 -- 172.21.2.116:0/1993134062 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+131252 (secure 0 0 0) 0x7fad2801a070 con 0x7fad200037d0 2021-11-21T21:54:27.189 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:27.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fad240534f0 msgr2=0x7fad240559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fad240534f0 0x7fad240559b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fad30008000 tx=0x7fad30009040).stop 2021-11-21T21:54:27.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 msgr2=0x7fad40146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad40146bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fad40061b50 tx=0x7fad28015000).stop 2021-11-21T21:54:27.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 shutdown_connections 2021-11-21T21:54:27.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad4013b180 0x7fad4013b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad40145290 0x7fad4015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fad240534f0 0x7fad240559b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 --2- 172.21.2.116:0/1993134062 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad200037d0 0x7fad40146bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 >> 172.21.2.116:0/1993134062 conn(0x7fad4005bd20 msgr2=0x7fad40149b20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 shutdown_connections 2021-11-21T21:54:27.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.182+0000 7fad45c0f700 1 -- 172.21.2.116:0/1993134062 wait complete. 2021-11-21T21:54:27.193 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":4,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6832","nonce":14582},{"type":"v1","addr":"172.21.2.117:6833","nonce":14582}]},"active_addr":"172.21.2.117:6833/14582","active_change":"2021-11-21T21:54:13.039032+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.2.117:0","nonce":1177440434}]},{"addrvec":[{"type":"v2","addr":"172.21.2.117:0","nonce":3873533881}]},{"addrvec":[{"type":"v2","addr":"172.21.2.117:0","nonce":99592683}]}]}} 2021-11-21T21:54:27.198 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2021-11-21T21:54:27.199 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2021-11-21T21:54:27.199 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:27.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.250+0000 7f7a44688700 1 Processor -- start 2021-11-21T21:54:27.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.250+0000 7f7a44688700 1 -- start start 2021-11-21T21:54:27.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a3c0aed90 0x7f7a3c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7a3c0af360 0x7f7a3c13c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c0b00d0 con 0x7f7a3c0b24a0 2021-11-21T21:54:27.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c0adba0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c0b2880 con 0x7f7a3c0af360 2021-11-21T21:54:27.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a3c0aed90 0x7f7a3c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a3c0aed90 0x7f7a3c0b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44600/0 (socket says 172.21.2.116:44600) 2021-11-21T21:54:27.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 -- 172.21.2.116:0/2403134351 learned_addr learned my addr 172.21.2.116:0/2403134351 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 -- 172.21.2.116:0/2403134351 >> v2:172.21.2.117:3300/0 conn(0x7f7a3c0af360 msgr2=0x7f7a3c13c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 --2- 172.21.2.116:0/2403134351 >> v2:172.21.2.117:3300/0 conn(0x7f7a3c0af360 0x7f7a3c13c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 -- 172.21.2.116:0/2403134351 >> v1:172.21.2.116:6789/0 conn(0x7f7a3c0b24a0 legacy=0x7f7a3c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 -- 172.21.2.116:0/2403134351 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c0b14a0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41c23700 1 --2- 172.21.2.116:0/2403134351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c0b1a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7a2c002f70 tx=0x7f7a2c006eb0).ready entity=mon.0 client_cookie=cbff723baf4ed970 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41422700 1 -- 172.21.2.116:0/2403134351 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2c00fab0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41422700 1 -- 172.21.2.116:0/2403134351 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c000e90 con 0x7f7a3c0aed90 2021-11-21T21:54:27.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a41422700 1 -- 172.21.2.116:0/2403134351 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2c00fe00 con 0x7f7a3c0aed90 2021-11-21T21:54:27.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- 172.21.2.116:0/2403134351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 msgr2=0x7f7a3c0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- 172.21.2.116:0/2403134351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c0b1a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7a2c002f70 tx=0x7f7a2c006eb0).stop 2021-11-21T21:54:27.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- 172.21.2.116:0/2403134351 shutdown_connections 2021-11-21T21:54:27.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- 172.21.2.116:0/2403134351 >> v2:172.21.2.117:3300/0 conn(0x7f7a3c0af360 0x7f7a3c13c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- 172.21.2.116:0/2403134351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c0b1a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- 172.21.2.116:0/2403134351 >> 172.21.2.116:0/2403134351 conn(0x7f7a3c05fd00 msgr2=0x7f7a3c0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- 172.21.2.116:0/2403134351 shutdown_connections 2021-11-21T21:54:27.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- 172.21.2.116:0/2403134351 wait complete. 2021-11-21T21:54:27.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 Processor -- start 2021-11-21T21:54:27.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- start start 2021-11-21T21:54:27.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a3c0af360 0x7f7a3c13b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a3c0b24a0 0x7f7a3c13b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c1371f0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c13bb70 con 0x7f7a3c0af360 2021-11-21T21:54:27.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0b51e0 con 0x7f7a3c0b24a0 2021-11-21T21:54:27.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44610/0 (socket says 172.21.2.116:44610) 2021-11-21T21:54:27.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 -- 172.21.2.116:0/2537224558 learned_addr learned my addr 172.21.2.116:0/2537224558 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 -- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a3c0b24a0 msgr2=0x7f7a3c13b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a3c0b24a0 0x7f7a3c13b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 -- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a3c0af360 msgr2=0x7f7a3c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a3c0af360 0x7f7a3c13b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 -- 172.21.2.116:0/2537224558 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c00b1e0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a42424700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c156510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7a3c13bcb0 tx=0x7f7a38007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a2b7fe700 1 -- 172.21.2.116:0/2537224558 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a3800ce20 con 0x7f7a3c0aed90 2021-11-21T21:54:27.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a2b7fe700 1 -- 172.21.2.116:0/2537224558 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3800f470 con 0x7f7a3c0aed90 2021-11-21T21:54:27.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c0b4020 con 0x7f7a3c0aed90 2021-11-21T21:54:27.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a2b7fe700 1 -- 172.21.2.116:0/2537224558 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a3800f7d0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.254+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c0b5440 con 0x7f7a3c0aed90 2021-11-21T21:54:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.258+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c0b42f0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.258+0000 7f7a2b7fe700 1 -- 172.21.2.116:0/2537224558 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7a38010070 con 0x7f7a3c0aed90 2021-11-21T21:54:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.258+0000 7f7a2b7fe700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a1c053520 0x7f7a1c0559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.258+0000 7f7a2b7fe700 1 -- 172.21.2.116:0/2537224558 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7a3806aa20 con 0x7f7a3c0aed90 2021-11-21T21:54:27.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.258+0000 7f7a41c23700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a1c053520 0x7f7a1c0559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.258+0000 7f7a41c23700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a1c053520 0x7f7a1c0559e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7a2c002f70 tx=0x7f7a2c017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.258+0000 7f7a2b7fe700 1 -- 172.21.2.116:0/2537224558 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a38041a70 con 0x7f7a3c0aed90 2021-11-21T21:54:27.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a3c0573d0 con 0x7f7a3c0aed90 2021-11-21T21:54:27.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a2b7fe700 1 -- 172.21.2.116:0/2537224558 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f7a38045090 con 0x7f7a3c0aed90 2021-11-21T21:54:27.346 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:27.347 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:27.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a1c053520 msgr2=0x7f7a1c0559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a1c053520 0x7f7a1c0559e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7a2c002f70 tx=0x7f7a2c017000).stop 2021-11-21T21:54:27.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 msgr2=0x7f7a3c156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c156510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7a3c13bcb0 tx=0x7f7a38007c50).stop 2021-11-21T21:54:27.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 shutdown_connections 2021-11-21T21:54:27.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a3c0b24a0 0x7f7a3c13b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a3c0af360 0x7f7a3c13b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a1c053520 0x7f7a1c0559e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 --2- 172.21.2.116:0/2537224558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a3c0aed90 0x7f7a3c156510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 >> 172.21.2.116:0/2537224558 conn(0x7f7a3c05fd00 msgr2=0x7f7a3c0b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 shutdown_connections 2021-11-21T21:54:27.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.342+0000 7f7a44688700 1 -- 172.21.2.116:0/2537224558 wait complete. 2021-11-21T21:54:27.354 INFO:tasks.ceph.ceph_manager.ceph:all up! 2021-11-21T21:54:27.354 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:27.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 Processor -- start 2021-11-21T21:54:27.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- start start 2021-11-21T21:54:27.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb9413d470 0x7feb940af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7feb940adb90 0x7feb9413fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940ac090 con 0x7feb940abcb0 2021-11-21T21:54:27.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb940b1590 con 0x7feb9413d470 2021-11-21T21:54:27.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb940ae7c0 con 0x7feb940adb90 2021-11-21T21:54:27.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb9413d470 0x7feb940af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb9413d470 0x7feb940af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44618/0 (socket says 172.21.2.116:44618) 2021-11-21T21:54:27.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 -- 172.21.2.116:0/2661873420 learned_addr learned my addr 172.21.2.116:0/2661873420 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 -- 172.21.2.116:0/2661873420 >> v2:172.21.2.117:3300/0 conn(0x7feb940adb90 msgr2=0x7feb9413fb10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:27.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 --2- 172.21.2.116:0/2661873420 >> v2:172.21.2.117:3300/0 conn(0x7feb940adb90 0x7feb9413fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 -- 172.21.2.116:0/2661873420 >> v1:172.21.2.116:6789/0 conn(0x7feb940abcb0 legacy=0x7feb9413cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 -- 172.21.2.116:0/2661873420 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb94062590 con 0x7feb9413d470 2021-11-21T21:54:27.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb937fe700 1 --2- 172.21.2.116:0/2661873420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb9413d470 0x7feb940af290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7feb84002f70 tx=0x7feb84006eb0).ready entity=mon.0 client_cookie=288ad831c4c47fb1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb92ffd700 1 -- 172.21.2.116:0/2661873420 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8400d610 con 0x7feb9413d470 2021-11-21T21:54:27.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb92ffd700 1 -- 172.21.2.116:0/2661873420 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84000e90 con 0x7feb9413d470 2021-11-21T21:54:27.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb92ffd700 1 -- 172.21.2.116:0/2661873420 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8400d9a0 con 0x7feb9413d470 2021-11-21T21:54:27.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- 172.21.2.116:0/2661873420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb9413d470 msgr2=0x7feb940af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/2661873420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb9413d470 0x7feb940af290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7feb84002f70 tx=0x7feb84006eb0).stop 2021-11-21T21:54:27.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- 172.21.2.116:0/2661873420 shutdown_connections 2021-11-21T21:54:27.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/2661873420 >> v2:172.21.2.117:3300/0 conn(0x7feb940adb90 0x7feb9413fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/2661873420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb9413d470 0x7feb940af290 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- 172.21.2.116:0/2661873420 >> 172.21.2.116:0/2661873420 conn(0x7feb9405bd20 msgr2=0x7feb9405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- 172.21.2.116:0/2661873420 shutdown_connections 2021-11-21T21:54:27.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- 172.21.2.116:0/2661873420 wait complete. 2021-11-21T21:54:27.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 Processor -- start 2021-11-21T21:54:27.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 -- start start 2021-11-21T21:54:27.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.402+0000 7feb9a3ee700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb940abcb0 0x7feb94145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb940adb90 0x7feb940aa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb9413d470 0x7feb940aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940a9c80 con 0x7feb940abcb0 2021-11-21T21:54:27.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9413ec60 con 0x7feb940adb90 2021-11-21T21:54:27.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9413f4b0 con 0x7feb9413d470 2021-11-21T21:54:27.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb940abcb0 0x7feb94145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb940abcb0 0x7feb94145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44620/0 (socket says 172.21.2.116:44620) 2021-11-21T21:54:27.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 -- 172.21.2.116:0/3972493562 learned_addr learned my addr 172.21.2.116:0/3972493562 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 -- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb9413d470 msgr2=0x7feb940aa600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:27.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb9413d470 0x7feb940aa600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 -- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb940adb90 msgr2=0x7feb940aa110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:27.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb940adb90 0x7feb940aa110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 -- 172.21.2.116:0/3972493562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8400b1e0 con 0x7feb940abcb0 2021-11-21T21:54:27.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb93fff700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb940abcb0 0x7feb94145cb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7feb9413e100 tx=0x7feb88007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb917fa700 1 -- 172.21.2.116:0/3972493562 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb88007030 con 0x7feb940abcb0 2021-11-21T21:54:27.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb917fa700 1 -- 172.21.2.116:0/3972493562 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb88007190 con 0x7feb940abcb0 2021-11-21T21:54:27.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb917fa700 1 -- 172.21.2.116:0/3972493562 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb88006510 con 0x7feb940abcb0 2021-11-21T21:54:27.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9413e860 con 0x7feb940abcb0 2021-11-21T21:54:27.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9413f210 con 0x7feb940abcb0 2021-11-21T21:54:27.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb94145700 con 0x7feb940abcb0 2021-11-21T21:54:27.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb917fa700 1 -- 172.21.2.116:0/3972493562 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7feb88006670 con 0x7feb940abcb0 2021-11-21T21:54:27.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb917fa700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb74053660 0x7feb74055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb917fa700 1 -- 172.21.2.116:0/3972493562 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7feb8806b4a0 con 0x7feb940abcb0 2021-11-21T21:54:27.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb937fe700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb74053660 0x7feb74055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.406+0000 7feb937fe700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb74053660 0x7feb74055b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7feb84002f70 tx=0x7feb84015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.410+0000 7feb917fa700 1 -- 172.21.2.116:0/3972493562 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feb88042490 con 0x7feb940abcb0 2021-11-21T21:54:27.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.490+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb94062830 con 0x7feb940abcb0 2021-11-21T21:54:27.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.490+0000 7feb917fa700 1 -- 172.21.2.116:0/3972493562 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7feb88045ab0 con 0x7feb940abcb0 2021-11-21T21:54:27.495 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:27.496 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:27.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb74053660 msgr2=0x7feb74055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb74053660 0x7feb74055b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7feb84002f70 tx=0x7feb84015000).stop 2021-11-21T21:54:27.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb940abcb0 msgr2=0x7feb94145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb940abcb0 0x7feb94145cb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7feb9413e100 tx=0x7feb88007be0).stop 2021-11-21T21:54:27.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 shutdown_connections 2021-11-21T21:54:27.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb9413d470 0x7feb940aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb940adb90 0x7feb940aa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb74053660 0x7feb74055b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 --2- 172.21.2.116:0/3972493562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb940abcb0 0x7feb94145cb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 >> 172.21.2.116:0/3972493562 conn(0x7feb9405bd20 msgr2=0x7feb940b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 shutdown_connections 2021-11-21T21:54:27.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.494+0000 7feb9a3ee700 1 -- 172.21.2.116:0/3972493562 wait complete. 2021-11-21T21:54:27.503 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2021-11-21T21:54:27.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 Processor -- start 2021-11-21T21:54:27.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 -- start start 2021-11-21T21:54:27.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfe8144810 0x7fbfe81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbfe80abce0 0x7fbfe80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfe80b1db0 0x7fbfe80ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe8147e80 con 0x7fbfe8144810 2021-11-21T21:54:27.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe80af5a0 con 0x7fbfe80abce0 2021-11-21T21:54:27.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfed5da700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80b1490 con 0x7fbfe80b1db0 2021-11-21T21:54:27.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfe8144810 0x7fbfe81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfe8144810 0x7fbfe81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44628/0 (socket says 172.21.2.116:44628) 2021-11-21T21:54:27.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/3270966236 learned_addr learned my addr 172.21.2.116:0/3270966236 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/3270966236 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfe80b1db0 msgr2=0x7fbfe80ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:27.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 --2- 172.21.2.116:0/3270966236 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfe80b1db0 0x7fbfe80ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/3270966236 >> v2:172.21.2.117:3300/0 conn(0x7fbfe80abce0 msgr2=0x7fbfe80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:27.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 --2- 172.21.2.116:0/3270966236 >> v2:172.21.2.117:3300/0 conn(0x7fbfe80abce0 0x7fbfe80b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/3270966236 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80af060 con 0x7fbfe8144810 2021-11-21T21:54:27.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe6ffd700 1 --2- 172.21.2.116:0/3270966236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe8144810 0x7fbfe81478c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fbfd8002f70 tx=0x7fbfd8006eb0).ready entity=mon.0 client_cookie=6d71ce2edf695f18 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe5ffb700 1 -- 172.21.2.116:0/3270966236 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd800d570 con 0x7fbfe8144810 2021-11-21T21:54:27.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe5ffb700 1 -- 172.21.2.116:0/3270966236 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8000e90 con 0x7fbfe8144810 2021-11-21T21:54:27.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.550+0000 7fbfe5ffb700 1 -- 172.21.2.116:0/3270966236 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd800d900 con 0x7fbfe8144810 2021-11-21T21:54:27.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/3270966236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe8144810 msgr2=0x7fbfe81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 --2- 172.21.2.116:0/3270966236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe8144810 0x7fbfe81478c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fbfd8002f70 tx=0x7fbfd8006eb0).stop 2021-11-21T21:54:27.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/3270966236 shutdown_connections 2021-11-21T21:54:27.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 --2- 172.21.2.116:0/3270966236 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfe80b1db0 0x7fbfe80ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 --2- 172.21.2.116:0/3270966236 >> v2:172.21.2.117:3300/0 conn(0x7fbfe80abce0 0x7fbfe80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 --2- 172.21.2.116:0/3270966236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe8144810 0x7fbfe81478c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/3270966236 >> 172.21.2.116:0/3270966236 conn(0x7fbfe805bd20 msgr2=0x7fbfe805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/3270966236 shutdown_connections 2021-11-21T21:54:27.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/3270966236 wait complete. 2021-11-21T21:54:27.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 Processor -- start 2021-11-21T21:54:27.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- start start 2021-11-21T21:54:27.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe80abce0 0x7fbfe81499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfe8144810 0x7fbfe81466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfe81455d0 0x7fbfe813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8147510 con 0x7fbfe80abce0 2021-11-21T21:54:27.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8144d20 con 0x7fbfe8144810 2021-11-21T21:54:27.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8146280 con 0x7fbfe81455d0 2021-11-21T21:54:27.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe80abce0 0x7fbfe81499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe80abce0 0x7fbfe81499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44636/0 (socket says 172.21.2.116:44636) 2021-11-21T21:54:27.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/184258120 learned_addr learned my addr 172.21.2.116:0/184258120 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfe81455d0 msgr2=0x7fbfe813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfe81455d0 0x7fbfe813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfe8144810 msgr2=0x7fbfe81466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfe8144810 0x7fbfe81466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 -- 172.21.2.116:0/184258120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd800b1e0 con 0x7fbfe80abce0 2021-11-21T21:54:27.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe6ffd700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe80abce0 0x7fbfe81499e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbfe813f680 tx=0x7fbfd8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd8011ca0 con 0x7fbfe80abce0 2021-11-21T21:54:27.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8011e00 con 0x7fbfe80abce0 2021-11-21T21:54:27.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd801e650 con 0x7fbfe80abce0 2021-11-21T21:54:27.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe8147270 con 0x7fbfe80abce0 2021-11-21T21:54:27.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe80a81a0 con 0x7fbfe80abce0 2021-11-21T21:54:27.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfe81463c0 con 0x7fbfe80abce0 2021-11-21T21:54:27.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbfd8025030 con 0x7fbfe80abce0 2021-11-21T21:54:27.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfcc0536f0 0x7fbfcc055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbfd806cf00 con 0x7fbfe80abce0 2021-11-21T21:54:27.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7fbfcc05abe0 0x7fbfcc05d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 --> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfcc05d5c0 con 0x7fbfcc05abe0 2021-11-21T21:54:27.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd806f4c0 con 0x7fbfe80abce0 2021-11-21T21:54:27.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.554+0000 7fbfe77fe700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7fbfcc05abe0 0x7fbfcc05d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.558+0000 7fbfe67fc700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfcc0536f0 0x7fbfcc055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.558+0000 7fbfe77fe700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7fbfcc05abe0 0x7fbfcc05d000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.558+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== osd.0 v2:172.21.2.116:6800/14707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fbfcc05d5c0 con 0x7fbfcc05abe0 2021-11-21T21:54:27.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.558+0000 7fbfe67fc700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfcc0536f0 0x7fbfcc055bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fbfd0002fb0 tx=0x7fbfd00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 --> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbfe80af300 con 0x7fbfcc05abe0 2021-11-21T21:54:27.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfdffff700 1 -- 172.21.2.116:0/184258120 <== osd.0 v2:172.21.2.116:6800/14707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbfe80af300 con 0x7fbfcc05abe0 2021-11-21T21:54:27.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7fbfcc05abe0 msgr2=0x7fbfcc05d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7fbfcc05abe0 0x7fbfcc05d000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfcc0536f0 msgr2=0x7fbfcc055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfcc0536f0 0x7fbfcc055bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fbfd0002fb0 tx=0x7fbfd00069b0).stop 2021-11-21T21:54:27.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe80abce0 msgr2=0x7fbfe81499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 --2- 172.21.2.116:0/184258120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfe80abce0 0x7fbfe81499e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbfe813f680 tx=0x7fbfd8015000).stop 2021-11-21T21:54:27.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfe77fe700 1 -- 172.21.2.116:0/184258120 reap_dead start 2021-11-21T21:54:27.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 shutdown_connections 2021-11-21T21:54:27.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 >> 172.21.2.116:0/184258120 conn(0x7fbfe805bd20 msgr2=0x7fbfe8148280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 shutdown_connections 2021-11-21T21:54:27.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.566+0000 7fbfed5da700 1 -- 172.21.2.116:0/184258120 wait complete. 2021-11-21T21:54:27.581 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:27.582 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2021-11-21T21:54:27.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 Processor -- start 2021-11-21T21:54:27.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- start start 2021-11-21T21:54:27.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5e7405f730 0x7f5e74141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5e74142370 0x7f5e74142800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e740b06c0 con 0x7f5e740b02e0 2021-11-21T21:54:27.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e740adf20 con 0x7f5e7405f730 2021-11-21T21:54:27.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740b2470 con 0x7f5e74142370 2021-11-21T21:54:27.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5e7405f730 0x7f5e74141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5e7405f730 0x7f5e74141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44648/0 (socket says 172.21.2.116:44648) 2021-11-21T21:54:27.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 -- 172.21.2.116:0/466808688 learned_addr learned my addr 172.21.2.116:0/466808688 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 -- 172.21.2.116:0/466808688 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5e74142370 msgr2=0x7f5e74142800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:27.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 --2- 172.21.2.116:0/466808688 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5e74142370 0x7f5e74142800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 -- 172.21.2.116:0/466808688 >> v1:172.21.2.117:6789/0 conn(0x7f5e740b02e0 legacy=0x7f5e740af620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:27.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 -- 172.21.2.116:0/466808688 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740b26d0 con 0x7f5e7405f730 2021-11-21T21:54:27.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e796f4700 1 --2- 172.21.2.116:0/466808688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e7405f730 0x7f5e74141e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5e74063350 tx=0x7f5e68006f40).ready entity=mon.0 client_cookie=efd80d36633a7d69 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e78ef3700 1 -- 172.21.2.116:0/466808688 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6800dc30 con 0x7f5e7405f730 2021-11-21T21:54:27.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e78ef3700 1 -- 172.21.2.116:0/466808688 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e68000e90 con 0x7f5e7405f730 2021-11-21T21:54:27.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e78ef3700 1 -- 172.21.2.116:0/466808688 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6800be90 con 0x7f5e7405f730 2021-11-21T21:54:27.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- 172.21.2.116:0/466808688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e7405f730 msgr2=0x7f5e74141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 --2- 172.21.2.116:0/466808688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e7405f730 0x7f5e74141e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5e74063350 tx=0x7f5e68006f40).stop 2021-11-21T21:54:27.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- 172.21.2.116:0/466808688 shutdown_connections 2021-11-21T21:54:27.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 --2- 172.21.2.116:0/466808688 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5e74142370 0x7f5e74142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 --2- 172.21.2.116:0/466808688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e7405f730 0x7f5e74141e80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- 172.21.2.116:0/466808688 >> 172.21.2.116:0/466808688 conn(0x7f5e740b5ff0 msgr2=0x7f5e740b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- 172.21.2.116:0/466808688 shutdown_connections 2021-11-21T21:54:27.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.630+0000 7f5e7c159700 1 -- 172.21.2.116:0/466808688 wait complete. 2021-11-21T21:54:27.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 Processor -- start 2021-11-21T21:54:27.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 -- start start 2021-11-21T21:54:27.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5e7405f730 0x7f5e74145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e740b02e0 0x7f5e740aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5e74142370 0x7f5e740ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74153090 con 0x7f5e740b02e0 2021-11-21T21:54:27.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740667a0 con 0x7f5e7405f730 2021-11-21T21:54:27.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74066a40 con 0x7f5e74142370 2021-11-21T21:54:27.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e740b02e0 0x7f5e740aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e740b02e0 0x7f5e740aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44652/0 (socket says 172.21.2.116:44652) 2021-11-21T21:54:27.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 -- 172.21.2.116:0/3314914868 learned_addr learned my addr 172.21.2.116:0/3314914868 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 -- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5e74142370 msgr2=0x7f5e740ab3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:27.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5e74142370 0x7f5e740ab3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 -- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5e7405f730 msgr2=0x7f5e74145740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:27.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5e7405f730 0x7f5e74145740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 -- 172.21.2.116:0/3314914868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6800b460 con 0x7f5e740b02e0 2021-11-21T21:54:27.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e796f4700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e740b02e0 0x7f5e740aaeb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f5e74063350 tx=0x7f5e740b7000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6801d070 con 0x7f5e740b02e0 2021-11-21T21:54:27.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e68022410 con 0x7f5e740b02e0 2021-11-21T21:54:27.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e74066ce0 con 0x7f5e740b02e0 2021-11-21T21:54:27.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e680227b0 con 0x7f5e740b02e0 2021-11-21T21:54:27.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e740671b0 con 0x7f5e740b02e0 2021-11-21T21:54:27.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e740aac10 con 0x7f5e740b02e0 2021-11-21T21:54:27.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5e68029070 con 0x7f5e740b02e0 2021-11-21T21:54:27.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5e54053580 0x7f5e54055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5e6806ce20 con 0x7f5e740b02e0 2021-11-21T21:54:27.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f5e5405a970 0x7f5e5405cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 --> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e5405d3f0 con 0x7f5e5405a970 2021-11-21T21:54:27.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e6806f4b0 con 0x7f5e740b02e0 2021-11-21T21:54:27.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7a6f6700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f5e5405a970 0x7f5e5405cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e7a6f6700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f5e5405a970 0x7f5e5405cd90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e79ef5700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5e54053580 0x7f5e54055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.634+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== osd.1 v2:172.21.2.116:6808/14708 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f5e5405d3f0 con 0x7f5e5405a970 2021-11-21T21:54:27.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.638+0000 7f5e79ef5700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5e54053580 0x7f5e54055a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5e70002fb0 tx=0x7f5e70006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.642+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 --> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e740661d0 con 0x7f5e5405a970 2021-11-21T21:54:27.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.642+0000 7f5e66ffd700 1 -- 172.21.2.116:0/3314914868 <== osd.1 v2:172.21.2.116:6808/14708 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5e740661d0 con 0x7f5e5405a970 2021-11-21T21:54:27.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f5e5405a970 msgr2=0x7f5e5405cd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f5e5405a970 0x7f5e5405cd90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5e54053580 msgr2=0x7f5e54055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5e54053580 0x7f5e54055a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5e70002fb0 tx=0x7f5e70006930).stop 2021-11-21T21:54:27.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e740b02e0 msgr2=0x7f5e740aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 --2- 172.21.2.116:0/3314914868 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5e740b02e0 0x7f5e740aaeb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f5e74063350 tx=0x7f5e740b7000).stop 2021-11-21T21:54:27.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7a6f6700 1 -- 172.21.2.116:0/3314914868 reap_dead start 2021-11-21T21:54:27.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 shutdown_connections 2021-11-21T21:54:27.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 >> 172.21.2.116:0/3314914868 conn(0x7f5e740b5ff0 msgr2=0x7f5e740b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 shutdown_connections 2021-11-21T21:54:27.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.646+0000 7f5e7c159700 1 -- 172.21.2.116:0/3314914868 wait complete. 2021-11-21T21:54:27.656 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:27.656 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2021-11-21T21:54:27.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 Processor -- start 2021-11-21T21:54:27.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- start start 2021-11-21T21:54:27.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f70340adc10 0x7f70340b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7034144a10 0x7f7034141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70340b3c50 con 0x7f7034141510 2021-11-21T21:54:27.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70340b3ef0 con 0x7f70340adc10 2021-11-21T21:54:27.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70340b4190 con 0x7f7034144a10 2021-11-21T21:54:27.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f70340adc10 0x7f70340b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f70340adc10 0x7f70340b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44660/0 (socket says 172.21.2.116:44660) 2021-11-21T21:54:27.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 -- 172.21.2.116:0/2317330814 learned_addr learned my addr 172.21.2.116:0/2317330814 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 -- 172.21.2.116:0/2317330814 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7034144a10 msgr2=0x7f7034141020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:27.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 --2- 172.21.2.116:0/2317330814 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7034144a10 0x7f7034141020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 -- 172.21.2.116:0/2317330814 >> v1:172.21.2.116:6789/0 conn(0x7f7034141510 legacy=0x7f703413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7032ffd700 1 -- 172.21.2.116:0/2317330814 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767536915 0 0) 0x7f70340b3c50 con 0x7f7034141510 2021-11-21T21:54:27.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 -- 172.21.2.116:0/2317330814 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7034140b80 con 0x7f70340adc10 2021-11-21T21:54:27.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7033fff700 1 --2- 172.21.2.116:0/2317330814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70340adc10 0x7f70340b2030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f701c002f70 tx=0x7f701c006eb0).ready entity=mon.0 client_cookie=6ff43219523b42f4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7032ffd700 1 -- 172.21.2.116:0/2317330814 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701c00d5c0 con 0x7f70340adc10 2021-11-21T21:54:27.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7032ffd700 1 -- 172.21.2.116:0/2317330814 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c000e90 con 0x7f70340adc10 2021-11-21T21:54:27.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7032ffd700 1 -- 172.21.2.116:0/2317330814 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701c00d910 con 0x7f70340adc10 2021-11-21T21:54:27.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- 172.21.2.116:0/2317330814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70340adc10 msgr2=0x7f70340b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- 172.21.2.116:0/2317330814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70340adc10 0x7f70340b2030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f701c002f70 tx=0x7f701c006eb0).stop 2021-11-21T21:54:27.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- 172.21.2.116:0/2317330814 shutdown_connections 2021-11-21T21:54:27.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- 172.21.2.116:0/2317330814 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7034144a10 0x7f7034141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- 172.21.2.116:0/2317330814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70340adc10 0x7f70340b2030 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- 172.21.2.116:0/2317330814 >> 172.21.2.116:0/2317330814 conn(0x7f703405bd20 msgr2=0x7f703405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- 172.21.2.116:0/2317330814 shutdown_connections 2021-11-21T21:54:27.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- 172.21.2.116:0/2317330814 wait complete. 2021-11-21T21:54:27.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 Processor -- start 2021-11-21T21:54:27.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- start start 2021-11-21T21:54:27.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70340adc10 0x7f7034067430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7034141510 0x7f703413dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7034144a10 0x7f7034153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034150ad0 con 0x7f7034144a10 2021-11-21T21:54:27.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034110c90 con 0x7f70340adc10 2021-11-21T21:54:27.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f703a394700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70341530f0 con 0x7f7034141510 2021-11-21T21:54:27.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7038931700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7034144a10 0x7f7034153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7038931700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7034144a10 0x7f7034153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44670/0 (socket says 172.21.2.116:44670) 2021-11-21T21:54:27.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.706+0000 7f7038931700 1 -- 172.21.2.116:0/71489897 learned_addr learned my addr 172.21.2.116:0/71489897 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7038931700 1 -- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7034141510 msgr2=0x7f703413dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7038931700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7034141510 0x7f703413dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7038931700 1 -- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70340adc10 msgr2=0x7f7034067430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7038931700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70340adc10 0x7f7034067430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7038931700 1 -- 172.21.2.116:0/71489897 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c00b1e0 con 0x7f7034144a10 2021-11-21T21:54:27.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7038931700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7034144a10 0x7f7034153520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f703413ef60 tx=0x7f702c007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f702c00e4a0 con 0x7f7034144a10 2021-11-21T21:54:27.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c00e600 con 0x7f7034144a10 2021-11-21T21:54:27.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7034110ec0 con 0x7f7034144a10 2021-11-21T21:54:27.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f702c00e960 con 0x7f7034144a10 2021-11-21T21:54:27.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703414d760 con 0x7f7034144a10 2021-11-21T21:54:27.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7034153280 con 0x7f7034144a10 2021-11-21T21:54:27.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f702c010070 con 0x7f7034144a10 2021-11-21T21:54:27.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7018053640 0x7f7018055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f702c06ad50 con 0x7f7034144a10 2021-11-21T21:54:27.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f701805a9b0 0x7f701805cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 --> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f701805d390 con 0x7f701805a9b0 2021-11-21T21:54:27.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f702c06d3e0 con 0x7f7034144a10 2021-11-21T21:54:27.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70337fe700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f701805a9b0 0x7f701805cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70337fe700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f701805a9b0 0x7f701805cdd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7033fff700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7018053640 0x7f7018055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== osd.2 v2:172.21.2.116:6816/14709 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f701805d390 con 0x7f701805a9b0 2021-11-21T21:54:27.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.710+0000 7f7033fff700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7018053640 0x7f7018055b00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f701c002f70 tx=0x7f701c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 --> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70340573d0 con 0x7f701805a9b0 2021-11-21T21:54:27.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f70317fa700 1 -- 172.21.2.116:0/71489897 <== osd.2 v2:172.21.2.116:6816/14709 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f70340573d0 con 0x7f701805a9b0 2021-11-21T21:54:27.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f701805a9b0 msgr2=0x7f701805cdd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f701805a9b0 0x7f701805cdd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7018053640 msgr2=0x7f7018055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7018053640 0x7f7018055b00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f701c002f70 tx=0x7f701c015000).stop 2021-11-21T21:54:27.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7034144a10 msgr2=0x7f7034153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 --2- 172.21.2.116:0/71489897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7034144a10 0x7f7034153520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f703413ef60 tx=0x7f702c007d90).stop 2021-11-21T21:54:27.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f7038931700 1 -- 172.21.2.116:0/71489897 reap_dead start 2021-11-21T21:54:27.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 shutdown_connections 2021-11-21T21:54:27.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 >> 172.21.2.116:0/71489897 conn(0x7f703405bd20 msgr2=0x7f703405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 shutdown_connections 2021-11-21T21:54:27.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.718+0000 7f703a394700 1 -- 172.21.2.116:0/71489897 wait complete. 2021-11-21T21:54:27.731 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:27.732 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2021-11-21T21:54:27.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.782+0000 7f91d7a62700 1 Processor -- start 2021-11-21T21:54:27.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.782+0000 7f91d7a62700 1 -- start start 2021-11-21T21:54:27.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91d00abce0 0x7f91d01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f91d00ac600 0x7f91d00acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d0147e80 con 0x7f91d0144810 2021-11-21T21:54:27.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d00b1490 con 0x7f91d00ac600 2021-11-21T21:54:27.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00b1fd0 con 0x7f91d00abce0 2021-11-21T21:54:27.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f91d0144810 0x7f91d00b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56632/0 (socket says 172.21.2.116:56632) 2021-11-21T21:54:27.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 -- 172.21.2.116:0/3521891935 learned_addr learned my addr 172.21.2.116:0/3521891935 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:27.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080191612 0 0) 0x7f91d0147e80 con 0x7f91d0144810 2021-11-21T21:54:27.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b8003260 con 0x7f91d0144810 2021-11-21T21:54:27.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2123741713 0 0) 0x7f91b8003260 con 0x7f91d0144810 2021-11-21T21:54:27.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91d00abce0 msgr2=0x7f91d01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91d00abce0 0x7f91d01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 >> v2:172.21.2.117:3300/0 conn(0x7f91d00ac600 msgr2=0x7f91d00acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 --2- 172.21.2.116:0/3521891935 >> v2:172.21.2.117:3300/0 conn(0x7f91d00ac600 0x7f91d00acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d00af5a0 con 0x7f91d0144810 2021-11-21T21:54:27.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f91c4001660 con 0x7f91d0144810 2021-11-21T21:54:27.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 >> v1:172.21.2.116:6789/0 conn(0x7f91d0144810 legacy=0x7f91d00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91b80037d0 0x7f91b8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91b8004500 0x7f91b8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f91b8005070 0x7f91b800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00b1490 con 0x7f91b80037d0 2021-11-21T21:54:27.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00b1fd0 con 0x7f91b8005070 2021-11-21T21:54:27.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b800bf00 con 0x7f91b8004500 2021-11-21T21:54:27.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91b80037d0 0x7f91b8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 -- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91b8004500 msgr2=0x7f91b8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91b8004500 0x7f91b8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 -- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f91b8005070 msgr2=0x7f91b800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f91b8005070 0x7f91b800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 -- 172.21.2.116:0/3521891935 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8003260 con 0x7f91b80037d0 2021-11-21T21:54:27.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d57fe700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91b80037d0 0x7f91b8003bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f91c4001370 tx=0x7f91c4001d80).ready entity=mon.0 client_cookie=1f91bbfab3522997 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91c40070d0 con 0x7f91b80037d0 2021-11-21T21:54:27.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4007230 con 0x7f91b80037d0 2021-11-21T21:54:27.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c3fff700 1 -- 172.21.2.116:0/3521891935 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91c400c8f0 con 0x7f91b80037d0 2021-11-21T21:54:27.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- 172.21.2.116:0/3521891935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91b80037d0 msgr2=0x7f91b8003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 --2- 172.21.2.116:0/3521891935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91b80037d0 0x7f91b8003bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f91c4001370 tx=0x7f91c4001d80).stop 2021-11-21T21:54:27.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d5fff700 1 -- 172.21.2.116:0/3521891935 reap_dead start 2021-11-21T21:54:27.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- 172.21.2.116:0/3521891935 shutdown_connections 2021-11-21T21:54:27.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- 172.21.2.116:0/3521891935 >> 172.21.2.116:0/3521891935 conn(0x7f91d005bd20 msgr2=0x7f91d005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- 172.21.2.116:0/3521891935 shutdown_connections 2021-11-21T21:54:27.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- 172.21.2.116:0/3521891935 wait complete. 2021-11-21T21:54:27.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 Processor -- start 2021-11-21T21:54:27.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- start start 2021-11-21T21:54:27.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91b80037d0 0x7f91d013e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91d0159e60 0x7f91d015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f91d015a7d0 0x7f91d01450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0146530 con 0x7f91d0159e60 2021-11-21T21:54:27.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d013f340 con 0x7f91d015a7d0 2021-11-21T21:54:27.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00b4870 con 0x7f91b80037d0 2021-11-21T21:54:27.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91d0159e60 0x7f91d015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91d0159e60 0x7f91d015a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44688/0 (socket says 172.21.2.116:44688) 2021-11-21T21:54:27.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 -- 172.21.2.116:0/884787207 learned_addr learned my addr 172.21.2.116:0/884787207 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 -- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91b80037d0 msgr2=0x7f91d013e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f91b80037d0 0x7f91d013e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 -- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f91d015a7d0 msgr2=0x7f91d01450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f91d015a7d0 0x7f91d01450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 -- 172.21.2.116:0/884787207 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d0146290 con 0x7f91d0159e60 2021-11-21T21:54:27.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d4ffd700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91d0159e60 0x7f91d015a2e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f91cc000b60 tx=0x7f91cc006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91cc009e40 con 0x7f91d0159e60 2021-11-21T21:54:27.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc013460 con 0x7f91d0159e60 2021-11-21T21:54:27.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d00b4aa0 con 0x7f91d0159e60 2021-11-21T21:54:27.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91cc0137c0 con 0x7f91d0159e60 2021-11-21T21:54:27.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.786+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d00b3ea0 con 0x7f91d0159e60 2021-11-21T21:54:27.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91d00b4390 con 0x7f91d0159e60 2021-11-21T21:54:27.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f91cc01a060 con 0x7f91d0159e60 2021-11-21T21:54:27.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91c27fc700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f91b00536d0 0x7f91b0055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f91cc06b270 con 0x7f91d0159e60 2021-11-21T21:54:27.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91c27fc700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f91b005aac0 0x7f91b005cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 --> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91b005d4a0 con 0x7f91b005aac0 2021-11-21T21:54:27.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91cc06d900 con 0x7f91d0159e60 2021-11-21T21:54:27.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91d5fff700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f91b005aac0 0x7f91b005cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91d5fff700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f91b005aac0 0x7f91b005cee0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91d57fe700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f91b00536d0 0x7f91b0055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== osd.3 v2:172.21.2.116:6824/14710 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f91b005d4a0 con 0x7f91b005aac0 2021-11-21T21:54:27.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.790+0000 7f91d57fe700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f91b00536d0 0x7f91b0055b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f91c4001370 tx=0x7f91c4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 --> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f91d00af060 con 0x7f91b005aac0 2021-11-21T21:54:27.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91c27fc700 1 -- 172.21.2.116:0/884787207 <== osd.3 v2:172.21.2.116:6824/14710 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f91d00af060 con 0x7f91b005aac0 2021-11-21T21:54:27.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f91b005aac0 msgr2=0x7f91b005cee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f91b005aac0 0x7f91b005cee0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f91b00536d0 msgr2=0x7f91b0055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f91b00536d0 0x7f91b0055b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f91c4001370 tx=0x7f91c4015000).stop 2021-11-21T21:54:27.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91d0159e60 msgr2=0x7f91d015a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 --2- 172.21.2.116:0/884787207 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f91d0159e60 0x7f91d015a2e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f91cc000b60 tx=0x7f91cc006de0).stop 2021-11-21T21:54:27.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d5fff700 1 -- 172.21.2.116:0/884787207 reap_dead start 2021-11-21T21:54:27.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 shutdown_connections 2021-11-21T21:54:27.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 >> 172.21.2.116:0/884787207 conn(0x7f91d005bd20 msgr2=0x7f91d00b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 shutdown_connections 2021-11-21T21:54:27.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.798+0000 7f91d7a62700 1 -- 172.21.2.116:0/884787207 wait complete. 2021-11-21T21:54:27.815 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:27.815 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2021-11-21T21:54:27.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 Processor -- start 2021-11-21T21:54:27.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- start start 2021-11-21T21:54:27.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f96840b02e0 0x7f968405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96840b06c0 con 0x7f96840af620 2021-11-21T21:54:27.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96840adf20 con 0x7f96840afab0 2021-11-21T21:54:27.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96840b2470 con 0x7f96840b02e0 2021-11-21T21:54:27.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f96840b02e0 0x7f968405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f96840b02e0 0x7f968405f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44698/0 (socket says 172.21.2.116:44698) 2021-11-21T21:54:27.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968259c700 1 -- 172.21.2.116:0/1901181737 learned_addr learned my addr 172.21.2.116:0/1901181737 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968259c700 1 -- 172.21.2.116:0/1901181737 >> v1:172.21.2.117:6789/0 conn(0x7f96840afab0 legacy=0x7f9684143680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:27.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968259c700 1 -- 172.21.2.116:0/1901181737 >> v1:172.21.2.116:6789/0 conn(0x7f96840af620 legacy=0x7f9684142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:27.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968259c700 1 -- 172.21.2.116:0/1901181737 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9684143d90 con 0x7f96840b02e0 2021-11-21T21:54:27.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968259c700 1 --2- 172.21.2.116:0/1901181737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f968405f730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f9684063350 tx=0x7f9674006eb0).ready entity=mon.0 client_cookie=729f080a8317705a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968159a700 1 -- 172.21.2.116:0/1901181737 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f967400d5c0 con 0x7f96840b02e0 2021-11-21T21:54:27.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968159a700 1 -- 172.21.2.116:0/1901181737 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674000e90 con 0x7f96840b02e0 2021-11-21T21:54:27.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f968159a700 1 -- 172.21.2.116:0/1901181737 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f967400d950 con 0x7f96840b02e0 2021-11-21T21:54:27.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- 172.21.2.116:0/1901181737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 msgr2=0x7f968405f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 --2- 172.21.2.116:0/1901181737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f968405f730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f9684063350 tx=0x7f9674006eb0).stop 2021-11-21T21:54:27.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- 172.21.2.116:0/1901181737 shutdown_connections 2021-11-21T21:54:27.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 --2- 172.21.2.116:0/1901181737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f968405f730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- 172.21.2.116:0/1901181737 >> 172.21.2.116:0/1901181737 conn(0x7f96840b5ff0 msgr2=0x7f96840b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- 172.21.2.116:0/1901181737 shutdown_connections 2021-11-21T21:54:27.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- 172.21.2.116:0/1901181737 wait complete. 2021-11-21T21:54:27.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 Processor -- start 2021-11-21T21:54:27.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- start start 2021-11-21T21:54:27.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96840af620 0x7f968414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96840afab0 0x7f9684144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f9684144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684067a00 con 0x7f96840b02e0 2021-11-21T21:54:27.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684144280 con 0x7f96840afab0 2021-11-21T21:54:27.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9684146900 con 0x7f96840af620 2021-11-21T21:54:27.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f9684144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f9684144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44704/0 (socket says 172.21.2.116:44704) 2021-11-21T21:54:27.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 -- 172.21.2.116:0/3062284370 learned_addr learned my addr 172.21.2.116:0/3062284370 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:27.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 -- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96840af620 msgr2=0x7f968414a680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:27.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96840af620 0x7f968414a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 -- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96840afab0 msgr2=0x7f9684144980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:27.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96840afab0 0x7f9684144980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:27.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 -- 172.21.2.116:0/3062284370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967400b1e0 con 0x7f96840b02e0 2021-11-21T21:54:27.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9682d9d700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f9684144e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f967c003330 tx=0x7f967c009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f967c001520 con 0x7f96840b02e0 2021-11-21T21:54:27.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c001680 con 0x7f96840b02e0 2021-11-21T21:54:27.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9684146ba0 con 0x7f96840b02e0 2021-11-21T21:54:27.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96840a9600 con 0x7f96840b02e0 2021-11-21T21:54:27.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.866+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f967c0019e0 con 0x7f96840b02e0 2021-11-21T21:54:27.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96841446e0 con 0x7f96840b02e0 2021-11-21T21:54:27.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f967c018030 con 0x7f96840b02e0 2021-11-21T21:54:27.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f967b7fe700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96680536f0 0x7f9668055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f967c06b470 con 0x7f96840b02e0 2021-11-21T21:54:27.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f967b7fe700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f966805abe0 0x7f966805d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:27.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 --> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f966805d5c0 con 0x7f966805abe0 2021-11-21T21:54:27.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967c06d9d0 con 0x7f96840b02e0 2021-11-21T21:54:27.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f968259c700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96680536f0 0x7f9668055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f9681d9b700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f966805abe0 0x7f966805d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:27.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f968259c700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96680536f0 0x7f9668055bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9684063350 tx=0x7f9674015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.870+0000 7f9681d9b700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f966805abe0 0x7f966805d000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:27.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.874+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== osd.4 v2:172.21.2.117:6800/14956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f966805d5c0 con 0x7f966805abe0 2021-11-21T21:54:27.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.910+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 --> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96840b26d0 con 0x7f966805abe0 2021-11-21T21:54:27.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f967b7fe700 1 -- 172.21.2.116:0/3062284370 <== osd.4 v2:172.21.2.117:6800/14956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96840b26d0 con 0x7f966805abe0 2021-11-21T21:54:27.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f966805abe0 msgr2=0x7f966805d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f966805abe0 0x7f966805d000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:27.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96680536f0 msgr2=0x7f9668055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96680536f0 0x7f9668055bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9684063350 tx=0x7f9674015000).stop 2021-11-21T21:54:27.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 msgr2=0x7f9684144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:27.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 --2- 172.21.2.116:0/3062284370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96840b02e0 0x7f9684144e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f967c003330 tx=0x7f967c009120).stop 2021-11-21T21:54:27.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9682d9d700 1 -- 172.21.2.116:0/3062284370 reap_dead start 2021-11-21T21:54:27.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 shutdown_connections 2021-11-21T21:54:27.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 >> 172.21.2.116:0/3062284370 conn(0x7f96840b5ff0 msgr2=0x7f9684064a40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:27.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 shutdown_connections 2021-11-21T21:54:27.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.914+0000 7f9688891700 1 -- 172.21.2.116:0/3062284370 wait complete. 2021-11-21T21:54:27.929 INFO:teuthology.orchestra.run.gibba016.stdout:55834574853 2021-11-21T21:54:27.929 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2021-11-21T21:54:28.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 Processor -- start 2021-11-21T21:54:28.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 -- start start 2021-11-21T21:54:28.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc034144810 0x7fc0341478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0340abce0 0x7fc0340b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc0340b1db0 0x7fc0340ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc034147e80 con 0x7fc034144810 2021-11-21T21:54:28.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0340af5a0 con 0x7fc0340b1db0 2021-11-21T21:54:28.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc03b2e7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340b1490 con 0x7fc0340abce0 2021-11-21T21:54:28.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc034144810 0x7fc0341478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc034144810 0x7fc0341478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44714/0 (socket says 172.21.2.116:44714) 2021-11-21T21:54:28.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 -- 172.21.2.116:0/1139051613 learned_addr learned my addr 172.21.2.116:0/1139051613 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:28.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 -- 172.21.2.116:0/1139051613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0340abce0 msgr2=0x7fc0340b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 --2- 172.21.2.116:0/1139051613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0340abce0 0x7fc0340b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 -- 172.21.2.116:0/1139051613 >> v2:172.21.2.117:3300/0 conn(0x7fc0340b1db0 msgr2=0x7fc0340ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 --2- 172.21.2.116:0/1139051613 >> v2:172.21.2.117:3300/0 conn(0x7fc0340b1db0 0x7fc0340ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 -- 172.21.2.116:0/1139051613 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340af060 con 0x7fc034144810 2021-11-21T21:54:28.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.994+0000 7fc039083700 1 --2- 172.21.2.116:0/1139051613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc0341478c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024006eb0).ready entity=mon.0 client_cookie=3933cd038ae2cb7c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc023fff700 1 -- 172.21.2.116:0/1139051613 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02400d570 con 0x7fc034144810 2021-11-21T21:54:28.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc023fff700 1 -- 172.21.2.116:0/1139051613 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024000e90 con 0x7fc034144810 2021-11-21T21:54:28.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc023fff700 1 -- 172.21.2.116:0/1139051613 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02400d900 con 0x7fc034144810 2021-11-21T21:54:28.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/1139051613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 msgr2=0x7fc0341478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 --2- 172.21.2.116:0/1139051613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc0341478c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024006eb0).stop 2021-11-21T21:54:28.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/1139051613 shutdown_connections 2021-11-21T21:54:28.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 --2- 172.21.2.116:0/1139051613 >> v2:172.21.2.117:3300/0 conn(0x7fc0340b1db0 0x7fc0340ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 --2- 172.21.2.116:0/1139051613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0340abce0 0x7fc0340b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 --2- 172.21.2.116:0/1139051613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc0341478c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/1139051613 >> 172.21.2.116:0/1139051613 conn(0x7fc03405bd20 msgr2=0x7fc03405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/1139051613 shutdown_connections 2021-11-21T21:54:28.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/1139051613 wait complete. 2021-11-21T21:54:28.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 Processor -- start 2021-11-21T21:54:28.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- start start 2021-11-21T21:54:28.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc0340abce0 0x7fc034063ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc034066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0340643e0 0x7fc034067090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034145a00 con 0x7fc034144810 2021-11-21T21:54:28.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034145200 con 0x7fc0340abce0 2021-11-21T21:54:28.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc034159e50 con 0x7fc0340643e0 2021-11-21T21:54:28.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc034066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc034066ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44720/0 (socket says 172.21.2.116:44720) 2021-11-21T21:54:28.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 -- 172.21.2.116:0/605659059 learned_addr learned my addr 172.21.2.116:0/605659059 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:28.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 -- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0340643e0 msgr2=0x7fc034067090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:28.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0340643e0 0x7fc034067090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 -- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc0340abce0 msgr2=0x7fc034063ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:28.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc0340abce0 0x7fc034063ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 -- 172.21.2.116:0/605659059 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02400b1e0 con 0x7fc034144810 2021-11-21T21:54:28.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc038882700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc034066ba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fc030002f70 tx=0x7fc030006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc030013460 con 0x7fc034144810 2021-11-21T21:54:28.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0300135c0 con 0x7fc034144810 2021-11-21T21:54:28.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc030013920 con 0x7fc034144810 2021-11-21T21:54:28.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc034067980 con 0x7fc034144810 2021-11-21T21:54:28.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc034067720 con 0x7fc034144810 2021-11-21T21:54:28.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc034145400 con 0x7fc034144810 2021-11-21T21:54:28.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc03000f070 con 0x7fc034144810 2021-11-21T21:54:28.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc014053580 0x7fc014055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc03006bd00 con 0x7fc034144810 2021-11-21T21:54:28.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7fc01405a9f0 0x7fc01405ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 --> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc01405d470 con 0x7fc01405a9f0 2021-11-21T21:54:28.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:27.998+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc03006c010 con 0x7fc034144810 2021-11-21T21:54:28.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.002+0000 7fc039083700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc014053580 0x7fc014055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.002+0000 7fc039884700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7fc01405a9f0 0x7fc01405ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.002+0000 7fc039884700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7fc01405a9f0 0x7fc01405ce10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.002+0000 7fc039083700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc014053580 0x7fc014055a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.006+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== osd.5 v2:172.21.2.117:6808/14957 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fc01405d470 con 0x7fc01405a9f0 2021-11-21T21:54:28.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.010+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 --> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0340573d0 con 0x7fc01405a9f0 2021-11-21T21:54:28.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc0227fc700 1 -- 172.21.2.116:0/605659059 <== osd.5 v2:172.21.2.117:6808/14957 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc0340573d0 con 0x7fc01405a9f0 2021-11-21T21:54:28.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7fc01405a9f0 msgr2=0x7fc01405ce10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7fc01405a9f0 0x7fc01405ce10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc014053580 msgr2=0x7fc014055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc014053580 0x7fc014055a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024015000).stop 2021-11-21T21:54:28.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 msgr2=0x7fc034066ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 --2- 172.21.2.116:0/605659059 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc034144810 0x7fc034066ba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fc030002f70 tx=0x7fc030006e90).stop 2021-11-21T21:54:28.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc039884700 1 -- 172.21.2.116:0/605659059 reap_dead start 2021-11-21T21:54:28.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 shutdown_connections 2021-11-21T21:54:28.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 >> 172.21.2.116:0/605659059 conn(0x7fc03405bd20 msgr2=0x7fc034140020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 shutdown_connections 2021-11-21T21:54:28.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.014+0000 7fc03b2e7700 1 -- 172.21.2.116:0/605659059 wait complete. 2021-11-21T21:54:28.023 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:28.023 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2021-11-21T21:54:28.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.070+0000 7fbe130cf700 1 Processor -- start 2021-11-21T21:54:28.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.070+0000 7fbe130cf700 1 -- start start 2021-11-21T21:54:28.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.070+0000 7fbe130cf700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbe0c062c20 0x7fbe0c05ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.070+0000 7fbe130cf700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbe0c0b6240 0x7fbe0c12ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.070+0000 7fbe130cf700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0c063000 con 0x7fbe0c0b5620 2021-11-21T21:54:28.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.070+0000 7fbe130cf700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c142dd0 con 0x7fbe0c062c20 2021-11-21T21:54:28.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.070+0000 7fbe130cf700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c143070 con 0x7fbe0c0b6240 2021-11-21T21:54:28.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe10e6b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbe0c062c20 0x7fbe0c05ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe0bfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbe0c0b5620 0x7fbe0c0b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56690/0 (socket says 172.21.2.116:56690) 2021-11-21T21:54:28.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe0bfff700 1 -- 172.21.2.116:0/1590158163 learned_addr learned my addr 172.21.2.116:0/1590158163 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:28.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe0b7fe700 1 -- 172.21.2.116:0/1590158163 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577126646 0 0) 0x7fbe0c063000 con 0x7fbe0c0b5620 2021-11-21T21:54:28.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe0b7fe700 1 -- 172.21.2.116:0/1590158163 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdec003260 con 0x7fbe0c0b5620 2021-11-21T21:54:28.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe10e6b700 1 -- 172.21.2.116:0/1590158163 >> v2:172.21.2.117:3300/0 conn(0x7fbe0c0b6240 msgr2=0x7fbe0c12ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe10e6b700 1 --2- 172.21.2.116:0/1590158163 >> v2:172.21.2.117:3300/0 conn(0x7fbe0c0b6240 0x7fbe0c12ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe10e6b700 1 -- 172.21.2.116:0/1590158163 >> v1:172.21.2.116:6789/0 conn(0x7fbe0c0b5620 legacy=0x7fbe0c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe10e6b700 1 -- 172.21.2.116:0/1590158163 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c143310 con 0x7fbe0c062c20 2021-11-21T21:54:28.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe10e6b700 1 --2- 172.21.2.116:0/1590158163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c062c20 0x7fbe0c05ce60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fbdf4002f70 tx=0x7fbdf4006df0).ready entity=mon.0 client_cookie=5824bf34487883b0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe0b7fe700 1 -- 172.21.2.116:0/1590158163 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf400d570 con 0x7fbe0c062c20 2021-11-21T21:54:28.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe0b7fe700 1 -- 172.21.2.116:0/1590158163 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf4000e90 con 0x7fbe0c062c20 2021-11-21T21:54:28.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe0b7fe700 1 -- 172.21.2.116:0/1590158163 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf400d8c0 con 0x7fbe0c062c20 2021-11-21T21:54:28.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/1590158163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c062c20 msgr2=0x7fbe0c05ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 --2- 172.21.2.116:0/1590158163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c062c20 0x7fbe0c05ce60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fbdf4002f70 tx=0x7fbdf4006df0).stop 2021-11-21T21:54:28.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/1590158163 shutdown_connections 2021-11-21T21:54:28.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 --2- 172.21.2.116:0/1590158163 >> v2:172.21.2.117:3300/0 conn(0x7fbe0c0b6240 0x7fbe0c12ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 --2- 172.21.2.116:0/1590158163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c062c20 0x7fbe0c05ce60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/1590158163 >> 172.21.2.116:0/1590158163 conn(0x7fbe0c0ac9b0 msgr2=0x7fbe0c069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/1590158163 shutdown_connections 2021-11-21T21:54:28.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/1590158163 wait complete. 2021-11-21T21:54:28.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 Processor -- start 2021-11-21T21:54:28.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- start start 2021-11-21T21:54:28.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe0c062c20 0x7fbe0c1459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe0c0b5620 0x7fbe0c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c0b6240 0x7fbe0c0af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c0afe80 con 0x7fbe0c0b6240 2021-11-21T21:54:28.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c146d80 con 0x7fbe0c0b5620 2021-11-21T21:54:28.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c146ff0 con 0x7fbe0c062c20 2021-11-21T21:54:28.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c0b6240 0x7fbe0c0af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c0b6240 0x7fbe0c0af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44740/0 (socket says 172.21.2.116:44740) 2021-11-21T21:54:28.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 -- 172.21.2.116:0/3829003337 learned_addr learned my addr 172.21.2.116:0/3829003337 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:28.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 -- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe0c062c20 msgr2=0x7fbe0c1459f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:28.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe0c062c20 0x7fbe0c1459f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 -- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe0c0b5620 msgr2=0x7fbe0c0af4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:28.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe0c0b5620 0x7fbe0c0af4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 -- 172.21.2.116:0/3829003337 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf400b180 con 0x7fbe0c0b6240 2021-11-21T21:54:28.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe1166c700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c0b6240 0x7fbe0c0af990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fbe0c146f10 tx=0x7fbe040091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe0400d4a0 con 0x7fbe0c0b6240 2021-11-21T21:54:28.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0400d600 con 0x7fbe0c0b6240 2021-11-21T21:54:28.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe0400d960 con 0x7fbe0c0b6240 2021-11-21T21:54:28.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c143f40 con 0x7fbe0c0b6240 2021-11-21T21:54:28.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c0ae3d0 con 0x7fbe0c0b6240 2021-11-21T21:54:28.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe0c1435b0 con 0x7fbe0c0b6240 2021-11-21T21:54:28.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbe04018030 con 0x7fbe0c0b6240 2021-11-21T21:54:28.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe09ffb700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbdf00536a0 0x7fbdf0055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.074+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbe0406b320 con 0x7fbe0c0b6240 2021-11-21T21:54:28.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.078+0000 7fbe09ffb700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7fbdf005ab10 0x7fbdf005cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.078+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 --> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdf005d4f0 con 0x7fbdf005ab10 2021-11-21T21:54:28.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.078+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe0406d8e0 con 0x7fbe0c0b6240 2021-11-21T21:54:28.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.078+0000 7fbe10e6b700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbdf00536a0 0x7fbdf0055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.078+0000 7fbe0bfff700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7fbdf005ab10 0x7fbdf005cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.078+0000 7fbe10e6b700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbdf00536a0 0x7fbdf0055b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbdf4002f70 tx=0x7fbdf4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.078+0000 7fbe0bfff700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7fbdf005ab10 0x7fbdf005cf30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.082+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== osd.6 v2:172.21.2.117:6816/14958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fbdf005d4f0 con 0x7fbdf005ab10 2021-11-21T21:54:28.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 --> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe0c0573d0 con 0x7fbdf005ab10 2021-11-21T21:54:28.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe09ffb700 1 -- 172.21.2.116:0/3829003337 <== osd.6 v2:172.21.2.117:6816/14958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe0c0573d0 con 0x7fbdf005ab10 2021-11-21T21:54:28.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7fbdf005ab10 msgr2=0x7fbdf005cf30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7fbdf005ab10 0x7fbdf005cf30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbdf00536a0 msgr2=0x7fbdf0055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbdf00536a0 0x7fbdf0055b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbdf4002f70 tx=0x7fbdf4014000).stop 2021-11-21T21:54:28.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c0b6240 msgr2=0x7fbe0c0af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 --2- 172.21.2.116:0/3829003337 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe0c0b6240 0x7fbe0c0af990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fbe0c146f10 tx=0x7fbe040091a0).stop 2021-11-21T21:54:28.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe1166c700 1 -- 172.21.2.116:0/3829003337 reap_dead start 2021-11-21T21:54:28.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 shutdown_connections 2021-11-21T21:54:28.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 >> 172.21.2.116:0/3829003337 conn(0x7fbe0c0ac9b0 msgr2=0x7fbe0c0b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 shutdown_connections 2021-11-21T21:54:28.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.090+0000 7fbe130cf700 1 -- 172.21.2.116:0/3829003337 wait complete. 2021-11-21T21:54:28.099 INFO:teuthology.orchestra.run.gibba016.stdout:55834574854 2021-11-21T21:54:28.100 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2021-11-21T21:54:28.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39d206700 1 Processor -- start 2021-11-21T21:54:28.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39d206700 1 -- start start 2021-11-21T21:54:28.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39d206700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd39805f730 0x7fd398141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39d206700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd398142370 0x7fd398142800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39d206700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3980b06c0 con 0x7fd3980b02e0 2021-11-21T21:54:28.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39d206700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3980adf20 con 0x7fd39805f730 2021-11-21T21:54:28.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39d206700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980b2470 con 0x7fd398142370 2021-11-21T21:54:28.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd396d9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd3980b02e0 0x7fd3980af620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56702/0 (socket says 172.21.2.116:56702) 2021-11-21T21:54:28.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd396d9d700 1 -- 172.21.2.116:0/2453912752 learned_addr learned my addr 172.21.2.116:0/2453912752 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:28.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370902766 0 0) 0x7fd3980b06c0 con 0x7fd3980b02e0 2021-11-21T21:54:28.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd378003260 con 0x7fd3980b02e0 2021-11-21T21:54:28.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732461295 0 0) 0x7fd378003260 con 0x7fd3980b02e0 2021-11-21T21:54:28.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd398142370 msgr2=0x7fd398142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd398142370 0x7fd398142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 >> v2:172.21.2.117:3300/0 conn(0x7fd39805f730 msgr2=0x7fd398141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 --2- 172.21.2.116:0/2453912752 >> v2:172.21.2.117:3300/0 conn(0x7fd39805f730 0x7fd398141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 --2- 172.21.2.116:0/2453912752 >> v2:172.21.2.117:3300/0 conn(0x7fd39805f730 0x7fd398141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3980b26d0 con 0x7fd3980b02e0 2021-11-21T21:54:28.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd388001660 con 0x7fd3980b02e0 2021-11-21T21:54:28.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 >> v1:172.21.2.116:6789/0 conn(0x7fd3980b02e0 legacy=0x7fd3980af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3780037d0 0x7fd378003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd378004500 0x7fd378004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd398142370 0x7fd37800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980adf20 con 0x7fd378004500 2021-11-21T21:54:28.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980b2470 con 0x7fd3780037d0 2021-11-21T21:54:28.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd378005300 con 0x7fd398142370 2021-11-21T21:54:28.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd378004500 0x7fd378004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd396d9d700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3780037d0 0x7fd378003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 -- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd398142370 msgr2=0x7fd37800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd398142370 0x7fd37800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 -- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3780037d0 msgr2=0x7fd378003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3780037d0 0x7fd378003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 -- 172.21.2.116:0/2453912752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378003260 con 0x7fd378004500 2021-11-21T21:54:28.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd396d9d700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3780037d0 0x7fd378003bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:28.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd39659c700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd378004500 0x7fd378004980 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fd380007000 tx=0x7fd380008040).ready entity=mon.0 client_cookie=76a1e30c21fc8766 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd380012470 con 0x7fd378004500 2021-11-21T21:54:28.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38000c030 con 0x7fd378004500 2021-11-21T21:54:28.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.154+0000 7fd395d9b700 1 -- 172.21.2.116:0/2453912752 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3800127c0 con 0x7fd378004500 2021-11-21T21:54:28.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2453912752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd378004500 msgr2=0x7fd378004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 --2- 172.21.2.116:0/2453912752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd378004500 0x7fd378004980 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fd380007000 tx=0x7fd380008040).stop 2021-11-21T21:54:28.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39759e700 1 -- 172.21.2.116:0/2453912752 reap_dead start 2021-11-21T21:54:28.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2453912752 shutdown_connections 2021-11-21T21:54:28.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2453912752 >> 172.21.2.116:0/2453912752 conn(0x7fd3980b5ff0 msgr2=0x7fd3980b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2453912752 shutdown_connections 2021-11-21T21:54:28.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2453912752 wait complete. 2021-11-21T21:54:28.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 Processor -- start 2021-11-21T21:54:28.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- start start 2021-11-21T21:54:28.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd378004500 0x7fd3980ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd398145180 0x7fd3980ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3980ace30 0x7fd3980ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd398067830 con 0x7fd398145180 2021-11-21T21:54:28.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980abca0 con 0x7fd3980ace30 2021-11-21T21:54:28.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd398144910 con 0x7fd378004500 2021-11-21T21:54:28.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd398145180 0x7fd3980ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd398145180 0x7fd3980ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44760/0 (socket says 172.21.2.116:44760) 2021-11-21T21:54:28.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 -- 172.21.2.116:0/2795598778 learned_addr learned my addr 172.21.2.116:0/2795598778 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:28.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 -- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd378004500 msgr2=0x7fd3980ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd378004500 0x7fd3980ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 -- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3980ace30 msgr2=0x7fd3980ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3980ace30 0x7fd3980ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:28.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 -- 172.21.2.116:0/2795598778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398146ee0 con 0x7fd398145180 2021-11-21T21:54:28.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39659c700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd398145180 0x7fd3980ab7b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fd3980aa5f0 tx=0x7fd380008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd380021070 con 0x7fd398145180 2021-11-21T21:54:28.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380000e90 con 0x7fd398145180 2021-11-21T21:54:28.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd398145da0 con 0x7fd398145180 2021-11-21T21:54:28.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd38001d420 con 0x7fd398145180 2021-11-21T21:54:28.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd398146040 con 0x7fd398145180 2021-11-21T21:54:28.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd398144f10 con 0x7fd398145180 2021-11-21T21:54:28.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd38001d580 con 0x7fd398145180 2021-11-21T21:54:28.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd37c053690 0x7fd37c055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd38006d270 con 0x7fd398145180 2021-11-21T21:54:28.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fd37c05ab00 0x7fd37c05cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 --> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd37c05d4e0 con 0x7fd37c05ab00 2021-11-21T21:54:28.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.158+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd38006d5d0 con 0x7fd398145180 2021-11-21T21:54:28.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.162+0000 7fd396d9d700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd37c053690 0x7fd37c055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.162+0000 7fd39759e700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fd37c05ab00 0x7fd37c05cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.162+0000 7fd39759e700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fd37c05ab00 0x7fd37c05cf20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.162+0000 7fd396d9d700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd37c053690 0x7fd37c055b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd388008000 tx=0x7fd388009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.162+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== osd.7 v2:172.21.2.117:6823/14960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fd37c05d4e0 con 0x7fd37c05ab00 2021-11-21T21:54:28.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.170+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 --> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd398143480 con 0x7fd37c05ab00 2021-11-21T21:54:28.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd38ffff700 1 -- 172.21.2.116:0/2795598778 <== osd.7 v2:172.21.2.117:6823/14960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd398143480 con 0x7fd37c05ab00 2021-11-21T21:54:28.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fd37c05ab00 msgr2=0x7fd37c05cf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fd37c05ab00 0x7fd37c05cf20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd37c053690 msgr2=0x7fd37c055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd37c053690 0x7fd37c055b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd388008000 tx=0x7fd388009040).stop 2021-11-21T21:54:28.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd398145180 msgr2=0x7fd3980ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 --2- 172.21.2.116:0/2795598778 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd398145180 0x7fd3980ab7b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fd3980aa5f0 tx=0x7fd380008040).stop 2021-11-21T21:54:28.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39759e700 1 -- 172.21.2.116:0/2795598778 reap_dead start 2021-11-21T21:54:28.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 shutdown_connections 2021-11-21T21:54:28.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 >> 172.21.2.116:0/2795598778 conn(0x7fd3980b5ff0 msgr2=0x7fd3980b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 shutdown_connections 2021-11-21T21:54:28.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.174+0000 7fd39d206700 1 -- 172.21.2.116:0/2795598778 wait complete. 2021-11-21T21:54:28.323 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:28.323 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-11-21T21:54:28.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.390+0000 7f02c1fb0700 1 Processor -- start 2021-11-21T21:54:28.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.390+0000 7f02c1fb0700 1 -- start start 2021-11-21T21:54:28.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02c1fb0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc144810 0x7f02bc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02c1fb0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f02bc0abce0 0x7f02bc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02c1fb0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc147e80 con 0x7f02bc0b1db0 2021-11-21T21:54:28.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02c1fb0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02bc0b1490 con 0x7f02bc0abce0 2021-11-21T21:54:28.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02c1fb0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0af060 con 0x7f02bc144810 2021-11-21T21:54:28.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02baffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f02bc0abce0 0x7f02bc0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02bb7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc144810 0x7f02bc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02bbfff700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f02bc0b1db0 0x7f02bc0ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:55634/0 (socket says 172.21.2.116:55634) 2021-11-21T21:54:28.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02bbfff700 1 -- 172.21.2.116:0/2611924853 learned_addr learned my addr 172.21.2.116:0/2611924853 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:28.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02baffd700 1 -- 172.21.2.116:0/2611924853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc144810 msgr2=0x7f02bc1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:28.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02baffd700 1 --2- 172.21.2.116:0/2611924853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc144810 0x7f02bc1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02baffd700 1 -- 172.21.2.116:0/2611924853 >> v1:172.21.2.117:6789/0 conn(0x7f02bc0b1db0 legacy=0x7f02bc0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02baffd700 1 -- 172.21.2.116:0/2611924853 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0af5a0 con 0x7f02bc0abce0 2021-11-21T21:54:28.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.394+0000 7f02baffd700 1 --2- 172.21.2.116:0/2611924853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc0abce0 0x7f02bc0b18c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f02b4001490 tx=0x7f02a4006df0).ready entity=mon.1 client_cookie=70b84da9e5062d02 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02ba7fc700 1 -- 172.21.2.116:0/2611924853 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02a400d4a0 con 0x7f02bc0abce0 2021-11-21T21:54:28.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02ba7fc700 1 -- 172.21.2.116:0/2611924853 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a4000e90 con 0x7f02bc0abce0 2021-11-21T21:54:28.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02ba7fc700 1 -- 172.21.2.116:0/2611924853 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02a400d7f0 con 0x7f02bc0abce0 2021-11-21T21:54:28.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- 172.21.2.116:0/2611924853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc0abce0 msgr2=0x7f02bc0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/2611924853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc0abce0 0x7f02bc0b18c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f02b4001490 tx=0x7f02a4006df0).stop 2021-11-21T21:54:28.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- 172.21.2.116:0/2611924853 shutdown_connections 2021-11-21T21:54:28.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/2611924853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc0abce0 0x7f02bc0b18c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/2611924853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc144810 0x7f02bc1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- 172.21.2.116:0/2611924853 >> 172.21.2.116:0/2611924853 conn(0x7f02bc05bd20 msgr2=0x7f02bc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- 172.21.2.116:0/2611924853 shutdown_connections 2021-11-21T21:54:28.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- 172.21.2.116:0/2611924853 wait complete. 2021-11-21T21:54:28.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 Processor -- start 2021-11-21T21:54:28.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- start start 2021-11-21T21:54:28.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc0abce0 0x7f02bc146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02bc144810 0x7f02bc13d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc14a5d0 0x7f02bc13db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc13e070 con 0x7f02bc144810 2021-11-21T21:54:28.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0b2bc0 con 0x7f02bc14a5d0 2021-11-21T21:54:28.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc159ca0 con 0x7f02bc0abce0 2021-11-21T21:54:28.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02bc144810 0x7f02bc13d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02bc144810 0x7f02bc13d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44774/0 (socket says 172.21.2.116:44774) 2021-11-21T21:54:28.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 -- 172.21.2.116:0/1645401900 learned_addr learned my addr 172.21.2.116:0/1645401900 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:28.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02bbfff700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc14a5d0 0x7f02bc13db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02bb7fe700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc0abce0 0x7f02bc146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 -- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc0abce0 msgr2=0x7f02bc146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc0abce0 0x7f02bc146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 -- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc14a5d0 msgr2=0x7f02bc13db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc14a5d0 0x7f02bc13db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 -- 172.21.2.116:0/1645401900 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a400b180 con 0x7f02bc144810 2021-11-21T21:54:28.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02bb7fe700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc0abce0 0x7f02bc146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:28.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02baffd700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02bc144810 0x7f02bc13d690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f02b4001490 tx=0x7f02a4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02b8ff9700 1 -- 172.21.2.116:0/1645401900 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02a4016070 con 0x7f02bc144810 2021-11-21T21:54:28.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02b8ff9700 1 -- 172.21.2.116:0/1645401900 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a4013910 con 0x7f02bc144810 2021-11-21T21:54:28.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc149fd0 con 0x7f02bc144810 2021-11-21T21:54:28.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02b8ff9700 1 -- 172.21.2.116:0/1645401900 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02a4013c70 con 0x7f02bc144810 2021-11-21T21:54:28.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.398+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc0b37d0 con 0x7f02bc144810 2021-11-21T21:54:28.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.402+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc0af300 con 0x7f02bc144810 2021-11-21T21:54:28.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.402+0000 7f02b8ff9700 1 -- 172.21.2.116:0/1645401900 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f02a400f030 con 0x7f02bc144810 2021-11-21T21:54:28.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.402+0000 7f02b8ff9700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02a0053580 0x7f02a0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:28.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.402+0000 7f02b8ff9700 1 -- 172.21.2.116:0/1645401900 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f02a406c670 con 0x7f02bc144810 2021-11-21T21:54:28.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.402+0000 7f02bb7fe700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02a0053580 0x7f02a0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:28.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.402+0000 7f02bb7fe700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02a0053580 0x7f02a0055a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f02ac00b000 tx=0x7f02ac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:28.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.402+0000 7f02b8ff9700 1 -- 172.21.2.116:0/1645401900 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02a4043740 con 0x7f02bc144810 2021-11-21T21:54:28.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.486+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f02bc0573d0 con 0x7f02bc144810 2021-11-21T21:54:28.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.486+0000 7f02b8ff9700 1 -- 172.21.2.116:0/1645401900 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02a4023090 con 0x7f02bc144810 2021-11-21T21:54:28.493 INFO:teuthology.orchestra.run.gibba016.stdout:55834574851 2021-11-21T21:54:28.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02a0053580 msgr2=0x7f02a0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02a0053580 0x7f02a0055a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f02ac00b000 tx=0x7f02ac008040).stop 2021-11-21T21:54:28.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02bc144810 msgr2=0x7f02bc13d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:28.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02bc144810 0x7f02bc13d690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f02b4001490 tx=0x7f02a4014000).stop 2021-11-21T21:54:28.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 shutdown_connections 2021-11-21T21:54:28.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02bc14a5d0 0x7f02bc13db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02a0053580 0x7f02a0055a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02bc144810 0x7f02bc13d690 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 --2- 172.21.2.116:0/1645401900 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02bc0abce0 0x7f02bc146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:28.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 >> 172.21.2.116:0/1645401900 conn(0x7f02bc05bd20 msgr2=0x7f02bc1417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:28.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 shutdown_connections 2021-11-21T21:54:28.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:28.490+0000 7f02c1fb0700 1 -- 172.21.2.116:0/1645401900 wait complete. 2021-11-21T21:54:28.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2021-11-21T21:54:29.503 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-11-21T21:54:29.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f1698d37700 1 Processor -- start 2021-11-21T21:54:29.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f1698d37700 1 -- start start 2021-11-21T21:54:29.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f1698d37700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1694144810 0x7f16941478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:29.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f1698d37700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f16940ac600 0x7f16940aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:29.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f1698d37700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1694147e80 con 0x7f16940abce0 2021-11-21T21:54:29.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f1698d37700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16940b1490 con 0x7f1694144810 2021-11-21T21:54:29.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f1698d37700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16940b1fd0 con 0x7f16940ac600 2021-11-21T21:54:29.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1694144810 0x7f16941478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:29.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1694144810 0x7f16941478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44778/0 (socket says 172.21.2.116:44778) 2021-11-21T21:54:29.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 -- 172.21.2.116:0/3234868736 learned_addr learned my addr 172.21.2.116:0/3234868736 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:29.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 -- 172.21.2.116:0/3234868736 >> v2:172.21.2.117:3300/0 conn(0x7f16940ac600 msgr2=0x7f16940aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:29.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 --2- 172.21.2.116:0/3234868736 >> v2:172.21.2.117:3300/0 conn(0x7f16940ac600 0x7f16940aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:29.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 -- 172.21.2.116:0/3234868736 >> v1:172.21.2.116:6789/0 conn(0x7f16940abce0 legacy=0x7f16940b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:29.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 -- 172.21.2.116:0/3234868736 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16940af5a0 con 0x7f1694144810 2021-11-21T21:54:29.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169259c700 1 --2- 172.21.2.116:0/3234868736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1694144810 0x7f16941478c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f1684002f70 tx=0x7f1684006eb0).ready entity=mon.0 client_cookie=ed880d30c817674d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:29.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.554+0000 7f169159a700 1 -- 172.21.2.116:0/3234868736 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f168400d610 con 0x7f1694144810 2021-11-21T21:54:29.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169159a700 1 -- 172.21.2.116:0/3234868736 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1684000e90 con 0x7f1694144810 2021-11-21T21:54:29.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169159a700 1 -- 172.21.2.116:0/3234868736 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f168400d9a0 con 0x7f1694144810 2021-11-21T21:54:29.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/3234868736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1694144810 msgr2=0x7f16941478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:29.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 --2- 172.21.2.116:0/3234868736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1694144810 0x7f16941478c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f1684002f70 tx=0x7f1684006eb0).stop 2021-11-21T21:54:29.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/3234868736 shutdown_connections 2021-11-21T21:54:29.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 --2- 172.21.2.116:0/3234868736 >> v2:172.21.2.117:3300/0 conn(0x7f16940ac600 0x7f16940aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:29.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 --2- 172.21.2.116:0/3234868736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1694144810 0x7f16941478c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:29.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/3234868736 >> 172.21.2.116:0/3234868736 conn(0x7f169405bd20 msgr2=0x7f169405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:29.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/3234868736 shutdown_connections 2021-11-21T21:54:29.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/3234868736 wait complete. 2021-11-21T21:54:29.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 Processor -- start 2021-11-21T21:54:29.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- start start 2021-11-21T21:54:29.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16940abce0 0x7f169413b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:29.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16940ac600 0x7f16940b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:29.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1694144810 0x7f16940b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:29.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16940b2c80 con 0x7f16940abce0 2021-11-21T21:54:29.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941423e0 con 0x7f1694144810 2021-11-21T21:54:29.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16941427c0 con 0x7f16940ac600 2021-11-21T21:54:29.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16940abce0 0x7f169413b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:29.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16940abce0 0x7f169413b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44784/0 (socket says 172.21.2.116:44784) 2021-11-21T21:54:29.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 -- 172.21.2.116:0/2603311412 learned_addr learned my addr 172.21.2.116:0/2603311412 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:29.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 -- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16940ac600 msgr2=0x7f16940b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:29.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16940ac600 0x7f16940b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:29.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 -- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1694144810 msgr2=0x7f16940b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:29.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1694144810 0x7f16940b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:29.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 -- 172.21.2.116:0/2603311412 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168400b1e0 con 0x7f16940abce0 2021-11-21T21:54:29.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f169259c700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16940abce0 0x7f169413b8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1694146730 tx=0x7f1684015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:29.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f16837fe700 1 -- 172.21.2.116:0/2603311412 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16840119c0 con 0x7f16940abce0 2021-11-21T21:54:29.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f16837fe700 1 -- 172.21.2.116:0/2603311412 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1684011b20 con 0x7f16940abce0 2021-11-21T21:54:29.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f16837fe700 1 -- 172.21.2.116:0/2603311412 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1684011e80 con 0x7f16940abce0 2021-11-21T21:54:29.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1694146810 con 0x7f16940abce0 2021-11-21T21:54:29.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16941429f0 con 0x7f16940abce0 2021-11-21T21:54:29.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.558+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16940af060 con 0x7f16940abce0 2021-11-21T21:54:29.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.562+0000 7f16837fe700 1 -- 172.21.2.116:0/2603311412 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1684025030 con 0x7f16940abce0 2021-11-21T21:54:29.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.562+0000 7f16837fe700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16700536a0 0x7f1670055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:29.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.562+0000 7f16837fe700 1 -- 172.21.2.116:0/2603311412 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f168406ce60 con 0x7f16940abce0 2021-11-21T21:54:29.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.562+0000 7f1691d9b700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16700536a0 0x7f1670055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:29.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.562+0000 7f1691d9b700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16700536a0 0x7f1670055b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f16940b4e00 tx=0x7f1688007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:29.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.562+0000 7f16837fe700 1 -- 172.21.2.116:0/2603311412 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1684043eb0 con 0x7f16940abce0 2021-11-21T21:54:29.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.642+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f16940573d0 con 0x7f16940abce0 2021-11-21T21:54:29.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.642+0000 7f16837fe700 1 -- 172.21.2.116:0/2603311412 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16840474d0 con 0x7f16940abce0 2021-11-21T21:54:29.648 INFO:teuthology.orchestra.run.gibba016.stdout:55834574851 2021-11-21T21:54:29.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16700536a0 msgr2=0x7f1670055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:29.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16700536a0 0x7f1670055b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f16940b4e00 tx=0x7f1688007810).stop 2021-11-21T21:54:29.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16940abce0 msgr2=0x7f169413b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:29.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16940abce0 0x7f169413b8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1694146730 tx=0x7f1684015000).stop 2021-11-21T21:54:29.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 shutdown_connections 2021-11-21T21:54:29.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1694144810 0x7f16940b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:29.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16940ac600 0x7f16940b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:29.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16700536a0 0x7f1670055b60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:29.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 --2- 172.21.2.116:0/2603311412 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16940abce0 0x7f169413b8f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:29.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 >> 172.21.2.116:0/2603311412 conn(0x7f169405bd20 msgr2=0x7f169414a370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:29.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 shutdown_connections 2021-11-21T21:54:29.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:29.646+0000 7f1698d37700 1 -- 172.21.2.116:0/2603311412 wait complete. 2021-11-21T21:54:29.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2021-11-21T21:54:30.656 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-11-21T21:54:30.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43e68e9700 1 Processor -- start 2021-11-21T21:54:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43e68e9700 1 -- start start 2021-11-21T21:54:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43e68e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 0x7f43e00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43e68e9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e00b3c50 con 0x7f43e00adc10 2021-11-21T21:54:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43e68e9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e00b3ef0 con 0x7f43e0144a10 2021-11-21T21:54:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43e68e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00b4190 con 0x7f43e00b3690 2021-11-21T21:54:30.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43dffff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f43e00adc10 0x7f43e0141020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56748/0 (socket says 172.21.2.116:56748) 2021-11-21T21:54:30.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.710+0000 7f43dffff700 1 -- 172.21.2.116:0/1919369410 learned_addr learned my addr 172.21.2.116:0/1919369410 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:30.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 0x7f43e00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112696735 0 0) 0x7f43e00b3c50 con 0x7f43e00adc10 2021-11-21T21:54:30.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c0003260 con 0x7f43e00adc10 2021-11-21T21:54:30.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197767733 0 0) 0x7f43e00b3ef0 con 0x7f43e0144a10 2021-11-21T21:54:30.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 --> v1:172.21.2.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e00b3c50 con 0x7f43e0144a10 2021-11-21T21:54:30.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517059436 0 0) 0x7f43c0003260 con 0x7f43e00adc10 2021-11-21T21:54:30.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 msgr2=0x7f43e00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:30.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 0x7f43e00b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 >> v1:172.21.2.117:6789/0 conn(0x7f43e0144a10 legacy=0x7f43e013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0140b80 con 0x7f43e00adc10 2021-11-21T21:54:30.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f43d0001600 con 0x7f43e00adc10 2021-11-21T21:54:30.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 >> v1:172.21.2.116:6789/0 conn(0x7f43e00adc10 legacy=0x7f43e0141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 0x7f43c0003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43c0003e00 0x7f43c0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c00049f0 0x7f43c0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00b4190 con 0x7f43c0003e00 2021-11-21T21:54:30.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c000fbc0 con 0x7f43c00049f0 2021-11-21T21:54:30.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c000fd80 con 0x7f43e00b3690 2021-11-21T21:54:30.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43c0003e00 0x7f43c0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c00049f0 0x7f43c0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43dffff700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 0x7f43c0003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 -- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 msgr2=0x7f43c0003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:30.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b3690 0x7f43c0003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 -- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c00049f0 msgr2=0x7f43c0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:30.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c00049f0 0x7f43c0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 -- 172.21.2.116:0/1919369410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0003260 con 0x7f43c0003e00 2021-11-21T21:54:30.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43c0003e00 0x7f43c0004500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f43c8001110 tx=0x7f43c8007a00).ready entity=mon.0 client_cookie=523362640e34f7d2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:30.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c8007d10 con 0x7f43c0003e00 2021-11-21T21:54:30.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8007030 con 0x7f43c0003e00 2021-11-21T21:54:30.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43deffd700 1 -- 172.21.2.116:0/1919369410 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c8007390 con 0x7f43c0003e00 2021-11-21T21:54:30.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- 172.21.2.116:0/1919369410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43c0003e00 msgr2=0x7f43c0004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 --2- 172.21.2.116:0/1919369410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43c0003e00 0x7f43c0004500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f43c8001110 tx=0x7f43c8007a00).stop 2021-11-21T21:54:30.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 -- 172.21.2.116:0/1919369410 reap_dead start 2021-11-21T21:54:30.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- 172.21.2.116:0/1919369410 shutdown_connections 2021-11-21T21:54:30.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- 172.21.2.116:0/1919369410 >> 172.21.2.116:0/1919369410 conn(0x7f43e005bd20 msgr2=0x7f43e005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:30.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- 172.21.2.116:0/1919369410 shutdown_connections 2021-11-21T21:54:30.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- 172.21.2.116:0/1919369410 wait complete. 2021-11-21T21:54:30.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 Processor -- start 2021-11-21T21:54:30.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- start start 2021-11-21T21:54:30.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c0003e00 0x7f43e0142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b17a0 0x7f43e013d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43e01102a0 0x7f43e013dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00b0430 con 0x7f43e01102a0 2021-11-21T21:54:30.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00af6a0 con 0x7f43c0003e00 2021-11-21T21:54:30.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00af940 con 0x7f43e00b17a0 2021-11-21T21:54:30.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43e01102a0 0x7f43e013dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43e01102a0 0x7f43e013dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44808/0 (socket says 172.21.2.116:44808) 2021-11-21T21:54:30.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 -- 172.21.2.116:0/2025356666 learned_addr learned my addr 172.21.2.116:0/2025356666 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:30.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43dffff700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c0003e00 0x7f43e0142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43df7fe700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b17a0 0x7f43e013d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 -- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b17a0 msgr2=0x7f43e013d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b17a0 0x7f43e013d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 -- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c0003e00 msgr2=0x7f43e0142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c0003e00 0x7f43e0142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 -- 172.21.2.116:0/2025356666 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e00afbe0 con 0x7f43e01102a0 2021-11-21T21:54:30.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43dffff700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c0003e00 0x7f43e0142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:30.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e4e86700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43e01102a0 0x7f43e013dbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f43d800b000 tx=0x7f43d80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:30.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43dd7fa700 1 -- 172.21.2.116:0/2025356666 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d8013750 con 0x7f43e01102a0 2021-11-21T21:54:30.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43dd7fa700 1 -- 172.21.2.116:0/2025356666 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d80138b0 con 0x7f43e01102a0 2021-11-21T21:54:30.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e00b00b0 con 0x7f43e01102a0 2021-11-21T21:54:30.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43dd7fa700 1 -- 172.21.2.116:0/2025356666 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d8013c10 con 0x7f43e01102a0 2021-11-21T21:54:30.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.714+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e0151080 con 0x7f43e01102a0 2021-11-21T21:54:30.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.718+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e0150e20 con 0x7f43e01102a0 2021-11-21T21:54:30.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.718+0000 7f43dd7fa700 1 -- 172.21.2.116:0/2025356666 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f43d8013d70 con 0x7f43e01102a0 2021-11-21T21:54:30.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.718+0000 7f43dd7fa700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43c40535d0 0x7f43c4055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.718+0000 7f43dd7fa700 1 -- 172.21.2.116:0/2025356666 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f43d806c070 con 0x7f43e01102a0 2021-11-21T21:54:30.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.718+0000 7f43dffff700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43c40535d0 0x7f43c4055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.718+0000 7f43dffff700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43c40535d0 0x7f43c4055a90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f43d000d000 tx=0x7f43d0007040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:30.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.718+0000 7f43dd7fa700 1 -- 172.21.2.116:0/2025356666 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43d80430c0 con 0x7f43e01102a0 2021-11-21T21:54:30.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.802+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f43e005db30 con 0x7f43e01102a0 2021-11-21T21:54:30.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.802+0000 7f43dd7fa700 1 -- 172.21.2.116:0/2025356666 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f43d8018070 con 0x7f43e01102a0 2021-11-21T21:54:30.808 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:30.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43c40535d0 msgr2=0x7f43c4055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43c40535d0 0x7f43c4055a90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f43d000d000 tx=0x7f43d0007040).stop 2021-11-21T21:54:30.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43e01102a0 msgr2=0x7f43e013dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43e01102a0 0x7f43e013dbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f43d800b000 tx=0x7f43d80091d0).stop 2021-11-21T21:54:30.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 shutdown_connections 2021-11-21T21:54:30.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43c40535d0 0x7f43c4055a90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43e01102a0 0x7f43e013dbb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43e00b17a0 0x7f43e013d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 --2- 172.21.2.116:0/2025356666 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c0003e00 0x7f43e0142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 >> 172.21.2.116:0/2025356666 conn(0x7f43e005bd20 msgr2=0x7f43e0061760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:30.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 shutdown_connections 2021-11-21T21:54:30.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.806+0000 7f43e68e9700 1 -- 172.21.2.116:0/2025356666 wait complete. 2021-11-21T21:54:30.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2021-11-21T21:54:30.816 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2021-11-21T21:54:30.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.862+0000 7f19a543a700 1 Processor -- start 2021-11-21T21:54:30.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.862+0000 7f19a543a700 1 -- start start 2021-11-21T21:54:30.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f19a0144810 0x7f19a01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 0x7f19a00b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a0147e80 con 0x7f19a00b1db0 2021-11-21T21:54:30.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a00b1490 con 0x7f19a0144810 2021-11-21T21:54:30.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00af060 con 0x7f19a00abce0 2021-11-21T21:54:30.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f19a0144810 0x7f19a01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f19a0144810 0x7f19a01478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44812/0 (socket says 172.21.2.116:44812) 2021-11-21T21:54:30.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 -- 172.21.2.116:0/1878970127 learned_addr learned my addr 172.21.2.116:0/1878970127 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:30.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 -- 172.21.2.116:0/1878970127 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 msgr2=0x7f19a00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:30.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 --2- 172.21.2.116:0/1878970127 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 0x7f19a00b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:30.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 -- 172.21.2.116:0/1878970127 >> v1:172.21.2.116:6789/0 conn(0x7f19a00b1db0 legacy=0x7f19a00ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 -- 172.21.2.116:0/1878970127 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00af5a0 con 0x7f19a0144810 2021-11-21T21:54:30.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199effd700 1 --2- 172.21.2.116:0/1878970127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a01478c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f1990002f70 tx=0x7f1990006eb0).ready entity=mon.0 client_cookie=35e6c1e529ed376e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:30.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199dffb700 1 -- 172.21.2.116:0/1878970127 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199000d5c0 con 0x7f19a0144810 2021-11-21T21:54:30.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199dffb700 1 -- 172.21.2.116:0/1878970127 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990000e90 con 0x7f19a0144810 2021-11-21T21:54:30.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199dffb700 1 -- 172.21.2.116:0/1878970127 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199000d950 con 0x7f19a0144810 2021-11-21T21:54:30.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- 172.21.2.116:0/1878970127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 msgr2=0x7f19a01478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- 172.21.2.116:0/1878970127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a01478c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f1990002f70 tx=0x7f1990006eb0).stop 2021-11-21T21:54:30.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- 172.21.2.116:0/1878970127 shutdown_connections 2021-11-21T21:54:30.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- 172.21.2.116:0/1878970127 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 0x7f19a00b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:30.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- 172.21.2.116:0/1878970127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a01478c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- 172.21.2.116:0/1878970127 >> 172.21.2.116:0/1878970127 conn(0x7f19a005bd20 msgr2=0x7f19a005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:30.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- 172.21.2.116:0/1878970127 shutdown_connections 2021-11-21T21:54:30.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- 172.21.2.116:0/1878970127 wait complete. 2021-11-21T21:54:30.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 Processor -- start 2021-11-21T21:54:30.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- start start 2021-11-21T21:54:30.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 0x7f19a0143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a013a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19a00b2f30 0x7f19a013a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a0143930 con 0x7f19a0144810 2021-11-21T21:54:30.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a015a970 con 0x7f19a00b2f30 2021-11-21T21:54:30.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a0145fe0 con 0x7f19a00abce0 2021-11-21T21:54:30.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a013a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a013a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44818/0 (socket says 172.21.2.116:44818) 2021-11-21T21:54:30.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 -- 172.21.2.116:0/3758685217 learned_addr learned my addr 172.21.2.116:0/3758685217 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:30.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 -- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 msgr2=0x7f19a0143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 0x7f19a0143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:30.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 -- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19a00b2f30 msgr2=0x7f19a013a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19a00b2f30 0x7f19a013a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:30.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 -- 172.21.2.116:0/3758685217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199000b1e0 con 0x7f19a0144810 2021-11-21T21:54:30.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f199e7fc700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a013a190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1988002f70 tx=0x7f1988006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:30.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f1997fff700 1 -- 172.21.2.116:0/3758685217 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1988013460 con 0x7f19a0144810 2021-11-21T21:54:30.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f1997fff700 1 -- 172.21.2.116:0/3758685217 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19880135c0 con 0x7f19a0144810 2021-11-21T21:54:30.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a0146280 con 0x7f19a0144810 2021-11-21T21:54:30.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f1997fff700 1 -- 172.21.2.116:0/3758685217 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1988013920 con 0x7f19a0144810 2021-11-21T21:54:30.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.866+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a00b2ab0 con 0x7f19a0144810 2021-11-21T21:54:30.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.870+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a013ad10 con 0x7f19a0144810 2021-11-21T21:54:30.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.870+0000 7f1997fff700 1 -- 172.21.2.116:0/3758685217 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f198801a060 con 0x7f19a0144810 2021-11-21T21:54:30.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.870+0000 7f1997fff700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1984053640 0x7f1984055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:30.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.870+0000 7f1997fff700 1 -- 172.21.2.116:0/3758685217 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f198806b370 con 0x7f19a0144810 2021-11-21T21:54:30.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.870+0000 7f199effd700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1984053640 0x7f1984055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:30.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.870+0000 7f199effd700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1984053640 0x7f1984055b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f1990002f70 tx=0x7f1990015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:30.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.870+0000 7f1997fff700 1 -- 172.21.2.116:0/3758685217 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f19880424d0 con 0x7f19a0144810 2021-11-21T21:54:30.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.954+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f19a00af300 con 0x7f19a0144810 2021-11-21T21:54:30.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.954+0000 7f1997fff700 1 -- 172.21.2.116:0/3758685217 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1988045af0 con 0x7f19a0144810 2021-11-21T21:54:30.959 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:30.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1984053640 msgr2=0x7f1984055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1984053640 0x7f1984055b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f1990002f70 tx=0x7f1990015000).stop 2021-11-21T21:54:30.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 msgr2=0x7f19a013a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:30.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a013a190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1988002f70 tx=0x7f1988006e90).stop 2021-11-21T21:54:30.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 shutdown_connections 2021-11-21T21:54:30.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19a00b2f30 0x7f19a013a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:30.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1984053640 0x7f1984055b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19a0144810 0x7f19a013a190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:30.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 --2- 172.21.2.116:0/3758685217 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19a00abce0 0x7f19a0143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:30.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 >> 172.21.2.116:0/3758685217 conn(0x7f19a005bd20 msgr2=0x7f19a013ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:30.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 shutdown_connections 2021-11-21T21:54:30.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:30.958+0000 7f19a543a700 1 -- 172.21.2.116:0/3758685217 wait complete. 2021-11-21T21:54:30.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2021-11-21T21:54:30.967 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2021-11-21T21:54:31.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b7890f700 1 Processor -- start 2021-11-21T21:54:31.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b7890f700 1 -- start start 2021-11-21T21:54:31.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b7890f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 0x7f0b740b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b7890f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0b74097ec0 0x7f0b7414a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b7890f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b740995c0 con 0x7f0b740b5490 2021-11-21T21:54:31.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b7890f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b74144b40 con 0x7f0b74097ec0 2021-11-21T21:54:31.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b7890f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7414b140 con 0x7f0b740af720 2021-11-21T21:54:31.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0b74097ec0 0x7f0b7414a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0b74097ec0 0x7f0b7414a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44830/0 (socket says 172.21.2.116:44830) 2021-11-21T21:54:31.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 -- 172.21.2.116:0/1446067433 learned_addr learned my addr 172.21.2.116:0/1446067433 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 -- 172.21.2.116:0/1446067433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 msgr2=0x7f0b740b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 --2- 172.21.2.116:0/1446067433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 0x7f0b740b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 -- 172.21.2.116:0/1446067433 >> v1:172.21.2.116:6789/0 conn(0x7f0b740b5490 legacy=0x7f0b74146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 -- 172.21.2.116:0/1446067433 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b740997f0 con 0x7f0b74097ec0 2021-11-21T21:54:31.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.014+0000 7f0b73fff700 1 --2- 172.21.2.116:0/1446067433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a3b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0b6c0032f0 tx=0x7f0b6c009290).ready entity=mon.0 client_cookie=1cec662c549eadaf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b727fc700 1 -- 172.21.2.116:0/1446067433 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b6c00b8a0 con 0x7f0b74097ec0 2021-11-21T21:54:31.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b727fc700 1 -- 172.21.2.116:0/1446067433 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c00ba00 con 0x7f0b74097ec0 2021-11-21T21:54:31.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b727fc700 1 -- 172.21.2.116:0/1446067433 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b6c00bb70 con 0x7f0b74097ec0 2021-11-21T21:54:31.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/1446067433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 msgr2=0x7f0b7414a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 --2- 172.21.2.116:0/1446067433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a3b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0b6c0032f0 tx=0x7f0b6c009290).stop 2021-11-21T21:54:31.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/1446067433 shutdown_connections 2021-11-21T21:54:31.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 --2- 172.21.2.116:0/1446067433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a3b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 --2- 172.21.2.116:0/1446067433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 0x7f0b740b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/1446067433 >> 172.21.2.116:0/1446067433 conn(0x7f0b7405c7a0 msgr2=0x7f0b740b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/1446067433 shutdown_connections 2021-11-21T21:54:31.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/1446067433 wait complete. 2021-11-21T21:54:31.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 Processor -- start 2021-11-21T21:54:31.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- start start 2021-11-21T21:54:31.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 0x7f0b74159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b740b5490 0x7f0b74159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740ae040 con 0x7f0b74097ec0 2021-11-21T21:54:31.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740af160 con 0x7f0b740b5490 2021-11-21T21:54:31.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740ae2a0 con 0x7f0b740af720 2021-11-21T21:54:31.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44832/0 (socket says 172.21.2.116:44832) 2021-11-21T21:54:31.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 -- 172.21.2.116:0/202030309 learned_addr learned my addr 172.21.2.116:0/202030309 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 -- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 msgr2=0x7f0b74159a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:31.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 0x7f0b74159a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 -- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b740b5490 msgr2=0x7f0b74159ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:31.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b740b5490 0x7f0b74159ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 -- 172.21.2.116:0/202030309 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c009530 con 0x7f0b74097ec0 2021-11-21T21:54:31.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b737fe700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0b74099e10 tx=0x7f0b5c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b70ff9700 1 -- 172.21.2.116:0/202030309 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b5c00ce30 con 0x7f0b74097ec0 2021-11-21T21:54:31.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b70ff9700 1 -- 172.21.2.116:0/202030309 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c00f470 con 0x7f0b74097ec0 2021-11-21T21:54:31.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b70ff9700 1 -- 172.21.2.116:0/202030309 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b5c00f7d0 con 0x7f0b74097ec0 2021-11-21T21:54:31.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b740ae540 con 0x7f0b74097ec0 2021-11-21T21:54:31.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7415a3e0 con 0x7f0b74097ec0 2021-11-21T21:54:31.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b741596f0 con 0x7f0b74097ec0 2021-11-21T21:54:31.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b70ff9700 1 -- 172.21.2.116:0/202030309 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0b5c019030 con 0x7f0b74097ec0 2021-11-21T21:54:31.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b70ff9700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b580535b0 0x7f0b58055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.018+0000 7f0b70ff9700 1 -- 172.21.2.116:0/202030309 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0b5c06adb0 con 0x7f0b74097ec0 2021-11-21T21:54:31.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.022+0000 7f0b72ffd700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b580535b0 0x7f0b58055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.022+0000 7f0b72ffd700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b580535b0 0x7f0b58055a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0b64002f70 tx=0x7f0b64006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.022+0000 7f0b70ff9700 1 -- 172.21.2.116:0/202030309 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b5c041e00 con 0x7f0b74097ec0 2021-11-21T21:54:31.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0b74099a90 con 0x7f0b74097ec0 2021-11-21T21:54:31.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b70ff9700 1 -- 172.21.2.116:0/202030309 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b5c045420 con 0x7f0b74097ec0 2021-11-21T21:54:31.111 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:31.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b580535b0 msgr2=0x7f0b58055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b580535b0 0x7f0b58055a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0b64002f70 tx=0x7f0b64006a00).stop 2021-11-21T21:54:31.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 msgr2=0x7f0b7414a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0b74099e10 tx=0x7f0b5c007c50).stop 2021-11-21T21:54:31.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 shutdown_connections 2021-11-21T21:54:31.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b740b5490 0x7f0b74159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b740af720 0x7f0b74159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b580535b0 0x7f0b58055a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.106+0000 7f0b7890f700 1 --2- 172.21.2.116:0/202030309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b74097ec0 0x7f0b7414a940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.110+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 >> 172.21.2.116:0/202030309 conn(0x7f0b7405c7a0 msgr2=0x7f0b74142a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.110+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 shutdown_connections 2021-11-21T21:54:31.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.110+0000 7f0b7890f700 1 -- 172.21.2.116:0/202030309 wait complete. 2021-11-21T21:54:31.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2021-11-21T21:54:31.119 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2021-11-21T21:54:31.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f3da0700 1 Processor -- start 2021-11-21T21:54:31.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f3da0700 1 -- start start 2021-11-21T21:54:31.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f3da0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb6ec0adc10 0x7fb6ec0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f3da0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6ec144a10 0x7fb6ec141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f3da0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ec0b3c50 con 0x7fb6ec141510 2021-11-21T21:54:31.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f3da0700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b3ef0 con 0x7fb6ec144a10 2021-11-21T21:54:31.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f3da0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b4190 con 0x7fb6ec0adc10 2021-11-21T21:54:31.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6ec144a10 0x7fb6ec141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6ec144a10 0x7fb6ec141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44842/0 (socket says 172.21.2.116:44842) 2021-11-21T21:54:31.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 -- 172.21.2.116:0/2817627813 learned_addr learned my addr 172.21.2.116:0/2817627813 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 -- 172.21.2.116:0/2817627813 >> v2:172.21.2.117:3300/0 conn(0x7fb6ec0adc10 msgr2=0x7fb6ec0b2030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:31.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 --2- 172.21.2.116:0/2817627813 >> v2:172.21.2.117:3300/0 conn(0x7fb6ec0adc10 0x7fb6ec0b2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 -- 172.21.2.116:0/2817627813 >> v1:172.21.2.116:6789/0 conn(0x7fb6ec141510 legacy=0x7fb6ec13a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 -- 172.21.2.116:0/2817627813 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec140b80 con 0x7fb6ec144a10 2021-11-21T21:54:31.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f133b700 1 --2- 172.21.2.116:0/2817627813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec144a10 0x7fb6ec141020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fb6dc002f70 tx=0x7fb6dc006eb0).ready entity=mon.0 client_cookie=89340391c6ded4d6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f0b3a700 1 -- 172.21.2.116:0/2817627813 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6dc00d610 con 0x7fb6ec144a10 2021-11-21T21:54:31.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f0b3a700 1 -- 172.21.2.116:0/2817627813 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc000e90 con 0x7fb6ec144a10 2021-11-21T21:54:31.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.166+0000 7fb6f0b3a700 1 -- 172.21.2.116:0/2817627813 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6dc00d9a0 con 0x7fb6ec144a10 2021-11-21T21:54:31.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/2817627813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec144a10 msgr2=0x7fb6ec141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/2817627813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec144a10 0x7fb6ec141020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fb6dc002f70 tx=0x7fb6dc006eb0).stop 2021-11-21T21:54:31.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/2817627813 shutdown_connections 2021-11-21T21:54:31.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/2817627813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec144a10 0x7fb6ec141020 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/2817627813 >> v2:172.21.2.117:3300/0 conn(0x7fb6ec0adc10 0x7fb6ec0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/2817627813 >> 172.21.2.116:0/2817627813 conn(0x7fb6ec05bd20 msgr2=0x7fb6ec05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/2817627813 shutdown_connections 2021-11-21T21:54:31.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/2817627813 wait complete. 2021-11-21T21:54:31.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 Processor -- start 2021-11-21T21:54:31.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- start start 2021-11-21T21:54:31.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec0adc10 0x7fb6ec0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6ec141510 0x7fb6ec146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6ec144a10 0x7fb6ec146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b11a0 con 0x7fb6ec0adc10 2021-11-21T21:54:31.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b1440 con 0x7fb6ec141510 2021-11-21T21:54:31.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec144f00 con 0x7fb6ec144a10 2021-11-21T21:54:31.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec0adc10 0x7fb6ec0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec0adc10 0x7fb6ec0b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44848/0 (socket says 172.21.2.116:44848) 2021-11-21T21:54:31.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 -- 172.21.2.116:0/1839605771 learned_addr learned my addr 172.21.2.116:0/1839605771 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 -- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6ec144a10 msgr2=0x7fb6ec146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6ec144a10 0x7fb6ec146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 -- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6ec141510 msgr2=0x7fb6ec146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6ec141510 0x7fb6ec146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 -- 172.21.2.116:0/1839605771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc00b1e0 con 0x7fb6ec0adc10 2021-11-21T21:54:31.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f1b3c700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec0adc10 0x7fb6ec0b04e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb6e8002fb0 tx=0x7fb6e8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6daffd700 1 -- 172.21.2.116:0/1839605771 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e8013460 con 0x7fb6ec0adc10 2021-11-21T21:54:31.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6daffd700 1 -- 172.21.2.116:0/1839605771 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e80135c0 con 0x7fb6ec0adc10 2021-11-21T21:54:31.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6daffd700 1 -- 172.21.2.116:0/1839605771 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e8013920 con 0x7fb6ec0adc10 2021-11-21T21:54:31.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec1451a0 con 0x7fb6ec0adc10 2021-11-21T21:54:31.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec143f70 con 0x7fb6ec0adc10 2021-11-21T21:54:31.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec0573d0 con 0x7fb6ec0adc10 2021-11-21T21:54:31.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6daffd700 1 -- 172.21.2.116:0/1839605771 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb6e800f070 con 0x7fb6ec0adc10 2021-11-21T21:54:31.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6daffd700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6cc0535d0 0x7fb6cc055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.170+0000 7fb6daffd700 1 -- 172.21.2.116:0/1839605771 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb6e806b520 con 0x7fb6ec0adc10 2021-11-21T21:54:31.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.174+0000 7fb6f133b700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6cc0535d0 0x7fb6cc055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.174+0000 7fb6f133b700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6cc0535d0 0x7fb6cc055a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb6dc002f70 tx=0x7fb6dc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.174+0000 7fb6daffd700 1 -- 172.21.2.116:0/1839605771 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6e8042570 con 0x7fb6ec0adc10 2021-11-21T21:54:31.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb6ec0b1af0 con 0x7fb6ec0adc10 2021-11-21T21:54:31.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6daffd700 1 -- 172.21.2.116:0/1839605771 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6e8045b90 con 0x7fb6ec0adc10 2021-11-21T21:54:31.263 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:31.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6cc0535d0 msgr2=0x7fb6cc055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6cc0535d0 0x7fb6cc055a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb6dc002f70 tx=0x7fb6dc015000).stop 2021-11-21T21:54:31.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec0adc10 msgr2=0x7fb6ec0b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec0adc10 0x7fb6ec0b04e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb6e8002fb0 tx=0x7fb6e8006df0).stop 2021-11-21T21:54:31.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 shutdown_connections 2021-11-21T21:54:31.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6ec144a10 0x7fb6ec146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6ec141510 0x7fb6ec146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6cc0535d0 0x7fb6cc055a90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 --2- 172.21.2.116:0/1839605771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6ec0adc10 0x7fb6ec0b04e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 >> 172.21.2.116:0/1839605771 conn(0x7fb6ec05bd20 msgr2=0x7fb6ec13fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 shutdown_connections 2021-11-21T21:54:31.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.258+0000 7fb6f3da0700 1 -- 172.21.2.116:0/1839605771 wait complete. 2021-11-21T21:54:31.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2021-11-21T21:54:31.270 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2021-11-21T21:54:31.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 Processor -- start 2021-11-21T21:54:31.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- start start 2021-11-21T21:54:31.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f50e01426e0 0x7f50e0142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e00ad470 0x7f50e00ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e00afae0 con 0x7f50e0062e60 2021-11-21T21:54:31.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e00b0f60 con 0x7f50e01426e0 2021-11-21T21:54:31.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0063470 con 0x7f50e00ad470 2021-11-21T21:54:31.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f50e01426e0 0x7f50e0142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f50e01426e0 0x7f50e0142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44854/0 (socket says 172.21.2.116:44854) 2021-11-21T21:54:31.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 -- 172.21.2.116:0/3565738279 learned_addr learned my addr 172.21.2.116:0/3565738279 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 -- 172.21.2.116:0/3565738279 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e00ad470 msgr2=0x7f50e00ad900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:31.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 --2- 172.21.2.116:0/3565738279 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e00ad470 0x7f50e00ad900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 -- 172.21.2.116:0/3565738279 >> v1:172.21.2.116:6789/0 conn(0x7f50e0062e60 legacy=0x7f50e00af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 -- 172.21.2.116:0/3565738279 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0143050 con 0x7f50e01426e0 2021-11-21T21:54:31.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50dffff700 1 --2- 172.21.2.116:0/3565738279 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e01426e0 0x7f50e0142ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f50c8002f70 tx=0x7f50c8006eb0).ready entity=mon.0 client_cookie=7e11a72bf9a7a047 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50deffd700 1 -- 172.21.2.116:0/3565738279 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c8010810 con 0x7f50e01426e0 2021-11-21T21:54:31.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50deffd700 1 -- 172.21.2.116:0/3565738279 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8000e90 con 0x7f50e01426e0 2021-11-21T21:54:31.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50deffd700 1 -- 172.21.2.116:0/3565738279 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c8010b60 con 0x7f50e01426e0 2021-11-21T21:54:31.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- 172.21.2.116:0/3565738279 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e01426e0 msgr2=0x7f50e0142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 --2- 172.21.2.116:0/3565738279 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e01426e0 0x7f50e0142ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f50c8002f70 tx=0x7f50c8006eb0).stop 2021-11-21T21:54:31.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- 172.21.2.116:0/3565738279 shutdown_connections 2021-11-21T21:54:31.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 --2- 172.21.2.116:0/3565738279 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e00ad470 0x7f50e00ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 --2- 172.21.2.116:0/3565738279 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e01426e0 0x7f50e0142ac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- 172.21.2.116:0/3565738279 >> 172.21.2.116:0/3565738279 conn(0x7f50e00b5730 msgr2=0x7f50e00b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- 172.21.2.116:0/3565738279 shutdown_connections 2021-11-21T21:54:31.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.318+0000 7f50e6777700 1 -- 172.21.2.116:0/3565738279 wait complete. 2021-11-21T21:54:31.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 Processor -- start 2021-11-21T21:54:31.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 -- start start 2021-11-21T21:54:31.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50e00ad470 0x7f50e013fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e01426e0 0x7f50e00637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e0144640 0x7f50e013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e013eb40 con 0x7f50e0144640 2021-11-21T21:54:31.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0145a10 con 0x7f50e00ad470 2021-11-21T21:54:31.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00b6220 con 0x7f50e01426e0 2021-11-21T21:54:31.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e0144640 0x7f50e013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e0144640 0x7f50e013cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44864/0 (socket says 172.21.2.116:44864) 2021-11-21T21:54:31.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 -- 172.21.2.116:0/3936076504 learned_addr learned my addr 172.21.2.116:0/3936076504 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 -- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e01426e0 msgr2=0x7f50e00637a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:31.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e01426e0 0x7f50e00637a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 -- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50e00ad470 msgr2=0x7f50e013fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50e00ad470 0x7f50e013fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 -- 172.21.2.116:0/3936076504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c800c2f0 con 0x7f50e0144640 2021-11-21T21:54:31.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e4d14700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e0144640 0x7f50e013cea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f50d8003330 tx=0x7f50d80091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50dd7fa700 1 -- 172.21.2.116:0/3936076504 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50d8001520 con 0x7f50e0144640 2021-11-21T21:54:31.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50dd7fa700 1 -- 172.21.2.116:0/3936076504 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8001680 con 0x7f50e0144640 2021-11-21T21:54:31.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e01462b0 con 0x7f50e0144640 2021-11-21T21:54:31.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e013e5d0 con 0x7f50e0144640 2021-11-21T21:54:31.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50dd7fa700 1 -- 172.21.2.116:0/3936076504 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50d80019e0 con 0x7f50e0144640 2021-11-21T21:54:31.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e013e430 con 0x7f50e0144640 2021-11-21T21:54:31.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50dd7fa700 1 -- 172.21.2.116:0/3936076504 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f50d8018030 con 0x7f50e0144640 2021-11-21T21:54:31.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50dd7fa700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f50c4053570 0x7f50c4055a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50dd7fa700 1 -- 172.21.2.116:0/3936076504 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f50d806b4b0 con 0x7f50e0144640 2021-11-21T21:54:31.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.322+0000 7f50dffff700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f50c4053570 0x7f50c4055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.326+0000 7f50dffff700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f50c4053570 0x7f50c4055a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f50c8002f70 tx=0x7f50c8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.326+0000 7f50dd7fa700 1 -- 172.21.2.116:0/3936076504 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f50e013e430 con 0x7f50e0144640 2021-11-21T21:54:31.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f50e013e430 con 0x7f50e0144640 2021-11-21T21:54:31.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50dd7fa700 1 -- 172.21.2.116:0/3936076504 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50e013e430 con 0x7f50e0144640 2021-11-21T21:54:31.413 INFO:teuthology.orchestra.run.gibba016.stdout:55834574853 2021-11-21T21:54:31.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f50c4053570 msgr2=0x7f50c4055a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f50c4053570 0x7f50c4055a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f50c8002f70 tx=0x7f50c8015000).stop 2021-11-21T21:54:31.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e0144640 msgr2=0x7f50e013cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e0144640 0x7f50e013cea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f50d8003330 tx=0x7f50d80091a0).stop 2021-11-21T21:54:31.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 shutdown_connections 2021-11-21T21:54:31.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f50c4053570 0x7f50c4055a30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50e0144640 0x7f50e013cea0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50e01426e0 0x7f50e00637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 --2- 172.21.2.116:0/3936076504 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50e00ad470 0x7f50e013fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 >> 172.21.2.116:0/3936076504 conn(0x7f50e00b5730 msgr2=0x7f50e00b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 shutdown_connections 2021-11-21T21:54:31.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.410+0000 7f50e6777700 1 -- 172.21.2.116:0/3936076504 wait complete. 2021-11-21T21:54:31.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2021-11-21T21:54:31.420 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2021-11-21T21:54:31.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33df245700 1 Processor -- start 2021-11-21T21:54:31.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33df245700 1 -- start start 2021-11-21T21:54:31.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33df245700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 0x7f33d805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33df245700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f33d80af620 0x7f33d8142f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33df245700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80b06c0 con 0x7f33d80afa80 2021-11-21T21:54:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33df245700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d80adf20 con 0x7f33d80af620 2021-11-21T21:54:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33df245700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80b2470 con 0x7f33d80b02e0 2021-11-21T21:54:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33d7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f33d80af620 0x7f33d8142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33d7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f33d80af620 0x7f33d8142f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44870/0 (socket says 172.21.2.116:44870) 2021-11-21T21:54:31.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33d7fff700 1 -- 172.21.2.116:0/3429887779 learned_addr learned my addr 172.21.2.116:0/3429887779 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33d7fff700 1 -- 172.21.2.116:0/3429887779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 msgr2=0x7f33d805f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33d7fff700 1 --2- 172.21.2.116:0/3429887779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 0x7f33d805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33d7fff700 1 -- 172.21.2.116:0/3429887779 >> v1:172.21.2.117:6789/0 conn(0x7f33d80afa80 legacy=0x7f33d8143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.466+0000 7f33d7fff700 1 -- 172.21.2.116:0/3429887779 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80b26d0 con 0x7f33d80af620 2021-11-21T21:54:31.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 --2- 172.21.2.116:0/3429887779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80af620 0x7f33d8142f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f33d80b2370 tx=0x7f33cc006f40).ready entity=mon.0 client_cookie=9aaae37a27cfdb1b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d77fe700 1 -- 172.21.2.116:0/3429887779 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33cc00dc30 con 0x7f33d80af620 2021-11-21T21:54:31.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d77fe700 1 -- 172.21.2.116:0/3429887779 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc000e90 con 0x7f33d80af620 2021-11-21T21:54:31.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d77fe700 1 -- 172.21.2.116:0/3429887779 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33cc00be90 con 0x7f33d80af620 2021-11-21T21:54:31.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/3429887779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80af620 msgr2=0x7f33d8142f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 --2- 172.21.2.116:0/3429887779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80af620 0x7f33d8142f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f33d80b2370 tx=0x7f33cc006f40).stop 2021-11-21T21:54:31.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/3429887779 shutdown_connections 2021-11-21T21:54:31.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 --2- 172.21.2.116:0/3429887779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80af620 0x7f33d8142f70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 --2- 172.21.2.116:0/3429887779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 0x7f33d805f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/3429887779 >> 172.21.2.116:0/3429887779 conn(0x7f33d80b5ff0 msgr2=0x7f33d80b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/3429887779 shutdown_connections 2021-11-21T21:54:31.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/3429887779 wait complete. 2021-11-21T21:54:31.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 Processor -- start 2021-11-21T21:54:31.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- start start 2021-11-21T21:54:31.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33d80af620 0x7f33d80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80afa80 0x7f33d80a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 0x7f33d80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8152fa0 con 0x7f33d80afa80 2021-11-21T21:54:31.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8152320 con 0x7f33d80af620 2021-11-21T21:54:31.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d81525c0 con 0x7f33d80b02e0 2021-11-21T21:54:31.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80afa80 0x7f33d80a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80afa80 0x7f33d80a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44874/0 (socket says 172.21.2.116:44874) 2021-11-21T21:54:31.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 -- 172.21.2.116:0/1644172349 learned_addr learned my addr 172.21.2.116:0/1644172349 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 -- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 msgr2=0x7f33d80a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:31.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 0x7f33d80a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 -- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33d80af620 msgr2=0x7f33d80a2b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:31.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33d80af620 0x7f33d80a2b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 -- 172.21.2.116:0/1644172349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc00b460 con 0x7f33d80afa80 2021-11-21T21:54:31.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d7fff700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80afa80 0x7f33d80a3520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f33d80b2370 tx=0x7f33d80b7000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d5ffb700 1 -- 172.21.2.116:0/1644172349 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33cc01d070 con 0x7f33d80afa80 2021-11-21T21:54:31.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d5ffb700 1 -- 172.21.2.116:0/1644172349 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc010e90 con 0x7f33d80afa80 2021-11-21T21:54:31.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d8152860 con 0x7f33d80afa80 2021-11-21T21:54:31.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d80a3070 con 0x7f33d80afa80 2021-11-21T21:54:31.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d5ffb700 1 -- 172.21.2.116:0/1644172349 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33cc022600 con 0x7f33d80afa80 2021-11-21T21:54:31.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d80b0da0 con 0x7f33d80afa80 2021-11-21T21:54:31.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d5ffb700 1 -- 172.21.2.116:0/1644172349 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f33cc029070 con 0x7f33d80afa80 2021-11-21T21:54:31.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d5ffb700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33b80536f0 0x7f33b8055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.470+0000 7f33d5ffb700 1 -- 172.21.2.116:0/1644172349 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f33cc06cc80 con 0x7f33d80afa80 2021-11-21T21:54:31.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.474+0000 7f33dcfe1700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33b80536f0 0x7f33b8055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.474+0000 7f33dcfe1700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33b80536f0 0x7f33b8055bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f33c8002f70 tx=0x7f33c8006960).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.474+0000 7f33d5ffb700 1 -- 172.21.2.116:0/1644172349 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33cc073080 con 0x7f33d80afa80 2021-11-21T21:54:31.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.554+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f33d80573d0 con 0x7f33d80afa80 2021-11-21T21:54:31.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33d5ffb700 1 -- 172.21.2.116:0/1644172349 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f33cc043cd0 con 0x7f33d80afa80 2021-11-21T21:54:31.560 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:31.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33b80536f0 msgr2=0x7f33b8055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33b80536f0 0x7f33b8055bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f33c8002f70 tx=0x7f33c8006960).stop 2021-11-21T21:54:31.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80afa80 msgr2=0x7f33d80a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80afa80 0x7f33d80a3520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f33d80b2370 tx=0x7f33d80b7000).stop 2021-11-21T21:54:31.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 shutdown_connections 2021-11-21T21:54:31.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33d80b02e0 0x7f33d80a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33b80536f0 0x7f33b8055bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33d80afa80 0x7f33d80a3520 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 --2- 172.21.2.116:0/1644172349 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33d80af620 0x7f33d80a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 >> 172.21.2.116:0/1644172349 conn(0x7f33d80b5ff0 msgr2=0x7f33d805c410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 shutdown_connections 2021-11-21T21:54:31.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.558+0000 7f33df245700 1 -- 172.21.2.116:0/1644172349 wait complete. 2021-11-21T21:54:31.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2021-11-21T21:54:31.568 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2021-11-21T21:54:31.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadcb397700 1 Processor -- start 2021-11-21T21:54:31.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadcb397700 1 -- start start 2021-11-21T21:54:31.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadcb397700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fadc41478c0 0x7fadc40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadcb397700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc4147e80 con 0x7fadc4144810 2021-11-21T21:54:31.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadcb397700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc40af5a0 con 0x7fadc40abce0 2021-11-21T21:54:31.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadcb397700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc40b1490 con 0x7fadc41478c0 2021-11-21T21:54:31.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadc9934700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fadc41478c0 0x7fadc40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadc9934700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fadc41478c0 0x7fadc40acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44882/0 (socket says 172.21.2.116:44882) 2021-11-21T21:54:31.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadc9934700 1 -- 172.21.2.116:0/2294187860 learned_addr learned my addr 172.21.2.116:0/2294187860 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadc9934700 1 -- 172.21.2.116:0/2294187860 >> v1:172.21.2.117:6789/0 conn(0x7fadc40abce0 legacy=0x7fadc40ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:31.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadc9934700 1 -- 172.21.2.116:0/2294187860 >> v1:172.21.2.116:6789/0 conn(0x7fadc4144810 legacy=0x7fadc40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadc9934700 1 -- 172.21.2.116:0/2294187860 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc40b1fd0 con 0x7fadc41478c0 2021-11-21T21:54:31.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadc9934700 1 --2- 172.21.2.116:0/2294187860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc40acd10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fadc40b4e00 tx=0x7fadbc009290).ready entity=mon.0 client_cookie=8ff55051a5054b44 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadb7fff700 1 -- 172.21.2.116:0/2294187860 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadbc00b860 con 0x7fadc41478c0 2021-11-21T21:54:31.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadb7fff700 1 -- 172.21.2.116:0/2294187860 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00b9c0 con 0x7fadc41478c0 2021-11-21T21:54:31.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.614+0000 7fadb7fff700 1 -- 172.21.2.116:0/2294187860 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadbc00bb30 con 0x7fadc41478c0 2021-11-21T21:54:31.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/2294187860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 msgr2=0x7fadc40acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 --2- 172.21.2.116:0/2294187860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc40acd10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fadc40b4e00 tx=0x7fadbc009290).stop 2021-11-21T21:54:31.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/2294187860 shutdown_connections 2021-11-21T21:54:31.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 --2- 172.21.2.116:0/2294187860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc40acd10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/2294187860 >> 172.21.2.116:0/2294187860 conn(0x7fadc405bd20 msgr2=0x7fadc405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/2294187860 shutdown_connections 2021-11-21T21:54:31.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/2294187860 wait complete. 2021-11-21T21:54:31.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 Processor -- start 2021-11-21T21:54:31.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- start start 2021-11-21T21:54:31.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fadc40abce0 0x7fadc40b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fadc4144810 0x7fadc4148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc4156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc413ce30 con 0x7fadc41478c0 2021-11-21T21:54:31.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40b3c30 con 0x7fadc4144810 2021-11-21T21:54:31.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40b3ed0 con 0x7fadc40abce0 2021-11-21T21:54:31.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc4156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc4156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44890/0 (socket says 172.21.2.116:44890) 2021-11-21T21:54:31.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 -- 172.21.2.116:0/3382674510 learned_addr learned my addr 172.21.2.116:0/3382674510 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 -- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fadc40abce0 msgr2=0x7fadc40b46b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:31.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fadc40abce0 0x7fadc40b46b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 -- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fadc4144810 msgr2=0x7fadc4148880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:31.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fadc4144810 0x7fadc4148880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 -- 172.21.2.116:0/3382674510 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc009530 con 0x7fadc41478c0 2021-11-21T21:54:31.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadc9934700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc4156f70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fadc40b4e00 tx=0x7fadbc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadb67fc700 1 -- 172.21.2.116:0/3382674510 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadbc01f3f0 con 0x7fadc41478c0 2021-11-21T21:54:31.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadb67fc700 1 -- 172.21.2.116:0/3382674510 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc01f550 con 0x7fadc41478c0 2021-11-21T21:54:31.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc40b4170 con 0x7fadc41478c0 2021-11-21T21:54:31.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadb67fc700 1 -- 172.21.2.116:0/3382674510 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadbc01f8b0 con 0x7fadc41478c0 2021-11-21T21:54:31.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc4157460 con 0x7fadc41478c0 2021-11-21T21:54:31.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc40573d0 con 0x7fadc41478c0 2021-11-21T21:54:31.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadb67fc700 1 -- 172.21.2.116:0/3382674510 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fadbc026030 con 0x7fadc41478c0 2021-11-21T21:54:31.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadb67fc700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fada40536a0 0x7fada4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.618+0000 7fadb67fc700 1 -- 172.21.2.116:0/3382674510 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fadbc06c7d0 con 0x7fadc41478c0 2021-11-21T21:54:31.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.622+0000 7fadc9133700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fada40536a0 0x7fada4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.622+0000 7fadc9133700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fada40536a0 0x7fada4055b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fadc4062500 tx=0x7fadb80077b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.622+0000 7fadb67fc700 1 -- 172.21.2.116:0/3382674510 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadbc043820 con 0x7fadc41478c0 2021-11-21T21:54:31.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fadc40ad630 con 0x7fadc41478c0 2021-11-21T21:54:31.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadb67fc700 1 -- 172.21.2.116:0/3382674510 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fadbc024090 con 0x7fadc41478c0 2021-11-21T21:54:31.709 INFO:teuthology.orchestra.run.gibba016.stdout:55834574854 2021-11-21T21:54:31.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fada40536a0 msgr2=0x7fada4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fada40536a0 0x7fada4055b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fadc4062500 tx=0x7fadb80077b0).stop 2021-11-21T21:54:31.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 msgr2=0x7fadc4156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc4156f70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fadc40b4e00 tx=0x7fadbc016000).stop 2021-11-21T21:54:31.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 shutdown_connections 2021-11-21T21:54:31.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fada40536a0 0x7fada4055b60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fadc41478c0 0x7fadc4156f70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fadc4144810 0x7fadc4148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 --2- 172.21.2.116:0/3382674510 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fadc40abce0 0x7fadc40b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 >> 172.21.2.116:0/3382674510 conn(0x7fadc405bd20 msgr2=0x7fadc413fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 shutdown_connections 2021-11-21T21:54:31.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.706+0000 7fadcb397700 1 -- 172.21.2.116:0/3382674510 wait complete. 2021-11-21T21:54:31.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2021-11-21T21:54:31.718 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2021-11-21T21:54:31.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 Processor -- start 2021-11-21T21:54:31.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- start start 2021-11-21T21:54:31.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe9180abce0 0x7fe9181478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe918147e80 con 0x7fe918144810 2021-11-21T21:54:31.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9180af5a0 con 0x7fe9180ac600 2021-11-21T21:54:31.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9180b1490 con 0x7fe9180abce0 2021-11-21T21:54:31.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe918144810 0x7fe9180b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56852/0 (socket says 172.21.2.116:56852) 2021-11-21T21:54:31.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 -- 172.21.2.116:0/4063936135 learned_addr learned my addr 172.21.2.116:0/4063936135 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:31.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455018218 0 0) 0x7fe918147e80 con 0x7fe918144810 2021-11-21T21:54:31.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f8003260 con 0x7fe918144810 2021-11-21T21:54:31.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012725169 0 0) 0x7fe8f8003260 con 0x7fe918144810 2021-11-21T21:54:31.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 >> v2:172.21.2.117:3300/0 conn(0x7fe9180abce0 msgr2=0x7fe9181478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 --2- 172.21.2.116:0/4063936135 >> v2:172.21.2.117:3300/0 conn(0x7fe9180abce0 0x7fe9181478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 >> v1:172.21.2.117:6789/0 conn(0x7fe9180ac600 legacy=0x7fe9180aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180b1fd0 con 0x7fe918144810 2021-11-21T21:54:31.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe908001660 con 0x7fe918144810 2021-11-21T21:54:31.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 >> v1:172.21.2.116:6789/0 conn(0x7fe918144810 legacy=0x7fe9180b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe8f8003750 0x7fe8f8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe8f8004500 0x7fe8f8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe8f8004f90 0x7fe8f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180b1490 con 0x7fe8f8003750 2021-11-21T21:54:31.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f800be80 con 0x7fe8f8004f90 2021-11-21T21:54:31.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f800c040 con 0x7fe8f8004500 2021-11-21T21:54:31.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe8f8003750 0x7fe8f8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 -- 172.21.2.116:0/4063936135 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe8f8004500 msgr2=0x7fe8f8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe8f8004500 0x7fe8f8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 -- 172.21.2.116:0/4063936135 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe8f8004f90 msgr2=0x7fe8f800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe8f8004f90 0x7fe8f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 -- 172.21.2.116:0/4063936135 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8003260 con 0x7fe8f8003750 2021-11-21T21:54:31.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe917fff700 1 -- 172.21.2.116:0/4063936135 reap_dead start 2021-11-21T21:54:31.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9177fe700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe8f8003750 0x7fe8f8003b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fe9180b4e00 tx=0x7fe918146e80).ready entity=mon.0 client_cookie=78aa25a372734fc7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9080071a0 con 0x7fe8f8003750 2021-11-21T21:54:31.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe908007300 con 0x7fe8f8003750 2021-11-21T21:54:31.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe9167fc700 1 -- 172.21.2.116:0/4063936135 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe90800c8f0 con 0x7fe8f8003750 2021-11-21T21:54:31.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- 172.21.2.116:0/4063936135 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe8f8003750 msgr2=0x7fe8f8003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe8f8003750 0x7fe8f8003b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fe9180b4e00 tx=0x7fe918146e80).stop 2021-11-21T21:54:31.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- 172.21.2.116:0/4063936135 shutdown_connections 2021-11-21T21:54:31.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/4063936135 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe8f8003750 0x7fe8f8003b50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- 172.21.2.116:0/4063936135 >> 172.21.2.116:0/4063936135 conn(0x7fe91805bd20 msgr2=0x7fe91805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- 172.21.2.116:0/4063936135 shutdown_connections 2021-11-21T21:54:31.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.766+0000 7fe91dc8a700 1 -- 172.21.2.116:0/4063936135 wait complete. 2021-11-21T21:54:31.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 Processor -- start 2021-11-21T21:54:31.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 -- start start 2021-11-21T21:54:31.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe8f8003750 0x7fe918143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe918145680 0x7fe9180b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9180b35c0 0x7fe91813cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe918145aa0 con 0x7fe918145680 2021-11-21T21:54:31.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe918144dc0 con 0x7fe8f8003750 2021-11-21T21:54:31.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe918145060 con 0x7fe9180b35c0 2021-11-21T21:54:31.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe918145680 0x7fe9180b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe918145680 0x7fe9180b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44912/0 (socket says 172.21.2.116:44912) 2021-11-21T21:54:31.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 -- 172.21.2.116:0/3482940327 learned_addr learned my addr 172.21.2.116:0/3482940327 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 -- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9180b35c0 msgr2=0x7fe91813cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9180b35c0 0x7fe91813cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 -- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe8f8003750 msgr2=0x7fe918143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe8f8003750 0x7fe918143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 -- 172.21.2.116:0/3482940327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180b2c30 con 0x7fe918145680 2021-11-21T21:54:31.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe916ffd700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe918145680 0x7fe9180b30d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fe900002f70 tx=0x7fe900006db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe914ff9700 1 -- 172.21.2.116:0/3482940327 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe900009e80 con 0x7fe918145680 2021-11-21T21:54:31.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe914ff9700 1 -- 172.21.2.116:0/3482940327 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe900013460 con 0x7fe918145680 2021-11-21T21:54:31.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe914ff9700 1 -- 172.21.2.116:0/3482940327 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9000137c0 con 0x7fe918145680 2021-11-21T21:54:31.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9180644c0 con 0x7fe918145680 2021-11-21T21:54:31.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91813d420 con 0x7fe918145680 2021-11-21T21:54:31.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9180b2dc0 con 0x7fe918145680 2021-11-21T21:54:31.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe914ff9700 1 -- 172.21.2.116:0/3482940327 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe90000f070 con 0x7fe918145680 2021-11-21T21:54:31.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe914ff9700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe8fc053610 0x7fe8fc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe914ff9700 1 -- 172.21.2.116:0/3482940327 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe90006b190 con 0x7fe918145680 2021-11-21T21:54:31.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.770+0000 7fe9177fe700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe8fc053610 0x7fe8fc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.774+0000 7fe9177fe700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe8fc053610 0x7fe8fc055ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe9180b4e00 tx=0x7fe908015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.774+0000 7fe914ff9700 1 -- 172.21.2.116:0/3482940327 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe9000422f0 con 0x7fe918145680 2021-11-21T21:54:31.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.854+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe918145220 con 0x7fe918145680 2021-11-21T21:54:31.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.854+0000 7fe914ff9700 1 -- 172.21.2.116:0/3482940327 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe900045910 con 0x7fe918145680 2021-11-21T21:54:31.860 INFO:teuthology.orchestra.run.gibba016.stdout:55834574852 2021-11-21T21:54:31.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe8fc053610 msgr2=0x7fe8fc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe8fc053610 0x7fe8fc055ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe9180b4e00 tx=0x7fe908015000).stop 2021-11-21T21:54:31.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe918145680 msgr2=0x7fe9180b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe918145680 0x7fe9180b30d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fe900002f70 tx=0x7fe900006db0).stop 2021-11-21T21:54:31.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 shutdown_connections 2021-11-21T21:54:31.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9180b35c0 0x7fe91813cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe8fc053610 0x7fe8fc055ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe918145680 0x7fe9180b30d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 --2- 172.21.2.116:0/3482940327 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe8f8003750 0x7fe918143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 >> 172.21.2.116:0/3482940327 conn(0x7fe91805bd20 msgr2=0x7fe91805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 shutdown_connections 2021-11-21T21:54:31.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.858+0000 7fe91dc8a700 1 -- 172.21.2.116:0/3482940327 wait complete. 2021-11-21T21:54:31.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2021-11-21T21:54:31.867 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2021-11-21T21:54:31.868 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-11-21T21:54:31.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.918+0000 7fe175aa0700 1 Processor -- start 2021-11-21T21:54:31.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.918+0000 7fe175aa0700 1 -- start start 2021-11-21T21:54:31.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.918+0000 7fe175aa0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe1700ad780 0x7fe17005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.918+0000 7fe175aa0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.918+0000 7fe175aa0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1700adb60 con 0x7fe1700af7e0 2021-11-21T21:54:31.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.918+0000 7fe175aa0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe170143160 con 0x7fe1700ad780 2021-11-21T21:54:31.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.918+0000 7fe175aa0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b0ce0 con 0x7fe1700affc0 2021-11-21T21:54:31.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16f7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe1700ad780 0x7fe17005f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16f7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe1700ad780 0x7fe17005f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:46086/0 (socket says 172.21.2.116:46086) 2021-11-21T21:54:31.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16f7fe700 1 -- 172.21.2.116:0/1811255845 learned_addr learned my addr 172.21.2.116:0/1811255845 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 -- 172.21.2.116:0/1811255845 >> v2:172.21.2.117:3300/0 conn(0x7fe1700ad780 msgr2=0x7fe17005f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:31.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 --2- 172.21.2.116:0/1811255845 >> v2:172.21.2.117:3300/0 conn(0x7fe1700ad780 0x7fe17005f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 -- 172.21.2.116:0/1811255845 >> v1:172.21.2.117:6789/0 conn(0x7fe1700af7e0 legacy=0x7fe1700b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 -- 172.21.2.116:0/1811255845 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700b0a40 con 0x7fe1700affc0 2021-11-21T21:54:31.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16f7fe700 1 --2- 172.21.2.116:0/1811255845 >> v2:172.21.2.117:3300/0 conn(0x7fe1700ad780 0x7fe17005f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:31.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 --2- 172.21.2.116:0/1811255845 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170143860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe158002f70 tx=0x7fe158006df0).ready entity=mon.2 client_cookie=ac127df5b3cfc728 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16e7fc700 1 -- 172.21.2.116:0/1811255845 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15800d570 con 0x7fe1700affc0 2021-11-21T21:54:31.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16e7fc700 1 -- 172.21.2.116:0/1811255845 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158000e90 con 0x7fe1700affc0 2021-11-21T21:54:31.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16e7fc700 1 -- 172.21.2.116:0/1811255845 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15800d8c0 con 0x7fe1700affc0 2021-11-21T21:54:31.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- 172.21.2.116:0/1811255845 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 msgr2=0x7fe170143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 --2- 172.21.2.116:0/1811255845 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170143860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe158002f70 tx=0x7fe158006df0).stop 2021-11-21T21:54:31.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- 172.21.2.116:0/1811255845 shutdown_connections 2021-11-21T21:54:31.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 --2- 172.21.2.116:0/1811255845 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170143860 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 --2- 172.21.2.116:0/1811255845 >> v2:172.21.2.117:3300/0 conn(0x7fe1700ad780 0x7fe17005f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:31.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- 172.21.2.116:0/1811255845 >> 172.21.2.116:0/1811255845 conn(0x7fe1700b6000 msgr2=0x7fe1700b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:31.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- 172.21.2.116:0/1811255845 shutdown_connections 2021-11-21T21:54:31.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- 172.21.2.116:0/1811255845 wait complete. 2021-11-21T21:54:31.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 Processor -- start 2021-11-21T21:54:31.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- start start 2021-11-21T21:54:31.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1700ad780 0x7fe1700665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1700af7e0 0x7fe1700aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170065c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700ac9e0 con 0x7fe1700af7e0 2021-11-21T21:54:31.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe170067b00 con 0x7fe1700ad780 2021-11-21T21:54:31.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe175aa0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700ac740 con 0x7fe1700affc0 2021-11-21T21:54:31.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1700af7e0 0x7fe1700aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1700af7e0 0x7fe1700aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44922/0 (socket says 172.21.2.116:44922) 2021-11-21T21:54:31.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.922+0000 7fe16effd700 1 -- 172.21.2.116:0/3924851759 learned_addr learned my addr 172.21.2.116:0/3924851759 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:31.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16effd700 1 -- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 msgr2=0x7fe170065c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16effd700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170065c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16effd700 1 -- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1700ad780 msgr2=0x7fe1700665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:31.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16effd700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1700ad780 0x7fe1700665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:31.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16effd700 1 -- 172.21.2.116:0/3924851759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16000b460 con 0x7fe1700af7e0 2021-11-21T21:54:31.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16effd700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1700af7e0 0x7fe1700aa920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fe158002f70 tx=0x7fe158014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16cff9700 1 -- 172.21.2.116:0/3924851759 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15800dea0 con 0x7fe1700af7e0 2021-11-21T21:54:31.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16cff9700 1 -- 172.21.2.116:0/3924851759 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15801e400 con 0x7fe1700af7e0 2021-11-21T21:54:31.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15800b180 con 0x7fe1700af7e0 2021-11-21T21:54:31.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16cff9700 1 -- 172.21.2.116:0/3924851759 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15801e760 con 0x7fe1700af7e0 2021-11-21T21:54:31.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1700aa010 con 0x7fe1700af7e0 2021-11-21T21:54:31.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1701433c0 con 0x7fe1700af7e0 2021-11-21T21:54:31.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16cff9700 1 -- 172.21.2.116:0/3924851759 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe158025030 con 0x7fe1700af7e0 2021-11-21T21:54:31.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16cff9700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe154053580 0x7fe154055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:31.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16cff9700 1 -- 172.21.2.116:0/3924851759 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe15806c760 con 0x7fe1700af7e0 2021-11-21T21:54:31.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16f7fe700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe154053580 0x7fe154055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:31.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16f7fe700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe154053580 0x7fe154055a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe16000e000 tx=0x7fe160006f40).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:31.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:31.926+0000 7fe16cff9700 1 -- 172.21.2.116:0/3924851759 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1580438c0 con 0x7fe1700af7e0 2021-11-21T21:54:32.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.010+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1700ac5c0 con 0x7fe154053580 2021-11-21T21:54:32.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe16cff9700 1 -- 172.21.2.116:0/3924851759 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82205 (secure 0 0 0) 0x7fe160000e90 con 0x7fe154053580 2021-11-21T21:54:32.019 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:32.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe154053580 msgr2=0x7fe154055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe154053580 0x7fe154055a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe16000e000 tx=0x7fe160006f40).stop 2021-11-21T21:54:32.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1700af7e0 msgr2=0x7fe1700aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1700af7e0 0x7fe1700aa920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fe158002f70 tx=0x7fe158014000).stop 2021-11-21T21:54:32.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 shutdown_connections 2021-11-21T21:54:32.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1700affc0 0x7fe170065c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe154053580 0x7fe154055a40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1700af7e0 0x7fe1700aa920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 --2- 172.21.2.116:0/3924851759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1700ad780 0x7fe1700665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 >> 172.21.2.116:0/3924851759 conn(0x7fe1700b6000 msgr2=0x7fe170143780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 shutdown_connections 2021-11-21T21:54:32.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.014+0000 7fe175aa0700 1 -- 172.21.2.116:0/3924851759 wait complete. 2021-11-21T21:54:32.023 INFO:teuthology.orchestra.run.gibba016.stderr:dumped all 2021-11-21T21:54:32.025 INFO:teuthology.orchestra.run.gibba016.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2021-11-21T21:54:31.051220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":704299008,"kb_used":6087020,"kb_used_data":698213028,"kb_used_omap":589,"kb_used_meta":0,"kb_avail":698211988,"statfs":{"total":721202184192,"available":714969075712,"internally_reserved":1064960,"allocated":714970140672,"data_stored":714970140672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":603284,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.138813"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.936765+0000","last_change":"2021-11-21T21:54:25.936621+0000","last_active":"2021-11-21T21:54:25.936765+0000","last_peered":"2021-11-21T21:54:25.936765+0000","last_clean":"2021-11-21T21:54:25.936765+0000","last_became_active":"2021-11-21T21:54:25.936370+0000","last_became_peered":"2021-11-21T21:54:25.936370+0000","last_unstale":"2021-11-21T21:54:25.936765+0000","last_undegraded":"2021-11-21T21:54:25.936765+0000","last_fullsized":"2021-11-21T21:54:25.936765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:26.788863+0000","last_change":"2021-11-21T21:54:16.981838+0000","last_active":"2021-11-21T21:54:26.788863+0000","last_peered":"2021-11-21T21:54:26.788863+0000","last_clean":"2021-11-21T21:54:26.788863+0000","last_became_active":"2021-11-21T21:54:16.981409+0000","last_became_peered":"2021-11-21T21:54:16.981409+0000","last_unstale":"2021-11-21T21:54:26.788863+0000","last_undegraded":"2021-11-21T21:54:26.788863+0000","last_fullsized":"2021-11-21T21:54:26.788863+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_clean_scrub_stamp":"2021-11-21T21:54:13.901475+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923551+0000","last_change":"2021-11-21T21:54:22.713500+0000","last_active":"2021-11-21T21:54:25.923551+0000","last_peered":"2021-11-21T21:54:25.923551+0000","last_clean":"2021-11-21T21:54:25.923551+0000","last_became_active":"2021-11-21T21:54:22.713118+0000","last_became_peered":"2021-11-21T21:54:22.713118+0000","last_unstale":"2021-11-21T21:54:25.923551+0000","last_undegraded":"2021-11-21T21:54:25.923551+0000","last_fullsized":"2021-11-21T21:54:25.923551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.910169+0000","last_change":"2021-11-21T21:54:24.724398+0000","last_active":"2021-11-21T21:54:24.910169+0000","last_peered":"2021-11-21T21:54:24.910169+0000","last_clean":"2021-11-21T21:54:24.910169+0000","last_became_active":"2021-11-21T21:54:24.724211+0000","last_became_peered":"2021-11-21T21:54:24.724211+0000","last_unstale":"2021-11-21T21:54:24.910169+0000","last_undegraded":"2021-11-21T21:54:24.910169+0000","last_fullsized":"2021-11-21T21:54:24.910169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.911804+0000","last_change":"2021-11-21T21:54:24.910954+0000","last_active":"2021-11-21T21:54:24.911804+0000","last_peered":"2021-11-21T21:54:24.911804+0000","last_clean":"2021-11-21T21:54:24.911804+0000","last_became_active":"2021-11-21T21:54:24.910760+0000","last_became_peered":"2021-11-21T21:54:24.910760+0000","last_unstale":"2021-11-21T21:54:24.911804+0000","last_undegraded":"2021-11-21T21:54:24.911804+0000","last_fullsized":"2021-11-21T21:54:24.911804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:26.788779+0000","last_change":"2021-11-21T21:54:22.713430+0000","last_active":"2021-11-21T21:54:26.788779+0000","last_peered":"2021-11-21T21:54:26.788779+0000","last_clean":"2021-11-21T21:54:26.788779+0000","last_became_active":"2021-11-21T21:54:22.712862+0000","last_became_peered":"2021-11-21T21:54:22.712862+0000","last_unstale":"2021-11-21T21:54:26.788779+0000","last_undegraded":"2021-11-21T21:54:26.788779+0000","last_fullsized":"2021-11-21T21:54:26.788779+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.909998+0000","last_change":"2021-11-21T21:54:24.724353+0000","last_active":"2021-11-21T21:54:24.909998+0000","last_peered":"2021-11-21T21:54:24.909998+0000","last_clean":"2021-11-21T21:54:24.909998+0000","last_became_active":"2021-11-21T21:54:24.724130+0000","last_became_peered":"2021-11-21T21:54:24.724130+0000","last_unstale":"2021-11-21T21:54:24.909998+0000","last_undegraded":"2021-11-21T21:54:24.909998+0000","last_fullsized":"2021-11-21T21:54:24.909998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.915142+0000","last_change":"2021-11-21T21:54:24.914982+0000","last_active":"2021-11-21T21:54:24.915142+0000","last_peered":"2021-11-21T21:54:24.915142+0000","last_clean":"2021-11-21T21:54:24.915142+0000","last_became_active":"2021-11-21T21:54:24.914758+0000","last_became_peered":"2021-11-21T21:54:24.914758+0000","last_unstale":"2021-11-21T21:54:24.915142+0000","last_undegraded":"2021-11-21T21:54:24.915142+0000","last_fullsized":"2021-11-21T21:54:24.915142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.912190+0000","last_change":"2021-11-21T21:54:22.711838+0000","last_active":"2021-11-21T21:54:24.912190+0000","last_peered":"2021-11-21T21:54:24.912190+0000","last_clean":"2021-11-21T21:54:24.912190+0000","last_became_active":"2021-11-21T21:54:22.711687+0000","last_became_peered":"2021-11-21T21:54:22.711687+0000","last_unstale":"2021-11-21T21:54:24.912190+0000","last_undegraded":"2021-11-21T21:54:24.912190+0000","last_fullsized":"2021-11-21T21:54:24.912190+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.942770+0000","last_change":"2021-11-21T21:54:24.724120+0000","last_active":"2021-11-21T21:54:24.942770+0000","last_peered":"2021-11-21T21:54:24.942770+0000","last_clean":"2021-11-21T21:54:24.942770+0000","last_became_active":"2021-11-21T21:54:24.723914+0000","last_became_peered":"2021-11-21T21:54:24.723914+0000","last_unstale":"2021-11-21T21:54:24.942770+0000","last_undegraded":"2021-11-21T21:54:24.942770+0000","last_fullsized":"2021-11-21T21:54:24.942770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.942134+0000","last_change":"2021-11-21T21:54:24.942009+0000","last_active":"2021-11-21T21:54:24.942134+0000","last_peered":"2021-11-21T21:54:24.942134+0000","last_clean":"2021-11-21T21:54:24.942134+0000","last_became_active":"2021-11-21T21:54:24.941840+0000","last_became_peered":"2021-11-21T21:54:24.941840+0000","last_unstale":"2021-11-21T21:54:24.942134+0000","last_undegraded":"2021-11-21T21:54:24.942134+0000","last_fullsized":"2021-11-21T21:54:24.942134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923515+0000","last_change":"2021-11-21T21:54:22.713478+0000","last_active":"2021-11-21T21:54:25.923515+0000","last_peered":"2021-11-21T21:54:25.923515+0000","last_clean":"2021-11-21T21:54:25.923515+0000","last_became_active":"2021-11-21T21:54:22.713254+0000","last_became_peered":"2021-11-21T21:54:22.713254+0000","last_unstale":"2021-11-21T21:54:25.923515+0000","last_undegraded":"2021-11-21T21:54:25.923515+0000","last_fullsized":"2021-11-21T21:54:25.923515+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'5","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.935504+0000","last_change":"2021-11-21T21:54:24.724548+0000","last_active":"2021-11-21T21:54:25.935504+0000","last_peered":"2021-11-21T21:54:25.935504+0000","last_clean":"2021-11-21T21:54:25.935504+0000","last_became_active":"2021-11-21T21:54:24.724266+0000","last_became_peered":"2021-11-21T21:54:24.724266+0000","last_unstale":"2021-11-21T21:54:25.935504+0000","last_undegraded":"2021-11-21T21:54:25.935504+0000","last_fullsized":"2021-11-21T21:54:25.935504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.941782+0000","last_change":"2021-11-21T21:54:24.914955+0000","last_active":"2021-11-21T21:54:24.941782+0000","last_peered":"2021-11-21T21:54:24.941782+0000","last_clean":"2021-11-21T21:54:24.941782+0000","last_became_active":"2021-11-21T21:54:24.914773+0000","last_became_peered":"2021-11-21T21:54:24.914773+0000","last_unstale":"2021-11-21T21:54:24.941782+0000","last_undegraded":"2021-11-21T21:54:24.941782+0000","last_fullsized":"2021-11-21T21:54:24.941782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923528+0000","last_change":"2021-11-21T21:54:24.724846+0000","last_active":"2021-11-21T21:54:25.923528+0000","last_peered":"2021-11-21T21:54:25.923528+0000","last_clean":"2021-11-21T21:54:25.923528+0000","last_became_active":"2021-11-21T21:54:24.724707+0000","last_became_peered":"2021-11-21T21:54:24.724707+0000","last_unstale":"2021-11-21T21:54:25.923528+0000","last_undegraded":"2021-11-21T21:54:25.923528+0000","last_fullsized":"2021-11-21T21:54:25.923528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.908638+0000","last_change":"2021-11-21T21:54:22.786234+0000","last_active":"2021-11-21T21:54:24.908638+0000","last_peered":"2021-11-21T21:54:24.908638+0000","last_clean":"2021-11-21T21:54:24.908638+0000","last_became_active":"2021-11-21T21:54:22.786041+0000","last_became_peered":"2021-11-21T21:54:22.786041+0000","last_unstale":"2021-11-21T21:54:24.908638+0000","last_undegraded":"2021-11-21T21:54:24.908638+0000","last_fullsized":"2021-11-21T21:54:24.908638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.915106+0000","last_change":"2021-11-21T21:54:24.915014+0000","last_active":"2021-11-21T21:54:24.915106+0000","last_peered":"2021-11-21T21:54:24.915106+0000","last_clean":"2021-11-21T21:54:24.915106+0000","last_became_active":"2021-11-21T21:54:24.914814+0000","last_became_peered":"2021-11-21T21:54:24.914814+0000","last_unstale":"2021-11-21T21:54:24.915106+0000","last_undegraded":"2021-11-21T21:54:24.915106+0000","last_fullsized":"2021-11-21T21:54:24.915106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.942897+0000","last_change":"2021-11-21T21:54:24.724216+0000","last_active":"2021-11-21T21:54:24.942897+0000","last_peered":"2021-11-21T21:54:24.942897+0000","last_clean":"2021-11-21T21:54:24.942897+0000","last_became_active":"2021-11-21T21:54:24.723947+0000","last_became_peered":"2021-11-21T21:54:24.723947+0000","last_unstale":"2021-11-21T21:54:24.942897+0000","last_undegraded":"2021-11-21T21:54:24.942897+0000","last_fullsized":"2021-11-21T21:54:24.942897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:26.788853+0000","last_change":"2021-11-21T21:54:22.788120+0000","last_active":"2021-11-21T21:54:26.788853+0000","last_peered":"2021-11-21T21:54:26.788853+0000","last_clean":"2021-11-21T21:54:26.788853+0000","last_became_active":"2021-11-21T21:54:22.787305+0000","last_became_peered":"2021-11-21T21:54:22.787305+0000","last_unstale":"2021-11-21T21:54:26.788853+0000","last_undegraded":"2021-11-21T21:54:26.788853+0000","last_fullsized":"2021-11-21T21:54:26.788853+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.914781+0000","last_change":"2021-11-21T21:54:24.914781+0000","last_active":"2021-11-21T21:54:24.914781+0000","last_peered":"2021-11-21T21:54:24.914781+0000","last_clean":"2021-11-21T21:54:24.914781+0000","last_became_active":"2021-11-21T21:54:24.914595+0000","last_became_peered":"2021-11-21T21:54:24.914595+0000","last_unstale":"2021-11-21T21:54:24.914781+0000","last_undegraded":"2021-11-21T21:54:24.914781+0000","last_fullsized":"2021-11-21T21:54:24.914781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'7","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.914967+0000","last_change":"2021-11-21T21:54:24.724673+0000","last_active":"2021-11-21T21:54:24.914967+0000","last_peered":"2021-11-21T21:54:24.914967+0000","last_clean":"2021-11-21T21:54:24.914967+0000","last_became_active":"2021-11-21T21:54:24.724477+0000","last_became_peered":"2021-11-21T21:54:24.724477+0000","last_unstale":"2021-11-21T21:54:24.914967+0000","last_undegraded":"2021-11-21T21:54:24.914967+0000","last_fullsized":"2021-11-21T21:54:24.914967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.908642+0000","last_change":"2021-11-21T21:54:22.713690+0000","last_active":"2021-11-21T21:54:24.908642+0000","last_peered":"2021-11-21T21:54:24.908642+0000","last_clean":"2021-11-21T21:54:24.908642+0000","last_became_active":"2021-11-21T21:54:22.713554+0000","last_became_peered":"2021-11-21T21:54:22.713554+0000","last_unstale":"2021-11-21T21:54:24.908642+0000","last_undegraded":"2021-11-21T21:54:24.908642+0000","last_fullsized":"2021-11-21T21:54:24.908642+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.935482+0000","last_change":"2021-11-21T21:54:24.916112+0000","last_active":"2021-11-21T21:54:25.935482+0000","last_peered":"2021-11-21T21:54:25.935482+0000","last_clean":"2021-11-21T21:54:25.935482+0000","last_became_active":"2021-11-21T21:54:24.915840+0000","last_became_peered":"2021-11-21T21:54:24.915840+0000","last_unstale":"2021-11-21T21:54:25.935482+0000","last_undegraded":"2021-11-21T21:54:25.935482+0000","last_fullsized":"2021-11-21T21:54:25.935482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.912474+0000","last_change":"2021-11-21T21:54:24.723551+0000","last_active":"2021-11-21T21:54:24.912474+0000","last_peered":"2021-11-21T21:54:24.912474+0000","last_clean":"2021-11-21T21:54:24.912474+0000","last_became_active":"2021-11-21T21:54:24.723376+0000","last_became_peered":"2021-11-21T21:54:24.723376+0000","last_unstale":"2021-11-21T21:54:24.912474+0000","last_undegraded":"2021-11-21T21:54:24.912474+0000","last_fullsized":"2021-11-21T21:54:24.912474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923482+0000","last_change":"2021-11-21T21:54:22.714589+0000","last_active":"2021-11-21T21:54:25.923482+0000","last_peered":"2021-11-21T21:54:25.923482+0000","last_clean":"2021-11-21T21:54:25.923482+0000","last_became_active":"2021-11-21T21:54:22.714383+0000","last_became_peered":"2021-11-21T21:54:22.714383+0000","last_unstale":"2021-11-21T21:54:25.923482+0000","last_undegraded":"2021-11-21T21:54:25.923482+0000","last_fullsized":"2021-11-21T21:54:25.923482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760596,"kb_used_data":87276780,"kb_used_omap":76,"kb_used_meta":0,"kb_avail":87276780,"statfs":{"total":90150273024,"available":89371422720,"internally_reserved":0,"allocated":89371422720,"data_stored":89371422720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77938,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760588,"kb_used_data":87276788,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":87276788,"statfs":{"total":90150273024,"available":89371430912,"internally_reserved":0,"allocated":89371430912,"data_stored":89371430912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43305,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.786}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7030000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":761012,"kb_used_data":87276540,"kb_used_omap":78,"kb_used_meta":0,"kb_avail":87276364,"statfs":{"total":90150273024,"available":89370996736,"internally_reserved":180224,"allocated":89371176960,"data_stored":89371176960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80002,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760956,"kb_used_data":87276564,"kb_used_omap":57,"kb_used_meta":0,"kb_avail":87276420,"statfs":{"total":90150273024,"available":89371054080,"internally_reserved":147456,"allocated":89371201536,"data_stored":89371201536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58532,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.032}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760936,"kb_used_data":87276600,"kb_used_omap":65,"kb_used_meta":0,"kb_avail":87276440,"statfs":{"total":90150273024,"available":89371074560,"internally_reserved":163840,"allocated":89371238400,"data_stored":89371238400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":67441,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":761016,"kb_used_data":87276516,"kb_used_omap":67,"kb_used_meta":0,"kb_avail":87276360,"statfs":{"total":90150273024,"available":89370992640,"internally_reserved":159744,"allocated":89371152384,"data_stored":89371152384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":69462,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760864,"kb_used_data":87276672,"kb_used_omap":96,"kb_used_meta":0,"kb_avail":87276512,"statfs":{"total":90150273024,"available":89371148288,"internally_reserved":163840,"allocated":89371312128,"data_stored":89371312128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":99195,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":761052,"kb_used_data":87276568,"kb_used_omap":104,"kb_used_meta":0,"kb_avail":87276324,"statfs":{"total":90150273024,"available":89370955776,"internally_reserved":249856,"allocated":89371205632,"data_stored":89371205632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107409,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]}]}]}} 2021-11-21T21:54:32.028 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-11-21T21:54:32.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b9d7b8700 1 Processor -- start 2021-11-21T21:54:32.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b9d7b8700 1 -- start start 2021-11-21T21:54:32.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b9d7b8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0b980b24a0 0x7f0b980b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b9d7b8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 0x7f0b980ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b9d7b8700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b980b00d0 con 0x7f0b980aeb70 2021-11-21T21:54:32.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b9d7b8700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b980adba0 con 0x7f0b980b24a0 2021-11-21T21:54:32.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b9d7b8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980af7a0 con 0x7f0b980af360 2021-11-21T21:54:32.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b96ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0b980b24a0 0x7f0b980b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b977fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0b980aeb70 0x7f0b9813c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56888/0 (socket says 172.21.2.116:56888) 2021-11-21T21:54:32.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b977fe700 1 -- 172.21.2.116:0/2426278804 learned_addr learned my addr 172.21.2.116:0/2426278804 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:32.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.074+0000 7f0b96ffd700 1 -- 172.21.2.116:0/2426278804 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 msgr2=0x7f0b980ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 --2- 172.21.2.116:0/2426278804 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 0x7f0b980ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 -- 172.21.2.116:0/2426278804 >> v1:172.21.2.116:6789/0 conn(0x7f0b980aeb70 legacy=0x7f0b9813c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b95ffb700 1 -- 172.21.2.116:0/2426278804 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033394670 0 0) 0x7f0b980b00d0 con 0x7f0b980aeb70 2021-11-21T21:54:32.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 -- 172.21.2.116:0/2426278804 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980b2880 con 0x7f0b980b24a0 2021-11-21T21:54:32.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 --2- 172.21.2.116:0/2426278804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980b24a0 0x7f0b980b1a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0b80002f70 tx=0x7f0b80006df0).ready entity=mon.0 client_cookie=a7f718617ee9aa2a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b95ffb700 1 -- 172.21.2.116:0/2426278804 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b8000d570 con 0x7f0b980b24a0 2021-11-21T21:54:32.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b95ffb700 1 -- 172.21.2.116:0/2426278804 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80000e90 con 0x7f0b980b24a0 2021-11-21T21:54:32.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b95ffb700 1 -- 172.21.2.116:0/2426278804 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b8000d8c0 con 0x7f0b980b24a0 2021-11-21T21:54:32.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2426278804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980b24a0 msgr2=0x7f0b980b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2426278804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980b24a0 0x7f0b980b1a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0b80002f70 tx=0x7f0b80006df0).stop 2021-11-21T21:54:32.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2426278804 shutdown_connections 2021-11-21T21:54:32.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2426278804 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 0x7f0b980ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2426278804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980b24a0 0x7f0b980b1a40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2426278804 >> 172.21.2.116:0/2426278804 conn(0x7f0b9805fd00 msgr2=0x7f0b980600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2426278804 shutdown_connections 2021-11-21T21:54:32.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2426278804 wait complete. 2021-11-21T21:54:32.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 Processor -- start 2021-11-21T21:54:32.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- start start 2021-11-21T21:54:32.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980aeb70 0x7f0b9813e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 0x7f0b98064730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b980b24a0 0x7f0b98064c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98065400 con 0x7f0b980aeb70 2021-11-21T21:54:32.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98155310 con 0x7f0b980b24a0 2021-11-21T21:54:32.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98065630 con 0x7f0b980af360 2021-11-21T21:54:32.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980aeb70 0x7f0b9813e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980aeb70 0x7f0b9813e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44936/0 (socket says 172.21.2.116:44936) 2021-11-21T21:54:32.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 -- 172.21.2.116:0/2356476844 learned_addr learned my addr 172.21.2.116:0/2356476844 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 -- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 msgr2=0x7f0b98064730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 0x7f0b98064730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 -- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b980b24a0 msgr2=0x7f0b98064c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b980b24a0 0x7f0b98064c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 -- 172.21.2.116:0/2356476844 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8000b180 con 0x7f0b980aeb70 2021-11-21T21:54:32.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b96ffd700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980aeb70 0x7f0b9813e350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f0b80002f70 tx=0x7f0b80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b8ffff700 1 -- 172.21.2.116:0/2356476844 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b800119d0 con 0x7f0b980aeb70 2021-11-21T21:54:32.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b8ffff700 1 -- 172.21.2.116:0/2356476844 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80011b30 con 0x7f0b980aeb70 2021-11-21T21:54:32.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b981424d0 con 0x7f0b980aeb70 2021-11-21T21:54:32.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b8ffff700 1 -- 172.21.2.116:0/2356476844 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b80011e90 con 0x7f0b980aeb70 2021-11-21T21:54:32.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b98142a10 con 0x7f0b980aeb70 2021-11-21T21:54:32.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.078+0000 7f0b8ffff700 1 -- 172.21.2.116:0/2356476844 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0b80025030 con 0x7f0b980aeb70 2021-11-21T21:54:32.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.082+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b980573d0 con 0x7f0b980aeb70 2021-11-21T21:54:32.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.082+0000 7f0b8ffff700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b7c053580 0x7f0b7c055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.082+0000 7f0b8ffff700 1 -- 172.21.2.116:0/2356476844 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0b8006cc10 con 0x7f0b980aeb70 2021-11-21T21:54:32.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.082+0000 7f0b967fc700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b7c053580 0x7f0b7c055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.082+0000 7f0b967fc700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b7c053580 0x7f0b7c055a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f0b88002f70 tx=0x7f0b88006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.082+0000 7f0b8ffff700 1 -- 172.21.2.116:0/2356476844 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b80043d70 con 0x7f0b980aeb70 2021-11-21T21:54:32.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.166+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b9805f2d0 con 0x7f0b7c053580 2021-11-21T21:54:32.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.170+0000 7f0b8ffff700 1 -- 172.21.2.116:0/2356476844 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82205 (secure 0 0 0) 0x7f0b9805f2d0 con 0x7f0b7c053580 2021-11-21T21:54:32.177 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:32.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b7c053580 msgr2=0x7f0b7c055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b7c053580 0x7f0b7c055a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f0b88002f70 tx=0x7f0b88006a80).stop 2021-11-21T21:54:32.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980aeb70 msgr2=0x7f0b9813e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980aeb70 0x7f0b9813e350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f0b80002f70 tx=0x7f0b80015000).stop 2021-11-21T21:54:32.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 shutdown_connections 2021-11-21T21:54:32.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b980b24a0 0x7f0b98064c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b980af360 0x7f0b98064730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b7c053580 0x7f0b7c055a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 --2- 172.21.2.116:0/2356476844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b980aeb70 0x7f0b9813e350 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 >> 172.21.2.116:0/2356476844 conn(0x7f0b9805fd00 msgr2=0x7f0b98069680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 shutdown_connections 2021-11-21T21:54:32.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.174+0000 7f0b9d7b8700 1 -- 172.21.2.116:0/2356476844 wait complete. 2021-11-21T21:54:32.181 INFO:teuthology.orchestra.run.gibba016.stderr:dumped all 2021-11-21T21:54:32.183 INFO:teuthology.orchestra.run.gibba016.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2021-11-21T21:54:31.051220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":704299008,"kb_used":6087020,"kb_used_data":698213028,"kb_used_omap":589,"kb_used_meta":0,"kb_avail":698211988,"statfs":{"total":721202184192,"available":714969075712,"internally_reserved":1064960,"allocated":714970140672,"data_stored":714970140672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":603284,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.138813"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.936765+0000","last_change":"2021-11-21T21:54:25.936621+0000","last_active":"2021-11-21T21:54:25.936765+0000","last_peered":"2021-11-21T21:54:25.936765+0000","last_clean":"2021-11-21T21:54:25.936765+0000","last_became_active":"2021-11-21T21:54:25.936370+0000","last_became_peered":"2021-11-21T21:54:25.936370+0000","last_unstale":"2021-11-21T21:54:25.936765+0000","last_undegraded":"2021-11-21T21:54:25.936765+0000","last_fullsized":"2021-11-21T21:54:25.936765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:26.788863+0000","last_change":"2021-11-21T21:54:16.981838+0000","last_active":"2021-11-21T21:54:26.788863+0000","last_peered":"2021-11-21T21:54:26.788863+0000","last_clean":"2021-11-21T21:54:26.788863+0000","last_became_active":"2021-11-21T21:54:16.981409+0000","last_became_peered":"2021-11-21T21:54:16.981409+0000","last_unstale":"2021-11-21T21:54:26.788863+0000","last_undegraded":"2021-11-21T21:54:26.788863+0000","last_fullsized":"2021-11-21T21:54:26.788863+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_clean_scrub_stamp":"2021-11-21T21:54:13.901475+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923551+0000","last_change":"2021-11-21T21:54:22.713500+0000","last_active":"2021-11-21T21:54:25.923551+0000","last_peered":"2021-11-21T21:54:25.923551+0000","last_clean":"2021-11-21T21:54:25.923551+0000","last_became_active":"2021-11-21T21:54:22.713118+0000","last_became_peered":"2021-11-21T21:54:22.713118+0000","last_unstale":"2021-11-21T21:54:25.923551+0000","last_undegraded":"2021-11-21T21:54:25.923551+0000","last_fullsized":"2021-11-21T21:54:25.923551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.910169+0000","last_change":"2021-11-21T21:54:24.724398+0000","last_active":"2021-11-21T21:54:24.910169+0000","last_peered":"2021-11-21T21:54:24.910169+0000","last_clean":"2021-11-21T21:54:24.910169+0000","last_became_active":"2021-11-21T21:54:24.724211+0000","last_became_peered":"2021-11-21T21:54:24.724211+0000","last_unstale":"2021-11-21T21:54:24.910169+0000","last_undegraded":"2021-11-21T21:54:24.910169+0000","last_fullsized":"2021-11-21T21:54:24.910169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.911804+0000","last_change":"2021-11-21T21:54:24.910954+0000","last_active":"2021-11-21T21:54:24.911804+0000","last_peered":"2021-11-21T21:54:24.911804+0000","last_clean":"2021-11-21T21:54:24.911804+0000","last_became_active":"2021-11-21T21:54:24.910760+0000","last_became_peered":"2021-11-21T21:54:24.910760+0000","last_unstale":"2021-11-21T21:54:24.911804+0000","last_undegraded":"2021-11-21T21:54:24.911804+0000","last_fullsized":"2021-11-21T21:54:24.911804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:26.788779+0000","last_change":"2021-11-21T21:54:22.713430+0000","last_active":"2021-11-21T21:54:26.788779+0000","last_peered":"2021-11-21T21:54:26.788779+0000","last_clean":"2021-11-21T21:54:26.788779+0000","last_became_active":"2021-11-21T21:54:22.712862+0000","last_became_peered":"2021-11-21T21:54:22.712862+0000","last_unstale":"2021-11-21T21:54:26.788779+0000","last_undegraded":"2021-11-21T21:54:26.788779+0000","last_fullsized":"2021-11-21T21:54:26.788779+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.909998+0000","last_change":"2021-11-21T21:54:24.724353+0000","last_active":"2021-11-21T21:54:24.909998+0000","last_peered":"2021-11-21T21:54:24.909998+0000","last_clean":"2021-11-21T21:54:24.909998+0000","last_became_active":"2021-11-21T21:54:24.724130+0000","last_became_peered":"2021-11-21T21:54:24.724130+0000","last_unstale":"2021-11-21T21:54:24.909998+0000","last_undegraded":"2021-11-21T21:54:24.909998+0000","last_fullsized":"2021-11-21T21:54:24.909998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.915142+0000","last_change":"2021-11-21T21:54:24.914982+0000","last_active":"2021-11-21T21:54:24.915142+0000","last_peered":"2021-11-21T21:54:24.915142+0000","last_clean":"2021-11-21T21:54:24.915142+0000","last_became_active":"2021-11-21T21:54:24.914758+0000","last_became_peered":"2021-11-21T21:54:24.914758+0000","last_unstale":"2021-11-21T21:54:24.915142+0000","last_undegraded":"2021-11-21T21:54:24.915142+0000","last_fullsized":"2021-11-21T21:54:24.915142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.912190+0000","last_change":"2021-11-21T21:54:22.711838+0000","last_active":"2021-11-21T21:54:24.912190+0000","last_peered":"2021-11-21T21:54:24.912190+0000","last_clean":"2021-11-21T21:54:24.912190+0000","last_became_active":"2021-11-21T21:54:22.711687+0000","last_became_peered":"2021-11-21T21:54:22.711687+0000","last_unstale":"2021-11-21T21:54:24.912190+0000","last_undegraded":"2021-11-21T21:54:24.912190+0000","last_fullsized":"2021-11-21T21:54:24.912190+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.942770+0000","last_change":"2021-11-21T21:54:24.724120+0000","last_active":"2021-11-21T21:54:24.942770+0000","last_peered":"2021-11-21T21:54:24.942770+0000","last_clean":"2021-11-21T21:54:24.942770+0000","last_became_active":"2021-11-21T21:54:24.723914+0000","last_became_peered":"2021-11-21T21:54:24.723914+0000","last_unstale":"2021-11-21T21:54:24.942770+0000","last_undegraded":"2021-11-21T21:54:24.942770+0000","last_fullsized":"2021-11-21T21:54:24.942770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.942134+0000","last_change":"2021-11-21T21:54:24.942009+0000","last_active":"2021-11-21T21:54:24.942134+0000","last_peered":"2021-11-21T21:54:24.942134+0000","last_clean":"2021-11-21T21:54:24.942134+0000","last_became_active":"2021-11-21T21:54:24.941840+0000","last_became_peered":"2021-11-21T21:54:24.941840+0000","last_unstale":"2021-11-21T21:54:24.942134+0000","last_undegraded":"2021-11-21T21:54:24.942134+0000","last_fullsized":"2021-11-21T21:54:24.942134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923515+0000","last_change":"2021-11-21T21:54:22.713478+0000","last_active":"2021-11-21T21:54:25.923515+0000","last_peered":"2021-11-21T21:54:25.923515+0000","last_clean":"2021-11-21T21:54:25.923515+0000","last_became_active":"2021-11-21T21:54:22.713254+0000","last_became_peered":"2021-11-21T21:54:22.713254+0000","last_unstale":"2021-11-21T21:54:25.923515+0000","last_undegraded":"2021-11-21T21:54:25.923515+0000","last_fullsized":"2021-11-21T21:54:25.923515+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'5","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.935504+0000","last_change":"2021-11-21T21:54:24.724548+0000","last_active":"2021-11-21T21:54:25.935504+0000","last_peered":"2021-11-21T21:54:25.935504+0000","last_clean":"2021-11-21T21:54:25.935504+0000","last_became_active":"2021-11-21T21:54:24.724266+0000","last_became_peered":"2021-11-21T21:54:24.724266+0000","last_unstale":"2021-11-21T21:54:25.935504+0000","last_undegraded":"2021-11-21T21:54:25.935504+0000","last_fullsized":"2021-11-21T21:54:25.935504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.941782+0000","last_change":"2021-11-21T21:54:24.914955+0000","last_active":"2021-11-21T21:54:24.941782+0000","last_peered":"2021-11-21T21:54:24.941782+0000","last_clean":"2021-11-21T21:54:24.941782+0000","last_became_active":"2021-11-21T21:54:24.914773+0000","last_became_peered":"2021-11-21T21:54:24.914773+0000","last_unstale":"2021-11-21T21:54:24.941782+0000","last_undegraded":"2021-11-21T21:54:24.941782+0000","last_fullsized":"2021-11-21T21:54:24.941782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923528+0000","last_change":"2021-11-21T21:54:24.724846+0000","last_active":"2021-11-21T21:54:25.923528+0000","last_peered":"2021-11-21T21:54:25.923528+0000","last_clean":"2021-11-21T21:54:25.923528+0000","last_became_active":"2021-11-21T21:54:24.724707+0000","last_became_peered":"2021-11-21T21:54:24.724707+0000","last_unstale":"2021-11-21T21:54:25.923528+0000","last_undegraded":"2021-11-21T21:54:25.923528+0000","last_fullsized":"2021-11-21T21:54:25.923528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.908638+0000","last_change":"2021-11-21T21:54:22.786234+0000","last_active":"2021-11-21T21:54:24.908638+0000","last_peered":"2021-11-21T21:54:24.908638+0000","last_clean":"2021-11-21T21:54:24.908638+0000","last_became_active":"2021-11-21T21:54:22.786041+0000","last_became_peered":"2021-11-21T21:54:22.786041+0000","last_unstale":"2021-11-21T21:54:24.908638+0000","last_undegraded":"2021-11-21T21:54:24.908638+0000","last_fullsized":"2021-11-21T21:54:24.908638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.915106+0000","last_change":"2021-11-21T21:54:24.915014+0000","last_active":"2021-11-21T21:54:24.915106+0000","last_peered":"2021-11-21T21:54:24.915106+0000","last_clean":"2021-11-21T21:54:24.915106+0000","last_became_active":"2021-11-21T21:54:24.914814+0000","last_became_peered":"2021-11-21T21:54:24.914814+0000","last_unstale":"2021-11-21T21:54:24.915106+0000","last_undegraded":"2021-11-21T21:54:24.915106+0000","last_fullsized":"2021-11-21T21:54:24.915106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.942897+0000","last_change":"2021-11-21T21:54:24.724216+0000","last_active":"2021-11-21T21:54:24.942897+0000","last_peered":"2021-11-21T21:54:24.942897+0000","last_clean":"2021-11-21T21:54:24.942897+0000","last_became_active":"2021-11-21T21:54:24.723947+0000","last_became_peered":"2021-11-21T21:54:24.723947+0000","last_unstale":"2021-11-21T21:54:24.942897+0000","last_undegraded":"2021-11-21T21:54:24.942897+0000","last_fullsized":"2021-11-21T21:54:24.942897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:26.788853+0000","last_change":"2021-11-21T21:54:22.788120+0000","last_active":"2021-11-21T21:54:26.788853+0000","last_peered":"2021-11-21T21:54:26.788853+0000","last_clean":"2021-11-21T21:54:26.788853+0000","last_became_active":"2021-11-21T21:54:22.787305+0000","last_became_peered":"2021-11-21T21:54:22.787305+0000","last_unstale":"2021-11-21T21:54:26.788853+0000","last_undegraded":"2021-11-21T21:54:26.788853+0000","last_fullsized":"2021-11-21T21:54:26.788853+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.914781+0000","last_change":"2021-11-21T21:54:24.914781+0000","last_active":"2021-11-21T21:54:24.914781+0000","last_peered":"2021-11-21T21:54:24.914781+0000","last_clean":"2021-11-21T21:54:24.914781+0000","last_became_active":"2021-11-21T21:54:24.914595+0000","last_became_peered":"2021-11-21T21:54:24.914595+0000","last_unstale":"2021-11-21T21:54:24.914781+0000","last_undegraded":"2021-11-21T21:54:24.914781+0000","last_fullsized":"2021-11-21T21:54:24.914781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'7","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.914967+0000","last_change":"2021-11-21T21:54:24.724673+0000","last_active":"2021-11-21T21:54:24.914967+0000","last_peered":"2021-11-21T21:54:24.914967+0000","last_clean":"2021-11-21T21:54:24.914967+0000","last_became_active":"2021-11-21T21:54:24.724477+0000","last_became_peered":"2021-11-21T21:54:24.724477+0000","last_unstale":"2021-11-21T21:54:24.914967+0000","last_undegraded":"2021-11-21T21:54:24.914967+0000","last_fullsized":"2021-11-21T21:54:24.914967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.908642+0000","last_change":"2021-11-21T21:54:22.713690+0000","last_active":"2021-11-21T21:54:24.908642+0000","last_peered":"2021-11-21T21:54:24.908642+0000","last_clean":"2021-11-21T21:54:24.908642+0000","last_became_active":"2021-11-21T21:54:22.713554+0000","last_became_peered":"2021-11-21T21:54:22.713554+0000","last_unstale":"2021-11-21T21:54:24.908642+0000","last_undegraded":"2021-11-21T21:54:24.908642+0000","last_fullsized":"2021-11-21T21:54:24.908642+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.935482+0000","last_change":"2021-11-21T21:54:24.916112+0000","last_active":"2021-11-21T21:54:25.935482+0000","last_peered":"2021-11-21T21:54:25.935482+0000","last_clean":"2021-11-21T21:54:25.935482+0000","last_became_active":"2021-11-21T21:54:24.915840+0000","last_became_peered":"2021-11-21T21:54:24.915840+0000","last_unstale":"2021-11-21T21:54:25.935482+0000","last_undegraded":"2021-11-21T21:54:25.935482+0000","last_fullsized":"2021-11-21T21:54:25.935482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:24.709816+0000","last_clean_scrub_stamp":"2021-11-21T21:54:24.709816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2021-11-21T21:54:24.912474+0000","last_change":"2021-11-21T21:54:24.723551+0000","last_active":"2021-11-21T21:54:24.912474+0000","last_peered":"2021-11-21T21:54:24.912474+0000","last_clean":"2021-11-21T21:54:24.912474+0000","last_became_active":"2021-11-21T21:54:24.723376+0000","last_became_peered":"2021-11-21T21:54:24.723376+0000","last_unstale":"2021-11-21T21:54:24.912474+0000","last_undegraded":"2021-11-21T21:54:24.912474+0000","last_fullsized":"2021-11-21T21:54:24.912474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:23.703971+0000","last_clean_scrub_stamp":"2021-11-21T21:54:23.703971+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2021-11-21T21:54:25.923482+0000","last_change":"2021-11-21T21:54:22.714589+0000","last_active":"2021-11-21T21:54:25.923482+0000","last_peered":"2021-11-21T21:54:25.923482+0000","last_clean":"2021-11-21T21:54:25.923482+0000","last_became_active":"2021-11-21T21:54:22.714383+0000","last_became_peered":"2021-11-21T21:54:22.714383+0000","last_unstale":"2021-11-21T21:54:25.923482+0000","last_undegraded":"2021-11-21T21:54:25.923482+0000","last_fullsized":"2021-11-21T21:54:25.923482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760596,"kb_used_data":87276780,"kb_used_omap":76,"kb_used_meta":0,"kb_avail":87276780,"statfs":{"total":90150273024,"available":89371422720,"internally_reserved":0,"allocated":89371422720,"data_stored":89371422720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77938,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760588,"kb_used_data":87276788,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":87276788,"statfs":{"total":90150273024,"available":89371430912,"internally_reserved":0,"allocated":89371430912,"data_stored":89371430912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43305,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.786}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7030000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":761012,"kb_used_data":87276540,"kb_used_omap":78,"kb_used_meta":0,"kb_avail":87276364,"statfs":{"total":90150273024,"available":89370996736,"internally_reserved":180224,"allocated":89371176960,"data_stored":89371176960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80002,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760956,"kb_used_data":87276564,"kb_used_omap":57,"kb_used_meta":0,"kb_avail":87276420,"statfs":{"total":90150273024,"available":89371054080,"internally_reserved":147456,"allocated":89371201536,"data_stored":89371201536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58532,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.032}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760936,"kb_used_data":87276600,"kb_used_omap":65,"kb_used_meta":0,"kb_avail":87276440,"statfs":{"total":90150273024,"available":89371074560,"internally_reserved":163840,"allocated":89371238400,"data_stored":89371238400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":67441,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":761016,"kb_used_data":87276516,"kb_used_omap":67,"kb_used_meta":0,"kb_avail":87276360,"statfs":{"total":90150273024,"available":89370992640,"internally_reserved":159744,"allocated":89371152384,"data_stored":89371152384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":69462,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":760864,"kb_used_data":87276672,"kb_used_omap":96,"kb_used_meta":0,"kb_avail":87276512,"statfs":{"total":90150273024,"available":89371148288,"internally_reserved":163840,"allocated":89371312128,"data_stored":89371312128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":99195,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":761052,"kb_used_data":87276568,"kb_used_omap":104,"kb_used_meta":0,"kb_avail":87276324,"statfs":{"total":90150273024,"available":89370955776,"internally_reserved":249856,"allocated":89371205632,"data_stored":89371205632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107409,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]}]}]}} 2021-11-21T21:54:32.185 INFO:tasks.ceph.ceph_manager.ceph:clean! 2021-11-21T21:54:32.186 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-11-21T21:54:32.186 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2021-11-21T21:54:32.186 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2021-11-21T21:54:32.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 Processor -- start 2021-11-21T21:54:32.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- start start 2021-11-21T21:54:32.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9a2c0a6aa0 0x7f9a2c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2c0a6e80 con 0x7f9a2c062410 2021-11-21T21:54:32.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2c14a7b0 con 0x7f9a2c0a86b0 2021-11-21T21:54:32.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2c0a8bc0 con 0x7f9a2c0a6aa0 2021-11-21T21:54:32.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3161c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9a2c062410 0x7f9a2c1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56904/0 (socket says 172.21.2.116:56904) 2021-11-21T21:54:32.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3161c700 1 -- 172.21.2.116:0/3507834220 learned_addr learned my addr 172.21.2.116:0/3507834220 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:32.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525791909 0 0) 0x7f9a2c0a6e80 con 0x7f9a2c062410 2021-11-21T21:54:32.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a0c003260 con 0x7f9a2c062410 2021-11-21T21:54:32.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/3507834220 >> v2:172.21.2.117:3300/0 conn(0x7f9a2c0a6aa0 0x7f9a2c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 51290360 0 0) 0x7f9a0c003260 con 0x7f9a2c062410 2021-11-21T21:54:32.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 >> v2:172.21.2.117:3300/0 conn(0x7f9a2c0a6aa0 msgr2=0x7f9a2c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 --2- 172.21.2.116:0/3507834220 >> v2:172.21.2.117:3300/0 conn(0x7f9a2c0a6aa0 0x7f9a2c14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 >> v1:172.21.2.117:6789/0 conn(0x7f9a2c0a86b0 legacy=0x7f9a2c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c14aa10 con 0x7f9a2c062410 2021-11-21T21:54:32.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9a24001a00 con 0x7f9a2c062410 2021-11-21T21:54:32.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 >> v1:172.21.2.116:6789/0 conn(0x7f9a2c062410 legacy=0x7f9a2c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a0c003750 0x7f9a0c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c004500 0x7f9a0c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a0c004e70 0x7f9a0c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c0a8bc0 con 0x7f9a0c003750 2021-11-21T21:54:32.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c00bd20 con 0x7f9a0c004500 2021-11-21T21:54:32.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c00bee0 con 0x7f9a0c004e70 2021-11-21T21:54:32.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a0c003750 0x7f9a0c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2bfff700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c004500 0x7f9a0c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3161c700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a0c004e70 0x7f9a0c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 -- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a0c004e70 msgr2=0x7f9a0c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a0c004e70 0x7f9a0c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 -- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c004500 msgr2=0x7f9a0c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c004500 0x7f9a0c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 -- 172.21.2.116:0/3507834220 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c003260 con 0x7f9a0c003750 2021-11-21T21:54:32.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3161c700 1 -- 172.21.2.116:0/3507834220 reap_dead start 2021-11-21T21:54:32.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a0c003750 0x7f9a0c003b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f9a1c00c000 tx=0x7f9a1c006df0).ready entity=mon.0 client_cookie=a3fa940520300dd6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a1c00e830 con 0x7f9a0c003750 2021-11-21T21:54:32.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c00e990 con 0x7f9a0c003750 2021-11-21T21:54:32.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a2b7fe700 1 -- 172.21.2.116:0/3507834220 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a1c00ecf0 con 0x7f9a0c003750 2021-11-21T21:54:32.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- 172.21.2.116:0/3507834220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a0c003750 msgr2=0x7f9a0c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a0c003750 0x7f9a0c003b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f9a1c00c000 tx=0x7f9a1c006df0).stop 2021-11-21T21:54:32.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- 172.21.2.116:0/3507834220 shutdown_connections 2021-11-21T21:54:32.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 --2- 172.21.2.116:0/3507834220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a0c003750 0x7f9a0c003b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- 172.21.2.116:0/3507834220 >> 172.21.2.116:0/3507834220 conn(0x7f9a2c05bd20 msgr2=0x7f9a2c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- 172.21.2.116:0/3507834220 shutdown_connections 2021-11-21T21:54:32.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.234+0000 7f9a3307f700 1 -- 172.21.2.116:0/3507834220 wait complete. 2021-11-21T21:54:32.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 Processor -- start 2021-11-21T21:54:32.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 -- start start 2021-11-21T21:54:32.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c003750 0x7f9a2c0a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a2c15a1e0 0x7f9a2c156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a2c065620 0x7f9a2c156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c156d10 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c15a680 con 0x7f9a0c003750 2021-11-21T21:54:32.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c064210 con 0x7f9a2c065620 2021-11-21T21:54:32.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a2c15a1e0 0x7f9a2c156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a2c15a1e0 0x7f9a2c156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44958/0 (socket says 172.21.2.116:44958) 2021-11-21T21:54:32.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 -- 172.21.2.116:0/1036821096 learned_addr learned my addr 172.21.2.116:0/1036821096 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3161c700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a2c065620 0x7f9a2c156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c003750 0x7f9a2c0a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 -- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a2c065620 msgr2=0x7f9a2c156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a2c065620 0x7f9a2c156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 -- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c003750 msgr2=0x7f9a2c0a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c003750 0x7f9a2c0a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 -- 172.21.2.116:0/1036821096 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c0644b0 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3161c700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a2c065620 0x7f9a2c156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:54:32.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a2bfff700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a2c15a1e0 0x7f9a2c156330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f9a1400b000 tx=0x7f9a14009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a29ffb700 1 -- 172.21.2.116:0/1036821096 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a14010640 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a29ffb700 1 -- 172.21.2.116:0/1036821096 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a140107a0 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c064750 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c0a9c90 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a29ffb700 1 -- 172.21.2.116:0/1036821096 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a14010b00 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2c0648e0 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a29ffb700 1 -- 172.21.2.116:0/1036821096 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9a14010c60 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a29ffb700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a100535d0 0x7f9a10055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a29ffb700 1 -- 172.21.2.116:0/1036821096 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9a1406b850 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a100535d0 0x7f9a10055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.238+0000 7f9a30e1b700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a100535d0 0x7f9a10055a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f9a1c01b000 tx=0x7f9a1c019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.242+0000 7f9a29ffb700 1 -- 172.21.2.116:0/1036821096 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a14042950 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9a2c0573d0 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a29ffb700 1 -- 172.21.2.116:0/1036821096 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f9a14045f70 con 0x7f9a2c15a1e0 2021-11-21T21:54:32.355 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:32.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a100535d0 msgr2=0x7f9a10055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a100535d0 0x7f9a10055a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f9a1c01b000 tx=0x7f9a1c019000).stop 2021-11-21T21:54:32.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a2c15a1e0 msgr2=0x7f9a2c156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a2c15a1e0 0x7f9a2c156330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f9a1400b000 tx=0x7f9a14009040).stop 2021-11-21T21:54:32.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 shutdown_connections 2021-11-21T21:54:32.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a2c065620 0x7f9a2c156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a100535d0 0x7f9a10055a90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a2c15a1e0 0x7f9a2c156330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 --2- 172.21.2.116:0/1036821096 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a0c003750 0x7f9a2c0a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.350+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 >> 172.21.2.116:0/1036821096 conn(0x7f9a2c05bd20 msgr2=0x7f9a2c149d00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.354+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 shutdown_connections 2021-11-21T21:54:32.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.354+0000 7f9a3307f700 1 -- 172.21.2.116:0/1036821096 wait complete. 2021-11-21T21:54:32.362 INFO:teuthology.orchestra.run.gibba016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-11-21T21:54:32.362 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2021-11-21T21:54:32.363 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:32.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 Processor -- start 2021-11-21T21:54:32.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- start start 2021-11-21T21:54:32.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2374062410 0x7f2374144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740a6e80 con 0x7f23740a6aa0 2021-11-21T21:54:32.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237414a7b0 con 0x7f23740a86b0 2021-11-21T21:54:32.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740a8bc0 con 0x7f2374062410 2021-11-21T21:54:32.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2373fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f23740a6aa0 0x7f23741470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56920/0 (socket says 172.21.2.116:56920) 2021-11-21T21:54:32.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2373fff700 1 -- 172.21.2.116:0/3781440616 learned_addr learned my addr 172.21.2.116:0/3781440616 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:32.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2378915700 1 --2- 172.21.2.116:0/3781440616 >> v2:172.21.2.116:3300/0 conn(0x7f2374062410 0x7f2374144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2372ffd700 1 -- 172.21.2.116:0/3781440616 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397626058 0 0) 0x7f23740a6e80 con 0x7f23740a6aa0 2021-11-21T21:54:32.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2372ffd700 1 -- 172.21.2.116:0/3781440616 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2354003260 con 0x7f23740a6aa0 2021-11-21T21:54:32.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2378915700 1 -- 172.21.2.116:0/3781440616 >> v1:172.21.2.117:6789/0 conn(0x7f23740a86b0 legacy=0x7f23741443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2378915700 1 -- 172.21.2.116:0/3781440616 >> v1:172.21.2.116:6789/0 conn(0x7f23740a6aa0 legacy=0x7f23741470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2378915700 1 -- 172.21.2.116:0/3781440616 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237414aa10 con 0x7f2374062410 2021-11-21T21:54:32.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2378915700 1 --2- 172.21.2.116:0/3781440616 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2374062410 0x7f2374144b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f235c001110 tx=0x7f236c0091d0).ready entity=mon.0 client_cookie=15c57bb3d9633604 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2372ffd700 1 -- 172.21.2.116:0/3781440616 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236c00b640 con 0x7f2374062410 2021-11-21T21:54:32.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2372ffd700 1 -- 172.21.2.116:0/3781440616 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c00b7a0 con 0x7f2374062410 2021-11-21T21:54:32.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f2372ffd700 1 -- 172.21.2.116:0/3781440616 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236c00b910 con 0x7f2374062410 2021-11-21T21:54:32.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- 172.21.2.116:0/3781440616 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2374062410 msgr2=0x7f2374144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 --2- 172.21.2.116:0/3781440616 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2374062410 0x7f2374144b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f235c001110 tx=0x7f236c0091d0).stop 2021-11-21T21:54:32.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- 172.21.2.116:0/3781440616 shutdown_connections 2021-11-21T21:54:32.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 --2- 172.21.2.116:0/3781440616 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2374062410 0x7f2374144b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- 172.21.2.116:0/3781440616 >> 172.21.2.116:0/3781440616 conn(0x7f237405bd20 msgr2=0x7f237405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- 172.21.2.116:0/3781440616 shutdown_connections 2021-11-21T21:54:32.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- 172.21.2.116:0/3781440616 wait complete. 2021-11-21T21:54:32.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 Processor -- start 2021-11-21T21:54:32.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- start start 2021-11-21T21:54:32.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2374062410 0x7f23740b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23740a6aa0 0x7f237413feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23740a86b0 0x7f237413cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2374140550 con 0x7f23740a6aa0 2021-11-21T21:54:32.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741408d0 con 0x7f23740a86b0 2021-11-21T21:54:32.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f237a378700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2374140b70 con 0x7f2374062410 2021-11-21T21:54:32.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f23737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23740a6aa0 0x7f237413feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f23737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23740a6aa0 0x7f237413feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44972/0 (socket says 172.21.2.116:44972) 2021-11-21T21:54:32.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.422+0000 7f23737fe700 1 -- 172.21.2.116:0/4277022134 learned_addr learned my addr 172.21.2.116:0/4277022134 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23737fe700 1 -- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2374062410 msgr2=0x7f23740b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23737fe700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2374062410 0x7f23740b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23737fe700 1 -- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23740a86b0 msgr2=0x7f237413cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23737fe700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23740a86b0 0x7f237413cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23737fe700 1 -- 172.21.2.116:0/4277022134 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c009470 con 0x7f23740a6aa0 2021-11-21T21:54:32.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23737fe700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23740a6aa0 0x7f237413feb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f2364002f10 tx=0x7f2364006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23717fa700 1 -- 172.21.2.116:0/4277022134 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236400f510 con 0x7f23740a6aa0 2021-11-21T21:54:32.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23717fa700 1 -- 172.21.2.116:0/4277022134 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400f670 con 0x7f23740a6aa0 2021-11-21T21:54:32.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23717fa700 1 -- 172.21.2.116:0/4277022134 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236400f7e0 con 0x7f23740a6aa0 2021-11-21T21:54:32.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2374140e10 con 0x7f23740a6aa0 2021-11-21T21:54:32.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2374141520 con 0x7f23740a6aa0 2021-11-21T21:54:32.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237413ca80 con 0x7f23740a6aa0 2021-11-21T21:54:32.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23717fa700 1 -- 172.21.2.116:0/4277022134 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2364017020 con 0x7f23740a6aa0 2021-11-21T21:54:32.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23717fa700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2358053580 0x7f2358055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23717fa700 1 -- 172.21.2.116:0/4277022134 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f236406aaf0 con 0x7f23740a6aa0 2021-11-21T21:54:32.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f2373fff700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2358053580 0x7f2358055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f2373fff700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2358053580 0x7f2358055a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f235c000ed0 tx=0x7f235c0076e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.426+0000 7f23717fa700 1 -- 172.21.2.116:0/4277022134 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2364041bf0 con 0x7f23740a6aa0 2021-11-21T21:54:32.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23740573d0 con 0x7f23740a6aa0 2021-11-21T21:54:32.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f23717fa700 1 -- 172.21.2.116:0/4277022134 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f2364015030 con 0x7f23740a6aa0 2021-11-21T21:54:32.515 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:32.515 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:32.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2358053580 msgr2=0x7f2358055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2358053580 0x7f2358055a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f235c000ed0 tx=0x7f235c0076e0).stop 2021-11-21T21:54:32.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23740a6aa0 msgr2=0x7f237413feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23740a6aa0 0x7f237413feb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f2364002f10 tx=0x7f2364006d70).stop 2021-11-21T21:54:32.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 shutdown_connections 2021-11-21T21:54:32.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23740a86b0 0x7f237413cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2358053580 0x7f2358055a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23740a6aa0 0x7f237413feb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 --2- 172.21.2.116:0/4277022134 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2374062410 0x7f23740b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.510+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 >> 172.21.2.116:0/4277022134 conn(0x7f237405bd20 msgr2=0x7f2374148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.514+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 shutdown_connections 2021-11-21T21:54:32.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.514+0000 7f237a378700 1 -- 172.21.2.116:0/4277022134 wait complete. 2021-11-21T21:54:32.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:32.523 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:32.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 Processor -- start 2021-11-21T21:54:32.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- start start 2021-11-21T21:54:32.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc5f8144810 0x7fc5f81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f8147e80 con 0x7fc5f80ac600 2021-11-21T21:54:32.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f80af5a0 con 0x7fc5f80abce0 2021-11-21T21:54:32.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f80b1490 con 0x7fc5f8144810 2021-11-21T21:54:32.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fdd3b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc5f8144810 0x7fc5f81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fdd3b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc5f8144810 0x7fc5f81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44978/0 (socket says 172.21.2.116:44978) 2021-11-21T21:54:32.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/2891600935 learned_addr learned my addr 172.21.2.116:0/2891600935 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/2891600935 >> v1:172.21.2.117:6789/0 conn(0x7fc5f80abce0 legacy=0x7fc5f80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:32.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/2891600935 >> v1:172.21.2.116:6789/0 conn(0x7fc5f80ac600 legacy=0x7fc5f80aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/2891600935 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80b1fd0 con 0x7fc5f8144810 2021-11-21T21:54:32.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fdd3b700 1 --2- 172.21.2.116:0/2891600935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f8144810 0x7fc5f81478c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fc5f805d170 tx=0x7fc5ec006eb0).ready entity=mon.0 client_cookie=9aa0c514f754dfa server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fcd39700 1 -- 172.21.2.116:0/2891600935 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ec00d610 con 0x7fc5f8144810 2021-11-21T21:54:32.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fcd39700 1 -- 172.21.2.116:0/2891600935 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec000e90 con 0x7fc5f8144810 2021-11-21T21:54:32.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fcd39700 1 -- 172.21.2.116:0/2891600935 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ec00d960 con 0x7fc5f8144810 2021-11-21T21:54:32.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- 172.21.2.116:0/2891600935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f8144810 msgr2=0x7fc5f81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/2891600935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f8144810 0x7fc5f81478c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fc5f805d170 tx=0x7fc5ec006eb0).stop 2021-11-21T21:54:32.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- 172.21.2.116:0/2891600935 shutdown_connections 2021-11-21T21:54:32.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/2891600935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f8144810 0x7fc5f81478c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- 172.21.2.116:0/2891600935 >> 172.21.2.116:0/2891600935 conn(0x7fc5f805bd20 msgr2=0x7fc5f805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- 172.21.2.116:0/2891600935 shutdown_connections 2021-11-21T21:54:32.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.614+0000 7fc5fff9f700 1 -- 172.21.2.116:0/2891600935 wait complete. 2021-11-21T21:54:32.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 Processor -- start 2021-11-21T21:54:32.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 -- start start 2021-11-21T21:54:32.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f80abce0 0x7fc5f8064160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc5f80ac600 0x7fc5f8066d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5f8144810 0x7fc5f8067260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8156e20 con 0x7fc5f80abce0 2021-11-21T21:54:32.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8157980 con 0x7fc5f8144810 2021-11-21T21:54:32.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80b2d50 con 0x7fc5f80ac600 2021-11-21T21:54:32.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f80abce0 0x7fc5f8064160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f80abce0 0x7fc5f8064160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:44984/0 (socket says 172.21.2.116:44984) 2021-11-21T21:54:32.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/3996246052 learned_addr learned my addr 172.21.2.116:0/3996246052 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc5f80ac600 msgr2=0x7fc5f8066d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:32.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc5f80ac600 0x7fc5f8066d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5f8144810 msgr2=0x7fc5f8067260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:32.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5f8144810 0x7fc5f8067260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 -- 172.21.2.116:0/3996246052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec00b1e0 con 0x7fc5f80abce0 2021-11-21T21:54:32.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fdd3b700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f80abce0 0x7fc5f8064160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fc5f805d170 tx=0x7fc5ec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5eaffd700 1 -- 172.21.2.116:0/3996246052 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ec00d290 con 0x7fc5f80abce0 2021-11-21T21:54:32.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5eaffd700 1 -- 172.21.2.116:0/3996246052 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec002f70 con 0x7fc5f80abce0 2021-11-21T21:54:32.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f80b3690 con 0x7fc5f80abce0 2021-11-21T21:54:32.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f8156b80 con 0x7fc5f80abce0 2021-11-21T21:54:32.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5eaffd700 1 -- 172.21.2.116:0/3996246052 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ec01e460 con 0x7fc5f80abce0 2021-11-21T21:54:32.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f80b3820 con 0x7fc5f80abce0 2021-11-21T21:54:32.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5eaffd700 1 -- 172.21.2.116:0/3996246052 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc5ec025030 con 0x7fc5f80abce0 2021-11-21T21:54:32.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5eaffd700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5d80536f0 0x7fc5d8055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5eaffd700 1 -- 172.21.2.116:0/3996246052 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc5ec06cee0 con 0x7fc5f80abce0 2021-11-21T21:54:32.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.618+0000 7fc5fd53a700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5d80536f0 0x7fc5d8055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.622+0000 7fc5fd53a700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5d80536f0 0x7fc5d8055bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc5f4002fb0 tx=0x7fc5f4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.622+0000 7fc5eaffd700 1 -- 172.21.2.116:0/3996246052 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc5ec043f30 con 0x7fc5f80abce0 2021-11-21T21:54:32.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.702+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc5f80af060 con 0x7fc5f80abce0 2021-11-21T21:54:32.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.702+0000 7fc5eaffd700 1 -- 172.21.2.116:0/3996246052 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc5ec0230b0 con 0x7fc5f80abce0 2021-11-21T21:54:32.707 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:32.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5d80536f0 msgr2=0x7fc5d8055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5d80536f0 0x7fc5d8055bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc5f4002fb0 tx=0x7fc5f4006930).stop 2021-11-21T21:54:32.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f80abce0 msgr2=0x7fc5f8064160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f80abce0 0x7fc5f8064160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fc5f805d170 tx=0x7fc5ec015000).stop 2021-11-21T21:54:32.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 shutdown_connections 2021-11-21T21:54:32.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5f8144810 0x7fc5f8067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc5f80ac600 0x7fc5f8066d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5d80536f0 0x7fc5d8055bb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 --2- 172.21.2.116:0/3996246052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5f80abce0 0x7fc5f8064160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 >> 172.21.2.116:0/3996246052 conn(0x7fc5f805bd20 msgr2=0x7fc5f8148280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 shutdown_connections 2021-11-21T21:54:32.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.706+0000 7fc5fff9f700 1 -- 172.21.2.116:0/3996246052 wait complete. 2021-11-21T21:54:32.714 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:32.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe091acb700 1 Processor -- start 2021-11-21T21:54:32.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe091acb700 1 -- start start 2021-11-21T21:54:32.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe091acb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe08c05f730 0x7fe08c141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe091acb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c142370 0x7fe08c142800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe091acb700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c0b06c0 con 0x7fe08c0b02e0 2021-11-21T21:54:32.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe091acb700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c0adf20 con 0x7fe08c05f730 2021-11-21T21:54:32.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe091acb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0b2470 con 0x7fe08c142370 2021-11-21T21:54:32.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe08b7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe08c0b02e0 0x7fe08c0af620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56948/0 (socket says 172.21.2.116:56948) 2021-11-21T21:54:32.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe08b7fe700 1 -- 172.21.2.116:0/3336123080 learned_addr learned my addr 172.21.2.116:0/3336123080 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:32.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972058581 0 0) 0x7fe08c0b06c0 con 0x7fe08c0b02e0 2021-11-21T21:54:32.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.762+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe06c003260 con 0x7fe08c0b02e0 2021-11-21T21:54:32.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863757885 0 0) 0x7fe06c003260 con 0x7fe08c0b02e0 2021-11-21T21:54:32.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c142370 msgr2=0x7fe08c142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c142370 0x7fe08c142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 >> v2:172.21.2.117:3300/0 conn(0x7fe08c05f730 msgr2=0x7fe08c141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 --2- 172.21.2.116:0/3336123080 >> v2:172.21.2.117:3300/0 conn(0x7fe08c05f730 0x7fe08c141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0b26d0 con 0x7fe08c0b02e0 2021-11-21T21:54:32.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe07c001660 con 0x7fe08c0b02e0 2021-11-21T21:54:32.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 >> v1:172.21.2.116:6789/0 conn(0x7fe08c0b02e0 legacy=0x7fe08c0af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe06c0037d0 0x7fe06c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c142370 0x7fe06c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe06c004bf0 0x7fe06c00d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0adf20 con 0x7fe06c0037d0 2021-11-21T21:54:32.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0b2470 con 0x7fe06c004bf0 2021-11-21T21:54:32.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c005300 con 0x7fe08c142370 2021-11-21T21:54:32.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08b7fe700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe06c0037d0 0x7fe06c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08b7fe700 1 -- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c142370 msgr2=0x7fe06c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08b7fe700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c142370 0x7fe06c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08b7fe700 1 -- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe06c004bf0 msgr2=0x7fe06c00d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:32.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08b7fe700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe06c004bf0 0x7fe06c00d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08b7fe700 1 -- 172.21.2.116:0/3336123080 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c003260 con 0x7fe06c0037d0 2021-11-21T21:54:32.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08b7fe700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe06c0037d0 0x7fe06c003bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fe07c001370 tx=0x7fe08c0b2860).ready entity=mon.0 client_cookie=ad7a6c9fcafe12a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe07c0070d0 con 0x7fe06c0037d0 2021-11-21T21:54:32.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c007230 con 0x7fe06c0037d0 2021-11-21T21:54:32.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08a7fc700 1 -- 172.21.2.116:0/3336123080 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe07c00c8f0 con 0x7fe06c0037d0 2021-11-21T21:54:32.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- 172.21.2.116:0/3336123080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe06c0037d0 msgr2=0x7fe06c003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 --2- 172.21.2.116:0/3336123080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe06c0037d0 0x7fe06c003bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fe07c001370 tx=0x7fe08c0b2860).stop 2021-11-21T21:54:32.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 -- 172.21.2.116:0/3336123080 reap_dead start 2021-11-21T21:54:32.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- 172.21.2.116:0/3336123080 shutdown_connections 2021-11-21T21:54:32.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- 172.21.2.116:0/3336123080 >> 172.21.2.116:0/3336123080 conn(0x7fe08c0b5ff0 msgr2=0x7fe08c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- 172.21.2.116:0/3336123080 shutdown_connections 2021-11-21T21:54:32.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- 172.21.2.116:0/3336123080 wait complete. 2021-11-21T21:54:32.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 Processor -- start 2021-11-21T21:54:32.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- start start 2021-11-21T21:54:32.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe06c0037d0 0x7fe08c0a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c0aa740 0x7fe08c0662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe08c0667d0 0x7fe08c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0a8320 con 0x7fe08c0667d0 2021-11-21T21:54:32.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0a84f0 con 0x7fe06c0037d0 2021-11-21T21:54:32.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0a8790 con 0x7fe08c0aa740 2021-11-21T21:54:32.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe08c0667d0 0x7fe08c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe08c0667d0 0x7fe08c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45004/0 (socket says 172.21.2.116:45004) 2021-11-21T21:54:32.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 -- 172.21.2.116:0/932068177 learned_addr learned my addr 172.21.2.116:0/932068177 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 -- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c0aa740 msgr2=0x7fe08c0662e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:32.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c0aa740 0x7fe08c0662e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 -- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe06c0037d0 msgr2=0x7fe08c0a9240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:32.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe06c0037d0 0x7fe08c0a9240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 -- 172.21.2.116:0/932068177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0a8a30 con 0x7fe08c0667d0 2021-11-21T21:54:32.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe08bfff700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe08c0667d0 0x7fe08c159980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fe07c001840 tx=0x7fe08400a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe088ff9700 1 -- 172.21.2.116:0/932068177 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe084005a20 con 0x7fe08c0667d0 2021-11-21T21:54:32.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe088ff9700 1 -- 172.21.2.116:0/932068177 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe084005b80 con 0x7fe08c0667d0 2021-11-21T21:54:32.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c0a7be0 con 0x7fe08c0667d0 2021-11-21T21:54:32.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c0ac7a0 con 0x7fe08c0667d0 2021-11-21T21:54:32.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.766+0000 7fe088ff9700 1 -- 172.21.2.116:0/932068177 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe084007070 con 0x7fe08c0667d0 2021-11-21T21:54:32.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.770+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe064003410 con 0x7fe08c0667d0 2021-11-21T21:54:32.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.770+0000 7fe088ff9700 1 -- 172.21.2.116:0/932068177 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe08401b060 con 0x7fe08c0667d0 2021-11-21T21:54:32.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.770+0000 7fe088ff9700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0700535a0 0x7fe070055a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.770+0000 7fe088ff9700 1 -- 172.21.2.116:0/932068177 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe08406b350 con 0x7fe08c0667d0 2021-11-21T21:54:32.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.770+0000 7fe08b7fe700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0700535a0 0x7fe070055a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.770+0000 7fe08b7fe700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0700535a0 0x7fe070055a60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe07c001370 tx=0x7fe07c013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.770+0000 7fe088ff9700 1 -- 172.21.2.116:0/932068177 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe084042340 con 0x7fe08c0667d0 2021-11-21T21:54:32.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe064003c40 con 0x7fe08c0667d0 2021-11-21T21:54:32.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe088ff9700 1 -- 172.21.2.116:0/932068177 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe084019020 con 0x7fe08c0667d0 2021-11-21T21:54:32.857 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:32.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0700535a0 msgr2=0x7fe070055a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0700535a0 0x7fe070055a60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe07c001370 tx=0x7fe07c013000).stop 2021-11-21T21:54:32.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe08c0667d0 msgr2=0x7fe08c159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe08c0667d0 0x7fe08c159980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fe07c001840 tx=0x7fe08400a060).stop 2021-11-21T21:54:32.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 shutdown_connections 2021-11-21T21:54:32.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0700535a0 0x7fe070055a60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe08c0667d0 0x7fe08c159980 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe08c0aa740 0x7fe08c0662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 --2- 172.21.2.116:0/932068177 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe06c0037d0 0x7fe08c0a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 >> 172.21.2.116:0/932068177 conn(0x7fe08c0b5ff0 msgr2=0x7fe08c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 shutdown_connections 2021-11-21T21:54:32.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.854+0000 7fe091acb700 1 -- 172.21.2.116:0/932068177 wait complete. 2021-11-21T21:54:32.864 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:32.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 Processor -- start 2021-11-21T21:54:32.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- start start 2021-11-21T21:54:32.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f840adc10 0x7f5f840b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5f840b3690 0x7f5f8413a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f840b3c50 con 0x7f5f84144a10 2021-11-21T21:54:32.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f840b3ef0 con 0x7f5f840b3690 2021-11-21T21:54:32.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840b4190 con 0x7f5f840adc10 2021-11-21T21:54:32.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5f840b3690 0x7f5f8413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5f840b3690 0x7f5f8413a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45016/0 (socket says 172.21.2.116:45016) 2021-11-21T21:54:32.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 -- 172.21.2.116:0/832987633 learned_addr learned my addr 172.21.2.116:0/832987633 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 -- 172.21.2.116:0/832987633 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f840adc10 msgr2=0x7f5f840b2030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:32.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 --2- 172.21.2.116:0/832987633 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f840adc10 0x7f5f840b2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 -- 172.21.2.116:0/832987633 >> v1:172.21.2.117:6789/0 conn(0x7f5f84144a10 legacy=0x7f5f84141020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:32.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 -- 172.21.2.116:0/832987633 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f84140b80 con 0x7f5f840b3690 2021-11-21T21:54:32.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f892f4700 1 --2- 172.21.2.116:0/832987633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840b3690 0x7f5f8413a540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5f840b7000 tx=0x7f5f7c009290).ready entity=mon.0 client_cookie=e5abdf74d70af2e5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f837fe700 1 -- 172.21.2.116:0/832987633 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7c00b860 con 0x7f5f840b3690 2021-11-21T21:54:32.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f837fe700 1 -- 172.21.2.116:0/832987633 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c00b9c0 con 0x7f5f840b3690 2021-11-21T21:54:32.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f837fe700 1 -- 172.21.2.116:0/832987633 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7c00bb70 con 0x7f5f840b3690 2021-11-21T21:54:32.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- 172.21.2.116:0/832987633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840b3690 msgr2=0x7f5f8413a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/832987633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840b3690 0x7f5f8413a540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5f840b7000 tx=0x7f5f7c009290).stop 2021-11-21T21:54:32.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- 172.21.2.116:0/832987633 shutdown_connections 2021-11-21T21:54:32.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/832987633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840b3690 0x7f5f8413a540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:32.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/832987633 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f840adc10 0x7f5f840b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- 172.21.2.116:0/832987633 >> 172.21.2.116:0/832987633 conn(0x7f5f8405bd20 msgr2=0x7f5f8405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- 172.21.2.116:0/832987633 shutdown_connections 2021-11-21T21:54:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- 172.21.2.116:0/832987633 wait complete. 2021-11-21T21:54:32.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 Processor -- start 2021-11-21T21:54:32.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- start start 2021-11-21T21:54:32.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840adc10 0x7f5f8413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f840b3690 0x7f5f84152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f84144a10 0x7f5f841554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f84063330 con 0x7f5f840adc10 2021-11-21T21:54:32.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840635d0 con 0x7f5f840b3690 2021-11-21T21:54:32.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f8ad57700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841521b0 con 0x7f5f84144a10 2021-11-21T21:54:32.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f88af3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840adc10 0x7f5f8413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f88af3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840adc10 0x7f5f8413fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45018/0 (socket says 172.21.2.116:45018) 2021-11-21T21:54:32.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.914+0000 7f5f88af3700 1 -- 172.21.2.116:0/132795602 learned_addr learned my addr 172.21.2.116:0/132795602 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:32.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f88af3700 1 -- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f84144a10 msgr2=0x7f5f841554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f88af3700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f84144a10 0x7f5f841554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f88af3700 1 -- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f840b3690 msgr2=0x7f5f84152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:32.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f88af3700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f840b3690 0x7f5f84152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:32.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f88af3700 1 -- 172.21.2.116:0/132795602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c009530 con 0x7f5f840adc10 2021-11-21T21:54:32.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f88af3700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840adc10 0x7f5f8413fa30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5f74002fb0 tx=0x7f5f74006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f81ffb700 1 -- 172.21.2.116:0/132795602 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f74013460 con 0x7f5f840adc10 2021-11-21T21:54:32.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f81ffb700 1 -- 172.21.2.116:0/132795602 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f740135c0 con 0x7f5f840adc10 2021-11-21T21:54:32.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f84115bb0 con 0x7f5f840adc10 2021-11-21T21:54:32.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f81ffb700 1 -- 172.21.2.116:0/132795602 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f74013920 con 0x7f5f840adc10 2021-11-21T21:54:32.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8413ff20 con 0x7f5f840adc10 2021-11-21T21:54:32.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f84152370 con 0x7f5f840adc10 2021-11-21T21:54:32.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f81ffb700 1 -- 172.21.2.116:0/132795602 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5f7400f070 con 0x7f5f840adc10 2021-11-21T21:54:32.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f81ffb700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053580 0x7f5f68055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:32.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f81ffb700 1 -- 172.21.2.116:0/132795602 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5f7406b370 con 0x7f5f840adc10 2021-11-21T21:54:32.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f83fff700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053580 0x7f5f68055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:32.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.918+0000 7f5f83fff700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053580 0x7f5f68055a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5f6c002ef0 tx=0x7f5f6c006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:32.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:32.922+0000 7f5f81ffb700 1 -- 172.21.2.116:0/132795602 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f740424d0 con 0x7f5f840adc10 2021-11-21T21:54:33.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f840573d0 con 0x7f5f840adc10 2021-11-21T21:54:33.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f81ffb700 1 -- 172.21.2.116:0/132795602 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5f74045af0 con 0x7f5f840adc10 2021-11-21T21:54:33.009 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:33.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053580 msgr2=0x7f5f68055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053580 0x7f5f68055a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5f6c002ef0 tx=0x7f5f6c006910).stop 2021-11-21T21:54:33.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840adc10 msgr2=0x7f5f8413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840adc10 0x7f5f8413fa30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5f74002fb0 tx=0x7f5f74006df0).stop 2021-11-21T21:54:33.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 shutdown_connections 2021-11-21T21:54:33.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5f84144a10 0x7f5f841554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5f840b3690 0x7f5f84152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5f68053580 0x7f5f68055a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 --2- 172.21.2.116:0/132795602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5f840adc10 0x7f5f8413fa30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 >> 172.21.2.116:0/132795602 conn(0x7f5f8405bd20 msgr2=0x7f5f8413de70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:33.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 shutdown_connections 2021-11-21T21:54:33.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.006+0000 7f5f8ad57700 1 -- 172.21.2.116:0/132795602 wait complete. 2021-11-21T21:54:33.016 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:33.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 Processor -- start 2021-11-21T21:54:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- start start 2021-11-21T21:54:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff21c0b1270 0x7ff21c0b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c0ad580 0x7ff21c0ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21c0b0070 con 0x7ff21c0b6770 2021-11-21T21:54:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c0b05c0 con 0x7ff21c0b1270 2021-11-21T21:54:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0b2230 con 0x7ff21c0ad580 2021-11-21T21:54:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff221470700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff21c0b6770 0x7ff21c1394f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:56984/0 (socket says 172.21.2.116:56984) 2021-11-21T21:54:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff221470700 1 -- 172.21.2.116:0/3505755724 learned_addr learned my addr 172.21.2.116:0/3505755724 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624819114 0 0) 0x7ff21c0b0070 con 0x7ff21c0b6770 2021-11-21T21:54:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff204003260 con 0x7ff21c0b6770 2021-11-21T21:54:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 --2- 172.21.2.116:0/3505755724 >> v2:172.21.2.117:3300/0 conn(0x7ff21c0b1270 0x7ff21c0b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104415435 0 0) 0x7ff204003260 con 0x7ff21c0b6770 2021-11-21T21:54:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c0ad580 msgr2=0x7ff21c0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c0ad580 0x7ff21c0ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 >> v2:172.21.2.117:3300/0 conn(0x7ff21c0b1270 msgr2=0x7ff21c0b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 --2- 172.21.2.116:0/3505755724 >> v2:172.21.2.117:3300/0 conn(0x7ff21c0b1270 0x7ff21c0b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0b24d0 con 0x7ff21c0b6770 2021-11-21T21:54:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7ff218001680 con 0x7ff21c0b6770 2021-11-21T21:54:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 >> v1:172.21.2.116:6789/0 conn(0x7ff21c0b6770 legacy=0x7ff21c1394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2040037d0 0x7ff204003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff204004500 0x7ff204004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c0ad580 0x7ff204004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0b05c0 con 0x7ff204004500 2021-11-21T21:54:33.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0b2230 con 0x7ff2040037d0 2021-11-21T21:54:33.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20400b990 con 0x7ff21c0ad580 2021-11-21T21:54:33.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff204004500 0x7ff204004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 -- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c0ad580 msgr2=0x7ff204004fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c0ad580 0x7ff204004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 -- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2040037d0 msgr2=0x7ff204003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2040037d0 0x7ff204003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 -- 172.21.2.116:0/3505755724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204003260 con 0x7ff204004500 2021-11-21T21:54:33.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff220c6f700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff204004500 0x7ff204004980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff20c00a000 tx=0x7ff20c008040).ready entity=mon.0 client_cookie=786f10ae25a5e1b9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff20c013670 con 0x7ff204004500 2021-11-21T21:54:33.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c0137d0 con 0x7ff204004500 2021-11-21T21:54:33.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff20bfff700 1 -- 172.21.2.116:0/3505755724 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff20c013b30 con 0x7ff204004500 2021-11-21T21:54:33.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- 172.21.2.116:0/3505755724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff204004500 msgr2=0x7ff204004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 --2- 172.21.2.116:0/3505755724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff204004500 0x7ff204004980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff20c00a000 tx=0x7ff20c008040).stop 2021-11-21T21:54:33.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff221c71700 1 -- 172.21.2.116:0/3505755724 reap_dead start 2021-11-21T21:54:33.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- 172.21.2.116:0/3505755724 shutdown_connections 2021-11-21T21:54:33.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- 172.21.2.116:0/3505755724 >> 172.21.2.116:0/3505755724 conn(0x7ff21c062040 msgr2=0x7ff21c0b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:33.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- 172.21.2.116:0/3505755724 shutdown_connections 2021-11-21T21:54:33.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- 172.21.2.116:0/3505755724 wait complete. 2021-11-21T21:54:33.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 Processor -- start 2021-11-21T21:54:33.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.066+0000 7ff2236d4700 1 -- start start 2021-11-21T21:54:33.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff204004500 0x7ff21c05c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff21c0b5830 0x7ff21c1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c155dd0 0x7ff21c12c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c1425c0 con 0x7ff21c0b5830 2021-11-21T21:54:33.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c13cca0 con 0x7ff204004500 2021-11-21T21:54:33.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c156250 con 0x7ff21c155dd0 2021-11-21T21:54:33.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff21c0b5830 0x7ff21c1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff21c0b5830 0x7ff21c1558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45040/0 (socket says 172.21.2.116:45040) 2021-11-21T21:54:33.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 -- 172.21.2.116:0/1748156402 learned_addr learned my addr 172.21.2.116:0/1748156402 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:33.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 -- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c155dd0 msgr2=0x7ff21c12c780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:33.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c155dd0 0x7ff21c12c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 -- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff204004500 msgr2=0x7ff21c05c060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:33.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff204004500 0x7ff21c05c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 -- 172.21.2.116:0/1748156402 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c13ced0 con 0x7ff21c0b5830 2021-11-21T21:54:33.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff220c6f700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff21c0b5830 0x7ff21c1558e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7ff20c00a000 tx=0x7ff20c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff20a7fc700 1 -- 172.21.2.116:0/1748156402 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff20c012d30 con 0x7ff21c0b5830 2021-11-21T21:54:33.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff20a7fc700 1 -- 172.21.2.116:0/1748156402 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c000e90 con 0x7ff21c0b5830 2021-11-21T21:54:33.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c1410a0 con 0x7ff21c0b5830 2021-11-21T21:54:33.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff20a7fc700 1 -- 172.21.2.116:0/1748156402 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff20c01d600 con 0x7ff21c0b5830 2021-11-21T21:54:33.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c140bc0 con 0x7ff21c0b5830 2021-11-21T21:54:33.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c13d060 con 0x7ff21c0b5830 2021-11-21T21:54:33.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff20a7fc700 1 -- 172.21.2.116:0/1748156402 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff20c01d760 con 0x7ff21c0b5830 2021-11-21T21:54:33.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff20a7fc700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff1fc053520 0x7ff1fc0559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff20a7fc700 1 -- 172.21.2.116:0/1748156402 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff20c06d4a0 con 0x7ff21c0b5830 2021-11-21T21:54:33.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.070+0000 7ff221470700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff1fc053520 0x7ff1fc0559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.074+0000 7ff221470700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff1fc053520 0x7ff1fc0559e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff218001390 tx=0x7ff21c13f7c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.074+0000 7ff20a7fc700 1 -- 172.21.2.116:0/1748156402 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff20c0444f0 con 0x7ff21c0b5830 2021-11-21T21:54:33.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.154+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff21c0573d0 con 0x7ff21c0b5830 2021-11-21T21:54:33.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.154+0000 7ff20a7fc700 1 -- 172.21.2.116:0/1748156402 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff20c047b10 con 0x7ff21c0b5830 2021-11-21T21:54:33.160 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:33.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff1fc053520 msgr2=0x7ff1fc0559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff1fc053520 0x7ff1fc0559e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff218001390 tx=0x7ff21c13f7c0).stop 2021-11-21T21:54:33.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff21c0b5830 msgr2=0x7ff21c1558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff21c0b5830 0x7ff21c1558e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7ff20c00a000 tx=0x7ff20c008040).stop 2021-11-21T21:54:33.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 shutdown_connections 2021-11-21T21:54:33.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff21c155dd0 0x7ff21c12c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff1fc053520 0x7ff1fc0559e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff21c0b5830 0x7ff21c1558e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 --2- 172.21.2.116:0/1748156402 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff204004500 0x7ff21c05c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 >> 172.21.2.116:0/1748156402 conn(0x7ff21c062040 msgr2=0x7ff21c13dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:33.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 shutdown_connections 2021-11-21T21:54:33.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.158+0000 7ff2236d4700 1 -- 172.21.2.116:0/1748156402 wait complete. 2021-11-21T21:54:33.167 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:33.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fd1eb7700 1 Processor -- start 2021-11-21T21:54:33.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fd1eb7700 1 -- start start 2021-11-21T21:54:33.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fd1eb7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc0a86b0 0x7f5fcc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fd1eb7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5fcc062410 0x7f5fcc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fd1eb7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fcc0a6e80 con 0x7f5fcc0a6aa0 2021-11-21T21:54:33.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fd1eb7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fcc0a8bc0 con 0x7f5fcc062410 2021-11-21T21:54:33.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fd1eb7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc14a7b0 con 0x7f5fcc0a86b0 2021-11-21T21:54:33.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fcb7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5fcc0a6aa0 0x7f5fcc1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57002/0 (socket says 172.21.2.116:57002) 2021-11-21T21:54:33.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fcb7fe700 1 -- 172.21.2.116:0/2195280246 learned_addr learned my addr 172.21.2.116:0/2195280246 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:33.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648344873 0 0) 0x7f5fcc0a6e80 con 0x7f5fcc0a6aa0 2021-11-21T21:54:33.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fac003260 con 0x7f5fcc0a6aa0 2021-11-21T21:54:33.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199208713 0 0) 0x7f5fac003260 con 0x7f5fcc0a6aa0 2021-11-21T21:54:33.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc0a86b0 msgr2=0x7f5fcc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc0a86b0 0x7f5fcc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 >> v2:172.21.2.117:3300/0 conn(0x7f5fcc062410 msgr2=0x7f5fcc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 --2- 172.21.2.116:0/2195280246 >> v2:172.21.2.117:3300/0 conn(0x7f5fcc062410 0x7f5fcc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc14aa10 con 0x7f5fcc0a6aa0 2021-11-21T21:54:33.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f5fbc001680 con 0x7f5fcc0a6aa0 2021-11-21T21:54:33.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 >> v1:172.21.2.116:6789/0 conn(0x7f5fcc0a6aa0 legacy=0x7f5fcc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fac003750 0x7f5fac003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc0a86b0 0x7f5fac004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fac004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0a8bc0 con 0x7f5fac004b10 2021-11-21T21:54:33.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc14a7b0 con 0x7f5fac003750 2021-11-21T21:54:33.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.214+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac00b990 con 0x7f5fcc0a86b0 2021-11-21T21:54:33.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fac004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 -- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc0a86b0 msgr2=0x7f5fac004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc0a86b0 0x7f5fac004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 -- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fac003750 msgr2=0x7f5fac003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fac003750 0x7f5fac003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 -- 172.21.2.116:0/2195280246 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac003260 con 0x7f5fac004b10 2021-11-21T21:54:33.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fac004f90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5fc40032f0 tx=0x7f5fc4006f90).ready entity=mon.0 client_cookie=75d165573564d2e4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc400b460 con 0x7f5fac004b10 2021-11-21T21:54:33.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc400b5c0 con 0x7f5fac004b10 2021-11-21T21:54:33.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fca7fc700 1 -- 172.21.2.116:0/2195280246 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc400b730 con 0x7f5fac004b10 2021-11-21T21:54:33.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2195280246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 msgr2=0x7f5fac004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 --2- 172.21.2.116:0/2195280246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fac004f90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5fc40032f0 tx=0x7f5fc4006f90).stop 2021-11-21T21:54:33.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcbfff700 1 -- 172.21.2.116:0/2195280246 reap_dead start 2021-11-21T21:54:33.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2195280246 shutdown_connections 2021-11-21T21:54:33.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2195280246 >> 172.21.2.116:0/2195280246 conn(0x7f5fcc05bd20 msgr2=0x7f5fcc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:33.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2195280246 shutdown_connections 2021-11-21T21:54:33.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2195280246 wait complete. 2021-11-21T21:54:33.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 Processor -- start 2021-11-21T21:54:33.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- start start 2021-11-21T21:54:33.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fcc066fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fcc0a9700 0x7f5fcc140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc065f70 0x7f5fcc140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc156220 con 0x7f5fac004b10 2021-11-21T21:54:33.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0b19e0 con 0x7f5fcc0a9700 2021-11-21T21:54:33.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0b1c80 con 0x7f5fcc065f70 2021-11-21T21:54:33.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fcc066fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fcc066fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45062/0 (socket says 172.21.2.116:45062) 2021-11-21T21:54:33.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 -- 172.21.2.116:0/2835549956 learned_addr learned my addr 172.21.2.116:0/2835549956 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:33.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 -- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc065f70 msgr2=0x7f5fcc140810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:33.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc065f70 0x7f5fcc140810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 -- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fcc0a9700 msgr2=0x7f5fcc140320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:33.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fcc0a9700 0x7f5fcc140320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 -- 172.21.2.116:0/2835549956 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0b1f20 con 0x7f5fac004b10 2021-11-21T21:54:33.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fcb7fe700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fcc066fc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f5fbc001390 tx=0x7f5fbc001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fc8ff9700 1 -- 172.21.2.116:0/2835549956 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fbc001bc0 con 0x7f5fac004b10 2021-11-21T21:54:33.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fc8ff9700 1 -- 172.21.2.116:0/2835549956 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fbc007170 con 0x7f5fac004b10 2021-11-21T21:54:33.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc0b4090 con 0x7f5fac004b10 2021-11-21T21:54:33.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc140ea0 con 0x7f5fac004b10 2021-11-21T21:54:33.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fc8ff9700 1 -- 172.21.2.116:0/2835549956 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fbc004730 con 0x7f5fac004b10 2021-11-21T21:54:33.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa4003410 con 0x7f5fac004b10 2021-11-21T21:54:33.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fc8ff9700 1 -- 172.21.2.116:0/2835549956 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5fbc004890 con 0x7f5fac004b10 2021-11-21T21:54:33.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fc8ff9700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fb0053720 0x7f5fb0055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.218+0000 7f5fc8ff9700 1 -- 172.21.2.116:0/2835549956 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5fbc06b3c0 con 0x7f5fac004b10 2021-11-21T21:54:33.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.222+0000 7f5fcaffd700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fb0053720 0x7f5fb0055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.222+0000 7f5fcaffd700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fb0053720 0x7f5fb0055be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f5fb4002f70 tx=0x7f5fb40069a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.222+0000 7f5fc8ff9700 1 -- 172.21.2.116:0/2835549956 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5fbc042410 con 0x7f5fac004b10 2021-11-21T21:54:33.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.330+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fa4003c40 con 0x7f5fac004b10 2021-11-21T21:54:33.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.330+0000 7f5fc8ff9700 1 -- 172.21.2.116:0/2835549956 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f5fbc045a30 con 0x7f5fac004b10 2021-11-21T21:54:33.335 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:33.335 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/862809391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":862809391},{"type":"v1","addr":"172.21.2.117:6835","nonce":862809391}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2422183040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2422183040},{"type":"v1","addr":"172.21.2.116:6833","nonce":2422183040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:24.904554+0000","modified":"2021-11-21T21:54:25.909207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/2778157441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2778157441},{"type":"v1","addr":"172.21.2.116:6835","nonce":2778157441}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-11-21T21:54:33.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.330+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fb0053720 msgr2=0x7f5fb0055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.330+0000 7f5fd1eb7700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fb0053720 0x7f5fb0055be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f5fb4002f70 tx=0x7f5fb40069a0).stop 2021-11-21T21:54:33.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.330+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 msgr2=0x7f5fcc066fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.330+0000 7f5fd1eb7700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fcc066fc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f5fbc001390 tx=0x7f5fbc001b50).stop 2021-11-21T21:54:33.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 shutdown_connections 2021-11-21T21:54:33.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fcc065f70 0x7f5fcc140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fcc0a9700 0x7f5fcc140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fb0053720 0x7f5fb0055be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 --2- 172.21.2.116:0/2835549956 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fac004b10 0x7f5fcc066fc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 >> 172.21.2.116:0/2835549956 conn(0x7f5fcc05bd20 msgr2=0x7f5fcc0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:33.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 shutdown_connections 2021-11-21T21:54:33.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.334+0000 7f5fd1eb7700 1 -- 172.21.2.116:0/2835549956 wait complete. 2021-11-21T21:54:33.345 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 5 2021-11-21T21:54:33.378 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:33.388 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:33.398 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:33.409 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:54:33.649 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:54:24.904554+0000', 'modified': '2021-11-21T21:54:25.909207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.116:6835/2778157441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2778157441}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2778157441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:54:33.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:54:33.724 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:54:33.789 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":4239,"want_state":"up:active","state":"up:active","rank_uptime":8.8783804049999997,"mdsmap_epoch":5,"osdmap_epoch":22,"osdmap_epoch_barrier":20,"uptime":11.028288235} 2021-11-21T21:54:33.790 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 4239, "mdsmap_epoch": 5, "osdmap_epoch": 22, "osdmap_epoch_barrier": 20, "rank_uptime": 8.878380405, "state": "up:active", "uptime": 11.028288235, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:54:33.790 INFO:teuthology.run_tasks:Running task ceph-fuse... 2021-11-21T21:54:33.810 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2021-11-21T21:54:33.811 INFO:tasks.ceph_fuse:config is None 2021-11-21T21:54:33.811 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None} 2021-11-21T21:54:33.811 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2021-11-21T21:54:33.812 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:33.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.858+0000 7f8763e24700 1 Processor -- start 2021-11-21T21:54:33.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- start start 2021-11-21T21:54:33.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f875c0a6aa0 0x7f875c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c0a86b0 0x7f875c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875c0a6e80 con 0x7f875c1475d0 2021-11-21T21:54:33.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875c0a8bc0 con 0x7f875c0a6aa0 2021-11-21T21:54:33.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c14a7b0 con 0x7f875c0a86b0 2021-11-21T21:54:33.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f875c0a6aa0 0x7f875c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f875c0a6aa0 0x7f875c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45066/0 (socket says 172.21.2.116:45066) 2021-11-21T21:54:33.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 -- 172.21.2.116:0/2197904052 learned_addr learned my addr 172.21.2.116:0/2197904052 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:33.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 -- 172.21.2.116:0/2197904052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c0a86b0 msgr2=0x7f875c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:33.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 --2- 172.21.2.116:0/2197904052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c0a86b0 0x7f875c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 -- 172.21.2.116:0/2197904052 >> v1:172.21.2.116:6789/0 conn(0x7f875c1475d0 legacy=0x7f875c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 -- 172.21.2.116:0/2197904052 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c14aa10 con 0x7f875c0a6aa0 2021-11-21T21:54:33.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 --2- 172.21.2.116:0/2197904052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c14b150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f8754001490 tx=0x7f874c006eb0).ready entity=mon.0 client_cookie=9edda10055763a25 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8760bbe700 1 -- 172.21.2.116:0/2197904052 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f874c00d570 con 0x7f875c0a6aa0 2021-11-21T21:54:33.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8760bbe700 1 -- 172.21.2.116:0/2197904052 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c000e90 con 0x7f875c0a6aa0 2021-11-21T21:54:33.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8760bbe700 1 -- 172.21.2.116:0/2197904052 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f874c00d900 con 0x7f875c0a6aa0 2021-11-21T21:54:33.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- 172.21.2.116:0/2197904052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 msgr2=0x7f875c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- 172.21.2.116:0/2197904052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c14b150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f8754001490 tx=0x7f874c006eb0).stop 2021-11-21T21:54:33.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- 172.21.2.116:0/2197904052 shutdown_connections 2021-11-21T21:54:33.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- 172.21.2.116:0/2197904052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c0a86b0 0x7f875c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- 172.21.2.116:0/2197904052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c14b150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- 172.21.2.116:0/2197904052 >> 172.21.2.116:0/2197904052 conn(0x7f875c05bd20 msgr2=0x7f875c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:33.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- 172.21.2.116:0/2197904052 shutdown_connections 2021-11-21T21:54:33.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- 172.21.2.116:0/2197904052 wait complete. 2021-11-21T21:54:33.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 Processor -- start 2021-11-21T21:54:33.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- start start 2021-11-21T21:54:33.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f875c0a86b0 0x7f875c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c1475d0 0x7f875c156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c142820 con 0x7f875c0a6aa0 2021-11-21T21:54:33.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c066620 con 0x7f875c0a86b0 2021-11-21T21:54:33.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8763e24700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c15a170 con 0x7f875c1475d0 2021-11-21T21:54:33.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45072/0 (socket says 172.21.2.116:45072) 2021-11-21T21:54:33.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.862+0000 7f8761bc0700 1 -- 172.21.2.116:0/3885175544 learned_addr learned my addr 172.21.2.116:0/3885175544 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:33.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8761bc0700 1 -- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c1475d0 msgr2=0x7f875c156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8761bc0700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c1475d0 0x7f875c156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8761bc0700 1 -- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f875c0a86b0 msgr2=0x7f875c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8761bc0700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f875c0a86b0 0x7f875c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8761bc0700 1 -- 172.21.2.116:0/3885175544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c00b1e0 con 0x7f875c0a6aa0 2021-11-21T21:54:33.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8761bc0700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c146220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f8754001490 tx=0x7f874c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f874affd700 1 -- 172.21.2.116:0/3885175544 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f874c011860 con 0x7f875c0a6aa0 2021-11-21T21:54:33.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f874affd700 1 -- 172.21.2.116:0/3885175544 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c0119c0 con 0x7f875c0a6aa0 2021-11-21T21:54:33.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c066380 con 0x7f875c0a6aa0 2021-11-21T21:54:33.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c1472d0 con 0x7f875c0a6aa0 2021-11-21T21:54:33.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f874affd700 1 -- 172.21.2.116:0/3885175544 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f874c011d20 con 0x7f875c0a6aa0 2021-11-21T21:54:33.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c0573d0 con 0x7f875c0a6aa0 2021-11-21T21:54:33.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f874affd700 1 -- 172.21.2.116:0/3885175544 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f874c025030 con 0x7f875c0a6aa0 2021-11-21T21:54:33.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f874affd700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f873c0536d0 0x7f873c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:33.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f874affd700 1 -- 172.21.2.116:0/3885175544 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f874c06cbe0 con 0x7f875c0a6aa0 2021-11-21T21:54:33.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f87613bf700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f873c0536d0 0x7f873c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:33.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.866+0000 7f87613bf700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f873c0536d0 0x7f873c055b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8758002f70 tx=0x7f8758006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:33.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.870+0000 7f874affd700 1 -- 172.21.2.116:0/3885175544 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f874c043d40 con 0x7f875c0a6aa0 2021-11-21T21:54:33.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.950+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f875c05db70 con 0x7f875c0a6aa0 2021-11-21T21:54:33.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f874affd700 1 -- 172.21.2.116:0/3885175544 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f874c023020 con 0x7f875c0a6aa0 2021-11-21T21:54:33.957 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:33.957 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:33.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f873c0536d0 msgr2=0x7f873c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f873c0536d0 0x7f873c055b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8758002f70 tx=0x7f8758006a80).stop 2021-11-21T21:54:33.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 msgr2=0x7f875c146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:33.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c146220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f8754001490 tx=0x7f874c015000).stop 2021-11-21T21:54:33.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 shutdown_connections 2021-11-21T21:54:33.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f875c1475d0 0x7f875c156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f875c0a86b0 0x7f875c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:33.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f873c0536d0 0x7f873c055b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 --2- 172.21.2.116:0/3885175544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f875c0a6aa0 0x7f875c146220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:33.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 >> 172.21.2.116:0/3885175544 conn(0x7f875c05bd20 msgr2=0x7f875c148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:33.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 shutdown_connections 2021-11-21T21:54:33.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:33.954+0000 7f8763e24700 1 -- 172.21.2.116:0/3885175544 wait complete. 2021-11-21T21:54:33.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:33.965 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:34.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 Processor -- start 2021-11-21T21:54:34.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 -- start start 2021-11-21T21:54:34.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd1100abce0 0x7fd1101478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd110147e80 con 0x7fd110144810 2021-11-21T21:54:34.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1100af5a0 con 0x7fd1100ac600 2021-11-21T21:54:34.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1100b1490 con 0x7fd1100abce0 2021-11-21T21:54:34.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10effd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd110144810 0x7fd1100b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57036/0 (socket says 172.21.2.116:57036) 2021-11-21T21:54:34.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10effd700 1 -- 172.21.2.116:0/4024550812 learned_addr learned my addr 172.21.2.116:0/4024550812 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:34.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10e7fc700 1 --2- 172.21.2.116:0/4024550812 >> v2:172.21.2.116:3300/0 conn(0x7fd1100abce0 0x7fd1101478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10e7fc700 1 -- 172.21.2.116:0/4024550812 >> v1:172.21.2.117:6789/0 conn(0x7fd1100ac600 legacy=0x7fd1100aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:34.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10e7fc700 1 -- 172.21.2.116:0/4024550812 >> v1:172.21.2.116:6789/0 conn(0x7fd110144810 legacy=0x7fd1100b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10e7fc700 1 -- 172.21.2.116:0/4024550812 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1100b1fd0 con 0x7fd1100abce0 2021-11-21T21:54:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10e7fc700 1 --2- 172.21.2.116:0/4024550812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd1101478c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd1100b4e00 tx=0x7fd100006df0).ready entity=mon.0 client_cookie=f4b8531d8fd3ec2f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10dffb700 1 -- 172.21.2.116:0/4024550812 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd10000d5c0 con 0x7fd1100abce0 2021-11-21T21:54:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10dffb700 1 -- 172.21.2.116:0/4024550812 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100000e90 con 0x7fd1100abce0 2021-11-21T21:54:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd10dffb700 1 -- 172.21.2.116:0/4024550812 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd10000d910 con 0x7fd1100abce0 2021-11-21T21:54:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 -- 172.21.2.116:0/4024550812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 msgr2=0x7fd1101478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 --2- 172.21.2.116:0/4024550812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd1101478c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd1100b4e00 tx=0x7fd100006df0).stop 2021-11-21T21:54:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 -- 172.21.2.116:0/4024550812 shutdown_connections 2021-11-21T21:54:34.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 --2- 172.21.2.116:0/4024550812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd1101478c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.014+0000 7fd1156ba700 1 -- 172.21.2.116:0/4024550812 >> 172.21.2.116:0/4024550812 conn(0x7fd11005bd20 msgr2=0x7fd11005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- 172.21.2.116:0/4024550812 shutdown_connections 2021-11-21T21:54:34.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- 172.21.2.116:0/4024550812 wait complete. 2021-11-21T21:54:34.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 Processor -- start 2021-11-21T21:54:34.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- start start 2021-11-21T21:54:34.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd110145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1100ac600 0x7fd110156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd110144810 0x7fd110157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110156b00 con 0x7fd1100abce0 2021-11-21T21:54:34.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1101431d0 con 0x7fd110144810 2021-11-21T21:54:34.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd110143470 con 0x7fd1100ac600 2021-11-21T21:54:34.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd110145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd110145310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45090/0 (socket says 172.21.2.116:45090) 2021-11-21T21:54:34.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 -- 172.21.2.116:0/3703822241 learned_addr learned my addr 172.21.2.116:0/3703822241 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 -- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1100ac600 msgr2=0x7fd110156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1100ac600 0x7fd110156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 -- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd110144810 msgr2=0x7fd110157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd110144810 0x7fd110157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 -- 172.21.2.116:0/3703822241 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10000b180 con 0x7fd1100abce0 2021-11-21T21:54:34.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10effd700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd110145310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fd110062500 tx=0x7fd0f8007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd107fff700 1 -- 172.21.2.116:0/3703822241 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f800ce30 con 0x7fd1100abce0 2021-11-21T21:54:34.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd107fff700 1 -- 172.21.2.116:0/3703822241 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f8007320 con 0x7fd1100abce0 2021-11-21T21:54:34.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd110143710 con 0x7fd1100abce0 2021-11-21T21:54:34.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd107fff700 1 -- 172.21.2.116:0/3703822241 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f8005220 con 0x7fd1100abce0 2021-11-21T21:54:34.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd110157910 con 0x7fd1100abce0 2021-11-21T21:54:34.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1101474f0 con 0x7fd1100abce0 2021-11-21T21:54:34.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd107fff700 1 -- 172.21.2.116:0/3703822241 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd0f801b070 con 0x7fd1100abce0 2021-11-21T21:54:34.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd107fff700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0f40536f0 0x7fd0f4055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd107fff700 1 -- 172.21.2.116:0/3703822241 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd0f806aef0 con 0x7fd1100abce0 2021-11-21T21:54:34.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.018+0000 7fd10e7fc700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0f40536f0 0x7fd0f4055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.022+0000 7fd10e7fc700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0f40536f0 0x7fd0f4055bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd1100b4e00 tx=0x7fd100012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.022+0000 7fd107fff700 1 -- 172.21.2.116:0/3703822241 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0f8041f40 con 0x7fd1100abce0 2021-11-21T21:54:34.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd1100573d0 con 0x7fd1100abce0 2021-11-21T21:54:34.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd107fff700 1 -- 172.21.2.116:0/3703822241 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd0f8045560 con 0x7fd1100abce0 2021-11-21T21:54:34.110 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:34.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0f40536f0 msgr2=0x7fd0f4055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0f40536f0 0x7fd0f4055bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd1100b4e00 tx=0x7fd100012000).stop 2021-11-21T21:54:34.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 msgr2=0x7fd110145310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd110145310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fd110062500 tx=0x7fd0f8007c60).stop 2021-11-21T21:54:34.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 shutdown_connections 2021-11-21T21:54:34.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd110144810 0x7fd110157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1100ac600 0x7fd110156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd0f40536f0 0x7fd0f4055bb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 --2- 172.21.2.116:0/3703822241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1100abce0 0x7fd110145310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 >> 172.21.2.116:0/3703822241 conn(0x7fd11005bd20 msgr2=0x7fd1101401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 shutdown_connections 2021-11-21T21:54:34.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.106+0000 7fd1156ba700 1 -- 172.21.2.116:0/3703822241 wait complete. 2021-11-21T21:54:34.117 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:34.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 Processor -- start 2021-11-21T21:54:34.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- start start 2021-11-21T21:54:34.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb2e80b6770 0x7fb2e80b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e80b1270 0x7fb2e81394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb2e81399e0 0x7fb2e80ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e80b0070 con 0x7fb2e80b6770 2021-11-21T21:54:34.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e80b05c0 con 0x7fb2e81399e0 2021-11-21T21:54:34.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80b2230 con 0x7fb2e80b1270 2021-11-21T21:54:34.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb2e80b6770 0x7fb2e80b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb2e80b6770 0x7fb2e80b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45094/0 (socket says 172.21.2.116:45094) 2021-11-21T21:54:34.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 -- 172.21.2.116:0/1656092736 learned_addr learned my addr 172.21.2.116:0/1656092736 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 -- 172.21.2.116:0/1656092736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e80b1270 msgr2=0x7fb2e81394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:34.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 --2- 172.21.2.116:0/1656092736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e80b1270 0x7fb2e81394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 -- 172.21.2.116:0/1656092736 >> v2:172.21.2.117:3300/0 conn(0x7fb2e81399e0 msgr2=0x7fb2e80ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:34.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 --2- 172.21.2.116:0/1656092736 >> v2:172.21.2.117:3300/0 conn(0x7fb2e81399e0 0x7fb2e80ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 -- 172.21.2.116:0/1656092736 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e80b24d0 con 0x7fb2e80b6770 2021-11-21T21:54:34.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e659c700 1 --2- 172.21.2.116:0/1656092736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b6770 0x7fb2e80b6b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fb2d0002f70 tx=0x7fb2d0006f40).ready entity=mon.0 client_cookie=72f77ee3e17b1814 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e559a700 1 -- 172.21.2.116:0/1656092736 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2d000dc70 con 0x7fb2e80b6770 2021-11-21T21:54:34.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e559a700 1 -- 172.21.2.116:0/1656092736 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0000e90 con 0x7fb2e80b6770 2021-11-21T21:54:34.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2e559a700 1 -- 172.21.2.116:0/1656092736 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2d000be90 con 0x7fb2e80b6770 2021-11-21T21:54:34.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- 172.21.2.116:0/1656092736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b6770 msgr2=0x7fb2e80b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/1656092736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b6770 0x7fb2e80b6b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fb2d0002f70 tx=0x7fb2d0006f40).stop 2021-11-21T21:54:34.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- 172.21.2.116:0/1656092736 shutdown_connections 2021-11-21T21:54:34.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/1656092736 >> v2:172.21.2.117:3300/0 conn(0x7fb2e81399e0 0x7fb2e80ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/1656092736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e80b1270 0x7fb2e81394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/1656092736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b6770 0x7fb2e80b6b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- 172.21.2.116:0/1656092736 >> 172.21.2.116:0/1656092736 conn(0x7fb2e8062040 msgr2=0x7fb2e80b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- 172.21.2.116:0/1656092736 shutdown_connections 2021-11-21T21:54:34.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.166+0000 7fb2ec84a700 1 -- 172.21.2.116:0/1656092736 wait complete. 2021-11-21T21:54:34.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 Processor -- start 2021-11-21T21:54:34.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 -- start start 2021-11-21T21:54:34.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b1270 0x7fb2e805bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2e80b6770 0x7fb2e8133490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e81399e0 0x7fb2e8133980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8142fd0 con 0x7fb2e80b1270 2021-11-21T21:54:34.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e813bac0 con 0x7fb2e80b6770 2021-11-21T21:54:34.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8140d40 con 0x7fb2e81399e0 2021-11-21T21:54:34.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b1270 0x7fb2e805bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b1270 0x7fb2e805bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45098/0 (socket says 172.21.2.116:45098) 2021-11-21T21:54:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 -- 172.21.2.116:0/221874014 learned_addr learned my addr 172.21.2.116:0/221874014 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 -- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e81399e0 msgr2=0x7fb2e8133980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e81399e0 0x7fb2e8133980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 -- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2e80b6770 msgr2=0x7fb2e8133490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2e80b6770 0x7fb2e8133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 -- 172.21.2.116:0/221874014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d000b460 con 0x7fb2e80b1270 2021-11-21T21:54:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e659c700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b1270 0x7fb2e805bec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb2d0002f70 tx=0x7fb2e80af4d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2df7fe700 1 -- 172.21.2.116:0/221874014 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2d0010e20 con 0x7fb2e80b1270 2021-11-21T21:54:34.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2df7fe700 1 -- 172.21.2.116:0/221874014 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0024450 con 0x7fb2e80b1270 2021-11-21T21:54:34.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e8140fe0 con 0x7fb2e80b1270 2021-11-21T21:54:34.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2df7fe700 1 -- 172.21.2.116:0/221874014 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2d00247b0 con 0x7fb2e80b1270 2021-11-21T21:54:34.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e8141630 con 0x7fb2e80b1270 2021-11-21T21:54:34.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e80573d0 con 0x7fb2e80b1270 2021-11-21T21:54:34.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2df7fe700 1 -- 172.21.2.116:0/221874014 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb2d0021070 con 0x7fb2e80b1270 2021-11-21T21:54:34.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2df7fe700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2cc0536d0 0x7fb2cc055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2df7fe700 1 -- 172.21.2.116:0/221874014 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb2d006f370 con 0x7fb2e80b1270 2021-11-21T21:54:34.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.170+0000 7fb2e5d9b700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2cc0536d0 0x7fb2cc055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.174+0000 7fb2e5d9b700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2cc0536d0 0x7fb2cc055b90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb2d8002f70 tx=0x7fb2d8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.174+0000 7fb2df7fe700 1 -- 172.21.2.116:0/221874014 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb2d0046580 con 0x7fb2e80b1270 2021-11-21T21:54:34.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb2e8067ca0 con 0x7fb2e80b1270 2021-11-21T21:54:34.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2df7fe700 1 -- 172.21.2.116:0/221874014 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb2d0049ba0 con 0x7fb2e80b1270 2021-11-21T21:54:34.261 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:34.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2cc0536d0 msgr2=0x7fb2cc055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2cc0536d0 0x7fb2cc055b90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb2d8002f70 tx=0x7fb2d8006a80).stop 2021-11-21T21:54:34.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b1270 msgr2=0x7fb2e805bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b1270 0x7fb2e805bec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb2d0002f70 tx=0x7fb2e80af4d0).stop 2021-11-21T21:54:34.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 shutdown_connections 2021-11-21T21:54:34.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2e81399e0 0x7fb2e8133980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2e80b6770 0x7fb2e8133490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2cc0536d0 0x7fb2cc055b90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 --2- 172.21.2.116:0/221874014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2e80b1270 0x7fb2e805bec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 >> 172.21.2.116:0/221874014 conn(0x7fb2e8062040 msgr2=0x7fb2e813f030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 shutdown_connections 2021-11-21T21:54:34.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.258+0000 7fb2ec84a700 1 -- 172.21.2.116:0/221874014 wait complete. 2021-11-21T21:54:34.268 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:34.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 Processor -- start 2021-11-21T21:54:34.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- start start 2021-11-21T21:54:34.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe034144810 0x7fe0341478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe0340abce0 0x7fe0340b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe034147e80 con 0x7fe0340b1db0 2021-11-21T21:54:34.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0340b1490 con 0x7fe0340abce0 2021-11-21T21:54:34.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340af060 con 0x7fe034144810 2021-11-21T21:54:34.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe034144810 0x7fe0341478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe034144810 0x7fe0341478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:48450/0 (socket says 172.21.2.116:48450) 2021-11-21T21:54:34.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe038ff2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe0340abce0 0x7fe0340b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 -- 172.21.2.116:0/4066763040 learned_addr learned my addr 172.21.2.116:0/4066763040 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 -- 172.21.2.116:0/4066763040 >> v2:172.21.2.117:3300/0 conn(0x7fe0340abce0 msgr2=0x7fe0340b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:34.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 --2- 172.21.2.116:0/4066763040 >> v2:172.21.2.117:3300/0 conn(0x7fe0340abce0 0x7fe0340b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 -- 172.21.2.116:0/4066763040 >> v1:172.21.2.117:6789/0 conn(0x7fe0340b1db0 legacy=0x7fe0340ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 -- 172.21.2.116:0/4066763040 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340af5a0 con 0x7fe034144810 2021-11-21T21:54:34.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe038ff2700 1 --2- 172.21.2.116:0/4066763040 >> v2:172.21.2.117:3300/0 conn(0x7fe0340abce0 0x7fe0340b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:34.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe0397f3700 1 --2- 172.21.2.116:0/4066763040 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe034144810 0x7fe0341478c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe024002f70 tx=0x7fe024006eb0).ready entity=mon.2 client_cookie=2f14afaeb1e1c788 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe023fff700 1 -- 172.21.2.116:0/4066763040 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe02400d570 con 0x7fe034144810 2021-11-21T21:54:34.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe023fff700 1 -- 172.21.2.116:0/4066763040 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe024000e90 con 0x7fe034144810 2021-11-21T21:54:34.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe023fff700 1 -- 172.21.2.116:0/4066763040 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe02400d900 con 0x7fe034144810 2021-11-21T21:54:34.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- 172.21.2.116:0/4066763040 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe034144810 msgr2=0x7fe0341478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 --2- 172.21.2.116:0/4066763040 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe034144810 0x7fe0341478c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe024002f70 tx=0x7fe024006eb0).stop 2021-11-21T21:54:34.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- 172.21.2.116:0/4066763040 shutdown_connections 2021-11-21T21:54:34.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 --2- 172.21.2.116:0/4066763040 >> v2:172.21.2.117:3300/0 conn(0x7fe0340abce0 0x7fe0340b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 --2- 172.21.2.116:0/4066763040 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe034144810 0x7fe0341478c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- 172.21.2.116:0/4066763040 >> 172.21.2.116:0/4066763040 conn(0x7fe03405bd20 msgr2=0x7fe03405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- 172.21.2.116:0/4066763040 shutdown_connections 2021-11-21T21:54:34.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.318+0000 7fe03ba57700 1 -- 172.21.2.116:0/4066763040 wait complete. 2021-11-21T21:54:34.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 Processor -- start 2021-11-21T21:54:34.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 -- start start 2021-11-21T21:54:34.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0340abce0 0x7fe034143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe034144810 0x7fe0340b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe03413b9b0 0x7fe0340b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341466f0 con 0x7fe034144810 2021-11-21T21:54:34.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03413bec0 con 0x7fe03413b9b0 2021-11-21T21:54:34.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03415a640 con 0x7fe0340abce0 2021-11-21T21:54:34.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe034144810 0x7fe0340b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe034144810 0x7fe0340b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45114/0 (socket says 172.21.2.116:45114) 2021-11-21T21:54:34.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 -- 172.21.2.116:0/182987565 learned_addr learned my addr 172.21.2.116:0/182987565 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0397f3700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0340abce0 0x7fe034143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe039ff4700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe03413b9b0 0x7fe0340b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 -- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0340abce0 msgr2=0x7fe034143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0340abce0 0x7fe034143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 -- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe03413b9b0 msgr2=0x7fe0340b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe03413b9b0 0x7fe0340b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 -- 172.21.2.116:0/182987565 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03000b180 con 0x7fe034144810 2021-11-21T21:54:34.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe039ff4700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe03413b9b0 0x7fe0340b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:54:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe038ff2700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe034144810 0x7fe0340b4000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fe03000c000 tx=0x7fe030006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0227fc700 1 -- 172.21.2.116:0/182987565 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe03000c890 con 0x7fe034144810 2021-11-21T21:54:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0227fc700 1 -- 172.21.2.116:0/182987565 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03000c9f0 con 0x7fe034144810 2021-11-21T21:54:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02400b1e0 con 0x7fe034144810 2021-11-21T21:54:34.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0227fc700 1 -- 172.21.2.116:0/182987565 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe03000cd50 con 0x7fe034144810 2021-11-21T21:54:34.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03413c120 con 0x7fe034144810 2021-11-21T21:54:34.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03413c2e0 con 0x7fe034144810 2021-11-21T21:54:34.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0227fc700 1 -- 172.21.2.116:0/182987565 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe03001c460 con 0x7fe034144810 2021-11-21T21:54:34.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0227fc700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe014053580 0x7fe014055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0227fc700 1 -- 172.21.2.116:0/182987565 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe03006bfb0 con 0x7fe034144810 2021-11-21T21:54:34.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0397f3700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe014053580 0x7fe014055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.322+0000 7fe0397f3700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe014053580 0x7fe014055a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fe02401c000 tx=0x7fe024015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.326+0000 7fe0227fc700 1 -- 172.21.2.116:0/182987565 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe0300430b0 con 0x7fe034144810 2021-11-21T21:54:34.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.406+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0340af300 con 0x7fe034144810 2021-11-21T21:54:34.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe0227fc700 1 -- 172.21.2.116:0/182987565 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe0300466d0 con 0x7fe034144810 2021-11-21T21:54:34.412 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:34.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe014053580 msgr2=0x7fe014055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe014053580 0x7fe014055a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fe02401c000 tx=0x7fe024015000).stop 2021-11-21T21:54:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe034144810 msgr2=0x7fe0340b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe034144810 0x7fe0340b4000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fe03000c000 tx=0x7fe030006df0).stop 2021-11-21T21:54:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 shutdown_connections 2021-11-21T21:54:34.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe03413b9b0 0x7fe0340b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe014053580 0x7fe014055a40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe034144810 0x7fe0340b4000 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 --2- 172.21.2.116:0/182987565 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0340abce0 0x7fe034143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 >> 172.21.2.116:0/182987565 conn(0x7fe03405bd20 msgr2=0x7fe0340b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 shutdown_connections 2021-11-21T21:54:34.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.410+0000 7fe03ba57700 1 -- 172.21.2.116:0/182987565 wait complete. 2021-11-21T21:54:34.420 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:34.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 Processor -- start 2021-11-21T21:54:34.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- start start 2021-11-21T21:54:34.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e40adc10 0x7f88e40b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f88e4144a10 0x7f88e4141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e40b3c50 con 0x7f88e4141510 2021-11-21T21:54:34.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e40b3ef0 con 0x7f88e4144a10 2021-11-21T21:54:34.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40b4190 con 0x7f88e40adc10 2021-11-21T21:54:34.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f88e4144a10 0x7f88e4141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f88e4144a10 0x7f88e4141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45124/0 (socket says 172.21.2.116:45124) 2021-11-21T21:54:34.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 -- 172.21.2.116:0/847042497 learned_addr learned my addr 172.21.2.116:0/847042497 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 -- 172.21.2.116:0/847042497 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e40adc10 msgr2=0x7f88e40b2030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:34.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 --2- 172.21.2.116:0/847042497 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e40adc10 0x7f88e40b2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 -- 172.21.2.116:0/847042497 >> v1:172.21.2.116:6789/0 conn(0x7f88e4141510 legacy=0x7f88e413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 -- 172.21.2.116:0/847042497 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4140b80 con 0x7f88e4144a10 2021-11-21T21:54:34.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e8efd700 1 -- 172.21.2.116:0/847042497 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138457720 0 0) 0x7f88e40b3c50 con 0x7f88e4141510 2021-11-21T21:54:34.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e96fe700 1 --2- 172.21.2.116:0/847042497 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e4144a10 0x7f88e4141020 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f88e0002f70 tx=0x7f88e0006eb0).ready entity=mon.0 client_cookie=339b31a3d543fda4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e8efd700 1 -- 172.21.2.116:0/847042497 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e000d5c0 con 0x7f88e4144a10 2021-11-21T21:54:34.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e8efd700 1 -- 172.21.2.116:0/847042497 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e0000e90 con 0x7f88e4144a10 2021-11-21T21:54:34.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88e8efd700 1 -- 172.21.2.116:0/847042497 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e000d950 con 0x7f88e4144a10 2021-11-21T21:54:34.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- 172.21.2.116:0/847042497 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e4144a10 msgr2=0x7f88e4141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 --2- 172.21.2.116:0/847042497 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e4144a10 0x7f88e4141020 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f88e0002f70 tx=0x7f88e0006eb0).stop 2021-11-21T21:54:34.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- 172.21.2.116:0/847042497 shutdown_connections 2021-11-21T21:54:34.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 --2- 172.21.2.116:0/847042497 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e4144a10 0x7f88e4141020 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 --2- 172.21.2.116:0/847042497 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e40adc10 0x7f88e40b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- 172.21.2.116:0/847042497 >> 172.21.2.116:0/847042497 conn(0x7f88e405bd20 msgr2=0x7f88e405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- 172.21.2.116:0/847042497 shutdown_connections 2021-11-21T21:54:34.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.470+0000 7f88ec163700 1 -- 172.21.2.116:0/847042497 wait complete. 2021-11-21T21:54:34.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 Processor -- start 2021-11-21T21:54:34.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 -- start start 2021-11-21T21:54:34.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e40adc10 0x7f88e413d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e4141510 0x7f88e40670c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88e4144a10 0x7f88e4155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4115d00 con 0x7f88e40adc10 2021-11-21T21:54:34.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4155180 con 0x7f88e4144a10 2021-11-21T21:54:34.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4152b40 con 0x7f88e4141510 2021-11-21T21:54:34.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e40adc10 0x7f88e413d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e40adc10 0x7f88e413d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45126/0 (socket says 172.21.2.116:45126) 2021-11-21T21:54:34.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 -- 172.21.2.116:0/2797134496 learned_addr learned my addr 172.21.2.116:0/2797134496 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 -- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e4141510 msgr2=0x7f88e40670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e4141510 0x7f88e40670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 -- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88e4144a10 msgr2=0x7f88e4155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88e4144a10 0x7f88e4155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 -- 172.21.2.116:0/2797134496 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e000b1e0 con 0x7f88e40adc10 2021-11-21T21:54:34.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e9eff700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e40adc10 0x7f88e413d2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f88e41150f0 tx=0x7f88d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88d2ffd700 1 -- 172.21.2.116:0/2797134496 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d4013460 con 0x7f88e40adc10 2021-11-21T21:54:34.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88d2ffd700 1 -- 172.21.2.116:0/2797134496 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d40135c0 con 0x7f88e40adc10 2021-11-21T21:54:34.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e40af5f0 con 0x7f88e40adc10 2021-11-21T21:54:34.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88d2ffd700 1 -- 172.21.2.116:0/2797134496 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d4013920 con 0x7f88e40adc10 2021-11-21T21:54:34.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e40af820 con 0x7f88e40adc10 2021-11-21T21:54:34.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e40afc60 con 0x7f88e40adc10 2021-11-21T21:54:34.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88d2ffd700 1 -- 172.21.2.116:0/2797134496 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f88d400f070 con 0x7f88e40adc10 2021-11-21T21:54:34.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88d2ffd700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88c40536a0 0x7f88c4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88d2ffd700 1 -- 172.21.2.116:0/2797134496 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f88d406b3b0 con 0x7f88e40adc10 2021-11-21T21:54:34.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.474+0000 7f88e96fe700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88c40536a0 0x7f88c4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.478+0000 7f88e96fe700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88c40536a0 0x7f88c4055b60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f88e0002f70 tx=0x7f88e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.478+0000 7f88d2ffd700 1 -- 172.21.2.116:0/2797134496 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f88d4042510 con 0x7f88e40adc10 2021-11-21T21:54:34.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88e405db30 con 0x7f88e40adc10 2021-11-21T21:54:34.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88d2ffd700 1 -- 172.21.2.116:0/2797134496 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f88d4045b30 con 0x7f88e40adc10 2021-11-21T21:54:34.566 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88c40536a0 msgr2=0x7f88c4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88c40536a0 0x7f88c4055b60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f88e0002f70 tx=0x7f88e0015000).stop 2021-11-21T21:54:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e40adc10 msgr2=0x7f88e413d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e40adc10 0x7f88e413d2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f88e41150f0 tx=0x7f88d4006df0).stop 2021-11-21T21:54:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 shutdown_connections 2021-11-21T21:54:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88e4144a10 0x7f88e4155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88e4141510 0x7f88e40670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88c40536a0 0x7f88c4055b60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 --2- 172.21.2.116:0/2797134496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88e40adc10 0x7f88e413d2b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 >> 172.21.2.116:0/2797134496 conn(0x7f88e405bd20 msgr2=0x7f88e40b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 shutdown_connections 2021-11-21T21:54:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.562+0000 7f88ec163700 1 -- 172.21.2.116:0/2797134496 wait complete. 2021-11-21T21:54:34.573 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:54:34.574 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:34.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 Processor -- start 2021-11-21T21:54:34.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- start start 2021-11-21T21:54:34.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b0062410 0x7f06b0144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b00a6e80 con 0x7f06b00a6aa0 2021-11-21T21:54:34.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b014a7b0 con 0x7f06b00a86b0 2021-11-21T21:54:34.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00a8bc0 con 0x7f06b0062410 2021-11-21T21:54:34.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f06b00a6aa0 0x7f06b01470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57088/0 (socket says 172.21.2.116:57088) 2021-11-21T21:54:34.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 -- 172.21.2.116:0/1751348118 learned_addr learned my addr 172.21.2.116:0/1751348118 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:34.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824530441 0 0) 0x7f06b00a6e80 con 0x7f06b00a6aa0 2021-11-21T21:54:34.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f069c003260 con 0x7f06b00a6aa0 2021-11-21T21:54:34.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 810365296 0 0) 0x7f069c003260 con 0x7f06b00a6aa0 2021-11-21T21:54:34.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b0062410 msgr2=0x7f06b0144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:34.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b0062410 0x7f06b0144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 >> v1:172.21.2.117:6789/0 conn(0x7f06b00a86b0 legacy=0x7f06b01443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:34.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b014aa10 con 0x7f06b00a6aa0 2021-11-21T21:54:34.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f06a8001a00 con 0x7f06b00a6aa0 2021-11-21T21:54:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 >> v1:172.21.2.116:6789/0 conn(0x7f06b00a6aa0 legacy=0x7f06b01470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f069c003750 0x7f069c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b0062410 0x7f069c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f069c004b10 0x7f069c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00a8bc0 con 0x7f069c003750 2021-11-21T21:54:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c00b940 con 0x7f069c004b10 2021-11-21T21:54:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c00bb00 con 0x7f06b0062410 2021-11-21T21:54:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f069c003750 0x7f069c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 -- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b0062410 msgr2=0x7f069c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b0062410 0x7f069c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 -- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f069c004b10 msgr2=0x7f069c004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f069c004b10 0x7f069c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 -- 172.21.2.116:0/1751348118 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c003260 con 0x7f069c003750 2021-11-21T21:54:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06affff700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f069c003750 0x7f069c003b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f06b0061b50 tx=0x7f06b0062060).ready entity=mon.0 client_cookie=19922c9ab11491b0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a800cbc0 con 0x7f069c003750 2021-11-21T21:54:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800cd20 con 0x7f069c003750 2021-11-21T21:54:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06aeffd700 1 -- 172.21.2.116:0/1751348118 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a8004b50 con 0x7f069c003750 2021-11-21T21:54:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- 172.21.2.116:0/1751348118 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f069c003750 msgr2=0x7f069c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 --2- 172.21.2.116:0/1751348118 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f069c003750 0x7f069c003b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f06b0061b50 tx=0x7f06b0062060).stop 2021-11-21T21:54:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b4a59700 1 -- 172.21.2.116:0/1751348118 reap_dead start 2021-11-21T21:54:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- 172.21.2.116:0/1751348118 shutdown_connections 2021-11-21T21:54:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- 172.21.2.116:0/1751348118 >> 172.21.2.116:0/1751348118 conn(0x7f06b005bd20 msgr2=0x7f06b005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- 172.21.2.116:0/1751348118 shutdown_connections 2021-11-21T21:54:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- 172.21.2.116:0/1751348118 wait complete. 2021-11-21T21:54:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 Processor -- start 2021-11-21T21:54:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.622+0000 7f06b64bc700 1 -- start start 2021-11-21T21:54:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f069c003750 0x7f06b014e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b013e900 0x7f06b0139250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b0139740 0x7f06b01564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b013db00 con 0x7f06b013e900 2021-11-21T21:54:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01560f0 con 0x7f06b0139740 2021-11-21T21:54:34.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b013bf40 con 0x7f069c003750 2021-11-21T21:54:34.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b013e900 0x7f06b0139250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b013e900 0x7f06b0139250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45146/0 (socket says 172.21.2.116:45146) 2021-11-21T21:54:34.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 -- 172.21.2.116:0/1000393292 learned_addr learned my addr 172.21.2.116:0/1000393292 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:34.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 -- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f069c003750 msgr2=0x7f06b014e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f069c003750 0x7f06b014e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 -- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b0139740 msgr2=0x7f06b01564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b0139740 0x7f06b01564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 -- 172.21.2.116:0/1000393292 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b014c2b0 con 0x7f06b013e900 2021-11-21T21:54:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06af7fe700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b013e900 0x7f06b0139250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f06b01563c0 tx=0x7f06a4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06ad7fa700 1 -- 172.21.2.116:0/1000393292 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a4009e60 con 0x7f06b013e900 2021-11-21T21:54:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06ad7fa700 1 -- 172.21.2.116:0/1000393292 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a4013460 con 0x7f06b013e900 2021-11-21T21:54:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b014c550 con 0x7f06b013e900 2021-11-21T21:54:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06ad7fa700 1 -- 172.21.2.116:0/1000393292 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a40137c0 con 0x7f06b013e900 2021-11-21T21:54:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b0139bc0 con 0x7f06b013e900 2021-11-21T21:54:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b014c9f0 con 0x7f06b013e900 2021-11-21T21:54:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06ad7fa700 1 -- 172.21.2.116:0/1000393292 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f06a400f070 con 0x7f06b013e900 2021-11-21T21:54:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06ad7fa700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0690053610 0x7f0690055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06ad7fa700 1 -- 172.21.2.116:0/1000393292 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f06a406b120 con 0x7f06b013e900 2021-11-21T21:54:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06affff700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0690053610 0x7f0690055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.626+0000 7f06affff700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0690053610 0x7f0690055ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f06b0061b50 tx=0x7f06a8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.630+0000 7f06ad7fa700 1 -- 172.21.2.116:0/1000393292 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06a40421e0 con 0x7f06b013e900 2021-11-21T21:54:34.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.738+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06b00573d0 con 0x7f06b013e900 2021-11-21T21:54:34.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.738+0000 7f06ad7fa700 1 -- 172.21.2.116:0/1000393292 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f06a4045800 con 0x7f06b013e900 2021-11-21T21:54:34.743 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:34.744 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/862809391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":862809391},{"type":"v1","addr":"172.21.2.117:6835","nonce":862809391}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2422183040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2422183040},{"type":"v1","addr":"172.21.2.116:6833","nonce":2422183040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:24.904554+0000","modified":"2021-11-21T21:54:25.909207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/2778157441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2778157441},{"type":"v1","addr":"172.21.2.116:6835","nonce":2778157441}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-11-21T21:54:34.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.738+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0690053610 msgr2=0x7f0690055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.738+0000 7f06b64bc700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0690053610 0x7f0690055ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f06b0061b50 tx=0x7f06a8015000).stop 2021-11-21T21:54:34.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.738+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b013e900 msgr2=0x7f06b0139250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:34.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.738+0000 7f06b64bc700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b013e900 0x7f06b0139250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f06b01563c0 tx=0x7f06a4006df0).stop 2021-11-21T21:54:34.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 shutdown_connections 2021-11-21T21:54:34.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b0139740 0x7f06b01564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0690053610 0x7f0690055ad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b013e900 0x7f06b0139250 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:34.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 --2- 172.21.2.116:0/1000393292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f069c003750 0x7f06b014e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:34.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 >> 172.21.2.116:0/1000393292 conn(0x7f06b005bd20 msgr2=0x7f06b0142da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:34.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 shutdown_connections 2021-11-21T21:54:34.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:34.742+0000 7f06b64bc700 1 -- 172.21.2.116:0/1000393292 wait complete. 2021-11-21T21:54:34.747 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 5 2021-11-21T21:54:35.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:54:24.904554+0000', 'modified': '2021-11-21T21:54:25.909207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.116:6835/2778157441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2778157441}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2778157441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:54:35.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:54:35.053 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:54:35.119 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":4239,"want_state":"up:active","state":"up:active","rank_uptime":10.208400170000001,"mdsmap_epoch":5,"osdmap_epoch":22,"osdmap_epoch_barrier":20,"uptime":12.358311684} 2021-11-21T21:54:35.119 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 4239, "mdsmap_epoch": 5, "osdmap_epoch": 22, "osdmap_epoch_barrier": 20, "rank_uptime": 10.20840017, "state": "up:active", "uptime": 12.358311684, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:54:35.120 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:54:35.120 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T21:54:35.121 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T21:54:35.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:35.121 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:35.132 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:54:35.144 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:54:35.144 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:54:35.150 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:54:35.150 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:54:35.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:35.152 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:54:35.199 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:54:35.200 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:35.202 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:54:35.211 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:54:35.212 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:54:35.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T21:54:35.244+0000 7fab21e8b080 -1 init, newargv = 0x55d2923889e0 newargc=9 2021-11-21T21:54:35.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[13977]: starting ceph client 2021-11-21T21:54:35.254 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:54:35.254 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:54:35.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:35.256 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:54:35.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[13977]: starting fuse 2021-11-21T21:54:35.267 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T21:54:35.268 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:54:35.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:35.269 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T21:54:35.269 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T21:54:35.269 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T21:54:35.270 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:54:35.270 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T21:54:35.270 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:35.271 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T21:54:35.271 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T21:54:35.271 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T21:54:35.272 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:35.272 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T21:54:35.272 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T21:54:35.272 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T21:54:35.273 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T21:54:35.273 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:35.273 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T21:54:35.274 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:54:35.274 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:54:35.274 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T21:54:35.275 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:54:35.275 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:35.275 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T21:54:35.276 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:54:35.371 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.13977.asok 2021-11-21T21:54:35.375 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13977.asok 2021-11-21T21:54:35.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:35.376 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13977.asok status 2021-11-21T21:54:35.436 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T21:54:35.436 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T21:54:35.436 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:54:35.436 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:54:35.437 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T21:54:35.437 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T21:54:35.437 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T21:54:35.437 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "13977", 2021-11-21T21:54:35.438 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T21:54:35.438 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:35.438 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T21:54:35.438 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T21:54:35.439 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4309, 2021-11-21T21:54:35.439 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T21:54:35.439 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T21:54:35.439 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T21:54:35.440 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4309 2021-11-21T21:54:35.440 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:35.440 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:54:35.440 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:54:35.440 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:54:35.441 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3063010776 2021-11-21T21:54:35.441 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T21:54:35.441 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:35.441 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:54:35.441 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:54:35.442 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:54:35.442 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3063010776 2021-11-21T21:54:35.442 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:35.442 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4309 172.21.2.119:0/3063010776", 2021-11-21T21:54:35.443 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3063010776", 2021-11-21T21:54:35.443 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T21:54:35.443 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 5, 2021-11-21T21:54:35.443 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 22, 2021-11-21T21:54:35.443 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:54:35.444 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T21:54:35.444 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T21:54:35.444 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T21:54:35.445 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:35.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 Processor -- start 2021-11-21T21:54:35.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- start start 2021-11-21T21:54:35.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 0x7fb5e814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb5e8062410 0x7fb5e81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e80a6e80 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e80a8bc0 con 0x7fb5e8062410 2021-11-21T21:54:35.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e814a7b0 con 0x7fb5e80a86b0 2021-11-21T21:54:35.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb5e8062410 0x7fb5e81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb5e8062410 0x7fb5e81443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45154/0 (socket says 172.21.2.116:45154) 2021-11-21T21:54:35.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 -- 172.21.2.116:0/3186650925 learned_addr learned my addr 172.21.2.116:0/3186650925 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:35.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 -- 172.21.2.116:0/3186650925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 msgr2=0x7fb5e814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:35.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 --2- 172.21.2.116:0/3186650925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 0x7fb5e814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 -- 172.21.2.116:0/3186650925 >> v1:172.21.2.117:6789/0 conn(0x7fb5e80a6aa0 legacy=0x7fb5e81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:35.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 -- 172.21.2.116:0/3186650925 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e814aa10 con 0x7fb5e8062410 2021-11-21T21:54:35.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e7fff700 1 --2- 172.21.2.116:0/3186650925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e8062410 0x7fb5e81443f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb5e8061ab0 tx=0x7fb5e0009290).ready entity=mon.0 client_cookie=5067af524cb13e6c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e67fc700 1 -- 172.21.2.116:0/3186650925 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e000b830 con 0x7fb5e8062410 2021-11-21T21:54:35.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e67fc700 1 -- 172.21.2.116:0/3186650925 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e000b990 con 0x7fb5e8062410 2021-11-21T21:54:35.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5e67fc700 1 -- 172.21.2.116:0/3186650925 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e000bb00 con 0x7fb5e8062410 2021-11-21T21:54:35.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- 172.21.2.116:0/3186650925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e8062410 msgr2=0x7fb5e81443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/3186650925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e8062410 0x7fb5e81443f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb5e8061ab0 tx=0x7fb5e0009290).stop 2021-11-21T21:54:35.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- 172.21.2.116:0/3186650925 shutdown_connections 2021-11-21T21:54:35.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/3186650925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e8062410 0x7fb5e81443f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/3186650925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 0x7fb5e814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- 172.21.2.116:0/3186650925 >> 172.21.2.116:0/3186650925 conn(0x7fb5e805bd20 msgr2=0x7fb5e805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:35.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- 172.21.2.116:0/3186650925 shutdown_connections 2021-11-21T21:54:35.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- 172.21.2.116:0/3186650925 wait complete. 2021-11-21T21:54:35.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 Processor -- start 2021-11-21T21:54:35.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.498+0000 7fb5edf2c700 1 -- start start 2021-11-21T21:54:35.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5e8062410 0x7fb5e814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e80a6aa0 0x7fb5e8066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 0x7fb5e8066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80ab060 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80ab290 con 0x7fb5e8062410 2021-11-21T21:54:35.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80671c0 con 0x7fb5e80a86b0 2021-11-21T21:54:35.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e80a6aa0 0x7fb5e8066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e80a6aa0 0x7fb5e8066420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45162/0 (socket says 172.21.2.116:45162) 2021-11-21T21:54:35.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 -- 172.21.2.116:0/2252160681 learned_addr learned my addr 172.21.2.116:0/2252160681 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:35.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 -- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 msgr2=0x7fb5e8066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 0x7fb5e8066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 -- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5e8062410 msgr2=0x7fb5e814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5e8062410 0x7fb5e814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 -- 172.21.2.116:0/2252160681 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e0009530 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e6ffd700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e80a6aa0 0x7fb5e8066420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fb5d0002f70 tx=0x7fb5d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e4ff9700 1 -- 172.21.2.116:0/2252160681 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5d0013460 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e4ff9700 1 -- 172.21.2.116:0/2252160681 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d00135c0 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e80aba80 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e8063c90 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e4ff9700 1 -- 172.21.2.116:0/2252160681 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5d0013920 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e80673c0 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e4ff9700 1 -- 172.21.2.116:0/2252160681 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb5d001a060 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e4ff9700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5cc0536f0 0x7fb5cc055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e4ff9700 1 -- 172.21.2.116:0/2252160681 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb5d006b430 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.502+0000 7fb5e77fe700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5cc0536f0 0x7fb5cc055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.506+0000 7fb5e77fe700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5cc0536f0 0x7fb5cc055bb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb5d8002f90 tx=0x7fb5d80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.506+0000 7fb5e4ff9700 1 -- 172.21.2.116:0/2252160681 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5d0042480 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.586+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5e8064b00 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.586+0000 7fb5e4ff9700 1 -- 172.21.2.116:0/2252160681 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7fb5d0018020 con 0x7fb5e80a6aa0 2021-11-21T21:54:35.593 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:35.594 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:35.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5cc0536f0 msgr2=0x7fb5cc055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5cc0536f0 0x7fb5cc055bb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb5d8002f90 tx=0x7fb5d80069b0).stop 2021-11-21T21:54:35.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e80a6aa0 msgr2=0x7fb5e8066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e80a6aa0 0x7fb5e8066420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fb5d0002f70 tx=0x7fb5d0006df0).stop 2021-11-21T21:54:35.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 shutdown_connections 2021-11-21T21:54:35.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5e80a86b0 0x7fb5e8066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5cc0536f0 0x7fb5cc055bb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5e80a6aa0 0x7fb5e8066420 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 --2- 172.21.2.116:0/2252160681 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5e8062410 0x7fb5e814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 >> 172.21.2.116:0/2252160681 conn(0x7fb5e805bd20 msgr2=0x7fb5e8149020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:35.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 shutdown_connections 2021-11-21T21:54:35.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.590+0000 7fb5edf2c700 1 -- 172.21.2.116:0/2252160681 wait complete. 2021-11-21T21:54:35.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:35.601 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:35.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 Processor -- start 2021-11-21T21:54:35.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- start start 2021-11-21T21:54:35.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f47c40abce0 0x7f47c41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40ac600 0x7f47c40aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c4147e80 con 0x7f47c4144810 2021-11-21T21:54:35.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c40b1490 con 0x7f47c40abce0 2021-11-21T21:54:35.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40b1fd0 con 0x7f47c40ac600 2021-11-21T21:54:35.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f47c40abce0 0x7f47c41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f47c40abce0 0x7f47c41478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45168/0 (socket says 172.21.2.116:45168) 2021-11-21T21:54:35.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 -- 172.21.2.116:0/4026379437 learned_addr learned my addr 172.21.2.116:0/4026379437 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:35.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 -- 172.21.2.116:0/4026379437 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40ac600 msgr2=0x7f47c40aca90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:35.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 --2- 172.21.2.116:0/4026379437 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40ac600 0x7f47c40aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 -- 172.21.2.116:0/4026379437 >> v1:172.21.2.117:6789/0 conn(0x7f47c4144810 legacy=0x7f47c40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:35.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 -- 172.21.2.116:0/4026379437 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40af5a0 con 0x7f47c40abce0 2021-11-21T21:54:35.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c3fff700 1 --2- 172.21.2.116:0/4026379437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40abce0 0x7f47c41478c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f47c4062500 tx=0x7f47ac006eb0).ready entity=mon.0 client_cookie=ea8365b6070b4b4c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c37fe700 1 -- 172.21.2.116:0/4026379437 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47ac00d570 con 0x7f47c40abce0 2021-11-21T21:54:35.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c37fe700 1 -- 172.21.2.116:0/4026379437 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac000e90 con 0x7f47c40abce0 2021-11-21T21:54:35.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47c37fe700 1 -- 172.21.2.116:0/4026379437 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47ac00d8c0 con 0x7f47c40abce0 2021-11-21T21:54:35.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- 172.21.2.116:0/4026379437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40abce0 msgr2=0x7f47c41478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 --2- 172.21.2.116:0/4026379437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40abce0 0x7f47c41478c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f47c4062500 tx=0x7f47ac006eb0).stop 2021-11-21T21:54:35.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- 172.21.2.116:0/4026379437 shutdown_connections 2021-11-21T21:54:35.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 --2- 172.21.2.116:0/4026379437 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40ac600 0x7f47c40aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 --2- 172.21.2.116:0/4026379437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40abce0 0x7f47c41478c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- 172.21.2.116:0/4026379437 >> 172.21.2.116:0/4026379437 conn(0x7f47c405bd20 msgr2=0x7f47c405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:35.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- 172.21.2.116:0/4026379437 shutdown_connections 2021-11-21T21:54:35.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.650+0000 7f47cb227700 1 -- 172.21.2.116:0/4026379437 wait complete. 2021-11-21T21:54:35.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 Processor -- start 2021-11-21T21:54:35.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 -- start start 2021-11-21T21:54:35.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40abce0 0x7f47c40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40ac600 0x7f47c4063e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c4144810 0x7f47c4064310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4146f90 con 0x7f47c40ac600 2021-11-21T21:54:35.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4146290 con 0x7f47c4144810 2021-11-21T21:54:35.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4145040 con 0x7f47c40abce0 2021-11-21T21:54:35.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40ac600 0x7f47c4063e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40ac600 0x7f47c4063e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45172/0 (socket says 172.21.2.116:45172) 2021-11-21T21:54:35.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 -- 172.21.2.116:0/3616698202 learned_addr learned my addr 172.21.2.116:0/3616698202 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:35.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 -- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40abce0 msgr2=0x7f47c40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40abce0 0x7f47c40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 -- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c4144810 msgr2=0x7f47c4064310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c4144810 0x7f47c4064310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 -- 172.21.2.116:0/3616698202 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac00b1e0 con 0x7f47c40ac600 2021-11-21T21:54:35.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c3fff700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40ac600 0x7f47c4063e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f47c4062500 tx=0x7f47ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c1ffb700 1 -- 172.21.2.116:0/3616698202 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47ac00d8f0 con 0x7f47c40ac600 2021-11-21T21:54:35.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c1ffb700 1 -- 172.21.2.116:0/3616698202 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac011ce0 con 0x7f47c40ac600 2021-11-21T21:54:35.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c1ffb700 1 -- 172.21.2.116:0/3616698202 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47ac01e650 con 0x7f47c40ac600 2021-11-21T21:54:35.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c41452e0 con 0x7f47c40ac600 2021-11-21T21:54:35.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c4145580 con 0x7f47c40ac600 2021-11-21T21:54:35.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c4159fb0 con 0x7f47c40ac600 2021-11-21T21:54:35.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c1ffb700 1 -- 172.21.2.116:0/3616698202 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f47ac01a040 con 0x7f47c40ac600 2021-11-21T21:54:35.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c1ffb700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47a8053610 0x7f47a8055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c1ffb700 1 -- 172.21.2.116:0/3616698202 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f47ac06ca80 con 0x7f47c40ac600 2021-11-21T21:54:35.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.654+0000 7f47c8fc3700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47a8053610 0x7f47a8055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.658+0000 7f47c8fc3700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47a8053610 0x7f47a8055ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f47b4002f10 tx=0x7f47b4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.658+0000 7f47c1ffb700 1 -- 172.21.2.116:0/3616698202 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f47ac043b10 con 0x7f47c40ac600 2021-11-21T21:54:35.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.738+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f47c40af060 con 0x7f47c40ac600 2021-11-21T21:54:35.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.738+0000 7f47c1ffb700 1 -- 172.21.2.116:0/3616698202 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f47ac0470f0 con 0x7f47c40ac600 2021-11-21T21:54:35.744 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:35.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47a8053610 msgr2=0x7f47a8055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47a8053610 0x7f47a8055ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f47b4002f10 tx=0x7f47b4006930).stop 2021-11-21T21:54:35.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40ac600 msgr2=0x7f47c4063e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40ac600 0x7f47c4063e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f47c4062500 tx=0x7f47ac015000).stop 2021-11-21T21:54:35.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 shutdown_connections 2021-11-21T21:54:35.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c4144810 0x7f47c4064310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47a8053610 0x7f47a8055ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c40ac600 0x7f47c4063e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 --2- 172.21.2.116:0/3616698202 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c40abce0 0x7f47c40afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 >> 172.21.2.116:0/3616698202 conn(0x7f47c405bd20 msgr2=0x7f47c4148580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:35.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 shutdown_connections 2021-11-21T21:54:35.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.742+0000 7f47cb227700 1 -- 172.21.2.116:0/3616698202 wait complete. 2021-11-21T21:54:35.751 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:35.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 Processor -- start 2021-11-21T21:54:35.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- start start 2021-11-21T21:54:35.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7feb300b5380 0x7feb300b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb300b13b0 0x7feb3013c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb300b0500 con 0x7feb3013c6b0 2021-11-21T21:54:35.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300adc70 con 0x7feb300b13b0 2021-11-21T21:54:35.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3013ec80 con 0x7feb300b5380 2021-11-21T21:54:35.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2e7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb300b13b0 0x7feb3013c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2f7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7feb3013c6b0 0x7feb301424e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57140/0 (socket says 172.21.2.116:57140) 2021-11-21T21:54:35.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2f7fe700 1 -- 172.21.2.116:0/4239959128 learned_addr learned my addr 172.21.2.116:0/4239959128 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:35.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2e7fc700 1 -- 172.21.2.116:0/4239959128 >> v2:172.21.2.117:3300/0 conn(0x7feb300b5380 msgr2=0x7feb300b5760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:35.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2e7fc700 1 --2- 172.21.2.116:0/4239959128 >> v2:172.21.2.117:3300/0 conn(0x7feb300b5380 0x7feb300b5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2e7fc700 1 -- 172.21.2.116:0/4239959128 >> v1:172.21.2.116:6789/0 conn(0x7feb3013c6b0 legacy=0x7feb301424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2e7fc700 1 -- 172.21.2.116:0/4239959128 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300b1b50 con 0x7feb300b13b0 2021-11-21T21:54:35.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2e7fc700 1 --2- 172.21.2.116:0/4239959128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb3013c1c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7feb28004b10 tx=0x7feb18006e80).ready entity=mon.0 client_cookie=85aca745fd8380e5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2dffb700 1 -- 172.21.2.116:0/4239959128 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb1800b900 con 0x7feb300b13b0 2021-11-21T21:54:35.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2dffb700 1 -- 172.21.2.116:0/4239959128 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18000e90 con 0x7feb300b13b0 2021-11-21T21:54:35.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb2dffb700 1 -- 172.21.2.116:0/4239959128 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb1800bc90 con 0x7feb300b13b0 2021-11-21T21:54:35.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- 172.21.2.116:0/4239959128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 msgr2=0x7feb3013c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 --2- 172.21.2.116:0/4239959128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb3013c1c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7feb28004b10 tx=0x7feb18006e80).stop 2021-11-21T21:54:35.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- 172.21.2.116:0/4239959128 shutdown_connections 2021-11-21T21:54:35.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 --2- 172.21.2.116:0/4239959128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb3013c1c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 --2- 172.21.2.116:0/4239959128 >> v2:172.21.2.117:3300/0 conn(0x7feb300b5380 0x7feb300b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- 172.21.2.116:0/4239959128 >> 172.21.2.116:0/4239959128 conn(0x7feb30061900 msgr2=0x7feb30069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:35.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- 172.21.2.116:0/4239959128 shutdown_connections 2021-11-21T21:54:35.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- 172.21.2.116:0/4239959128 wait complete. 2021-11-21T21:54:35.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 Processor -- start 2021-11-21T21:54:35.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.802+0000 7feb35269700 1 -- start start 2021-11-21T21:54:35.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb30141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb300b5380 0x7feb30137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb3013c6b0 0x7feb3014e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3014f210 con 0x7feb300b13b0 2021-11-21T21:54:35.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb30141fd0 con 0x7feb3013c6b0 2021-11-21T21:54:35.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb30139b30 con 0x7feb300b5380 2021-11-21T21:54:35.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb30141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb30141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45188/0 (socket says 172.21.2.116:45188) 2021-11-21T21:54:35.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 -- 172.21.2.116:0/4212009046 learned_addr learned my addr 172.21.2.116:0/4212009046 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:35.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 -- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb300b5380 msgr2=0x7feb30137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb300b5380 0x7feb30137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 -- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb3013c6b0 msgr2=0x7feb3014e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb3013c6b0 0x7feb3014e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 -- 172.21.2.116:0/4212009046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1800d410 con 0x7feb300b13b0 2021-11-21T21:54:35.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2effd700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb30141ae0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7feb3013a0c0 tx=0x7feb20006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb27fff700 1 -- 172.21.2.116:0/4212009046 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb20013460 con 0x7feb300b13b0 2021-11-21T21:54:35.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb27fff700 1 -- 172.21.2.116:0/4212009046 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200135c0 con 0x7feb300b13b0 2021-11-21T21:54:35.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb27fff700 1 -- 172.21.2.116:0/4212009046 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb20013920 con 0x7feb300b13b0 2021-11-21T21:54:35.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb30139dd0 con 0x7feb300b13b0 2021-11-21T21:54:35.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb30141640 con 0x7feb300b13b0 2021-11-21T21:54:35.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb300afd20 con 0x7feb300b13b0 2021-11-21T21:54:35.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb27fff700 1 -- 172.21.2.116:0/4212009046 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7feb2000f070 con 0x7feb300b13b0 2021-11-21T21:54:35.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb27fff700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb14053610 0x7feb14055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb27fff700 1 -- 172.21.2.116:0/4212009046 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7feb2006b070 con 0x7feb300b13b0 2021-11-21T21:54:35.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2e7fc700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb14053610 0x7feb14055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.806+0000 7feb2e7fc700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb14053610 0x7feb14055ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7feb28004b10 tx=0x7feb300ad500).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.810+0000 7feb27fff700 1 -- 172.21.2.116:0/4212009046 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feb200421d0 con 0x7feb300b13b0 2021-11-21T21:54:35.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feb300573d0 con 0x7feb300b13b0 2021-11-21T21:54:35.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb27fff700 1 -- 172.21.2.116:0/4212009046 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7feb20018030 con 0x7feb300b13b0 2021-11-21T21:54:35.897 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:35.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb14053610 msgr2=0x7feb14055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb14053610 0x7feb14055ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7feb28004b10 tx=0x7feb300ad500).stop 2021-11-21T21:54:35.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 msgr2=0x7feb30141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb30141ae0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7feb3013a0c0 tx=0x7feb20006df0).stop 2021-11-21T21:54:35.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 shutdown_connections 2021-11-21T21:54:35.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb3013c6b0 0x7feb3014e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb300b5380 0x7feb30137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb14053610 0x7feb14055ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 --2- 172.21.2.116:0/4212009046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb300b13b0 0x7feb30141ae0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 >> 172.21.2.116:0/4212009046 conn(0x7feb30061900 msgr2=0x7feb3013b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:35.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 shutdown_connections 2021-11-21T21:54:35.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.894+0000 7feb35269700 1 -- 172.21.2.116:0/4212009046 wait complete. 2021-11-21T21:54:35.904 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:35.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 Processor -- start 2021-11-21T21:54:35.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- start start 2021-11-21T21:54:35.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb0281426e0 0x7fb028142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb028062e60 0x7fb0280af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 0x7fb0280ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0280b0f60 con 0x7fb028062e60 2021-11-21T21:54:35.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb028063470 con 0x7fb0281426e0 2021-11-21T21:54:35.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb028143050 con 0x7fb028069e70 2021-11-21T21:54:35.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb028062e60 0x7fb0280af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb028062e60 0x7fb0280af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45194/0 (socket says 172.21.2.116:45194) 2021-11-21T21:54:35.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 -- 172.21.2.116:0/1935485925 learned_addr learned my addr 172.21.2.116:0/1935485925 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:35.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 -- 172.21.2.116:0/1935485925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 msgr2=0x7fb0280ad470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:35.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 --2- 172.21.2.116:0/1935485925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 0x7fb0280ad470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 -- 172.21.2.116:0/1935485925 >> v2:172.21.2.117:3300/0 conn(0x7fb0281426e0 msgr2=0x7fb028142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:35.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 --2- 172.21.2.116:0/1935485925 >> v2:172.21.2.117:3300/0 conn(0x7fb0281426e0 0x7fb028142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 -- 172.21.2.116:0/1935485925 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280b1e00 con 0x7fb028062e60 2021-11-21T21:54:35.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 --2- 172.21.2.116:0/1935485925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb028062e60 0x7fb0280af3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fb01c002f70 tx=0x7fb01c006eb0).ready entity=mon.0 client_cookie=e38e96663e316709 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02c9ff700 1 -- 172.21.2.116:0/1935485925 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb01c00d570 con 0x7fb028062e60 2021-11-21T21:54:35.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02c9ff700 1 -- 172.21.2.116:0/1935485925 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c000e90 con 0x7fb028062e60 2021-11-21T21:54:35.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02c9ff700 1 -- 172.21.2.116:0/1935485925 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb01c00d900 con 0x7fb028062e60 2021-11-21T21:54:35.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- 172.21.2.116:0/1935485925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb028062e60 msgr2=0x7fb0280af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:35.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1935485925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb028062e60 0x7fb0280af3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fb01c002f70 tx=0x7fb01c006eb0).stop 2021-11-21T21:54:35.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- 172.21.2.116:0/1935485925 shutdown_connections 2021-11-21T21:54:35.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1935485925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 0x7fb0280ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1935485925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb028062e60 0x7fb0280af3d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:35.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1935485925 >> v2:172.21.2.117:3300/0 conn(0x7fb0281426e0 0x7fb028142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- 172.21.2.116:0/1935485925 >> 172.21.2.116:0/1935485925 conn(0x7fb0280b5730 msgr2=0x7fb0280b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:35.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- 172.21.2.116:0/1935485925 shutdown_connections 2021-11-21T21:54:35.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- 172.21.2.116:0/1935485925 wait complete. 2021-11-21T21:54:35.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 Processor -- start 2021-11-21T21:54:35.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- start start 2021-11-21T21:54:35.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 0x7fb02805c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0281426e0 0x7fb0281597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb028156c70 0x7fb028159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02812a0f0 con 0x7fb0281426e0 2021-11-21T21:54:35.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02805d190 con 0x7fb028156c70 2021-11-21T21:54:35.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02fc65700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb028129710 con 0x7fb028069e70 2021-11-21T21:54:35.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0281426e0 0x7fb0281597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0281426e0 0x7fb0281597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45200/0 (socket says 172.21.2.116:45200) 2021-11-21T21:54:35.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.954+0000 7fb02d200700 1 -- 172.21.2.116:0/1351834702 learned_addr learned my addr 172.21.2.116:0/1351834702 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:35.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02d200700 1 -- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 msgr2=0x7fb02805c140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:35.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02d200700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 0x7fb02805c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02d200700 1 -- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb028156c70 msgr2=0x7fb028159cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:35.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02d200700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb028156c70 0x7fb028159cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:35.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02d200700 1 -- 172.21.2.116:0/1351834702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c00b1e0 con 0x7fb0281426e0 2021-11-21T21:54:35.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02d200700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0281426e0 0x7fb0281597c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fb01c002f70 tx=0x7fb01c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb01affd700 1 -- 172.21.2.116:0/1351834702 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb01c00de20 con 0x7fb0281426e0 2021-11-21T21:54:35.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb01affd700 1 -- 172.21.2.116:0/1351834702 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c011ce0 con 0x7fb0281426e0 2021-11-21T21:54:35.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0281299b0 con 0x7fb0281426e0 2021-11-21T21:54:35.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb01affd700 1 -- 172.21.2.116:0/1351834702 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb01c01e650 con 0x7fb0281426e0 2021-11-21T21:54:35.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02813b8d0 con 0x7fb0281426e0 2021-11-21T21:54:35.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0280573d0 con 0x7fb0281426e0 2021-11-21T21:54:35.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb01affd700 1 -- 172.21.2.116:0/1351834702 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb01c025030 con 0x7fb0281426e0 2021-11-21T21:54:35.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb01affd700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb008053660 0x7fb008055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:35.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb01affd700 1 -- 172.21.2.116:0/1351834702 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb01c06ce60 con 0x7fb0281426e0 2021-11-21T21:54:35.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02da01700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb008053660 0x7fb008055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:35.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.958+0000 7fb02da01700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb008053660 0x7fb008055b20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fb024000ed0 tx=0x7fb0240068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:35.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:35.962+0000 7fb01affd700 1 -- 172.21.2.116:0/1351834702 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb01c043eb0 con 0x7fb0281426e0 2021-11-21T21:54:36.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.042+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb02805f2d0 con 0x7fb0281426e0 2021-11-21T21:54:36.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb01affd700 1 -- 172.21.2.116:0/1351834702 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb01c023020 con 0x7fb0281426e0 2021-11-21T21:54:36.048 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:36.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb008053660 msgr2=0x7fb008055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb008053660 0x7fb008055b20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fb024000ed0 tx=0x7fb0240068b0).stop 2021-11-21T21:54:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0281426e0 msgr2=0x7fb0281597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0281426e0 0x7fb0281597c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fb01c002f70 tx=0x7fb01c015000).stop 2021-11-21T21:54:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 shutdown_connections 2021-11-21T21:54:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb028156c70 0x7fb028159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb008053660 0x7fb008055b20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0281426e0 0x7fb0281597c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 --2- 172.21.2.116:0/1351834702 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb028069e70 0x7fb02805c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 >> 172.21.2.116:0/1351834702 conn(0x7fb0280b5730 msgr2=0x7fb028141580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:36.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 shutdown_connections 2021-11-21T21:54:36.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.046+0000 7fb02fc65700 1 -- 172.21.2.116:0/1351834702 wait complete. 2021-11-21T21:54:36.055 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:36.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.102+0000 7fc884756700 1 Processor -- start 2021-11-21T21:54:36.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.102+0000 7fc884756700 1 -- start start 2021-11-21T21:54:36.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc87c1426e0 0x7fc87c142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c0afae0 con 0x7fc87c062e60 2021-11-21T21:54:36.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c0b0f60 con 0x7fc87c0ad470 2021-11-21T21:54:36.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c063470 con 0x7fc87c1426e0 2021-11-21T21:54:36.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc881cf1700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc87c062e60 0x7fc87c0af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57166/0 (socket says 172.21.2.116:57166) 2021-11-21T21:54:36.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc881cf1700 1 -- 172.21.2.116:0/2655097192 learned_addr learned my addr 172.21.2.116:0/2655097192 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:36.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993635295 0 0) 0x7fc87c0afae0 con 0x7fc87c062e60 2021-11-21T21:54:36.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc864003260 con 0x7fc87c062e60 2021-11-21T21:54:36.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 36544003 0 0) 0x7fc864003260 con 0x7fc87c062e60 2021-11-21T21:54:36.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 >> v2:172.21.2.117:3300/0 conn(0x7fc87c1426e0 msgr2=0x7fc87c142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:36.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 --2- 172.21.2.116:0/2655097192 >> v2:172.21.2.117:3300/0 conn(0x7fc87c1426e0 0x7fc87c142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 >> v1:172.21.2.117:6789/0 conn(0x7fc87c0ad470 legacy=0x7fc87c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:36.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c143050 con 0x7fc87c062e60 2021-11-21T21:54:36.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fc878001660 con 0x7fc87c062e60 2021-11-21T21:54:36.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 >> v1:172.21.2.116:6789/0 conn(0x7fc87c062e60 legacy=0x7fc87c0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc864003820 0x7fc864003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc864004500 0x7fc864004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc864005050 0x7fc864007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c063470 con 0x7fc864005050 2021-11-21T21:54:36.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864007f20 con 0x7fc864003820 2021-11-21T21:54:36.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640080e0 con 0x7fc864004500 2021-11-21T21:54:36.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc864005050 0x7fc864007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:36.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 -- 172.21.2.116:0/2655097192 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc864004500 msgr2=0x7fc864004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:36.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc864004500 0x7fc864004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 -- 172.21.2.116:0/2655097192 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc864003820 msgr2=0x7fc864003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:36.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc864003820 0x7fc864003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 -- 172.21.2.116:0/2655097192 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864003260 con 0x7fc864005050 2021-11-21T21:54:36.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 -- 172.21.2.116:0/2655097192 reap_dead start 2021-11-21T21:54:36.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc882cf3700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc864005050 0x7fc864007730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc8740012f0 tx=0x7fc874007d60).ready entity=mon.0 client_cookie=29ed1789f1951a32 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:36.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8740014c0 con 0x7fc864005050 2021-11-21T21:54:36.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc874006870 con 0x7fc864005050 2021-11-21T21:54:36.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc8814f0700 1 -- 172.21.2.116:0/2655097192 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc874006bd0 con 0x7fc864005050 2021-11-21T21:54:36.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- 172.21.2.116:0/2655097192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc864005050 msgr2=0x7fc864007730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc864005050 0x7fc864007730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc8740012f0 tx=0x7fc874007d60).stop 2021-11-21T21:54:36.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- 172.21.2.116:0/2655097192 shutdown_connections 2021-11-21T21:54:36.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 --2- 172.21.2.116:0/2655097192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc864005050 0x7fc864007730 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- 172.21.2.116:0/2655097192 >> 172.21.2.116:0/2655097192 conn(0x7fc87c0b5730 msgr2=0x7fc87c0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:36.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- 172.21.2.116:0/2655097192 shutdown_connections 2021-11-21T21:54:36.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- 172.21.2.116:0/2655097192 wait complete. 2021-11-21T21:54:36.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 Processor -- start 2021-11-21T21:54:36.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- start start 2021-11-21T21:54:36.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc864005050 0x7fc87c05c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc87c13efe0 0x7fc87c14d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc87c0639d0 0x7fc87c123ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c065150 con 0x7fc87c13efe0 2021-11-21T21:54:36.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c064290 con 0x7fc87c0639d0 2021-11-21T21:54:36.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc884756700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c066450 con 0x7fc864005050 2021-11-21T21:54:36.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc881cf1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc87c13efe0 0x7fc87c14d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:36.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc881cf1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc87c13efe0 0x7fc87c14d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45224/0 (socket says 172.21.2.116:45224) 2021-11-21T21:54:36.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.106+0000 7fc881cf1700 1 -- 172.21.2.116:0/4050601205 learned_addr learned my addr 172.21.2.116:0/4050601205 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:36.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc881cf1700 1 -- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc864005050 msgr2=0x7fc87c05c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc881cf1700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc864005050 0x7fc87c05c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc881cf1700 1 -- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc87c0639d0 msgr2=0x7fc87c123ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc881cf1700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc87c0639d0 0x7fc87c123ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc881cf1700 1 -- 172.21.2.116:0/4050601205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c13fd60 con 0x7fc87c13efe0 2021-11-21T21:54:36.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc881cf1700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc87c13efe0 0x7fc87c14d840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fc87c067ce0 tx=0x7fc87c0b07d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:36.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc86f7fe700 1 -- 172.21.2.116:0/4050601205 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc878007170 con 0x7fc87c13efe0 2021-11-21T21:54:36.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc86f7fe700 1 -- 172.21.2.116:0/4050601205 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8780072d0 con 0x7fc87c13efe0 2021-11-21T21:54:36.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c0644c0 con 0x7fc87c13efe0 2021-11-21T21:54:36.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc86f7fe700 1 -- 172.21.2.116:0/4050601205 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc878004920 con 0x7fc87c13efe0 2021-11-21T21:54:36.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c13e8c0 con 0x7fc87c13efe0 2021-11-21T21:54:36.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c13e660 con 0x7fc87c13efe0 2021-11-21T21:54:36.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc86f7fe700 1 -- 172.21.2.116:0/4050601205 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc878004a80 con 0x7fc87c13efe0 2021-11-21T21:54:36.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc86f7fe700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc85c053610 0x7fc85c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc86f7fe700 1 -- 172.21.2.116:0/4050601205 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc87806b480 con 0x7fc87c13efe0 2021-11-21T21:54:36.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc8824f2700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc85c053610 0x7fc85c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:36.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.110+0000 7fc8824f2700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc85c053610 0x7fc85c055ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc870002f70 tx=0x7fc870006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:36.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.114+0000 7fc86f7fe700 1 -- 172.21.2.116:0/4050601205 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8780425e0 con 0x7fc87c13efe0 2021-11-21T21:54:36.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc87c0b1e00 con 0x7fc87c13efe0 2021-11-21T21:54:36.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc86f7fe700 1 -- 172.21.2.116:0/4050601205 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc878045c00 con 0x7fc87c13efe0 2021-11-21T21:54:36.201 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:36.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc85c053610 msgr2=0x7fc85c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc85c053610 0x7fc85c055ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc870002f70 tx=0x7fc870006a00).stop 2021-11-21T21:54:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc87c13efe0 msgr2=0x7fc87c14d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc87c13efe0 0x7fc87c14d840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fc87c067ce0 tx=0x7fc87c0b07d0).stop 2021-11-21T21:54:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 shutdown_connections 2021-11-21T21:54:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc87c0639d0 0x7fc87c123ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc85c053610 0x7fc85c055ad0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc87c13efe0 0x7fc87c14d840 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 --2- 172.21.2.116:0/4050601205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc864005050 0x7fc87c05c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 >> 172.21.2.116:0/4050601205 conn(0x7fc87c0b5730 msgr2=0x7fc87c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:36.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 shutdown_connections 2021-11-21T21:54:36.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.198+0000 7fc884756700 1 -- 172.21.2.116:0/4050601205 wait complete. 2021-11-21T21:54:36.208 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:54:36.209 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:36.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.254+0000 7f0ab73c2700 1 Processor -- start 2021-11-21T21:54:36.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.254+0000 7f0ab73c2700 1 -- start start 2021-11-21T21:54:36.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ab00a86b0 0x7f0ab014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0ab0062410 0x7f0ab01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab00a6e80 con 0x7f0ab00a6aa0 2021-11-21T21:54:36.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab00a8bc0 con 0x7f0ab00a86b0 2021-11-21T21:54:36.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab014a7b0 con 0x7f0ab0062410 2021-11-21T21:54:36.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ab00a86b0 0x7f0ab014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:36.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ab00a86b0 0x7f0ab014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45228/0 (socket says 172.21.2.116:45228) 2021-11-21T21:54:36.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 -- 172.21.2.116:0/3836417960 learned_addr learned my addr 172.21.2.116:0/3836417960 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:36.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 -- 172.21.2.116:0/3836417960 >> v2:172.21.2.117:3300/0 conn(0x7f0ab0062410 msgr2=0x7f0ab01443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:36.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 --2- 172.21.2.116:0/3836417960 >> v2:172.21.2.117:3300/0 conn(0x7f0ab0062410 0x7f0ab01443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 -- 172.21.2.116:0/3836417960 >> v1:172.21.2.117:6789/0 conn(0x7f0ab00a6aa0 legacy=0x7f0ab01470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:36.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 -- 172.21.2.116:0/3836417960 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab014aa10 con 0x7f0ab00a86b0 2021-11-21T21:54:36.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab495d700 1 --2- 172.21.2.116:0/3836417960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab00a86b0 0x7f0ab014b150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f0ab0061ab0 tx=0x7f0aac006eb0).ready entity=mon.0 client_cookie=35d115398c501bd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:36.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0aa3fff700 1 -- 172.21.2.116:0/3836417960 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aac00d5c0 con 0x7f0ab00a86b0 2021-11-21T21:54:36.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0aa3fff700 1 -- 172.21.2.116:0/3836417960 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac000e90 con 0x7f0ab00a86b0 2021-11-21T21:54:36.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0aa3fff700 1 -- 172.21.2.116:0/3836417960 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aac00d910 con 0x7f0ab00a86b0 2021-11-21T21:54:36.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- 172.21.2.116:0/3836417960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab00a86b0 msgr2=0x7f0ab014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/3836417960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab00a86b0 0x7f0ab014b150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f0ab0061ab0 tx=0x7f0aac006eb0).stop 2021-11-21T21:54:36.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- 172.21.2.116:0/3836417960 shutdown_connections 2021-11-21T21:54:36.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/3836417960 >> v2:172.21.2.117:3300/0 conn(0x7f0ab0062410 0x7f0ab01443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/3836417960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab00a86b0 0x7f0ab014b150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- 172.21.2.116:0/3836417960 >> 172.21.2.116:0/3836417960 conn(0x7f0ab005bd20 msgr2=0x7f0ab005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:36.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- 172.21.2.116:0/3836417960 shutdown_connections 2021-11-21T21:54:36.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- 172.21.2.116:0/3836417960 wait complete. 2021-11-21T21:54:36.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 Processor -- start 2021-11-21T21:54:36.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- start start 2021-11-21T21:54:36.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab0062410 0x7f0ab0144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ab00a6aa0 0x7f0ab00aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ab00a86b0 0x7f0ab00ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00639d0 con 0x7f0ab0062410 2021-11-21T21:54:36.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab0063c70 con 0x7f0ab00a86b0 2021-11-21T21:54:36.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab0063f10 con 0x7f0ab00a6aa0 2021-11-21T21:54:36.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab0062410 0x7f0ab0144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:36.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab0062410 0x7f0ab0144500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45234/0 (socket says 172.21.2.116:45234) 2021-11-21T21:54:36.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 -- 172.21.2.116:0/348187813 learned_addr learned my addr 172.21.2.116:0/348187813 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:36.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 -- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ab00a6aa0 msgr2=0x7f0ab00aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ab00a6aa0 0x7f0ab00aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 -- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ab00a86b0 msgr2=0x7f0ab00ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ab00a86b0 0x7f0ab00ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 -- 172.21.2.116:0/348187813 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac00b1e0 con 0x7f0ab0062410 2021-11-21T21:54:36.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab515e700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab0062410 0x7f0ab0144500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f0aa4002fb0 tx=0x7f0aa4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:36.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0aa27fc700 1 -- 172.21.2.116:0/348187813 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aa4013460 con 0x7f0ab0062410 2021-11-21T21:54:36.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0aa27fc700 1 -- 172.21.2.116:0/348187813 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa40135c0 con 0x7f0ab0062410 2021-11-21T21:54:36.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab00660e0 con 0x7f0ab0062410 2021-11-21T21:54:36.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0aa27fc700 1 -- 172.21.2.116:0/348187813 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aa4013920 con 0x7f0ab0062410 2021-11-21T21:54:36.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.258+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab00aa2c0 con 0x7f0ab0062410 2021-11-21T21:54:36.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.262+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab0063830 con 0x7f0ab0062410 2021-11-21T21:54:36.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.262+0000 7f0aa27fc700 1 -- 172.21.2.116:0/348187813 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0aa400f070 con 0x7f0ab0062410 2021-11-21T21:54:36.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.262+0000 7f0aa27fc700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a90053610 0x7f0a90055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:36.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.262+0000 7f0aa27fc700 1 -- 172.21.2.116:0/348187813 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0aa406b1f0 con 0x7f0ab0062410 2021-11-21T21:54:36.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.262+0000 7f0ab495d700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a90053610 0x7f0a90055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:36.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.262+0000 7f0ab495d700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a90053610 0x7f0a90055ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0ab0061ab0 tx=0x7f0aac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:36.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.262+0000 7f0aa27fc700 1 -- 172.21.2.116:0/348187813 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ab0063830 con 0x7f0ab0062410 2021-11-21T21:54:36.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab0063830 con 0x7f0ab0062410 2021-11-21T21:54:36.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0aa27fc700 1 -- 172.21.2.116:0/348187813 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f0ab0063830 con 0x7f0ab0062410 2021-11-21T21:54:36.377 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:36.377 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/862809391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":862809391},{"type":"v1","addr":"172.21.2.117:6835","nonce":862809391}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2422183040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2422183040},{"type":"v1","addr":"172.21.2.116:6833","nonce":2422183040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:24.904554+0000","modified":"2021-11-21T21:54:25.909207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/2778157441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2778157441},{"type":"v1","addr":"172.21.2.116:6835","nonce":2778157441}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-11-21T21:54:36.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a90053610 msgr2=0x7f0a90055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a90053610 0x7f0a90055ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0ab0061ab0 tx=0x7f0aac015000).stop 2021-11-21T21:54:36.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab0062410 msgr2=0x7f0ab0144500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:36.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab0062410 0x7f0ab0144500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f0aa4002fb0 tx=0x7f0aa4006df0).stop 2021-11-21T21:54:36.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 shutdown_connections 2021-11-21T21:54:36.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ab00a86b0 0x7f0ab00ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ab00a6aa0 0x7f0ab00aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:36.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a90053610 0x7f0a90055ad0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 --2- 172.21.2.116:0/348187813 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ab0062410 0x7f0ab0144500 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:36.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 >> 172.21.2.116:0/348187813 conn(0x7f0ab005bd20 msgr2=0x7f0ab0149140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:36.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 shutdown_connections 2021-11-21T21:54:36.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:36.374+0000 7f0ab73c2700 1 -- 172.21.2.116:0/348187813 wait complete. 2021-11-21T21:54:36.381 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 5 2021-11-21T21:54:36.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:54:24.904554+0000', 'modified': '2021-11-21T21:54:25.909207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.116:6835/2778157441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2778157441}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2778157441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:54:36.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:54:36.688 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:54:36.753 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":4239,"want_state":"up:active","state":"up:active","rank_uptime":11.842276132,"mdsmap_epoch":5,"osdmap_epoch":22,"osdmap_epoch_barrier":20,"uptime":13.992184246000001} 2021-11-21T21:54:36.753 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 4239, "mdsmap_epoch": 5, "osdmap_epoch": 22, "osdmap_epoch_barrier": 20, "rank_uptime": 11.842276132, "state": "up:active", "uptime": 13.992184246, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:54:36.754 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:54:36.754 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T21:54:36.754 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T21:54:36.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:36.755 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:36.766 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T21:54:36.778 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:54:36.779 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:54:36.785 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:54:36.786 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:54:36.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:36.787 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T21:54:36.832 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:54:36.832 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:36.835 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T21:54:36.845 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:54:36.846 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:54:36.885 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T21:54:36.884+0000 7f2fbe349080 -1 init, newargv = 0x55f947711950 newargc=9 2021-11-21T21:54:36.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[13978]: starting ceph client 2021-11-21T21:54:36.891 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:54:36.892 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:54:36.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:36.893 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T21:54:36.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[13978]: starting fuse 2021-11-21T21:54:36.898 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T21:54:36.898 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:54:36.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:36.899 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T21:54:36.899 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T21:54:36.900 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T21:54:36.900 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T21:54:36.900 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T21:54:36.901 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T21:54:36.901 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T21:54:36.901 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T21:54:36.902 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T21:54:36.902 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T21:54:36.902 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T21:54:36.903 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T21:54:36.903 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T21:54:36.903 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T21:54:36.904 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T21:54:36.904 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T21:54:36.904 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:54:36.905 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:54:36.905 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T21:54:36.905 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:54:36.906 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T21:54:36.906 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T21:54:36.906 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T21:54:36.962 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.13978.asok 2021-11-21T21:54:36.965 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.13978.asok 2021-11-21T21:54:36.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:36.965 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.13978.asok status 2021-11-21T21:54:37.017 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T21:54:37.017 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T21:54:37.017 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:54:37.018 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:54:37.018 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T21:54:37.018 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T21:54:37.018 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T21:54:37.019 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "13978", 2021-11-21T21:54:37.019 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T21:54:37.019 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T21:54:37.019 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T21:54:37.020 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T21:54:37.020 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 4665, 2021-11-21T21:54:37.020 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T21:54:37.020 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T21:54:37.020 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T21:54:37.021 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 4665 2021-11-21T21:54:37.021 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T21:54:37.021 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T21:54:37.022 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T21:54:37.022 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T21:54:37.022 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 678101452 2021-11-21T21:54:37.022 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T21:54:37.023 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T21:54:37.023 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T21:54:37.023 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T21:54:37.023 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T21:54:37.024 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 678101452 2021-11-21T21:54:37.025 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T21:54:37.025 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.4665 172.21.2.136:0/678101452", 2021-11-21T21:54:37.025 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/678101452", 2021-11-21T21:54:37.025 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T21:54:37.025 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 5, 2021-11-21T21:54:37.026 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 22, 2021-11-21T21:54:37.026 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:54:37.026 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T21:54:37.026 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T21:54:37.027 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T21:54:37.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:37.029 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:37.029 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:37.040 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:54:37.041 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:54:37.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:37.042 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:37.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:37.102 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:37.103 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:37.111 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T21:54:37.112 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T21:54:37.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:37.113 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:37.186 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2021-11-21T21:54:37.203 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:37.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d91af700 1 Processor -- start 2021-11-21T21:54:37.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d91af700 1 -- start start 2021-11-21T21:54:37.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d91af700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44d40ad780 0x7f44d405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d91af700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40af7e0 0x7f44d413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d91af700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d4143f70 con 0x7f44d40affc0 2021-11-21T21:54:37.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d91af700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d40adb60 con 0x7f44d40ad780 2021-11-21T21:54:37.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d91af700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d4143160 con 0x7f44d40af7e0 2021-11-21T21:54:37.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d2d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44d40ad780 0x7f44d405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d2d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44d40ad780 0x7f44d405f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45242/0 (socket says 172.21.2.116:45242) 2021-11-21T21:54:37.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d2d9d700 1 -- 172.21.2.116:0/2978109004 learned_addr learned my addr 172.21.2.116:0/2978109004 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d259c700 1 --1- 172.21.2.116:0/2978109004 >> v1:172.21.2.116:6789/0 conn(0x7f44d40affc0 0x7f44d4143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57200/0 (socket says 172.21.2.116:57200) 2021-11-21T21:54:37.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d2d9d700 1 -- 172.21.2.116:0/2978109004 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40af7e0 msgr2=0x7f44d413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:37.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d2d9d700 1 --2- 172.21.2.116:0/2978109004 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40af7e0 0x7f44d413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d2d9d700 1 -- 172.21.2.116:0/2978109004 >> v1:172.21.2.116:6789/0 conn(0x7f44d40affc0 legacy=0x7f44d4143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d1d9b700 1 -- 172.21.2.116:0/2978109004 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871513039 0 0) 0x7f44d4143f70 con 0x7f44d40affc0 2021-11-21T21:54:37.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.254+0000 7f44d2d9d700 1 -- 172.21.2.116:0/2978109004 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40b0ce0 con 0x7f44d40ad780 2021-11-21T21:54:37.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d2d9d700 1 --2- 172.21.2.116:0/2978109004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40ad780 0x7f44d405f730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f44c4002f70 tx=0x7f44c4006eb0).ready entity=mon.0 client_cookie=e82da01b76221abe server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d1d9b700 1 -- 172.21.2.116:0/2978109004 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c400f9c0 con 0x7f44d40ad780 2021-11-21T21:54:37.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d1d9b700 1 -- 172.21.2.116:0/2978109004 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c4000e90 con 0x7f44d40ad780 2021-11-21T21:54:37.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d1d9b700 1 -- 172.21.2.116:0/2978109004 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c400fd50 con 0x7f44d40ad780 2021-11-21T21:54:37.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- 172.21.2.116:0/2978109004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40ad780 msgr2=0x7f44d405f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 --2- 172.21.2.116:0/2978109004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40ad780 0x7f44d405f730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f44c4002f70 tx=0x7f44c4006eb0).stop 2021-11-21T21:54:37.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- 172.21.2.116:0/2978109004 shutdown_connections 2021-11-21T21:54:37.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 --2- 172.21.2.116:0/2978109004 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40af7e0 0x7f44d413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 --2- 172.21.2.116:0/2978109004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40ad780 0x7f44d405f730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- 172.21.2.116:0/2978109004 >> 172.21.2.116:0/2978109004 conn(0x7f44d40b6000 msgr2=0x7f44d40b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- 172.21.2.116:0/2978109004 shutdown_connections 2021-11-21T21:54:37.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- 172.21.2.116:0/2978109004 wait complete. 2021-11-21T21:54:37.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 Processor -- start 2021-11-21T21:54:37.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- start start 2021-11-21T21:54:37.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40ad780 0x7f44d405c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40af7e0 0x7f44d40aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44d40affc0 0x7f44d40b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40b3f60 con 0x7f44d40af7e0 2021-11-21T21:54:37.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40b41c0 con 0x7f44d40affc0 2021-11-21T21:54:37.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40b4460 con 0x7f44d40ad780 2021-11-21T21:54:37.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40af7e0 0x7f44d40aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40af7e0 0x7f44d40aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45248/0 (socket says 172.21.2.116:45248) 2021-11-21T21:54:37.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 -- 172.21.2.116:0/3826554425 learned_addr learned my addr 172.21.2.116:0/3826554425 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 -- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40ad780 msgr2=0x7f44d405c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40ad780 0x7f44d405c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 -- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44d40affc0 msgr2=0x7f44d40b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44d40affc0 0x7f44d40b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 -- 172.21.2.116:0/3826554425 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c400b1e0 con 0x7f44d40af7e0 2021-11-21T21:54:37.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d259c700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40af7e0 0x7f44d40aa060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f44bc000bc0 tx=0x7f44bc007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44cbfff700 1 -- 172.21.2.116:0/3826554425 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44bc00ce20 con 0x7f44d40af7e0 2021-11-21T21:54:37.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44cbfff700 1 -- 172.21.2.116:0/3826554425 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc0072e0 con 0x7f44d40af7e0 2021-11-21T21:54:37.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d4152440 con 0x7f44d40af7e0 2021-11-21T21:54:37.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44cbfff700 1 -- 172.21.2.116:0/3826554425 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44bc005740 con 0x7f44d40af7e0 2021-11-21T21:54:37.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.258+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d40b5420 con 0x7f44d40af7e0 2021-11-21T21:54:37.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.262+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d40b4620 con 0x7f44d40af7e0 2021-11-21T21:54:37.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.262+0000 7f44cbfff700 1 -- 172.21.2.116:0/3826554425 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f44bc010070 con 0x7f44d40af7e0 2021-11-21T21:54:37.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.262+0000 7f44cbfff700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44b80536a0 0x7f44b8055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.262+0000 7f44cbfff700 1 -- 172.21.2.116:0/3826554425 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f44bc06b0a0 con 0x7f44d40af7e0 2021-11-21T21:54:37.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.262+0000 7f44d2d9d700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44b80536a0 0x7f44b8055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.262+0000 7f44d2d9d700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44b80536a0 0x7f44b8055b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f44c4002f70 tx=0x7f44c4017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.262+0000 7f44cbfff700 1 -- 172.21.2.116:0/3826554425 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44bc042200 con 0x7f44d40af7e0 2021-11-21T21:54:37.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44d40b0a40 con 0x7f44d40af7e0 2021-11-21T21:54:37.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44cbfff700 1 -- 172.21.2.116:0/3826554425 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f44bc045820 con 0x7f44d40af7e0 2021-11-21T21:54:37.350 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:37.350 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:37.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44b80536a0 msgr2=0x7f44b8055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44b80536a0 0x7f44b8055b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f44c4002f70 tx=0x7f44c4017000).stop 2021-11-21T21:54:37.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40af7e0 msgr2=0x7f44d40aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40af7e0 0x7f44d40aa060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f44bc000bc0 tx=0x7f44bc007a60).stop 2021-11-21T21:54:37.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 shutdown_connections 2021-11-21T21:54:37.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44d40affc0 0x7f44d40b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44b80536a0 0x7f44b8055b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44d40af7e0 0x7f44d40aa060 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 --2- 172.21.2.116:0/3826554425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44d40ad780 0x7f44d405c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 >> 172.21.2.116:0/3826554425 conn(0x7f44d40b6000 msgr2=0x7f44d4141000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 shutdown_connections 2021-11-21T21:54:37.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.346+0000 7f44d91af700 1 -- 172.21.2.116:0/3826554425 wait complete. 2021-11-21T21:54:37.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:37.358 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:37.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b570ea700 1 Processor -- start 2021-11-21T21:54:37.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b570ea700 1 -- start start 2021-11-21T21:54:37.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b570ea700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1b500abce0 0x7f1b501478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b570ea700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1b500ac600 0x7f1b500aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b570ea700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b50147e80 con 0x7f1b50144810 2021-11-21T21:54:37.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b570ea700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b500b1490 con 0x7f1b500abce0 2021-11-21T21:54:37.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b570ea700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b500b1fd0 con 0x7f1b500ac600 2021-11-21T21:54:37.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b54e86700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f1b50144810 0x7f1b500b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57212/0 (socket says 172.21.2.116:57212) 2021-11-21T21:54:37.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b54e86700 1 -- 172.21.2.116:0/3753225730 learned_addr learned my addr 172.21.2.116:0/3753225730 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:37.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4ffff700 1 --2- 172.21.2.116:0/3753225730 >> v2:172.21.2.116:3300/0 conn(0x7f1b500abce0 0x7f1b501478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4f7fe700 1 -- 172.21.2.116:0/3753225730 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223641071 0 0) 0x7f1b50147e80 con 0x7f1b50144810 2021-11-21T21:54:37.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4f7fe700 1 -- 172.21.2.116:0/3753225730 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b30003260 con 0x7f1b50144810 2021-11-21T21:54:37.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4ffff700 1 -- 172.21.2.116:0/3753225730 >> v2:172.21.2.117:3300/0 conn(0x7f1b500ac600 msgr2=0x7f1b500aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:37.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4ffff700 1 --2- 172.21.2.116:0/3753225730 >> v2:172.21.2.117:3300/0 conn(0x7f1b500ac600 0x7f1b500aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4ffff700 1 -- 172.21.2.116:0/3753225730 >> v1:172.21.2.116:6789/0 conn(0x7f1b50144810 legacy=0x7f1b500b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4ffff700 1 -- 172.21.2.116:0/3753225730 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b500af5a0 con 0x7f1b500abce0 2021-11-21T21:54:37.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4ffff700 1 --2- 172.21.2.116:0/3753225730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b500abce0 0x7f1b501478c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f1b40002f70 tx=0x7f1b40006df0).ready entity=mon.0 client_cookie=3635e1a0be095672 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4f7fe700 1 -- 172.21.2.116:0/3753225730 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4000d570 con 0x7f1b500abce0 2021-11-21T21:54:37.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4f7fe700 1 -- 172.21.2.116:0/3753225730 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b40000e90 con 0x7f1b500abce0 2021-11-21T21:54:37.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.406+0000 7f1b4f7fe700 1 -- 172.21.2.116:0/3753225730 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4000d8c0 con 0x7f1b500abce0 2021-11-21T21:54:37.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3753225730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b500abce0 msgr2=0x7f1b501478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3753225730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b500abce0 0x7f1b501478c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f1b40002f70 tx=0x7f1b40006df0).stop 2021-11-21T21:54:37.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3753225730 shutdown_connections 2021-11-21T21:54:37.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3753225730 >> v2:172.21.2.117:3300/0 conn(0x7f1b500ac600 0x7f1b500aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3753225730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b500abce0 0x7f1b501478c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3753225730 >> 172.21.2.116:0/3753225730 conn(0x7f1b5005bd20 msgr2=0x7f1b5005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3753225730 shutdown_connections 2021-11-21T21:54:37.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3753225730 wait complete. 2021-11-21T21:54:37.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 Processor -- start 2021-11-21T21:54:37.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- start start 2021-11-21T21:54:37.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1b500abce0 0x7f1b5013f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1b500ac600 0x7f1b500629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b50144810 0x7f1b50156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500b4cc0 con 0x7f1b50144810 2021-11-21T21:54:37.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b50062eb0 con 0x7f1b500ac600 2021-11-21T21:54:37.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500630d0 con 0x7f1b500abce0 2021-11-21T21:54:37.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b50144810 0x7f1b50156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b50144810 0x7f1b50156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45264/0 (socket says 172.21.2.116:45264) 2021-11-21T21:54:37.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 -- 172.21.2.116:0/3550346582 learned_addr learned my addr 172.21.2.116:0/3550346582 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 -- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1b500abce0 msgr2=0x7f1b5013f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1b500abce0 0x7f1b5013f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 -- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1b500ac600 msgr2=0x7f1b500629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1b500ac600 0x7f1b500629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 -- 172.21.2.116:0/3550346582 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4000b180 con 0x7f1b50144810 2021-11-21T21:54:37.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b55687700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b50144810 0x7f1b50156ba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f1b501472e0 tx=0x7f1b48009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b4dffb700 1 -- 172.21.2.116:0/3550346582 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4800d580 con 0x7f1b50144810 2021-11-21T21:54:37.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b4dffb700 1 -- 172.21.2.116:0/3550346582 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4800d6e0 con 0x7f1b50144810 2021-11-21T21:54:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b50063370 con 0x7f1b50144810 2021-11-21T21:54:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b50156700 con 0x7f1b50144810 2021-11-21T21:54:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b4dffb700 1 -- 172.21.2.116:0/3550346582 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4800da40 con 0x7f1b50144810 2021-11-21T21:54:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b28003410 con 0x7f1b50144810 2021-11-21T21:54:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b4dffb700 1 -- 172.21.2.116:0/3550346582 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1b4800dba0 con 0x7f1b50144810 2021-11-21T21:54:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b4dffb700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1b34053580 0x7f1b34055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.410+0000 7f1b4dffb700 1 -- 172.21.2.116:0/3550346582 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1b4806bcd0 con 0x7f1b50144810 2021-11-21T21:54:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.414+0000 7f1b54e86700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1b34053580 0x7f1b34055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.414+0000 7f1b54e86700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1b34053580 0x7f1b34055a40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f1b38001110 tx=0x7f1b380076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.414+0000 7f1b4dffb700 1 -- 172.21.2.116:0/3550346582 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b48042dd0 con 0x7f1b50144810 2021-11-21T21:54:37.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1b28003c40 con 0x7f1b50144810 2021-11-21T21:54:37.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b4dffb700 1 -- 172.21.2.116:0/3550346582 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1b480463f0 con 0x7f1b50144810 2021-11-21T21:54:37.501 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:37.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1b34053580 msgr2=0x7f1b34055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1b34053580 0x7f1b34055a40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f1b38001110 tx=0x7f1b380076c0).stop 2021-11-21T21:54:37.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b50144810 msgr2=0x7f1b50156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b50144810 0x7f1b50156ba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f1b501472e0 tx=0x7f1b48009220).stop 2021-11-21T21:54:37.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 shutdown_connections 2021-11-21T21:54:37.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1b34053580 0x7f1b34055a40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1b50144810 0x7f1b50156ba0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1b500ac600 0x7f1b500629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 --2- 172.21.2.116:0/3550346582 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1b500abce0 0x7f1b5013f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 >> 172.21.2.116:0/3550346582 conn(0x7f1b5005bd20 msgr2=0x7f1b500aca50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 shutdown_connections 2021-11-21T21:54:37.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.498+0000 7f1b570ea700 1 -- 172.21.2.116:0/3550346582 wait complete. 2021-11-21T21:54:37.509 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:37.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 Processor -- start 2021-11-21T21:54:37.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- start start 2021-11-21T21:54:37.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8340144810 0x7f83401478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 0x7f83400b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8340147e80 con 0x7f83400b1db0 2021-11-21T21:54:37.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83400b1490 con 0x7f8340144810 2021-11-21T21:54:37.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400af060 con 0x7f83400abce0 2021-11-21T21:54:37.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8340144810 0x7f83401478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8340144810 0x7f83401478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45270/0 (socket says 172.21.2.116:45270) 2021-11-21T21:54:37.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833f59e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f83400b1db0 0x7f83400ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57228/0 (socket says 172.21.2.116:57228) 2021-11-21T21:54:37.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 -- 172.21.2.116:0/1091831766 learned_addr learned my addr 172.21.2.116:0/1091831766 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 -- 172.21.2.116:0/1091831766 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 msgr2=0x7f83400b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:37.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 --2- 172.21.2.116:0/1091831766 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 0x7f83400b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 -- 172.21.2.116:0/1091831766 >> v1:172.21.2.116:6789/0 conn(0x7f83400b1db0 legacy=0x7f83400ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 -- 172.21.2.116:0/1091831766 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400af5a0 con 0x7f8340144810 2021-11-21T21:54:37.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833ed9d700 1 --2- 172.21.2.116:0/1091831766 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340144810 0x7f83401478c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8328002f70 tx=0x7f8328006eb0).ready entity=mon.0 client_cookie=4ff38f9bbee56e0e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833dd9b700 1 -- 172.21.2.116:0/1091831766 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f832800d5c0 con 0x7f8340144810 2021-11-21T21:54:37.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833dd9b700 1 -- 172.21.2.116:0/1091831766 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328000e90 con 0x7f8340144810 2021-11-21T21:54:37.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f833dd9b700 1 -- 172.21.2.116:0/1091831766 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f832800d950 con 0x7f8340144810 2021-11-21T21:54:37.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- 172.21.2.116:0/1091831766 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340144810 msgr2=0x7f83401478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 --2- 172.21.2.116:0/1091831766 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340144810 0x7f83401478c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8328002f70 tx=0x7f8328006eb0).stop 2021-11-21T21:54:37.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- 172.21.2.116:0/1091831766 shutdown_connections 2021-11-21T21:54:37.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 --2- 172.21.2.116:0/1091831766 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 0x7f83400b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 --2- 172.21.2.116:0/1091831766 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340144810 0x7f83401478c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- 172.21.2.116:0/1091831766 >> 172.21.2.116:0/1091831766 conn(0x7f834005bd20 msgr2=0x7f834005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- 172.21.2.116:0/1091831766 shutdown_connections 2021-11-21T21:54:37.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.558+0000 7f8345195700 1 -- 172.21.2.116:0/1091831766 wait complete. 2021-11-21T21:54:37.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 Processor -- start 2021-11-21T21:54:37.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 -- start start 2021-11-21T21:54:37.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 0x7f8340143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340144810 0x7f834013a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83400b2f30 0x7f834013a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8340143930 con 0x7f83400b2f30 2021-11-21T21:54:37.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834015a970 con 0x7f8340144810 2021-11-21T21:54:37.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8340145fe0 con 0x7f83400abce0 2021-11-21T21:54:37.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83400b2f30 0x7f834013a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83400b2f30 0x7f834013a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45278/0 (socket says 172.21.2.116:45278) 2021-11-21T21:54:37.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 -- 172.21.2.116:0/138268228 learned_addr learned my addr 172.21.2.116:0/138268228 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 -- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 msgr2=0x7f8340143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 0x7f8340143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 -- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340144810 msgr2=0x7f834013a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340144810 0x7f834013a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 -- 172.21.2.116:0/138268228 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832800b1e0 con 0x7f83400b2f30 2021-11-21T21:54:37.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833f59e700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83400b2f30 0x7f834013a680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f83400b2a30 tx=0x7f8338007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8337fff700 1 -- 172.21.2.116:0/138268228 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8338004ab0 con 0x7f83400b2f30 2021-11-21T21:54:37.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8337fff700 1 -- 172.21.2.116:0/138268228 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8338004c10 con 0x7f83400b2f30 2021-11-21T21:54:37.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8340146280 con 0x7f83400b2f30 2021-11-21T21:54:37.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8337fff700 1 -- 172.21.2.116:0/138268228 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8338007170 con 0x7f83400b2f30 2021-11-21T21:54:37.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400b2ab0 con 0x7f83400b2f30 2021-11-21T21:54:37.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834013ad10 con 0x7f83400b2f30 2021-11-21T21:54:37.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8337fff700 1 -- 172.21.2.116:0/138268228 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f83380072d0 con 0x7f83400b2f30 2021-11-21T21:54:37.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8337fff700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8324053640 0x7f8324055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f8337fff700 1 -- 172.21.2.116:0/138268228 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f833806b650 con 0x7f83400b2f30 2021-11-21T21:54:37.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.562+0000 7f833ed9d700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8324053640 0x7f8324055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.566+0000 7f833ed9d700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8324053640 0x7f8324055b00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8328002f70 tx=0x7f8328015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.566+0000 7f8337fff700 1 -- 172.21.2.116:0/138268228 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83380426a0 con 0x7f83400b2f30 2021-11-21T21:54:37.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.646+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f83400af300 con 0x7f83400b2f30 2021-11-21T21:54:37.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.646+0000 7f8337fff700 1 -- 172.21.2.116:0/138268228 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8338045cc0 con 0x7f83400b2f30 2021-11-21T21:54:37.653 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:37.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8324053640 msgr2=0x7f8324055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8324053640 0x7f8324055b00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8328002f70 tx=0x7f8328015000).stop 2021-11-21T21:54:37.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83400b2f30 msgr2=0x7f834013a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83400b2f30 0x7f834013a680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f83400b2a30 tx=0x7f8338007f00).stop 2021-11-21T21:54:37.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 shutdown_connections 2021-11-21T21:54:37.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8324053640 0x7f8324055b00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83400b2f30 0x7f834013a680 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340144810 0x7f834013a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 --2- 172.21.2.116:0/138268228 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83400abce0 0x7f8340143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 >> 172.21.2.116:0/138268228 conn(0x7f834005bd20 msgr2=0x7f834013ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 shutdown_connections 2021-11-21T21:54:37.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.650+0000 7f8345195700 1 -- 172.21.2.116:0/138268228 wait complete. 2021-11-21T21:54:37.661 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:37.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfbaed700 1 Processor -- start 2021-11-21T21:54:37.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfbaed700 1 -- start start 2021-11-21T21:54:37.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfbaed700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf4144810 0x7febf41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfbaed700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7febf40ac600 0x7febf40aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfbaed700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf4147e80 con 0x7febf40abce0 2021-11-21T21:54:37.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfbaed700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf40b1490 con 0x7febf40ac600 2021-11-21T21:54:37.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfbaed700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf40b1fd0 con 0x7febf4144810 2021-11-21T21:54:37.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7febf40ac600 0x7febf40aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7febf40ac600 0x7febf40aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45288/0 (socket says 172.21.2.116:45288) 2021-11-21T21:54:37.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 -- 172.21.2.116:0/395369290 learned_addr learned my addr 172.21.2.116:0/395369290 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 -- 172.21.2.116:0/395369290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf4144810 msgr2=0x7febf41478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:37.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 --2- 172.21.2.116:0/395369290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf4144810 0x7febf41478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 -- 172.21.2.116:0/395369290 >> v1:172.21.2.117:6789/0 conn(0x7febf40abce0 legacy=0x7febf40b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:37.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 -- 172.21.2.116:0/395369290 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf40af5a0 con 0x7febf40ac600 2021-11-21T21:54:37.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febfa08a700 1 --2- 172.21.2.116:0/395369290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40ac600 0x7febf40aca90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7febf40b2a30 tx=0x7febec009290).ready entity=mon.0 client_cookie=de9ec05b9655a03 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febf8887700 1 -- 172.21.2.116:0/395369290 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febec00b860 con 0x7febf40ac600 2021-11-21T21:54:37.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febf8887700 1 -- 172.21.2.116:0/395369290 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec00b9c0 con 0x7febf40ac600 2021-11-21T21:54:37.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.710+0000 7febf8887700 1 -- 172.21.2.116:0/395369290 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febec00bb70 con 0x7febf40ac600 2021-11-21T21:54:37.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/395369290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40ac600 msgr2=0x7febf40aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 --2- 172.21.2.116:0/395369290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40ac600 0x7febf40aca90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7febf40b2a30 tx=0x7febec009290).stop 2021-11-21T21:54:37.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/395369290 shutdown_connections 2021-11-21T21:54:37.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 --2- 172.21.2.116:0/395369290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40ac600 0x7febf40aca90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 --2- 172.21.2.116:0/395369290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf4144810 0x7febf41478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/395369290 >> 172.21.2.116:0/395369290 conn(0x7febf405bd20 msgr2=0x7febf405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/395369290 shutdown_connections 2021-11-21T21:54:37.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/395369290 wait complete. 2021-11-21T21:54:37.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 Processor -- start 2021-11-21T21:54:37.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- start start 2021-11-21T21:54:37.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40abce0 0x7febf4145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf40ac600 0x7febf415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7febf4144810 0x7febf4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf415ab00 con 0x7febf40abce0 2021-11-21T21:54:37.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40b30d0 con 0x7febf4144810 2021-11-21T21:54:37.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf40b3300 con 0x7febf40ac600 2021-11-21T21:54:37.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40abce0 0x7febf4145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40abce0 0x7febf4145b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45292/0 (socket says 172.21.2.116:45292) 2021-11-21T21:54:37.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 -- 172.21.2.116:0/3150861623 learned_addr learned my addr 172.21.2.116:0/3150861623 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 -- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf40ac600 msgr2=0x7febf415a610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:37.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf40ac600 0x7febf415a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 -- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7febf4144810 msgr2=0x7febf4156920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:37.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7febf4144810 0x7febf4156920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 -- 172.21.2.116:0/3150861623 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec009530 con 0x7febf40abce0 2021-11-21T21:54:37.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febf9889700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40abce0 0x7febf4145b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7febf415ac40 tx=0x7febf0006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febe2ffd700 1 -- 172.21.2.116:0/3150861623 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febf0012410 con 0x7febf40abce0 2021-11-21T21:54:37.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febe2ffd700 1 -- 172.21.2.116:0/3150861623 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf0012570 con 0x7febf40abce0 2021-11-21T21:54:37.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febe2ffd700 1 -- 172.21.2.116:0/3150861623 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febf00128d0 con 0x7febf40abce0 2021-11-21T21:54:37.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf40b35a0 con 0x7febf40abce0 2021-11-21T21:54:37.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf40b3ae0 con 0x7febf40abce0 2021-11-21T21:54:37.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf40af060 con 0x7febf40abce0 2021-11-21T21:54:37.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febe2ffd700 1 -- 172.21.2.116:0/3150861623 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7febf000f030 con 0x7febf40abce0 2021-11-21T21:54:37.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febe2ffd700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7febd4053520 0x7febd40559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.714+0000 7febe2ffd700 1 -- 172.21.2.116:0/3150861623 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7febf006b490 con 0x7febf40abce0 2021-11-21T21:54:37.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.718+0000 7febf9088700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7febd4053520 0x7febd40559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.718+0000 7febf9088700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7febd4053520 0x7febd40559e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7febe4002fb0 tx=0x7febe4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.718+0000 7febe2ffd700 1 -- 172.21.2.116:0/3150861623 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7febf00425f0 con 0x7febf40abce0 2021-11-21T21:54:37.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7febf40ad630 con 0x7febf40abce0 2021-11-21T21:54:37.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febe2ffd700 1 -- 172.21.2.116:0/3150861623 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7febf0045c10 con 0x7febf40abce0 2021-11-21T21:54:37.805 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:37.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7febd4053520 msgr2=0x7febd40559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7febd4053520 0x7febd40559e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7febe4002fb0 tx=0x7febe4006930).stop 2021-11-21T21:54:37.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40abce0 msgr2=0x7febf4145b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40abce0 0x7febf4145b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7febf415ac40 tx=0x7febf0006d70).stop 2021-11-21T21:54:37.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 shutdown_connections 2021-11-21T21:54:37.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7febf4144810 0x7febf4156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7febf40ac600 0x7febf415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7febd4053520 0x7febd40559e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 --2- 172.21.2.116:0/3150861623 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7febf40abce0 0x7febf4145b40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 >> 172.21.2.116:0/3150861623 conn(0x7febf405bd20 msgr2=0x7febf405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 shutdown_connections 2021-11-21T21:54:37.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.802+0000 7febfbaed700 1 -- 172.21.2.116:0/3150861623 wait complete. 2021-11-21T21:54:37.813 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:37.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f040b990700 1 Processor -- start 2021-11-21T21:54:37.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f040b990700 1 -- start start 2021-11-21T21:54:37.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f040b990700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f04040abce0 0x7f04041478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f040b990700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f04040ac600 0x7f04040aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f040b990700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0404147e80 con 0x7f0404144810 2021-11-21T21:54:37.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f040b990700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040b1490 con 0x7f04040ac600 2021-11-21T21:54:37.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f040b990700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040b1fd0 con 0x7f04040abce0 2021-11-21T21:54:37.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f04040ac600 0x7f04040aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f04040ac600 0x7f04040aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45298/0 (socket says 172.21.2.116:45298) 2021-11-21T21:54:37.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 -- 172.21.2.116:0/859170486 learned_addr learned my addr 172.21.2.116:0/859170486 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 -- 172.21.2.116:0/859170486 >> v2:172.21.2.117:3300/0 conn(0x7f04040abce0 msgr2=0x7f04041478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:37.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 --2- 172.21.2.116:0/859170486 >> v2:172.21.2.117:3300/0 conn(0x7f04040abce0 0x7f04041478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 -- 172.21.2.116:0/859170486 >> v1:172.21.2.117:6789/0 conn(0x7f0404144810 legacy=0x7f04040b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:37.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 -- 172.21.2.116:0/859170486 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04040af5a0 con 0x7f04040ac600 2021-11-21T21:54:37.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.862+0000 7f0409f2d700 1 --2- 172.21.2.116:0/859170486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f04040aca50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f0404062500 tx=0x7f03fc009290).ready entity=mon.0 client_cookie=a97672b898f6197 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f03f3fff700 1 -- 172.21.2.116:0/859170486 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03fc00b830 con 0x7f04040ac600 2021-11-21T21:54:37.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f03f3fff700 1 -- 172.21.2.116:0/859170486 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc00b990 con 0x7f04040ac600 2021-11-21T21:54:37.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f03f3fff700 1 -- 172.21.2.116:0/859170486 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03fc00bb40 con 0x7f04040ac600 2021-11-21T21:54:37.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/859170486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 msgr2=0x7f04040aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 --2- 172.21.2.116:0/859170486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f04040aca50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f0404062500 tx=0x7f03fc009290).stop 2021-11-21T21:54:37.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/859170486 shutdown_connections 2021-11-21T21:54:37.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 --2- 172.21.2.116:0/859170486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f04040aca50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 --2- 172.21.2.116:0/859170486 >> v2:172.21.2.117:3300/0 conn(0x7f04040abce0 0x7f04041478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/859170486 >> 172.21.2.116:0/859170486 conn(0x7f040405bd20 msgr2=0x7f040405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/859170486 shutdown_connections 2021-11-21T21:54:37.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/859170486 wait complete. 2021-11-21T21:54:37.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 Processor -- start 2021-11-21T21:54:37.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- start start 2021-11-21T21:54:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04040abce0 0x7f0404143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f0404156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0404144810 0x7f0404157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0404157710 con 0x7f04040ac600 2021-11-21T21:54:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041467b0 con 0x7f04040abce0 2021-11-21T21:54:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0404156900 con 0x7f0404144810 2021-11-21T21:54:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f0404156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f0404156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45306/0 (socket says 172.21.2.116:45306) 2021-11-21T21:54:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 -- 172.21.2.116:0/1347926778 learned_addr learned my addr 172.21.2.116:0/1347926778 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 -- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0404144810 msgr2=0x7f0404157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0409f2d700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0404144810 0x7f0404157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0404144810 0x7f0404157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 -- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04040abce0 msgr2=0x7f0404143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04040abce0 0x7f0404143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 -- 172.21.2.116:0/1347926778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc009530 con 0x7f04040ac600 2021-11-21T21:54:37.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f0408f2b700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f0404156d30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f0404157850 tx=0x7f0400006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f03f27fc700 1 -- 172.21.2.116:0/1347926778 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0400013460 con 0x7f04040ac600 2021-11-21T21:54:37.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f03f27fc700 1 -- 172.21.2.116:0/1347926778 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04000135c0 con 0x7f04040ac600 2021-11-21T21:54:37.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f03f27fc700 1 -- 172.21.2.116:0/1347926778 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0400013920 con 0x7f04040ac600 2021-11-21T21:54:37.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04040b33e0 con 0x7f04040ac600 2021-11-21T21:54:37.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04040b2ab0 con 0x7f04040ac600 2021-11-21T21:54:37.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.866+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0404156a40 con 0x7f04040ac600 2021-11-21T21:54:37.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.870+0000 7f03f27fc700 1 -- 172.21.2.116:0/1347926778 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f040000f070 con 0x7f04040ac600 2021-11-21T21:54:37.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.870+0000 7f03f27fc700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03e4053610 0x7f03e4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:37.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.870+0000 7f03f27fc700 1 -- 172.21.2.116:0/1347926778 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f040006b1d0 con 0x7f04040ac600 2021-11-21T21:54:37.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.870+0000 7f040972c700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03e4053610 0x7f03e4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:37.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.870+0000 7f040972c700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03e4053610 0x7f03e4055ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f03f4002f10 tx=0x7f03f4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:37.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.870+0000 7f03f27fc700 1 -- 172.21.2.116:0/1347926778 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04000421c0 con 0x7f04040ac600 2021-11-21T21:54:37.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04040b33e0 con 0x7f04040ac600 2021-11-21T21:54:37.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f03f27fc700 1 -- 172.21.2.116:0/1347926778 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f04040b33e0 con 0x7f04040ac600 2021-11-21T21:54:37.959 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:37.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03e4053610 msgr2=0x7f03e4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03e4053610 0x7f03e4055ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f03f4002f10 tx=0x7f03f4006930).stop 2021-11-21T21:54:37.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 msgr2=0x7f0404156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:37.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f0404156d30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f0404157850 tx=0x7f0400006e90).stop 2021-11-21T21:54:37.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 shutdown_connections 2021-11-21T21:54:37.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0404144810 0x7f0404157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03e4053610 0x7f03e4055ad0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04040ac600 0x7f0404156d30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:37.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 --2- 172.21.2.116:0/1347926778 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04040abce0 0x7f0404143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:37.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 >> 172.21.2.116:0/1347926778 conn(0x7f040405bd20 msgr2=0x7f0404140a80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:37.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 shutdown_connections 2021-11-21T21:54:37.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:37.954+0000 7f040b990700 1 -- 172.21.2.116:0/1347926778 wait complete. 2021-11-21T21:54:37.967 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:38.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf41820700 1 Processor -- start 2021-11-21T21:54:38.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf41820700 1 -- start start 2021-11-21T21:54:38.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf41820700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbf3c0acf40 0x7fbf3c05bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf41820700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 0x7fbf3c0ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf41820700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c0b0670 con 0x7fbf3c0af210 2021-11-21T21:54:38.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf41820700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0aed70 con 0x7fbf3c0acf40 2021-11-21T21:54:38.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf41820700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0ad810 con 0x7fbf3c0b1de0 2021-11-21T21:54:38.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf4101f700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbf3c0af210 0x7fbf3c1427a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57270/0 (socket says 172.21.2.116:57270) 2021-11-21T21:54:38.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf4101f700 1 -- 172.21.2.116:0/1139464361 learned_addr learned my addr 172.21.2.116:0/1139464361 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:38.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf3bfff700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 0x7fbf3c0ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf4081e700 1 --2- 172.21.2.116:0/1139464361 >> v2:172.21.2.117:3300/0 conn(0x7fbf3c0acf40 0x7fbf3c05bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327665156 0 0) 0x7fbf3c0b0670 con 0x7fbf3c0af210 2021-11-21T21:54:38.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.018+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf1c003260 con 0x7fbf3c0af210 2021-11-21T21:54:38.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2478710524 0 0) 0x7fbf1c003260 con 0x7fbf3c0af210 2021-11-21T21:54:38.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 msgr2=0x7fbf3c0ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 0x7fbf3c0ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 >> v2:172.21.2.117:3300/0 conn(0x7fbf3c0acf40 msgr2=0x7fbf3c05bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 --2- 172.21.2.116:0/1139464361 >> v2:172.21.2.117:3300/0 conn(0x7fbf3c0acf40 0x7fbf3c05bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0b0e00 con 0x7fbf3c0af210 2021-11-21T21:54:38.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fbf340019e0 con 0x7fbf3c0af210 2021-11-21T21:54:38.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 >> v1:172.21.2.116:6789/0 conn(0x7fbf3c0af210 legacy=0x7fbf3c1427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 0x7fbf1c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf1c003c80 0x7fbf1c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf1c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0aed70 con 0x7fbf1c0049f0 2021-11-21T21:54:38.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0ad810 con 0x7fbf1c003c80 2021-11-21T21:54:38.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c00b750 con 0x7fbf3c0b1de0 2021-11-21T21:54:38.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf1c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3bfff700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf1c003c80 0x7fbf1c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 0x7fbf1c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 -- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 msgr2=0x7fbf1c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c0b1de0 0x7fbf1c003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 -- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf1c003c80 msgr2=0x7fbf1c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf1c003c80 0x7fbf1c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 -- 172.21.2.116:0/1139464361 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c003260 con 0x7fbf1c0049f0 2021-11-21T21:54:38.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf1c004e70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbf1c0054d0 tx=0x7fbf34002190).ready entity=mon.0 client_cookie=6480faaa0d8f227a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf3400cd00 con 0x7fbf1c0049f0 2021-11-21T21:54:38.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3400ce60 con 0x7fbf1c0049f0 2021-11-21T21:54:38.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3b7fe700 1 -- 172.21.2.116:0/1139464361 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf34005b80 con 0x7fbf1c0049f0 2021-11-21T21:54:38.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/1139464361 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 msgr2=0x7fbf1c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 --2- 172.21.2.116:0/1139464361 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf1c004e70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbf1c0054d0 tx=0x7fbf34002190).stop 2021-11-21T21:54:38.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 -- 172.21.2.116:0/1139464361 reap_dead start 2021-11-21T21:54:38.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/1139464361 shutdown_connections 2021-11-21T21:54:38.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/1139464361 >> 172.21.2.116:0/1139464361 conn(0x7fbf3c0b57c0 msgr2=0x7fbf3c0b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/1139464361 shutdown_connections 2021-11-21T21:54:38.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/1139464361 wait complete. 2021-11-21T21:54:38.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 Processor -- start 2021-11-21T21:54:38.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- start start 2021-11-21T21:54:38.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf3c066170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf3c1560f0 0x7fbf3c13edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c066af0 0x7fbf3c13f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0ab910 con 0x7fbf1c0049f0 2021-11-21T21:54:38.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0abb40 con 0x7fbf3c1560f0 2021-11-21T21:54:38.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0abde0 con 0x7fbf3c066af0 2021-11-21T21:54:38.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf3c066170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf3c066170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45324/0 (socket says 172.21.2.116:45324) 2021-11-21T21:54:38.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 -- 172.21.2.116:0/2243450710 learned_addr learned my addr 172.21.2.116:0/2243450710 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4101f700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c066af0 0x7fbf3c13f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf3bfff700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf3c1560f0 0x7fbf3c13edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 -- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c066af0 msgr2=0x7fbf3c13f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c066af0 0x7fbf3c13f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 -- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf3c1560f0 msgr2=0x7fbf3c13edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf3c1560f0 0x7fbf3c13edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 -- 172.21.2.116:0/2243450710 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0ac080 con 0x7fbf1c0049f0 2021-11-21T21:54:38.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf4081e700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf3c066170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fbf240030b0 tx=0x7fbf2400f040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf39ffb700 1 -- 172.21.2.116:0/2243450710 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf24012040 con 0x7fbf1c0049f0 2021-11-21T21:54:38.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf39ffb700 1 -- 172.21.2.116:0/2243450710 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2400aa20 con 0x7fbf1c0049f0 2021-11-21T21:54:38.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c1596e0 con 0x7fbf1c0049f0 2021-11-21T21:54:38.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf39ffb700 1 -- 172.21.2.116:0/2243450710 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf2400b040 con 0x7fbf1c0049f0 2021-11-21T21:54:38.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c0a0cf0 con 0x7fbf1c0049f0 2021-11-21T21:54:38.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.022+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c159870 con 0x7fbf1c0049f0 2021-11-21T21:54:38.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.026+0000 7fbf39ffb700 1 -- 172.21.2.116:0/2243450710 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbf2400ad80 con 0x7fbf1c0049f0 2021-11-21T21:54:38.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.026+0000 7fbf39ffb700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf200536d0 0x7fbf20055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.026+0000 7fbf39ffb700 1 -- 172.21.2.116:0/2243450710 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbf2406ba70 con 0x7fbf1c0049f0 2021-11-21T21:54:38.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.026+0000 7fbf3bfff700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf200536d0 0x7fbf20055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.026+0000 7fbf3bfff700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf200536d0 0x7fbf20055b90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbf2c00b000 tx=0x7fbf2c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.026+0000 7fbf39ffb700 1 -- 172.21.2.116:0/2243450710 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf24042bd0 con 0x7fbf1c0049f0 2021-11-21T21:54:38.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.110+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf3c0573d0 con 0x7fbf1c0049f0 2021-11-21T21:54:38.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.110+0000 7fbf39ffb700 1 -- 172.21.2.116:0/2243450710 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7fbf24028ec0 con 0x7fbf1c0049f0 2021-11-21T21:54:38.117 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:38.117 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:38.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf200536d0 msgr2=0x7fbf20055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf200536d0 0x7fbf20055b90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbf2c00b000 tx=0x7fbf2c008040).stop 2021-11-21T21:54:38.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 msgr2=0x7fbf3c066170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf3c066170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fbf240030b0 tx=0x7fbf2400f040).stop 2021-11-21T21:54:38.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 shutdown_connections 2021-11-21T21:54:38.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf3c066af0 0x7fbf3c13f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf3c1560f0 0x7fbf3c13edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf200536d0 0x7fbf20055b90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 --2- 172.21.2.116:0/2243450710 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf1c0049f0 0x7fbf3c066170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 >> 172.21.2.116:0/2243450710 conn(0x7fbf3c0b57c0 msgr2=0x7fbf3c0b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 shutdown_connections 2021-11-21T21:54:38.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.114+0000 7fbf41820700 1 -- 172.21.2.116:0/2243450710 wait complete. 2021-11-21T21:54:38.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:38.128 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:38.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 Processor -- start 2021-11-21T21:54:38.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- start start 2021-11-21T21:54:38.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbad8144810 0x7fbad81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbad80ac600 0x7fbad80aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad8147e80 con 0x7fbad80abce0 2021-11-21T21:54:38.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80b1490 con 0x7fbad80ac600 2021-11-21T21:54:38.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80b1fd0 con 0x7fbad8144810 2021-11-21T21:54:38.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbad80ac600 0x7fbad80aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbad80ac600 0x7fbad80aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45336/0 (socket says 172.21.2.116:45336) 2021-11-21T21:54:38.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/1467610559 learned_addr learned my addr 172.21.2.116:0/1467610559 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/1467610559 >> v2:172.21.2.117:3300/0 conn(0x7fbad8144810 msgr2=0x7fbad81478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:38.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- 172.21.2.116:0/1467610559 >> v2:172.21.2.117:3300/0 conn(0x7fbad8144810 0x7fbad81478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/1467610559 >> v1:172.21.2.117:6789/0 conn(0x7fbad80abce0 legacy=0x7fbad80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:38.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/1467610559 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80af5a0 con 0x7fbad80ac600 2021-11-21T21:54:38.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- 172.21.2.116:0/1467610559 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad80ac600 0x7fbad80aca50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fbad80b4e00 tx=0x7fbad0009290).ready entity=mon.0 client_cookie=5f259c04bfb710a4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad67fc700 1 -- 172.21.2.116:0/1467610559 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad000b830 con 0x7fbad80ac600 2021-11-21T21:54:38.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad67fc700 1 -- 172.21.2.116:0/1467610559 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad000b990 con 0x7fbad80ac600 2021-11-21T21:54:38.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad67fc700 1 -- 172.21.2.116:0/1467610559 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad000bb40 con 0x7fbad80ac600 2021-11-21T21:54:38.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- 172.21.2.116:0/1467610559 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad80ac600 msgr2=0x7fbad80aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/1467610559 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad80ac600 0x7fbad80aca50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fbad80b4e00 tx=0x7fbad0009290).stop 2021-11-21T21:54:38.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- 172.21.2.116:0/1467610559 shutdown_connections 2021-11-21T21:54:38.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/1467610559 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad80ac600 0x7fbad80aca50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/1467610559 >> v2:172.21.2.117:3300/0 conn(0x7fbad8144810 0x7fbad81478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- 172.21.2.116:0/1467610559 >> 172.21.2.116:0/1467610559 conn(0x7fbad805bd20 msgr2=0x7fbad805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- 172.21.2.116:0/1467610559 shutdown_connections 2021-11-21T21:54:38.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- 172.21.2.116:0/1467610559 wait complete. 2021-11-21T21:54:38.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 Processor -- start 2021-11-21T21:54:38.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- start start 2021-11-21T21:54:38.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbad80abce0 0x7fbad813c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbad80ac600 0x7fbad80b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad8144810 0x7fbad80b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8143810 con 0x7fbad8144810 2021-11-21T21:54:38.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8143a40 con 0x7fbad80abce0 2021-11-21T21:54:38.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8143ce0 con 0x7fbad80ac600 2021-11-21T21:54:38.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad8144810 0x7fbad80b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad8144810 0x7fbad80b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45338/0 (socket says 172.21.2.116:45338) 2021-11-21T21:54:38.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/3723682108 learned_addr learned my addr 172.21.2.116:0/3723682108 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbad80ac600 msgr2=0x7fbad80b32a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:38.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbad80ac600 0x7fbad80b32a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbad80abce0 msgr2=0x7fbad813c350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:38.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbad80abce0 0x7fbad813c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 -- 172.21.2.116:0/3723682108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009530 con 0x7fbad8144810 2021-11-21T21:54:38.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad7fff700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad8144810 0x7fbad80b3790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbad80b4e00 tx=0x7fbad0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad4ff9700 1 -- 172.21.2.116:0/3723682108 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad0021620 con 0x7fbad8144810 2021-11-21T21:54:38.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad4ff9700 1 -- 172.21.2.116:0/3723682108 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad0021780 con 0x7fbad8144810 2021-11-21T21:54:38.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbad4ff9700 1 -- 172.21.2.116:0/3723682108 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad0021ae0 con 0x7fbad8144810 2021-11-21T21:54:38.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad80b2e00 con 0x7fbad8144810 2021-11-21T21:54:38.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.178+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad813c980 con 0x7fbad8144810 2021-11-21T21:54:38.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.182+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad8143ee0 con 0x7fbad8144810 2021-11-21T21:54:38.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.182+0000 7fbad4ff9700 1 -- 172.21.2.116:0/3723682108 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbad000f070 con 0x7fbad8144810 2021-11-21T21:54:38.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.182+0000 7fbad4ff9700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbabc053610 0x7fbabc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.182+0000 7fbad4ff9700 1 -- 172.21.2.116:0/3723682108 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbad006d0d0 con 0x7fbad8144810 2021-11-21T21:54:38.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.182+0000 7fbad77fe700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbabc053610 0x7fbabc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.182+0000 7fbad77fe700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbabc053610 0x7fbabc055ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fbac0000ed0 tx=0x7fbac00068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.182+0000 7fbad4ff9700 1 -- 172.21.2.116:0/3723682108 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbad0044120 con 0x7fbad8144810 2021-11-21T21:54:38.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbad80b2f90 con 0x7fbad8144810 2021-11-21T21:54:38.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbad4ff9700 1 -- 172.21.2.116:0/3723682108 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbad0015030 con 0x7fbad8144810 2021-11-21T21:54:38.270 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:38.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbabc053610 msgr2=0x7fbabc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbabc053610 0x7fbabc055ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fbac0000ed0 tx=0x7fbac00068b0).stop 2021-11-21T21:54:38.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad8144810 msgr2=0x7fbad80b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad8144810 0x7fbad80b3790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbad80b4e00 tx=0x7fbad0016000).stop 2021-11-21T21:54:38.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 shutdown_connections 2021-11-21T21:54:38.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbabc053610 0x7fbabc055ad0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbad8144810 0x7fbad80b3790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbad80ac600 0x7fbad80b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 --2- 172.21.2.116:0/3723682108 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbad80abce0 0x7fbad813c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 >> 172.21.2.116:0/3723682108 conn(0x7fbad805bd20 msgr2=0x7fbad813ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 shutdown_connections 2021-11-21T21:54:38.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.266+0000 7fbaddc7f700 1 -- 172.21.2.116:0/3723682108 wait complete. 2021-11-21T21:54:38.277 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:38.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb0b7fa700 1 Processor -- start 2021-11-21T21:54:38.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb0b7fa700 1 -- start start 2021-11-21T21:54:38.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb0b7fa700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ffb04144810 0x7ffb041478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb0b7fa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb040ac600 0x7ffb040aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb0b7fa700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb04147e80 con 0x7ffb040abce0 2021-11-21T21:54:38.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb0b7fa700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb040b1490 con 0x7ffb04144810 2021-11-21T21:54:38.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb0b7fa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b1fd0 con 0x7ffb040ac600 2021-11-21T21:54:38.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb09596700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ffb04144810 0x7ffb041478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb09d97700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb040ac600 0x7ffb040aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb08d95700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7ffb040abce0 0x7ffb040b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:56216/0 (socket says 172.21.2.116:56216) 2021-11-21T21:54:38.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.326+0000 7ffb08d95700 1 -- 172.21.2.116:0/2068908753 learned_addr learned my addr 172.21.2.116:0/2068908753 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:38.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb09d97700 1 -- 172.21.2.116:0/2068908753 >> v2:172.21.2.117:3300/0 conn(0x7ffb04144810 msgr2=0x7ffb041478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb09d97700 1 --2- 172.21.2.116:0/2068908753 >> v2:172.21.2.117:3300/0 conn(0x7ffb04144810 0x7ffb041478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb09d97700 1 -- 172.21.2.116:0/2068908753 >> v1:172.21.2.117:6789/0 conn(0x7ffb040abce0 legacy=0x7ffb040b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb09d97700 1 -- 172.21.2.116:0/2068908753 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040af5a0 con 0x7ffb040ac600 2021-11-21T21:54:38.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb09d97700 1 --2- 172.21.2.116:0/2068908753 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb040ac600 0x7ffb040aca90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffafc0032f0 tx=0x7ffafc0091d0).ready entity=mon.2 client_cookie=9dd50e4a25e8ae7a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffaf3fff700 1 -- 172.21.2.116:0/2068908753 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffafc00b6c0 con 0x7ffb040ac600 2021-11-21T21:54:38.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffaf3fff700 1 -- 172.21.2.116:0/2068908753 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc00b820 con 0x7ffb040ac600 2021-11-21T21:54:38.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffaf3fff700 1 -- 172.21.2.116:0/2068908753 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffafc00b990 con 0x7ffb040ac600 2021-11-21T21:54:38.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2068908753 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb040ac600 msgr2=0x7ffb040aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2068908753 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb040ac600 0x7ffb040aca90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffafc0032f0 tx=0x7ffafc0091d0).stop 2021-11-21T21:54:38.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2068908753 shutdown_connections 2021-11-21T21:54:38.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2068908753 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb040ac600 0x7ffb040aca90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2068908753 >> v2:172.21.2.117:3300/0 conn(0x7ffb04144810 0x7ffb041478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2068908753 >> 172.21.2.116:0/2068908753 conn(0x7ffb0405bd20 msgr2=0x7ffb0405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2068908753 shutdown_connections 2021-11-21T21:54:38.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2068908753 wait complete. 2021-11-21T21:54:38.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 Processor -- start 2021-11-21T21:54:38.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- start start 2021-11-21T21:54:38.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb040abce0 0x7ffb04143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb040ac600 0x7ffb041567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb04144810 0x7ffb04156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b41d0 con 0x7ffb040abce0 2021-11-21T21:54:38.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04062da0 con 0x7ffb040ac600 2021-11-21T21:54:38.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.330+0000 7ffb0b7fa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04063040 con 0x7ffb04144810 2021-11-21T21:54:38.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb040abce0 0x7ffb04143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb040abce0 0x7ffb04143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45352/0 (socket says 172.21.2.116:45352) 2021-11-21T21:54:38.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 -- 172.21.2.116:0/2988177921 learned_addr learned my addr 172.21.2.116:0/2988177921 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb08d95700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb040ac600 0x7ffb041567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 -- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb04144810 msgr2=0x7ffb04156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb04144810 0x7ffb04156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 -- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb040ac600 msgr2=0x7ffb041567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb040ac600 0x7ffb041567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 -- 172.21.2.116:0/2988177921 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009470 con 0x7ffb040abce0 2021-11-21T21:54:38.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb09596700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb040abce0 0x7ffb04143320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ffb0000c000 tx=0x7ffb00006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffaf27fc700 1 -- 172.21.2.116:0/2988177921 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb00010780 con 0x7ffb040abce0 2021-11-21T21:54:38.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffaf27fc700 1 -- 172.21.2.116:0/2988177921 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb000108e0 con 0x7ffb040abce0 2021-11-21T21:54:38.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb040b3cc0 con 0x7ffb040abce0 2021-11-21T21:54:38.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffaf27fc700 1 -- 172.21.2.116:0/2988177921 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb00010c40 con 0x7ffb040abce0 2021-11-21T21:54:38.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb04146c50 con 0x7ffb040abce0 2021-11-21T21:54:38.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb040b4390 con 0x7ffb040abce0 2021-11-21T21:54:38.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffaf27fc700 1 -- 172.21.2.116:0/2988177921 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ffb0001c460 con 0x7ffb040abce0 2021-11-21T21:54:38.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffaf27fc700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffae40536a0 0x7ffae4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffaf27fc700 1 -- 172.21.2.116:0/2988177921 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ffb0006bea0 con 0x7ffb040abce0 2021-11-21T21:54:38.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb08d95700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffae40536a0 0x7ffae4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.334+0000 7ffb08d95700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffae40536a0 0x7ffae4055b60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ffaf4008000 tx=0x7ffaf4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.338+0000 7ffaf27fc700 1 -- 172.21.2.116:0/2988177921 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb00042e90 con 0x7ffb040abce0 2021-11-21T21:54:38.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.418+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffb040af060 con 0x7ffb040abce0 2021-11-21T21:54:38.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.418+0000 7ffaf27fc700 1 -- 172.21.2.116:0/2988177921 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffb000464b0 con 0x7ffb040abce0 2021-11-21T21:54:38.424 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:38.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffae40536a0 msgr2=0x7ffae4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffae40536a0 0x7ffae4055b60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ffaf4008000 tx=0x7ffaf4009040).stop 2021-11-21T21:54:38.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb040abce0 msgr2=0x7ffb04143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb040abce0 0x7ffb04143320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ffb0000c000 tx=0x7ffb00006df0).stop 2021-11-21T21:54:38.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 shutdown_connections 2021-11-21T21:54:38.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb04144810 0x7ffb04156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb040ac600 0x7ffb041567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffae40536a0 0x7ffae4055b60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 --2- 172.21.2.116:0/2988177921 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb040abce0 0x7ffb04143320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 >> 172.21.2.116:0/2988177921 conn(0x7ffb0405bd20 msgr2=0x7ffb0413d600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 shutdown_connections 2021-11-21T21:54:38.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.422+0000 7ffb0b7fa700 1 -- 172.21.2.116:0/2988177921 wait complete. 2021-11-21T21:54:38.431 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:38.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.478+0000 7f1c94909700 1 Processor -- start 2021-11-21T21:54:38.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.478+0000 7f1c94909700 1 -- start start 2021-11-21T21:54:38.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1c90144810 0x7f1c901478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1c900ac600 0x7f1c900aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c90147e80 con 0x7f1c900abce0 2021-11-21T21:54:38.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c900b1490 con 0x7f1c90144810 2021-11-21T21:54:38.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c900b1fd0 con 0x7f1c900ac600 2021-11-21T21:54:38.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1c90144810 0x7f1c901478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1c90144810 0x7f1c901478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45360/0 (socket says 172.21.2.116:45360) 2021-11-21T21:54:38.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 -- 172.21.2.116:0/3480719558 learned_addr learned my addr 172.21.2.116:0/3480719558 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 -- 172.21.2.116:0/3480719558 >> v2:172.21.2.117:3300/0 conn(0x7f1c900ac600 msgr2=0x7f1c900aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:38.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 --2- 172.21.2.116:0/3480719558 >> v2:172.21.2.117:3300/0 conn(0x7f1c900ac600 0x7f1c900aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 -- 172.21.2.116:0/3480719558 >> v1:172.21.2.117:6789/0 conn(0x7f1c900abce0 legacy=0x7f1c900b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:38.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 -- 172.21.2.116:0/3480719558 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c900af5a0 con 0x7f1c90144810 2021-11-21T21:54:38.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 --2- 172.21.2.116:0/3480719558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c90144810 0x7f1c901478c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f1c900b4e00 tx=0x7f1c84006eb0).ready entity=mon.0 client_cookie=bdfe4fcb8e5b450c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8d59a700 1 -- 172.21.2.116:0/3480719558 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8400d5c0 con 0x7f1c90144810 2021-11-21T21:54:38.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8d59a700 1 -- 172.21.2.116:0/3480719558 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c84000e90 con 0x7f1c90144810 2021-11-21T21:54:38.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8d59a700 1 -- 172.21.2.116:0/3480719558 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8400d910 con 0x7f1c90144810 2021-11-21T21:54:38.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- 172.21.2.116:0/3480719558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c90144810 msgr2=0x7f1c901478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- 172.21.2.116:0/3480719558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c90144810 0x7f1c901478c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f1c900b4e00 tx=0x7f1c84006eb0).stop 2021-11-21T21:54:38.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- 172.21.2.116:0/3480719558 shutdown_connections 2021-11-21T21:54:38.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- 172.21.2.116:0/3480719558 >> v2:172.21.2.117:3300/0 conn(0x7f1c900ac600 0x7f1c900aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- 172.21.2.116:0/3480719558 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c90144810 0x7f1c901478c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- 172.21.2.116:0/3480719558 >> 172.21.2.116:0/3480719558 conn(0x7f1c9005bd20 msgr2=0x7f1c9005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- 172.21.2.116:0/3480719558 shutdown_connections 2021-11-21T21:54:38.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- 172.21.2.116:0/3480719558 wait complete. 2021-11-21T21:54:38.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 Processor -- start 2021-11-21T21:54:38.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- start start 2021-11-21T21:54:38.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c900abce0 0x7f1c90143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c900ac600 0x7f1c9013a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c90144810 0x7f1c9013a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90148e80 con 0x7f1c900ac600 2021-11-21T21:54:38.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90159e30 con 0x7f1c90144810 2021-11-21T21:54:38.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900b30b0 con 0x7f1c900abce0 2021-11-21T21:54:38.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c900ac600 0x7f1c9013a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c900ac600 0x7f1c9013a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45370/0 (socket says 172.21.2.116:45370) 2021-11-21T21:54:38.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 -- 172.21.2.116:0/1105243550 learned_addr learned my addr 172.21.2.116:0/1105243550 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c900abce0 0x7f1c90143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8ed9d700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c90144810 0x7f1c9013a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 -- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c900abce0 msgr2=0x7f1c90143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c900abce0 0x7f1c90143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 -- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c90144810 msgr2=0x7f1c9013a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c90144810 0x7f1c9013a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 -- 172.21.2.116:0/1105243550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8400b1e0 con 0x7f1c900ac600 2021-11-21T21:54:38.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8ed9d700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c90144810 0x7f1c9013a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:38.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8e59c700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c900abce0 0x7f1c90143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:38.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c8dd9b700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c900ac600 0x7f1c9013a4d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1c88003330 tx=0x7f1c880091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c7f7fe700 1 -- 172.21.2.116:0/1105243550 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c88001710 con 0x7f1c900ac600 2021-11-21T21:54:38.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c7f7fe700 1 -- 172.21.2.116:0/1105243550 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88001870 con 0x7f1c900ac600 2021-11-21T21:54:38.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c90146bf0 con 0x7f1c900ac600 2021-11-21T21:54:38.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c7f7fe700 1 -- 172.21.2.116:0/1105243550 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c88001bd0 con 0x7f1c900ac600 2021-11-21T21:54:38.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.482+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c901463e0 con 0x7f1c900ac600 2021-11-21T21:54:38.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.486+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c900b32b0 con 0x7f1c900ac600 2021-11-21T21:54:38.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.486+0000 7f1c7f7fe700 1 -- 172.21.2.116:0/1105243550 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1c88018030 con 0x7f1c900ac600 2021-11-21T21:54:38.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.486+0000 7f1c7f7fe700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c6c053610 0x7f1c6c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.486+0000 7f1c7f7fe700 1 -- 172.21.2.116:0/1105243550 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1c8806b880 con 0x7f1c900ac600 2021-11-21T21:54:38.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.486+0000 7f1c8e59c700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c6c053610 0x7f1c6c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.486+0000 7f1c8e59c700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c6c053610 0x7f1c6c055ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f1c84017000 tx=0x7f1c84015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.486+0000 7f1c7f7fe700 1 -- 172.21.2.116:0/1105243550 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c88041e90 con 0x7f1c900ac600 2021-11-21T21:54:38.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c90159c40 con 0x7f1c900ac600 2021-11-21T21:54:38.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c7f7fe700 1 -- 172.21.2.116:0/1105243550 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1c900b32b0 con 0x7f1c900ac600 2021-11-21T21:54:38.574 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:38.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c6c053610 msgr2=0x7f1c6c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c6c053610 0x7f1c6c055ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f1c84017000 tx=0x7f1c84015000).stop 2021-11-21T21:54:38.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c900ac600 msgr2=0x7f1c9013a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c900ac600 0x7f1c9013a4d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1c88003330 tx=0x7f1c880091d0).stop 2021-11-21T21:54:38.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 shutdown_connections 2021-11-21T21:54:38.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c90144810 0x7f1c9013a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c6c053610 0x7f1c6c055ad0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c900ac600 0x7f1c9013a4d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 --2- 172.21.2.116:0/1105243550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c900abce0 0x7f1c90143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 >> 172.21.2.116:0/1105243550 conn(0x7f1c9005bd20 msgr2=0x7f1c901400c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 shutdown_connections 2021-11-21T21:54:38.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.570+0000 7f1c94909700 1 -- 172.21.2.116:0/1105243550 wait complete. 2021-11-21T21:54:38.581 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:38.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 Processor -- start 2021-11-21T21:54:38.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- start start 2021-11-21T21:54:38.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1a000abce0 0x7f1a001478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a00147e80 con 0x7f1a000ac600 2021-11-21T21:54:38.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a000af5a0 con 0x7f1a00144810 2021-11-21T21:54:38.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a000b1490 con 0x7f1a000abce0 2021-11-21T21:54:38.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f1a000ac600 0x7f1a000aca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57334/0 (socket says 172.21.2.116:57334) 2021-11-21T21:54:38.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 -- 172.21.2.116:0/2682837809 learned_addr learned my addr 172.21.2.116:0/2682837809 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:38.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752601679 0 0) 0x7f1a00147e80 con 0x7f1a000ac600 2021-11-21T21:54:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e8003260 con 0x7f1a000ac600 2021-11-21T21:54:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781172698 0 0) 0x7f19e8003260 con 0x7f1a000ac600 2021-11-21T21:54:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 >> v2:172.21.2.117:3300/0 conn(0x7f1a000abce0 msgr2=0x7f1a001478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 --2- 172.21.2.116:0/2682837809 >> v2:172.21.2.117:3300/0 conn(0x7f1a000abce0 0x7f1a001478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 >> v1:172.21.2.117:6789/0 conn(0x7f1a00144810 legacy=0x7f1a000b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000b1fd0 con 0x7f1a000ac600 2021-11-21T21:54:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f19f80019e0 con 0x7f1a000ac600 2021-11-21T21:54:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 >> v1:172.21.2.116:6789/0 conn(0x7f1a000ac600 legacy=0x7f1a000aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19e8003820 0x7f19e8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19e8004500 0x7f19e8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f19e800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b1490 con 0x7f19e8005050 2021-11-21T21:54:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e800bf40 con 0x7f19e8003820 2021-11-21T21:54:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e800c100 con 0x7f19e8004500 2021-11-21T21:54:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f19e800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 -- 172.21.2.116:0/2682837809 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19e8004500 msgr2=0x7f19e8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19e8004500 0x7f19e8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 -- 172.21.2.116:0/2682837809 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19e8003820 msgr2=0x7f19e8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19e8003820 0x7f19e8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 -- 172.21.2.116:0/2682837809 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e8003260 con 0x7f19e8005050 2021-11-21T21:54:38.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 -- 172.21.2.116:0/2682837809 reap_dead start 2021-11-21T21:54:38.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a05c8a700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f19e800b750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f1a000b4e00 tx=0x7f1a00146e80).ready entity=mon.0 client_cookie=11507d1e314ad815 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19f800cc60 con 0x7f19e8005050 2021-11-21T21:54:38.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f800cdc0 con 0x7f19e8005050 2021-11-21T21:54:38.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f19f3fff700 1 -- 172.21.2.116:0/2682837809 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19f8005b80 con 0x7f19e8005050 2021-11-21T21:54:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- 172.21.2.116:0/2682837809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 msgr2=0x7f19e800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f19e800b750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f1a000b4e00 tx=0x7f1a00146e80).stop 2021-11-21T21:54:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- 172.21.2.116:0/2682837809 shutdown_connections 2021-11-21T21:54:38.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 --2- 172.21.2.116:0/2682837809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f19e800b750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- 172.21.2.116:0/2682837809 >> 172.21.2.116:0/2682837809 conn(0x7f1a0005bd20 msgr2=0x7f1a0005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- 172.21.2.116:0/2682837809 shutdown_connections 2021-11-21T21:54:38.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.658+0000 7f1a076ed700 1 -- 172.21.2.116:0/2682837809 wait complete. 2021-11-21T21:54:38.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 Processor -- start 2021-11-21T21:54:38.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 -- start start 2021-11-21T21:54:38.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f1a0014ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a000b40e0 0x7f1a00159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a000b4500 0x7f1a0015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b3870 con 0x7f19e8005050 2021-11-21T21:54:38.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b3aa0 con 0x7f1a000b4500 2021-11-21T21:54:38.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b3d40 con 0x7f1a000b40e0 2021-11-21T21:54:38.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f1a0014ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f1a0014ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45386/0 (socket says 172.21.2.116:45386) 2021-11-21T21:54:38.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 -- 172.21.2.116:0/1669077474 learned_addr learned my addr 172.21.2.116:0/1669077474 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 -- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a000b40e0 msgr2=0x7f1a00159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a000b40e0 0x7f1a00159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 -- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a000b4500 msgr2=0x7f1a0015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a000b4500 0x7f1a0015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 -- 172.21.2.116:0/1669077474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0015a710 con 0x7f19e8005050 2021-11-21T21:54:38.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a05489700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f1a0014ae00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f19f4000f70 tx=0x7f19f4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f19f27fc700 1 -- 172.21.2.116:0/1669077474 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19f4013460 con 0x7f19e8005050 2021-11-21T21:54:38.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f19f27fc700 1 -- 172.21.2.116:0/1669077474 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f40135c0 con 0x7f19e8005050 2021-11-21T21:54:38.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a000b2c10 con 0x7f19e8005050 2021-11-21T21:54:38.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f19f27fc700 1 -- 172.21.2.116:0/1669077474 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19f4013920 con 0x7f19e8005050 2021-11-21T21:54:38.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a00143180 con 0x7f19e8005050 2021-11-21T21:54:38.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a000af060 con 0x7f19e8005050 2021-11-21T21:54:38.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f19f27fc700 1 -- 172.21.2.116:0/1669077474 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f19f401a060 con 0x7f19e8005050 2021-11-21T21:54:38.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f19f27fc700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19e00536f0 0x7f19e0055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f19f27fc700 1 -- 172.21.2.116:0/1669077474 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f19f406b380 con 0x7f19e8005050 2021-11-21T21:54:38.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.662+0000 7f1a04c88700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19e00536f0 0x7f19e0055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.666+0000 7f1a04c88700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19e00536f0 0x7f19e0055bb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f19fc002f70 tx=0x7f19fc006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.666+0000 7f19f27fc700 1 -- 172.21.2.116:0/1669077474 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f19f40423d0 con 0x7f19e8005050 2021-11-21T21:54:38.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.746+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a000ad630 con 0x7f19e8005050 2021-11-21T21:54:38.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.746+0000 7f19f27fc700 1 -- 172.21.2.116:0/1669077474 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f19f4018020 con 0x7f19e8005050 2021-11-21T21:54:38.752 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:38.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19e00536f0 msgr2=0x7f19e0055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19e00536f0 0x7f19e0055bb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f19fc002f70 tx=0x7f19fc006a00).stop 2021-11-21T21:54:38.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 msgr2=0x7f1a0014ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f1a0014ae00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f19f4000f70 tx=0x7f19f4006df0).stop 2021-11-21T21:54:38.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 shutdown_connections 2021-11-21T21:54:38.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a000b4500 0x7f1a0015a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a000b40e0 0x7f1a00159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19e00536f0 0x7f19e0055bb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 --2- 172.21.2.116:0/1669077474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19e8005050 0x7f1a0014ae00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 >> 172.21.2.116:0/1669077474 conn(0x7f1a0005bd20 msgr2=0x7f1a0013d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 shutdown_connections 2021-11-21T21:54:38.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.750+0000 7f1a076ed700 1 -- 172.21.2.116:0/1669077474 wait complete. 2021-11-21T21:54:38.764 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.818+0000 7f7777a5c700 1 Processor -- start 2021-11-21T21:54:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.818+0000 7f7777a5c700 1 -- start start 2021-11-21T21:54:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f777005c060 0x7f7770141200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7770060510 0x7f77701416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77700b1ae0 con 0x7f777005f730 2021-11-21T21:54:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77700b2250 con 0x7f7770060510 2021-11-21T21:54:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777005c440 con 0x7f777005c060 2021-11-21T21:54:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7775ff9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7770060510 0x7f77701416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f77757f8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f777005c060 0x7f7770141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7775ff9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7770060510 0x7f77701416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:46564/0 (socket says 172.21.2.116:46564) 2021-11-21T21:54:38.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f77757f8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f777005c060 0x7f7770141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:48738/0 (socket says 172.21.2.116:48738) 2021-11-21T21:54:38.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7775ff9700 1 -- 172.21.2.116:0/1535948288 learned_addr learned my addr 172.21.2.116:0/1535948288 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7774ff7700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f777005f730 0x7f777005fe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:56262/0 (socket says 172.21.2.116:56262) 2021-11-21T21:54:38.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f77757f8700 1 -- 172.21.2.116:0/1535948288 >> v2:172.21.2.117:3300/0 conn(0x7f7770060510 msgr2=0x7f77701416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f77757f8700 1 --2- 172.21.2.116:0/1535948288 >> v2:172.21.2.117:3300/0 conn(0x7f7770060510 0x7f77701416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f77757f8700 1 -- 172.21.2.116:0/1535948288 >> v1:172.21.2.117:6789/0 conn(0x7f777005f730 legacy=0x7f777005fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f77757f8700 1 -- 172.21.2.116:0/1535948288 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770140da0 con 0x7f777005c060 2021-11-21T21:54:38.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f77757f8700 1 --2- 172.21.2.116:0/1535948288 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f777005c060 0x7f7770141200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7760002f70 tx=0x7f7760006df0).ready entity=mon.2 client_cookie=3163e5e365849439 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f775ffff700 1 -- 172.21.2.116:0/1535948288 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f776000d570 con 0x7f777005c060 2021-11-21T21:54:38.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f775ffff700 1 -- 172.21.2.116:0/1535948288 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7760000e90 con 0x7f777005c060 2021-11-21T21:54:38.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f775ffff700 1 -- 172.21.2.116:0/1535948288 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f776000d8c0 con 0x7f777005c060 2021-11-21T21:54:38.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- 172.21.2.116:0/1535948288 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f777005c060 msgr2=0x7f7770141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 --2- 172.21.2.116:0/1535948288 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f777005c060 0x7f7770141200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7760002f70 tx=0x7f7760006df0).stop 2021-11-21T21:54:38.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- 172.21.2.116:0/1535948288 shutdown_connections 2021-11-21T21:54:38.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 --2- 172.21.2.116:0/1535948288 >> v2:172.21.2.117:3300/0 conn(0x7f7770060510 0x7f77701416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 --2- 172.21.2.116:0/1535948288 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f777005c060 0x7f7770141200 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- 172.21.2.116:0/1535948288 >> 172.21.2.116:0/1535948288 conn(0x7f77700b3be0 msgr2=0x7f77700b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- 172.21.2.116:0/1535948288 shutdown_connections 2021-11-21T21:54:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.822+0000 7f7777a5c700 1 -- 172.21.2.116:0/1535948288 wait complete. 2021-11-21T21:54:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 Processor -- start 2021-11-21T21:54:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 -- start start 2021-11-21T21:54:38.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f777005c060 0x7f777009c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f777005f730 0x7f77700a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770060510 0x7f77700aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777013fae0 con 0x7f777005c060 2021-11-21T21:54:38.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777009d000 con 0x7f777005f730 2021-11-21T21:54:38.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77700a5f40 con 0x7f7770060510 2021-11-21T21:54:38.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f777005c060 0x7f777009c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f777005c060 0x7f777009c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45402/0 (socket says 172.21.2.116:45402) 2021-11-21T21:54:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 -- 172.21.2.116:0/3736433499 learned_addr learned my addr 172.21.2.116:0/3736433499 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 -- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770060510 msgr2=0x7f77700aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770060510 0x7f77700aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 -- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f777005f730 msgr2=0x7f77700a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f777005f730 0x7f77700a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 -- 172.21.2.116:0/3736433499 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776000b180 con 0x7f777005c060 2021-11-21T21:54:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f77757f8700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f777005c060 0x7f777009c5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f777009d2a0 tx=0x7f7760014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f775e7fc700 1 -- 172.21.2.116:0/3736433499 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f776000dea0 con 0x7f777005c060 2021-11-21T21:54:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f775e7fc700 1 -- 172.21.2.116:0/3736433499 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776001e400 con 0x7f777005c060 2021-11-21T21:54:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777015a850 con 0x7f777005c060 2021-11-21T21:54:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f775e7fc700 1 -- 172.21.2.116:0/3736433499 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f776001e760 con 0x7f777005c060 2021-11-21T21:54:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77700a6410 con 0x7f777005c060 2021-11-21T21:54:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77700a65d0 con 0x7f777005c060 2021-11-21T21:54:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f775e7fc700 1 -- 172.21.2.116:0/3736433499 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7760025030 con 0x7f777005c060 2021-11-21T21:54:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f775e7fc700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500534f0 0x7f77500559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f775e7fc700 1 -- 172.21.2.116:0/3736433499 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f776006c810 con 0x7f777005c060 2021-11-21T21:54:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.826+0000 7f7774ff7700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500534f0 0x7f77500559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.830+0000 7f7774ff7700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500534f0 0x7f77500559b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f7770060ec0 tx=0x7f776c007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.830+0000 7f775e7fc700 1 -- 172.21.2.116:0/3736433499 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7760043970 con 0x7f777005c060 2021-11-21T21:54:38.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77700b1d10 con 0x7f777005c060 2021-11-21T21:54:38.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f775e7fc700 1 -- 172.21.2.116:0/3736433499 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f7760023020 con 0x7f777005c060 2021-11-21T21:54:38.919 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:38.920 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:38.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500534f0 msgr2=0x7f77500559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500534f0 0x7f77500559b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f7770060ec0 tx=0x7f776c007800).stop 2021-11-21T21:54:38.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f777005c060 msgr2=0x7f777009c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f777005c060 0x7f777009c5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f777009d2a0 tx=0x7f7760014000).stop 2021-11-21T21:54:38.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 shutdown_connections 2021-11-21T21:54:38.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770060510 0x7f77700aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f777005f730 0x7f77700a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:38.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500534f0 0x7f77500559b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 --2- 172.21.2.116:0/3736433499 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f777005c060 0x7f777009c5a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 >> 172.21.2.116:0/3736433499 conn(0x7f77700b3be0 msgr2=0x7f7770141120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 shutdown_connections 2021-11-21T21:54:38.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.914+0000 7f7777a5c700 1 -- 172.21.2.116:0/3736433499 wait complete. 2021-11-21T21:54:38.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:38.929 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:38.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.982+0000 7fcc2aa60700 1 Processor -- start 2021-11-21T21:54:38.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.982+0000 7fcc2aa60700 1 -- start start 2021-11-21T21:54:38.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcc240abce0 0x7fcc241478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcc240ac600 0x7fcc240aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc24147e80 con 0x7fcc24144810 2021-11-21T21:54:38.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc240b1490 con 0x7fcc240ac600 2021-11-21T21:54:38.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc240b1fd0 con 0x7fcc240abce0 2021-11-21T21:54:38.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcc240ac600 0x7fcc240aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcc240ac600 0x7fcc240aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45408/0 (socket says 172.21.2.116:45408) 2021-11-21T21:54:38.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 -- 172.21.2.116:0/3532077927 learned_addr learned my addr 172.21.2.116:0/3532077927 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:38.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 --2- 172.21.2.116:0/3532077927 >> v2:172.21.2.117:3300/0 conn(0x7fcc240abce0 0x7fcc241478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:38.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 -- 172.21.2.116:0/3532077927 >> v2:172.21.2.117:3300/0 conn(0x7fcc240abce0 msgr2=0x7fcc241478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 --2- 172.21.2.116:0/3532077927 >> v2:172.21.2.117:3300/0 conn(0x7fcc240abce0 0x7fcc241478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 -- 172.21.2.116:0/3532077927 >> v1:172.21.2.117:6789/0 conn(0x7fcc24144810 legacy=0x7fcc240b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:38.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 -- 172.21.2.116:0/3532077927 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240af5a0 con 0x7fcc240ac600 2021-11-21T21:54:38.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc28ffd700 1 --2- 172.21.2.116:0/3532077927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc240aca50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fcc24062500 tx=0x7fcc1c009290).ready entity=mon.0 client_cookie=3b20406f184c6eae server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:38.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc22ffd700 1 -- 172.21.2.116:0/3532077927 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1c00b830 con 0x7fcc240ac600 2021-11-21T21:54:38.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc22ffd700 1 -- 172.21.2.116:0/3532077927 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00b990 con 0x7fcc240ac600 2021-11-21T21:54:38.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc22ffd700 1 -- 172.21.2.116:0/3532077927 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1c00bb40 con 0x7fcc240ac600 2021-11-21T21:54:38.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- 172.21.2.116:0/3532077927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 msgr2=0x7fcc240aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:38.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/3532077927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc240aca50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fcc24062500 tx=0x7fcc1c009290).stop 2021-11-21T21:54:38.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- 172.21.2.116:0/3532077927 shutdown_connections 2021-11-21T21:54:38.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/3532077927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc240aca50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/3532077927 >> v2:172.21.2.117:3300/0 conn(0x7fcc240abce0 0x7fcc241478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:38.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- 172.21.2.116:0/3532077927 >> 172.21.2.116:0/3532077927 conn(0x7fcc2405bd20 msgr2=0x7fcc2405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:38.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- 172.21.2.116:0/3532077927 shutdown_connections 2021-11-21T21:54:38.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- 172.21.2.116:0/3532077927 wait complete. 2021-11-21T21:54:38.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 Processor -- start 2021-11-21T21:54:38.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- start start 2021-11-21T21:54:38.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc240abce0 0x7fcc241570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc24145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:38.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc24144810 0x7fcc240b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc24156520 con 0x7fcc240ac600 2021-11-21T21:54:39.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc24156710 con 0x7fcc240abce0 2021-11-21T21:54:39.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc241569b0 con 0x7fcc24144810 2021-11-21T21:54:39.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc24145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc24145980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45416/0 (socket says 172.21.2.116:45416) 2021-11-21T21:54:39.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 -- 172.21.2.116:0/1534849982 learned_addr learned my addr 172.21.2.116:0/1534849982 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 -- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc24144810 msgr2=0x7fcc240b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc24144810 0x7fcc240b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 -- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc240abce0 msgr2=0x7fcc241570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc240abce0 0x7fcc241570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 -- 172.21.2.116:0/1534849982 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009530 con 0x7fcc240ac600 2021-11-21T21:54:39.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc237fe700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc24145980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fcc0c007000 tx=0x7fcc0c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc217fa700 1 -- 172.21.2.116:0/1534849982 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc0c010580 con 0x7fcc240ac600 2021-11-21T21:54:39.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc217fa700 1 -- 172.21.2.116:0/1534849982 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c0106e0 con 0x7fcc240ac600 2021-11-21T21:54:39.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc24156c50 con 0x7fcc240ac600 2021-11-21T21:54:39.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc217fa700 1 -- 172.21.2.116:0/1534849982 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc0c010a40 con 0x7fcc240ac600 2021-11-21T21:54:39.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.986+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc24146cd0 con 0x7fcc240ac600 2021-11-21T21:54:39.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.990+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc24156de0 con 0x7fcc240ac600 2021-11-21T21:54:39.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.990+0000 7fcc217fa700 1 -- 172.21.2.116:0/1534849982 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcc0c010ba0 con 0x7fcc240ac600 2021-11-21T21:54:39.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.990+0000 7fcc217fa700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc08053610 0x7fcc08055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.990+0000 7fcc217fa700 1 -- 172.21.2.116:0/1534849982 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcc0c06ba30 con 0x7fcc240ac600 2021-11-21T21:54:39.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.990+0000 7fcc23fff700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc08053610 0x7fcc08055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.990+0000 7fcc23fff700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc08053610 0x7fcc08055ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcc14002f30 tx=0x7fcc14006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:38.990+0000 7fcc217fa700 1 -- 172.21.2.116:0/1534849982 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc0c042b90 con 0x7fcc240ac600 2021-11-21T21:54:39.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.074+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcc240573d0 con 0x7fcc240ac600 2021-11-21T21:54:39.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.074+0000 7fcc217fa700 1 -- 172.21.2.116:0/1534849982 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcc0c0461b0 con 0x7fcc240ac600 2021-11-21T21:54:39.080 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:39.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc08053610 msgr2=0x7fcc08055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc08053610 0x7fcc08055ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcc14002f30 tx=0x7fcc14006930).stop 2021-11-21T21:54:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 msgr2=0x7fcc24145980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc24145980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fcc0c007000 tx=0x7fcc0c008040).stop 2021-11-21T21:54:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 shutdown_connections 2021-11-21T21:54:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc24144810 0x7fcc240b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc08053610 0x7fcc08055ad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc240ac600 0x7fcc24145980 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 --2- 172.21.2.116:0/1534849982 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc240abce0 0x7fcc241570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 >> 172.21.2.116:0/1534849982 conn(0x7fcc2405bd20 msgr2=0x7fcc24149510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 shutdown_connections 2021-11-21T21:54:39.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.078+0000 7fcc2aa60700 1 -- 172.21.2.116:0/1534849982 wait complete. 2021-11-21T21:54:39.095 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:39.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 Processor -- start 2021-11-21T21:54:39.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- start start 2021-11-21T21:54:39.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb7800adc10 0x7fb7800b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb780144a10 0x7fb780141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780141510 0x7fb78013a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800b3c50 con 0x7fb7800adc10 2021-11-21T21:54:39.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800b3ef0 con 0x7fb780144a10 2021-11-21T21:54:39.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b4190 con 0x7fb780141510 2021-11-21T21:54:39.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb7800adc10 0x7fb7800b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb7800adc10 0x7fb7800b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45424/0 (socket says 172.21.2.116:45424) 2021-11-21T21:54:39.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 -- 172.21.2.116:0/3980211369 learned_addr learned my addr 172.21.2.116:0/3980211369 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 -- 172.21.2.116:0/3980211369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780141510 msgr2=0x7fb78013a540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:39.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 --2- 172.21.2.116:0/3980211369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780141510 0x7fb78013a540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 -- 172.21.2.116:0/3980211369 >> v2:172.21.2.117:3300/0 conn(0x7fb780144a10 msgr2=0x7fb780141020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:39.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 --2- 172.21.2.116:0/3980211369 >> v2:172.21.2.117:3300/0 conn(0x7fb780144a10 0x7fb780141020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 -- 172.21.2.116:0/3980211369 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb780140b80 con 0x7fb7800adc10 2021-11-21T21:54:39.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb784e61700 1 --2- 172.21.2.116:0/3980211369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb7800adc10 0x7fb7800b2030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb770002f70 tx=0x7fb770006eb0).ready entity=mon.0 client_cookie=dcbf5df23c67b285 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb77f7fe700 1 -- 172.21.2.116:0/3980211369 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb770010b60 con 0x7fb7800adc10 2021-11-21T21:54:39.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb77f7fe700 1 -- 172.21.2.116:0/3980211369 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb770000e90 con 0x7fb7800adc10 2021-11-21T21:54:39.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb77f7fe700 1 -- 172.21.2.116:0/3980211369 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7700183d0 con 0x7fb7800adc10 2021-11-21T21:54:39.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- 172.21.2.116:0/3980211369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb7800adc10 msgr2=0x7fb7800b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3980211369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb7800adc10 0x7fb7800b2030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb770002f70 tx=0x7fb770006eb0).stop 2021-11-21T21:54:39.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- 172.21.2.116:0/3980211369 shutdown_connections 2021-11-21T21:54:39.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3980211369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780141510 0x7fb78013a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3980211369 >> v2:172.21.2.117:3300/0 conn(0x7fb780144a10 0x7fb780141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3980211369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb7800adc10 0x7fb7800b2030 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- 172.21.2.116:0/3980211369 >> 172.21.2.116:0/3980211369 conn(0x7fb78005bd20 msgr2=0x7fb78005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- 172.21.2.116:0/3980211369 shutdown_connections 2021-11-21T21:54:39.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.150+0000 7fb7870c5700 1 -- 172.21.2.116:0/3980211369 wait complete. 2021-11-21T21:54:39.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 Processor -- start 2021-11-21T21:54:39.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 -- start start 2021-11-21T21:54:39.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb7800adc10 0x7fb78014e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb780141510 0x7fb780067220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780144a10 0x7fb7800b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78014e980 con 0x7fb780141510 2021-11-21T21:54:39.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800afd20 con 0x7fb7800adc10 2021-11-21T21:54:39.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801105b0 con 0x7fb780144a10 2021-11-21T21:54:39.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb780141510 0x7fb780067220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb780141510 0x7fb780067220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45430/0 (socket says 172.21.2.116:45430) 2021-11-21T21:54:39.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 -- 172.21.2.116:0/3362789194 learned_addr learned my addr 172.21.2.116:0/3362789194 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 -- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780144a10 msgr2=0x7fb7800b02e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:39.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780144a10 0x7fb7800b02e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 -- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb7800adc10 msgr2=0x7fb78014e490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:39.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb7800adc10 0x7fb78014e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 -- 172.21.2.116:0/3362789194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77000b1e0 con 0x7fb780141510 2021-11-21T21:54:39.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77ffff700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb780141510 0x7fb780067220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fb768000ed0 tx=0x7fb768006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77dffb700 1 -- 172.21.2.116:0/3362789194 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb768009e00 con 0x7fb780141510 2021-11-21T21:54:39.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77dffb700 1 -- 172.21.2.116:0/3362789194 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768012410 con 0x7fb780141510 2021-11-21T21:54:39.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77dffb700 1 -- 172.21.2.116:0/3362789194 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb768012770 con 0x7fb780141510 2021-11-21T21:54:39.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78014dab0 con 0x7fb780141510 2021-11-21T21:54:39.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78014dff0 con 0x7fb780141510 2021-11-21T21:54:39.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7800573d0 con 0x7fb780141510 2021-11-21T21:54:39.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77dffb700 1 -- 172.21.2.116:0/3362789194 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb76800f030 con 0x7fb780141510 2021-11-21T21:54:39.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77dffb700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb764053610 0x7fb764055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb77dffb700 1 -- 172.21.2.116:0/3362789194 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb76806b4f0 con 0x7fb780141510 2021-11-21T21:54:39.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.154+0000 7fb784e61700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb764053610 0x7fb764055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.158+0000 7fb784e61700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb764053610 0x7fb764055ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fb770002f70 tx=0x7fb770017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.158+0000 7fb77dffb700 1 -- 172.21.2.116:0/3362789194 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb768042650 con 0x7fb780141510 2021-11-21T21:54:39.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.238+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb780068320 con 0x7fb780141510 2021-11-21T21:54:39.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.238+0000 7fb77dffb700 1 -- 172.21.2.116:0/3362789194 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb768045c70 con 0x7fb780141510 2021-11-21T21:54:39.250 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:39.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb764053610 msgr2=0x7fb764055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb764053610 0x7fb764055ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fb770002f70 tx=0x7fb770017000).stop 2021-11-21T21:54:39.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb780141510 msgr2=0x7fb780067220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb780141510 0x7fb780067220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fb768000ed0 tx=0x7fb768006d60).stop 2021-11-21T21:54:39.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 shutdown_connections 2021-11-21T21:54:39.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb780144a10 0x7fb7800b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb764053610 0x7fb764055ad0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb780141510 0x7fb780067220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 --2- 172.21.2.116:0/3362789194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb7800adc10 0x7fb78014e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 >> 172.21.2.116:0/3362789194 conn(0x7fb78005bd20 msgr2=0x7fb78013ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 shutdown_connections 2021-11-21T21:54:39.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.242+0000 7fb7870c5700 1 -- 172.21.2.116:0/3362789194 wait complete. 2021-11-21T21:54:39.256 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:39.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 Processor -- start 2021-11-21T21:54:39.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- start start 2021-11-21T21:54:39.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5d30144810 0x7f5d301478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d300abce0 0x7f5d300b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5d300b1db0 0x7f5d300ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d30147e80 con 0x7f5d300b1db0 2021-11-21T21:54:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d300af5a0 con 0x7f5d30144810 2021-11-21T21:54:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d300b1490 con 0x7f5d300abce0 2021-11-21T21:54:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5d300b1db0 0x7f5d300ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5d300b1db0 0x7f5d300ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45436/0 (socket says 172.21.2.116:45436) 2021-11-21T21:54:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1411799823 learned_addr learned my addr 172.21.2.116:0/1411799823 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1411799823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d300abce0 msgr2=0x7f5d300b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:39.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- 172.21.2.116:0/1411799823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d300abce0 0x7f5d300b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1411799823 >> v2:172.21.2.117:3300/0 conn(0x7f5d30144810 msgr2=0x7f5d301478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:39.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- 172.21.2.116:0/1411799823 >> v2:172.21.2.117:3300/0 conn(0x7f5d30144810 0x7f5d301478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1411799823 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d300af060 con 0x7f5d300b1db0 2021-11-21T21:54:39.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- 172.21.2.116:0/1411799823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d300b1db0 0x7f5d300ac600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f5d280032f0 tx=0x7f5d28009290).ready entity=mon.0 client_cookie=b406ede695c07dfb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2e7fc700 1 -- 172.21.2.116:0/1411799823 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d2800b870 con 0x7f5d300b1db0 2021-11-21T21:54:39.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2e7fc700 1 -- 172.21.2.116:0/1411799823 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2800b9d0 con 0x7f5d300b1db0 2021-11-21T21:54:39.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2e7fc700 1 -- 172.21.2.116:0/1411799823 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d2800bb80 con 0x7f5d300b1db0 2021-11-21T21:54:39.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- 172.21.2.116:0/1411799823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d300b1db0 msgr2=0x7f5d300ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1411799823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d300b1db0 0x7f5d300ac600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f5d280032f0 tx=0x7f5d28009290).stop 2021-11-21T21:54:39.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- 172.21.2.116:0/1411799823 shutdown_connections 2021-11-21T21:54:39.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1411799823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d300b1db0 0x7f5d300ac600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1411799823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d300abce0 0x7f5d300b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1411799823 >> v2:172.21.2.117:3300/0 conn(0x7f5d30144810 0x7f5d301478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- 172.21.2.116:0/1411799823 >> 172.21.2.116:0/1411799823 conn(0x7f5d3005bd20 msgr2=0x7f5d3005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- 172.21.2.116:0/1411799823 shutdown_connections 2021-11-21T21:54:39.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- 172.21.2.116:0/1411799823 wait complete. 2021-11-21T21:54:39.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 Processor -- start 2021-11-21T21:54:39.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- start start 2021-11-21T21:54:39.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d300abce0 0x7f5d30143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d30144810 0x7f5d3015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d3014a8f0 0x7f5d30156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30159cb0 con 0x7f5d3014a8f0 2021-11-21T21:54:39.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30159f50 con 0x7f5d300abce0 2021-11-21T21:54:39.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3015a1f0 con 0x7f5d30144810 2021-11-21T21:54:39.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d3014a8f0 0x7f5d30156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d3014a8f0 0x7f5d30156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45446/0 (socket says 172.21.2.116:45446) 2021-11-21T21:54:39.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1238566908 learned_addr learned my addr 172.21.2.116:0/1238566908 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2f7fe700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d300abce0 0x7f5d30143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d30144810 msgr2=0x7f5d3015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d30144810 0x7f5d3015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d300abce0 msgr2=0x7f5d30143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d300abce0 0x7f5d30143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 -- 172.21.2.116:0/1238566908 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d28009530 con 0x7f5d3014a8f0 2021-11-21T21:54:39.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2ffff700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d3014a8f0 0x7f5d30156650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f5d280032f0 tx=0x7f5d28016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2cff9700 1 -- 172.21.2.116:0/1238566908 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d28021620 con 0x7f5d3014a8f0 2021-11-21T21:54:39.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2cff9700 1 -- 172.21.2.116:0/1238566908 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28021780 con 0x7f5d3014a8f0 2021-11-21T21:54:39.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d30146230 con 0x7f5d3014a8f0 2021-11-21T21:54:39.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d2cff9700 1 -- 172.21.2.116:0/1238566908 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d28021b20 con 0x7f5d3014a8f0 2021-11-21T21:54:39.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.306+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3014ad80 con 0x7f5d3014a8f0 2021-11-21T21:54:39.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.310+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3015a3b0 con 0x7f5d3014a8f0 2021-11-21T21:54:39.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.310+0000 7f5d2cff9700 1 -- 172.21.2.116:0/1238566908 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5d2800f070 con 0x7f5d3014a8f0 2021-11-21T21:54:39.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.310+0000 7f5d2cff9700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d14053610 0x7f5d14055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.310+0000 7f5d2cff9700 1 -- 172.21.2.116:0/1238566908 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5d2806cfd0 con 0x7f5d3014a8f0 2021-11-21T21:54:39.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.310+0000 7f5d2f7fe700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d14053610 0x7f5d14055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.310+0000 7f5d2f7fe700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d14053610 0x7f5d14055ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f5d18007000 tx=0x7f5d18008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.310+0000 7f5d2cff9700 1 -- 172.21.2.116:0/1238566908 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d28044130 con 0x7f5d3014a8f0 2021-11-21T21:54:39.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d300af300 con 0x7f5d3014a8f0 2021-11-21T21:54:39.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d2cff9700 1 -- 172.21.2.116:0/1238566908 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5d28015070 con 0x7f5d3014a8f0 2021-11-21T21:54:39.399 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:39.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d14053610 msgr2=0x7f5d14055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d14053610 0x7f5d14055ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f5d18007000 tx=0x7f5d18008040).stop 2021-11-21T21:54:39.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d3014a8f0 msgr2=0x7f5d30156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d3014a8f0 0x7f5d30156650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f5d280032f0 tx=0x7f5d28016000).stop 2021-11-21T21:54:39.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 shutdown_connections 2021-11-21T21:54:39.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d14053610 0x7f5d14055ad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d3014a8f0 0x7f5d30156650 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5d30144810 0x7f5d3015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 --2- 172.21.2.116:0/1238566908 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d300abce0 0x7f5d30143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.394+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 >> 172.21.2.116:0/1238566908 conn(0x7f5d3005bd20 msgr2=0x7f5d30148250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.398+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 shutdown_connections 2021-11-21T21:54:39.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.398+0000 7f5d361ba700 1 -- 172.21.2.116:0/1238566908 wait complete. 2021-11-21T21:54:39.406 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:39.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a22392700 1 Processor -- start 2021-11-21T21:54:39.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a22392700 1 -- start start 2021-11-21T21:54:39.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a22392700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9a1c0b02e0 0x7f9a1c05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a22392700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c0af620 0x7f9a1c142f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a22392700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c0b06c0 con 0x7f9a1c0afa80 2021-11-21T21:54:39.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a22392700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c0adf20 con 0x7f9a1c0b02e0 2021-11-21T21:54:39.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a22392700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0b2470 con 0x7f9a1c0af620 2021-11-21T21:54:39.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a2092f700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9a1c0afa80 0x7f9a1c143460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57408/0 (socket says 172.21.2.116:57408) 2021-11-21T21:54:39.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a2092f700 1 -- 172.21.2.116:0/2625941252 learned_addr learned my addr 172.21.2.116:0/2625941252 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:39.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811939641 0 0) 0x7f9a1c0b06c0 con 0x7f9a1c0afa80 2021-11-21T21:54:39.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99fc003260 con 0x7f9a1c0afa80 2021-11-21T21:54:39.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2071284070 0 0) 0x7f99fc003260 con 0x7f9a1c0afa80 2021-11-21T21:54:39.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c0af620 msgr2=0x7f9a1c142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c0af620 0x7f9a1c142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 >> v2:172.21.2.117:3300/0 conn(0x7f9a1c0b02e0 msgr2=0x7f9a1c05f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 --2- 172.21.2.116:0/2625941252 >> v2:172.21.2.117:3300/0 conn(0x7f9a1c0b02e0 0x7f9a1c05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0b26d0 con 0x7f9a1c0afa80 2021-11-21T21:54:39.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9a140019e0 con 0x7f9a1c0afa80 2021-11-21T21:54:39.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 >> v1:172.21.2.116:6789/0 conn(0x7f9a1c0afa80 legacy=0x7f9a1c143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc003750 0x7f99fc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c0af620 0x7f99fc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99fc004b10 0x7f99fc004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0adf20 con 0x7f99fc004b10 2021-11-21T21:54:39.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0b2470 con 0x7f99fc003750 2021-11-21T21:54:39.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc00b9c0 con 0x7f9a1c0af620 2021-11-21T21:54:39.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.454+0000 7f9a2092f700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99fc004b10 0x7f99fc004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c0af620 msgr2=0x7f99fc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c0af620 0x7f99fc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc003750 msgr2=0x7f99fc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc003750 0x7f99fc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/2625941252 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc003260 con 0x7f99fc004b10 2021-11-21T21:54:39.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99fc004b10 0x7f99fc004f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f9a140016f0 tx=0x7f9a1c142e90).ready entity=mon.0 client_cookie=7b678362a4296a72 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a1400cda0 con 0x7f99fc004b10 2021-11-21T21:54:39.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a14005990 con 0x7f99fc004b10 2021-11-21T21:54:39.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a1affd700 1 -- 172.21.2.116:0/2625941252 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a14005cf0 con 0x7f99fc004b10 2021-11-21T21:54:39.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- 172.21.2.116:0/2625941252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99fc004b10 msgr2=0x7f99fc004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 --2- 172.21.2.116:0/2625941252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99fc004b10 0x7f99fc004f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f9a140016f0 tx=0x7f9a1c142e90).stop 2021-11-21T21:54:39.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/2625941252 reap_dead start 2021-11-21T21:54:39.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- 172.21.2.116:0/2625941252 shutdown_connections 2021-11-21T21:54:39.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- 172.21.2.116:0/2625941252 >> 172.21.2.116:0/2625941252 conn(0x7f9a1c0b5ff0 msgr2=0x7f9a1c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- 172.21.2.116:0/2625941252 shutdown_connections 2021-11-21T21:54:39.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- 172.21.2.116:0/2625941252 wait complete. 2021-11-21T21:54:39.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 Processor -- start 2021-11-21T21:54:39.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- start start 2021-11-21T21:54:39.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc004b10 0x7f9a1c0ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c140360 0x7f9a1c1407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a1c140cd0 0x7f9a1c13e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c13f740 con 0x7f9a1c140cd0 2021-11-21T21:54:39.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0acc70 con 0x7f99fc004b10 2021-11-21T21:54:39.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c143bf0 con 0x7f9a1c140360 2021-11-21T21:54:39.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a1c140cd0 0x7f9a1c13e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a1c140cd0 0x7f9a1c13e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45466/0 (socket says 172.21.2.116:45466) 2021-11-21T21:54:39.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/3207163087 learned_addr learned my addr 172.21.2.116:0/3207163087 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a1b7fe700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c140360 0x7f9a1c1407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a1bfff700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc004b10 0x7f9a1c0ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c140360 msgr2=0x7f9a1c1407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c140360 0x7f9a1c1407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc004b10 msgr2=0x7f9a1c0ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc004b10 0x7f9a1c0ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 -- 172.21.2.116:0/3207163087 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c13f4a0 con 0x7f9a1c140cd0 2021-11-21T21:54:39.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a1bfff700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc004b10 0x7f9a1c0ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:39.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a2092f700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a1c140cd0 0x7f9a1c13e280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f9a1c0a9970 tx=0x7f9a14013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a197fa700 1 -- 172.21.2.116:0/3207163087 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a1401a070 con 0x7f9a1c140cd0 2021-11-21T21:54:39.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a197fa700 1 -- 172.21.2.116:0/3207163087 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1401b040 con 0x7f9a1c140cd0 2021-11-21T21:54:39.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c0ad200 con 0x7f9a1c140cd0 2021-11-21T21:54:39.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a197fa700 1 -- 172.21.2.116:0/3207163087 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a14004e90 con 0x7f9a1c140cd0 2021-11-21T21:54:39.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.458+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c0ac670 con 0x7f9a1c140cd0 2021-11-21T21:54:39.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.462+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c13f300 con 0x7f9a1c140cd0 2021-11-21T21:54:39.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.462+0000 7f9a197fa700 1 -- 172.21.2.116:0/3207163087 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9a14010070 con 0x7f9a1c140cd0 2021-11-21T21:54:39.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.462+0000 7f9a197fa700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a00053600 0x7f9a00055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.462+0000 7f9a197fa700 1 -- 172.21.2.116:0/3207163087 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9a1406c420 con 0x7f9a1c140cd0 2021-11-21T21:54:39.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.462+0000 7f9a1bfff700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a00053600 0x7f9a00055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.462+0000 7f9a1bfff700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a00053600 0x7f9a00055ac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f9a0c007000 tx=0x7f9a0c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.462+0000 7f9a197fa700 1 -- 172.21.2.116:0/3207163087 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a1c13f300 con 0x7f9a1c140cd0 2021-11-21T21:54:39.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a1c13f300 con 0x7f9a1c140cd0 2021-11-21T21:54:39.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a197fa700 1 -- 172.21.2.116:0/3207163087 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9a1c13f300 con 0x7f9a1c140cd0 2021-11-21T21:54:39.549 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:39.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a00053600 msgr2=0x7f9a00055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a00053600 0x7f9a00055ac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f9a0c007000 tx=0x7f9a0c008040).stop 2021-11-21T21:54:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a1c140cd0 msgr2=0x7f9a1c13e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a1c140cd0 0x7f9a1c13e280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f9a1c0a9970 tx=0x7f9a14013000).stop 2021-11-21T21:54:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 shutdown_connections 2021-11-21T21:54:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a00053600 0x7f9a00055ac0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a1c140cd0 0x7f9a1c13e280 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a1c140360 0x7f9a1c1407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 --2- 172.21.2.116:0/3207163087 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99fc004b10 0x7f9a1c0ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 >> 172.21.2.116:0/3207163087 conn(0x7f9a1c0b5ff0 msgr2=0x7f9a1c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 shutdown_connections 2021-11-21T21:54:39.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.546+0000 7f9a22392700 1 -- 172.21.2.116:0/3207163087 wait complete. 2021-11-21T21:54:39.556 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:39.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.602+0000 7f274236a700 1 Processor -- start 2021-11-21T21:54:39.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.602+0000 7f274236a700 1 -- start start 2021-11-21T21:54:39.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f273c0af7e0 0x7f273c05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273c143f70 con 0x7f273c0ad780 2021-11-21T21:54:39.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273c0adb60 con 0x7f273c0affc0 2021-11-21T21:54:39.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c143160 con 0x7f273c0af7e0 2021-11-21T21:54:39.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f273c0ad780 0x7f273c143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57426/0 (socket says 172.21.2.116:57426) 2021-11-21T21:54:39.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 -- 172.21.2.116:0/1243065878 learned_addr learned my addr 172.21.2.116:0/1243065878 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:39.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f2740907700 1 --2- 172.21.2.116:0/1243065878 >> v2:172.21.2.116:3300/0 conn(0x7f273c0af7e0 0x7f273c05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273affd700 1 -- 172.21.2.116:0/1243065878 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247885891 0 0) 0x7f273c143f70 con 0x7f273c0ad780 2021-11-21T21:54:39.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f2740907700 1 -- 172.21.2.116:0/1243065878 >> v1:172.21.2.117:6789/0 conn(0x7f273c0affc0 legacy=0x7f273c13fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:39.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f2740907700 1 -- 172.21.2.116:0/1243065878 >> v1:172.21.2.116:6789/0 conn(0x7f273c0ad780 legacy=0x7f273c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f2740907700 1 -- 172.21.2.116:0/1243065878 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0b0ce0 con 0x7f273c0af7e0 2021-11-21T21:54:39.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f2740907700 1 --2- 172.21.2.116:0/1243065878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0af7e0 0x7f273c05f730 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f273c0685b0 tx=0x7f273400b470).ready entity=mon.0 client_cookie=98e815bf7639b03e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273affd700 1 -- 172.21.2.116:0/1243065878 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f273400dd10 con 0x7f273c0af7e0 2021-11-21T21:54:39.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273affd700 1 -- 172.21.2.116:0/1243065878 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273400de70 con 0x7f273c0af7e0 2021-11-21T21:54:39.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273affd700 1 -- 172.21.2.116:0/1243065878 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f273400f5f0 con 0x7f273c0af7e0 2021-11-21T21:54:39.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- 172.21.2.116:0/1243065878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0af7e0 msgr2=0x7f273c05f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 --2- 172.21.2.116:0/1243065878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0af7e0 0x7f273c05f730 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f273c0685b0 tx=0x7f273400b470).stop 2021-11-21T21:54:39.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- 172.21.2.116:0/1243065878 shutdown_connections 2021-11-21T21:54:39.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 --2- 172.21.2.116:0/1243065878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0af7e0 0x7f273c05f730 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- 172.21.2.116:0/1243065878 >> 172.21.2.116:0/1243065878 conn(0x7f273c0b6000 msgr2=0x7f273c0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- 172.21.2.116:0/1243065878 shutdown_connections 2021-11-21T21:54:39.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- 172.21.2.116:0/1243065878 wait complete. 2021-11-21T21:54:39.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 Processor -- start 2021-11-21T21:54:39.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- start start 2021-11-21T21:54:39.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0ad780 0x7f273c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f273c0af7e0 0x7f273c14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f273c0affc0 0x7f273c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0ac760 con 0x7f273c0ad780 2021-11-21T21:54:39.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c1427f0 con 0x7f273c0affc0 2021-11-21T21:54:39.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c142a90 con 0x7f273c0af7e0 2021-11-21T21:54:39.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0ad780 0x7f273c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0ad780 0x7f273c0b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45478/0 (socket says 172.21.2.116:45478) 2021-11-21T21:54:39.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 -- 172.21.2.116:0/1074249808 learned_addr learned my addr 172.21.2.116:0/1074249808 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 -- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f273c0af7e0 msgr2=0x7f273c14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f273c0af7e0 0x7f273c14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 -- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f273c0affc0 msgr2=0x7f273c14e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f273c0affc0 0x7f273c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 -- 172.21.2.116:0/1074249808 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273400b710 con 0x7f273c0ad780 2021-11-21T21:54:39.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f273bfff700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0ad780 0x7f273c0b4ed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f272c000ed0 tx=0x7f272c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f27397fa700 1 -- 172.21.2.116:0/1074249808 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f272c00ce50 con 0x7f273c0ad780 2021-11-21T21:54:39.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f27397fa700 1 -- 172.21.2.116:0/1074249808 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c0072e0 con 0x7f273c0ad780 2021-11-21T21:54:39.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c142d30 con 0x7f273c0ad780 2021-11-21T21:54:39.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c13ca70 con 0x7f273c0ad780 2021-11-21T21:54:39.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.606+0000 7f27397fa700 1 -- 172.21.2.116:0/1074249808 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f272c005610 con 0x7f273c0ad780 2021-11-21T21:54:39.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.610+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c0ac8a0 con 0x7f273c0ad780 2021-11-21T21:54:39.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.610+0000 7f27397fa700 1 -- 172.21.2.116:0/1074249808 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f272c01a060 con 0x7f273c0ad780 2021-11-21T21:54:39.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.610+0000 7f27397fa700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f27200535d0 0x7f2720055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.610+0000 7f27397fa700 1 -- 172.21.2.116:0/1074249808 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f272c06b380 con 0x7f273c0ad780 2021-11-21T21:54:39.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.610+0000 7f273b7fe700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f27200535d0 0x7f2720055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.610+0000 7f273b7fe700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f27200535d0 0x7f2720055a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f2724000ed0 tx=0x7f2724006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.610+0000 7f27397fa700 1 -- 172.21.2.116:0/1074249808 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f272c0423d0 con 0x7f273c0ad780 2021-11-21T21:54:39.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.694+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f273c0573d0 con 0x7f273c0ad780 2021-11-21T21:54:39.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.694+0000 7f27397fa700 1 -- 172.21.2.116:0/1074249808 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f272c018020 con 0x7f273c0ad780 2021-11-21T21:54:39.699 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:39.700 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":22,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:26.920796+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:23.490451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:23.890830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:39.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.694+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f27200535d0 msgr2=0x7f2720055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.694+0000 7f274236a700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f27200535d0 0x7f2720055a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f2724000ed0 tx=0x7f2724006930).stop 2021-11-21T21:54:39.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.694+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0ad780 msgr2=0x7f273c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.694+0000 7f274236a700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0ad780 0x7f273c0b4ed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f272c000ed0 tx=0x7f272c007ac0).stop 2021-11-21T21:54:39.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 shutdown_connections 2021-11-21T21:54:39.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f273c0affc0 0x7f273c14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f273c0af7e0 0x7f273c14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f27200535d0 0x7f2720055a90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 --2- 172.21.2.116:0/1074249808 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f273c0ad780 0x7f273c0b4ed0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 >> 172.21.2.116:0/1074249808 conn(0x7f273c0b6000 msgr2=0x7f273c05c500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 shutdown_connections 2021-11-21T21:54:39.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.698+0000 7f274236a700 1 -- 172.21.2.116:0/1074249808 wait complete. 2021-11-21T21:54:39.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:23.490451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:23.890830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:39.707 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:39.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.754+0000 7f47ced7e700 1 Processor -- start 2021-11-21T21:54:39.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.754+0000 7f47ced7e700 1 -- start start 2021-11-21T21:54:39.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.754+0000 7f47ced7e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f47c8144810 0x7f47c81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.754+0000 7f47ced7e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 0x7f47c80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c8147e80 con 0x7f47c80b1db0 2021-11-21T21:54:39.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c80b1490 con 0x7f47c8144810 2021-11-21T21:54:39.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80af060 con 0x7f47c80abce0 2021-11-21T21:54:39.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f47c8144810 0x7f47c81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f47c8144810 0x7f47c81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45484/0 (socket says 172.21.2.116:45484) 2021-11-21T21:54:39.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 -- 172.21.2.116:0/4237374563 learned_addr learned my addr 172.21.2.116:0/4237374563 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 -- 172.21.2.116:0/4237374563 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 msgr2=0x7f47c80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:39.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 --2- 172.21.2.116:0/4237374563 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 0x7f47c80b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 -- 172.21.2.116:0/4237374563 >> v1:172.21.2.117:6789/0 conn(0x7f47c80b1db0 legacy=0x7f47c80ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:39.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 -- 172.21.2.116:0/4237374563 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80af5a0 con 0x7f47c8144810 2021-11-21T21:54:39.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ccb1a700 1 --2- 172.21.2.116:0/4237374563 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c81478c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f47c80b2a30 tx=0x7f47b0006eb0).ready entity=mon.0 client_cookie=bd90cfb6e6297f89 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c77fe700 1 -- 172.21.2.116:0/4237374563 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b000d570 con 0x7f47c8144810 2021-11-21T21:54:39.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c77fe700 1 -- 172.21.2.116:0/4237374563 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0000e90 con 0x7f47c8144810 2021-11-21T21:54:39.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c77fe700 1 -- 172.21.2.116:0/4237374563 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b000d8c0 con 0x7f47c8144810 2021-11-21T21:54:39.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- 172.21.2.116:0/4237374563 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 msgr2=0x7f47c81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 --2- 172.21.2.116:0/4237374563 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c81478c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f47c80b2a30 tx=0x7f47b0006eb0).stop 2021-11-21T21:54:39.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- 172.21.2.116:0/4237374563 shutdown_connections 2021-11-21T21:54:39.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 --2- 172.21.2.116:0/4237374563 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 0x7f47c80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 --2- 172.21.2.116:0/4237374563 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c81478c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- 172.21.2.116:0/4237374563 >> 172.21.2.116:0/4237374563 conn(0x7f47c805bd20 msgr2=0x7f47c805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- 172.21.2.116:0/4237374563 shutdown_connections 2021-11-21T21:54:39.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- 172.21.2.116:0/4237374563 wait complete. 2021-11-21T21:54:39.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 Processor -- start 2021-11-21T21:54:39.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- start start 2021-11-21T21:54:39.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 0x7f47c80b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c80b4890 0x7f47c813a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c813aeb0 con 0x7f47c8144810 2021-11-21T21:54:39.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c813ce30 con 0x7f47c80b4890 2021-11-21T21:54:39.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c813c600 con 0x7f47c80abce0 2021-11-21T21:54:39.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c813a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45494/0 (socket says 172.21.2.116:45494) 2021-11-21T21:54:39.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 -- 172.21.2.116:0/1915148570 learned_addr learned my addr 172.21.2.116:0/1915148570 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 -- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 msgr2=0x7f47c80b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 0x7f47c80b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 -- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c80b4890 msgr2=0x7f47c813a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c80b4890 0x7f47c813a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 -- 172.21.2.116:0/1915148570 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b000b1e0 con 0x7f47c8144810 2021-11-21T21:54:39.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c7fff700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c813a4d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f47c80b3100 tx=0x7f47b8006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c5ffb700 1 -- 172.21.2.116:0/1915148570 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b8013460 con 0x7f47c8144810 2021-11-21T21:54:39.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c5ffb700 1 -- 172.21.2.116:0/1915148570 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b80135c0 con 0x7f47c8144810 2021-11-21T21:54:39.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c813c360 con 0x7f47c8144810 2021-11-21T21:54:39.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c813cab0 con 0x7f47c8144810 2021-11-21T21:54:39.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.758+0000 7f47c5ffb700 1 -- 172.21.2.116:0/1915148570 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b8013920 con 0x7f47c8144810 2021-11-21T21:54:39.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.762+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c81567f0 con 0x7f47c8144810 2021-11-21T21:54:39.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.762+0000 7f47c5ffb700 1 -- 172.21.2.116:0/1915148570 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f47b800f070 con 0x7f47c8144810 2021-11-21T21:54:39.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.762+0000 7f47c5ffb700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47ac0536f0 0x7f47ac055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.762+0000 7f47c5ffb700 1 -- 172.21.2.116:0/1915148570 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f47b806b3a0 con 0x7f47c8144810 2021-11-21T21:54:39.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.762+0000 7f47ccb1a700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47ac0536f0 0x7f47ac055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.762+0000 7f47ccb1a700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47ac0536f0 0x7f47ac055bb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f47c80b2a30 tx=0x7f47b0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.762+0000 7f47c5ffb700 1 -- 172.21.2.116:0/1915148570 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f47b8042500 con 0x7f47c8144810 2021-11-21T21:54:39.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f47c80af300 con 0x7f47c8144810 2021-11-21T21:54:39.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47c5ffb700 1 -- 172.21.2.116:0/1915148570 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f47b8045b20 con 0x7f47c8144810 2021-11-21T21:54:39.849 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:39.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47ac0536f0 msgr2=0x7f47ac055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47ac0536f0 0x7f47ac055bb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f47c80b2a30 tx=0x7f47b0015000).stop 2021-11-21T21:54:39.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 msgr2=0x7f47c813a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c813a4d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f47c80b3100 tx=0x7f47b8006de0).stop 2021-11-21T21:54:39.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 shutdown_connections 2021-11-21T21:54:39.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f47c80b4890 0x7f47c813a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f47ac0536f0 0x7f47ac055bb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f47c8144810 0x7f47c813a4d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 --2- 172.21.2.116:0/1915148570 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f47c80abce0 0x7f47c80b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 >> 172.21.2.116:0/1915148570 conn(0x7f47c805bd20 msgr2=0x7f47c80b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 shutdown_connections 2021-11-21T21:54:39.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.846+0000 7f47ced7e700 1 -- 172.21.2.116:0/1915148570 wait complete. 2021-11-21T21:54:39.856 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:39.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 Processor -- start 2021-11-21T21:54:39.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- start start 2021-11-21T21:54:39.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb8b40b6770 0x7fb8b40b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b40b0070 con 0x7fb8b40ad580 2021-11-21T21:54:39.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b40b05c0 con 0x7fb8b40b1270 2021-11-21T21:54:39.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b40b2230 con 0x7fb8b40b6770 2021-11-21T21:54:39.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b37fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb8b40ad580 0x7fb8b40ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57458/0 (socket says 172.21.2.116:57458) 2021-11-21T21:54:39.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b37fe700 1 -- 172.21.2.116:0/1167051367 learned_addr learned my addr 172.21.2.116:0/1167051367 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:39.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734339788 0 0) 0x7fb8b40b0070 con 0x7fb8b40ad580 2021-11-21T21:54:39.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb898003260 con 0x7fb8b40ad580 2021-11-21T21:54:39.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4101344709 0 0) 0x7fb898003260 con 0x7fb8b40ad580 2021-11-21T21:54:39.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 >> v2:172.21.2.117:3300/0 conn(0x7fb8b40b6770 msgr2=0x7fb8b40b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 --2- 172.21.2.116:0/1167051367 >> v2:172.21.2.117:3300/0 conn(0x7fb8b40b6770 0x7fb8b40b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 >> v1:172.21.2.117:6789/0 conn(0x7fb8b40b1270 legacy=0x7fb8b41394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b40b24d0 con 0x7fb8b40ad580 2021-11-21T21:54:39.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb8ac0019e0 con 0x7fb8b40ad580 2021-11-21T21:54:39.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 >> v1:172.21.2.116:6789/0 conn(0x7fb8b40ad580 legacy=0x7fb8b40ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb898003820 0x7fb898003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb898004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb898005050 0x7fb89800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40b2230 con 0x7fb898004500 2021-11-21T21:54:39.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89800bf40 con 0x7fb898003820 2021-11-21T21:54:39.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89800c100 con 0x7fb898005050 2021-11-21T21:54:39.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b27fc700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb898004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b27fc700 1 -- 172.21.2.116:0/1167051367 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb898005050 msgr2=0x7fb89800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b27fc700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb898005050 0x7fb89800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b27fc700 1 -- 172.21.2.116:0/1167051367 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb898003820 msgr2=0x7fb898003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:39.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b27fc700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb898003820 0x7fb898003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b37fe700 1 -- 172.21.2.116:0/1167051367 reap_dead start 2021-11-21T21:54:39.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b27fc700 1 -- 172.21.2.116:0/1167051367 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898003260 con 0x7fb898004500 2021-11-21T21:54:39.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b27fc700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb898004980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fb8a4000f70 tx=0x7fb8a40079e0).ready entity=mon.0 client_cookie=cb754ac955301bef server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a4007cf0 con 0x7fb898004500 2021-11-21T21:54:39.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a4014410 con 0x7fb898004500 2021-11-21T21:54:39.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b1ffb700 1 -- 172.21.2.116:0/1167051367 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a4014770 con 0x7fb898004500 2021-11-21T21:54:39.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1167051367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 msgr2=0x7fb898004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb898004980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fb8a4000f70 tx=0x7fb8a40079e0).stop 2021-11-21T21:54:39.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1167051367 shutdown_connections 2021-11-21T21:54:39.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1167051367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb898004980 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:39.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1167051367 >> 172.21.2.116:0/1167051367 conn(0x7fb8b4062040 msgr2=0x7fb8b40b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:39.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1167051367 shutdown_connections 2021-11-21T21:54:39.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.906+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1167051367 wait complete. 2021-11-21T21:54:39.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 Processor -- start 2021-11-21T21:54:39.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 -- start start 2021-11-21T21:54:39.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb8b40b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b4141fd0 0x7fb8b414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b4152b50 0x7fb8b414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b4142a70 con 0x7fb898004500 2021-11-21T21:54:39.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b4152410 con 0x7fb8b4141fd0 2021-11-21T21:54:39.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b41526b0 con 0x7fb8b4152b50 2021-11-21T21:54:39.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb8b40b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb8b40b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45510/0 (socket says 172.21.2.116:45510) 2021-11-21T21:54:39.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 -- 172.21.2.116:0/1221955069 learned_addr learned my addr 172.21.2.116:0/1221955069 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:39.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 -- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b4152b50 msgr2=0x7fb8b414bed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:39.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b4152b50 0x7fb8b414bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 -- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b4141fd0 msgr2=0x7fb8b414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:39.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b4141fd0 0x7fb8b414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:39.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 -- 172.21.2.116:0/1221955069 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4142d10 con 0x7fb898004500 2021-11-21T21:54:39.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b2ffd700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb8b40b5120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb8a8002f70 tx=0x7fb8a8006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8a3fff700 1 -- 172.21.2.116:0/1221955069 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a8009e80 con 0x7fb898004500 2021-11-21T21:54:39.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8a3fff700 1 -- 172.21.2.116:0/1221955069 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8013460 con 0x7fb898004500 2021-11-21T21:54:39.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b41426f0 con 0x7fb898004500 2021-11-21T21:54:39.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b4152fd0 con 0x7fb898004500 2021-11-21T21:54:39.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8a3fff700 1 -- 172.21.2.116:0/1221955069 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a80137c0 con 0x7fb898004500 2021-11-21T21:54:39.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b4152870 con 0x7fb898004500 2021-11-21T21:54:39.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8a3fff700 1 -- 172.21.2.116:0/1221955069 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb8a801a060 con 0x7fb898004500 2021-11-21T21:54:39.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8a3fff700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb8900536d0 0x7fb890055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:39.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8a3fff700 1 -- 172.21.2.116:0/1221955069 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb8a806b300 con 0x7fb898004500 2021-11-21T21:54:39.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b27fc700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb8900536d0 0x7fb890055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:39.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.910+0000 7fb8b27fc700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb8900536d0 0x7fb890055b90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb8a4000f70 tx=0x7fb8a4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:39.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.914+0000 7fb8a3fff700 1 -- 172.21.2.116:0/1221955069 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8a8042350 con 0x7fb898004500 2021-11-21T21:54:40.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb8b40573d0 con 0x7fb898004500 2021-11-21T21:54:40.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8a3fff700 1 -- 172.21.2.116:0/1221955069 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb8a8018020 con 0x7fb898004500 2021-11-21T21:54:40.001 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:40.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb8900536d0 msgr2=0x7fb890055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb8900536d0 0x7fb890055b90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb8a4000f70 tx=0x7fb8a4015000).stop 2021-11-21T21:54:40.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 msgr2=0x7fb8b40b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb8b40b5120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb8a8002f70 tx=0x7fb8a8006de0).stop 2021-11-21T21:54:40.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 shutdown_connections 2021-11-21T21:54:40.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b4152b50 0x7fb8b414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b4141fd0 0x7fb8b414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb8900536d0 0x7fb890055b90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 --2- 172.21.2.116:0/1221955069 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb898004500 0x7fb8b40b5120 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 >> 172.21.2.116:0/1221955069 conn(0x7fb8b4062040 msgr2=0x7fb8b413c800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:40.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 shutdown_connections 2021-11-21T21:54:40.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:39.998+0000 7fb8b99e9700 1 -- 172.21.2.116:0/1221955069 wait complete. 2021-11-21T21:54:40.010 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:40.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.058+0000 7f28a7f98700 1 Processor -- start 2021-11-21T21:54:40.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.058+0000 7f28a7f98700 1 -- start start 2021-11-21T21:54:40.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f28a01426e0 0x7f28a0142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f28a0062e60 0x7f28a00af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a00b0f60 con 0x7f28a00ad470 2021-11-21T21:54:40.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a0063470 con 0x7f28a0062e60 2021-11-21T21:54:40.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a0143050 con 0x7f28a01426e0 2021-11-21T21:54:40.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a5533700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f28a0062e60 0x7f28a00af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f28a00ad470 0x7f28a00ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57476/0 (socket says 172.21.2.116:57476) 2021-11-21T21:54:40.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 -- 172.21.2.116:0/2204746922 learned_addr learned my addr 172.21.2.116:0/2204746922 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:40.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a5533700 1 -- 172.21.2.116:0/2204746922 >> v2:172.21.2.117:3300/0 conn(0x7f28a01426e0 msgr2=0x7f28a0142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:40.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a5533700 1 --2- 172.21.2.116:0/2204746922 >> v2:172.21.2.117:3300/0 conn(0x7f28a01426e0 0x7f28a0142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a5533700 1 -- 172.21.2.116:0/2204746922 >> v1:172.21.2.116:6789/0 conn(0x7f28a00ad470 legacy=0x7f28a00ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a5533700 1 -- 172.21.2.116:0/2204746922 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a00adfc0 con 0x7f28a0062e60 2021-11-21T21:54:40.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a5533700 1 --2- 172.21.2.116:0/2204746922 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a0062e60 0x7f28a00af3d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f2894002f70 tx=0x7f2894006df0).ready entity=mon.0 client_cookie=9fc151d39ac392b2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a4d32700 1 -- 172.21.2.116:0/2204746922 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f289400d5c0 con 0x7f28a0062e60 2021-11-21T21:54:40.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a4d32700 1 -- 172.21.2.116:0/2204746922 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2894000e90 con 0x7f28a0062e60 2021-11-21T21:54:40.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a4d32700 1 -- 172.21.2.116:0/2204746922 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f289400d950 con 0x7f28a0062e60 2021-11-21T21:54:40.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- 172.21.2.116:0/2204746922 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a0062e60 msgr2=0x7f28a00af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- 172.21.2.116:0/2204746922 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a0062e60 0x7f28a00af3d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f2894002f70 tx=0x7f2894006df0).stop 2021-11-21T21:54:40.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- 172.21.2.116:0/2204746922 shutdown_connections 2021-11-21T21:54:40.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- 172.21.2.116:0/2204746922 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a0062e60 0x7f28a00af3d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- 172.21.2.116:0/2204746922 >> v2:172.21.2.117:3300/0 conn(0x7f28a01426e0 0x7f28a0142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- 172.21.2.116:0/2204746922 >> 172.21.2.116:0/2204746922 conn(0x7f28a00b5730 msgr2=0x7f28a00b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:40.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- 172.21.2.116:0/2204746922 shutdown_connections 2021-11-21T21:54:40.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- 172.21.2.116:0/2204746922 wait complete. 2021-11-21T21:54:40.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 Processor -- start 2021-11-21T21:54:40.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- start start 2021-11-21T21:54:40.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28a00ad470 0x7f28a0125850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28a01426e0 0x7f28a0122a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a013fcb0 0x7f28a0122f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a005d070 con 0x7f28a013fcb0 2021-11-21T21:54:40.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0066e70 con 0x7f28a00ad470 2021-11-21T21:54:40.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0067110 con 0x7f28a01426e0 2021-11-21T21:54:40.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a013fcb0 0x7f28a0122f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a013fcb0 0x7f28a0122f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45526/0 (socket says 172.21.2.116:45526) 2021-11-21T21:54:40.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 -- 172.21.2.116:0/95905014 learned_addr learned my addr 172.21.2.116:0/95905014 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:40.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 -- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28a01426e0 msgr2=0x7f28a0122a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28a01426e0 0x7f28a0122a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 -- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28a00ad470 msgr2=0x7f28a0125850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28a00ad470 0x7f28a0125850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 -- 172.21.2.116:0/95905014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289400b180 con 0x7f28a013fcb0 2021-11-21T21:54:40.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a6535700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a013fcb0 0x7f28a0122f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f28a0062d10 tx=0x7f2898007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f2892ffd700 1 -- 172.21.2.116:0/95905014 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2898005b80 con 0x7f28a013fcb0 2021-11-21T21:54:40.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f2892ffd700 1 -- 172.21.2.116:0/95905014 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2898005ce0 con 0x7f28a013fcb0 2021-11-21T21:54:40.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f2892ffd700 1 -- 172.21.2.116:0/95905014 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2898007300 con 0x7f28a013fcb0 2021-11-21T21:54:40.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a00673b0 con 0x7f28a013fcb0 2021-11-21T21:54:40.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.062+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a0125d40 con 0x7f28a013fcb0 2021-11-21T21:54:40.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.066+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a00b1e00 con 0x7f28a013fcb0 2021-11-21T21:54:40.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.066+0000 7f2892ffd700 1 -- 172.21.2.116:0/95905014 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2898010030 con 0x7f28a013fcb0 2021-11-21T21:54:40.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.066+0000 7f2892ffd700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2880053610 0x7f2880055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.066+0000 7f2892ffd700 1 -- 172.21.2.116:0/95905014 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f289806b970 con 0x7f28a013fcb0 2021-11-21T21:54:40.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.066+0000 7f28a5d34700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2880053610 0x7f2880055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.066+0000 7f28a5d34700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2880053610 0x7f2880055ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f289c002ef0 tx=0x7f289c006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.066+0000 7f2892ffd700 1 -- 172.21.2.116:0/95905014 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28980429c0 con 0x7f28a013fcb0 2021-11-21T21:54:40.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28a00573d0 con 0x7f28a013fcb0 2021-11-21T21:54:40.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f2892ffd700 1 -- 172.21.2.116:0/95905014 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2898045fe0 con 0x7f28a013fcb0 2021-11-21T21:54:40.154 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:40.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2880053610 msgr2=0x7f2880055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2880053610 0x7f2880055ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f289c002ef0 tx=0x7f289c006910).stop 2021-11-21T21:54:40.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a013fcb0 msgr2=0x7f28a0122f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a013fcb0 0x7f28a0122f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f28a0062d10 tx=0x7f2898007ef0).stop 2021-11-21T21:54:40.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 shutdown_connections 2021-11-21T21:54:40.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2880053610 0x7f2880055ad0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28a013fcb0 0x7f28a0122f50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28a01426e0 0x7f28a0122a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 --2- 172.21.2.116:0/95905014 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28a00ad470 0x7f28a0125850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 >> 172.21.2.116:0/95905014 conn(0x7f28a00b5730 msgr2=0x7f28a00af360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:40.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 shutdown_connections 2021-11-21T21:54:40.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.150+0000 7f28a7f98700 1 -- 172.21.2.116:0/95905014 wait complete. 2021-11-21T21:54:40.161 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:40.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff897ec7700 1 Processor -- start 2021-11-21T21:54:40.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff897ec7700 1 -- start start 2021-11-21T21:54:40.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff897ec7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff8900abce0 0x7ff8901478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff897ec7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff8900aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff897ec7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff890147e80 con 0x7ff890144810 2021-11-21T21:54:40.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff897ec7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8900b1490 con 0x7ff8900abce0 2021-11-21T21:54:40.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff897ec7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900b1fd0 con 0x7ff8900ac600 2021-11-21T21:54:40.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff895462700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff8900abce0 0x7ff8901478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff896464700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff8900aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff895c63700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7ff890144810 0x7ff8900b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:56398/0 (socket says 172.21.2.116:56398) 2021-11-21T21:54:40.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff896464700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff8900aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:48876/0 (socket says 172.21.2.116:48876) 2021-11-21T21:54:40.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff895462700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff8900abce0 0x7ff8901478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:46704/0 (socket says 172.21.2.116:46704) 2021-11-21T21:54:40.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.210+0000 7ff895c63700 1 -- 172.21.2.116:0/218634341 learned_addr learned my addr 172.21.2.116:0/218634341 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:40.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff895462700 1 -- 172.21.2.116:0/218634341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 msgr2=0x7ff8900aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:40.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff895462700 1 --2- 172.21.2.116:0/218634341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff8900aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff895462700 1 -- 172.21.2.116:0/218634341 >> v1:172.21.2.117:6789/0 conn(0x7ff890144810 legacy=0x7ff8900b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff895462700 1 -- 172.21.2.116:0/218634341 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900af5a0 con 0x7ff8900abce0 2021-11-21T21:54:40.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff895462700 1 --2- 172.21.2.116:0/218634341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 0x7ff8901478c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff88c002f70 tx=0x7ff88c006df0).ready entity=mon.1 client_cookie=e5ce7d1517b5a4ee server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff894c61700 1 -- 172.21.2.116:0/218634341 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88c00d570 con 0x7ff8900abce0 2021-11-21T21:54:40.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff894c61700 1 -- 172.21.2.116:0/218634341 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c000e90 con 0x7ff8900abce0 2021-11-21T21:54:40.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff894c61700 1 -- 172.21.2.116:0/218634341 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88c00d8c0 con 0x7ff8900abce0 2021-11-21T21:54:40.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- 172.21.2.116:0/218634341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 msgr2=0x7ff8901478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 --2- 172.21.2.116:0/218634341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 0x7ff8901478c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff88c002f70 tx=0x7ff88c006df0).stop 2021-11-21T21:54:40.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- 172.21.2.116:0/218634341 shutdown_connections 2021-11-21T21:54:40.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 --2- 172.21.2.116:0/218634341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff8900aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 --2- 172.21.2.116:0/218634341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 0x7ff8901478c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- 172.21.2.116:0/218634341 >> 172.21.2.116:0/218634341 conn(0x7ff89005bd20 msgr2=0x7ff89005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:40.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- 172.21.2.116:0/218634341 shutdown_connections 2021-11-21T21:54:40.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- 172.21.2.116:0/218634341 wait complete. 2021-11-21T21:54:40.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 Processor -- start 2021-11-21T21:54:40.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- start start 2021-11-21T21:54:40.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 0x7ff890143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff89015a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff890144810 0x7ff89015a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890145e40 con 0x7ff890144810 2021-11-21T21:54:40.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890146310 con 0x7ff8900abce0 2021-11-21T21:54:40.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff897ec7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900b4b00 con 0x7ff8900ac600 2021-11-21T21:54:40.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff890144810 0x7ff89015a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff890144810 0x7ff89015a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45542/0 (socket says 172.21.2.116:45542) 2021-11-21T21:54:40.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 -- 172.21.2.116:0/2282979194 learned_addr learned my addr 172.21.2.116:0/2282979194 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:40.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff895c63700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 0x7ff890143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff895462700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff89015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 -- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 msgr2=0x7ff89015a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff89015a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 -- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 msgr2=0x7ff890143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 0x7ff890143900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.214+0000 7ff896464700 1 -- 172.21.2.116:0/2282979194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c00b180 con 0x7ff890144810 2021-11-21T21:54:40.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff896464700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff890144810 0x7ff89015a640 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ff888006e00 tx=0x7ff8880091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff882ffd700 1 -- 172.21.2.116:0/2282979194 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88800c850 con 0x7ff890144810 2021-11-21T21:54:40.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff882ffd700 1 -- 172.21.2.116:0/2282979194 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88800c9b0 con 0x7ff890144810 2021-11-21T21:54:40.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff882ffd700 1 -- 172.21.2.116:0/2282979194 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88800cd10 con 0x7ff890144810 2021-11-21T21:54:40.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff890146070 con 0x7ff890144810 2021-11-21T21:54:40.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff890064470 con 0x7ff890144810 2021-11-21T21:54:40.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff890146690 con 0x7ff890144810 2021-11-21T21:54:40.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff882ffd700 1 -- 172.21.2.116:0/2282979194 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff8880123f0 con 0x7ff890144810 2021-11-21T21:54:40.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff882ffd700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff870053580 0x7ff870055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff882ffd700 1 -- 172.21.2.116:0/2282979194 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff88806bdb0 con 0x7ff890144810 2021-11-21T21:54:40.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff895c63700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff870053580 0x7ff870055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.218+0000 7ff895c63700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff870053580 0x7ff870055a40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff884008000 tx=0x7ff884009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.222+0000 7ff882ffd700 1 -- 172.21.2.116:0/2282979194 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff888070070 con 0x7ff890144810 2021-11-21T21:54:40.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.302+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8900af060 con 0x7ff890144810 2021-11-21T21:54:40.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.302+0000 7ff882ffd700 1 -- 172.21.2.116:0/2282979194 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff888042da0 con 0x7ff890144810 2021-11-21T21:54:40.307 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:40.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff870053580 msgr2=0x7ff870055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff870053580 0x7ff870055a40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff884008000 tx=0x7ff884009040).stop 2021-11-21T21:54:40.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff890144810 msgr2=0x7ff89015a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff890144810 0x7ff89015a640 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ff888006e00 tx=0x7ff8880091d0).stop 2021-11-21T21:54:40.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 shutdown_connections 2021-11-21T21:54:40.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff870053580 0x7ff870055a40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff890144810 0x7ff89015a640 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8900ac600 0x7ff89015a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 --2- 172.21.2.116:0/2282979194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8900abce0 0x7ff890143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 >> 172.21.2.116:0/2282979194 conn(0x7ff89005bd20 msgr2=0x7ff890148720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:40.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 shutdown_connections 2021-11-21T21:54:40.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.306+0000 7ff897ec7700 1 -- 172.21.2.116:0/2282979194 wait complete. 2021-11-21T21:54:40.322 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2021-11-21T21:54:40.323 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2021-11-21T21:54:40.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd618e11700 1 Processor -- start 2021-11-21T21:54:40.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd618e11700 1 -- start start 2021-11-21T21:54:40.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd618e11700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd6140b09c0 0x7fd61413d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd618e11700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd614060c80 con 0x7fd6140aeb90 2021-11-21T21:54:40.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd618e11700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6140aef70 con 0x7fd6140b16a0 2021-11-21T21:54:40.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd618e11700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61413f9c0 con 0x7fd6140b09c0 2021-11-21T21:54:40.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd611d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd6140b09c0 0x7fd61413d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd61259c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd6140aeb90 0x7fd61413cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57502/0 (socket says 172.21.2.116:57502) 2021-11-21T21:54:40.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd61259c700 1 -- 172.21.2.116:0/2945121493 learned_addr learned my addr 172.21.2.116:0/2945121493 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:40.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.370+0000 7fd611d9b700 1 --2- 172.21.2.116:0/2945121493 >> v2:172.21.2.116:3300/0 conn(0x7fd6140b09c0 0x7fd61413d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45548/0 (socket says 172.21.2.116:45548) 2021-11-21T21:54:40.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 -- 172.21.2.116:0/2945121493 >> v1:172.21.2.117:6789/0 conn(0x7fd6140b16a0 legacy=0x7fd61413fd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:40.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 -- 172.21.2.116:0/2945121493 >> v1:172.21.2.116:6789/0 conn(0x7fd6140aeb90 legacy=0x7fd61413cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 -- 172.21.2.116:0/2945121493 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6140609e0 con 0x7fd6140b09c0 2021-11-21T21:54:40.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 --2- 172.21.2.116:0/2945121493 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd61413d350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fd6140634b0 tx=0x7fd5fc006df0).ready entity=mon.0 client_cookie=f66b3f969c8e33e7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd61159a700 1 -- 172.21.2.116:0/2945121493 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5fc014070 con 0x7fd6140b09c0 2021-11-21T21:54:40.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd61159a700 1 -- 172.21.2.116:0/2945121493 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc000e90 con 0x7fd6140b09c0 2021-11-21T21:54:40.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd61159a700 1 -- 172.21.2.116:0/2945121493 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5fc00fbb0 con 0x7fd6140b09c0 2021-11-21T21:54:40.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2945121493 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 msgr2=0x7fd61413d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:40.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 --2- 172.21.2.116:0/2945121493 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd61413d350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fd6140634b0 tx=0x7fd5fc006df0).stop 2021-11-21T21:54:40.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2945121493 shutdown_connections 2021-11-21T21:54:40.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 --2- 172.21.2.116:0/2945121493 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd61413d350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:40.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2945121493 >> 172.21.2.116:0/2945121493 conn(0x7fd614063bb0 msgr2=0x7fd614063f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:40.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2945121493 shutdown_connections 2021-11-21T21:54:40.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2945121493 wait complete. 2021-11-21T21:54:40.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 Processor -- start 2021-11-21T21:54:40.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- start start 2021-11-21T21:54:40.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6140aeb90 0x7fd6140b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd614066410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6140b16a0 0x7fd614064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd614065fa0 con 0x7fd6140b09c0 2021-11-21T21:54:40.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6141392d0 con 0x7fd6140aeb90 2021-11-21T21:54:40.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd614143700 con 0x7fd6140b16a0 2021-11-21T21:54:40.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd614066410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd614066410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45552/0 (socket says 172.21.2.116:45552) 2021-11-21T21:54:40.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 -- 172.21.2.116:0/2861618694 learned_addr learned my addr 172.21.2.116:0/2861618694 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:40.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 -- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6140b16a0 msgr2=0x7fd614064840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:40.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6140b16a0 0x7fd614064840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 -- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6140aeb90 msgr2=0x7fd6140b4ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:40.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6140aeb90 0x7fd6140b4ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:40.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 -- 172.21.2.116:0/2861618694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc00b180 con 0x7fd6140b09c0 2021-11-21T21:54:40.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd611d9b700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd614066410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd6140634b0 tx=0x7fd5fc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd60b7fe700 1 -- 172.21.2.116:0/2861618694 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5fc00e1d0 con 0x7fd6140b09c0 2021-11-21T21:54:40.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd60b7fe700 1 -- 172.21.2.116:0/2861618694 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc002f50 con 0x7fd6140b09c0 2021-11-21T21:54:40.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6140b53e0 con 0x7fd6140b09c0 2021-11-21T21:54:40.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd60b7fe700 1 -- 172.21.2.116:0/2861618694 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5fc01c840 con 0x7fd6140b09c0 2021-11-21T21:54:40.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd614066aa0 con 0x7fd6140b09c0 2021-11-21T21:54:40.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.374+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6140661a0 con 0x7fd6140b09c0 2021-11-21T21:54:40.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.378+0000 7fd60b7fe700 1 -- 172.21.2.116:0/2861618694 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd5fc022020 con 0x7fd6140b09c0 2021-11-21T21:54:40.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.378+0000 7fd60b7fe700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd5f80536d0 0x7fd5f8055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:40.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.378+0000 7fd60b7fe700 1 -- 172.21.2.116:0/2861618694 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd5fc06d4b0 con 0x7fd6140b09c0 2021-11-21T21:54:40.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.378+0000 7fd61259c700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd5f80536d0 0x7fd5f8055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:40.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.378+0000 7fd61259c700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd5f80536d0 0x7fd5f8055b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd614063470 tx=0x7fd604007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:40.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.378+0000 7fd60b7fe700 1 -- 172.21.2.116:0/2861618694 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5fc044610 con 0x7fd6140b09c0 2021-11-21T21:54:40.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:40.490+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7fd6140661a0 con 0x7fd6140b09c0 2021-11-21T21:54:41.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.034+0000 7fd60b7fe700 1 -- 172.21.2.116:0/2861618694 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v29) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd5fc047c30 con 0x7fd6140b09c0 2021-11-21T21:54:41.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd5f80536d0 msgr2=0x7fd5f8055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd5f80536d0 0x7fd5f8055b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd614063470 tx=0x7fd604007790).stop 2021-11-21T21:54:41.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 msgr2=0x7fd614066410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd614066410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd6140634b0 tx=0x7fd5fc014000).stop 2021-11-21T21:54:41.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 shutdown_connections 2021-11-21T21:54:41.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6140b16a0 0x7fd614064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd5f80536d0 0x7fd5f8055b90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd6140b09c0 0x7fd614066410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 --2- 172.21.2.116:0/2861618694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6140aeb90 0x7fd6140b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 >> 172.21.2.116:0/2861618694 conn(0x7fd614063bb0 msgr2=0x7fd61413ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:41.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 shutdown_connections 2021-11-21T21:54:41.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.038+0000 7fd618e11700 1 -- 172.21.2.116:0/2861618694 wait complete. 2021-11-21T21:54:41.047 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T21:54:41.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.102+0000 7f10a8c02700 1 Processor -- start 2021-11-21T21:54:41.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.102+0000 7f10a8c02700 1 -- start start 2021-11-21T21:54:41.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.102+0000 7f10a8c02700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a40adc10 0x7f10a40b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.102+0000 7f10a8c02700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f10a4144a10 0x7f10a4141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.102+0000 7f10a8c02700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40b3c50 con 0x7f10a4141510 2021-11-21T21:54:41.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.102+0000 7f10a8c02700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40b3ef0 con 0x7f10a4144a10 2021-11-21T21:54:41.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.102+0000 7f10a8c02700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b4190 con 0x7f10a40adc10 2021-11-21T21:54:41.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f10a4144a10 0x7f10a4141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:41.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f10a4144a10 0x7f10a4141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45562/0 (socket says 172.21.2.116:45562) 2021-11-21T21:54:41.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 -- 172.21.2.116:0/3555132386 learned_addr learned my addr 172.21.2.116:0/3555132386 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:41.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 -- 172.21.2.116:0/3555132386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a40adc10 msgr2=0x7f10a40b2030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:41.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 --2- 172.21.2.116:0/3555132386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a40adc10 0x7f10a40b2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 -- 172.21.2.116:0/3555132386 >> v1:172.21.2.116:6789/0 conn(0x7f10a4141510 legacy=0x7f10a413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 -- 172.21.2.116:0/3555132386 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a4140b80 con 0x7f10a4144a10 2021-11-21T21:54:41.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a1d9b700 1 --2- 172.21.2.116:0/3555132386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a4144a10 0x7f10a4141020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f1094002f70 tx=0x7f1094006eb0).ready entity=mon.0 client_cookie=801f7f65a74e12ce server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:41.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a159a700 1 -- 172.21.2.116:0/3555132386 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109400d5c0 con 0x7f10a4144a10 2021-11-21T21:54:41.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a159a700 1 -- 172.21.2.116:0/3555132386 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1094000e90 con 0x7f10a4144a10 2021-11-21T21:54:41.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a159a700 1 -- 172.21.2.116:0/3555132386 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109400d950 con 0x7f10a4144a10 2021-11-21T21:54:41.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- 172.21.2.116:0/3555132386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a4144a10 msgr2=0x7f10a4141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3555132386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a4144a10 0x7f10a4141020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f1094002f70 tx=0x7f1094006eb0).stop 2021-11-21T21:54:41.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- 172.21.2.116:0/3555132386 shutdown_connections 2021-11-21T21:54:41.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3555132386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a4144a10 0x7f10a4141020 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3555132386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a40adc10 0x7f10a40b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- 172.21.2.116:0/3555132386 >> 172.21.2.116:0/3555132386 conn(0x7f10a405bd20 msgr2=0x7f10a405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:41.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- 172.21.2.116:0/3555132386 shutdown_connections 2021-11-21T21:54:41.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- 172.21.2.116:0/3555132386 wait complete. 2021-11-21T21:54:41.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 Processor -- start 2021-11-21T21:54:41.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- start start 2021-11-21T21:54:41.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a40adc10 0x7f10a4067860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a4141510 0x7f10a40ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f10a4144a10 0x7f10a4141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4155e10 con 0x7f10a40adc10 2021-11-21T21:54:41.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4114750 con 0x7f10a4144a10 2021-11-21T21:54:41.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41560b0 con 0x7f10a4141510 2021-11-21T21:54:41.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a40adc10 0x7f10a4067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:41.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a40adc10 0x7f10a4067860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45568/0 (socket says 172.21.2.116:45568) 2021-11-21T21:54:41.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 -- 172.21.2.116:0/3560245916 learned_addr learned my addr 172.21.2.116:0/3560245916 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:41.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 -- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a4141510 msgr2=0x7f10a40ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a4141510 0x7f10a40ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 -- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f10a4144a10 msgr2=0x7f10a4141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f10a4144a10 0x7f10a4141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 -- 172.21.2.116:0/3560245916 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109400b1e0 con 0x7f10a40adc10 2021-11-21T21:54:41.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a259c700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a40adc10 0x7f10a4067860 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f108c002fb0 tx=0x7f108c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:41.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f109b7fe700 1 -- 172.21.2.116:0/3560245916 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c013460 con 0x7f10a40adc10 2021-11-21T21:54:41.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f109b7fe700 1 -- 172.21.2.116:0/3560245916 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c0135c0 con 0x7f10a40adc10 2021-11-21T21:54:41.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a4143d80 con 0x7f10a40adc10 2021-11-21T21:54:41.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f109b7fe700 1 -- 172.21.2.116:0/3560245916 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c013920 con 0x7f10a40adc10 2021-11-21T21:54:41.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.106+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a4114d90 con 0x7f10a40adc10 2021-11-21T21:54:41.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.110+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a4155c50 con 0x7f10a40adc10 2021-11-21T21:54:41.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.110+0000 7f109b7fe700 1 -- 172.21.2.116:0/3560245916 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f108c00f070 con 0x7f10a40adc10 2021-11-21T21:54:41.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.110+0000 7f109b7fe700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10880535b0 0x7f1088055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.110+0000 7f109b7fe700 1 -- 172.21.2.116:0/3560245916 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f108c06b000 con 0x7f10a40adc10 2021-11-21T21:54:41.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.110+0000 7f10a1d9b700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10880535b0 0x7f1088055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:41.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.110+0000 7f10a1d9b700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10880535b0 0x7f1088055a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1094002f70 tx=0x7f1094015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:41.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.110+0000 7f109b7fe700 1 -- 172.21.2.116:0/3560245916 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f108c042160 con 0x7f10a40adc10 2021-11-21T21:54:41.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.194+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f10a4114920 con 0x7f10a40adc10 2021-11-21T21:54:41.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.198+0000 7f109b7fe700 1 -- 172.21.2.116:0/3560245916 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f108c018030 con 0x7f10a40adc10 2021-11-21T21:54:41.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10880535b0 msgr2=0x7f1088055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10880535b0 0x7f1088055a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1094002f70 tx=0x7f1094015000).stop 2021-11-21T21:54:41.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a40adc10 msgr2=0x7f10a4067860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a40adc10 0x7f10a4067860 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f108c002fb0 tx=0x7f108c006df0).stop 2021-11-21T21:54:41.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 shutdown_connections 2021-11-21T21:54:41.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f10a4144a10 0x7f10a4141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f10a4141510 0x7f10a40ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10880535b0 0x7f1088055a70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 --2- 172.21.2.116:0/3560245916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f10a40adc10 0x7f10a4067860 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 >> 172.21.2.116:0/3560245916 conn(0x7f10a405bd20 msgr2=0x7f10a413f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:41.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 shutdown_connections 2021-11-21T21:54:41.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.202+0000 7f10a8c02700 1 -- 172.21.2.116:0/3560245916 wait complete. 2021-11-21T21:54:41.211 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:54:41.212 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:54:41.220 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:54:41.229 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:54:41.243 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:54:41.257 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:54:41.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.258 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.259 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:41.276 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:54:41.277 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:54:41.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.278 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.279 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:41.329 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:54:41.331 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:54:41.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.332 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.332 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:41.341 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-11-21T21:54:41.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:41.343 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-11-21T21:54:41.343 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:54:41.343 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:54:41.344 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:41.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.406 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.407 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:41.413 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T21:54:41.415 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:54:41.416 INFO:teuthology.orchestra.run:waiting for 900 2021-11-21T21:54:41.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:54:41.417 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:41.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.476 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.476 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:41.487 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T21:54:41.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T21:54:41.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.490 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.490 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:41.501 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T21:54:41.503 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T21:54:41.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.504 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.504 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:41.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:54:41.514 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2021-11-21T21:54:41.514 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2021-11-21T21:54:41.515 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T21:54:41.515 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:54:41.516 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:41.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:41.589 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:41.590 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:41.599 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T21:54:41.600 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:54:41.601 INFO:teuthology.orchestra.run:waiting for 900 2021-11-21T21:54:41.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:54:41.602 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:54:41.616 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:54:41.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.666+0000 7ff2af111700 1 Processor -- start 2021-11-21T21:54:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.666+0000 7ff2af111700 1 -- start start 2021-11-21T21:54:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.666+0000 7ff2af111700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff2a813e480 0x7ff2a80b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.666+0000 7ff2af111700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a8144970 con 0x7ff2a80a9d00 2021-11-21T21:54:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.666+0000 7ff2af111700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a80aa0e0 con 0x7ff2a805c840 2021-11-21T21:54:41.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.666+0000 7ff2af111700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80b1200 con 0x7ff2a813e480 2021-11-21T21:54:41.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff2a80a9d00 0x7ff2a813dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57530/0 (socket says 172.21.2.116:57530) 2021-11-21T21:54:41.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 -- 172.21.2.116:0/1611229180 learned_addr learned my addr 172.21.2.116:0/1611229180 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:41.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776249591 0 0) 0x7ff2a8144970 con 0x7ff2a80a9d00 2021-11-21T21:54:41.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff288003260 con 0x7ff2a80a9d00 2021-11-21T21:54:41.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2ad6ae700 1 --2- 172.21.2.116:0/1611229180 >> v2:172.21.2.117:3300/0 conn(0x7ff2a813e480 0x7ff2a80b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:41.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1074861085 0 0) 0x7ff288003260 con 0x7ff2a80a9d00 2021-11-21T21:54:41.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 >> v2:172.21.2.117:3300/0 conn(0x7ff2a813e480 msgr2=0x7ff2a80b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:41.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 --2- 172.21.2.116:0/1611229180 >> v2:172.21.2.117:3300/0 conn(0x7ff2a813e480 0x7ff2a80b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 >> v1:172.21.2.117:6789/0 conn(0x7ff2a805c840 legacy=0x7ff2a80b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:41.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80abb80 con 0x7ff2a80a9d00 2021-11-21T21:54:41.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7ff2a00019e0 con 0x7ff2a80a9d00 2021-11-21T21:54:41.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 >> v1:172.21.2.116:6789/0 conn(0x7ff2a80a9d00 legacy=0x7ff2a813dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff288003820 0x7ff288003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff288004500 0x7ff288004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff288004f70 0x7ff28800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28800bec0 con 0x7ff288003820 2021-11-21T21:54:41.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28800c030 con 0x7ff288004500 2021-11-21T21:54:41.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28800c1f0 con 0x7ff288004f70 2021-11-21T21:54:41.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff288003820 0x7ff288003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:41.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 -- 172.21.2.116:0/1611229180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff288004f70 msgr2=0x7ff28800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:41.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff288004f70 0x7ff28800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 -- 172.21.2.116:0/1611229180 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff288004500 msgr2=0x7ff288004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:41.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff288004500 0x7ff288004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 -- 172.21.2.116:0/1611229180 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288003260 con 0x7ff288003820 2021-11-21T21:54:41.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2ad6ae700 1 -- 172.21.2.116:0/1611229180 reap_dead start 2021-11-21T21:54:41.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2acead700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff288003820 0x7ff288003c20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff2a8061b50 tx=0x7ff2a80a9520).ready entity=mon.0 client_cookie=c981af531c37862d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:41.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a000cc60 con 0x7ff288003820 2021-11-21T21:54:41.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000cdc0 con 0x7ff288003820 2021-11-21T21:54:41.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a77fe700 1 -- 172.21.2.116:0/1611229180 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a0005b80 con 0x7ff288003820 2021-11-21T21:54:41.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- 172.21.2.116:0/1611229180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff288003820 msgr2=0x7ff288003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff288003820 0x7ff288003c20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff2a8061b50 tx=0x7ff2a80a9520).stop 2021-11-21T21:54:41.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- 172.21.2.116:0/1611229180 shutdown_connections 2021-11-21T21:54:41.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 --2- 172.21.2.116:0/1611229180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff288003820 0x7ff288003c20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- 172.21.2.116:0/1611229180 >> 172.21.2.116:0/1611229180 conn(0x7ff2a805bd20 msgr2=0x7ff2a805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:41.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- 172.21.2.116:0/1611229180 shutdown_connections 2021-11-21T21:54:41.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- 172.21.2.116:0/1611229180 wait complete. 2021-11-21T21:54:41.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 Processor -- start 2021-11-21T21:54:41.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- start start 2021-11-21T21:54:41.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff288003820 0x7ff2a81430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2a80671a0 0x7ff2a81567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2a8159500 0x7ff2a81599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8143850 con 0x7ff2a80671a0 2021-11-21T21:54:41.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81560b0 con 0x7ff288003820 2021-11-21T21:54:41.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2af111700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8065330 con 0x7ff2a8159500 2021-11-21T21:54:41.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2a80671a0 0x7ff2a81567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:41.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2a80671a0 0x7ff2a81567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45588/0 (socket says 172.21.2.116:45588) 2021-11-21T21:54:41.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 -- 172.21.2.116:0/799466594 learned_addr learned my addr 172.21.2.116:0/799466594 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:41.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 -- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2a8159500 msgr2=0x7ff2a81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2a8159500 0x7ff2a81599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 -- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff288003820 msgr2=0x7ff2a81430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff288003820 0x7ff2a81430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.670+0000 7ff2a7fff700 1 -- 172.21.2.116:0/799466594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a8065090 con 0x7ff2a80671a0 2021-11-21T21:54:41.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a7fff700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2a80671a0 0x7ff2a81567b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff2a80654c0 tx=0x7ff290007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:41.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a5ffb700 1 -- 172.21.2.116:0/799466594 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff290007030 con 0x7ff2a80671a0 2021-11-21T21:54:41.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a5ffb700 1 -- 172.21.2.116:0/799466594 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff290000e90 con 0x7ff2a80671a0 2021-11-21T21:54:41.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a5ffb700 1 -- 172.21.2.116:0/799466594 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff290007380 con 0x7ff2a80671a0 2021-11-21T21:54:41.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a8156310 con 0x7ff2a80671a0 2021-11-21T21:54:41.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a8064d90 con 0x7ff2a80671a0 2021-11-21T21:54:41.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a81564a0 con 0x7ff2a80671a0 2021-11-21T21:54:41.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a5ffb700 1 -- 172.21.2.116:0/799466594 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff29001a060 con 0x7ff2a80671a0 2021-11-21T21:54:41.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a5ffb700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c053540 0x7ff28c055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:41.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a5ffb700 1 -- 172.21.2.116:0/799466594 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff29006afe0 con 0x7ff2a80671a0 2021-11-21T21:54:41.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2acead700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c053540 0x7ff28c055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:41.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2acead700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c053540 0x7ff28c055a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff2a8061b50 tx=0x7ff2a0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:41.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.674+0000 7ff2a5ffb700 1 -- 172.21.2.116:0/799466594 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff290042140 con 0x7ff2a80671a0 2021-11-21T21:54:41.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff2a80b0f60 con 0x7ff2a80671a0 2021-11-21T21:54:41.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2a5ffb700 1 -- 172.21.2.116:0/799466594 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v22) v1 ==== 81+0+33645 (secure 0 0 0) 0x7ff290018070 con 0x7ff2a80671a0 2021-11-21T21:54:41.762 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:41.762 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:54:41.762 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 22, 2021-11-21T21:54:41.763 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:54:41.763 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:54:41.763 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:54:26.920796+0000", 2021-11-21T21:54:41.763 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:54:41.764 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:54:41.764 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:54:41.764 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:54:41.764 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:54:41.764 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:54:41.765 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:54:41.765 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:54:41.765 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:54:41.765 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:54:41.766 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:54:41.766 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:54:41.766 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:54:41.766 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:54:41.767 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:54:41.767 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 4, 2021-11-21T21:54:41.767 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:54:41.767 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:54:41.767 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:54:41.768 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:54:41.768 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:54:41.768 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.768 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:54:41.769 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:54:41.769 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:54:41.769 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:54:41.769 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:54:41.769 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:54:41.770 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:54:41.770 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:54:41.770 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:54:41.771 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:54:41.771 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:54:41.771 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:54:41.771 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:54:41.772 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:54:41.772 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:54:41.773 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:54:41.773 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:54:41.773 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:54:41.773 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:54:41.774 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:54:41.774 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:54:41.774 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:54:41.774 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:54:41.775 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.775 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:54:41.775 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:54:41.775 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:54:41.775 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:54:41.776 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:54:41.776 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:54:41.777 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:54:41.777 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:54:41.777 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:54:41.777 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:54:41.778 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:54:41.778 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:54:41.778 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:54:41.778 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:54:41.779 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:54:41.779 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:54:41.779 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:54:41.779 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:54:41.780 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:54:41.780 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:54:41.780 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:54:41.781 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:54:41.781 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:54:41.781 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:54:41.781 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:54:41.782 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:54:41.782 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:54:41.782 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.782 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:54:41.783 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:54:41.783 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:54:41.783 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:54:41.783 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:54:41.783 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:54:41.784 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:54:41.784 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:54:41.784 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:54:41.784 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:54:41.785 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:54:41.785 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:54:41.785 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:54:41.785 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.785 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:54:41.786 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:54:41.786 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.786 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.786 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.787 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:54:41.787 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:54:41.787 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:54:41.787 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:54:41.788 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:54:41.788 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:54:41.788 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:54:41.788 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:54:41.788 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:54:41.789 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:54:41.789 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:54:41.789 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:54:41.789 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:54:41.789 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:54:41.790 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:54:41.790 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:54:41.790 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:54:41.791 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:54:41.791 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:54:41.791 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:54:41.791 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:54:41.792 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:54:41.792 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:54:41.792 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.792 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:54:41.792 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:54:41.793 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:54:41.793 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:54:41.793 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:54:41.793 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:54:41.794 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:54:41.794 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:54:41.794 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:54:41.794 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:54:41.794 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:54:41.795 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:54:41.795 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:54:41.795 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:54:41.795 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:54:41.796 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:54:41.796 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:54:41.796 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:54:41.796 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:54:41.796 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:54:41.797 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:54:41.797 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:54:41.797 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:54:41.797 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:54:41.798 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:54:41.798 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:54:41.798 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:54:41.798 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.798 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:54:41.799 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:54:41.799 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:54:41.799 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:54:41.799 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:54:41.800 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:54:41.800 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:54:41.800 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:54:41.800 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:54:41.800 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:54:41.801 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:54:41.801 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:54:41.801 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:54:41.802 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:54:41.802 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.802 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.802 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.803 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 3, 2021-11-21T21:54:41.803 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T21:54:41.803 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:23.490451+0000", 2021-11-21T21:54:41.803 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:54:41.804 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:54:41.804 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:54:41.804 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:54:41.804 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:54:41.804 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:54:41.805 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:54:41.805 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:54:41.805 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:54:41.805 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:54:41.806 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:54:41.806 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:54:41.806 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:54:41.806 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:54:41.806 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:54:41.807 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:54:41.807 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:54:41.807 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:54:41.807 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:54:41.808 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:54:41.808 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.808 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "20", 2021-11-21T21:54:41.808 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:54:41.808 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:54:41.809 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:54:41.809 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:54:41.809 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:54:41.809 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:54:41.809 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:54:41.810 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:54:41.810 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:54:41.810 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:54:41.810 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:54:41.811 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:54:41.811 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:54:41.811 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:54:41.812 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:54:41.812 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:54:41.812 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:54:41.812 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:54:41.812 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:54:41.813 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:54:41.813 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:54:41.813 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:54:41.813 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:54:41.814 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:54:41.814 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:54:41.814 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:54:41.814 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.814 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:54:41.815 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:54:41.815 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:54:41.815 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:54:41.815 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:54:41.816 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:54:41.816 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:54:41.816 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:54:41.816 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:54:41.816 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:54:41.817 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:54:41.817 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:54:41.817 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T21:54:41.817 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T21:54:41.818 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T21:54:41.818 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.818 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:54:41.818 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:54:41.818 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T21:54:41.819 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.819 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.819 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.819 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.820 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 4, 2021-11-21T21:54:41.820 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T21:54:41.820 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:23.890830+0000", 2021-11-21T21:54:41.820 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:54:41.821 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:54:41.821 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:54:41.821 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:54:41.821 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:54:41.821 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:54:41.822 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:54:41.822 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:54:41.822 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:54:41.822 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:54:41.823 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:54:41.823 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:54:41.823 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:54:41.823 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:54:41.823 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:54:41.824 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:54:41.824 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:54:41.824 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:54:41.824 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:54:41.825 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:54:41.825 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.825 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "22", 2021-11-21T21:54:41.825 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:54:41.825 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:54:41.826 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:54:41.826 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:54:41.826 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:54:41.826 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:54:41.827 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:54:41.827 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:54:41.827 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:54:41.827 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:54:41.827 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:54:41.828 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:54:41.828 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:54:41.828 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:54:41.828 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:54:41.829 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:54:41.829 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:54:41.829 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:54:41.829 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:54:41.829 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:54:41.830 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:54:41.830 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:54:41.830 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:54:41.830 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:54:41.831 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:54:41.831 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:54:41.831 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.831 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:54:41.832 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:54:41.832 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:54:41.832 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:54:41.832 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:54:41.832 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:54:41.833 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:54:41.833 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:54:41.833 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:54:41.833 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:54:41.833 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:54:41.834 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:54:41.834 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:54:41.834 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:54:41.834 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T21:54:41.835 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.835 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.835 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.835 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:54:41.836 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:54:41.836 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.836 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:54:41.836 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:54:41.836 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.837 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.837 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.837 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.837 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.837 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.838 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.838 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 19, 2021-11-21T21:54:41.838 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.838 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.839 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.839 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.839 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.839 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.839 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:54:41.840 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.840 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.840 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.840 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.841 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:54:41.841 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.841 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.841 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.841 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.842 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.842 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.842 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.842 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.843 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:54:41.843 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.843 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.843 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.843 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.844 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:54:41.844 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.844 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.844 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.845 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.845 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.845 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.845 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.845 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.846 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:54:41.846 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.846 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.846 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.846 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.847 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:54:41.847 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.847 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.847 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.848 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.848 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.848 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.848 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.848 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.849 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:54:41.849 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.849 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.849 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.849 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.850 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:54:41.850 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:54:41.850 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.850 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.851 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.851 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:54:41.851 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:54:41.852 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:54:41.852 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:54:41.852 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.852 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.852 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.853 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.853 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.853 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.853 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:54:41.853 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:54:41.854 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.854 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.854 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.854 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.855 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.855 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.855 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.855 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 19, 2021-11-21T21:54:41.855 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.856 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.856 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.856 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.856 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.856 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.857 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:54:41.857 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.857 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.857 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.858 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.858 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:54:41.858 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.858 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.858 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.859 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.859 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.859 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.859 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.859 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.860 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:54:41.860 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.860 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.860 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.861 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.861 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:54:41.861 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.861 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.861 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.862 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.862 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.862 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.862 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.862 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.863 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:54:41.863 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.863 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.863 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.864 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.864 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:54:41.864 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.864 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.864 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.865 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.865 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.865 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.865 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.865 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.866 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:54:41.866 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.866 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.866 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.867 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.867 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:54:41.867 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:54:41.867 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.867 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.868 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.868 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:54:41.868 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:54:41.868 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:54:41.868 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:54:41.869 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.869 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.869 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.869 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.870 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.870 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.870 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:54:41.870 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:54:41.870 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.871 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.871 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.871 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.871 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.871 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.872 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.872 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 16, 2021-11-21T21:54:41.872 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.872 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.873 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.873 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.873 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.873 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.873 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:54:41.874 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.874 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.874 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.874 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.875 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:54:41.875 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.875 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.875 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.875 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.876 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.876 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.876 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.876 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.877 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:54:41.877 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.877 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.877 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.877 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.878 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:54:41.878 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.878 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.878 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.879 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.879 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.879 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.879 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.879 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.880 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:54:41.880 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.880 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.880 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.880 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.881 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:54:41.881 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.881 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.881 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.882 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.882 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.882 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.882 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.883 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.883 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:54:41.883 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.883 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.883 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.884 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.884 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:54:41.884 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:54:41.884 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.885 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.885 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.885 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:54:41.885 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:54:41.885 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:54:41.886 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:54:41.886 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.886 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.886 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.887 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.887 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.887 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.887 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:54:41.887 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:54:41.888 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.888 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.888 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.888 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.889 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.889 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.889 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.889 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 19, 2021-11-21T21:54:41.889 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.890 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.890 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.890 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.890 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.891 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.891 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:54:41.891 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.891 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.891 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.892 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.892 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:54:41.892 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.892 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.892 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.893 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.893 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.893 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.893 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.894 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.894 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:54:41.894 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.894 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.894 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.895 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.895 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:54:41.895 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.895 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.895 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.896 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.896 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.896 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.896 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.896 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.897 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:54:41.897 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.897 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.897 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.898 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.898 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:54:41.898 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.898 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.898 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.899 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.899 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.899 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.899 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.900 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.900 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:54:41.900 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.900 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.900 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.901 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.901 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:54:41.901 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:54:41.901 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.901 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.902 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.902 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:54:41.902 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:54:41.902 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:54:41.903 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:54:41.903 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.903 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.903 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.903 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.904 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.904 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.904 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:54:41.904 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:54:41.904 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.905 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.905 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.905 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.905 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.906 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.906 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.906 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 19, 2021-11-21T21:54:41.906 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.906 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.907 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.907 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.907 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.907 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.907 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:54:41.908 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.908 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.908 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.908 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.909 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:54:41.909 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.909 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.909 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.909 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.910 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.910 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.910 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.910 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.910 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:54:41.911 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.911 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.911 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.911 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.912 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:54:41.912 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.912 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.912 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.912 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.913 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.913 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.913 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.913 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.913 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:54:41.914 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.914 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.914 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.914 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.915 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:54:41.915 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.915 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.915 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.915 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.916 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.916 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.916 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.916 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.917 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:54:41.917 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.917 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.917 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.917 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.918 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:54:41.918 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:54:41.918 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.918 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.919 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.919 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:54:41.919 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:54:41.919 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:54:41.919 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:54:41.920 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.920 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.920 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.920 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.920 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.921 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.921 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:54:41.921 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:54:41.921 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.922 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.922 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.922 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.922 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.922 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.923 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.923 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 18, 2021-11-21T21:54:41.923 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.923 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.924 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.924 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.924 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.924 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.924 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:54:41.925 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.925 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.925 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.925 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.926 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:54:41.926 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.926 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.926 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.926 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.927 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.927 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.927 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.927 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.928 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:54:41.928 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.928 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.928 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.928 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.929 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:54:41.929 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.929 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.929 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.929 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.930 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.930 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.930 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.930 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.931 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:54:41.931 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.931 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.931 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.931 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.932 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:54:41.932 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.932 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.932 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.932 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.933 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.933 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.933 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.933 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.934 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:54:41.934 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.934 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.934 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.934 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.935 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:54:41.935 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:54:41.935 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.935 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.935 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.936 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:54:41.936 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:54:41.936 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:54:41.936 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:54:41.937 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.937 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.937 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.937 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.937 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.938 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.938 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:54:41.938 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:54:41.938 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.938 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.939 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.939 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.939 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.939 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.940 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.940 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 20, 2021-11-21T21:54:41.940 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.940 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.940 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.941 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.941 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.941 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.941 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:54:41.941 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.942 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.942 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.942 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.942 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:54:41.943 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.943 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.943 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.943 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.943 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.944 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.944 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.944 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.944 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:54:41.945 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.945 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.945 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.945 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.945 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:54:41.946 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.946 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.946 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.946 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.946 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.947 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.947 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.947 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.947 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:54:41.947 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.948 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.948 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.948 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.948 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:54:41.949 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.949 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.949 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.949 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.949 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.950 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.950 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.950 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.950 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:54:41.951 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.951 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.951 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.951 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.951 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:54:41.952 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:54:41.952 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.952 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.952 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.952 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:54:41.953 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:54:41.953 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:54:41.953 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:54:41.953 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.954 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.954 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.954 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.954 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.954 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.955 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:54:41.955 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:54:41.955 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:54:41.955 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:54:41.955 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:54:41.956 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:54:41.956 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:54:41.956 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:54:41.956 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:54:41.957 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 16, 2021-11-21T21:54:41.957 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:54:41.957 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:54:41.957 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:54:41.957 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.958 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.958 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.958 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:54:41.958 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.958 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.959 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.959 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.959 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:54:41.959 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.960 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.960 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.960 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.960 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:54:41.960 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.961 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.961 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.961 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:54:41.961 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.962 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.962 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.962 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.962 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:54:41.962 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.963 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.963 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.963 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.963 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:54:41.964 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.964 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.964 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.964 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:54:41.964 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.965 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.965 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.965 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.965 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:54:41.965 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.966 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.966 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.966 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.966 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:54:41.967 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:54:41.967 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.967 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:54:41.967 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:54:41.967 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.968 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.968 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.968 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:54:41.968 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:54:41.969 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:54:41.969 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.969 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.969 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.969 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:54:41.970 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:54:41.970 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:54:41.970 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:54:41.970 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:54:41.970 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:54:41.971 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:54:41.971 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:41.971 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.971 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:54:41.972 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:54:41.972 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.972 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:54:41.972 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.972 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.973 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.973 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.973 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.973 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:54:41.974 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.974 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.974 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.974 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:54:41.974 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.975 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.975 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.975 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.975 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.976 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:54:41.976 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.976 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.976 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.976 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:54:41.977 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.977 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.977 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.977 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.978 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.978 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:54:41.978 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.978 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.979 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.979 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:54:41.979 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.979 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.979 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.980 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.980 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.980 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:54:41.980 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.980 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.981 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.981 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:54:41.981 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.981 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.982 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.982 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.982 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.982 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:54:41.982 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.983 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.983 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.983 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:54:41.983 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.984 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.984 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.984 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.984 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.985 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:54:41.985 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.985 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.985 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.986 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:54:41.986 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.986 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.986 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.986 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.987 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.987 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:54:41.987 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.987 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.988 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:41.988 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:54:41.988 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:54:41.988 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:54:41.988 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:54:41.989 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:54:41.989 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:54:41.989 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:54:41.989 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:54:41.989 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.990 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:54:41.990 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:54:41.990 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:54:41.990 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:54:41.991 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:54:41.991 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T21:54:41.991 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:54:41.991 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:54:41.991 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:54:41.992 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:54:41.992 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:54:41.992 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:54:41.992 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:54:41.993 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:54:41.993 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:41.993 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:41.993 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:54:41.993 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:54:41.994 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:54:41.994 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:54:41.994 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:54:41.994 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:54:41.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c053540 msgr2=0x7ff28c055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c053540 0x7ff28c055a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff2a8061b50 tx=0x7ff2a0015000).stop 2021-11-21T21:54:41.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2a80671a0 msgr2=0x7ff2a81567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:41.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2a80671a0 0x7ff2a81567b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff2a80654c0 tx=0x7ff290007bd0).stop 2021-11-21T21:54:41.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 shutdown_connections 2021-11-21T21:54:41.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2a8159500 0x7ff2a81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c053540 0x7ff28c055a00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2a80671a0 0x7ff2a81567b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:41.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 --2- 172.21.2.116:0/799466594 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff288003820 0x7ff2a81430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:41.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 >> 172.21.2.116:0/799466594 conn(0x7ff2a805bd20 msgr2=0x7ff2a81442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:41.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 shutdown_connections 2021-11-21T21:54:41.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:41.758+0000 7ff2af111700 1 -- 172.21.2.116:0/799466594 wait complete. 2021-11-21T21:54:41.999 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:42.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 Processor -- start 2021-11-21T21:54:42.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- start start 2021-11-21T21:54:42.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f63dc0aed90 0x7f63dc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0af360 0x7f63dc069e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63dc0b00d0 con 0x7f63dc0b24a0 2021-11-21T21:54:42.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63dc0adba0 con 0x7f63dc0aed90 2021-11-21T21:54:42.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc0b2880 con 0x7f63dc0af360 2021-11-21T21:54:42.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f63dc0aed90 0x7f63dc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e1bfc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0af360 0x7f63dc069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f63dc0aed90 0x7f63dc0b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:46762/0 (socket says 172.21.2.116:46762) 2021-11-21T21:54:42.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e13fb700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f63dc0b24a0 0x7f63dc0ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:56458/0 (socket says 172.21.2.116:56458) 2021-11-21T21:54:42.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 -- 172.21.2.116:0/1054943607 learned_addr learned my addr 172.21.2.116:0/1054943607 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:42.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 -- 172.21.2.116:0/1054943607 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0af360 msgr2=0x7f63dc069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:42.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 --2- 172.21.2.116:0/1054943607 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0af360 0x7f63dc069e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 -- 172.21.2.116:0/1054943607 >> v1:172.21.2.117:6789/0 conn(0x7f63dc0b24a0 legacy=0x7f63dc0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 -- 172.21.2.116:0/1054943607 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc0b14a0 con 0x7f63dc0aed90 2021-11-21T21:54:42.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e0bfa700 1 --2- 172.21.2.116:0/1054943607 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc0b1a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f63cc002f70 tx=0x7f63cc006eb0).ready entity=mon.1 client_cookie=e67d2c472ddae015 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:42.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63cbfff700 1 -- 172.21.2.116:0/1054943607 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63cc00d570 con 0x7f63dc0aed90 2021-11-21T21:54:42.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63cbfff700 1 -- 172.21.2.116:0/1054943607 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc000e90 con 0x7f63dc0aed90 2021-11-21T21:54:42.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63cbfff700 1 -- 172.21.2.116:0/1054943607 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63cc00d900 con 0x7f63dc0aed90 2021-11-21T21:54:42.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- 172.21.2.116:0/1054943607 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 msgr2=0x7f63dc0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 --2- 172.21.2.116:0/1054943607 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc0b1a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f63cc002f70 tx=0x7f63cc006eb0).stop 2021-11-21T21:54:42.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- 172.21.2.116:0/1054943607 shutdown_connections 2021-11-21T21:54:42.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 --2- 172.21.2.116:0/1054943607 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0af360 0x7f63dc069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 --2- 172.21.2.116:0/1054943607 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc0b1a40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- 172.21.2.116:0/1054943607 >> 172.21.2.116:0/1054943607 conn(0x7f63dc05fd00 msgr2=0x7f63dc0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:42.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- 172.21.2.116:0/1054943607 shutdown_connections 2021-11-21T21:54:42.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.070+0000 7f63e365f700 1 -- 172.21.2.116:0/1054943607 wait complete. 2021-11-21T21:54:42.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 Processor -- start 2021-11-21T21:54:42.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 -- start start 2021-11-21T21:54:42.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc13b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63dc0af360 0x7f63dc066160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0b24a0 0x7f63dc066650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc13a120 con 0x7f63dc0af360 2021-11-21T21:54:42.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc13d010 con 0x7f63dc0aed90 2021-11-21T21:54:42.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc13d2b0 con 0x7f63dc0b24a0 2021-11-21T21:54:42.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e0bfa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63dc0af360 0x7f63dc066160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e13fb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc13b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e0bfa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63dc0af360 0x7f63dc066160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45602/0 (socket says 172.21.2.116:45602) 2021-11-21T21:54:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e0bfa700 1 -- 172.21.2.116:0/2667080898 learned_addr learned my addr 172.21.2.116:0/2667080898 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e1bfc700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0b24a0 0x7f63dc066650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e13fb700 1 -- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0b24a0 msgr2=0x7f63dc066650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e13fb700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0b24a0 0x7f63dc066650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e13fb700 1 -- 172.21.2.116:0/2667080898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63dc0af360 msgr2=0x7f63dc066160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e13fb700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63dc0af360 0x7f63dc066160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e13fb700 1 -- 172.21.2.116:0/2667080898 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc00b1e0 con 0x7f63dc0aed90 2021-11-21T21:54:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e0bfa700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63dc0af360 0x7f63dc066160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e13fb700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc13b420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f63dc0630d0 tx=0x7f63d8007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63ca7fc700 1 -- 172.21.2.116:0/2667080898 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63d80070d0 con 0x7f63dc0aed90 2021-11-21T21:54:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63ca7fc700 1 -- 172.21.2.116:0/2667080898 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d8007230 con 0x7f63dc0aed90 2021-11-21T21:54:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63ca7fc700 1 -- 172.21.2.116:0/2667080898 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63d8005550 con 0x7f63dc0aed90 2021-11-21T21:54:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63dc13b910 con 0x7f63dc0aed90 2021-11-21T21:54:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63dc139c10 con 0x7f63dc0aed90 2021-11-21T21:54:42.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63dc13a6c0 con 0x7f63dc0aed90 2021-11-21T21:54:42.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63ca7fc700 1 -- 172.21.2.116:0/2667080898 <== mon.1 v2:172.21.2.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f63d8010070 con 0x7f63dc0aed90 2021-11-21T21:54:42.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63ca7fc700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63bc0534f0 0x7f63bc0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63ca7fc700 1 -- 172.21.2.116:0/2667080898 <== mon.1 v2:172.21.2.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f63d806ae50 con 0x7f63dc0aed90 2021-11-21T21:54:42.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.074+0000 7f63e0bfa700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63bc0534f0 0x7f63bc0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.078+0000 7f63e0bfa700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63bc0534f0 0x7f63bc0559b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f63cc01c000 tx=0x7f63cc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:42.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.078+0000 7f63ca7fc700 1 -- 172.21.2.116:0/2667080898 <== mon.1 v2:172.21.2.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f63d8041ee0 con 0x7f63dc0aed90 2021-11-21T21:54:42.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.186+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63dc13a2f0 con 0x7f63dc0aed90 2021-11-21T21:54:42.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.186+0000 7f63ca7fc700 1 -- 172.21.2.116:0/2667080898 <== mon.1 v2:172.21.2.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f63d806c020 con 0x7f63dc0aed90 2021-11-21T21:54:42.192 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:42.192 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/862809391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":862809391},{"type":"v1","addr":"172.21.2.117:6835","nonce":862809391}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2422183040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2422183040},{"type":"v1","addr":"172.21.2.116:6833","nonce":2422183040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:24.904554+0000","modified":"2021-11-21T21:54:25.909207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/2778157441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2778157441},{"type":"v1","addr":"172.21.2.116:6835","nonce":2778157441}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-11-21T21:54:42.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63bc0534f0 msgr2=0x7f63bc0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63bc0534f0 0x7f63bc0559b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f63cc01c000 tx=0x7f63cc015000).stop 2021-11-21T21:54:42.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 msgr2=0x7f63dc13b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc13b420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f63dc0630d0 tx=0x7f63d8007b80).stop 2021-11-21T21:54:42.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 shutdown_connections 2021-11-21T21:54:42.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63dc0b24a0 0x7f63dc066650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63bc0534f0 0x7f63bc0559b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63dc0af360 0x7f63dc066160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 --2- 172.21.2.116:0/2667080898 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63dc0aed90 0x7f63dc13b420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 >> 172.21.2.116:0/2667080898 conn(0x7f63dc05fd00 msgr2=0x7f63dc0670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:42.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 shutdown_connections 2021-11-21T21:54:42.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.190+0000 7f63e365f700 1 -- 172.21.2.116:0/2667080898 wait complete. 2021-11-21T21:54:42.196 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 5 2021-11-21T21:54:42.199 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T21:54:42.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f21810d2700 1 Processor -- start 2021-11-21T21:54:42.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f21810d2700 1 -- start start 2021-11-21T21:54:42.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f21810d2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f217c142ac0 0x7f217c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f21810d2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0ae070 con 0x7f217c1426e0 2021-11-21T21:54:42.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f21810d2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0afae0 con 0x7f217c062e60 2021-11-21T21:54:42.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f21810d2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c0b0f60 con 0x7f217c142ac0 2021-11-21T21:54:42.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f217b59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f217c142ac0 0x7f217c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f217b59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f217c142ac0 0x7f217c0adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45608/0 (socket says 172.21.2.116:45608) 2021-11-21T21:54:42.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f217b59e700 1 -- 172.21.2.116:0/3635674874 learned_addr learned my addr 172.21.2.116:0/3635674874 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:42.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f217b59e700 1 -- 172.21.2.116:0/3635674874 >> v1:172.21.2.117:6789/0 conn(0x7f217c062e60 legacy=0x7f217c0ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:42.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f217b59e700 1 -- 172.21.2.116:0/3635674874 >> v1:172.21.2.116:6789/0 conn(0x7f217c1426e0 legacy=0x7f217c0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f217b59e700 1 -- 172.21.2.116:0/3635674874 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c063470 con 0x7f217c142ac0 2021-11-21T21:54:42.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f217b59e700 1 --2- 172.21.2.116:0/3635674874 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c0adb80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f217c062db0 tx=0x7f2174009290).ready entity=mon.0 client_cookie=6e51bf67b89503d2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:42.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f2179d9b700 1 -- 172.21.2.116:0/3635674874 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f217400b860 con 0x7f217c142ac0 2021-11-21T21:54:42.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f2179d9b700 1 -- 172.21.2.116:0/3635674874 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217400b9c0 con 0x7f217c142ac0 2021-11-21T21:54:42.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.246+0000 7f2179d9b700 1 -- 172.21.2.116:0/3635674874 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f217400bb30 con 0x7f217c142ac0 2021-11-21T21:54:42.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/3635674874 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 msgr2=0x7f217c0adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 --2- 172.21.2.116:0/3635674874 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c0adb80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f217c062db0 tx=0x7f2174009290).stop 2021-11-21T21:54:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/3635674874 shutdown_connections 2021-11-21T21:54:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 --2- 172.21.2.116:0/3635674874 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c0adb80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:42.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/3635674874 >> 172.21.2.116:0/3635674874 conn(0x7f217c0b5730 msgr2=0x7f217c0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:42.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/3635674874 shutdown_connections 2021-11-21T21:54:42.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/3635674874 wait complete. 2021-11-21T21:54:42.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 Processor -- start 2021-11-21T21:54:42.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- start start 2021-11-21T21:54:42.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f217c1426e0 0x7f217c11bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c11a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f217c141830 0x7f217c11a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c15a6f0 con 0x7f217c142ac0 2021-11-21T21:54:42.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c15a450 con 0x7f217c141830 2021-11-21T21:54:42.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c141e50 con 0x7f217c1426e0 2021-11-21T21:54:42.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c11a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45614/0 (socket says 172.21.2.116:45614) 2021-11-21T21:54:42.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 -- 172.21.2.116:0/2691867088 learned_addr learned my addr 172.21.2.116:0/2691867088 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:42.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 -- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f217c1426e0 msgr2=0x7f217c11bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f217c1426e0 0x7f217c11bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:42.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 -- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f217c141830 msgr2=0x7f217c11a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:42.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f217c141830 0x7f217c11a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:42.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 -- 172.21.2.116:0/2691867088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174009530 con 0x7f217c142ac0 2021-11-21T21:54:42.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f217a59c700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c11a160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f2164002f90 tx=0x7f2164006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:42.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f2173fff700 1 -- 172.21.2.116:0/2691867088 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2164013460 con 0x7f217c142ac0 2021-11-21T21:54:42.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f2173fff700 1 -- 172.21.2.116:0/2691867088 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21640135c0 con 0x7f217c142ac0 2021-11-21T21:54:42.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f2173fff700 1 -- 172.21.2.116:0/2691867088 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2164013920 con 0x7f217c142ac0 2021-11-21T21:54:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c1420f0 con 0x7f217c142ac0 2021-11-21T21:54:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c13fa00 con 0x7f217c142ac0 2021-11-21T21:54:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c142280 con 0x7f217c142ac0 2021-11-21T21:54:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f2173fff700 1 -- 172.21.2.116:0/2691867088 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f216400f070 con 0x7f217c142ac0 2021-11-21T21:54:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f2173fff700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21600536d0 0x7f2160055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:42.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.250+0000 7f2173fff700 1 -- 172.21.2.116:0/2691867088 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f216406b130 con 0x7f217c142ac0 2021-11-21T21:54:42.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.254+0000 7f217ad9d700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21600536d0 0x7f2160055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:42.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.254+0000 7f217ad9d700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21600536d0 0x7f2160055b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f217c062d10 tx=0x7f216c0077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:42.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.254+0000 7f2173fff700 1 -- 172.21.2.116:0/2691867088 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2164042290 con 0x7f217c142ac0 2021-11-21T21:54:42.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:42.362+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f217c143050 con 0x7f217c142ac0 2021-11-21T21:54:43.046 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T21:54:43.042+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T21:54:43.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.054+0000 7f2173fff700 1 -- 172.21.2.116:0/2691867088 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7f21640458b0 con 0x7f217c142ac0 2021-11-21T21:54:43.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21600536d0 msgr2=0x7f2160055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21600536d0 0x7f2160055b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f217c062d10 tx=0x7f216c0077d0).stop 2021-11-21T21:54:43.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 msgr2=0x7f217c11a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c11a160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f2164002f90 tx=0x7f2164006df0).stop 2021-11-21T21:54:43.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 shutdown_connections 2021-11-21T21:54:43.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f217c141830 0x7f217c11a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f21600536d0 0x7f2160055b90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:43.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f217c142ac0 0x7f217c11a160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:43.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 --2- 172.21.2.116:0/2691867088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f217c1426e0 0x7f217c11bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 >> 172.21.2.116:0/2691867088 conn(0x7f217c0b5730 msgr2=0x7f217c0b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:43.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 shutdown_connections 2021-11-21T21:54:43.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.058+0000 7f21810d2700 1 -- 172.21.2.116:0/2691867088 wait complete. 2021-11-21T21:54:43.065 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T21:54:43.068 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:43.072 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:54:43.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f218530e700 1 Processor -- start 2021-11-21T21:54:43.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f218530e700 1 -- start start 2021-11-21T21:54:43.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f218530e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 0x7f218014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f218530e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2180062410 0x7f21801443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f218530e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800a6e80 con 0x7f21800a86b0 2021-11-21T21:54:43.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f218530e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800a8bc0 con 0x7f2180062410 2021-11-21T21:54:43.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f218530e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218014a7b0 con 0x7f21800a6aa0 2021-11-21T21:54:43.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2180062410 0x7f21801443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:43.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2180062410 0x7f21801443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45642/0 (socket says 172.21.2.116:45642) 2021-11-21T21:54:43.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217f7fe700 1 -- 172.21.2.116:0/654572241 learned_addr learned my addr 172.21.2.116:0/654572241 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:43.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217f7fe700 1 -- 172.21.2.116:0/654572241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 msgr2=0x7f218014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:43.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217f7fe700 1 --2- 172.21.2.116:0/654572241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 0x7f218014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217f7fe700 1 -- 172.21.2.116:0/654572241 >> v1:172.21.2.116:6789/0 conn(0x7f21800a86b0 legacy=0x7f21801470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217f7fe700 1 -- 172.21.2.116:0/654572241 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218014aa10 con 0x7f2180062410 2021-11-21T21:54:43.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.114+0000 7f217dffb700 1 -- 172.21.2.116:0/654572241 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672885995 0 0) 0x7f21800a6e80 con 0x7f21800a86b0 2021-11-21T21:54:43.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217f7fe700 1 --2- 172.21.2.116:0/654572241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f21801443f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f21780032f0 tx=0x7f2178009290).ready entity=mon.0 client_cookie=d0750842a511d109 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:43.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217dffb700 1 -- 172.21.2.116:0/654572241 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f217800b8a0 con 0x7f2180062410 2021-11-21T21:54:43.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217dffb700 1 -- 172.21.2.116:0/654572241 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217800ba00 con 0x7f2180062410 2021-11-21T21:54:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217dffb700 1 -- 172.21.2.116:0/654572241 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f217800bb70 con 0x7f2180062410 2021-11-21T21:54:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/654572241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 msgr2=0x7f21801443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 --2- 172.21.2.116:0/654572241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f21801443f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f21780032f0 tx=0x7f2178009290).stop 2021-11-21T21:54:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/654572241 shutdown_connections 2021-11-21T21:54:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 --2- 172.21.2.116:0/654572241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f21801443f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:43.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 --2- 172.21.2.116:0/654572241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 0x7f218014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/654572241 >> 172.21.2.116:0/654572241 conn(0x7f218005bd20 msgr2=0x7f218005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:43.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/654572241 shutdown_connections 2021-11-21T21:54:43.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/654572241 wait complete. 2021-11-21T21:54:43.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 Processor -- start 2021-11-21T21:54:43.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- start start 2021-11-21T21:54:43.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f2180141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 0x7f2180139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21800a86b0 0x7f2180139830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218013bf50 con 0x7f2180062410 2021-11-21T21:54:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218013c180 con 0x7f21800a86b0 2021-11-21T21:54:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218013c420 con 0x7f21800a6aa0 2021-11-21T21:54:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f2180141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f2180141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45644/0 (socket says 172.21.2.116:45644) 2021-11-21T21:54:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 -- 172.21.2.116:0/3327567351 learned_addr learned my addr 172.21.2.116:0/3327567351 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 -- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 msgr2=0x7f2180139340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 0x7f2180139340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 -- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21800a86b0 msgr2=0x7f2180139830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21800a86b0 0x7f2180139830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 -- 172.21.2.116:0/3327567351 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2178009530 con 0x7f2180062410 2021-11-21T21:54:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f217effd700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f2180141ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f2168002f70 tx=0x7f2168006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f2177fff700 1 -- 172.21.2.116:0/3327567351 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2168013460 con 0x7f2180062410 2021-11-21T21:54:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f2177fff700 1 -- 172.21.2.116:0/3327567351 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21680135c0 con 0x7f2180062410 2021-11-21T21:54:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218013c6c0 con 0x7f2180062410 2021-11-21T21:54:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f2177fff700 1 -- 172.21.2.116:0/3327567351 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2168013920 con 0x7f2180062410 2021-11-21T21:54:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800a9a70 con 0x7f2180062410 2021-11-21T21:54:43.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f2177fff700 1 -- 172.21.2.116:0/3327567351 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f216801a060 con 0x7f2180062410 2021-11-21T21:54:43.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.118+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21800573d0 con 0x7f2180062410 2021-11-21T21:54:43.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.122+0000 7f2177fff700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2164053660 0x7f2164055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.122+0000 7f2177fff700 1 -- 172.21.2.116:0/3327567351 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f216806b500 con 0x7f2180062410 2021-11-21T21:54:43.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.122+0000 7f217e7fc700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2164053660 0x7f2164055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:43.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.122+0000 7f217e7fc700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2164053660 0x7f2164055b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2170000bc0 tx=0x7f21700077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:43.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.122+0000 7f2177fff700 1 -- 172.21.2.116:0/3327567351 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2168042500 con 0x7f2180062410 2021-11-21T21:54:43.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.230+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f218005db70 con 0x7f2180062410 2021-11-21T21:54:43.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f2177fff700 1 -- 172.21.2.116:0/3327567351 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2123 (secure 0 0 0) 0x7f2168018030 con 0x7f2180062410 2021-11-21T21:54:43.236 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:43.237 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/862809391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":862809391},{"type":"v1","addr":"172.21.2.117:6835","nonce":862809391}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2422183040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2422183040},{"type":"v1","addr":"172.21.2.116:6833","nonce":2422183040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:24.904554+0000","modified":"2021-11-21T21:54:43.046871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-11-21T21:54:43.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2164053660 msgr2=0x7f2164055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2164053660 0x7f2164055b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2170000bc0 tx=0x7f21700077d0).stop 2021-11-21T21:54:43.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 msgr2=0x7f2180141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f2180141ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f2168002f70 tx=0x7f2168006ec0).stop 2021-11-21T21:54:43.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 shutdown_connections 2021-11-21T21:54:43.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21800a86b0 0x7f2180139830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21800a6aa0 0x7f2180139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2164053660 0x7f2164055b20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:43.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 --2- 172.21.2.116:0/3327567351 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2180062410 0x7f2180141ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:43.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 >> 172.21.2.116:0/3327567351 conn(0x7f218005bd20 msgr2=0x7f21800b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:43.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 shutdown_connections 2021-11-21T21:54:43.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.234+0000 7f218530e700 1 -- 172.21.2.116:0/3327567351 wait complete. 2021-11-21T21:54:43.240 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 6 2021-11-21T21:54:43.243 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T21:54:43.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.338+0000 7fa4aeeb3700 1 Processor -- start 2021-11-21T21:54:43.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.338+0000 7fa4aeeb3700 1 -- start start 2021-11-21T21:54:43.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.338+0000 7fa4aeeb3700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa4a8144810 0x7fa4a81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.338+0000 7fa4aeeb3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa4a80ac600 0x7fa4a80aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.338+0000 7fa4aeeb3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a8147e80 con 0x7fa4a80abce0 2021-11-21T21:54:43.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.338+0000 7fa4aeeb3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a80b1490 con 0x7fa4a80ac600 2021-11-21T21:54:43.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.338+0000 7fa4aeeb3700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a80b1fd0 con 0x7fa4a8144810 2021-11-21T21:54:43.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4ad450700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa4a80ac600 0x7fa4a80aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:43.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a7fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa4a80abce0 0x7fa4a80b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57610/0 (socket says 172.21.2.116:57610) 2021-11-21T21:54:43.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a7fff700 1 -- 172.21.2.116:0/1317906288 learned_addr learned my addr 172.21.2.116:0/1317906288 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:43.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a77fe700 1 -- 172.21.2.116:0/1317906288 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336565692 0 0) 0x7fa4a8147e80 con 0x7fa4a80abce0 2021-11-21T21:54:43.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a77fe700 1 -- 172.21.2.116:0/1317906288 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa488003260 con 0x7fa4a80abce0 2021-11-21T21:54:43.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4ad450700 1 -- 172.21.2.116:0/1317906288 >> v2:172.21.2.117:3300/0 conn(0x7fa4a8144810 msgr2=0x7fa4a81478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:43.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4ad450700 1 --2- 172.21.2.116:0/1317906288 >> v2:172.21.2.117:3300/0 conn(0x7fa4a8144810 0x7fa4a81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4ad450700 1 -- 172.21.2.116:0/1317906288 >> v1:172.21.2.116:6789/0 conn(0x7fa4a80abce0 legacy=0x7fa4a80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4ad450700 1 -- 172.21.2.116:0/1317906288 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a80af5a0 con 0x7fa4a80ac600 2021-11-21T21:54:43.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4ad450700 1 --2- 172.21.2.116:0/1317906288 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80ac600 0x7fa4a80aca50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa4a00032f0 tx=0x7fa4a00091d0).ready entity=mon.0 client_cookie=a9586fe434d4becd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:43.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a77fe700 1 -- 172.21.2.116:0/1317906288 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4a000b680 con 0x7fa4a80ac600 2021-11-21T21:54:43.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a77fe700 1 -- 172.21.2.116:0/1317906288 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a000b7e0 con 0x7fa4a80ac600 2021-11-21T21:54:43.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a77fe700 1 -- 172.21.2.116:0/1317906288 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4a000b950 con 0x7fa4a80ac600 2021-11-21T21:54:43.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/1317906288 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80ac600 msgr2=0x7fa4a80aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/1317906288 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80ac600 0x7fa4a80aca50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa4a00032f0 tx=0x7fa4a00091d0).stop 2021-11-21T21:54:43.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/1317906288 shutdown_connections 2021-11-21T21:54:43.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/1317906288 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80ac600 0x7fa4a80aca50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:43.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/1317906288 >> v2:172.21.2.117:3300/0 conn(0x7fa4a8144810 0x7fa4a81478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/1317906288 >> 172.21.2.116:0/1317906288 conn(0x7fa4a805bd20 msgr2=0x7fa4a805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:43.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/1317906288 shutdown_connections 2021-11-21T21:54:43.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/1317906288 wait complete. 2021-11-21T21:54:43.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 Processor -- start 2021-11-21T21:54:43.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- start start 2021-11-21T21:54:43.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80abce0 0x7fa4a8145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4a80ac600 0x7fa4a80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4a8144810 0x7fa4a81535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a8065850 con 0x7fa4a80abce0 2021-11-21T21:54:43.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80b37b0 con 0x7fa4a8144810 2021-11-21T21:54:43.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a8065ab0 con 0x7fa4a80ac600 2021-11-21T21:54:43.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80abce0 0x7fa4a8145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:43.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80abce0 0x7fa4a8145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45662/0 (socket says 172.21.2.116:45662) 2021-11-21T21:54:43.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 -- 172.21.2.116:0/3985492599 learned_addr learned my addr 172.21.2.116:0/3985492599 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:43.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 -- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4a80ac600 msgr2=0x7fa4a80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4a80ac600 0x7fa4a80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 -- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4a8144810 msgr2=0x7fa4a81535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:43.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4a8144810 0x7fa4a81535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:43.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 -- 172.21.2.116:0/3985492599 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a0009470 con 0x7fa4a80abce0 2021-11-21T21:54:43.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4acc4f700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80abce0 0x7fa4a8145ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa490002f70 tx=0x7fa490006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:43.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a5ffb700 1 -- 172.21.2.116:0/3985492599 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa490013460 con 0x7fa4a80abce0 2021-11-21T21:54:43.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a5ffb700 1 -- 172.21.2.116:0/3985492599 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4900135c0 con 0x7fa4a80abce0 2021-11-21T21:54:43.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a8065d50 con 0x7fa4a80abce0 2021-11-21T21:54:43.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4a5ffb700 1 -- 172.21.2.116:0/3985492599 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa490013920 con 0x7fa4a80abce0 2021-11-21T21:54:43.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.342+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a80b4100 con 0x7fa4a80abce0 2021-11-21T21:54:43.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.346+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a80af060 con 0x7fa4a80abce0 2021-11-21T21:54:43.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.346+0000 7fa4a5ffb700 1 -- 172.21.2.116:0/3985492599 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa49001a060 con 0x7fa4a80abce0 2021-11-21T21:54:43.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.346+0000 7fa4a5ffb700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa48c0535b0 0x7fa48c055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:43.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.346+0000 7fa4a5ffb700 1 -- 172.21.2.116:0/3985492599 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fa49006b300 con 0x7fa4a80abce0 2021-11-21T21:54:43.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.346+0000 7fa4a7fff700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa48c0535b0 0x7fa48c055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:43.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.346+0000 7fa4a7fff700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa48c0535b0 0x7fa48c055a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fa498001110 tx=0x7fa4980076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:43.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.346+0000 7fa4a5ffb700 1 -- 172.21.2.116:0/3985492599 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa490042410 con 0x7fa4a80abce0 2021-11-21T21:54:43.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:43.454+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa4a80ad630 con 0x7fa4a80abce0 2021-11-21T21:54:44.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.062+0000 7fa4a5ffb700 1 -- 172.21.2.116:0/3985492599 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa490045a30 con 0x7fa4a80abce0 2021-11-21T21:54:44.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa48c0535b0 msgr2=0x7fa48c055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:44.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa48c0535b0 0x7fa48c055a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fa498001110 tx=0x7fa4980076c0).stop 2021-11-21T21:54:44.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80abce0 msgr2=0x7fa4a8145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:44.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80abce0 0x7fa4a8145ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa490002f70 tx=0x7fa490006e20).stop 2021-11-21T21:54:44.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 shutdown_connections 2021-11-21T21:54:44.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4a8144810 0x7fa4a81535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:44.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4a80ac600 0x7fa4a80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:44.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa48c0535b0 0x7fa48c055a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:44.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 --2- 172.21.2.116:0/3985492599 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4a80abce0 0x7fa4a8145ff0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:44.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 >> 172.21.2.116:0/3985492599 conn(0x7fa4a805bd20 msgr2=0x7fa4a8140630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:44.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 shutdown_connections 2021-11-21T21:54:44.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.066+0000 7fa4aeeb3700 1 -- 172.21.2.116:0/3985492599 wait complete. 2021-11-21T21:54:44.076 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T21:54:44.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43cb7b3700 1 Processor -- start 2021-11-21T21:54:44.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43cb7b3700 1 -- start start 2021-11-21T21:54:44.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43cb7b3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c40aed90 0x7f43c40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43cb7b3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40b00d0 con 0x7f43c40af360 2021-11-21T21:54:44.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43cb7b3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40b2880 con 0x7f43c40b24a0 2021-11-21T21:54:44.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43cb7b3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40adba0 con 0x7f43c40aed90 2021-11-21T21:54:44.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43c9d50700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f43c40af360 0x7f43c4069e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57624/0 (socket says 172.21.2.116:57624) 2021-11-21T21:54:44.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43c9d50700 1 -- 172.21.2.116:0/3499998677 learned_addr learned my addr 172.21.2.116:0/3499998677 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:44.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762268373 0 0) 0x7f43c40b00d0 con 0x7f43c40af360 2021-11-21T21:54:44.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43ac003260 con 0x7f43c40af360 2021-11-21T21:54:44.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c40aed90 0x7f43c40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:44.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1360604256 0 0) 0x7f43ac003260 con 0x7f43c40af360 2021-11-21T21:54:44.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c40aed90 msgr2=0x7f43c40b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:44.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c40aed90 0x7f43c40b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:44.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 >> v1:172.21.2.117:6789/0 conn(0x7f43c40b24a0 legacy=0x7f43c40ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:44.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40b14a0 con 0x7f43c40af360 2021-11-21T21:54:44.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f43bc0019e0 con 0x7f43c40af360 2021-11-21T21:54:44.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 >> v1:172.21.2.116:6789/0 conn(0x7f43c40af360 legacy=0x7f43c4069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:44.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43ac003800 0x7f43ac003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43ac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c40aed90 0x7f43ac004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40adba0 con 0x7f43ac004500 2021-11-21T21:54:44.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac00b910 con 0x7f43ac003800 2021-11-21T21:54:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac00bad0 con 0x7f43c40aed90 2021-11-21T21:54:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.134+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43ac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43ac003800 0x7f43ac003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c8d4e700 1 -- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c40aed90 msgr2=0x7f43ac004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c40aed90 0x7f43ac004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:44.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c8d4e700 1 -- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43ac003800 msgr2=0x7f43ac003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:44.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43ac003800 0x7f43ac003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:44.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c8d4e700 1 -- 172.21.2.116:0/3499998677 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac003260 con 0x7f43ac004500 2021-11-21T21:54:44.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43ac003800 0x7f43ac003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43ac004980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f43c000a000 tx=0x7f43c0008040).ready entity=mon.0 client_cookie=e45bcd5ffca53a20 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c0013670 con 0x7f43ac004500 2021-11-21T21:54:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c00137d0 con 0x7f43ac004500 2021-11-21T21:54:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43b3fff700 1 -- 172.21.2.116:0/3499998677 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c0013b30 con 0x7f43ac004500 2021-11-21T21:54:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- 172.21.2.116:0/3499998677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 msgr2=0x7f43ac004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 --2- 172.21.2.116:0/3499998677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43ac004980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f43c000a000 tx=0x7f43c0008040).stop 2021-11-21T21:54:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c9d50700 1 -- 172.21.2.116:0/3499998677 reap_dead start 2021-11-21T21:54:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- 172.21.2.116:0/3499998677 shutdown_connections 2021-11-21T21:54:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- 172.21.2.116:0/3499998677 >> 172.21.2.116:0/3499998677 conn(0x7f43c405fd00 msgr2=0x7f43c40600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- 172.21.2.116:0/3499998677 shutdown_connections 2021-11-21T21:54:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- 172.21.2.116:0/3499998677 wait complete. 2021-11-21T21:54:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 Processor -- start 2021-11-21T21:54:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- start start 2021-11-21T21:54:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43c413d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c413f580 0x7f43c40664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c4159900 0x7f43c412c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4066ea0 con 0x7f43ac004500 2021-11-21T21:54:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c413fa00 con 0x7f43c413f580 2021-11-21T21:54:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40669b0 con 0x7f43c4159900 2021-11-21T21:54:44.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43c413d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:44.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43c413d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45680/0 (socket says 172.21.2.116:45680) 2021-11-21T21:54:44.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 -- 172.21.2.116:0/1466076789 learned_addr learned my addr 172.21.2.116:0/1466076789 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:44.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c9d50700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c4159900 0x7f43c412c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c413f580 0x7f43c40664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 -- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c4159900 msgr2=0x7f43c412c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c4159900 0x7f43c412c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 -- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c413f580 msgr2=0x7f43c40664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c413f580 0x7f43c40664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 -- 172.21.2.116:0/1466076789 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4066c00 con 0x7f43ac004500 2021-11-21T21:54:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43c954f700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43c413d1d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f43b4008000 tx=0x7f43b4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43b27fc700 1 -- 172.21.2.116:0/1466076789 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43b4013470 con 0x7f43ac004500 2021-11-21T21:54:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43b27fc700 1 -- 172.21.2.116:0/1466076789 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40135d0 con 0x7f43ac004500 2021-11-21T21:54:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c413ee40 con 0x7f43ac004500 2021-11-21T21:54:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43b27fc700 1 -- 172.21.2.116:0/1466076789 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43b4013930 con 0x7f43ac004500 2021-11-21T21:54:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.138+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c413d800 con 0x7f43ac004500 2021-11-21T21:54:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.142+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c413ff70 con 0x7f43ac004500 2021-11-21T21:54:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.142+0000 7f43b27fc700 1 -- 172.21.2.116:0/1466076789 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f43b401a030 con 0x7f43ac004500 2021-11-21T21:54:44.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.142+0000 7f43b27fc700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43a40535b0 0x7f43a4055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:44.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.142+0000 7f43b27fc700 1 -- 172.21.2.116:0/1466076789 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f43b406b3d0 con 0x7f43ac004500 2021-11-21T21:54:44.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.142+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43a40535b0 0x7f43a4055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.142+0000 7f43c8d4e700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43a40535b0 0x7f43a4055a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f43c001b000 tx=0x7f43c0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.142+0000 7f43b27fc700 1 -- 172.21.2.116:0/1466076789 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43b4042440 con 0x7f43ac004500 2021-11-21T21:54:44.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:44.226+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f43c40573d0 con 0x7f43ac004500 2021-11-21T21:54:45.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43b27fc700 1 -- 172.21.2.116:0/1466076789 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 185+0+0 (secure 0 0 0) 0x7f43b4045a60 con 0x7f43ac004500 2021-11-21T21:54:45.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43a40535b0 msgr2=0x7f43a4055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:45.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43a40535b0 0x7f43a4055a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f43c001b000 tx=0x7f43c0008040).stop 2021-11-21T21:54:45.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 msgr2=0x7f43c413d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:45.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43c413d1d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f43b4008000 tx=0x7f43b4009040).stop 2021-11-21T21:54:45.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 shutdown_connections 2021-11-21T21:54:45.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43c4159900 0x7f43c412c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:45.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43c413f580 0x7f43c40664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:45.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43a40535b0 0x7f43a4055a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:45.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 --2- 172.21.2.116:0/1466076789 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43ac004500 0x7f43c413d1d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:45.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 >> 172.21.2.116:0/1466076789 conn(0x7f43c405fd00 msgr2=0x7f43c40b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:45.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 shutdown_connections 2021-11-21T21:54:45.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.066+0000 7f43cb7b3700 1 -- 172.21.2.116:0/1466076789 wait complete. 2021-11-21T21:54:45.074 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T21:54:45.078 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T21:54:45.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 Processor -- start 2021-11-21T21:54:45.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- start start 2021-11-21T21:54:45.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9c880affc0 0x7f9c8805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:45.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9c880af7e0 0x7f9c8813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:45.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c88143f70 con 0x7f9c880ad780 2021-11-21T21:54:45.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880adb60 con 0x7f9c880af7e0 2021-11-21T21:54:45.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c88143160 con 0x7f9c880affc0 2021-11-21T21:54:45.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8d828700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9c880ad780 0x7f9c88143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57642/0 (socket says 172.21.2.116:57642) 2021-11-21T21:54:45.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8d828700 1 -- 172.21.2.116:0/3249655378 learned_addr learned my addr 172.21.2.116:0/3249655378 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:45.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8e029700 1 --2- 172.21.2.116:0/3249655378 >> v2:172.21.2.116:3300/0 conn(0x7f9c880af7e0 0x7f9c8813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:45.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8e029700 1 -- 172.21.2.116:0/3249655378 >> v2:172.21.2.117:3300/0 conn(0x7f9c880affc0 msgr2=0x7f9c8805f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:45.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8e029700 1 --2- 172.21.2.116:0/3249655378 >> v2:172.21.2.117:3300/0 conn(0x7f9c880affc0 0x7f9c8805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:45.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8e029700 1 -- 172.21.2.116:0/3249655378 >> v1:172.21.2.116:6789/0 conn(0x7f9c880ad780 legacy=0x7f9c88143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:45.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8e029700 1 -- 172.21.2.116:0/3249655378 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880b0ce0 con 0x7f9c880af7e0 2021-11-21T21:54:45.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8e029700 1 --2- 172.21.2.116:0/3249655378 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af7e0 0x7f9c8813fe70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f9c800032f0 tx=0x7f9c800091d0).ready entity=mon.0 client_cookie=1b2f5e1118d48319 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:45.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8c826700 1 -- 172.21.2.116:0/3249655378 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c80009ea0 con 0x7f9c880af7e0 2021-11-21T21:54:45.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8c826700 1 -- 172.21.2.116:0/3249655378 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8000b870 con 0x7f9c880af7e0 2021-11-21T21:54:45.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8c826700 1 -- 172.21.2.116:0/3249655378 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8000b9e0 con 0x7f9c880af7e0 2021-11-21T21:54:45.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3249655378 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af7e0 msgr2=0x7f9c8813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:45.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3249655378 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af7e0 0x7f9c8813fe70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f9c800032f0 tx=0x7f9c800091d0).stop 2021-11-21T21:54:45.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3249655378 shutdown_connections 2021-11-21T21:54:45.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3249655378 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af7e0 0x7f9c8813fe70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:45.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3249655378 >> v2:172.21.2.117:3300/0 conn(0x7f9c880affc0 0x7f9c8805f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:45.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3249655378 >> 172.21.2.116:0/3249655378 conn(0x7f9c880b6000 msgr2=0x7f9c880b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:45.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3249655378 shutdown_connections 2021-11-21T21:54:45.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3249655378 wait complete. 2021-11-21T21:54:45.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 Processor -- start 2021-11-21T21:54:45.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.134+0000 7f9c8fa8c700 1 -- start start 2021-11-21T21:54:45.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880ad780 0x7f9c88142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:45.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af7e0 0x7f9c88152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:45.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880affc0 0x7f9c88152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:45.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8809ea40 con 0x7f9c880ad780 2021-11-21T21:54:45.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88066ab0 con 0x7f9c880af7e0 2021-11-21T21:54:45.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88065f90 con 0x7f9c880affc0 2021-11-21T21:54:45.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880ad780 0x7f9c88142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:45.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880ad780 0x7f9c88142300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45692/0 (socket says 172.21.2.116:45692) 2021-11-21T21:54:45.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 -- 172.21.2.116:0/3971231192 learned_addr learned my addr 172.21.2.116:0/3971231192 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:45.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 -- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880affc0 msgr2=0x7f9c88152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:45.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880affc0 0x7f9c88152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:45.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 -- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af7e0 msgr2=0x7f9c88152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:45.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af7e0 0x7f9c88152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:45.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 -- 172.21.2.116:0/3971231192 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c80009470 con 0x7f9c880ad780 2021-11-21T21:54:45.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d828700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880ad780 0x7f9c88142300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f9c88068820 tx=0x7f9c7c007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:45.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c7affd700 1 -- 172.21.2.116:0/3971231192 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c7c005610 con 0x7f9c880ad780 2021-11-21T21:54:45.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c7affd700 1 -- 172.21.2.116:0/3971231192 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c005770 con 0x7f9c880ad780 2021-11-21T21:54:45.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c7affd700 1 -- 172.21.2.116:0/3971231192 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c7c005ad0 con 0x7f9c880ad780 2021-11-21T21:54:45.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c88066ce0 con 0x7f9c880ad780 2021-11-21T21:54:45.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c881429d0 con 0x7f9c880ad780 2021-11-21T21:54:45.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c88067cf0 con 0x7f9c880ad780 2021-11-21T21:54:45.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c7affd700 1 -- 172.21.2.116:0/3971231192 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9c7c005c30 con 0x7f9c880ad780 2021-11-21T21:54:45.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c7affd700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c68057b00 0x7f9c68059fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:45.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c7affd700 1 -- 172.21.2.116:0/3971231192 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f9c7c06b520 con 0x7f9c880ad780 2021-11-21T21:54:45.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d027700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c68057b00 0x7f9c68059fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:45.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.138+0000 7f9c8d027700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c68057b00 0x7f9c68059fc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f9c84002f70 tx=0x7f9c840069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:45.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.142+0000 7f9c7affd700 1 -- 172.21.2.116:0/3971231192 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c7c0427e0 con 0x7f9c880ad780 2021-11-21T21:54:45.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:45.226+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c88066e70 con 0x7f9c880ad780 2021-11-21T21:54:46.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.078+0000 7f9c7affd700 1 -- 172.21.2.116:0/3971231192 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9c7c045e00 con 0x7f9c880ad780 2021-11-21T21:54:46.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c68057b00 msgr2=0x7f9c68059fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:46.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c68057b00 0x7f9c68059fc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f9c84002f70 tx=0x7f9c840069e0).stop 2021-11-21T21:54:46.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880ad780 msgr2=0x7f9c88142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:46.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880ad780 0x7f9c88142300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f9c88068820 tx=0x7f9c7c007c90).stop 2021-11-21T21:54:46.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 shutdown_connections 2021-11-21T21:54:46.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880affc0 0x7f9c88152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:46.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af7e0 0x7f9c88152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:46.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c68057b00 0x7f9c68059fc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:46.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 --2- 172.21.2.116:0/3971231192 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880ad780 0x7f9c88142300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:46.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 >> 172.21.2.116:0/3971231192 conn(0x7f9c880b6000 msgr2=0x7f9c880b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:46.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 shutdown_connections 2021-11-21T21:54:46.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.082+0000 7f9c8fa8c700 1 -- 172.21.2.116:0/3971231192 wait complete. 2021-11-21T21:54:46.088 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T21:54:46.091 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T21:54:46.092 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T21:54:46.092 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T21:54:46.092 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:54:46.093 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T21:54:46.093 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T21:54:46.093 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T21:54:46.094 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:54:46.094 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T21:54:46.094 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T21:54:46.095 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T21:54:46.095 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:54:46.098 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T21:54:46.098 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T21:54:46.100 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T21:54:46.100 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T21:54:46.101 INFO:tasks.ceph.mds.a:Started 2021-11-21T21:54:46.102 DEBUG:teuthology.parallel:result is None 2021-11-21T21:54:46.103 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T21:54:46.103 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T21:54:46.104 INFO:tasks.ceph.mds.c:Started 2021-11-21T21:54:46.104 DEBUG:teuthology.parallel:result is None 2021-11-21T21:54:46.106 INFO:tasks.ceph.mds.b:Started 2021-11-21T21:54:46.106 DEBUG:teuthology.parallel:result is None 2021-11-21T21:54:46.107 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T21:54:46.134 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T21:54:46.135 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:54:46.142 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T21:54:46.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.154+0000 7f89b9fc8700 1 Processor -- start 2021-11-21T21:54:46.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.154+0000 7f89b9fc8700 1 -- start start 2021-11-21T21:54:46.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f89b40b6770 0x7f89b40b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:46.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b1270 0x7f89b41394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:46.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40b0070 con 0x7f89b41399e0 2021-11-21T21:54:46.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40b05c0 con 0x7f89b40b6770 2021-11-21T21:54:46.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40b2230 con 0x7f89b40b1270 2021-11-21T21:54:46.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b37fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f89b40b6770 0x7f89b40b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:46.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b2ffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b1270 0x7f89b41394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:46.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b37fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f89b40b6770 0x7f89b40b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:46892/0 (socket says 172.21.2.116:46892) 2021-11-21T21:54:46.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b2ffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b1270 0x7f89b41394f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:49072/0 (socket says 172.21.2.116:49072) 2021-11-21T21:54:46.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b37fe700 1 -- 172.21.2.116:0/682986650 learned_addr learned my addr 172.21.2.116:0/682986650 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:46.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b3fff700 1 --1- 172.21.2.116:0/682986650 >> v1:172.21.2.117:6789/0 conn(0x7f89b41399e0 0x7f89b40ad580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:56592/0 (socket says 172.21.2.116:56592) 2021-11-21T21:54:46.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b2ffd700 1 -- 172.21.2.116:0/682986650 >> v2:172.21.2.117:3300/0 conn(0x7f89b40b6770 msgr2=0x7f89b40b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:46.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b2ffd700 1 --2- 172.21.2.116:0/682986650 >> v2:172.21.2.117:3300/0 conn(0x7f89b40b6770 0x7f89b40b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:46.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b2ffd700 1 -- 172.21.2.116:0/682986650 >> v1:172.21.2.117:6789/0 conn(0x7f89b41399e0 legacy=0x7f89b40ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:46.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b2ffd700 1 -- 172.21.2.116:0/682986650 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40b24d0 con 0x7f89b40b1270 2021-11-21T21:54:46.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b2ffd700 1 --2- 172.21.2.116:0/682986650 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b1270 0x7f89b41394f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f89a4002f70 tx=0x7f89a4006df0).ready entity=mon.2 client_cookie=266e7dd3a654445b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:46.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b27fc700 1 -- 172.21.2.116:0/682986650 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89a400d570 con 0x7f89b40b1270 2021-11-21T21:54:46.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b27fc700 1 -- 172.21.2.116:0/682986650 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4000e90 con 0x7f89b40b1270 2021-11-21T21:54:46.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b27fc700 1 -- 172.21.2.116:0/682986650 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89a400d8c0 con 0x7f89b40b1270 2021-11-21T21:54:46.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- 172.21.2.116:0/682986650 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b1270 msgr2=0x7f89b41394f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:46.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/682986650 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b1270 0x7f89b41394f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f89a4002f70 tx=0x7f89a4006df0).stop 2021-11-21T21:54:46.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- 172.21.2.116:0/682986650 shutdown_connections 2021-11-21T21:54:46.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/682986650 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b1270 0x7f89b41394f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:46.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/682986650 >> v2:172.21.2.117:3300/0 conn(0x7f89b40b6770 0x7f89b40b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:46.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- 172.21.2.116:0/682986650 >> 172.21.2.116:0/682986650 conn(0x7f89b4062040 msgr2=0x7f89b40b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:46.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- 172.21.2.116:0/682986650 shutdown_connections 2021-11-21T21:54:46.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.158+0000 7f89b9fc8700 1 -- 172.21.2.116:0/682986650 wait complete. 2021-11-21T21:54:46.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 Processor -- start 2021-11-21T21:54:46.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 -- start start 2021-11-21T21:54:46.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89b40b1270 0x7f89b4131ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:46.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b6770 0x7f89b413c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:46.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89b41399e0 0x7f89b413c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:46.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b413d850 con 0x7f89b41399e0 2021-11-21T21:54:46.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b413cbd0 con 0x7f89b40b1270 2021-11-21T21:54:46.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b413b6d0 con 0x7f89b40b6770 2021-11-21T21:54:46.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89b41399e0 0x7f89b413c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:46.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89b41399e0 0x7f89b413c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45732/0 (socket says 172.21.2.116:45732) 2021-11-21T21:54:46.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 -- 172.21.2.116:0/2993756621 learned_addr learned my addr 172.21.2.116:0/2993756621 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:46.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b37fe700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89b40b1270 0x7f89b4131ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:46.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 -- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b6770 msgr2=0x7f89b413c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:46.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b6770 0x7f89b413c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:46.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 -- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89b40b1270 msgr2=0x7f89b4131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:46.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89b40b1270 0x7f89b4131ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:46.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 -- 172.21.2.116:0/2993756621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a400b180 con 0x7f89b41399e0 2021-11-21T21:54:46.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b37fe700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89b40b1270 0x7f89b4131ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:46.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b3fff700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89b41399e0 0x7f89b413c6e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f89b4060900 tx=0x7f89ac007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:46.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b0ff9700 1 -- 172.21.2.116:0/2993756621 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ac005ae0 con 0x7f89b41399e0 2021-11-21T21:54:46.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b0ff9700 1 -- 172.21.2.116:0/2993756621 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac005c40 con 0x7f89b41399e0 2021-11-21T21:54:46.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b0ff9700 1 -- 172.21.2.116:0/2993756621 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ac0070d0 con 0x7f89b41399e0 2021-11-21T21:54:46.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b413b970 con 0x7f89b41399e0 2021-11-21T21:54:46.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b413da40 con 0x7f89b41399e0 2021-11-21T21:54:46.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b40573d0 con 0x7f89b41399e0 2021-11-21T21:54:46.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b0ff9700 1 -- 172.21.2.116:0/2993756621 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f89ac01a060 con 0x7f89b41399e0 2021-11-21T21:54:46.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b0ff9700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8998053660 0x7f8998055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:46.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b0ff9700 1 -- 172.21.2.116:0/2993756621 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f89ac06b110 con 0x7f89b41399e0 2021-11-21T21:54:46.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.162+0000 7f89b37fe700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8998053660 0x7f8998055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:46.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.166+0000 7f89b37fe700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8998053660 0x7f8998055b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f899c008000 tx=0x7f899c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:46.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.166+0000 7f89b0ff9700 1 -- 172.21.2.116:0/2993756621 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f89ac042550 con 0x7f89b41399e0 2021-11-21T21:54:46.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:46.258+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f89b40afe00 con 0x7f89b41399e0 2021-11-21T21:54:47.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b0ff9700 1 -- 172.21.2.116:0/2993756621 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f89ac018070 con 0x7f89b41399e0 2021-11-21T21:54:47.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8998053660 msgr2=0x7f8998055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8998053660 0x7f8998055b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f899c008000 tx=0x7f899c008040).stop 2021-11-21T21:54:47.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89b41399e0 msgr2=0x7f89b413c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89b41399e0 0x7f89b413c6e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f89b4060900 tx=0x7f89ac007f70).stop 2021-11-21T21:54:47.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 shutdown_connections 2021-11-21T21:54:47.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8998053660 0x7f8998055b20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89b41399e0 0x7f89b413c6e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89b40b6770 0x7f89b413c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 --2- 172.21.2.116:0/2993756621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89b40b1270 0x7f89b4131ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 >> 172.21.2.116:0/2993756621 conn(0x7f89b4062040 msgr2=0x7f89b4138ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 shutdown_connections 2021-11-21T21:54:47.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.142+0000 7f89b9fc8700 1 -- 172.21.2.116:0/2993756621 wait complete. 2021-11-21T21:54:47.149 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T21:54:47.156 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T21:54:47.156 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T21:54:47.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.250+0000 7f0f68440700 1 Processor -- start 2021-11-21T21:54:47.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.250+0000 7f0f68440700 1 -- start start 2021-11-21T21:54:47.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0f600a6aa0 0x7f0f6014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0f60062410 0x7f0f601443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f600a6e80 con 0x7f0f600a86b0 2021-11-21T21:54:47.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f600a8bc0 con 0x7f0f60062410 2021-11-21T21:54:47.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6014a7b0 con 0x7f0f600a6aa0 2021-11-21T21:54:47.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f659db700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0f600a86b0 0x7f0f601470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57698/0 (socket says 172.21.2.116:57698) 2021-11-21T21:54:47.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f659db700 1 -- 172.21.2.116:0/4180865368 learned_addr learned my addr 172.21.2.116:0/4180865368 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:47.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 --2- 172.21.2.116:0/4180865368 >> v2:172.21.2.116:3300/0 conn(0x7f0f60062410 0x7f0f601443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f651da700 1 -- 172.21.2.116:0/4180865368 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142398332 0 0) 0x7f0f600a6e80 con 0x7f0f600a86b0 2021-11-21T21:54:47.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 -- 172.21.2.116:0/4180865368 >> v2:172.21.2.117:3300/0 conn(0x7f0f600a6aa0 msgr2=0x7f0f6014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:47.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 --2- 172.21.2.116:0/4180865368 >> v2:172.21.2.117:3300/0 conn(0x7f0f600a6aa0 0x7f0f6014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 -- 172.21.2.116:0/4180865368 >> v1:172.21.2.116:6789/0 conn(0x7f0f600a86b0 legacy=0x7f0f601470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 -- 172.21.2.116:0/4180865368 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6014aa10 con 0x7f0f60062410 2021-11-21T21:54:47.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 --2- 172.21.2.116:0/4180865368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f60062410 0x7f0f601443f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f0f580032f0 tx=0x7f0f580091d0).ready entity=mon.0 client_cookie=9de1862d8adcc0ca server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f651da700 1 -- 172.21.2.116:0/4180865368 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f58009ea0 con 0x7f0f60062410 2021-11-21T21:54:47.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f651da700 1 -- 172.21.2.116:0/4180865368 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5800b870 con 0x7f0f60062410 2021-11-21T21:54:47.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f651da700 1 -- 172.21.2.116:0/4180865368 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f5800b9e0 con 0x7f0f60062410 2021-11-21T21:54:47.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- 172.21.2.116:0/4180865368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f60062410 msgr2=0x7f0f601443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- 172.21.2.116:0/4180865368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f60062410 0x7f0f601443f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f0f580032f0 tx=0x7f0f580091d0).stop 2021-11-21T21:54:47.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- 172.21.2.116:0/4180865368 shutdown_connections 2021-11-21T21:54:47.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- 172.21.2.116:0/4180865368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f60062410 0x7f0f601443f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- 172.21.2.116:0/4180865368 >> v2:172.21.2.117:3300/0 conn(0x7f0f600a6aa0 0x7f0f6014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- 172.21.2.116:0/4180865368 >> 172.21.2.116:0/4180865368 conn(0x7f0f6005bd20 msgr2=0x7f0f6005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- 172.21.2.116:0/4180865368 shutdown_connections 2021-11-21T21:54:47.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- 172.21.2.116:0/4180865368 wait complete. 2021-11-21T21:54:47.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 Processor -- start 2021-11-21T21:54:47.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- start start 2021-11-21T21:54:47.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f60062410 0x7f0f60142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f600a6aa0 0x7f0f6015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f600a86b0 0x7f0f60156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6014d670 con 0x7f0f600a86b0 2021-11-21T21:54:47.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f60159790 con 0x7f0f60062410 2021-11-21T21:54:47.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f68440700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6013efb0 con 0x7f0f600a6aa0 2021-11-21T21:54:47.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f600a86b0 0x7f0f60156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f600a86b0 0x7f0f60156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45750/0 (socket says 172.21.2.116:45750) 2021-11-21T21:54:47.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 -- 172.21.2.116:0/2426737314 learned_addr learned my addr 172.21.2.116:0/2426737314 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:47.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f661dc700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f60062410 0x7f0f60142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 -- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f600a6aa0 msgr2=0x7f0f6015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f600a6aa0 0x7f0f6015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 -- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f60062410 msgr2=0x7f0f60142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f60062410 0x7f0f60142010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.254+0000 7f0f669dd700 1 -- 172.21.2.116:0/2426737314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f58009470 con 0x7f0f600a86b0 2021-11-21T21:54:47.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f669dd700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f600a86b0 0x7f0f60156200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0f580032f0 tx=0x7f0f58016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f4f7fe700 1 -- 172.21.2.116:0/2426737314 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f5801f5e0 con 0x7f0f600a86b0 2021-11-21T21:54:47.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f4f7fe700 1 -- 172.21.2.116:0/2426737314 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5801f740 con 0x7f0f600a86b0 2021-11-21T21:54:47.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f4f7fe700 1 -- 172.21.2.116:0/2426737314 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f5801faa0 con 0x7f0f600a86b0 2021-11-21T21:54:47.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f601599c0 con 0x7f0f600a86b0 2021-11-21T21:54:47.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6013bdb0 con 0x7f0f600a86b0 2021-11-21T21:54:47.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f600573d0 con 0x7f0f600a86b0 2021-11-21T21:54:47.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f4f7fe700 1 -- 172.21.2.116:0/2426737314 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0f58026030 con 0x7f0f600a86b0 2021-11-21T21:54:47.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f4f7fe700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f400536a0 0x7f0f40055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f4f7fe700 1 -- 172.21.2.116:0/2426737314 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0f5806c980 con 0x7f0f600a86b0 2021-11-21T21:54:47.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f661dc700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f400536a0 0x7f0f40055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f661dc700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f400536a0 0x7f0f40055b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0f50007000 tx=0x7f0f50008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.258+0000 7f0f4f7fe700 1 -- 172.21.2.116:0/2426737314 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f58043d00 con 0x7f0f600a86b0 2021-11-21T21:54:47.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.370+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0f6005db70 con 0x7f0f600a86b0 2021-11-21T21:54:47.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.370+0000 7f0f4f7fe700 1 -- 172.21.2.116:0/2426737314 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+5621 (secure 0 0 0) 0x7f0f58024090 con 0x7f0f600a86b0 2021-11-21T21:54:47.376 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:47.377 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:54:47.377 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T21:54:47.377 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.377 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T21:54:47.378 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T21:54:47.378 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.378 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:54:47.378 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.379 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.379 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.379 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.379 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.380 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.380 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T21:54:47.380 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T21:54:47.380 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.380 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:54:47.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.381 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.381 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.381 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.382 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.382 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T21:54:47.382 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T21:54:47.382 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.383 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:54:47.383 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.383 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.383 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.383 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.384 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.384 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.384 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T21:54:47.384 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T21:54:47.385 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.385 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.385 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.386 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.386 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.386 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.387 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.388 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T21:54:47.388 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T21:54:47.388 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.388 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.388 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.389 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.389 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.389 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.389 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.390 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T21:54:47.390 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T21:54:47.390 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.390 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.391 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.391 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.391 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.391 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.392 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.392 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T21:54:47.392 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T21:54:47.392 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.392 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.393 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.393 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.393 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.393 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.394 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.394 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T21:54:47.394 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T21:54:47.394 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.394 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.395 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.395 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.395 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.395 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.395 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.396 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T21:54:47.396 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T21:54:47.396 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.396 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.397 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.397 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.397 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.398 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.398 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.398 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T21:54:47.398 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T21:54:47.399 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.399 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.399 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.399 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.399 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.400 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.400 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.400 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T21:54:47.400 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T21:54:47.400 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.401 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.401 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.401 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.401 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.402 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.402 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.402 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T21:54:47.402 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T21:54:47.402 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.403 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:54:47.403 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow r", 2021-11-21T21:54:47.403 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow rw", 2021-11-21T21:54:47.403 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rwx" 2021-11-21T21:54:47.404 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.404 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.404 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.404 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T21:54:47.404 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T21:54:47.405 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.405 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:54:47.405 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow r", 2021-11-21T21:54:47.405 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow rw", 2021-11-21T21:54:47.406 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rwx" 2021-11-21T21:54:47.406 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.406 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.406 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.406 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T21:54:47.407 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T21:54:47.407 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.407 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:54:47.407 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:54:47.408 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:54:47.408 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.408 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.409 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.409 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.409 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.bootstrap-mds", 2021-11-21T21:54:47.409 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQACwJphTGFHOhAAi/6fNODbEwJ9kzPin4KNHw==", 2021-11-21T21:54:47.410 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.410 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile bootstrap-mds" 2021-11-21T21:54:47.410 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.410 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.411 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.411 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.bootstrap-mgr", 2021-11-21T21:54:47.411 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQACwJphDYVHOhAA0xdr/ppsbKnyawJdkuaAaA==", 2021-11-21T21:54:47.411 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.411 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile bootstrap-mgr" 2021-11-21T21:54:47.412 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.412 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.412 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.412 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.bootstrap-osd", 2021-11-21T21:54:47.413 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQACwJphlaJHOhAAUk/9sPEcO0jHT/GY6e4Lfw==", 2021-11-21T21:54:47.413 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.413 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile bootstrap-osd" 2021-11-21T21:54:47.413 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.414 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.414 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.414 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.bootstrap-rbd", 2021-11-21T21:54:47.414 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQACwJphEsJHOhAAk17GP8fysL7sjBuy6MguFw==", 2021-11-21T21:54:47.415 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.415 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile bootstrap-rbd" 2021-11-21T21:54:47.415 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.415 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.416 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.416 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.bootstrap-rbd-mirror", 2021-11-21T21:54:47.416 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQACwJph3eBHOhAAV6dJ+DxfQfyoEuwgBCMWSA==", 2021-11-21T21:54:47.416 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.416 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2021-11-21T21:54:47.417 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.417 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.417 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.417 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.bootstrap-rgw", 2021-11-21T21:54:47.418 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQACwJphbwNIOhAATdIyPEmOtn7SySu0RgIL0g==", 2021-11-21T21:54:47.418 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.418 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile bootstrap-rgw" 2021-11-21T21:54:47.418 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.419 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.419 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T21:54:47.419 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T21:54:47.420 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.420 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:54:47.420 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:54:47.420 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.420 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.421 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:47.421 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:47.421 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T21:54:47.421 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T21:54:47.422 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:54:47.422 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:54:47.422 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:54:47.422 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:54:47.422 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.423 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:47.423 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:54:47.423 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:54:47.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f400536a0 msgr2=0x7f0f40055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f400536a0 0x7f0f40055b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0f50007000 tx=0x7f0f50008040).stop 2021-11-21T21:54:47.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f600a86b0 msgr2=0x7f0f60156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f600a86b0 0x7f0f60156200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0f580032f0 tx=0x7f0f58016000).stop 2021-11-21T21:54:47.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 shutdown_connections 2021-11-21T21:54:47.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f400536a0 0x7f0f40055b60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f600a86b0 0x7f0f60156200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f600a6aa0 0x7f0f6015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 --2- 172.21.2.116:0/2426737314 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f60062410 0x7f0f60142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 >> 172.21.2.116:0/2426737314 conn(0x7f0f6005bd20 msgr2=0x7f0f60149bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 shutdown_connections 2021-11-21T21:54:47.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.374+0000 7f0f68440700 1 -- 172.21.2.116:0/2426737314 wait complete. 2021-11-21T21:54:47.428 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2021-11-21T21:54:47.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 Processor -- start 2021-11-21T21:54:47.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- start start 2021-11-21T21:54:47.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff7f40abce0 0x7ff7f41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff7f40ac600 0x7ff7f40aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f4147e80 con 0x7ff7f4144810 2021-11-21T21:54:47.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40b1490 con 0x7ff7f40abce0 2021-11-21T21:54:47.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40b1fd0 con 0x7ff7f40ac600 2021-11-21T21:54:47.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff7f40abce0 0x7ff7f41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff7f40abce0 0x7ff7f41478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45758/0 (socket says 172.21.2.116:45758) 2021-11-21T21:54:47.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 -- 172.21.2.116:0/1098562462 learned_addr learned my addr 172.21.2.116:0/1098562462 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:47.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 -- 172.21.2.116:0/1098562462 >> v2:172.21.2.117:3300/0 conn(0x7ff7f40ac600 msgr2=0x7ff7f40aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:47.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 --2- 172.21.2.116:0/1098562462 >> v2:172.21.2.117:3300/0 conn(0x7ff7f40ac600 0x7ff7f40aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 -- 172.21.2.116:0/1098562462 >> v1:172.21.2.117:6789/0 conn(0x7ff7f4144810 legacy=0x7ff7f40b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:47.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 -- 172.21.2.116:0/1098562462 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f40af5a0 con 0x7ff7f40abce0 2021-11-21T21:54:47.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f3fff700 1 --2- 172.21.2.116:0/1098562462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f41478c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff7f4062500 tx=0x7ff7e4006eb0).ready entity=mon.0 client_cookie=e51989cacae8312b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f37fe700 1 -- 172.21.2.116:0/1098562462 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e400d5c0 con 0x7ff7f40abce0 2021-11-21T21:54:47.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f37fe700 1 -- 172.21.2.116:0/1098562462 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4000e90 con 0x7ff7f40abce0 2021-11-21T21:54:47.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7f37fe700 1 -- 172.21.2.116:0/1098562462 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e400d910 con 0x7ff7f40abce0 2021-11-21T21:54:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- 172.21.2.116:0/1098562462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 msgr2=0x7ff7f41478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/1098562462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f41478c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff7f4062500 tx=0x7ff7e4006eb0).stop 2021-11-21T21:54:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- 172.21.2.116:0/1098562462 shutdown_connections 2021-11-21T21:54:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/1098562462 >> v2:172.21.2.117:3300/0 conn(0x7ff7f40ac600 0x7ff7f40aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/1098562462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f41478c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- 172.21.2.116:0/1098562462 >> 172.21.2.116:0/1098562462 conn(0x7ff7f405bd20 msgr2=0x7ff7f405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- 172.21.2.116:0/1098562462 shutdown_connections 2021-11-21T21:54:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- 172.21.2.116:0/1098562462 wait complete. 2021-11-21T21:54:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 Processor -- start 2021-11-21T21:54:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.478+0000 7ff7fad8e700 1 -- start start 2021-11-21T21:54:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f4143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff7f40ac600 0x7ff7f415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff7f4144810 0x7ff7f40b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f414d350 con 0x7ff7f40abce0 2021-11-21T21:54:47.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f415a130 con 0x7ff7f4144810 2021-11-21T21:54:47.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f415a3d0 con 0x7ff7f40ac600 2021-11-21T21:54:47.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f4143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f4143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45764/0 (socket says 172.21.2.116:45764) 2021-11-21T21:54:47.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 -- 172.21.2.116:0/279523571 learned_addr learned my addr 172.21.2.116:0/279523571 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:47.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 -- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff7f40ac600 msgr2=0x7ff7f415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff7f40ac600 0x7ff7f415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 -- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff7f4144810 msgr2=0x7ff7f40b2b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:47.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff7f4144810 0x7ff7f40b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 -- 172.21.2.116:0/279523571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e400b1e0 con 0x7ff7f40abce0 2021-11-21T21:54:47.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f8b2a700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f4143510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7ff7dc000ed0 tx=0x7ff7dc006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f1ffb700 1 -- 172.21.2.116:0/279523571 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7dc012410 con 0x7ff7f40abce0 2021-11-21T21:54:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f1ffb700 1 -- 172.21.2.116:0/279523571 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc012570 con 0x7ff7f40abce0 2021-11-21T21:54:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f415a670 con 0x7ff7f40abce0 2021-11-21T21:54:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f1ffb700 1 -- 172.21.2.116:0/279523571 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7dc0128d0 con 0x7ff7f40abce0 2021-11-21T21:54:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f4146c40 con 0x7ff7f40abce0 2021-11-21T21:54:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f415a800 con 0x7ff7f40abce0 2021-11-21T21:54:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f1ffb700 1 -- 172.21.2.116:0/279523571 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff7dc00f030 con 0x7ff7f40abce0 2021-11-21T21:54:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f1ffb700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7d8053660 0x7ff7d8055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f1ffb700 1 -- 172.21.2.116:0/279523571 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff7dc06b350 con 0x7ff7f40abce0 2021-11-21T21:54:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.482+0000 7ff7f3fff700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7d8053660 0x7ff7d8055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.486+0000 7ff7f3fff700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7d8053660 0x7ff7d8055b20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ff7f4062500 tx=0x7ff7e4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.486+0000 7ff7f1ffb700 1 -- 172.21.2.116:0/279523571 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff7dc0426d0 con 0x7ff7f40abce0 2021-11-21T21:54:47.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.594+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7ff7f40573d0 con 0x7ff7f40abce0 2021-11-21T21:54:47.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.598+0000 7ff7f1ffb700 1 -- 172.21.2.116:0/279523571 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff7dc045cf0 con 0x7ff7f40abce0 2021-11-21T21:54:47.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7d8053660 msgr2=0x7ff7d8055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7d8053660 0x7ff7d8055b20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ff7f4062500 tx=0x7ff7e4015000).stop 2021-11-21T21:54:47.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 msgr2=0x7ff7f4143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f4143510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7ff7dc000ed0 tx=0x7ff7dc006d70).stop 2021-11-21T21:54:47.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 shutdown_connections 2021-11-21T21:54:47.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff7f4144810 0x7ff7f40b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff7f40ac600 0x7ff7f415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7d8053660 0x7ff7d8055b20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 --2- 172.21.2.116:0/279523571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7f40abce0 0x7ff7f4143510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 >> 172.21.2.116:0/279523571 conn(0x7ff7f405bd20 msgr2=0x7ff7f40afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 shutdown_connections 2021-11-21T21:54:47.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.602+0000 7ff7fad8e700 1 -- 172.21.2.116:0/279523571 wait complete. 2021-11-21T21:54:47.607 INFO:teuthology.orchestra.run.gibba016.stderr:updated 2021-11-21T21:54:47.611 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2021-11-21T21:54:47.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a4a6b3700 1 Processor -- start 2021-11-21T21:54:47.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a4a6b3700 1 -- start start 2021-11-21T21:54:47.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a4a6b3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 0x7f7a441478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a4a6b3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a440abce0 0x7f7a440b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a4a6b3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a44147e80 con 0x7f7a440b1db0 2021-11-21T21:54:47.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a4a6b3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440b1490 con 0x7f7a440abce0 2021-11-21T21:54:47.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a4a6b3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440af060 con 0x7f7a44144810 2021-11-21T21:54:47.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a440abce0 0x7f7a440b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a440abce0 0x7f7a440b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45772/0 (socket says 172.21.2.116:45772) 2021-11-21T21:54:47.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 -- 172.21.2.116:0/3998305468 learned_addr learned my addr 172.21.2.116:0/3998305468 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:47.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 -- 172.21.2.116:0/3998305468 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 msgr2=0x7f7a441478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:47.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 --2- 172.21.2.116:0/3998305468 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 0x7f7a441478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 -- 172.21.2.116:0/3998305468 >> v1:172.21.2.117:6789/0 conn(0x7f7a440b1db0 legacy=0x7f7a440ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:47.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 -- 172.21.2.116:0/3998305468 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a440af5a0 con 0x7f7a440abce0 2021-11-21T21:54:47.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a437fe700 1 --2- 172.21.2.116:0/3998305468 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a440b18c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f7a440b2a30 tx=0x7f7a34006eb0).ready entity=mon.0 client_cookie=b8fad55d53b35076 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a42ffd700 1 -- 172.21.2.116:0/3998305468 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a3400d570 con 0x7f7a440abce0 2021-11-21T21:54:47.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a42ffd700 1 -- 172.21.2.116:0/3998305468 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a34000e90 con 0x7f7a440abce0 2021-11-21T21:54:47.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.658+0000 7f7a42ffd700 1 -- 172.21.2.116:0/3998305468 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a3400d8c0 con 0x7f7a440abce0 2021-11-21T21:54:47.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/3998305468 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 msgr2=0x7f7a440b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/3998305468 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a440b18c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f7a440b2a30 tx=0x7f7a34006eb0).stop 2021-11-21T21:54:47.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/3998305468 shutdown_connections 2021-11-21T21:54:47.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/3998305468 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a440b18c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/3998305468 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 0x7f7a441478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/3998305468 >> 172.21.2.116:0/3998305468 conn(0x7f7a4405bd20 msgr2=0x7f7a4405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/3998305468 shutdown_connections 2021-11-21T21:54:47.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/3998305468 wait complete. 2021-11-21T21:54:47.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 Processor -- start 2021-11-21T21:54:47.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- start start 2021-11-21T21:54:47.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a44159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 0x7f7a44145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a440670c0 0x7f7a4413ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4414ad20 con 0x7f7a440abce0 2021-11-21T21:54:47.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440667b0 con 0x7f7a440670c0 2021-11-21T21:54:47.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440669c0 con 0x7f7a44144810 2021-11-21T21:54:47.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a44159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a44159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45774/0 (socket says 172.21.2.116:45774) 2021-11-21T21:54:47.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 -- 172.21.2.116:0/628392290 learned_addr learned my addr 172.21.2.116:0/628392290 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:47.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 -- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 msgr2=0x7f7a44145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 0x7f7a44145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 -- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a440670c0 msgr2=0x7f7a4413ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a440670c0 0x7f7a4413ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 -- 172.21.2.116:0/628392290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3400b1e0 con 0x7f7a440abce0 2021-11-21T21:54:47.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a43fff700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a44159db0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f7a38002f70 tx=0x7f7a38006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a417fa700 1 -- 172.21.2.116:0/628392290 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a38013460 con 0x7f7a440abce0 2021-11-21T21:54:47.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a417fa700 1 -- 172.21.2.116:0/628392290 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a380135c0 con 0x7f7a440abce0 2021-11-21T21:54:47.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4414af50 con 0x7f7a440abce0 2021-11-21T21:54:47.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a417fa700 1 -- 172.21.2.116:0/628392290 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a38013920 con 0x7f7a440abce0 2021-11-21T21:54:47.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a44149e20 con 0x7f7a440abce0 2021-11-21T21:54:47.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a440af300 con 0x7f7a440abce0 2021-11-21T21:54:47.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a417fa700 1 -- 172.21.2.116:0/628392290 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7a3801a060 con 0x7f7a440abce0 2021-11-21T21:54:47.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a417fa700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a24053660 0x7f7a24055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.662+0000 7f7a417fa700 1 -- 172.21.2.116:0/628392290 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7a3806b050 con 0x7f7a440abce0 2021-11-21T21:54:47.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.666+0000 7f7a437fe700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a24053660 0x7f7a24055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.666+0000 7f7a437fe700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a24053660 0x7f7a24055b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f7a440b2a30 tx=0x7f7a34015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.666+0000 7f7a417fa700 1 -- 172.21.2.116:0/628392290 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a380424e0 con 0x7f7a440abce0 2021-11-21T21:54:47.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.778+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f7a440573d0 con 0x7f7a440abce0 2021-11-21T21:54:47.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.782+0000 7f7a417fa700 1 -- 172.21.2.116:0/628392290 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7a38018070 con 0x7f7a440abce0 2021-11-21T21:54:47.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a24053660 msgr2=0x7f7a24055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a24053660 0x7f7a24055b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f7a440b2a30 tx=0x7f7a34015000).stop 2021-11-21T21:54:47.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 msgr2=0x7f7a44159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a44159db0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f7a38002f70 tx=0x7f7a38006e20).stop 2021-11-21T21:54:47.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 shutdown_connections 2021-11-21T21:54:47.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a440670c0 0x7f7a4413ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a44144810 0x7f7a44145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a24053660 0x7f7a24055b20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 --2- 172.21.2.116:0/628392290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a440abce0 0x7f7a44159db0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 >> 172.21.2.116:0/628392290 conn(0x7f7a4405bd20 msgr2=0x7f7a440af780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 shutdown_connections 2021-11-21T21:54:47.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.786+0000 7f7a4a6b3700 1 -- 172.21.2.116:0/628392290 wait complete. 2021-11-21T21:54:47.791 INFO:teuthology.orchestra.run.gibba016.stderr:updated 2021-11-21T21:54:47.794 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2021-11-21T21:54:47.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 Processor -- start 2021-11-21T21:54:47.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- start start 2021-11-21T21:54:47.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a40144810 0x7f4a401478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4a400ac600 0x7f4a400aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a40147e80 con 0x7f4a400abce0 2021-11-21T21:54:47.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a400b1490 con 0x7f4a400ac600 2021-11-21T21:54:47.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a400b1fd0 con 0x7f4a40144810 2021-11-21T21:54:47.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4a400ac600 0x7f4a400aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4a400ac600 0x7f4a400aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45786/0 (socket says 172.21.2.116:45786) 2021-11-21T21:54:47.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 -- 172.21.2.116:0/3310530157 learned_addr learned my addr 172.21.2.116:0/3310530157 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:47.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 -- 172.21.2.116:0/3310530157 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a40144810 msgr2=0x7f4a401478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:47.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 --2- 172.21.2.116:0/3310530157 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a40144810 0x7f4a401478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 -- 172.21.2.116:0/3310530157 >> v1:172.21.2.117:6789/0 conn(0x7f4a400abce0 legacy=0x7f4a400b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:47.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 -- 172.21.2.116:0/3310530157 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400af5a0 con 0x7f4a400ac600 2021-11-21T21:54:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a46d67700 1 --2- 172.21.2.116:0/3310530157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a400aca90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f4a400b2a30 tx=0x7f4a38009290).ready entity=mon.0 client_cookie=32b308dbc549dae6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a45564700 1 -- 172.21.2.116:0/3310530157 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a3800b860 con 0x7f4a400ac600 2021-11-21T21:54:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a45564700 1 -- 172.21.2.116:0/3310530157 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3800b9c0 con 0x7f4a400ac600 2021-11-21T21:54:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a45564700 1 -- 172.21.2.116:0/3310530157 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a3800bb70 con 0x7f4a400ac600 2021-11-21T21:54:47.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- 172.21.2.116:0/3310530157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 msgr2=0x7f4a400aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 --2- 172.21.2.116:0/3310530157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a400aca90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f4a400b2a30 tx=0x7f4a38009290).stop 2021-11-21T21:54:47.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- 172.21.2.116:0/3310530157 shutdown_connections 2021-11-21T21:54:47.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 --2- 172.21.2.116:0/3310530157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a400aca90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 --2- 172.21.2.116:0/3310530157 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a40144810 0x7f4a401478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- 172.21.2.116:0/3310530157 >> 172.21.2.116:0/3310530157 conn(0x7f4a4005bd20 msgr2=0x7f4a4005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- 172.21.2.116:0/3310530157 shutdown_connections 2021-11-21T21:54:47.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.842+0000 7f4a487ca700 1 -- 172.21.2.116:0/3310530157 wait complete. 2021-11-21T21:54:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 Processor -- start 2021-11-21T21:54:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 -- start start 2021-11-21T21:54:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a400abce0 0x7f4a40143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a4013c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4a40144810 0x7f4a4013c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a401569a0 con 0x7f4a400ac600 2021-11-21T21:54:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a401575d0 con 0x7f4a40144810 2021-11-21T21:54:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40156740 con 0x7f4a400abce0 2021-11-21T21:54:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a4013c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a4013c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45790/0 (socket says 172.21.2.116:45790) 2021-11-21T21:54:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 -- 172.21.2.116:0/1108666645 learned_addr learned my addr 172.21.2.116:0/1108666645 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 -- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a400abce0 msgr2=0x7f4a40143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a400abce0 0x7f4a40143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 -- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4a40144810 msgr2=0x7f4a4013c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4a40144810 0x7f4a4013c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 -- 172.21.2.116:0/1108666645 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a38009530 con 0x7f4a400ac600 2021-11-21T21:54:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a45d65700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a4013c380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f4a3c002f90 tx=0x7f4a3c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a337fe700 1 -- 172.21.2.116:0/1108666645 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a3c013460 con 0x7f4a400ac600 2021-11-21T21:54:47.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a337fe700 1 -- 172.21.2.116:0/1108666645 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c0135c0 con 0x7f4a400ac600 2021-11-21T21:54:47.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a337fe700 1 -- 172.21.2.116:0/1108666645 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a3c013920 con 0x7f4a400ac600 2021-11-21T21:54:47.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a401463b0 con 0x7f4a400ac600 2021-11-21T21:54:47.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a40143810 con 0x7f4a400ac600 2021-11-21T21:54:47.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a400af060 con 0x7f4a400ac600 2021-11-21T21:54:47.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a337fe700 1 -- 172.21.2.116:0/1108666645 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4a3c00f070 con 0x7f4a400ac600 2021-11-21T21:54:47.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a337fe700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a20053580 0x7f4a20055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:47.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a337fe700 1 -- 172.21.2.116:0/1108666645 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4a3c06ae00 con 0x7f4a400ac600 2021-11-21T21:54:47.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.846+0000 7f4a46566700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a20053580 0x7f4a20055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:47.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.850+0000 7f4a46566700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a20053580 0x7f4a20055a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4a34002f70 tx=0x7f4a340069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:47.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.850+0000 7f4a337fe700 1 -- 172.21.2.116:0/1108666645 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4a3c042290 con 0x7f4a400ac600 2021-11-21T21:54:47.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.958+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4a400573d0 con 0x7f4a400ac600 2021-11-21T21:54:47.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a337fe700 1 -- 172.21.2.116:0/1108666645 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4a3c0458b0 con 0x7f4a400ac600 2021-11-21T21:54:47.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a20053580 msgr2=0x7f4a20055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a20053580 0x7f4a20055a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4a34002f70 tx=0x7f4a340069e0).stop 2021-11-21T21:54:47.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 msgr2=0x7f4a4013c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:47.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a4013c380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f4a3c002f90 tx=0x7f4a3c006df0).stop 2021-11-21T21:54:47.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 shutdown_connections 2021-11-21T21:54:47.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4a40144810 0x7f4a4013c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a20053580 0x7f4a20055a40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4a400ac600 0x7f4a4013c380 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 --2- 172.21.2.116:0/1108666645 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4a400abce0 0x7f4a40143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 >> 172.21.2.116:0/1108666645 conn(0x7f4a4005bd20 msgr2=0x7f4a400b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 shutdown_connections 2021-11-21T21:54:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:47.962+0000 7f4a487ca700 1 -- 172.21.2.116:0/1108666645 wait complete. 2021-11-21T21:54:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:updated 2021-11-21T21:54:47.974 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2021-11-21T21:54:48.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c9d27700 1 Processor -- start 2021-11-21T21:54:48.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c9d27700 1 -- start start 2021-11-21T21:54:48.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c9d27700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd7c40abce0 0x7fd7c41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c9d27700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7c40ac600 0x7fd7c40aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c9d27700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c4147e80 con 0x7fd7c4144810 2021-11-21T21:54:48.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c9d27700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c40b1490 con 0x7fd7c40abce0 2021-11-21T21:54:48.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c9d27700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40b1fd0 con 0x7fd7c40ac600 2021-11-21T21:54:48.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c37fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd7c4144810 0x7fd7c40b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57756/0 (socket says 172.21.2.116:57756) 2021-11-21T21:54:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c37fe700 1 -- 172.21.2.116:0/1975858994 learned_addr learned my addr 172.21.2.116:0/1975858994 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272933959 0 0) 0x7fd7c4147e80 con 0x7fd7c4144810 2021-11-21T21:54:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a4003260 con 0x7fd7c4144810 2021-11-21T21:54:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/1975858994 >> v2:172.21.2.117:3300/0 conn(0x7fd7c40abce0 0x7fd7c41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c3fff700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7c40ac600 0x7fd7c40aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1703839546 0 0) 0x7fd7a4003260 con 0x7fd7c4144810 2021-11-21T21:54:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7c40ac600 msgr2=0x7fd7c40aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7c40ac600 0x7fd7c40aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 >> v2:172.21.2.117:3300/0 conn(0x7fd7c40abce0 msgr2=0x7fd7c41478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 --2- 172.21.2.116:0/1975858994 >> v2:172.21.2.117:3300/0 conn(0x7fd7c40abce0 0x7fd7c41478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40af5a0 con 0x7fd7c4144810 2021-11-21T21:54:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd7b4001660 con 0x7fd7c4144810 2021-11-21T21:54:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 >> v1:172.21.2.116:6789/0 conn(0x7fd7c4144810 legacy=0x7fd7c40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7a40037d0 0x7fd7a4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a4004500 0x7fd7a4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7c40ac600 0x7fd7a4004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40b1490 con 0x7fd7a4004500 2021-11-21T21:54:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40b1fd0 con 0x7fd7a40037d0 2021-11-21T21:54:48.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a400b8d0 con 0x7fd7c40ac600 2021-11-21T21:54:48.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a4004500 0x7fd7a4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 -- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7c40ac600 msgr2=0x7fd7a4004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7c40ac600 0x7fd7a4004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 -- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7a40037d0 msgr2=0x7fd7a4003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7a40037d0 0x7fd7a4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 -- 172.21.2.116:0/1975858994 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4003260 con 0x7fd7a4004500 2021-11-21T21:54:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.022+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a4004500 0x7fd7a4004980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fd7ac00a000 tx=0x7fd7ac008040).ready entity=mon.0 client_cookie=10156c329c8f703f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7ac013670 con 0x7fd7a4004500 2021-11-21T21:54:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac0137d0 con 0x7fd7a4004500 2021-11-21T21:54:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c27fc700 1 -- 172.21.2.116:0/1975858994 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7ac013b30 con 0x7fd7a4004500 2021-11-21T21:54:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/1975858994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a4004500 msgr2=0x7fd7a4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 --2- 172.21.2.116:0/1975858994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a4004500 0x7fd7a4004980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fd7ac00a000 tx=0x7fd7ac008040).stop 2021-11-21T21:54:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c3fff700 1 -- 172.21.2.116:0/1975858994 reap_dead start 2021-11-21T21:54:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/1975858994 shutdown_connections 2021-11-21T21:54:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/1975858994 >> 172.21.2.116:0/1975858994 conn(0x7fd7c405bd20 msgr2=0x7fd7c405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/1975858994 shutdown_connections 2021-11-21T21:54:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/1975858994 wait complete. 2021-11-21T21:54:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 Processor -- start 2021-11-21T21:54:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- start start 2021-11-21T21:54:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a4004500 0x7fd7c4156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7c4145e50 0x7fd7c40b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7c4156470 0x7fd7c41568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4143e40 con 0x7fd7c4145e50 2021-11-21T21:54:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4146cf0 con 0x7fd7c4156470 2021-11-21T21:54:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4141a20 con 0x7fd7a4004500 2021-11-21T21:54:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7c4145e50 0x7fd7c40b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7c4145e50 0x7fd7c40b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45810/0 (socket says 172.21.2.116:45810) 2021-11-21T21:54:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 -- 172.21.2.116:0/3090753805 learned_addr learned my addr 172.21.2.116:0/3090753805 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c37fe700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a4004500 0x7fd7c4156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c3fff700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7c4156470 0x7fd7c41568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 -- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a4004500 msgr2=0x7fd7c4156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a4004500 0x7fd7c4156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 -- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7c4156470 msgr2=0x7fd7c41568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7c4156470 0x7fd7c41568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 -- 172.21.2.116:0/3090753805 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4143ba0 con 0x7fd7c4145e50 2021-11-21T21:54:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c2ffd700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7c4145e50 0x7fd7c40b2ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fd7c4062e50 tx=0x7fd7ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c0ff9700 1 -- 172.21.2.116:0/3090753805 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7ac00b1a0 con 0x7fd7c4145e50 2021-11-21T21:54:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c0ff9700 1 -- 172.21.2.116:0/3090753805 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac013e40 con 0x7fd7c4145e50 2021-11-21T21:54:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c0ff9700 1 -- 172.21.2.116:0/3090753805 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7ac012e20 con 0x7fd7c4145e50 2021-11-21T21:54:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c4146a50 con 0x7fd7c4145e50 2021-11-21T21:54:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c413ecc0 con 0x7fd7c4145e50 2021-11-21T21:54:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c40af060 con 0x7fd7c4145e50 2021-11-21T21:54:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c0ff9700 1 -- 172.21.2.116:0/3090753805 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd7ac024020 con 0x7fd7c4145e50 2021-11-21T21:54:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c0ff9700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7a80534f0 0x7fd7a80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.026+0000 7fd7c0ff9700 1 -- 172.21.2.116:0/3090753805 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd7ac06c8c0 con 0x7fd7c4145e50 2021-11-21T21:54:48.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.030+0000 7fd7c37fe700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7a80534f0 0x7fd7a80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.030+0000 7fd7c37fe700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7a80534f0 0x7fd7a80559b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd7b400b000 tx=0x7fd7b4001710).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.030+0000 7fd7c0ff9700 1 -- 172.21.2.116:0/3090753805 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7ac043d50 con 0x7fd7c4145e50 2021-11-21T21:54:48.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.142+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fd7c40573d0 con 0x7fd7c4145e50 2021-11-21T21:54:48.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c0ff9700 1 -- 172.21.2.116:0/3090753805 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd7ac047370 con 0x7fd7c4145e50 2021-11-21T21:54:48.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7a80534f0 msgr2=0x7fd7a80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7a80534f0 0x7fd7a80559b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd7b400b000 tx=0x7fd7b4001710).stop 2021-11-21T21:54:48.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7c4145e50 msgr2=0x7fd7c40b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7c4145e50 0x7fd7c40b2ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fd7c4062e50 tx=0x7fd7ac017000).stop 2021-11-21T21:54:48.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 shutdown_connections 2021-11-21T21:54:48.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7c4156470 0x7fd7c41568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7a80534f0 0x7fd7a80559b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7c4145e50 0x7fd7c40b2ab0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 --2- 172.21.2.116:0/3090753805 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a4004500 0x7fd7c4156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 >> 172.21.2.116:0/3090753805 conn(0x7fd7c405bd20 msgr2=0x7fd7c40aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 shutdown_connections 2021-11-21T21:54:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.146+0000 7fd7c9d27700 1 -- 172.21.2.116:0/3090753805 wait complete. 2021-11-21T21:54:48.153 INFO:teuthology.orchestra.run.gibba016.stderr:updated 2021-11-21T21:54:48.156 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2021-11-21T21:54:48.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd8159700 1 Processor -- start 2021-11-21T21:54:48.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd8159700 1 -- start start 2021-11-21T21:54:48.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd8159700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fd0144810 0x7f0fd01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd8159700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd0147e80 con 0x7f0fd00abce0 2021-11-21T21:54:48.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd8159700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd00af5a0 con 0x7f0fd00ac600 2021-11-21T21:54:48.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd8159700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00b1490 con 0x7f0fd0144810 2021-11-21T21:54:48.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd56f4700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0fd00abce0 0x7f0fd00b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57772/0 (socket says 172.21.2.116:57772) 2021-11-21T21:54:48.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd56f4700 1 -- 172.21.2.116:0/420686844 learned_addr learned my addr 172.21.2.116:0/420686844 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:48.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226528875 0 0) 0x7f0fd0147e80 con 0x7f0fd00abce0 2021-11-21T21:54:48.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb8003260 con 0x7f0fd00abce0 2021-11-21T21:54:48.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110218066 0 0) 0x7f0fb8003260 con 0x7f0fd00abce0 2021-11-21T21:54:48.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fd0144810 msgr2=0x7f0fd01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fd0144810 0x7f0fd01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 >> v1:172.21.2.117:6789/0 conn(0x7f0fd00ac600 legacy=0x7f0fd00aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00b1fd0 con 0x7f0fd00abce0 2021-11-21T21:54:48.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f0fc4001660 con 0x7f0fd00abce0 2021-11-21T21:54:48.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 >> v1:172.21.2.116:6789/0 conn(0x7f0fd00abce0 legacy=0x7f0fd00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fb8003780 0x7f0fb8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fb8004500 0x7f0fb8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fd0144810 0x7f0fb8004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00b1490 con 0x7f0fb8004500 2021-11-21T21:54:48.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.202+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb800b910 con 0x7f0fb8003780 2021-11-21T21:54:48.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb800bad0 con 0x7f0fd0144810 2021-11-21T21:54:48.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fb8004500 0x7f0fb8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 -- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fd0144810 msgr2=0x7f0fb8004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fd0144810 0x7f0fb8004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 -- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fb8003780 msgr2=0x7f0fb8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fb8003780 0x7f0fb8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 -- 172.21.2.116:0/420686844 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb8003260 con 0x7f0fb8004500 2021-11-21T21:54:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fb8004500 0x7f0fb8004980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f0fd00ac4e0 tx=0x7f0fd0146e80).ready entity=mon.0 client_cookie=839c3a06fcdf7fc server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc40071a0 con 0x7f0fb8004500 2021-11-21T21:54:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc4007300 con 0x7f0fb8004500 2021-11-21T21:54:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd4ef3700 1 -- 172.21.2.116:0/420686844 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc400c8f0 con 0x7f0fb8004500 2021-11-21T21:54:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/420686844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fb8004500 msgr2=0x7f0fb8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 --2- 172.21.2.116:0/420686844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fb8004500 0x7f0fb8004980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f0fd00ac4e0 tx=0x7f0fd0146e80).stop 2021-11-21T21:54:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 -- 172.21.2.116:0/420686844 reap_dead start 2021-11-21T21:54:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/420686844 shutdown_connections 2021-11-21T21:54:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/420686844 >> 172.21.2.116:0/420686844 conn(0x7f0fd005bd20 msgr2=0x7f0fd005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/420686844 shutdown_connections 2021-11-21T21:54:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/420686844 wait complete. 2021-11-21T21:54:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 Processor -- start 2021-11-21T21:54:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- start start 2021-11-21T21:54:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fb8004500 0x7f0fd00b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fd00a8330 0x7f0fd0156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fd0156f60 0x7f0fd0159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00b4400 con 0x7f0fd0156f60 2021-11-21T21:54:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00a7ac0 con 0x7f0fd00a8330 2021-11-21T21:54:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00a7d60 con 0x7f0fb8004500 2021-11-21T21:54:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fd0156f60 0x7f0fd0159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fd0156f60 0x7f0fd0159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45832/0 (socket says 172.21.2.116:45832) 2021-11-21T21:54:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 -- 172.21.2.116:0/3228492099 learned_addr learned my addr 172.21.2.116:0/3228492099 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd56f4700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fd00a8330 0x7f0fd0156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 -- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fb8004500 msgr2=0x7f0fd00b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fb8004500 0x7f0fd00b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 -- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fd00a8330 msgr2=0x7f0fd0156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fd00a8330 0x7f0fd0156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 -- 172.21.2.116:0/3228492099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00a8000 con 0x7f0fd0156f60 2021-11-21T21:54:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd66f6700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fd0156f60 0x7f0fd0159e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f0fc4001370 tx=0x7f0fc8007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fc2ffd700 1 -- 172.21.2.116:0/3228492099 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc8004850 con 0x7f0fd0156f60 2021-11-21T21:54:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fc2ffd700 1 -- 172.21.2.116:0/3228492099 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc80049b0 con 0x7f0fd0156f60 2021-11-21T21:54:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd01599d0 con 0x7f0fd0156f60 2021-11-21T21:54:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fc2ffd700 1 -- 172.21.2.116:0/3228492099 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc8004d50 con 0x7f0fd0156f60 2021-11-21T21:54:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd0156400 con 0x7f0fd0156f60 2021-11-21T21:54:48.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.206+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd0159b60 con 0x7f0fd0156f60 2021-11-21T21:54:48.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.210+0000 7f0fc2ffd700 1 -- 172.21.2.116:0/3228492099 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0fc801b070 con 0x7f0fd0156f60 2021-11-21T21:54:48.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.210+0000 7f0fc2ffd700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0fb00536d0 0x7f0fb0055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.210+0000 7f0fc2ffd700 1 -- 172.21.2.116:0/3228492099 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0fc806aaf0 con 0x7f0fd0156f60 2021-11-21T21:54:48.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.210+0000 7f0fd5ef5700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0fb00536d0 0x7f0fb0055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.210+0000 7f0fd5ef5700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0fb00536d0 0x7f0fb0055b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0fcc002f70 tx=0x7f0fcc006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.210+0000 7f0fc2ffd700 1 -- 172.21.2.116:0/3228492099 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fc8041e70 con 0x7f0fd0156f60 2021-11-21T21:54:48.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.322+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f0fd00573d0 con 0x7f0fd0156f60 2021-11-21T21:54:48.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fc2ffd700 1 -- 172.21.2.116:0/3228492099 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0fc8045490 con 0x7f0fd0156f60 2021-11-21T21:54:48.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0fb00536d0 msgr2=0x7f0fb0055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0fb00536d0 0x7f0fb0055b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0fcc002f70 tx=0x7f0fcc006a10).stop 2021-11-21T21:54:48.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fd0156f60 msgr2=0x7f0fd0159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fd0156f60 0x7f0fd0159e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f0fc4001370 tx=0x7f0fc8007e50).stop 2021-11-21T21:54:48.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 shutdown_connections 2021-11-21T21:54:48.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0fb00536d0 0x7f0fb0055b90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fd0156f60 0x7f0fd0159e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fd00a8330 0x7f0fd0156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 --2- 172.21.2.116:0/3228492099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fb8004500 0x7f0fd00b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 >> 172.21.2.116:0/3228492099 conn(0x7f0fd005bd20 msgr2=0x7f0fd0147020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 shutdown_connections 2021-11-21T21:54:48.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.326+0000 7f0fd8159700 1 -- 172.21.2.116:0/3228492099 wait complete. 2021-11-21T21:54:48.335 INFO:teuthology.orchestra.run.gibba016.stderr:updated 2021-11-21T21:54:48.338 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2021-11-21T21:54:48.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.382+0000 7fa14126d700 1 Processor -- start 2021-11-21T21:54:48.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.382+0000 7fa14126d700 1 -- start start 2021-11-21T21:54:48.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa13c0b0ca0 0x7fa13c0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c067890 0x7fa13c0ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa13c140220 0x7fa13c0634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0b1080 con 0x7fa13c0b0ca0 2021-11-21T21:54:48.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0aca90 con 0x7fa13c140220 2021-11-21T21:54:48.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0acd20 con 0x7fa13c067890 2021-11-21T21:54:48.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa13c0b0ca0 0x7fa13c0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa13c0b0ca0 0x7fa13c0b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45836/0 (socket says 172.21.2.116:45836) 2021-11-21T21:54:48.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/4194067988 learned_addr learned my addr 172.21.2.116:0/4194067988 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:48.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13b7fe700 1 --2- 172.21.2.116:0/4194067988 >> v2:172.21.2.117:3300/0 conn(0x7fa13c140220 0x7fa13c0634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13a7fc700 1 --2- 172.21.2.116:0/4194067988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c067890 0x7fa13c0ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/4194067988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c067890 msgr2=0x7fa13c0ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- 172.21.2.116:0/4194067988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c067890 0x7fa13c0ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/4194067988 >> v2:172.21.2.117:3300/0 conn(0x7fa13c140220 msgr2=0x7fa13c0634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- 172.21.2.116:0/4194067988 >> v2:172.21.2.117:3300/0 conn(0x7fa13c140220 0x7fa13c0634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/4194067988 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0acfc0 con 0x7fa13c0b0ca0 2021-11-21T21:54:48.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- 172.21.2.116:0/4194067988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c0b0ca0 0x7fa13c0b1790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fa12c002f70 tx=0x7fa12c006f40).ready entity=mon.0 client_cookie=14b26180c375892c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa139ffb700 1 -- 172.21.2.116:0/4194067988 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c00dc70 con 0x7fa13c0b0ca0 2021-11-21T21:54:48.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa139ffb700 1 -- 172.21.2.116:0/4194067988 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c000e90 con 0x7fa13c0b0ca0 2021-11-21T21:54:48.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa139ffb700 1 -- 172.21.2.116:0/4194067988 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c00be90 con 0x7fa13c0b0ca0 2021-11-21T21:54:48.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- 172.21.2.116:0/4194067988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c0b0ca0 msgr2=0x7fa13c0b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- 172.21.2.116:0/4194067988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c0b0ca0 0x7fa13c0b1790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fa12c002f70 tx=0x7fa12c006f40).stop 2021-11-21T21:54:48.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- 172.21.2.116:0/4194067988 shutdown_connections 2021-11-21T21:54:48.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- 172.21.2.116:0/4194067988 >> v2:172.21.2.117:3300/0 conn(0x7fa13c140220 0x7fa13c0634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- 172.21.2.116:0/4194067988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c067890 0x7fa13c0ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- 172.21.2.116:0/4194067988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c0b0ca0 0x7fa13c0b1790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- 172.21.2.116:0/4194067988 >> 172.21.2.116:0/4194067988 conn(0x7fa13c05bd20 msgr2=0x7fa13c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- 172.21.2.116:0/4194067988 shutdown_connections 2021-11-21T21:54:48.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- 172.21.2.116:0/4194067988 wait complete. 2021-11-21T21:54:48.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 Processor -- start 2021-11-21T21:54:48.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- start start 2021-11-21T21:54:48.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c067890 0x7fa13c1360a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c0b0ca0 0x7fa13c13a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa13c140220 0x7fa13c0aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c13eef0 con 0x7fa13c067890 2021-11-21T21:54:48.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c13e970 con 0x7fa13c140220 2021-11-21T21:54:48.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa14126d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c13f120 con 0x7fa13c0b0ca0 2021-11-21T21:54:48.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c067890 0x7fa13c1360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c067890 0x7fa13c1360a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45844/0 (socket says 172.21.2.116:45844) 2021-11-21T21:54:48.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/1060999371 learned_addr learned my addr 172.21.2.116:0/1060999371 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:48.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c0b0ca0 msgr2=0x7fa13c13a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c0b0ca0 0x7fa13c13a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa13c140220 msgr2=0x7fa13c0aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa13c140220 0x7fa13c0aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.386+0000 7fa13affd700 1 -- 172.21.2.116:0/1060999371 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c00b460 con 0x7fa13c067890 2021-11-21T21:54:48.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa13affd700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c067890 0x7fa13c1360a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fa12c002f70 tx=0x7fa13c0b1390).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa133fff700 1 -- 172.21.2.116:0/1060999371 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c01d070 con 0x7fa13c067890 2021-11-21T21:54:48.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa133fff700 1 -- 172.21.2.116:0/1060999371 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c022410 con 0x7fa13c067890 2021-11-21T21:54:48.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa133fff700 1 -- 172.21.2.116:0/1060999371 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c022770 con 0x7fa13c067890 2021-11-21T21:54:48.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c13dec0 con 0x7fa13c067890 2021-11-21T21:54:48.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c13e4d0 con 0x7fa13c067890 2021-11-21T21:54:48.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c0b4ac0 con 0x7fa13c067890 2021-11-21T21:54:48.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa133fff700 1 -- 172.21.2.116:0/1060999371 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa12c029070 con 0x7fa13c067890 2021-11-21T21:54:48.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa133fff700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1200534f0 0x7fa1200559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa133fff700 1 -- 172.21.2.116:0/1060999371 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa12c06c9d0 con 0x7fa13c067890 2021-11-21T21:54:48.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa13a7fc700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1200534f0 0x7fa1200559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa13a7fc700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1200534f0 0x7fa1200559b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa124007000 tx=0x7fa124008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.390+0000 7fa133fff700 1 -- 172.21.2.116:0/1060999371 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa12c043e60 con 0x7fa13c067890 2021-11-21T21:54:48.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.502+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa13c13fa20 con 0x7fa13c067890 2021-11-21T21:54:48.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.506+0000 7fa133fff700 1 -- 172.21.2.116:0/1060999371 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa12c047480 con 0x7fa13c067890 2021-11-21T21:54:48.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1200534f0 msgr2=0x7fa1200559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1200534f0 0x7fa1200559b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa124007000 tx=0x7fa124008040).stop 2021-11-21T21:54:48.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c067890 msgr2=0x7fa13c1360a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c067890 0x7fa13c1360a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fa12c002f70 tx=0x7fa13c0b1390).stop 2021-11-21T21:54:48.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 shutdown_connections 2021-11-21T21:54:48.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa13c140220 0x7fa13c0aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa13c0b0ca0 0x7fa13c13a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1200534f0 0x7fa1200559b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 --2- 172.21.2.116:0/1060999371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa13c067890 0x7fa13c1360a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 >> 172.21.2.116:0/1060999371 conn(0x7fa13c05bd20 msgr2=0x7fa13c0af760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 shutdown_connections 2021-11-21T21:54:48.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.510+0000 7fa14126d700 1 -- 172.21.2.116:0/1060999371 wait complete. 2021-11-21T21:54:48.517 INFO:teuthology.orchestra.run.gibba016.stderr:updated 2021-11-21T21:54:48.520 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:48.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5fbba700 1 Processor -- start 2021-11-21T21:54:48.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5fbba700 1 -- start start 2021-11-21T21:54:48.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5fbba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3b580a6aa0 0x7f3b5814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5fbba700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b580a6e80 con 0x7f3b58062410 2021-11-21T21:54:48.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5fbba700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5814a7b0 con 0x7f3b580a86b0 2021-11-21T21:54:48.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5fbba700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b580a8bc0 con 0x7f3b580a6aa0 2021-11-21T21:54:48.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5e157700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f3b58062410 0x7f3b581443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57810/0 (socket says 172.21.2.116:57810) 2021-11-21T21:54:48.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5e157700 1 -- 172.21.2.116:0/989635120 learned_addr learned my addr 172.21.2.116:0/989635120 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:48.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5d956700 1 --2- 172.21.2.116:0/989635120 >> v2:172.21.2.116:3300/0 conn(0x7f3b580a6aa0 0x7f3b5814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5d956700 1 -- 172.21.2.116:0/989635120 >> v1:172.21.2.117:6789/0 conn(0x7f3b580a86b0 legacy=0x7f3b581470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5d956700 1 -- 172.21.2.116:0/989635120 >> v1:172.21.2.116:6789/0 conn(0x7f3b58062410 legacy=0x7f3b581443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5d956700 1 -- 172.21.2.116:0/989635120 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5814aa10 con 0x7f3b580a6aa0 2021-11-21T21:54:48.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5d956700 1 --2- 172.21.2.116:0/989635120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b580a6aa0 0x7f3b5814b150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3b58061b50 tx=0x7f3b48006df0).ready entity=mon.0 client_cookie=a331aab416b8289e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5c954700 1 -- 172.21.2.116:0/989635120 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b4800d5c0 con 0x7f3b580a6aa0 2021-11-21T21:54:48.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5c954700 1 -- 172.21.2.116:0/989635120 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b48000e90 con 0x7f3b580a6aa0 2021-11-21T21:54:48.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.566+0000 7f3b5c954700 1 -- 172.21.2.116:0/989635120 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b4800d910 con 0x7f3b580a6aa0 2021-11-21T21:54:48.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/989635120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b580a6aa0 msgr2=0x7f3b5814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/989635120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b580a6aa0 0x7f3b5814b150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3b58061b50 tx=0x7f3b48006df0).stop 2021-11-21T21:54:48.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/989635120 shutdown_connections 2021-11-21T21:54:48.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/989635120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b580a6aa0 0x7f3b5814b150 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/989635120 >> 172.21.2.116:0/989635120 conn(0x7f3b5805bd20 msgr2=0x7f3b5805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/989635120 shutdown_connections 2021-11-21T21:54:48.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/989635120 wait complete. 2021-11-21T21:54:48.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 Processor -- start 2021-11-21T21:54:48.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- start start 2021-11-21T21:54:48.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b58062410 0x7f3b58156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3b580a6aa0 0x7f3b58064aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3b580a86b0 0x7f3b5815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580b42f0 con 0x7f3b58062410 2021-11-21T21:54:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58156910 con 0x7f3b580a6aa0 2021-11-21T21:54:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580b4050 con 0x7f3b580a86b0 2021-11-21T21:54:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b58062410 0x7f3b58156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b58062410 0x7f3b58156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45860/0 (socket says 172.21.2.116:45860) 2021-11-21T21:54:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 -- 172.21.2.116:0/1290182849 learned_addr learned my addr 172.21.2.116:0/1290182849 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5e157700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3b580a86b0 0x7f3b5815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 -- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3b580a86b0 msgr2=0x7f3b5815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3b580a86b0 0x7f3b5815a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 -- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3b580a6aa0 msgr2=0x7f3b58064aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3b580a6aa0 0x7f3b58064aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 -- 172.21.2.116:0/1290182849 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4800b180 con 0x7f3b58062410 2021-11-21T21:54:48.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5d956700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b58062410 0x7f3b58156420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f3b58061b50 tx=0x7f3b48012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b46ffd700 1 -- 172.21.2.116:0/1290182849 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b48002f50 con 0x7f3b58062410 2021-11-21T21:54:48.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b46ffd700 1 -- 172.21.2.116:0/1290182849 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b48013d50 con 0x7f3b58062410 2021-11-21T21:54:48.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b46ffd700 1 -- 172.21.2.116:0/1290182849 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b4801f640 con 0x7f3b58062410 2021-11-21T21:54:48.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b58159b00 con 0x7f3b58062410 2021-11-21T21:54:48.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b58064f90 con 0x7f3b58062410 2021-11-21T21:54:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5815a8d0 con 0x7f3b58062410 2021-11-21T21:54:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.570+0000 7f3b46ffd700 1 -- 172.21.2.116:0/1290182849 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3b4801c070 con 0x7f3b58062410 2021-11-21T21:54:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.574+0000 7f3b46ffd700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3b380535d0 0x7f3b38055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.574+0000 7f3b46ffd700 1 -- 172.21.2.116:0/1290182849 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3b4806c530 con 0x7f3b58062410 2021-11-21T21:54:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.574+0000 7f3b5d155700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3b380535d0 0x7f3b38055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.574+0000 7f3b5d155700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3b380535d0 0x7f3b38055a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f3b54002f90 tx=0x7f3b540069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.574+0000 7f3b46ffd700 1 -- 172.21.2.116:0/1290182849 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3b480438b0 con 0x7f3b58062410 2021-11-21T21:54:48.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b580573d0 con 0x7f3b58062410 2021-11-21T21:54:48.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b46ffd700 1 -- 172.21.2.116:0/1290182849 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+13181 (secure 0 0 0) 0x7f3b48046ed0 con 0x7f3b58062410 2021-11-21T21:54:48.664 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:48.664 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":26,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:47.139931+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:48.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3b380535d0 msgr2=0x7f3b38055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3b380535d0 0x7f3b38055a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f3b54002f90 tx=0x7f3b540069b0).stop 2021-11-21T21:54:48.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b58062410 msgr2=0x7f3b58156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b58062410 0x7f3b58156420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f3b58061b50 tx=0x7f3b48012000).stop 2021-11-21T21:54:48.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 shutdown_connections 2021-11-21T21:54:48.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3b580a86b0 0x7f3b5815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3b580a6aa0 0x7f3b58064aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3b380535d0 0x7f3b38055a90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 --2- 172.21.2.116:0/1290182849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3b58062410 0x7f3b58156420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 >> 172.21.2.116:0/1290182849 conn(0x7f3b5805bd20 msgr2=0x7f3b580b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 shutdown_connections 2021-11-21T21:54:48.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.658+0000 7f3b5fbba700 1 -- 172.21.2.116:0/1290182849 wait complete. 2021-11-21T21:54:48.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T21:54:48.671 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:48.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 Processor -- start 2021-11-21T21:54:48.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- start start 2021-11-21T21:54:48.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b4144a10 0x7ff9b40b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff9b40b3690 0x7ff9b413a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40b3c50 con 0x7ff9b40adc10 2021-11-21T21:54:48.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40b3ef0 con 0x7ff9b40b3690 2021-11-21T21:54:48.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40b4190 con 0x7ff9b4144a10 2021-11-21T21:54:48.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b3fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff9b40adc10 0x7ff9b4141020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57820/0 (socket says 172.21.2.116:57820) 2021-11-21T21:54:48.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b3fff700 1 -- 172.21.2.116:0/65141443 learned_addr learned my addr 172.21.2.116:0/65141443 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:48.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 --2- 172.21.2.116:0/65141443 >> v2:172.21.2.116:3300/0 conn(0x7ff9b40b3690 0x7ff9b413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 -- 172.21.2.116:0/65141443 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b4144a10 msgr2=0x7ff9b40b2030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:48.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 --2- 172.21.2.116:0/65141443 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b4144a10 0x7ff9b40b2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 -- 172.21.2.116:0/65141443 >> v1:172.21.2.116:6789/0 conn(0x7ff9b40adc10 legacy=0x7ff9b4141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 -- 172.21.2.116:0/65141443 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4140b80 con 0x7ff9b40b3690 2021-11-21T21:54:48.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 --2- 172.21.2.116:0/65141443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40b3690 0x7ff9b413a540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7ff9ac0032f0 tx=0x7ff9ac0091d0).ready entity=mon.0 client_cookie=5a7523b51723965a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b2ffd700 1 -- 172.21.2.116:0/65141443 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9ac009ea0 con 0x7ff9b40b3690 2021-11-21T21:54:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b2ffd700 1 -- 172.21.2.116:0/65141443 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac00b8a0 con 0x7ff9b40b3690 2021-11-21T21:54:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b2ffd700 1 -- 172.21.2.116:0/65141443 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9ac00ba50 con 0x7ff9b40b3690 2021-11-21T21:54:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- 172.21.2.116:0/65141443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40b3690 msgr2=0x7ff9b413a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- 172.21.2.116:0/65141443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40b3690 0x7ff9b413a540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7ff9ac0032f0 tx=0x7ff9ac0091d0).stop 2021-11-21T21:54:48.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- 172.21.2.116:0/65141443 shutdown_connections 2021-11-21T21:54:48.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- 172.21.2.116:0/65141443 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40b3690 0x7ff9b413a540 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- 172.21.2.116:0/65141443 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b4144a10 0x7ff9b40b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- 172.21.2.116:0/65141443 >> 172.21.2.116:0/65141443 conn(0x7ff9b405bd20 msgr2=0x7ff9b405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- 172.21.2.116:0/65141443 shutdown_connections 2021-11-21T21:54:48.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- 172.21.2.116:0/65141443 wait complete. 2021-11-21T21:54:48.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 Processor -- start 2021-11-21T21:54:48.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- start start 2021-11-21T21:54:48.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40adc10 0x7ff9b4066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b40b3690 0x7ff9b4153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b4144a10 0x7ff9b4153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40aeb90 con 0x7ff9b4144a10 2021-11-21T21:54:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40ae420 con 0x7ff9b40b3690 2021-11-21T21:54:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9ba567700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40ae650 con 0x7ff9b40adc10 2021-11-21T21:54:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b4144a10 0x7ff9b4153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b4144a10 0x7ff9b4153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45870/0 (socket says 172.21.2.116:45870) 2021-11-21T21:54:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.722+0000 7ff9b8b04700 1 -- 172.21.2.116:0/1268649284 learned_addr learned my addr 172.21.2.116:0/1268649284 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b8b04700 1 -- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40adc10 msgr2=0x7ff9b4066680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b8b04700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40adc10 0x7ff9b4066680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b8b04700 1 -- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b40b3690 msgr2=0x7ff9b4153740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b8b04700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b40b3690 0x7ff9b4153740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b8b04700 1 -- 172.21.2.116:0/1268649284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac009470 con 0x7ff9b4144a10 2021-11-21T21:54:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b8b04700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b4144a10 0x7ff9b4153c30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7ff9ac0032f0 tx=0x7ff9ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b17fa700 1 -- 172.21.2.116:0/1268649284 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9ac009ea0 con 0x7ff9b4144a10 2021-11-21T21:54:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b17fa700 1 -- 172.21.2.116:0/1268649284 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac00bec0 con 0x7ff9b4144a10 2021-11-21T21:54:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b17fa700 1 -- 172.21.2.116:0/1268649284 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9ac01f5e0 con 0x7ff9b4144a10 2021-11-21T21:54:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b40ae8f0 con 0x7ff9b4144a10 2021-11-21T21:54:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4111220 con 0x7ff9b4144a10 2021-11-21T21:54:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b40573d0 con 0x7ff9b4144a10 2021-11-21T21:54:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b17fa700 1 -- 172.21.2.116:0/1268649284 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff9ac026030 con 0x7ff9b4144a10 2021-11-21T21:54:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b17fa700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053580 0x7ff998055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b17fa700 1 -- 172.21.2.116:0/1268649284 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff9ac06c690 con 0x7ff9b4144a10 2021-11-21T21:54:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b3fff700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053580 0x7ff998055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b3fff700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053580 0x7ff998055a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7ff9b40b7000 tx=0x7ff99c007770).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.726+0000 7ff9b17fa700 1 -- 172.21.2.116:0/1268649284 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff9ac043a50 con 0x7ff9b4144a10 2021-11-21T21:54:48.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.810+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff9b4141e10 con 0x7ff9b4144a10 2021-11-21T21:54:48.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.810+0000 7ff9b17fa700 1 -- 172.21.2.116:0/1268649284 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff9ac0150b0 con 0x7ff9b4144a10 2021-11-21T21:54:48.815 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:48.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.810+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053580 msgr2=0x7ff998055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.810+0000 7ff9ba567700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053580 0x7ff998055a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7ff9b40b7000 tx=0x7ff99c007770).stop 2021-11-21T21:54:48.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.810+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b4144a10 msgr2=0x7ff9b4153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.810+0000 7ff9ba567700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b4144a10 0x7ff9b4153c30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7ff9ac0032f0 tx=0x7ff9ac016000).stop 2021-11-21T21:54:48.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 shutdown_connections 2021-11-21T21:54:48.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053580 0x7ff998055a40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b4144a10 0x7ff9b4153c30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b40b3690 0x7ff9b4153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 --2- 172.21.2.116:0/1268649284 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40adc10 0x7ff9b4066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 >> 172.21.2.116:0/1268649284 conn(0x7ff9b405bd20 msgr2=0x7ff9b413f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 shutdown_connections 2021-11-21T21:54:48.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.814+0000 7ff9ba567700 1 -- 172.21.2.116:0/1268649284 wait complete. 2021-11-21T21:54:48.822 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:48.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5c78c700 1 Processor -- start 2021-11-21T21:54:48.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5c78c700 1 -- start start 2021-11-21T21:54:48.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5c78c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fce541478c0 0x7fce540acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5c78c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce54147e80 con 0x7fce54144810 2021-11-21T21:54:48.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5c78c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540af5a0 con 0x7fce540abce0 2021-11-21T21:54:48.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5c78c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce540b1490 con 0x7fce541478c0 2021-11-21T21:54:48.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5a528700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fce54144810 0x7fce540b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57834/0 (socket says 172.21.2.116:57834) 2021-11-21T21:54:48.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce5a528700 1 -- 172.21.2.116:0/445448765 learned_addr learned my addr 172.21.2.116:0/445448765 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:48.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290393728 0 0) 0x7fce54147e80 con 0x7fce54144810 2021-11-21T21:54:48.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c003260 con 0x7fce54144810 2021-11-21T21:54:48.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4135228555 0 0) 0x7fce3c003260 con 0x7fce54144810 2021-11-21T21:54:48.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 >> v2:172.21.2.117:3300/0 conn(0x7fce541478c0 msgr2=0x7fce540acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 --2- 172.21.2.116:0/445448765 >> v2:172.21.2.117:3300/0 conn(0x7fce541478c0 0x7fce540acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 >> v1:172.21.2.117:6789/0 conn(0x7fce540abce0 legacy=0x7fce540ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540b1fd0 con 0x7fce54144810 2021-11-21T21:54:48.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fce44001660 con 0x7fce54144810 2021-11-21T21:54:48.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 >> v1:172.21.2.116:6789/0 conn(0x7fce54144810 legacy=0x7fce540b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.870+0000 7fce59526700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce3c003750 0x7fce3c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004500 0x7fce3c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce3c004f90 0x7fce3c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b1490 con 0x7fce3c004f90 2021-11-21T21:54:48.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c00be80 con 0x7fce3c004500 2021-11-21T21:54:48.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c00c040 con 0x7fce3c003750 2021-11-21T21:54:48.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce3c004f90 0x7fce3c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 -- 172.21.2.116:0/445448765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce3c003750 msgr2=0x7fce3c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce3c003750 0x7fce3c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 -- 172.21.2.116:0/445448765 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004500 msgr2=0x7fce3c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:48.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004500 0x7fce3c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004500 0x7fce3c004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 -- 172.21.2.116:0/445448765 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c003260 con 0x7fce3c004f90 2021-11-21T21:54:48.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 -- 172.21.2.116:0/445448765 reap_dead start 2021-11-21T21:54:48.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5ad29700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce3c004f90 0x7fce3c00b750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fce4c0032f0 tx=0x7fce4c006f90).ready entity=mon.0 client_cookie=3b1a19e3f0cbe7d4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce4c00b460 con 0x7fce3c004f90 2021-11-21T21:54:48.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c00b5c0 con 0x7fce3c004f90 2021-11-21T21:54:48.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59526700 1 -- 172.21.2.116:0/445448765 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce4c00b770 con 0x7fce3c004f90 2021-11-21T21:54:48.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- 172.21.2.116:0/445448765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce3c004f90 msgr2=0x7fce3c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce3c004f90 0x7fce3c00b750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fce4c0032f0 tx=0x7fce4c006f90).stop 2021-11-21T21:54:48.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- 172.21.2.116:0/445448765 shutdown_connections 2021-11-21T21:54:48.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 --2- 172.21.2.116:0/445448765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce3c004f90 0x7fce3c00b750 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- 172.21.2.116:0/445448765 >> 172.21.2.116:0/445448765 conn(0x7fce5405bd20 msgr2=0x7fce5405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- 172.21.2.116:0/445448765 shutdown_connections 2021-11-21T21:54:48.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- 172.21.2.116:0/445448765 wait complete. 2021-11-21T21:54:48.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 Processor -- start 2021-11-21T21:54:48.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- start start 2021-11-21T21:54:48.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004f90 0x7fce541461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce54142c30 0x7fce5413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce5413c8d0 0x7fce54159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b3b10 con 0x7fce54142c30 2021-11-21T21:54:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b38b0 con 0x7fce3c004f90 2021-11-21T21:54:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce54143a00 con 0x7fce5413c8d0 2021-11-21T21:54:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce54142c30 0x7fce5413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce54142c30 0x7fce5413c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45892/0 (socket says 172.21.2.116:45892) 2021-11-21T21:54:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 -- 172.21.2.116:0/92666966 learned_addr learned my addr 172.21.2.116:0/92666966 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 -- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce5413c8d0 msgr2=0x7fce54159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce5413c8d0 0x7fce54159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 -- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004f90 msgr2=0x7fce541461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004f90 0x7fce541461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 -- 172.21.2.116:0/92666966 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54143ca0 con 0x7fce54142c30 2021-11-21T21:54:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce59d27700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce54142c30 0x7fce5413c3e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fce50008000 tx=0x7fce50009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce437fe700 1 -- 172.21.2.116:0/92666966 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce50007e40 con 0x7fce54142c30 2021-11-21T21:54:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce437fe700 1 -- 172.21.2.116:0/92666966 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000f440 con 0x7fce54142c30 2021-11-21T21:54:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5413cd50 con 0x7fce54142c30 2021-11-21T21:54:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce437fe700 1 -- 172.21.2.116:0/92666966 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5000f7a0 con 0x7fce54142c30 2021-11-21T21:54:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.874+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce54142850 con 0x7fce54142c30 2021-11-21T21:54:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.878+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54143e30 con 0x7fce54142c30 2021-11-21T21:54:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.878+0000 7fce437fe700 1 -- 172.21.2.116:0/92666966 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fce5000f900 con 0x7fce54142c30 2021-11-21T21:54:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.878+0000 7fce437fe700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce340536d0 0x7fce34055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:48.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.878+0000 7fce437fe700 1 -- 172.21.2.116:0/92666966 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fce5006b180 con 0x7fce54142c30 2021-11-21T21:54:48.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.878+0000 7fce5a528700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce340536d0 0x7fce34055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:48.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.878+0000 7fce5a528700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce340536d0 0x7fce34055b90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fce540b4e00 tx=0x7fce540aee70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:48.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.878+0000 7fce437fe700 1 -- 172.21.2.116:0/92666966 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce50042610 con 0x7fce54142c30 2021-11-21T21:54:48.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.962+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fce540573d0 con 0x7fce54142c30 2021-11-21T21:54:48.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.962+0000 7fce437fe700 1 -- 172.21.2.116:0/92666966 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fce50045c30 con 0x7fce54142c30 2021-11-21T21:54:48.967 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:48.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.962+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce340536d0 msgr2=0x7fce34055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.962+0000 7fce5c78c700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce340536d0 0x7fce34055b90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fce540b4e00 tx=0x7fce540aee70).stop 2021-11-21T21:54:48.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.962+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce54142c30 msgr2=0x7fce5413c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:48.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.962+0000 7fce5c78c700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce54142c30 0x7fce5413c3e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fce50008000 tx=0x7fce50009040).stop 2021-11-21T21:54:48.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 shutdown_connections 2021-11-21T21:54:48.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce5413c8d0 0x7fce54159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce340536d0 0x7fce34055b90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce54142c30 0x7fce5413c3e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:48.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 --2- 172.21.2.116:0/92666966 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce3c004f90 0x7fce541461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:48.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 >> 172.21.2.116:0/92666966 conn(0x7fce5405bd20 msgr2=0x7fce540b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:48.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 shutdown_connections 2021-11-21T21:54:48.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:48.966+0000 7fce5c78c700 1 -- 172.21.2.116:0/92666966 wait complete. 2021-11-21T21:54:48.975 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T21:54:49.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabbba62700 1 Processor -- start 2021-11-21T21:54:49.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabbba62700 1 -- start start 2021-11-21T21:54:49.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabbba62700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb40b1a40 0x7fabb413c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabbba62700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb40b00d0 con 0x7fabb40aed90 2021-11-21T21:54:49.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabbba62700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb40b2880 con 0x7fabb40b24a0 2021-11-21T21:54:49.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabbba62700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40adba0 con 0x7fabb40b1a40 2021-11-21T21:54:49.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb8ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fabb40aed90 0x7fabb413c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57856/0 (socket says 172.21.2.116:57856) 2021-11-21T21:54:49.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb8ffd700 1 -- 172.21.2.116:0/820713734 learned_addr learned my addr 172.21.2.116:0/820713734 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:49.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543449931 0 0) 0x7fabb40b00d0 con 0x7fabb40aed90 2021-11-21T21:54:49.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9c003260 con 0x7fabb40aed90 2021-11-21T21:54:49.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3670562897 0 0) 0x7fab9c003260 con 0x7fabb40aed90 2021-11-21T21:54:49.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb40b1a40 msgr2=0x7fabb413c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:49.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb40b1a40 0x7fabb413c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 >> v1:172.21.2.117:6789/0 conn(0x7fabb40b24a0 legacy=0x7fabb40ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:49.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40b14a0 con 0x7fabb40aed90 2021-11-21T21:54:49.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fabb0001680 con 0x7fabb40aed90 2021-11-21T21:54:49.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 >> v1:172.21.2.116:6789/0 conn(0x7fabb40aed90 legacy=0x7fabb413c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab9c003800 0x7fab9c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c004500 0x7fab9c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb40b1a40 0x7fab9c005720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40adba0 con 0x7fab9c003800 2021-11-21T21:54:49.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0052b0 con 0x7fab9c004500 2021-11-21T21:54:49.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c005c10 con 0x7fabb40b1a40 2021-11-21T21:54:49.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb97fe700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab9c003800 0x7fab9c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb97fe700 1 -- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb40b1a40 msgr2=0x7fab9c005720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:49.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb97fe700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb40b1a40 0x7fab9c005720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb97fe700 1 -- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c004500 msgr2=0x7fab9c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:49.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb97fe700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c004500 0x7fab9c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb97fe700 1 -- 172.21.2.116:0/820713734 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c003260 con 0x7fab9c003800 2021-11-21T21:54:49.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7fabb97fe700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab9c003800 0x7fab9c003c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7faba4000f70 tx=0x7faba40079e0).ready entity=mon.0 client_cookie=3acbe4c3d5012c44 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:49.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba4007cf0 con 0x7fab9c003800 2021-11-21T21:54:49.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4014410 con 0x7fab9c003800 2021-11-21T21:54:49.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.022+0000 7faba3fff700 1 -- 172.21.2.116:0/820713734 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba4014770 con 0x7fab9c003800 2021-11-21T21:54:49.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/820713734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab9c003800 msgr2=0x7fab9c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 --2- 172.21.2.116:0/820713734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab9c003800 0x7fab9c003c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7faba4000f70 tx=0x7faba40079e0).stop 2021-11-21T21:54:49.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 -- 172.21.2.116:0/820713734 reap_dead start 2021-11-21T21:54:49.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/820713734 shutdown_connections 2021-11-21T21:54:49.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/820713734 >> 172.21.2.116:0/820713734 conn(0x7fabb405fd00 msgr2=0x7fabb40600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:49.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/820713734 shutdown_connections 2021-11-21T21:54:49.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/820713734 wait complete. 2021-11-21T21:54:49.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 Processor -- start 2021-11-21T21:54:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- start start 2021-11-21T21:54:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c003800 0x7fabb413e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb410ec50 0x7fabb41511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb413f890 0x7fabb41516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb413ec80 con 0x7fabb413f890 2021-11-21T21:54:49.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4110e20 con 0x7fab9c003800 2021-11-21T21:54:49.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb413eeb0 con 0x7fabb410ec50 2021-11-21T21:54:49.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb413f890 0x7fabb41516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb413f890 0x7fabb41516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45914/0 (socket says 172.21.2.116:45914) 2021-11-21T21:54:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 -- 172.21.2.116:0/3096099178 learned_addr learned my addr 172.21.2.116:0/3096099178 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb97fe700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c003800 0x7fabb413e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 -- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb410ec50 msgr2=0x7fabb41511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb8ffd700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb410ec50 0x7fabb41511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb410ec50 0x7fabb41511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 -- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c003800 msgr2=0x7fabb413e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c003800 0x7fabb413e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 -- 172.21.2.116:0/3096099178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb413f150 con 0x7fabb413f890 2021-11-21T21:54:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb97fe700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c003800 0x7fabb413e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb8ffd700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb410ec50 0x7fabb41511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb9fff700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb413f890 0x7fabb41516c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fabb0001390 tx=0x7fabac00a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7faba27fc700 1 -- 172.21.2.116:0/3096099178 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabac00ce40 con 0x7fabb413f890 2021-11-21T21:54:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7faba27fc700 1 -- 172.21.2.116:0/3096099178 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac005c30 con 0x7fabb413f890 2021-11-21T21:54:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7faba27fc700 1 -- 172.21.2.116:0/3096099178 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabac007070 con 0x7fabb413f890 2021-11-21T21:54:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb413f3f0 con 0x7fabb413f890 2021-11-21T21:54:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb413d0d0 con 0x7fabb413f890 2021-11-21T21:54:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb413f5b0 con 0x7fabb413f890 2021-11-21T21:54:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7faba27fc700 1 -- 172.21.2.116:0/3096099178 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fabac010030 con 0x7fabb413f890 2021-11-21T21:54:49.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7faba27fc700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab940536a0 0x7fab94055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7faba27fc700 1 -- 172.21.2.116:0/3096099178 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fabac06ab70 con 0x7fabb413f890 2021-11-21T21:54:49.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.026+0000 7fabb97fe700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab940536a0 0x7fab94055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.030+0000 7fabb97fe700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab940536a0 0x7fab94055b60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7faba4017000 tx=0x7faba4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:49.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.030+0000 7faba27fc700 1 -- 172.21.2.116:0/3096099178 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fabac042000 con 0x7fabb413f890 2021-11-21T21:54:49.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fabb40b0300 con 0x7fabb413f890 2021-11-21T21:54:49.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7faba27fc700 1 -- 172.21.2.116:0/3096099178 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v26) v1 ==== 84+0+143 (secure 0 0 0) 0x7fabac045620 con 0x7fabb413f890 2021-11-21T21:54:49.118 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:49.118 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T21:54:49.118 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:49.119 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T21:54:49.119 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T21:54:49.119 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:54:49.119 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:54:49.119 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T21:54:49.120 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T21:54:49.120 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:54:49.120 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T21:54:49.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab940536a0 msgr2=0x7fab94055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab940536a0 0x7fab94055b60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7faba4017000 tx=0x7faba4015000).stop 2021-11-21T21:54:49.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb413f890 msgr2=0x7fabb41516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb413f890 0x7fabb41516c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fabb0001390 tx=0x7fabac00a060).stop 2021-11-21T21:54:49.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 shutdown_connections 2021-11-21T21:54:49.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab940536a0 0x7fab94055b60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:49.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb413f890 0x7fabb41516c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:49.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fabb410ec50 0x7fabb41511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:49.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 --2- 172.21.2.116:0/3096099178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab9c003800 0x7fabb413e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:49.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 >> 172.21.2.116:0/3096099178 conn(0x7fabb405fd00 msgr2=0x7fabb4069d00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:49.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 shutdown_connections 2021-11-21T21:54:49.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.114+0000 7fabbba62700 1 -- 172.21.2.116:0/3096099178 wait complete. 2021-11-21T21:54:49.125 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T21:54:49.126 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T21:54:49.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 Processor -- start 2021-11-21T21:54:49.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 -- start start 2021-11-21T21:54:49.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb300144810 0x7fb3001478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb3000abce0 0x7fb3000b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000b1db0 0x7fb3000ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb300147e80 con 0x7fb300144810 2021-11-21T21:54:49.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3000af5a0 con 0x7fb3000abce0 2021-11-21T21:54:49.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb307471700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000b1490 con 0x7fb3000b1db0 2021-11-21T21:54:49.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb300144810 0x7fb3001478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb300144810 0x7fb3001478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45922/0 (socket says 172.21.2.116:45922) 2021-11-21T21:54:49.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 -- 172.21.2.116:0/2143493832 learned_addr learned my addr 172.21.2.116:0/2143493832 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:49.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 -- 172.21.2.116:0/2143493832 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000b1db0 msgr2=0x7fb3000ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:49.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 --2- 172.21.2.116:0/2143493832 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000b1db0 0x7fb3000ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 -- 172.21.2.116:0/2143493832 >> v2:172.21.2.117:3300/0 conn(0x7fb3000abce0 msgr2=0x7fb3000b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:49.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 --2- 172.21.2.116:0/2143493832 >> v2:172.21.2.117:3300/0 conn(0x7fb3000abce0 0x7fb3000b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 -- 172.21.2.116:0/2143493832 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000af060 con 0x7fb300144810 2021-11-21T21:54:49.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb30520d700 1 --2- 172.21.2.116:0/2143493832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb300144810 0x7fb3001478c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fb2f0002f70 tx=0x7fb2f0006eb0).ready entity=mon.0 client_cookie=8241651a1da5d8ab server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:49.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb2effff700 1 -- 172.21.2.116:0/2143493832 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f000d570 con 0x7fb300144810 2021-11-21T21:54:49.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb2effff700 1 -- 172.21.2.116:0/2143493832 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f0000e90 con 0x7fb300144810 2021-11-21T21:54:49.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.174+0000 7fb2effff700 1 -- 172.21.2.116:0/2143493832 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f000d900 con 0x7fb300144810 2021-11-21T21:54:49.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/2143493832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb300144810 msgr2=0x7fb3001478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 --2- 172.21.2.116:0/2143493832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb300144810 0x7fb3001478c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fb2f0002f70 tx=0x7fb2f0006eb0).stop 2021-11-21T21:54:49.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/2143493832 shutdown_connections 2021-11-21T21:54:49.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 --2- 172.21.2.116:0/2143493832 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000b1db0 0x7fb3000ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 --2- 172.21.2.116:0/2143493832 >> v2:172.21.2.117:3300/0 conn(0x7fb3000abce0 0x7fb3000b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 --2- 172.21.2.116:0/2143493832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb300144810 0x7fb3001478c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:49.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/2143493832 >> 172.21.2.116:0/2143493832 conn(0x7fb30005bd20 msgr2=0x7fb30005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:49.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/2143493832 shutdown_connections 2021-11-21T21:54:49.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/2143493832 wait complete. 2021-11-21T21:54:49.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 Processor -- start 2021-11-21T21:54:49.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- start start 2021-11-21T21:54:49.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000abce0 0x7fb3001499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb300144810 0x7fb3001466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb3001455d0 0x7fb30013cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb300147510 con 0x7fb3001455d0 2021-11-21T21:54:49.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb300144d20 con 0x7fb300144810 2021-11-21T21:54:49.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb300146280 con 0x7fb3000abce0 2021-11-21T21:54:49.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb3001455d0 0x7fb30013cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb3001455d0 0x7fb30013cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45928/0 (socket says 172.21.2.116:45928) 2021-11-21T21:54:49.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 -- 172.21.2.116:0/189109441 learned_addr learned my addr 172.21.2.116:0/189109441 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:49.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 -- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000abce0 msgr2=0x7fb3001499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000abce0 0x7fb3001499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 -- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb300144810 msgr2=0x7fb3001466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:49.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb300144810 0x7fb3001466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:49.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 -- 172.21.2.116:0/189109441 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f000b1e0 con 0x7fb3001455d0 2021-11-21T21:54:49.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb305a0e700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb3001455d0 0x7fb30013cd00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fb30013f680 tx=0x7fb2f80090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:49.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb2ee7fc700 1 -- 172.21.2.116:0/189109441 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f8001520 con 0x7fb3001455d0 2021-11-21T21:54:49.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb2ee7fc700 1 -- 172.21.2.116:0/189109441 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f8001680 con 0x7fb3001455d0 2021-11-21T21:54:49.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb2ee7fc700 1 -- 172.21.2.116:0/189109441 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f80019e0 con 0x7fb3001455d0 2021-11-21T21:54:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb300147270 con 0x7fb3001455d0 2021-11-21T21:54:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3000a81a0 con 0x7fb3001455d0 2021-11-21T21:54:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3001463c0 con 0x7fb3001455d0 2021-11-21T21:54:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb2ee7fc700 1 -- 172.21.2.116:0/189109441 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb2f8018030 con 0x7fb3001455d0 2021-11-21T21:54:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb2ee7fc700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2e00536a0 0x7fb2e0055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb2ee7fc700 1 -- 172.21.2.116:0/189109441 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb2f806af70 con 0x7fb3001455d0 2021-11-21T21:54:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.178+0000 7fb30520d700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2e00536a0 0x7fb2e0055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.182+0000 7fb30520d700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2e00536a0 0x7fb2e0055b60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fb2f0002f70 tx=0x7fb2f0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.182+0000 7fb2ee7fc700 1 -- 172.21.2.116:0/189109441 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb2f80424b0 con 0x7fb3001455d0 2021-11-21T21:54:49.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:49.262+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb3000af300 con 0x7fb3001455d0 2021-11-21T21:54:50.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.110+0000 7fb2ee7fc700 1 -- 172.21.2.116:0/189109441 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v27) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb2f8016030 con 0x7fb3001455d0 2021-11-21T21:54:50.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.114+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2e00536a0 msgr2=0x7fb2e0055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:50.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.114+0000 7fb307471700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2e00536a0 0x7fb2e0055b60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fb2f0002f70 tx=0x7fb2f0015000).stop 2021-11-21T21:54:50.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.114+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb3001455d0 msgr2=0x7fb30013cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:50.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.114+0000 7fb307471700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb3001455d0 0x7fb30013cd00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fb30013f680 tx=0x7fb2f80090f0).stop 2021-11-21T21:54:50.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 shutdown_connections 2021-11-21T21:54:50.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2e00536a0 0x7fb2e0055b60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:50.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb3001455d0 0x7fb30013cd00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:50.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb300144810 0x7fb3001466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:50.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 --2- 172.21.2.116:0/189109441 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb3000abce0 0x7fb3001499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:50.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 >> 172.21.2.116:0/189109441 conn(0x7fb30005bd20 msgr2=0x7fb300148280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:50.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 shutdown_connections 2021-11-21T21:54:50.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.118+0000 7fb307471700 1 -- 172.21.2.116:0/189109441 wait complete. 2021-11-21T21:54:50.123 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T21:54:50.131 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T21:54:50.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3eeb4700 1 Processor -- start 2021-11-21T21:54:50.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3eeb4700 1 -- start start 2021-11-21T21:54:50.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3eeb4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2c38144810 0x7f2c381478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:50.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3eeb4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2c380abce0 0x7f2c380b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:50.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3eeb4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c38147e80 con 0x7f2c380b1db0 2021-11-21T21:54:50.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3eeb4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c380b1490 con 0x7f2c380abce0 2021-11-21T21:54:50.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3eeb4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c380af060 con 0x7f2c38144810 2021-11-21T21:54:50.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3d451700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f2c380b1db0 0x7f2c380ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57888/0 (socket says 172.21.2.116:57888) 2021-11-21T21:54:50.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c3d451700 1 -- 172.21.2.116:0/3984514586 learned_addr learned my addr 172.21.2.116:0/3984514586 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:50.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c37fff700 1 --2- 172.21.2.116:0/3984514586 >> v2:172.21.2.116:3300/0 conn(0x7f2c380abce0 0x7f2c380b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:50.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c377fe700 1 -- 172.21.2.116:0/3984514586 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251539092 0 0) 0x7f2c38147e80 con 0x7f2c380b1db0 2021-11-21T21:54:50.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c37fff700 1 -- 172.21.2.116:0/3984514586 >> v2:172.21.2.117:3300/0 conn(0x7f2c38144810 msgr2=0x7f2c381478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:50.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c37fff700 1 --2- 172.21.2.116:0/3984514586 >> v2:172.21.2.117:3300/0 conn(0x7f2c38144810 0x7f2c381478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:50.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c37fff700 1 -- 172.21.2.116:0/3984514586 >> v1:172.21.2.116:6789/0 conn(0x7f2c380b1db0 legacy=0x7f2c380ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:50.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c37fff700 1 -- 172.21.2.116:0/3984514586 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380af5a0 con 0x7f2c380abce0 2021-11-21T21:54:50.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c37fff700 1 --2- 172.21.2.116:0/3984514586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380abce0 0x7f2c380b18c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f2c28002f70 tx=0x7f2c28006df0).ready entity=mon.0 client_cookie=7f190256896efcc4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:50.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.182+0000 7f2c377fe700 1 -- 172.21.2.116:0/3984514586 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c2800d5c0 con 0x7f2c380abce0 2021-11-21T21:54:50.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c377fe700 1 -- 172.21.2.116:0/3984514586 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28000e90 con 0x7f2c380abce0 2021-11-21T21:54:50.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c377fe700 1 -- 172.21.2.116:0/3984514586 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c2800d910 con 0x7f2c380abce0 2021-11-21T21:54:50.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/3984514586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380abce0 msgr2=0x7f2c380b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:50.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/3984514586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380abce0 0x7f2c380b18c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f2c28002f70 tx=0x7f2c28006df0).stop 2021-11-21T21:54:50.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/3984514586 shutdown_connections 2021-11-21T21:54:50.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/3984514586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380abce0 0x7f2c380b18c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:50.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/3984514586 >> v2:172.21.2.117:3300/0 conn(0x7f2c38144810 0x7f2c381478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:50.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/3984514586 >> 172.21.2.116:0/3984514586 conn(0x7f2c3805bd20 msgr2=0x7f2c3805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:50.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/3984514586 shutdown_connections 2021-11-21T21:54:50.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/3984514586 wait complete. 2021-11-21T21:54:50.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 Processor -- start 2021-11-21T21:54:50.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- start start 2021-11-21T21:54:50.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2c380abce0 0x7f2c3813c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:50.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2c38144810 0x7f2c381475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:50.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380b49b0 0x7f2c381565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:50.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380b32d0 con 0x7f2c380b49b0 2021-11-21T21:54:50.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c38146ea0 con 0x7f2c38144810 2021-11-21T21:54:50.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c38147a90 con 0x7f2c380abce0 2021-11-21T21:54:50.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380b49b0 0x7f2c381565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:50.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380b49b0 0x7f2c381565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45942/0 (socket says 172.21.2.116:45942) 2021-11-21T21:54:50.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 -- 172.21.2.116:0/321210413 learned_addr learned my addr 172.21.2.116:0/321210413 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:50.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 -- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2c380abce0 msgr2=0x7f2c3813c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:50.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2c380abce0 0x7f2c3813c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:50.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 -- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2c38144810 msgr2=0x7f2c381475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:50.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2c38144810 0x7f2c381475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:50.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 -- 172.21.2.116:0/321210413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2800b180 con 0x7f2c380b49b0 2021-11-21T21:54:50.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3d451700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380b49b0 0x7f2c381565c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f2c30001250 tx=0x7f2c30007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:50.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c35ffb700 1 -- 172.21.2.116:0/321210413 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c30005c70 con 0x7f2c380b49b0 2021-11-21T21:54:50.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c35ffb700 1 -- 172.21.2.116:0/321210413 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30005dd0 con 0x7f2c380b49b0 2021-11-21T21:54:50.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c380b3030 con 0x7f2c380b49b0 2021-11-21T21:54:50.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c35ffb700 1 -- 172.21.2.116:0/321210413 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c30004a90 con 0x7f2c380b49b0 2021-11-21T21:54:50.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c38146a70 con 0x7f2c380b49b0 2021-11-21T21:54:50.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c38156c50 con 0x7f2c380b49b0 2021-11-21T21:54:50.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c35ffb700 1 -- 172.21.2.116:0/321210413 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2c30004bf0 con 0x7f2c380b49b0 2021-11-21T21:54:50.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c35ffb700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2c1c053580 0x7f2c1c055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:50.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c35ffb700 1 -- 172.21.2.116:0/321210413 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f2c3006b4a0 con 0x7f2c380b49b0 2021-11-21T21:54:50.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.186+0000 7f2c3cc50700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2c1c053580 0x7f2c1c055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:50.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.190+0000 7f2c3cc50700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2c1c053580 0x7f2c1c055a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f2c20002f70 tx=0x7f2c200069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:50.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.190+0000 7f2c35ffb700 1 -- 172.21.2.116:0/321210413 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2c300427d0 con 0x7f2c380b49b0 2021-11-21T21:54:50.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:50.274+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f2c38146d00 con 0x7f2c380b49b0 2021-11-21T21:54:51.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.158+0000 7f2c35ffb700 1 -- 172.21.2.116:0/321210413 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v30) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2c38146d00 con 0x7f2c380b49b0 2021-11-21T21:54:51.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2c1c053580 msgr2=0x7f2c1c055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2c1c053580 0x7f2c1c055a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f2c20002f70 tx=0x7f2c200069e0).stop 2021-11-21T21:54:51.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380b49b0 msgr2=0x7f2c381565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380b49b0 0x7f2c381565c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f2c30001250 tx=0x7f2c30007dc0).stop 2021-11-21T21:54:51.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 shutdown_connections 2021-11-21T21:54:51.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2c1c053580 0x7f2c1c055a40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2c380b49b0 0x7f2c381565c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2c38144810 0x7f2c381475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 --2- 172.21.2.116:0/321210413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2c380abce0 0x7f2c3813c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 >> 172.21.2.116:0/321210413 conn(0x7f2c3805bd20 msgr2=0x7f2c38149b70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 shutdown_connections 2021-11-21T21:54:51.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.162+0000 7f2c3eeb4700 1 -- 172.21.2.116:0/321210413 wait complete. 2021-11-21T21:54:51.170 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T21:54:51.179 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T21:54:51.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 Processor -- start 2021-11-21T21:54:51.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- start start 2021-11-21T21:54:51.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe24c0a8670 0x7fe24c147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe24c143e60 0x7fe24c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24c0b1940 con 0x7fe24c0b11c0 2021-11-21T21:54:51.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24c0b15a0 con 0x7fe24c0a8670 2021-11-21T21:54:51.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24c0b1b70 con 0x7fe24c143e60 2021-11-21T21:54:51.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24b7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe24c0b11c0 0x7fe24c147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57904/0 (socket says 172.21.2.116:57904) 2021-11-21T21:54:51.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24b7fe700 1 -- 172.21.2.116:0/2179223705 learned_addr learned my addr 172.21.2.116:0/2179223705 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:51.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24affd700 1 --2- 172.21.2.116:0/2179223705 >> v2:172.21.2.116:3300/0 conn(0x7fe24c0a8670 0x7fe24c147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24affd700 1 -- 172.21.2.116:0/2179223705 >> v2:172.21.2.117:3300/0 conn(0x7fe24c143e60 msgr2=0x7fe24c1442d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:51.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24affd700 1 --2- 172.21.2.116:0/2179223705 >> v2:172.21.2.117:3300/0 conn(0x7fe24c143e60 0x7fe24c1442d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24affd700 1 -- 172.21.2.116:0/2179223705 >> v1:172.21.2.116:6789/0 conn(0x7fe24c0b11c0 legacy=0x7fe24c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24affd700 1 -- 172.21.2.116:0/2179223705 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0b1e10 con 0x7fe24c0a8670 2021-11-21T21:54:51.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24affd700 1 --2- 172.21.2.116:0/2179223705 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0a8670 0x7fe24c147940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fe234002f70 tx=0x7fe234006df0).ready entity=mon.0 client_cookie=89ce91e80a623df6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24a7fc700 1 -- 172.21.2.116:0/2179223705 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe23400d5c0 con 0x7fe24c0a8670 2021-11-21T21:54:51.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24a7fc700 1 -- 172.21.2.116:0/2179223705 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234000e90 con 0x7fe24c0a8670 2021-11-21T21:54:51.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe24a7fc700 1 -- 172.21.2.116:0/2179223705 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe23400d950 con 0x7fe24c0a8670 2021-11-21T21:54:51.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- 172.21.2.116:0/2179223705 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0a8670 msgr2=0x7fe24c147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 --2- 172.21.2.116:0/2179223705 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0a8670 0x7fe24c147940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fe234002f70 tx=0x7fe234006df0).stop 2021-11-21T21:54:51.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- 172.21.2.116:0/2179223705 shutdown_connections 2021-11-21T21:54:51.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 --2- 172.21.2.116:0/2179223705 >> v2:172.21.2.117:3300/0 conn(0x7fe24c143e60 0x7fe24c1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 --2- 172.21.2.116:0/2179223705 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0a8670 0x7fe24c147940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- 172.21.2.116:0/2179223705 >> 172.21.2.116:0/2179223705 conn(0x7fe24c05bd20 msgr2=0x7fe24c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- 172.21.2.116:0/2179223705 shutdown_connections 2021-11-21T21:54:51.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.230+0000 7fe251f38700 1 -- 172.21.2.116:0/2179223705 wait complete. 2021-11-21T21:54:51.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 Processor -- start 2021-11-21T21:54:51.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 -- start start 2021-11-21T21:54:51.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe24c0a8670 0x7fe24c14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0b11c0 0x7fe24c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe24c143e60 0x7fe24c159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c064260 con 0x7fe24c0b11c0 2021-11-21T21:54:51.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c065a00 con 0x7fe24c0a8670 2021-11-21T21:54:51.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c065ca0 con 0x7fe24c143e60 2021-11-21T21:54:51.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0b11c0 0x7fe24c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0b11c0 0x7fe24c159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45954/0 (socket says 172.21.2.116:45954) 2021-11-21T21:54:51.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 -- 172.21.2.116:0/2519222807 learned_addr learned my addr 172.21.2.116:0/2519222807 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:51.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 -- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe24c143e60 msgr2=0x7fe24c159e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:51.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe24c143e60 0x7fe24c159e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 -- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe24c0a8670 msgr2=0x7fe24c14a4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:51.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe24c0a8670 0x7fe24c14a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 -- 172.21.2.116:0/2519222807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23400b180 con 0x7fe24c0b11c0 2021-11-21T21:54:51.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24affd700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0b11c0 0x7fe24c159920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fe234002f70 tx=0x7fe234014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe248ff9700 1 -- 172.21.2.116:0/2519222807 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe23400de70 con 0x7fe24c0b11c0 2021-11-21T21:54:51.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe248ff9700 1 -- 172.21.2.116:0/2519222807 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23401e400 con 0x7fe24c0b11c0 2021-11-21T21:54:51.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c065f40 con 0x7fe24c0b11c0 2021-11-21T21:54:51.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe248ff9700 1 -- 172.21.2.116:0/2519222807 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe23401e760 con 0x7fe24c0b11c0 2021-11-21T21:54:51.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c14b040 con 0x7fe24c0b11c0 2021-11-21T21:54:51.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c0660d0 con 0x7fe24c0b11c0 2021-11-21T21:54:51.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe248ff9700 1 -- 172.21.2.116:0/2519222807 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe234025030 con 0x7fe24c0b11c0 2021-11-21T21:54:51.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe248ff9700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe230053660 0x7fe230055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe248ff9700 1 -- 172.21.2.116:0/2519222807 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fe23406ca20 con 0x7fe24c0b11c0 2021-11-21T21:54:51.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.234+0000 7fe24b7fe700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe230053660 0x7fe230055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.238+0000 7fe24b7fe700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe230053660 0x7fe230055b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe24c147180 tx=0x7fe23c007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.238+0000 7fe248ff9700 1 -- 172.21.2.116:0/2519222807 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe234043b20 con 0x7fe24c0b11c0 2021-11-21T21:54:51.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.346+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe24c0b20b0 con 0x7fe24c0b11c0 2021-11-21T21:54:51.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe248ff9700 1 -- 172.21.2.116:0/2519222807 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v10) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe2340230d0 con 0x7fe24c0b11c0 2021-11-21T21:54:51.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe230053660 msgr2=0x7fe230055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe230053660 0x7fe230055b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe24c147180 tx=0x7fe23c007790).stop 2021-11-21T21:54:51.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0b11c0 msgr2=0x7fe24c159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0b11c0 0x7fe24c159920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fe234002f70 tx=0x7fe234014000).stop 2021-11-21T21:54:51.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 shutdown_connections 2021-11-21T21:54:51.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe24c143e60 0x7fe24c159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe230053660 0x7fe230055b20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe24c0b11c0 0x7fe24c159920 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0x7fe234002f70 tx=0x7fe234014000).stop 2021-11-21T21:54:51.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 --2- 172.21.2.116:0/2519222807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe24c0a8670 0x7fe24c14a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 >> 172.21.2.116:0/2519222807 conn(0x7fe24c05bd20 msgr2=0x7fe24c145970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 shutdown_connections 2021-11-21T21:54:51.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.354+0000 7fe251f38700 1 -- 172.21.2.116:0/2519222807 wait complete. 2021-11-21T21:54:51.363 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 5 and data pool 6 2021-11-21T21:54:51.366 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:51.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.410+0000 7f0ba5b2e700 1 Processor -- start 2021-11-21T21:54:51.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.410+0000 7f0ba5b2e700 1 -- start start 2021-11-21T21:54:51.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 0x7f0ba0140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ba00a9500 0x7f0ba005fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba005c440 con 0x7f0ba005c060 2021-11-21T21:54:51.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba0143630 con 0x7f0ba00a9500 2021-11-21T21:54:51.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00a9c10 con 0x7f0ba0060c50 2021-11-21T21:54:51.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0ba005c060 0x7f0ba013fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57930/0 (socket says 172.21.2.116:57930) 2021-11-21T21:54:51.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 -- 172.21.2.116:0/2269442672 learned_addr learned my addr 172.21.2.116:0/2269442672 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:51.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9ffff700 1 --2- 172.21.2.116:0/2269442672 >> v2:172.21.2.116:3300/0 conn(0x7f0ba00a9500 0x7f0ba005fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9e7fc700 1 -- 172.21.2.116:0/2269442672 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431631096 0 0) 0x7f0ba005c440 con 0x7f0ba005c060 2021-11-21T21:54:51.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9e7fc700 1 -- 172.21.2.116:0/2269442672 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b80003260 con 0x7f0ba005c060 2021-11-21T21:54:51.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9ffff700 1 -- 172.21.2.116:0/2269442672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 msgr2=0x7f0ba0140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:51.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9ffff700 1 --2- 172.21.2.116:0/2269442672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 0x7f0ba0140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9ffff700 1 -- 172.21.2.116:0/2269442672 >> v1:172.21.2.116:6789/0 conn(0x7f0ba005c060 legacy=0x7f0ba013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9ffff700 1 -- 172.21.2.116:0/2269442672 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba01438a0 con 0x7f0ba00a9500 2021-11-21T21:54:51.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9ffff700 1 --2- 172.21.2.116:0/2269442672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba00a9500 0x7f0ba005fe70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0b980032f0 tx=0x7f0b980091d0).ready entity=mon.0 client_cookie=5e177c74b77b3fef server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9e7fc700 1 -- 172.21.2.116:0/2269442672 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b9800b6c0 con 0x7f0ba00a9500 2021-11-21T21:54:51.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9e7fc700 1 -- 172.21.2.116:0/2269442672 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9800b820 con 0x7f0ba00a9500 2021-11-21T21:54:51.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9e7fc700 1 -- 172.21.2.116:0/2269442672 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b9800b990 con 0x7f0ba00a9500 2021-11-21T21:54:51.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/2269442672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba00a9500 msgr2=0x7f0ba005fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/2269442672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba00a9500 0x7f0ba005fe70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0b980032f0 tx=0x7f0b980091d0).stop 2021-11-21T21:54:51.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/2269442672 shutdown_connections 2021-11-21T21:54:51.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/2269442672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba00a9500 0x7f0ba005fe70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/2269442672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 0x7f0ba0140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/2269442672 >> 172.21.2.116:0/2269442672 conn(0x7f0ba00b6000 msgr2=0x7f0ba00b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/2269442672 shutdown_connections 2021-11-21T21:54:51.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/2269442672 wait complete. 2021-11-21T21:54:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 Processor -- start 2021-11-21T21:54:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- start start 2021-11-21T21:54:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba005c060 0x7f0ba013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 0x7f0ba01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ba00a9500 0x7f0ba0144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00acea0 con 0x7f0ba005c060 2021-11-21T21:54:51.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0146a30 con 0x7f0ba00a9500 2021-11-21T21:54:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00abe90 con 0x7f0ba0060c50 2021-11-21T21:54:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba005c060 0x7f0ba013dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba005c060 0x7f0ba013dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45980/0 (socket says 172.21.2.116:45980) 2021-11-21T21:54:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 -- 172.21.2.116:0/98005316 learned_addr learned my addr 172.21.2.116:0/98005316 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9effd700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 0x7f0ba01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9ffff700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ba00a9500 0x7f0ba0144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 -- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 msgr2=0x7f0ba01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 0x7f0ba01445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 -- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ba00a9500 msgr2=0x7f0ba0144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ba00a9500 0x7f0ba0144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 -- 172.21.2.116:0/98005316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98009470 con 0x7f0ba005c060 2021-11-21T21:54:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9f7fe700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba005c060 0x7f0ba013dd70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f0b90001110 tx=0x7f0b90007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9cff9700 1 -- 172.21.2.116:0/98005316 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b900072e0 con 0x7f0ba005c060 2021-11-21T21:54:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9cff9700 1 -- 172.21.2.116:0/98005316 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90007440 con 0x7f0ba005c060 2021-11-21T21:54:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba0146c60 con 0x7f0ba005c060 2021-11-21T21:54:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0b9cff9700 1 -- 172.21.2.116:0/98005316 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b90005800 con 0x7f0ba005c060 2021-11-21T21:54:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.414+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba0144170 con 0x7f0ba005c060 2021-11-21T21:54:51.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.418+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba0143b40 con 0x7f0ba005c060 2021-11-21T21:54:51.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.418+0000 7f0b9cff9700 1 -- 172.21.2.116:0/98005316 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0b90010070 con 0x7f0ba005c060 2021-11-21T21:54:51.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.418+0000 7f0b9cff9700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b840534f0 0x7f0b840559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.418+0000 7f0b9cff9700 1 -- 172.21.2.116:0/98005316 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0b9006b190 con 0x7f0ba005c060 2021-11-21T21:54:51.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.418+0000 7f0b9effd700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b840534f0 0x7f0b840559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.418+0000 7f0b9effd700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b840534f0 0x7f0b840559b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0b8800a000 tx=0x7f0b88008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.418+0000 7f0b9cff9700 1 -- 172.21.2.116:0/98005316 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b900421e0 con 0x7f0ba005c060 2021-11-21T21:54:51.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.502+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba00573d0 con 0x7f0ba005c060 2021-11-21T21:54:51.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0b9cff9700 1 -- 172.21.2.116:0/98005316 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16064 (secure 0 0 0) 0x7f0b90045800 con 0x7f0ba005c060 2021-11-21T21:54:51.509 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:51.509 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":31,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:51.352706+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:49.278376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:50.289265+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:51.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b840534f0 msgr2=0x7f0b840559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b840534f0 0x7f0b840559b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0b8800a000 tx=0x7f0b88008040).stop 2021-11-21T21:54:51.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba005c060 msgr2=0x7f0ba013dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba005c060 0x7f0ba013dd70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f0b90001110 tx=0x7f0b90007ad0).stop 2021-11-21T21:54:51.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 shutdown_connections 2021-11-21T21:54:51.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ba00a9500 0x7f0ba0144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ba0060c50 0x7f0ba01445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b840534f0 0x7f0b840559b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 --2- 172.21.2.116:0/98005316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ba005c060 0x7f0ba013dd70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 >> 172.21.2.116:0/98005316 conn(0x7f0ba00b6000 msgr2=0x7f0ba013fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 shutdown_connections 2021-11-21T21:54:51.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.506+0000 7f0ba5b2e700 1 -- 172.21.2.116:0/98005316 wait complete. 2021-11-21T21:54:51.517 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:51.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.562+0000 7f3726a36700 1 Processor -- start 2021-11-21T21:54:51.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.562+0000 7f3726a36700 1 -- start start 2021-11-21T21:54:51.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f372005c840 0x7f37200b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f372013e480 0x7f37200b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3720144970 con 0x7f37200a9d00 2021-11-21T21:54:51.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37200b1200 con 0x7f372005c840 2021-11-21T21:54:51.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37200aa0e0 con 0x7f372013e480 2021-11-21T21:54:51.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f372005c840 0x7f37200b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f372005c840 0x7f37200b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45992/0 (socket says 172.21.2.116:45992) 2021-11-21T21:54:51.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 -- 172.21.2.116:0/4294097347 learned_addr learned my addr 172.21.2.116:0/4294097347 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:51.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 -- 172.21.2.116:0/4294097347 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f372013e480 msgr2=0x7f37200b1ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:51.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 --2- 172.21.2.116:0/4294097347 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f372013e480 0x7f37200b1ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 -- 172.21.2.116:0/4294097347 >> v1:172.21.2.117:6789/0 conn(0x7f37200a9d00 legacy=0x7f372013dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:51.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 -- 172.21.2.116:0/4294097347 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200abb80 con 0x7f372005c840 2021-11-21T21:54:51.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371f7fe700 1 --2- 172.21.2.116:0/4294097347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b17b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f3720061ab0 tx=0x7f3708006eb0).ready entity=mon.0 client_cookie=75d927afb0164a01 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371effd700 1 -- 172.21.2.116:0/4294097347 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370800d570 con 0x7f372005c840 2021-11-21T21:54:51.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371effd700 1 -- 172.21.2.116:0/4294097347 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3708000e90 con 0x7f372005c840 2021-11-21T21:54:51.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371effd700 1 -- 172.21.2.116:0/4294097347 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370800d8c0 con 0x7f372005c840 2021-11-21T21:54:51.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- 172.21.2.116:0/4294097347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 msgr2=0x7f37200b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- 172.21.2.116:0/4294097347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b17b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f3720061ab0 tx=0x7f3708006eb0).stop 2021-11-21T21:54:51.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- 172.21.2.116:0/4294097347 shutdown_connections 2021-11-21T21:54:51.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- 172.21.2.116:0/4294097347 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f372013e480 0x7f37200b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- 172.21.2.116:0/4294097347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b17b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- 172.21.2.116:0/4294097347 >> 172.21.2.116:0/4294097347 conn(0x7f372005bd20 msgr2=0x7f372005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- 172.21.2.116:0/4294097347 shutdown_connections 2021-11-21T21:54:51.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- 172.21.2.116:0/4294097347 wait complete. 2021-11-21T21:54:51.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 Processor -- start 2021-11-21T21:54:51.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- start start 2021-11-21T21:54:51.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f37200a9d00 0x7f3720063850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f372013e480 0x7f3720063d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720064230 con 0x7f372005c840 2021-11-21T21:54:51.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37201401a0 con 0x7f372013e480 2021-11-21T21:54:51.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3726a36700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3720141180 con 0x7f37200a9d00 2021-11-21T21:54:51.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:45994/0 (socket says 172.21.2.116:45994) 2021-11-21T21:54:51.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 -- 172.21.2.116:0/846862567 learned_addr learned my addr 172.21.2.116:0/846862567 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:51.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f3724fd3700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f372013e480 0x7f3720063d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 -- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f37200a9d00 msgr2=0x7f3720063850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f37200a9d00 0x7f3720063850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 -- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f372013e480 msgr2=0x7f3720063d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f372013e480 0x7f3720063d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.566+0000 7f371ffff700 1 -- 172.21.2.116:0/846862567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370800b1e0 con 0x7f372005c840 2021-11-21T21:54:51.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371ffff700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b2e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f3710002f90 tx=0x7f3710006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371d7fa700 1 -- 172.21.2.116:0/846862567 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3710013460 con 0x7f372005c840 2021-11-21T21:54:51.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371d7fa700 1 -- 172.21.2.116:0/846862567 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37100135c0 con 0x7f372005c840 2021-11-21T21:54:51.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3720140a10 con 0x7f372005c840 2021-11-21T21:54:51.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371d7fa700 1 -- 172.21.2.116:0/846862567 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3710013920 con 0x7f372005c840 2021-11-21T21:54:51.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3720140770 con 0x7f372005c840 2021-11-21T21:54:51.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37200b0f60 con 0x7f372005c840 2021-11-21T21:54:51.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371d7fa700 1 -- 172.21.2.116:0/846862567 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f371001a060 con 0x7f372005c840 2021-11-21T21:54:51.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371d7fa700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3704053540 0x7f3704055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371d7fa700 1 -- 172.21.2.116:0/846862567 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f371006b310 con 0x7f372005c840 2021-11-21T21:54:51.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371f7fe700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3704053540 0x7f3704055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.570+0000 7f371f7fe700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3704053540 0x7f3704055a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f3720061ab0 tx=0x7f3708015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.574+0000 7f371d7fa700 1 -- 172.21.2.116:0/846862567 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3710042470 con 0x7f372005c840 2021-11-21T21:54:51.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.654+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37200573d0 con 0x7f372005c840 2021-11-21T21:54:51.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.654+0000 7f371d7fa700 1 -- 172.21.2.116:0/846862567 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16064 (secure 0 0 0) 0x7f3710045a90 con 0x7f372005c840 2021-11-21T21:54:51.659 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:51.660 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":31,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:51.352706+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:49.278376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:50.289265+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:51.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3704053540 msgr2=0x7f3704055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3704053540 0x7f3704055a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f3720061ab0 tx=0x7f3708015000).stop 2021-11-21T21:54:51.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 msgr2=0x7f37200b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b2e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f3710002f90 tx=0x7f3710006df0).stop 2021-11-21T21:54:51.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 shutdown_connections 2021-11-21T21:54:51.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f372013e480 0x7f3720063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f37200a9d00 0x7f3720063850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3704053540 0x7f3704055a00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 --2- 172.21.2.116:0/846862567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f372005c840 0x7f37200b2e50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 >> 172.21.2.116:0/846862567 conn(0x7f372005bd20 msgr2=0x7f37200ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 shutdown_connections 2021-11-21T21:54:51.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.658+0000 7f3726a36700 1 -- 172.21.2.116:0/846862567 wait complete. 2021-11-21T21:54:51.667 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T21:54:51.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.714+0000 7fd79d812700 1 Processor -- start 2021-11-21T21:54:51.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.714+0000 7fd79d812700 1 -- start start 2021-11-21T21:54:51.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd7980addb0 0x7fd7980b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd798143140 0x7fd79813fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7980ae190 con 0x7fd798063470 2021-11-21T21:54:51.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7980b41b0 con 0x7fd798143140 2021-11-21T21:54:51.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7981460a0 con 0x7fd7980addb0 2021-11-21T21:54:51.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7967fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd798063470 0x7fd798142a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57958/0 (socket says 172.21.2.116:57958) 2021-11-21T21:54:51.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7967fc700 1 -- 172.21.2.116:0/1114347194 learned_addr learned my addr 172.21.2.116:0/1114347194 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:51.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- 172.21.2.116:0/1114347194 >> v2:172.21.2.116:3300/0 conn(0x7fd798143140 0x7fd79813fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 -- 172.21.2.116:0/1114347194 >> v2:172.21.2.117:3300/0 conn(0x7fd7980addb0 msgr2=0x7fd7980b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:51.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- 172.21.2.116:0/1114347194 >> v2:172.21.2.117:3300/0 conn(0x7fd7980addb0 0x7fd7980b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 -- 172.21.2.116:0/1114347194 >> v1:172.21.2.116:6789/0 conn(0x7fd798063470 legacy=0x7fd798142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 -- 172.21.2.116:0/1114347194 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798146340 con 0x7fd798143140 2021-11-21T21:54:51.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- 172.21.2.116:0/1114347194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79813fb90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fd7900032f0 tx=0x7fd7900091d0).ready entity=mon.0 client_cookie=94279c941895624b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd795ffb700 1 -- 172.21.2.116:0/1114347194 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd790009ea0 con 0x7fd798143140 2021-11-21T21:54:51.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd795ffb700 1 -- 172.21.2.116:0/1114347194 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79000b870 con 0x7fd798143140 2021-11-21T21:54:51.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd795ffb700 1 -- 172.21.2.116:0/1114347194 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79000b9e0 con 0x7fd798143140 2021-11-21T21:54:51.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- 172.21.2.116:0/1114347194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 msgr2=0x7fd79813fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- 172.21.2.116:0/1114347194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79813fb90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fd7900032f0 tx=0x7fd7900091d0).stop 2021-11-21T21:54:51.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- 172.21.2.116:0/1114347194 shutdown_connections 2021-11-21T21:54:51.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- 172.21.2.116:0/1114347194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79813fb90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- 172.21.2.116:0/1114347194 >> v2:172.21.2.117:3300/0 conn(0x7fd7980addb0 0x7fd7980b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- 172.21.2.116:0/1114347194 >> 172.21.2.116:0/1114347194 conn(0x7fd79805bd20 msgr2=0x7fd79805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:51.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- 172.21.2.116:0/1114347194 shutdown_connections 2021-11-21T21:54:51.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- 172.21.2.116:0/1114347194 wait complete. 2021-11-21T21:54:51.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 Processor -- start 2021-11-21T21:54:51.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- start start 2021-11-21T21:54:51.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd798063470 0x7fd798114850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7980addb0 0x7fd79814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79814f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79814fd50 con 0x7fd798143140 2021-11-21T21:54:51.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980b1340 con 0x7fd798063470 2021-11-21T21:54:51.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd798152c10 con 0x7fd7980addb0 2021-11-21T21:54:51.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79814f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46012/0 (socket says 172.21.2.116:46012) 2021-11-21T21:54:51.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 -- 172.21.2.116:0/3731704284 learned_addr learned my addr 172.21.2.116:0/3731704284 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:51.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd796ffd700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd798063470 0x7fd798114850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 -- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7980addb0 msgr2=0x7fd79814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7980addb0 0x7fd79814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:51.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 -- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd798063470 msgr2=0x7fd798114850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:51.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd798063470 0x7fd798114850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:51.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 -- 172.21.2.116:0/3731704284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd790009470 con 0x7fd798143140 2021-11-21T21:54:51.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd7977fe700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79814f860 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd7900032f0 tx=0x7fd790016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd78ffff700 1 -- 172.21.2.116:0/3731704284 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd790006ea0 con 0x7fd798143140 2021-11-21T21:54:51.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd78ffff700 1 -- 172.21.2.116:0/3731704284 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7900107a0 con 0x7fd798143140 2021-11-21T21:54:51.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd798152eb0 con 0x7fd798143140 2021-11-21T21:54:51.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd78ffff700 1 -- 172.21.2.116:0/3731704284 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd790009ea0 con 0x7fd798143140 2021-11-21T21:54:51.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.718+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7980b0200 con 0x7fd798143140 2021-11-21T21:54:51.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.722+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7980b1540 con 0x7fd798143140 2021-11-21T21:54:51.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.722+0000 7fd78ffff700 1 -- 172.21.2.116:0/3731704284 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd790027020 con 0x7fd798143140 2021-11-21T21:54:51.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.722+0000 7fd78ffff700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd77c0536a0 0x7fd77c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:51.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.722+0000 7fd78ffff700 1 -- 172.21.2.116:0/3731704284 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd79006cd10 con 0x7fd798143140 2021-11-21T21:54:51.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.722+0000 7fd796ffd700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd77c0536a0 0x7fd77c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:51.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.722+0000 7fd796ffd700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd77c0536a0 0x7fd77c055b60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fd780007000 tx=0x7fd780008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:51.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.722+0000 7fd78ffff700 1 -- 172.21.2.116:0/3731704284 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd790043ef0 con 0x7fd798143140 2021-11-21T21:54:51.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:51.834+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd7981465e0 con 0x7fd798143140 2021-11-21T21:54:52.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd78ffff700 1 -- 172.21.2.116:0/3731704284 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd7900150b0 con 0x7fd798143140 2021-11-21T21:54:52.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd77c0536a0 msgr2=0x7fd77c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd77c0536a0 0x7fd77c055b60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fd780007000 tx=0x7fd780008040).stop 2021-11-21T21:54:52.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 msgr2=0x7fd79814f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79814f860 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd7900032f0 tx=0x7fd790016000).stop 2021-11-21T21:54:52.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 shutdown_connections 2021-11-21T21:54:52.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd77c0536a0 0x7fd77c055b60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd798143140 0x7fd79814f860 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7980addb0 0x7fd79814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 --2- 172.21.2.116:0/3731704284 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd798063470 0x7fd798114850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 >> 172.21.2.116:0/3731704284 conn(0x7fd79805bd20 msgr2=0x7fd79813f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 shutdown_connections 2021-11-21T21:54:52.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.362+0000 7fd79d812700 1 -- 172.21.2.116:0/3731704284 wait complete. 2021-11-21T21:54:52.375 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:52.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 Processor -- start 2021-11-21T21:54:52.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- start start 2021-11-21T21:54:52.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f661c0a6aa0 0x7f661c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f661c062410 0x7f661c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c0a6e80 con 0x7f661c0a86b0 2021-11-21T21:54:52.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c0a8bc0 con 0x7f661c0a6aa0 2021-11-21T21:54:52.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c14a7b0 con 0x7f661c062410 2021-11-21T21:54:52.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f661c0a6aa0 0x7f661c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f661c0a6aa0 0x7f661c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46018/0 (socket says 172.21.2.116:46018) 2021-11-21T21:54:52.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 -- 172.21.2.116:0/1584607575 learned_addr learned my addr 172.21.2.116:0/1584607575 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:52.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 -- 172.21.2.116:0/1584607575 >> v2:172.21.2.117:3300/0 conn(0x7f661c062410 msgr2=0x7f661c1443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:52.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 --2- 172.21.2.116:0/1584607575 >> v2:172.21.2.117:3300/0 conn(0x7f661c062410 0x7f661c1443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 -- 172.21.2.116:0/1584607575 >> v1:172.21.2.117:6789/0 conn(0x7f661c0a86b0 legacy=0x7f661c1470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:52.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 -- 172.21.2.116:0/1584607575 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c14aa10 con 0x7f661c0a6aa0 2021-11-21T21:54:52.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6621d07700 1 --2- 172.21.2.116:0/1584607575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a6aa0 0x7f661c14b150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f661c061ab0 tx=0x7f6610006eb0).ready entity=mon.0 client_cookie=21c7539bc3676365 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6620d05700 1 -- 172.21.2.116:0/1584607575 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f661000d5c0 con 0x7f661c0a6aa0 2021-11-21T21:54:52.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6620d05700 1 -- 172.21.2.116:0/1584607575 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610000e90 con 0x7f661c0a6aa0 2021-11-21T21:54:52.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6620d05700 1 -- 172.21.2.116:0/1584607575 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f661000d910 con 0x7f661c0a6aa0 2021-11-21T21:54:52.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- 172.21.2.116:0/1584607575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a6aa0 msgr2=0x7f661c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 --2- 172.21.2.116:0/1584607575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a6aa0 0x7f661c14b150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f661c061ab0 tx=0x7f6610006eb0).stop 2021-11-21T21:54:52.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- 172.21.2.116:0/1584607575 shutdown_connections 2021-11-21T21:54:52.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 --2- 172.21.2.116:0/1584607575 >> v2:172.21.2.117:3300/0 conn(0x7f661c062410 0x7f661c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 --2- 172.21.2.116:0/1584607575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a6aa0 0x7f661c14b150 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- 172.21.2.116:0/1584607575 >> 172.21.2.116:0/1584607575 conn(0x7f661c05bd20 msgr2=0x7f661c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- 172.21.2.116:0/1584607575 shutdown_connections 2021-11-21T21:54:52.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- 172.21.2.116:0/1584607575 wait complete. 2021-11-21T21:54:52.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 Processor -- start 2021-11-21T21:54:52.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.430+0000 7f6623f6b700 1 -- start start 2021-11-21T21:54:52.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f661c062410 0x7f661c0ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f661c0a6aa0 0x7f661c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a86b0 0x7f661c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0638f0 con 0x7f661c0a86b0 2021-11-21T21:54:52.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c063b50 con 0x7f661c062410 2021-11-21T21:54:52.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c063df0 con 0x7f661c0a6aa0 2021-11-21T21:54:52.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a86b0 0x7f661c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a86b0 0x7f661c156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46026/0 (socket says 172.21.2.116:46026) 2021-11-21T21:54:52.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 -- 172.21.2.116:0/3183727553 learned_addr learned my addr 172.21.2.116:0/3183727553 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:52.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 -- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f661c0a6aa0 msgr2=0x7f661c15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f661c0a6aa0 0x7f661c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 -- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f661c062410 msgr2=0x7f661c0ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f661c062410 0x7f661c0ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 -- 172.21.2.116:0/3183727553 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661000b1e0 con 0x7f661c0a86b0 2021-11-21T21:54:52.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6622508700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a86b0 0x7f661c156300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f6614003330 tx=0x7f66140091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f660effd700 1 -- 172.21.2.116:0/3183727553 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6614001520 con 0x7f661c0a86b0 2021-11-21T21:54:52.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f660effd700 1 -- 172.21.2.116:0/3183727553 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6614001680 con 0x7f661c0a86b0 2021-11-21T21:54:52.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c148760 con 0x7f661c0a86b0 2021-11-21T21:54:52.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f660effd700 1 -- 172.21.2.116:0/3183727553 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66140019e0 con 0x7f661c0a86b0 2021-11-21T21:54:52.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c0675c0 con 0x7f661c0a86b0 2021-11-21T21:54:52.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661c1488f0 con 0x7f661c0a86b0 2021-11-21T21:54:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f660effd700 1 -- 172.21.2.116:0/3183727553 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6614018030 con 0x7f661c0a86b0 2021-11-21T21:54:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f660effd700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f660effd700 1 -- 172.21.2.116:0/3183727553 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f661406b3c0 con 0x7f661c0a86b0 2021-11-21T21:54:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.434+0000 7f6621d07700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.438+0000 7f6621d07700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f661c061ab0 tx=0x7f6610015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.438+0000 7f660effd700 1 -- 172.21.2.116:0/3183727553 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6614042550 con 0x7f661c0a86b0 2021-11-21T21:54:52.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.546+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f661c05db70 con 0x7f661c0a86b0 2021-11-21T21:54:52.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.546+0000 7f660effd700 1 -- 172.21.2.116:0/3183727553 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2480 (secure 0 0 0) 0x7f6614045b70 con 0x7f661c0a86b0 2021-11-21T21:54:52.552 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:52.553 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:54:52.356582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2051796411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2051796411},{"type":"v1","addr":"172.21.2.116:6835","nonce":2051796411}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:54:52.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 msgr2=0x7f65fc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f661c061ab0 tx=0x7f6610015000).stop 2021-11-21T21:54:52.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a86b0 msgr2=0x7f661c156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a86b0 0x7f661c156300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f6614003330 tx=0x7f66140091a0).stop 2021-11-21T21:54:52.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 shutdown_connections 2021-11-21T21:54:52.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f661c0a86b0 0x7f661c156300 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f661c0a6aa0 0x7f661c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 --2- 172.21.2.116:0/3183727553 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f661c062410 0x7f661c0ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 >> 172.21.2.116:0/3183727553 conn(0x7f661c05bd20 msgr2=0x7f661c1426b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 shutdown_connections 2021-11-21T21:54:52.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.550+0000 7f6623f6b700 1 -- 172.21.2.116:0/3183727553 wait complete. 2021-11-21T21:54:52.557 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 11 2021-11-21T21:54:52.559 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:52.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 Processor -- start 2021-11-21T21:54:52.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 -- start start 2021-11-21T21:54:52.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbf5c0a86b0 0x7fbf5c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbf5c062410 0x7fbf5c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c0a6e80 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c0a8bc0 con 0x7fbf5c0a86b0 2021-11-21T21:54:52.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c14a7b0 con 0x7fbf5c062410 2021-11-21T21:54:52.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5bfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbf5c0a6aa0 0x7fbf5c1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:57986/0 (socket says 172.21.2.116:57986) 2021-11-21T21:54:52.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5bfff700 1 -- 172.21.2.116:0/3969065284 learned_addr learned my addr 172.21.2.116:0/3969065284 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:52.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5b7fe700 1 --2- 172.21.2.116:0/3969065284 >> v2:172.21.2.116:3300/0 conn(0x7fbf5c0a86b0 0x7fbf5c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5b7fe700 1 -- 172.21.2.116:0/3969065284 >> v2:172.21.2.117:3300/0 conn(0x7fbf5c062410 msgr2=0x7fbf5c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:52.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5b7fe700 1 --2- 172.21.2.116:0/3969065284 >> v2:172.21.2.117:3300/0 conn(0x7fbf5c062410 0x7fbf5c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5b7fe700 1 -- 172.21.2.116:0/3969065284 >> v1:172.21.2.116:6789/0 conn(0x7fbf5c0a6aa0 legacy=0x7fbf5c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5b7fe700 1 -- 172.21.2.116:0/3969065284 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c14aa10 con 0x7fbf5c0a86b0 2021-11-21T21:54:52.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5b7fe700 1 --2- 172.21.2.116:0/3969065284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a86b0 0x7fbf5c14b150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54006df0).ready entity=mon.0 client_cookie=12b97dc5d4fb0218 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5affd700 1 -- 172.21.2.116:0/3969065284 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf5400d5c0 con 0x7fbf5c0a86b0 2021-11-21T21:54:52.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5affd700 1 -- 172.21.2.116:0/3969065284 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf54000e90 con 0x7fbf5c0a86b0 2021-11-21T21:54:52.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf5affd700 1 -- 172.21.2.116:0/3969065284 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf5400d910 con 0x7fbf5c0a86b0 2021-11-21T21:54:52.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 -- 172.21.2.116:0/3969065284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a86b0 msgr2=0x7fbf5c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 --2- 172.21.2.116:0/3969065284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a86b0 0x7fbf5c14b150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54006df0).stop 2021-11-21T21:54:52.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 -- 172.21.2.116:0/3969065284 shutdown_connections 2021-11-21T21:54:52.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 --2- 172.21.2.116:0/3969065284 >> v2:172.21.2.117:3300/0 conn(0x7fbf5c062410 0x7fbf5c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 --2- 172.21.2.116:0/3969065284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a86b0 0x7fbf5c14b150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.606+0000 7fbf62438700 1 -- 172.21.2.116:0/3969065284 >> 172.21.2.116:0/3969065284 conn(0x7fbf5c05bd20 msgr2=0x7fbf5c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- 172.21.2.116:0/3969065284 shutdown_connections 2021-11-21T21:54:52.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- 172.21.2.116:0/3969065284 wait complete. 2021-11-21T21:54:52.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 Processor -- start 2021-11-21T21:54:52.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- start start 2021-11-21T21:54:52.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf5c062410 0x7fbf5c064cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a6aa0 0x7fbf5c065500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf5c0a86b0 0x7fbf5c063850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c064910 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0651a0 con 0x7fbf5c0a86b0 2021-11-21T21:54:52.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c065f10 con 0x7fbf5c062410 2021-11-21T21:54:52.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a6aa0 0x7fbf5c065500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a6aa0 0x7fbf5c065500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46040/0 (socket says 172.21.2.116:46040) 2021-11-21T21:54:52.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 -- 172.21.2.116:0/3142037217 learned_addr learned my addr 172.21.2.116:0/3142037217 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:52.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 -- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf5c062410 msgr2=0x7fbf5c064cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf5c062410 0x7fbf5c064cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 -- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf5c0a86b0 msgr2=0x7fbf5c063850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf5c0a86b0 0x7fbf5c063850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 -- 172.21.2.116:0/3142037217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5400b180 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5b7fe700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a6aa0 0x7fbf5c065500 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf597fa700 1 -- 172.21.2.116:0/3142037217 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf5401e400 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf597fa700 1 -- 172.21.2.116:0/3142037217 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5401e560 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf597fa700 1 -- 172.21.2.116:0/3142037217 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf5401e8c0 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c0644a0 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c1573d0 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c063ee0 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf597fa700 1 -- 172.21.2.116:0/3142037217 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbf54025030 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf597fa700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf400536a0 0x7fbf40055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf597fa700 1 -- 172.21.2.116:0/3142037217 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbf5406c8a0 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.610+0000 7fbf5bfff700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf400536a0 0x7fbf40055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.614+0000 7fbf5bfff700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf400536a0 0x7fbf40055b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fbf5c061ab0 tx=0x7fbf44007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.614+0000 7fbf597fa700 1 -- 172.21.2.116:0/3142037217 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf54043a00 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf5c145d90 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf597fa700 1 -- 172.21.2.116:0/3142037217 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16051 (secure 0 0 0) 0x7fbf5c145d90 con 0x7fbf5c0a6aa0 2021-11-21T21:54:52.701 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:52.702 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":32,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:52.357755+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:49.278376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:50.289265+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:52.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf400536a0 msgr2=0x7fbf40055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf400536a0 0x7fbf40055b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fbf5c061ab0 tx=0x7fbf44007820).stop 2021-11-21T21:54:52.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a6aa0 msgr2=0x7fbf5c065500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a6aa0 0x7fbf5c065500 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54014000).stop 2021-11-21T21:54:52.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 shutdown_connections 2021-11-21T21:54:52.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf5c0a86b0 0x7fbf5c063850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf400536a0 0x7fbf40055b60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf5c0a6aa0 0x7fbf5c065500 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 --2- 172.21.2.116:0/3142037217 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf5c062410 0x7fbf5c064cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 >> 172.21.2.116:0/3142037217 conn(0x7fbf5c05bd20 msgr2=0x7fbf5c141590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 shutdown_connections 2021-11-21T21:54:52.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.698+0000 7fbf62438700 1 -- 172.21.2.116:0/3142037217 wait complete. 2021-11-21T21:54:52.708 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:54:52.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f7d37700 1 Processor -- start 2021-11-21T21:54:52.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f7d37700 1 -- start start 2021-11-21T21:54:52.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f7d37700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f00b11c0 0x7fb5f0147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f7d37700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb5f00a8670 0x7fb5f0147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f7d37700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f00b1940 con 0x7fb5f0147c10 2021-11-21T21:54:52.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f7d37700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f00b15a0 con 0x7fb5f00a8670 2021-11-21T21:54:52.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f7d37700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b1b70 con 0x7fb5f00b11c0 2021-11-21T21:54:52.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f62d4700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb5f0147c10 0x7fb5f014a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58000/0 (socket says 172.21.2.116:58000) 2021-11-21T21:54:52.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f62d4700 1 -- 172.21.2.116:0/4028613722 learned_addr learned my addr 172.21.2.116:0/4028613722 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:52.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.758+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960649296 0 0) 0x7fb5f00b1940 con 0x7fb5f0147c10 2021-11-21T21:54:52.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d8003260 con 0x7fb5f0147c10 2021-11-21T21:54:52.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/4028613722 >> v2:172.21.2.117:3300/0 conn(0x7fb5f00a8670 0x7fb5f0147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f5ad3700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f00b11c0 0x7fb5f0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1559257695 0 0) 0x7fb5d8003260 con 0x7fb5f0147c10 2021-11-21T21:54:52.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f00b11c0 msgr2=0x7fb5f0147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:52.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f00b11c0 0x7fb5f0147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 >> v2:172.21.2.117:3300/0 conn(0x7fb5f00a8670 msgr2=0x7fb5f0147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:52.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 --2- 172.21.2.116:0/4028613722 >> v2:172.21.2.117:3300/0 conn(0x7fb5f00a8670 0x7fb5f0147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00b1e10 con 0x7fb5f0147c10 2021-11-21T21:54:52.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb5e80019e0 con 0x7fb5f0147c10 2021-11-21T21:54:52.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 >> v1:172.21.2.116:6789/0 conn(0x7fb5f0147c10 legacy=0x7fb5f014a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5d8003750 0x7fb5d8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5d8004500 0x7fb5d8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f00b11c0 0x7fb5d8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b15a0 con 0x7fb5d8004500 2021-11-21T21:54:52.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b1b70 con 0x7fb5d8003750 2021-11-21T21:54:52.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d800b7a0 con 0x7fb5f00b11c0 2021-11-21T21:54:52.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5d8004500 0x7fb5d8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 -- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f00b11c0 msgr2=0x7fb5d8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:52.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f00b11c0 0x7fb5d8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 -- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5d8003750 msgr2=0x7fb5d8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:52.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5d8003750 0x7fb5d8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 -- 172.21.2.116:0/4028613722 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8003260 con 0x7fb5d8004500 2021-11-21T21:54:52.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5d8004500 0x7fb5d8004980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb5ec00a000 tx=0x7fb5ec008040).ready entity=mon.0 client_cookie=df114fc6d6d759d3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5ec013670 con 0x7fb5d8004500 2021-11-21T21:54:52.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec0137d0 con 0x7fb5d8004500 2021-11-21T21:54:52.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f4ad1700 1 -- 172.21.2.116:0/4028613722 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5ec013b30 con 0x7fb5d8004500 2021-11-21T21:54:52.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- 172.21.2.116:0/4028613722 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5d8004500 msgr2=0x7fb5d8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 --2- 172.21.2.116:0/4028613722 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5d8004500 0x7fb5d8004980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb5ec00a000 tx=0x7fb5ec008040).stop 2021-11-21T21:54:52.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f62d4700 1 -- 172.21.2.116:0/4028613722 reap_dead start 2021-11-21T21:54:52.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- 172.21.2.116:0/4028613722 shutdown_connections 2021-11-21T21:54:52.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- 172.21.2.116:0/4028613722 >> 172.21.2.116:0/4028613722 conn(0x7fb5f005bd20 msgr2=0x7fb5f005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- 172.21.2.116:0/4028613722 shutdown_connections 2021-11-21T21:54:52.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- 172.21.2.116:0/4028613722 wait complete. 2021-11-21T21:54:52.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 Processor -- start 2021-11-21T21:54:52.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- start start 2021-11-21T21:54:52.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5d8004500 0x7fb5f01405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5f0140ae0 0x7fb5f013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f013e260 0x7fb5f01566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01562f0 con 0x7fb5f0140ae0 2021-11-21T21:54:52.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f013e7d0 con 0x7fb5d8004500 2021-11-21T21:54:52.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0141d30 con 0x7fb5f013e260 2021-11-21T21:54:52.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5f0140ae0 0x7fb5f013dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5f0140ae0 0x7fb5f013dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46054/0 (socket says 172.21.2.116:46054) 2021-11-21T21:54:52.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 -- 172.21.2.116:0/2447331943 learned_addr learned my addr 172.21.2.116:0/2447331943 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:52.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 -- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f013e260 msgr2=0x7fb5f01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f013e260 0x7fb5f01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 -- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5d8004500 msgr2=0x7fb5f01405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5d8004500 0x7fb5f01405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 -- 172.21.2.116:0/2447331943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0141f60 con 0x7fb5f0140ae0 2021-11-21T21:54:52.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f52d2700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5f0140ae0 0x7fb5f013dd70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fb5f00b3ef0 tx=0x7fb5ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5e2ffd700 1 -- 172.21.2.116:0/2447331943 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5ec012d20 con 0x7fb5f0140ae0 2021-11-21T21:54:52.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5e2ffd700 1 -- 172.21.2.116:0/2447331943 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec000e90 con 0x7fb5f0140ae0 2021-11-21T21:54:52.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f0142200 con 0x7fb5f0140ae0 2021-11-21T21:54:52.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5e2ffd700 1 -- 172.21.2.116:0/2447331943 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5ec01d600 con 0x7fb5f0140ae0 2021-11-21T21:54:52.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.762+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f01425a0 con 0x7fb5f0140ae0 2021-11-21T21:54:52.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.766+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f0142390 con 0x7fb5f0140ae0 2021-11-21T21:54:52.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.766+0000 7fb5e2ffd700 1 -- 172.21.2.116:0/2447331943 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb5ec01d760 con 0x7fb5f0140ae0 2021-11-21T21:54:52.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.766+0000 7fb5e2ffd700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5d0053640 0x7fb5d0055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.766+0000 7fb5e2ffd700 1 -- 172.21.2.116:0/2447331943 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb5ec06d250 con 0x7fb5f0140ae0 2021-11-21T21:54:52.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.766+0000 7fb5f5ad3700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5d0053640 0x7fb5d0055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.766+0000 7fb5f5ad3700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5d0053640 0x7fb5d0055b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fb5e400a000 tx=0x7fb5e4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.766+0000 7fb5e2ffd700 1 -- 172.21.2.116:0/2447331943 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5ec044240 con 0x7fb5f0140ae0 2021-11-21T21:54:52.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.878+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb5f00b20b0 con 0x7fb5f0140ae0 2021-11-21T21:54:52.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.882+0000 7fb5e2ffd700 1 -- 172.21.2.116:0/2447331943 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb5ec00b120 con 0x7fb5f0140ae0 2021-11-21T21:54:52.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5d0053640 msgr2=0x7fb5d0055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5d0053640 0x7fb5d0055b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fb5e400a000 tx=0x7fb5e4008040).stop 2021-11-21T21:54:52.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5f0140ae0 msgr2=0x7fb5f013dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5f0140ae0 0x7fb5f013dd70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fb5f00b3ef0 tx=0x7fb5ec008040).stop 2021-11-21T21:54:52.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 shutdown_connections 2021-11-21T21:54:52.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb5f013e260 0x7fb5f01566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5d0053640 0x7fb5d0055b00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb5f0140ae0 0x7fb5f013dd70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 --2- 172.21.2.116:0/2447331943 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb5d8004500 0x7fb5f01405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 >> 172.21.2.116:0/2447331943 conn(0x7fb5f005bd20 msgr2=0x7fb5f0145c90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 shutdown_connections 2021-11-21T21:54:52.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.886+0000 7fb5f7d37700 1 -- 172.21.2.116:0/2447331943 wait complete. 2021-11-21T21:54:52.892 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T21:54:52.895 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:54:52.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabac25a700 1 Processor -- start 2021-11-21T21:54:52.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabac25a700 1 -- start start 2021-11-21T21:54:52.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabac25a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faba40ad780 0x7faba405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabac25a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba4143f70 con 0x7faba40af7e0 2021-11-21T21:54:52.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabac25a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba40adb60 con 0x7faba40affc0 2021-11-21T21:54:52.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabac25a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba4143160 con 0x7faba40ad780 2021-11-21T21:54:52.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabaa7f7700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7faba40af7e0 0x7faba413fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58018/0 (socket says 172.21.2.116:58018) 2021-11-21T21:54:52.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7fabaa7f7700 1 -- 172.21.2.116:0/4011622286 learned_addr learned my addr 172.21.2.116:0/4011622286 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:52.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba9ff6700 1 --2- 172.21.2.116:0/4011622286 >> v2:172.21.2.116:3300/0 conn(0x7faba40ad780 0x7faba405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba9ff6700 1 -- 172.21.2.116:0/4011622286 >> v1:172.21.2.117:6789/0 conn(0x7faba40affc0 legacy=0x7faba4143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:52.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba9ff6700 1 -- 172.21.2.116:0/4011622286 >> v1:172.21.2.116:6789/0 conn(0x7faba40af7e0 legacy=0x7faba413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba9ff6700 1 -- 172.21.2.116:0/4011622286 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba40b0ce0 con 0x7faba40ad780 2021-11-21T21:54:52.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba9ff6700 1 --2- 172.21.2.116:0/4011622286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40ad780 0x7faba405f730 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7faba40685b0 tx=0x7faba0006e80).ready entity=mon.0 client_cookie=b0c2d5d14bbec590 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba8ff4700 1 -- 172.21.2.116:0/4011622286 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba000ba30 con 0x7faba40ad780 2021-11-21T21:54:52.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba8ff4700 1 -- 172.21.2.116:0/4011622286 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba0000e90 con 0x7faba40ad780 2021-11-21T21:54:52.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.946+0000 7faba8ff4700 1 -- 172.21.2.116:0/4011622286 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba000bd80 con 0x7faba40ad780 2021-11-21T21:54:52.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/4011622286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40ad780 msgr2=0x7faba405f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 --2- 172.21.2.116:0/4011622286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40ad780 0x7faba405f730 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7faba40685b0 tx=0x7faba0006e80).stop 2021-11-21T21:54:52.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/4011622286 shutdown_connections 2021-11-21T21:54:52.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 --2- 172.21.2.116:0/4011622286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40ad780 0x7faba405f730 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:52.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/4011622286 >> 172.21.2.116:0/4011622286 conn(0x7faba40b6000 msgr2=0x7faba40b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:52.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/4011622286 shutdown_connections 2021-11-21T21:54:52.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/4011622286 wait complete. 2021-11-21T21:54:52.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 Processor -- start 2021-11-21T21:54:52.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- start start 2021-11-21T21:54:52.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba40ad780 0x7faba414e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba40af7e0 0x7faba40abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40affc0 0x7faba414b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba414edd0 con 0x7faba40affc0 2021-11-21T21:54:52.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba409f6f0 con 0x7faba40af7e0 2021-11-21T21:54:52.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba409f990 con 0x7faba40ad780 2021-11-21T21:54:52.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40affc0 0x7faba414b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40affc0 0x7faba414b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46072/0 (socket says 172.21.2.116:46072) 2021-11-21T21:54:52.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 -- 172.21.2.116:0/1917623780 learned_addr learned my addr 172.21.2.116:0/1917623780 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:52.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 -- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba40ad780 msgr2=0x7faba414e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba40ad780 0x7faba414e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 -- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba40af7e0 msgr2=0x7faba40abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:52.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba40af7e0 0x7faba40abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:52.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 -- 172.21.2.116:0/1917623780 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba000d410 con 0x7faba40affc0 2021-11-21T21:54:52.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabaa7f7700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40affc0 0x7faba414b1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7faba4068820 tx=0x7fab9c007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fab92ffd700 1 -- 172.21.2.116:0/1917623780 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c005ba0 con 0x7faba40affc0 2021-11-21T21:54:52.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fab92ffd700 1 -- 172.21.2.116:0/1917623780 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c005d00 con 0x7faba40affc0 2021-11-21T21:54:52.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba409fc30 con 0x7faba40affc0 2021-11-21T21:54:52.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba40ac410 con 0x7faba40affc0 2021-11-21T21:54:52.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fab92ffd700 1 -- 172.21.2.116:0/1917623780 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c0072c0 con 0x7faba40affc0 2021-11-21T21:54:52.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba40b0a40 con 0x7faba40affc0 2021-11-21T21:54:52.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.950+0000 7fab92ffd700 1 -- 172.21.2.116:0/1917623780 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fab9c01a060 con 0x7faba40affc0 2021-11-21T21:54:52.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.954+0000 7fab92ffd700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab84053640 0x7fab84055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:52.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.954+0000 7fab92ffd700 1 -- 172.21.2.116:0/1917623780 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fab9c06b4d0 con 0x7faba40affc0 2021-11-21T21:54:52.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.954+0000 7faba9ff6700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab84053640 0x7fab84055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:52.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.954+0000 7faba9ff6700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab84053640 0x7fab84055b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7faba40685b0 tx=0x7faba0006e80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:52.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:52.954+0000 7fab92ffd700 1 -- 172.21.2.116:0/1917623780 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab9c042520 con 0x7faba40affc0 2021-11-21T21:54:53.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.066+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faba40573d0 con 0x7faba40affc0 2021-11-21T21:54:53.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fab92ffd700 1 -- 172.21.2.116:0/1917623780 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7fab9c018020 con 0x7faba40affc0 2021-11-21T21:54:53.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab84053640 msgr2=0x7fab84055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab84053640 0x7fab84055b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7faba40685b0 tx=0x7faba0006e80).stop 2021-11-21T21:54:53.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40affc0 msgr2=0x7faba414b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40affc0 0x7faba414b1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7faba4068820 tx=0x7fab9c007f40).stop 2021-11-21T21:54:53.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 shutdown_connections 2021-11-21T21:54:53.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab84053640 0x7fab84055b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faba40affc0 0x7faba414b1c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba40af7e0 0x7faba40abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 --2- 172.21.2.116:0/1917623780 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba40ad780 0x7faba414e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 >> 172.21.2.116:0/1917623780 conn(0x7faba40b6000 msgr2=0x7faba413f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.070+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 shutdown_connections 2021-11-21T21:54:53.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.074+0000 7fabac25a700 1 -- 172.21.2.116:0/1917623780 wait complete. 2021-11-21T21:54:53.079 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T21:54:53.082 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:53.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 Processor -- start 2021-11-21T21:54:53.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- start start 2021-11-21T21:54:53.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 0x7fce2c142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fce2c0b5730 0x7fce2c09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c143330 con 0x7fce2c0b67f0 2021-11-21T21:54:53.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c143e60 con 0x7fce2c0b5730 2021-11-21T21:54:53.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0b6bd0 con 0x7fce2c0b3ca0 2021-11-21T21:54:53.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fce2c0b5730 0x7fce2c09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce31b70700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fce2c0b67f0 0x7fce2c0b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58032/0 (socket says 172.21.2.116:58032) 2021-11-21T21:54:53.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce31b70700 1 -- 172.21.2.116:0/2183826446 learned_addr learned my addr 172.21.2.116:0/2183826446 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:53.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 -- 172.21.2.116:0/2183826446 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 msgr2=0x7fce2c142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 --2- 172.21.2.116:0/2183826446 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 0x7fce2c142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 -- 172.21.2.116:0/2183826446 >> v1:172.21.2.116:6789/0 conn(0x7fce2c0b67f0 legacy=0x7fce2c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 -- 172.21.2.116:0/2183826446 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0ac990 con 0x7fce2c0b5730 2021-11-21T21:54:53.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 --2- 172.21.2.116:0/2183826446 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b5730 0x7fce2c09b5a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fce240032f0 tx=0x7fce240091d0).ready entity=mon.0 client_cookie=ebef06f383aa6108 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce30b6e700 1 -- 172.21.2.116:0/2183826446 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce24009ea0 con 0x7fce2c0b5730 2021-11-21T21:54:53.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce30b6e700 1 -- 172.21.2.116:0/2183826446 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400b8a0 con 0x7fce2c0b5730 2021-11-21T21:54:53.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce30b6e700 1 -- 172.21.2.116:0/2183826446 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce2400ba10 con 0x7fce2c0b5730 2021-11-21T21:54:53.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- 172.21.2.116:0/2183826446 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b5730 msgr2=0x7fce2c09b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- 172.21.2.116:0/2183826446 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b5730 0x7fce2c09b5a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fce240032f0 tx=0x7fce240091d0).stop 2021-11-21T21:54:53.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- 172.21.2.116:0/2183826446 shutdown_connections 2021-11-21T21:54:53.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- 172.21.2.116:0/2183826446 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b5730 0x7fce2c09b5a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- 172.21.2.116:0/2183826446 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 0x7fce2c142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- 172.21.2.116:0/2183826446 >> 172.21.2.116:0/2183826446 conn(0x7fce2c0b9d80 msgr2=0x7fce2c0b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- 172.21.2.116:0/2183826446 shutdown_connections 2021-11-21T21:54:53.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- 172.21.2.116:0/2183826446 wait complete. 2021-11-21T21:54:53.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 Processor -- start 2021-11-21T21:54:53.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- start start 2021-11-21T21:54:53.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 0x7fce2c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce2c0b5730 0x7fce2c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b67f0 0x7fce2c09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c152be0 con 0x7fce2c0b67f0 2021-11-21T21:54:53.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0b1ef0 con 0x7fce2c0b5730 2021-11-21T21:54:53.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce33dd4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c152e70 con 0x7fce2c0b3ca0 2021-11-21T21:54:53.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b67f0 0x7fce2c09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b67f0 0x7fce2c09b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46084/0 (socket says 172.21.2.116:46084) 2021-11-21T21:54:53.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.130+0000 7fce32371700 1 -- 172.21.2.116:0/1700401420 learned_addr learned my addr 172.21.2.116:0/1700401420 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:53.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce32371700 1 -- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 msgr2=0x7fce2c0ad3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:53.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce32371700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 0x7fce2c0ad3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce32371700 1 -- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce2c0b5730 msgr2=0x7fce2c0ad890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:53.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce32371700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce2c0b5730 0x7fce2c0ad890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce32371700 1 -- 172.21.2.116:0/1700401420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24009470 con 0x7fce2c0b67f0 2021-11-21T21:54:53.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce32371700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b67f0 0x7fce2c09b5a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fce240032f0 tx=0x7fce24016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce1affd700 1 -- 172.21.2.116:0/1700401420 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce24021630 con 0x7fce2c0b67f0 2021-11-21T21:54:53.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce1affd700 1 -- 172.21.2.116:0/1700401420 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24021790 con 0x7fce2c0b67f0 2021-11-21T21:54:53.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c153110 con 0x7fce2c0b67f0 2021-11-21T21:54:53.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce1affd700 1 -- 172.21.2.116:0/1700401420 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce24021af0 con 0x7fce2c0b67f0 2021-11-21T21:54:53.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c13c080 con 0x7fce2c0b67f0 2021-11-21T21:54:53.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c0acc30 con 0x7fce2c0b67f0 2021-11-21T21:54:53.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce1affd700 1 -- 172.21.2.116:0/1700401420 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fce24027020 con 0x7fce2c0b67f0 2021-11-21T21:54:53.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce1affd700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce0c053720 0x7fce0c055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce1affd700 1 -- 172.21.2.116:0/1700401420 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fce2406d2a0 con 0x7fce2c0b67f0 2021-11-21T21:54:53.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce31b70700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce0c053720 0x7fce0c055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.134+0000 7fce31b70700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce0c053720 0x7fce0c055be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fce2c062710 tx=0x7fce1c007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.138+0000 7fce1affd700 1 -- 172.21.2.116:0/1700401420 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce24044480 con 0x7fce2c0b67f0 2021-11-21T21:54:53.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.242+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce2c0573d0 con 0x7fce2c0b67f0 2021-11-21T21:54:53.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce1affd700 1 -- 172.21.2.116:0/1700401420 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2480 (secure 0 0 0) 0x7fce24015020 con 0x7fce2c0b67f0 2021-11-21T21:54:53.249 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:53.249 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:54:52.356582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2051796411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2051796411},{"type":"v1","addr":"172.21.2.116:6835","nonce":2051796411}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:54:53.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce0c053720 msgr2=0x7fce0c055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce0c053720 0x7fce0c055be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fce2c062710 tx=0x7fce1c007790).stop 2021-11-21T21:54:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b67f0 msgr2=0x7fce2c09b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b67f0 0x7fce2c09b5a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fce240032f0 tx=0x7fce24016000).stop 2021-11-21T21:54:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 shutdown_connections 2021-11-21T21:54:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fce0c053720 0x7fce0c055be0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fce2c0b67f0 0x7fce2c09b5a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce2c0b5730 0x7fce2c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 --2- 172.21.2.116:0/1700401420 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce2c0b3ca0 0x7fce2c0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 >> 172.21.2.116:0/1700401420 conn(0x7fce2c0b9d80 msgr2=0x7fce2c09d370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 shutdown_connections 2021-11-21T21:54:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.246+0000 7fce33dd4700 1 -- 172.21.2.116:0/1700401420 wait complete. 2021-11-21T21:54:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 11 2021-11-21T21:54:53.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:54:51.352804+0000', 'modified': '2021-11-21T21:54:52.356582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4878}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4878': {'gid': 4878, 'name': 'c', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2051796411', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2051796411}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2051796411}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:54:53.564 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:54:53.565 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:54:53.627 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":4878,"want_state":"up:active","state":"up:active","rank_uptime":2.267890816,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":31,"uptime":7.4868938729999996} 2021-11-21T21:54:53.628 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 4878, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 31, "rank_uptime": 2.267890816, "state": "up:active", "uptime": 7.486893873, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:54:53.629 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:54:53.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507e9f0700 1 Processor -- start 2021-11-21T21:54:53.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507e9f0700 1 -- start start 2021-11-21T21:54:53.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507e9f0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f50780a6aa0 0x7f507814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507e9f0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f50780a86b0 0x7f50781470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507e9f0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50780a6e80 con 0x7f50781475d0 2021-11-21T21:54:53.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507e9f0700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50780a8bc0 con 0x7f50780a86b0 2021-11-21T21:54:53.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507e9f0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507814a7b0 con 0x7f50780a6aa0 2021-11-21T21:54:53.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f50777fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f50780a86b0 0x7f50781470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507cf8d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f50781475d0 0x7f50781443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58048/0 (socket says 172.21.2.116:58048) 2021-11-21T21:54:53.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.674+0000 7f507cf8d700 1 -- 172.21.2.116:0/1634108844 learned_addr learned my addr 172.21.2.116:0/1634108844 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:53.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 -- 172.21.2.116:0/1634108844 >> v2:172.21.2.117:3300/0 conn(0x7f50780a6aa0 msgr2=0x7f507814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 --2- 172.21.2.116:0/1634108844 >> v2:172.21.2.117:3300/0 conn(0x7f50780a6aa0 0x7f507814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 -- 172.21.2.116:0/1634108844 >> v1:172.21.2.116:6789/0 conn(0x7f50781475d0 legacy=0x7f50781443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 -- 172.21.2.116:0/1634108844 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507814aa10 con 0x7f50780a86b0 2021-11-21T21:54:53.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 --2- 172.21.2.116:0/1634108844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f50781470e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f5070001490 tx=0x7f5060006df0).ready entity=mon.0 client_cookie=cfb4bbbc0652dde3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f5076ffd700 1 -- 172.21.2.116:0/1634108844 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506000d570 con 0x7f50780a86b0 2021-11-21T21:54:53.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f5076ffd700 1 -- 172.21.2.116:0/1634108844 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5060000e90 con 0x7f50780a86b0 2021-11-21T21:54:53.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f5076ffd700 1 -- 172.21.2.116:0/1634108844 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506000d8c0 con 0x7f50780a86b0 2021-11-21T21:54:53.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1634108844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 msgr2=0x7f50781470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1634108844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f50781470e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f5070001490 tx=0x7f5060006df0).stop 2021-11-21T21:54:53.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1634108844 shutdown_connections 2021-11-21T21:54:53.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1634108844 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f50781470e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1634108844 >> v2:172.21.2.117:3300/0 conn(0x7f50780a6aa0 0x7f507814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1634108844 >> 172.21.2.116:0/1634108844 conn(0x7f507805bd20 msgr2=0x7f507805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1634108844 shutdown_connections 2021-11-21T21:54:53.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1634108844 wait complete. 2021-11-21T21:54:53.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 Processor -- start 2021-11-21T21:54:53.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- start start 2021-11-21T21:54:53.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50780a6aa0 0x7f5078141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f507813e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50781475d0 0x7f507813ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078064770 con 0x7f50780a86b0 2021-11-21T21:54:53.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078063850 con 0x7f50780a6aa0 2021-11-21T21:54:53.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5078067650 con 0x7f50781475d0 2021-11-21T21:54:53.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f507813e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f507813e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46098/0 (socket says 172.21.2.116:46098) 2021-11-21T21:54:53.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 -- 172.21.2.116:0/1966893501 learned_addr learned my addr 172.21.2.116:0/1966893501 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:53.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 -- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50781475d0 msgr2=0x7f507813ebb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:53.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50781475d0 0x7f507813ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 -- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50780a6aa0 msgr2=0x7f5078141cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:53.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50780a6aa0 0x7f5078141cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 -- 172.21.2.116:0/1966893501 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506000b180 con 0x7f50780a86b0 2021-11-21T21:54:53.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50777fe700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f507813e6c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f5070001490 tx=0x7f5060014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50757fa700 1 -- 172.21.2.116:0/1966893501 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5060016070 con 0x7f50780a86b0 2021-11-21T21:54:53.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50757fa700 1 -- 172.21.2.116:0/1966893501 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5060002f50 con 0x7f50780a86b0 2021-11-21T21:54:53.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5078146da0 con 0x7f50780a86b0 2021-11-21T21:54:53.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50757fa700 1 -- 172.21.2.116:0/1966893501 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5060006d60 con 0x7f50780a86b0 2021-11-21T21:54:53.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50781421a0 con 0x7f50780a86b0 2021-11-21T21:54:53.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50780573d0 con 0x7f50780a86b0 2021-11-21T21:54:53.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50757fa700 1 -- 172.21.2.116:0/1966893501 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5060025030 con 0x7f50780a86b0 2021-11-21T21:54:53.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50757fa700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f505c053640 0x7f505c055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.678+0000 7f50757fa700 1 -- 172.21.2.116:0/1966893501 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f506006c7d0 con 0x7f50780a86b0 2021-11-21T21:54:53.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.682+0000 7f5077fff700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f505c053640 0x7f505c055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.682+0000 7f5077fff700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f505c053640 0x7f505c055b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5068002f70 tx=0x7f5068006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.682+0000 7f50757fa700 1 -- 172.21.2.116:0/1966893501 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f50600439f0 con 0x7f50780a86b0 2021-11-21T21:54:53.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.762+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f507805db70 con 0x7f50780a86b0 2021-11-21T21:54:53.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.762+0000 7f50757fa700 1 -- 172.21.2.116:0/1966893501 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f5060023020 con 0x7f50780a86b0 2021-11-21T21:54:53.768 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:53.768 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:54:53.370027+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:54:49.278376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-11-21T21:54:50.289265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:54:53.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f505c053640 msgr2=0x7f505c055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f505c053640 0x7f505c055b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5068002f70 tx=0x7f5068006a00).stop 2021-11-21T21:54:53.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 msgr2=0x7f507813e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f507813e6c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f5070001490 tx=0x7f5060014000).stop 2021-11-21T21:54:53.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 shutdown_connections 2021-11-21T21:54:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f50781475d0 0x7f507813ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f505c053640 0x7f505c055b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f50780a86b0 0x7f507813e6c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 --2- 172.21.2.116:0/1966893501 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f50780a6aa0 0x7f5078141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 >> 172.21.2.116:0/1966893501 conn(0x7f507805bd20 msgr2=0x7f5078142f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 shutdown_connections 2021-11-21T21:54:53.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.766+0000 7f507e9f0700 1 -- 172.21.2.116:0/1966893501 wait complete. 2021-11-21T21:54:53.775 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:54:49.278376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:54:50.289265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:54:53.776 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:54:53.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 Processor -- start 2021-11-21T21:54:53.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- start start 2021-11-21T21:54:53.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa4dc144810 0x7fa4dc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4dc147e80 con 0x7fa4dc0ac600 2021-11-21T21:54:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4dc0af5a0 con 0x7fa4dc0abce0 2021-11-21T21:54:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4dc0b1490 con 0x7fa4dc144810 2021-11-21T21:54:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e0a59700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa4dc0ac600 0x7fa4dc0aca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58060/0 (socket says 172.21.2.116:58060) 2021-11-21T21:54:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e0a59700 1 -- 172.21.2.116:0/120717897 learned_addr learned my addr 172.21.2.116:0/120717897 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:53.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814673810 0 0) 0x7fa4dc147e80 con 0x7fa4dc0ac600 2021-11-21T21:54:53.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4bc003260 con 0x7fa4dc0ac600 2021-11-21T21:54:53.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199213072 0 0) 0x7fa4bc003260 con 0x7fa4dc0ac600 2021-11-21T21:54:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 >> v2:172.21.2.117:3300/0 conn(0x7fa4dc144810 msgr2=0x7fa4dc1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 --2- 172.21.2.116:0/120717897 >> v2:172.21.2.117:3300/0 conn(0x7fa4dc144810 0x7fa4dc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 >> v1:172.21.2.117:6789/0 conn(0x7fa4dc0abce0 legacy=0x7fa4dc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc0b1fd0 con 0x7fa4dc0ac600 2021-11-21T21:54:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fa4d40019e0 con 0x7fa4dc0ac600 2021-11-21T21:54:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 >> v1:172.21.2.116:6789/0 conn(0x7fa4dc0ac600 legacy=0x7fa4dc0aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4bc003820 0x7fa4bc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4bc004500 0x7fa4bc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4bc005050 0x7fa4bc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0b1490 con 0x7fa4bc003820 2021-11-21T21:54:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc00bf40 con 0x7fa4bc004500 2021-11-21T21:54:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc00c100 con 0x7fa4bc005050 2021-11-21T21:54:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4dbfff700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4bc003820 0x7fa4bc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4dbfff700 1 -- 172.21.2.116:0/120717897 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4bc005050 msgr2=0x7fa4bc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4dbfff700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4bc005050 0x7fa4bc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4dbfff700 1 -- 172.21.2.116:0/120717897 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4bc004500 msgr2=0x7fa4bc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4dbfff700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4bc004500 0x7fa4bc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4dbfff700 1 -- 172.21.2.116:0/120717897 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc003260 con 0x7fa4bc003820 2021-11-21T21:54:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e0a59700 1 -- 172.21.2.116:0/120717897 reap_dead start 2021-11-21T21:54:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4dbfff700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4bc003820 0x7fa4bc003c20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa4cc002f70 tx=0x7fa4cc006ca0).ready entity=mon.0 client_cookie=26c345c4212621b7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4cc013410 con 0x7fa4bc003820 2021-11-21T21:54:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc013570 con 0x7fa4bc003820 2021-11-21T21:54:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4daffd700 1 -- 172.21.2.116:0/120717897 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4cc0138d0 con 0x7fa4bc003820 2021-11-21T21:54:53.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- 172.21.2.116:0/120717897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4bc003820 msgr2=0x7fa4bc003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4bc003820 0x7fa4bc003c20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa4cc002f70 tx=0x7fa4cc006ca0).stop 2021-11-21T21:54:53.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- 172.21.2.116:0/120717897 shutdown_connections 2021-11-21T21:54:53.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/120717897 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4bc003820 0x7fa4bc003c20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- 172.21.2.116:0/120717897 >> 172.21.2.116:0/120717897 conn(0x7fa4dc05bd20 msgr2=0x7fa4dc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- 172.21.2.116:0/120717897 shutdown_connections 2021-11-21T21:54:53.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.826+0000 7fa4e24bc700 1 -- 172.21.2.116:0/120717897 wait complete. 2021-11-21T21:54:53.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 Processor -- start 2021-11-21T21:54:53.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 -- start start 2021-11-21T21:54:53.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4bc003820 0x7fa4dc13c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4dc156630 0x7fa4dc0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4dc0b3ce0 0x7fa4dc159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0b2f70 con 0x7fa4dc156630 2021-11-21T21:54:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0b43d0 con 0x7fa4bc003820 2021-11-21T21:54:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0b31a0 con 0x7fa4dc0b3ce0 2021-11-21T21:54:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4dc156630 0x7fa4dc0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4dc156630 0x7fa4dc0b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46116/0 (socket says 172.21.2.116:46116) 2021-11-21T21:54:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 -- 172.21.2.116:0/1983759289 learned_addr learned my addr 172.21.2.116:0/1983759289 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:53.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 -- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4dc0b3ce0 msgr2=0x7fa4dc159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4dc0b3ce0 0x7fa4dc159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 -- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4bc003820 msgr2=0x7fa4dc13c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4bc003820 0x7fa4dc13c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 -- 172.21.2.116:0/1983759289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc1462d0 con 0x7fa4dc156630 2021-11-21T21:54:53.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4db7fe700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4dc156630 0x7fa4dc0b37f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fa4c4000f70 tx=0x7fa4c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4d97fa700 1 -- 172.21.2.116:0/1983759289 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c4009e00 con 0x7fa4dc156630 2021-11-21T21:54:53.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4d97fa700 1 -- 172.21.2.116:0/1983759289 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4013460 con 0x7fa4dc156630 2021-11-21T21:54:53.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4d97fa700 1 -- 172.21.2.116:0/1983759289 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c40137c0 con 0x7fa4dc156630 2021-11-21T21:54:53.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc0b2ab0 con 0x7fa4dc156630 2021-11-21T21:54:53.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc1432a0 con 0x7fa4dc156630 2021-11-21T21:54:53.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc0af060 con 0x7fa4dc156630 2021-11-21T21:54:53.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4d97fa700 1 -- 172.21.2.116:0/1983759289 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa4c400f070 con 0x7fa4dc156630 2021-11-21T21:54:53.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4d97fa700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4c00536d0 0x7fa4c0055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4d97fa700 1 -- 172.21.2.116:0/1983759289 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa4c406b070 con 0x7fa4dc156630 2021-11-21T21:54:53.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.830+0000 7fa4dbfff700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4c00536d0 0x7fa4c0055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.834+0000 7fa4dbfff700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4c00536d0 0x7fa4c0055b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa4cc002f70 tx=0x7fa4cc014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.834+0000 7fa4d97fa700 1 -- 172.21.2.116:0/1983759289 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa4c40421d0 con 0x7fa4dc156630 2021-11-21T21:54:53.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.914+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa4dc0573d0 con 0x7fa4dc156630 2021-11-21T21:54:53.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4d97fa700 1 -- 172.21.2.116:0/1983759289 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v33) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa4c40457f0 con 0x7fa4dc156630 2021-11-21T21:54:53.921 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:54:53.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4c00536d0 msgr2=0x7fa4c0055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4c00536d0 0x7fa4c0055b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa4cc002f70 tx=0x7fa4cc014000).stop 2021-11-21T21:54:53.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4dc156630 msgr2=0x7fa4dc0b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4dc156630 0x7fa4dc0b37f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fa4c4000f70 tx=0x7fa4c4006df0).stop 2021-11-21T21:54:53.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 shutdown_connections 2021-11-21T21:54:53.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4dc0b3ce0 0x7fa4dc159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4c00536d0 0x7fa4c0055b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4dc156630 0x7fa4dc0b37f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 --2- 172.21.2.116:0/1983759289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4bc003820 0x7fa4dc13c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 >> 172.21.2.116:0/1983759289 conn(0x7fa4dc05bd20 msgr2=0x7fa4dc1417f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 shutdown_connections 2021-11-21T21:54:53.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.918+0000 7fa4e24bc700 1 -- 172.21.2.116:0/1983759289 wait complete. 2021-11-21T21:54:53.928 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:54:53.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 Processor -- start 2021-11-21T21:54:53.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- start start 2021-11-21T21:54:53.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f07e4144810 0x7f07e41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e4147e80 con 0x7f07e40abce0 2021-11-21T21:54:53.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40af5a0 con 0x7f07e40ac600 2021-11-21T21:54:53.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e40b1490 con 0x7f07e4144810 2021-11-21T21:54:53.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f07e40abce0 0x7f07e40b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58082/0 (socket says 172.21.2.116:58082) 2021-11-21T21:54:53.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 -- 172.21.2.116:0/2261417321 learned_addr learned my addr 172.21.2.116:0/2261417321 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:53.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866119027 0 0) 0x7f07e4147e80 con 0x7f07e40abce0 2021-11-21T21:54:53.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c4003260 con 0x7f07e40abce0 2021-11-21T21:54:53.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e259c700 1 --2- 172.21.2.116:0/2261417321 >> v2:172.21.2.117:3300/0 conn(0x7f07e4144810 0x7f07e41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 804384022 0 0) 0x7f07c4003260 con 0x7f07e40abce0 2021-11-21T21:54:53.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 >> v2:172.21.2.117:3300/0 conn(0x7f07e4144810 msgr2=0x7f07e41478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 --2- 172.21.2.116:0/2261417321 >> v2:172.21.2.117:3300/0 conn(0x7f07e4144810 0x7f07e41478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 >> v1:172.21.2.117:6789/0 conn(0x7f07e40ac600 legacy=0x7f07e40aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40b1fd0 con 0x7f07e40abce0 2021-11-21T21:54:53.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f07cc001660 con 0x7f07e40abce0 2021-11-21T21:54:53.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 >> v1:172.21.2.116:6789/0 conn(0x7f07e40abce0 legacy=0x7f07e40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07c4003750 0x7f07c4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07c4004500 0x7f07c4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07c4004ec0 0x7f07c400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b1490 con 0x7f07c4004500 2021-11-21T21:54:53.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c400bd80 con 0x7f07c4003750 2021-11-21T21:54:53.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c400bf40 con 0x7f07c4004ec0 2021-11-21T21:54:53.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07c4004500 0x7f07c4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 -- 172.21.2.116:0/2261417321 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07c4004ec0 msgr2=0x7f07c400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07c4004ec0 0x7f07c400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 -- 172.21.2.116:0/2261417321 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07c4003750 msgr2=0x7f07c4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:53.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07c4003750 0x7f07c4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 -- 172.21.2.116:0/2261417321 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4003260 con 0x7f07c4004500 2021-11-21T21:54:53.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e2d9d700 1 -- 172.21.2.116:0/2261417321 reap_dead start 2021-11-21T21:54:53.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e1d9b700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07c4004500 0x7f07c4004980 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f07e405d170 tx=0x7f07cc001e30).ready entity=mon.0 client_cookie=f18269db0be1742b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc0071a0 con 0x7f07c4004500 2021-11-21T21:54:53.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc007300 con 0x7f07c4004500 2021-11-21T21:54:53.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e159a700 1 -- 172.21.2.116:0/2261417321 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00c8f0 con 0x7f07c4004500 2021-11-21T21:54:53.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- 172.21.2.116:0/2261417321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07c4004500 msgr2=0x7f07c4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07c4004500 0x7f07c4004980 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f07e405d170 tx=0x7f07cc001e30).stop 2021-11-21T21:54:53.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- 172.21.2.116:0/2261417321 shutdown_connections 2021-11-21T21:54:53.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 --2- 172.21.2.116:0/2261417321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07c4004500 0x7f07c4004980 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:53.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- 172.21.2.116:0/2261417321 >> 172.21.2.116:0/2261417321 conn(0x7f07e405bd20 msgr2=0x7f07e405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:53.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- 172.21.2.116:0/2261417321 shutdown_connections 2021-11-21T21:54:53.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.978+0000 7f07e8eff700 1 -- 172.21.2.116:0/2261417321 wait complete. 2021-11-21T21:54:53.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 Processor -- start 2021-11-21T21:54:53.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 -- start start 2021-11-21T21:54:53.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07c4004500 0x7f07e40aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07e40a6a10 0x7f07e40a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07e41435f0 0x7f07e40a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:53.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b4450 con 0x7f07e41435f0 2021-11-21T21:54:53.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40a6f20 con 0x7f07e40a6a10 2021-11-21T21:54:53.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b4680 con 0x7f07c4004500 2021-11-21T21:54:53.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07e41435f0 0x7f07e40a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:53.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07e41435f0 0x7f07e40a7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46140/0 (socket says 172.21.2.116:46140) 2021-11-21T21:54:53.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 -- 172.21.2.116:0/4167027183 learned_addr learned my addr 172.21.2.116:0/4167027183 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:53.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 -- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07c4004500 msgr2=0x7f07e40aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07c4004500 0x7f07e40aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 -- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07e40a6a10 msgr2=0x7f07e40a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:53.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07e40a6a10 0x7f07e40a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:53.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 -- 172.21.2.116:0/4167027183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40b4920 con 0x7f07e41435f0 2021-11-21T21:54:53.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e2d9d700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07e41435f0 0x7f07e40a7f70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f07cc001110 tx=0x7f07dc007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:53.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07db7fe700 1 -- 172.21.2.116:0/4167027183 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07dc005880 con 0x7f07e41435f0 2021-11-21T21:54:53.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07db7fe700 1 -- 172.21.2.116:0/4167027183 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc0059e0 con 0x7f07e41435f0 2021-11-21T21:54:54.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40aa520 con 0x7f07e41435f0 2021-11-21T21:54:54.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07db7fe700 1 -- 172.21.2.116:0/4167027183 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07dc005d80 con 0x7f07e41435f0 2021-11-21T21:54:54.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e40a8680 con 0x7f07e41435f0 2021-11-21T21:54:54.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e40b4ab0 con 0x7f07e41435f0 2021-11-21T21:54:54.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07db7fe700 1 -- 172.21.2.116:0/4167027183 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f07dc01c070 con 0x7f07e41435f0 2021-11-21T21:54:54.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07db7fe700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07c80535b0 0x7f07c8055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07db7fe700 1 -- 172.21.2.116:0/4167027183 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f07dc06ab30 con 0x7f07e41435f0 2021-11-21T21:54:54.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.982+0000 7f07e259c700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07c80535b0 0x7f07c8055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.986+0000 7f07e259c700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07c80535b0 0x7f07c8055a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f07d400a000 tx=0x7f07d4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:53.986+0000 7f07db7fe700 1 -- 172.21.2.116:0/4167027183 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07dc041c90 con 0x7f07e41435f0 2021-11-21T21:54:54.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f07e40573d0 con 0x7f07e41435f0 2021-11-21T21:54:54.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07db7fe700 1 -- 172.21.2.116:0/4167027183 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f07dc0452b0 con 0x7f07e41435f0 2021-11-21T21:54:54.074 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:54.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07c80535b0 msgr2=0x7f07c8055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07c80535b0 0x7f07c8055a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f07d400a000 tx=0x7f07d4008040).stop 2021-11-21T21:54:54.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07e41435f0 msgr2=0x7f07e40a7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07e41435f0 0x7f07e40a7f70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f07cc001110 tx=0x7f07dc007dc0).stop 2021-11-21T21:54:54.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 shutdown_connections 2021-11-21T21:54:54.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07c80535b0 0x7f07c8055a70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07e41435f0 0x7f07e40a7f70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07e40a6a10 0x7f07e40a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 --2- 172.21.2.116:0/4167027183 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07c4004500 0x7f07e40aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 >> 172.21.2.116:0/4167027183 conn(0x7f07e405bd20 msgr2=0x7f07e40b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:54.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 shutdown_connections 2021-11-21T21:54:54.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.070+0000 7f07e8eff700 1 -- 172.21.2.116:0/4167027183 wait complete. 2021-11-21T21:54:54.080 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:54:54.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 Processor -- start 2021-11-21T21:54:54.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- start start 2021-11-21T21:54:54.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9a84144810 0x7f9a841478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840abce0 0x7f9a840b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a84147e80 con 0x7f9a840b1db0 2021-11-21T21:54:54.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a840b1490 con 0x7f9a84144810 2021-11-21T21:54:54.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a840af060 con 0x7f9a840abce0 2021-11-21T21:54:54.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a83fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9a840b1db0 0x7f9a840ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58100/0 (socket says 172.21.2.116:58100) 2021-11-21T21:54:54.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a83fff700 1 -- 172.21.2.116:0/3434369043 learned_addr learned my addr 172.21.2.116:0/3434369043 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:54.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a837fe700 1 --2- 172.21.2.116:0/3434369043 >> v2:172.21.2.116:3300/0 conn(0x7f9a84144810 0x7f9a841478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a827fc700 1 -- 172.21.2.116:0/3434369043 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474539737 0 0) 0x7f9a84147e80 con 0x7f9a840b1db0 2021-11-21T21:54:54.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a837fe700 1 -- 172.21.2.116:0/3434369043 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840abce0 msgr2=0x7f9a840b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:54:54.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a837fe700 1 --2- 172.21.2.116:0/3434369043 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840abce0 0x7f9a840b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a837fe700 1 -- 172.21.2.116:0/3434369043 >> v1:172.21.2.116:6789/0 conn(0x7f9a840b1db0 legacy=0x7f9a840ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a837fe700 1 -- 172.21.2.116:0/3434369043 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a840af5a0 con 0x7f9a84144810 2021-11-21T21:54:54.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a837fe700 1 --2- 172.21.2.116:0/3434369043 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a84144810 0x7f9a841478c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f9a78002f70 tx=0x7f9a78006df0).ready entity=mon.0 client_cookie=4b7c78f683e22d3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a827fc700 1 -- 172.21.2.116:0/3434369043 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a7800d570 con 0x7f9a84144810 2021-11-21T21:54:54.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a827fc700 1 -- 172.21.2.116:0/3434369043 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78000e90 con 0x7f9a84144810 2021-11-21T21:54:54.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a827fc700 1 -- 172.21.2.116:0/3434369043 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a7800d900 con 0x7f9a84144810 2021-11-21T21:54:54.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- 172.21.2.116:0/3434369043 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a84144810 msgr2=0x7f9a841478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3434369043 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a84144810 0x7f9a841478c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f9a78002f70 tx=0x7f9a78006df0).stop 2021-11-21T21:54:54.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- 172.21.2.116:0/3434369043 shutdown_connections 2021-11-21T21:54:54.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3434369043 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840abce0 0x7f9a840b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3434369043 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a84144810 0x7f9a841478c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- 172.21.2.116:0/3434369043 >> 172.21.2.116:0/3434369043 conn(0x7f9a8405bd20 msgr2=0x7f9a8405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:54.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- 172.21.2.116:0/3434369043 shutdown_connections 2021-11-21T21:54:54.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- 172.21.2.116:0/3434369043 wait complete. 2021-11-21T21:54:54.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 Processor -- start 2021-11-21T21:54:54.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.130+0000 7f9a89b66700 1 -- start start 2021-11-21T21:54:54.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a840abce0 0x7f9a84146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a84144810 0x7f9a8413c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840b4690 0x7f9a8413c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a84149dd0 con 0x7f9a840abce0 2021-11-21T21:54:54.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841491b0 con 0x7f9a84144810 2021-11-21T21:54:54.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8414a040 con 0x7f9a840b4690 2021-11-21T21:54:54.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a840abce0 0x7f9a84146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a840abce0 0x7f9a84146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46150/0 (socket says 172.21.2.116:46150) 2021-11-21T21:54:54.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 -- 172.21.2.116:0/3686391273 learned_addr learned my addr 172.21.2.116:0/3686391273 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:54.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 -- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840b4690 msgr2=0x7f9a8413c9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:54.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840b4690 0x7f9a8413c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 -- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a84144810 msgr2=0x7f9a8413c4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:54:54.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a84144810 0x7f9a8413c4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 -- 172.21.2.116:0/3686391273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7800b180 con 0x7f9a840abce0 2021-11-21T21:54:54.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a837fe700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a840abce0 0x7f9a84146640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9a78002f70 tx=0x7f9a78015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a80ff9700 1 -- 172.21.2.116:0/3686391273 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a78011ce0 con 0x7f9a840abce0 2021-11-21T21:54:54.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a80ff9700 1 -- 172.21.2.116:0/3686391273 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78011e40 con 0x7f9a840abce0 2021-11-21T21:54:54.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a80ff9700 1 -- 172.21.2.116:0/3686391273 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a7801e650 con 0x7f9a840abce0 2021-11-21T21:54:54.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a840b3b70 con 0x7f9a840abce0 2021-11-21T21:54:54.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a840b40b0 con 0x7f9a840abce0 2021-11-21T21:54:54.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a840af300 con 0x7f9a840abce0 2021-11-21T21:54:54.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a80ff9700 1 -- 172.21.2.116:0/3686391273 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9a78025030 con 0x7f9a840abce0 2021-11-21T21:54:54.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a80ff9700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a64053690 0x7f9a64055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a80ff9700 1 -- 172.21.2.116:0/3686391273 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9a7806ce00 con 0x7f9a840abce0 2021-11-21T21:54:54.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a82ffd700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a64053690 0x7f9a64055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.134+0000 7f9a82ffd700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a64053690 0x7f9a64055b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f9a74002fb0 tx=0x7f9a74006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.138+0000 7f9a80ff9700 1 -- 172.21.2.116:0/3686391273 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a78043f60 con 0x7f9a840abce0 2021-11-21T21:54:54.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.218+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a840ad630 con 0x7f9a840abce0 2021-11-21T21:54:54.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.218+0000 7f9a80ff9700 1 -- 172.21.2.116:0/3686391273 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9a78023020 con 0x7f9a840abce0 2021-11-21T21:54:54.224 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:54.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a64053690 msgr2=0x7f9a64055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a64053690 0x7f9a64055b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f9a74002fb0 tx=0x7f9a74006930).stop 2021-11-21T21:54:54.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a840abce0 msgr2=0x7f9a84146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a840abce0 0x7f9a84146640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9a78002f70 tx=0x7f9a78015000).stop 2021-11-21T21:54:54.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 shutdown_connections 2021-11-21T21:54:54.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a840b4690 0x7f9a8413c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a84144810 0x7f9a8413c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9a64053690 0x7f9a64055b50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 --2- 172.21.2.116:0/3686391273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a840abce0 0x7f9a84146640 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 >> 172.21.2.116:0/3686391273 conn(0x7f9a8405bd20 msgr2=0x7f9a84141870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:54.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 shutdown_connections 2021-11-21T21:54:54.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.222+0000 7f9a89b66700 1 -- 172.21.2.116:0/3686391273 wait complete. 2021-11-21T21:54:54.231 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:54:54.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.278+0000 7fb0e2cf0700 1 Processor -- start 2021-11-21T21:54:54.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.278+0000 7fb0e2cf0700 1 -- start start 2021-11-21T21:54:54.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e2cf0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb0dc0b0ca0 0x7fb0dc0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e2cf0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0ac5a0 0x7fb0dc0634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e2cf0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc140930 con 0x7fb0dc067890 2021-11-21T21:54:54.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e2cf0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b1080 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e2cf0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0acd20 con 0x7fb0dc0ac5a0 2021-11-21T21:54:54.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e128d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0ac5a0 0x7fb0dc0634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0dbfff700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fb0dc067890 0x7fb0dc140220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:57024/0 (socket says 172.21.2.116:57024) 2021-11-21T21:54:54.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0dbfff700 1 -- 172.21.2.116:0/4172967988 learned_addr learned my addr 172.21.2.116:0/4172967988 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:54.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e0a8c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb0dc0b0ca0 0x7fb0dc0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e0a8c700 1 -- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0ac5a0 msgr2=0x7fb0dc0634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:54.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e0a8c700 1 --2- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0ac5a0 0x7fb0dc0634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e0a8c700 1 -- 172.21.2.116:0/4172967988 >> v1:172.21.2.117:6789/0 conn(0x7fb0dc067890 legacy=0x7fb0dc140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e0a8c700 1 -- 172.21.2.116:0/4172967988 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0acfc0 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e128d700 1 --2- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0ac5a0 0x7fb0dc0634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:54:54.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0e0a8c700 1 --2- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b1790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fb0c4002f70 tx=0x7fb0c4006df0).ready entity=mon.1 client_cookie=cf16ec7697a462aa server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0db7fe700 1 -- 172.21.2.116:0/4172967988 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0c400d570 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0db7fe700 1 -- 172.21.2.116:0/4172967988 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c4000e90 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.282+0000 7fb0db7fe700 1 -- 172.21.2.116:0/4172967988 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0c400d8c0 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0b0ca0 msgr2=0x7fb0dc0b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b1790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fb0c4002f70 tx=0x7fb0c4006df0).stop 2021-11-21T21:54:54.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/4172967988 shutdown_connections 2021-11-21T21:54:54.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0ac5a0 0x7fb0dc0634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/4172967988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b1790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/4172967988 >> 172.21.2.116:0/4172967988 conn(0x7fb0dc05bd20 msgr2=0x7fb0dc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:54.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/4172967988 shutdown_connections 2021-11-21T21:54:54.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/4172967988 wait complete. 2021-11-21T21:54:54.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 Processor -- start 2021-11-21T21:54:54.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- start start 2021-11-21T21:54:54.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc067890 0x7fb0dc13e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0ac5a0 0x7fb0dc140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc13f5f0 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1423a0 con 0x7fb0dc0ac5a0 2021-11-21T21:54:54.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc141490 con 0x7fb0dc067890 2021-11-21T21:54:54.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46164/0 (socket says 172.21.2.116:46164) 2021-11-21T21:54:54.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 -- 172.21.2.116:0/2350290758 learned_addr learned my addr 172.21.2.116:0/2350290758 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:54.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e0a8c700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc067890 0x7fb0dc13e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0dbfff700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0ac5a0 0x7fb0dc140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 -- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc067890 msgr2=0x7fb0dc13e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc067890 0x7fb0dc13e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 -- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0ac5a0 msgr2=0x7fb0dc140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0ac5a0 0x7fb0dc140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 -- 172.21.2.116:0/2350290758 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4009470 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e0a8c700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc067890 0x7fb0dc13e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:54.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0dbfff700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0ac5a0 0x7fb0dc140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:54:54.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e128d700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b3530 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fb0d400c000 tx=0x7fb0d40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0d9ffb700 1 -- 172.21.2.116:0/2350290758 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0d400c9d0 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0d9ffb700 1 -- 172.21.2.116:0/2350290758 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d400cb30 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0d9ffb700 1 -- 172.21.2.116:0/2350290758 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0d400ce90 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c400b180 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc142600 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.286+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc0573d0 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.290+0000 7fb0d9ffb700 1 -- 172.21.2.116:0/2350290758 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb0d4022070 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.290+0000 7fb0d9ffb700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0c0053660 0x7fb0c0055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.290+0000 7fb0d9ffb700 1 -- 172.21.2.116:0/2350290758 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb0d406c260 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.290+0000 7fb0e0a8c700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0c0053660 0x7fb0c0055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.290+0000 7fb0e0a8c700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0c0053660 0x7fb0c0055b20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb0c4015000 tx=0x7fb0c4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.290+0000 7fb0d9ffb700 1 -- 172.21.2.116:0/2350290758 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb0d40432b0 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0dc0aeda0 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0d9ffb700 1 -- 172.21.2.116:0/2350290758 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb0d4016020 con 0x7fb0dc0b0ca0 2021-11-21T21:54:54.379 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:54:54.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0c0053660 msgr2=0x7fb0c0055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0c0053660 0x7fb0c0055b20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb0c4015000 tx=0x7fb0c4014000).stop 2021-11-21T21:54:54.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0b0ca0 msgr2=0x7fb0dc0b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b3530 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fb0d400c000 tx=0x7fb0d40091d0).stop 2021-11-21T21:54:54.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 shutdown_connections 2021-11-21T21:54:54.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0c0053660 0x7fb0c0055b20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0b0ca0 0x7fb0dc0b3530 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0ac5a0 0x7fb0dc140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 --2- 172.21.2.116:0/2350290758 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc067890 0x7fb0dc13e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 >> 172.21.2.116:0/2350290758 conn(0x7fb0dc05bd20 msgr2=0x7fb0dc0ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:54.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.374+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 shutdown_connections 2021-11-21T21:54:54.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.378+0000 7fb0e2cf0700 1 -- 172.21.2.116:0/2350290758 wait complete. 2021-11-21T21:54:54.386 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:54:54.387 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:54:54.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98bd67700 1 Processor -- start 2021-11-21T21:54:54.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98bd67700 1 -- start start 2021-11-21T21:54:54.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98bd67700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc9840ad780 0x7fc98405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98bd67700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc984143f70 con 0x7fc9840affc0 2021-11-21T21:54:54.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98bd67700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9840adb60 con 0x7fc9840af7e0 2021-11-21T21:54:54.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98bd67700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc984143160 con 0x7fc9840ad780 2021-11-21T21:54:54.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc989302700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc9840affc0 0x7fc984143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58130/0 (socket says 172.21.2.116:58130) 2021-11-21T21:54:54.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc989302700 1 -- 172.21.2.116:0/550133653 learned_addr learned my addr 172.21.2.116:0/550133653 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:54:54.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030520145 0 0) 0x7fc984143f70 con 0x7fc9840affc0 2021-11-21T21:54:54.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc96c003260 con 0x7fc9840affc0 2021-11-21T21:54:54.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161641207 0 0) 0x7fc96c003260 con 0x7fc9840affc0 2021-11-21T21:54:54.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 >> v2:172.21.2.117:3300/0 conn(0x7fc9840ad780 msgr2=0x7fc98405f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:54.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 --2- 172.21.2.116:0/550133653 >> v2:172.21.2.117:3300/0 conn(0x7fc9840ad780 0x7fc98405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 >> v1:172.21.2.117:6789/0 conn(0x7fc9840af7e0 legacy=0x7fc98413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:54.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840b0ce0 con 0x7fc9840affc0 2021-11-21T21:54:54.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fc974001660 con 0x7fc9840affc0 2021-11-21T21:54:54.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 >> v1:172.21.2.116:6789/0 conn(0x7fc9840affc0 legacy=0x7fc984143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc96c003820 0x7fc96c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc96c004500 0x7fc96c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc96c005050 0x7fc96c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc984143160 con 0x7fc96c005050 2021-11-21T21:54:54.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c00bf40 con 0x7fc96c004500 2021-11-21T21:54:54.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c00c100 con 0x7fc96c003820 2021-11-21T21:54:54.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98a304700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc96c005050 0x7fc96c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc989302700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc96c004500 0x7fc96c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.434+0000 7fc98a304700 1 -- 172.21.2.116:0/550133653 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc96c003820 msgr2=0x7fc96c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:54.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc96c003820 0x7fc96c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 -- 172.21.2.116:0/550133653 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc96c004500 msgr2=0x7fc96c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:54:54.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc96c004500 0x7fc96c004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 -- 172.21.2.116:0/550133653 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c003260 con 0x7fc96c005050 2021-11-21T21:54:54.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 -- 172.21.2.116:0/550133653 reap_dead start 2021-11-21T21:54:54.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc96c005050 0x7fc96c00b750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fc97c0012f0 tx=0x7fc97c007d60).ready entity=mon.0 client_cookie=a19afafd853ef8bc server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc97c0014c0 con 0x7fc96c005050 2021-11-21T21:54:54.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c006870 con 0x7fc96c005050 2021-11-21T21:54:54.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc988b01700 1 -- 172.21.2.116:0/550133653 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc97c006bd0 con 0x7fc96c005050 2021-11-21T21:54:54.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/550133653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc96c005050 msgr2=0x7fc96c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc96c005050 0x7fc96c00b750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fc97c0012f0 tx=0x7fc97c007d60).stop 2021-11-21T21:54:54.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/550133653 shutdown_connections 2021-11-21T21:54:54.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 --2- 172.21.2.116:0/550133653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc96c005050 0x7fc96c00b750 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/550133653 >> 172.21.2.116:0/550133653 conn(0x7fc9840b6000 msgr2=0x7fc9840b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:54.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/550133653 shutdown_connections 2021-11-21T21:54:54.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/550133653 wait complete. 2021-11-21T21:54:54.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 Processor -- start 2021-11-21T21:54:54.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- start start 2021-11-21T21:54:54.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc96c005050 0x7fc9841446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc984144bb0 0x7fc9840aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9840aabc0 0x7fc9840655d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc984064c50 con 0x7fc9840aabc0 2021-11-21T21:54:54.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840acb90 con 0x7fc984144bb0 2021-11-21T21:54:54.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840ace30 con 0x7fc96c005050 2021-11-21T21:54:54.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9840aabc0 0x7fc9840655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9840aabc0 0x7fc9840655d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46184/0 (socket says 172.21.2.116:46184) 2021-11-21T21:54:54.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 -- 172.21.2.116:0/1270802816 learned_addr learned my addr 172.21.2.116:0/1270802816 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:54:54.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 -- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc96c005050 msgr2=0x7fc9841446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc96c005050 0x7fc9841446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 -- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc984144bb0 msgr2=0x7fc9840aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc984144bb0 0x7fc9840aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 -- 172.21.2.116:0/1270802816 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc984066680 con 0x7fc9840aabc0 2021-11-21T21:54:54.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98a304700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9840aabc0 0x7fc9840655d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fc97c0012f0 tx=0x7fc97c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc972ffd700 1 -- 172.21.2.116:0/1270802816 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc97c014860 con 0x7fc9840aabc0 2021-11-21T21:54:54.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc984066b20 con 0x7fc9840aabc0 2021-11-21T21:54:54.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc972ffd700 1 -- 172.21.2.116:0/1270802816 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c0149c0 con 0x7fc9840aabc0 2021-11-21T21:54:54.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc972ffd700 1 -- 172.21.2.116:0/1270802816 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc97c014d20 con 0x7fc9840aabc0 2021-11-21T21:54:54.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9840ab070 con 0x7fc9840aabc0 2021-11-21T21:54:54.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984066810 con 0x7fc9840aabc0 2021-11-21T21:54:54.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc972ffd700 1 -- 172.21.2.116:0/1270802816 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc97c028030 con 0x7fc9840aabc0 2021-11-21T21:54:54.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc972ffd700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9640535b0 0x7fc964055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:54:54.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.438+0000 7fc972ffd700 1 -- 172.21.2.116:0/1270802816 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc97c06d0f0 con 0x7fc9840aabc0 2021-11-21T21:54:54.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.442+0000 7fc989b03700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9640535b0 0x7fc964055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:54:54.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.442+0000 7fc989b03700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9640535b0 0x7fc964055a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fc980002f70 tx=0x7fc980006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:54:54.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.442+0000 7fc972ffd700 1 -- 172.21.2.116:0/1270802816 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc97c044140 con 0x7fc9840aabc0 2021-11-21T21:54:54.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9840b0a40 con 0x7fc9840aabc0 2021-11-21T21:54:54.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc972ffd700 1 -- 172.21.2.116:0/1270802816 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2480 (secure 0 0 0) 0x7fc97c047760 con 0x7fc9840aabc0 2021-11-21T21:54:54.554 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:54:54.554 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:54:52.356582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2051796411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2051796411},{"type":"v1","addr":"172.21.2.116:6835","nonce":2051796411}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:54:54.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9640535b0 msgr2=0x7fc964055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9640535b0 0x7fc964055a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fc980002f70 tx=0x7fc980006a00).stop 2021-11-21T21:54:54.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9840aabc0 msgr2=0x7fc9840655d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:54:54.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9840aabc0 0x7fc9840655d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fc97c0012f0 tx=0x7fc97c016000).stop 2021-11-21T21:54:54.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 shutdown_connections 2021-11-21T21:54:54.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9640535b0 0x7fc964055a70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9840aabc0 0x7fc9840655d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:54:54.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc984144bb0 0x7fc9840aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 --2- 172.21.2.116:0/1270802816 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc96c005050 0x7fc9841446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:54:54.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 >> 172.21.2.116:0/1270802816 conn(0x7fc9840b6000 msgr2=0x7fc9840b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:54:54.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 shutdown_connections 2021-11-21T21:54:54.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:54:54.550+0000 7fc98bd67700 1 -- 172.21.2.116:0/1270802816 wait complete. 2021-11-21T21:54:54.558 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 11 2021-11-21T21:54:54.863 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:54:51.352804+0000', 'modified': '2021-11-21T21:54:52.356582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4878}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4878': {'gid': 4878, 'name': 'c', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2051796411', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2051796411}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2051796411}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:54:54.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:54:54.865 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:54:54.929 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":4878,"want_state":"up:active","state":"up:active","rank_uptime":3.5689381230000001,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":31,"uptime":8.7879406769999999} 2021-11-21T21:54:54.929 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 4878, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 31, "rank_uptime": 3.568938123, "state": "up:active", "uptime": 8.787940677, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:54:54.930 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:54:54.931 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T21:54:54.931 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T21:54:54.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:54.932 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:54.944 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:54:54.999 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:54:54.999 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:54:55.005 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:54:55.006 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:54:55.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:55.007 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:54:55.011 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:54:55.012 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:55.057 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:54:55.066 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:54:55.066 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:54:55.071 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:54:55.072 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:54:55.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:55.074 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:54:55.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T21:54:55.089+0000 7f9a50d39080 -1 init, newargv = 0x559da2c929f0 newargc=9 2021-11-21T21:54:55.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14058]: starting ceph client 2021-11-21T21:54:55.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14058]: starting fuse 2021-11-21T21:54:56.179 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:54:56.191 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:54:56.192 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:54:56.242 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:54:56.243 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:54:56.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:56.244 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:54:56.249 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T21:54:56.250 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:54:56.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:56.251 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T21:54:56.251 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T21:54:56.251 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T21:54:56.252 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:54:56.252 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T21:54:56.253 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:56.253 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T21:54:56.253 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T21:54:56.254 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T21:54:56.254 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:56.254 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T21:54:56.256 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T21:54:56.256 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T21:54:56.257 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T21:54:56.257 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:56.258 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T21:54:56.259 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:54:56.259 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:54:56.259 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T21:54:56.260 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:54:56.260 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:56.261 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T21:54:56.261 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:54:56.327 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14058.asok 2021-11-21T21:54:56.329 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14058.asok 2021-11-21T21:54:56.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:56.330 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14058.asok status 2021-11-21T21:54:56.380 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T21:54:56.380 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T21:54:56.381 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:54:56.381 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:54:56.381 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T21:54:56.381 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T21:54:56.382 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T21:54:56.382 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14058", 2021-11-21T21:54:56.382 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T21:54:56.382 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:56.382 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T21:54:56.383 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T21:54:56.383 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4417, 2021-11-21T21:54:56.383 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T21:54:56.383 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T21:54:56.384 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T21:54:56.384 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4417 2021-11-21T21:54:56.384 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:56.384 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:54:56.385 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:54:56.385 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:54:56.385 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 276183154 2021-11-21T21:54:56.385 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T21:54:56.385 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:56.386 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:54:56.386 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:54:56.386 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:54:56.386 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 276183154 2021-11-21T21:54:56.387 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:54:56.387 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4417 172.21.2.119:0/276183154", 2021-11-21T21:54:56.387 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/276183154", 2021-11-21T21:54:56.388 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T21:54:56.388 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 11, 2021-11-21T21:54:56.388 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 33, 2021-11-21T21:54:56.388 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:54:56.388 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T21:54:56.389 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T21:54:56.389 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T21:54:56.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:56.391 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:56.391 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:56.398 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:54:56.399 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:54:56.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:56.401 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:56.449 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2021-11-21T21:54:56.507 INFO:teuthology.orchestra.run.gibba016.stdout:{"mds_min_caps_per_client":"100"} 2021-11-21T21:54:56.508 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2021-11-21T21:54:56.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:54:56.509 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:54:56.510 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:54:56.522 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:54:56.523 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:54:56.524 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T21:54:56.524 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-11-21T21:54:56.525 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-11-21T21:54:56.525 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:54:56.526 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:56.526 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-11-21T21:54:56.527 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-11-21T21:54:56.527 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:56.528 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-11-21T21:54:56.528 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-11-21T21:54:56.529 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:54:56.529 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-11-21T21:54:56.530 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-11-21T21:54:56.530 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-11-21T21:54:56.530 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-11-21T21:54:56.531 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-11-21T21:54:56.531 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-11-21T21:54:56.532 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-11-21T21:54:56.532 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:55:03.493 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:03.496 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:03.499 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:03.503 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:11.448 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:55:11.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.498+0000 7fbf0ca89700 1 Processor -- start 2021-11-21T21:55:11.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.498+0000 7fbf0ca89700 1 -- start start 2021-11-21T21:55:11.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbf080a86b0 0x7fbf0814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080a6e80 con 0x7fbf08062410 2021-11-21T21:55:11.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0814a7b0 con 0x7fbf080a6aa0 2021-11-21T21:55:11.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf080a8bc0 con 0x7fbf080a86b0 2021-11-21T21:55:11.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf06d9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbf08062410 0x7fbf081443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58148/0 (socket says 172.21.2.116:58148) 2021-11-21T21:55:11.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf06d9d700 1 -- 172.21.2.116:0/498845752 learned_addr learned my addr 172.21.2.116:0/498845752 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:55:11.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629775430 0 0) 0x7fbf080a6e80 con 0x7fbf08062410 2021-11-21T21:55:11.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee8003260 con 0x7fbf08062410 2021-11-21T21:55:11.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962740129 0 0) 0x7fbee8003260 con 0x7fbf08062410 2021-11-21T21:55:11.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf05d9b700 1 --2- 172.21.2.116:0/498845752 >> v2:172.21.2.117:3300/0 conn(0x7fbf080a86b0 0x7fbf0814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 >> v2:172.21.2.117:3300/0 conn(0x7fbf080a86b0 msgr2=0x7fbf0814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 --2- 172.21.2.116:0/498845752 >> v2:172.21.2.117:3300/0 conn(0x7fbf080a86b0 0x7fbf0814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 >> v1:172.21.2.117:6789/0 conn(0x7fbf080a6aa0 legacy=0x7fbf081470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0814aa10 con 0x7fbf08062410 2021-11-21T21:55:11.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fbf00001a00 con 0x7fbf08062410 2021-11-21T21:55:11.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 >> v1:172.21.2.116:6789/0 conn(0x7fbf08062410 legacy=0x7fbf081443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbee8003820 0x7fbee8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8004500 0x7fbee8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbee8005050 0x7fbee800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080a8bc0 con 0x7fbee8003820 2021-11-21T21:55:11.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee800bf40 con 0x7fbee8005050 2021-11-21T21:55:11.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee800c100 con 0x7fbee8004500 2021-11-21T21:55:11.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf05d9b700 1 --2- 172.21.2.116:0/498845752 >> v2:172.21.2.117:3300/0 conn(0x7fbf080a86b0 0x7fbf0814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:55:11.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0659c700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbee8003820 0x7fbee8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf06d9d700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbee8005050 0x7fbee800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0659c700 1 -- 172.21.2.116:0/498845752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8004500 msgr2=0x7fbee8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0659c700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8004500 0x7fbee8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0659c700 1 -- 172.21.2.116:0/498845752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbee8005050 msgr2=0x7fbee800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0659c700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbee8005050 0x7fbee800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0659c700 1 -- 172.21.2.116:0/498845752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8003260 con 0x7fbee8003820 2021-11-21T21:55:11.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf06d9d700 1 -- 172.21.2.116:0/498845752 reap_dead start 2021-11-21T21:55:11.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0659c700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbee8003820 0x7fbee8003c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fbef8001fa0 tx=0x7fbef8007a00).ready entity=mon.0 client_cookie=5ecd6c9e8628e5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef8007070 con 0x7fbee8003820 2021-11-21T21:55:11.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef80071d0 con 0x7fbee8003820 2021-11-21T21:55:11.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0559a700 1 -- 172.21.2.116:0/498845752 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef8006510 con 0x7fbee8003820 2021-11-21T21:55:11.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- 172.21.2.116:0/498845752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbee8003820 msgr2=0x7fbee8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbee8003820 0x7fbee8003c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fbef8001fa0 tx=0x7fbef8007a00).stop 2021-11-21T21:55:11.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- 172.21.2.116:0/498845752 shutdown_connections 2021-11-21T21:55:11.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/498845752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbee8003820 0x7fbee8003c20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- 172.21.2.116:0/498845752 >> 172.21.2.116:0/498845752 conn(0x7fbf0805bd20 msgr2=0x7fbf0805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:11.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- 172.21.2.116:0/498845752 shutdown_connections 2021-11-21T21:55:11.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- 172.21.2.116:0/498845752 wait complete. 2021-11-21T21:55:11.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 Processor -- start 2021-11-21T21:55:11.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.502+0000 7fbf0ca89700 1 -- start start 2021-11-21T21:55:11.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8003820 0x7fbf08157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf0814e2a0 0x7fbf080642c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf080656f0 0x7fbf08066250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0814e700 con 0x7fbf080656f0 2021-11-21T21:55:11.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf08156440 con 0x7fbf0814e2a0 2021-11-21T21:55:11.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080647b0 con 0x7fbee8003820 2021-11-21T21:55:11.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf080656f0 0x7fbf08066250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf080656f0 0x7fbf08066250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46208/0 (socket says 172.21.2.116:46208) 2021-11-21T21:55:11.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 -- 172.21.2.116:0/880421748 learned_addr learned my addr 172.21.2.116:0/880421748 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:11.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0659c700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8003820 0x7fbf08157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf05d9b700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf0814e2a0 0x7fbf080642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 -- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8003820 msgr2=0x7fbf08157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8003820 0x7fbf08157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 -- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf0814e2a0 msgr2=0x7fbf080642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf0814e2a0 0x7fbf080642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 -- 172.21.2.116:0/880421748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf08156630 con 0x7fbf080656f0 2021-11-21T21:55:11.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0659c700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8003820 0x7fbf08157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:55:11.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf05d9b700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf0814e2a0 0x7fbf080642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:55:11.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf06d9d700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf080656f0 0x7fbf08066250 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fbf08064080 tx=0x7fbf080a8520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbeff7fe700 1 -- 172.21.2.116:0/880421748 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf00010710 con 0x7fbf080656f0 2021-11-21T21:55:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbeff7fe700 1 -- 172.21.2.116:0/880421748 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00010870 con 0x7fbf080656f0 2021-11-21T21:55:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbeff7fe700 1 -- 172.21.2.116:0/880421748 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf00010bd0 con 0x7fbf080656f0 2021-11-21T21:55:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf081568d0 con 0x7fbf080656f0 2021-11-21T21:55:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf080a9f50 con 0x7fbf080656f0 2021-11-21T21:55:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf080573d0 con 0x7fbf080656f0 2021-11-21T21:55:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbeff7fe700 1 -- 172.21.2.116:0/880421748 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbf0001d070 con 0x7fbf080656f0 2021-11-21T21:55:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbeff7fe700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbeec053580 0x7fbeec055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbeff7fe700 1 -- 172.21.2.116:0/880421748 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbf0006b460 con 0x7fbf080656f0 2021-11-21T21:55:11.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0659c700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbeec053580 0x7fbeec055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.506+0000 7fbf0659c700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbeec053580 0x7fbeec055a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fbef8016000 tx=0x7fbef8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.510+0000 7fbeff7fe700 1 -- 172.21.2.116:0/880421748 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf000425c0 con 0x7fbf080656f0 2021-11-21T21:55:11.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.618+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf0805db70 con 0x7fbf080656f0 2021-11-21T21:55:11.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.618+0000 7fbeff7fe700 1 -- 172.21.2.116:0/880421748 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2480 (secure 0 0 0) 0x7fbf00045be0 con 0x7fbf080656f0 2021-11-21T21:55:11.624 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:55:11.624 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:54:52.356582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2051796411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2051796411},{"type":"v1","addr":"172.21.2.116:6835","nonce":2051796411}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:55:11.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.618+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbeec053580 msgr2=0x7fbeec055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.618+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbeec053580 0x7fbeec055a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fbef8016000 tx=0x7fbef8015000).stop 2021-11-21T21:55:11.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.618+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf080656f0 msgr2=0x7fbf08066250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.618+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf080656f0 0x7fbf08066250 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fbf08064080 tx=0x7fbf080a8520).stop 2021-11-21T21:55:11.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 shutdown_connections 2021-11-21T21:55:11.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbeec053580 0x7fbeec055a40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf080656f0 0x7fbf08066250 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf0814e2a0 0x7fbf080642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 --2- 172.21.2.116:0/880421748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbee8003820 0x7fbf08157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 >> 172.21.2.116:0/880421748 conn(0x7fbf0805bd20 msgr2=0x7fbf08149d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:11.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 shutdown_connections 2021-11-21T21:55:11.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.622+0000 7fbf0ca89700 1 -- 172.21.2.116:0/880421748 wait complete. 2021-11-21T21:55:11.628 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 11 2021-11-21T21:55:11.631 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2021-11-21T21:55:11.688 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-11-21T21:55:11.689 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-11-21T21:55:11.690 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:55:11.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.734+0000 7f597ee89700 1 Processor -- start 2021-11-21T21:55:11.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.734+0000 7f597ee89700 1 -- start start 2021-11-21T21:55:11.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f59780a6aa0 0x7f597814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f59780a86b0 0x7f59781470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59780a6e80 con 0x7f59781475d0 2021-11-21T21:55:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59780a8bc0 con 0x7f59780a86b0 2021-11-21T21:55:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597814a7b0 con 0x7f59780a6aa0 2021-11-21T21:55:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f59780a86b0 0x7f59781470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f59780a86b0 0x7f59781470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46216/0 (socket says 172.21.2.116:46216) 2021-11-21T21:55:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 -- 172.21.2.116:0/672005346 learned_addr learned my addr 172.21.2.116:0/672005346 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 -- 172.21.2.116:0/672005346 >> v2:172.21.2.117:3300/0 conn(0x7f59780a6aa0 msgr2=0x7f597814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 --2- 172.21.2.116:0/672005346 >> v2:172.21.2.117:3300/0 conn(0x7f59780a6aa0 0x7f597814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 -- 172.21.2.116:0/672005346 >> v1:172.21.2.117:6789/0 conn(0x7f59781475d0 legacy=0x7f59781443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 -- 172.21.2.116:0/672005346 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597814aa10 con 0x7f59780a86b0 2021-11-21T21:55:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5977fff700 1 --2- 172.21.2.116:0/672005346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59780a86b0 0x7f59781470e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f5978061ab0 tx=0x7f5960006eb0).ready entity=mon.0 client_cookie=2d6574ea0a283b81 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f59777fe700 1 -- 172.21.2.116:0/672005346 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596000d5c0 con 0x7f59780a86b0 2021-11-21T21:55:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f59777fe700 1 -- 172.21.2.116:0/672005346 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5960000e90 con 0x7f59780a86b0 2021-11-21T21:55:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f59777fe700 1 -- 172.21.2.116:0/672005346 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596000d910 con 0x7f59780a86b0 2021-11-21T21:55:11.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- 172.21.2.116:0/672005346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59780a86b0 msgr2=0x7f59781470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- 172.21.2.116:0/672005346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59780a86b0 0x7f59781470e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f5978061ab0 tx=0x7f5960006eb0).stop 2021-11-21T21:55:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- 172.21.2.116:0/672005346 shutdown_connections 2021-11-21T21:55:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- 172.21.2.116:0/672005346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59780a86b0 0x7f59781470e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- 172.21.2.116:0/672005346 >> v2:172.21.2.117:3300/0 conn(0x7f59780a6aa0 0x7f597814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- 172.21.2.116:0/672005346 >> 172.21.2.116:0/672005346 conn(0x7f597805bd20 msgr2=0x7f597805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- 172.21.2.116:0/672005346 shutdown_connections 2021-11-21T21:55:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- 172.21.2.116:0/672005346 wait complete. 2021-11-21T21:55:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 Processor -- start 2021-11-21T21:55:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- start start 2021-11-21T21:55:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59780a6aa0 0x7f59780aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59780a86b0 0x7f59780aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59781475d0 0x7f5978064770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780a94f0 con 0x7f59781475d0 2021-11-21T21:55:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780a9240 con 0x7f59780a6aa0 2021-11-21T21:55:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5978157420 con 0x7f59780a86b0 2021-11-21T21:55:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59781475d0 0x7f5978064770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59781475d0 0x7f5978064770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46222/0 (socket says 172.21.2.116:46222) 2021-11-21T21:55:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 -- 172.21.2.116:0/3259934943 learned_addr learned my addr 172.21.2.116:0/3259934943 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 -- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59780a86b0 msgr2=0x7f59780aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59780a86b0 0x7f59780aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 -- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59780a6aa0 msgr2=0x7f59780aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59780a6aa0 0x7f59780aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 -- 172.21.2.116:0/3259934943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596000b1e0 con 0x7f59781475d0 2021-11-21T21:55:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597d426700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59781475d0 0x7f5978064770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f5970003270 tx=0x7f59700090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5975ffb700 1 -- 172.21.2.116:0/3259934943 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f597000d4a0 con 0x7f59781475d0 2021-11-21T21:55:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5975ffb700 1 -- 172.21.2.116:0/3259934943 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597000d600 con 0x7f59781475d0 2021-11-21T21:55:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59780a9720 con 0x7f59781475d0 2021-11-21T21:55:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f5975ffb700 1 -- 172.21.2.116:0/3259934943 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f597000d960 con 0x7f59781475d0 2021-11-21T21:55:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.738+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978146d60 con 0x7f59781475d0 2021-11-21T21:55:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.742+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59780573d0 con 0x7f59781475d0 2021-11-21T21:55:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.742+0000 7f5975ffb700 1 -- 172.21.2.116:0/3259934943 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5970018030 con 0x7f59781475d0 2021-11-21T21:55:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.742+0000 7f5975ffb700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f595c053640 0x7f595c055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.742+0000 7f5975ffb700 1 -- 172.21.2.116:0/3259934943 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f597006b1d0 con 0x7f59781475d0 2021-11-21T21:55:11.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.742+0000 7f597cc25700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f595c053640 0x7f595c055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.742+0000 7f597cc25700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f595c053640 0x7f595c055b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5968002f70 tx=0x7f5968006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.742+0000 7f5975ffb700 1 -- 172.21.2.116:0/3259934943 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59700423e0 con 0x7f59781475d0 2021-11-21T21:55:11.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.850+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59780b2de0 con 0x7f59781475d0 2021-11-21T21:55:11.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.850+0000 7f5975ffb700 1 -- 172.21.2.116:0/3259934943 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2480 (secure 0 0 0) 0x7f5970045a00 con 0x7f59781475d0 2021-11-21T21:55:11.856 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:55:11.857 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:54:52.356582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2051796411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2051796411},{"type":"v1","addr":"172.21.2.116:6835","nonce":2051796411}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:55:11.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f595c053640 msgr2=0x7f595c055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f595c053640 0x7f595c055b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5968002f70 tx=0x7f5968006a80).stop 2021-11-21T21:55:11.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59781475d0 msgr2=0x7f5978064770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59781475d0 0x7f5978064770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f5970003270 tx=0x7f59700090f0).stop 2021-11-21T21:55:11.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 shutdown_connections 2021-11-21T21:55:11.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f595c053640 0x7f595c055b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59781475d0 0x7f5978064770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:11.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59780a86b0 0x7f59780aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 --2- 172.21.2.116:0/3259934943 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59780a6aa0 0x7f59780aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 >> 172.21.2.116:0/3259934943 conn(0x7f597805bd20 msgr2=0x7f597813f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:11.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 shutdown_connections 2021-11-21T21:55:11.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.854+0000 7f597ee89700 1 -- 172.21.2.116:0/3259934943 wait complete. 2021-11-21T21:55:11.860 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 11 2021-11-21T21:55:11.864 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2021-11-21T21:55:11.921 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-11-21T21:55:11.921 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-11-21T21:55:11.922 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:55:11.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 Processor -- start 2021-11-21T21:55:11.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- start start 2021-11-21T21:55:11.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b1a50 0x7ff28c060b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28c144410 con 0x7ff28c141530 2021-11-21T21:55:11.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28c1446b0 con 0x7ff28c060740 2021-11-21T21:55:11.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c144950 con 0x7ff28c0b1a50 2021-11-21T21:55:11.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29116a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff28c141530 0x7ff28c067200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58184/0 (socket says 172.21.2.116:58184) 2021-11-21T21:55:11.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29116a700 1 -- 172.21.2.116:0/3900422405 learned_addr learned my addr 172.21.2.116:0/3900422405 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:55:11.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020708380 0 0) 0x7ff28c144410 con 0x7ff28c141530 2021-11-21T21:55:11.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff274003260 con 0x7ff28c141530 2021-11-21T21:55:11.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4213328230 0 0) 0x7ff274003260 con 0x7ff28c141530 2021-11-21T21:55:11.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b1a50 msgr2=0x7ff28c060b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b1a50 0x7ff28c060b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 >> v1:172.21.2.117:6789/0 conn(0x7ff28c060740 legacy=0x7ff28c140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c144bf0 con 0x7ff28c141530 2021-11-21T21:55:11.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7ff280001660 con 0x7ff28c141530 2021-11-21T21:55:11.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 >> v1:172.21.2.116:6789/0 conn(0x7ff28c141530 legacy=0x7ff28c067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff274003800 0x7ff274003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b1a50 0x7ff274004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff274004bf0 0x7ff27400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c144950 con 0x7ff274003800 2021-11-21T21:55:11.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740052a0 con 0x7ff274004bf0 2021-11-21T21:55:11.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27400bc40 con 0x7ff28c0b1a50 2021-11-21T21:55:11.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29196b700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff274003800 0x7ff274003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29196b700 1 -- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b1a50 msgr2=0x7ff274004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29196b700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b1a50 0x7ff274004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29196b700 1 -- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff274004bf0 msgr2=0x7ff27400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:11.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29196b700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff274004bf0 0x7ff27400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29196b700 1 -- 172.21.2.116:0/3900422405 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274003260 con 0x7ff274003800 2021-11-21T21:55:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29196b700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff274003800 0x7ff274003c00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7ff288000f70 tx=0x7ff2880079e0).ready entity=mon.0 client_cookie=24493fd878d8e69a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff288007cf0 con 0x7ff274003800 2021-11-21T21:55:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288016030 con 0x7ff274003800 2021-11-21T21:55:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff290969700 1 -- 172.21.2.116:0/3900422405 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff288011940 con 0x7ff274003800 2021-11-21T21:55:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- 172.21.2.116:0/3900422405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff274003800 msgr2=0x7ff274003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 --2- 172.21.2.116:0/3900422405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff274003800 0x7ff274003c00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7ff288000f70 tx=0x7ff2880079e0).stop 2021-11-21T21:55:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff29216c700 1 -- 172.21.2.116:0/3900422405 reap_dead start 2021-11-21T21:55:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- 172.21.2.116:0/3900422405 shutdown_connections 2021-11-21T21:55:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- 172.21.2.116:0/3900422405 >> 172.21.2.116:0/3900422405 conn(0x7ff28c0b7b80 msgr2=0x7ff28c0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- 172.21.2.116:0/3900422405 shutdown_connections 2021-11-21T21:55:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.970+0000 7ff293bcf700 1 -- 172.21.2.116:0/3900422405 wait complete. 2021-11-21T21:55:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 Processor -- start 2021-11-21T21:55:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 -- start start 2021-11-21T21:55:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff274003800 0x7ff28c13d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b2af0 0x7ff28c0ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff28c0ab6c0 0x7ff28c151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c09bcc0 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c14ead0 con 0x7ff274003800 2021-11-21T21:55:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c14ed70 con 0x7ff28c0b2af0 2021-11-21T21:55:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff28c0ab6c0 0x7ff28c151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff28c0ab6c0 0x7ff28c151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46238/0 (socket says 172.21.2.116:46238) 2021-11-21T21:55:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 -- 172.21.2.116:0/3731283177 learned_addr learned my addr 172.21.2.116:0/3731283177 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 -- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b2af0 msgr2=0x7ff28c0ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b2af0 0x7ff28c0ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 -- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff274003800 msgr2=0x7ff28c13d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff274003800 0x7ff28c13d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 -- 172.21.2.116:0/3731283177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c14e7d0 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29216c700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff28c0ab6c0 0x7ff28c151370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7ff280001370 tx=0x7ff284007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff27effd700 1 -- 172.21.2.116:0/3731283177 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff284005a20 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff27effd700 1 -- 172.21.2.116:0/3731283177 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284005b80 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff27effd700 1 -- 172.21.2.116:0/3731283177 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff284007070 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c09b8b0 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c141310 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28c0573d0 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff27effd700 1 -- 172.21.2.116:0/3731283177 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff28401b060 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff27effd700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff26c053540 0x7ff26c055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff27effd700 1 -- 172.21.2.116:0/3731283177 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff28406aed0 con 0x7ff28c0ab6c0 2021-11-21T21:55:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29196b700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff26c053540 0x7ff26c055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.974+0000 7ff29196b700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff26c053540 0x7ff26c055a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff288000f70 tx=0x7ff288017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:11.978+0000 7ff27effd700 1 -- 172.21.2.116:0/3731283177 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff284042030 con 0x7ff28c0ab6c0 2021-11-21T21:55:12.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff28c068740 con 0x7ff28c0ab6c0 2021-11-21T21:55:12.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff27effd700 1 -- 172.21.2.116:0/3731283177 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2480 (secure 0 0 0) 0x7ff284019030 con 0x7ff28c0ab6c0 2021-11-21T21:55:12.090 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:55:12.090 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:54:52.356582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2051796411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2051796411},{"type":"v1","addr":"172.21.2.116:6835","nonce":2051796411}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:55:12.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff26c053540 msgr2=0x7ff26c055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff26c053540 0x7ff26c055a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff288000f70 tx=0x7ff288017000).stop 2021-11-21T21:55:12.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff28c0ab6c0 msgr2=0x7ff28c151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff28c0ab6c0 0x7ff28c151370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7ff280001370 tx=0x7ff284007f90).stop 2021-11-21T21:55:12.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 shutdown_connections 2021-11-21T21:55:12.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff26c053540 0x7ff26c055a00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:12.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff28c0ab6c0 0x7ff28c151370 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:12.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28c0b2af0 0x7ff28c0ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 --2- 172.21.2.116:0/3731283177 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff274003800 0x7ff28c13d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 >> 172.21.2.116:0/3731283177 conn(0x7ff28c0b7b80 msgr2=0x7ff28c0b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:12.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 shutdown_connections 2021-11-21T21:55:12.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.086+0000 7ff293bcf700 1 -- 172.21.2.116:0/3731283177 wait complete. 2021-11-21T21:55:12.094 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 11 2021-11-21T21:55:12.097 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 2021-11-21T21:55:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.146+0000 7fe76c1b6700 1 Processor -- start 2021-11-21T21:55:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.146+0000 7fe76c1b6700 1 -- start start 2021-11-21T21:55:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe7640abce0 0x7fe7641478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe7640ac600 0x7fe7640aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe764147e80 con 0x7fe764144810 2021-11-21T21:55:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640b1490 con 0x7fe7640ac600 2021-11-21T21:55:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640b1fd0 con 0x7fe7640abce0 2021-11-21T21:55:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe7640ac600 0x7fe7640aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe7640ac600 0x7fe7640aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46250/0 (socket says 172.21.2.116:46250) 2021-11-21T21:55:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 -- 172.21.2.116:0/1072819926 learned_addr learned my addr 172.21.2.116:0/1072819926 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 -- 172.21.2.116:0/1072819926 >> v2:172.21.2.117:3300/0 conn(0x7fe7640abce0 msgr2=0x7fe7641478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 --2- 172.21.2.116:0/1072819926 >> v2:172.21.2.117:3300/0 conn(0x7fe7640abce0 0x7fe7641478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 -- 172.21.2.116:0/1072819926 >> v1:172.21.2.117:6789/0 conn(0x7fe764144810 legacy=0x7fe7640b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 -- 172.21.2.116:0/1072819926 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640af5a0 con 0x7fe7640ac600 2021-11-21T21:55:12.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76a753700 1 --2- 172.21.2.116:0/1072819926 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640ac600 0x7fe7640aca50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fe764062500 tx=0x7fe75c009290).ready entity=mon.0 client_cookie=88174e63c3dbea66 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:12.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe768f50700 1 -- 172.21.2.116:0/1072819926 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe75c00b830 con 0x7fe7640ac600 2021-11-21T21:55:12.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe768f50700 1 -- 172.21.2.116:0/1072819926 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00b990 con 0x7fe7640ac600 2021-11-21T21:55:12.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe768f50700 1 -- 172.21.2.116:0/1072819926 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe75c00bb40 con 0x7fe7640ac600 2021-11-21T21:55:12.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1072819926 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640ac600 msgr2=0x7fe7640aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/1072819926 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640ac600 0x7fe7640aca50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fe764062500 tx=0x7fe75c009290).stop 2021-11-21T21:55:12.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1072819926 shutdown_connections 2021-11-21T21:55:12.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/1072819926 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640ac600 0x7fe7640aca50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:12.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/1072819926 >> v2:172.21.2.117:3300/0 conn(0x7fe7640abce0 0x7fe7641478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1072819926 >> 172.21.2.116:0/1072819926 conn(0x7fe76405bd20 msgr2=0x7fe76405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:12.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1072819926 shutdown_connections 2021-11-21T21:55:12.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1072819926 wait complete. 2021-11-21T21:55:12.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 Processor -- start 2021-11-21T21:55:12.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- start start 2021-11-21T21:55:12.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640abce0 0x7fe764146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7640ac600 0x7fe7640b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe764144810 0x7fe7640b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764147630 con 0x7fe7640abce0 2021-11-21T21:55:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764147390 con 0x7fe7640ac600 2021-11-21T21:55:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe764157700 con 0x7fe764144810 2021-11-21T21:55:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640abce0 0x7fe764146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640abce0 0x7fe764146ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46252/0 (socket says 172.21.2.116:46252) 2021-11-21T21:55:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 -- 172.21.2.116:0/711399193 learned_addr learned my addr 172.21.2.116:0/711399193 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 -- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe764144810 msgr2=0x7fe7640b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe764144810 0x7fe7640b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 -- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7640ac600 msgr2=0x7fe7640b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7640ac600 0x7fe7640b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 -- 172.21.2.116:0/711399193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c009530 con 0x7fe7640abce0 2021-11-21T21:55:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe769f52700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640abce0 0x7fe764146ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fe760002f10 tx=0x7fe760006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:12.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe756ffd700 1 -- 172.21.2.116:0/711399193 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe76000f510 con 0x7fe7640abce0 2021-11-21T21:55:12.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe756ffd700 1 -- 172.21.2.116:0/711399193 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76000f670 con 0x7fe7640abce0 2021-11-21T21:55:12.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7641579a0 con 0x7fe7640abce0 2021-11-21T21:55:12.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe756ffd700 1 -- 172.21.2.116:0/711399193 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe76000f7e0 con 0x7fe7640abce0 2021-11-21T21:55:12.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.150+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe764146fc0 con 0x7fe7640abce0 2021-11-21T21:55:12.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe756ffd700 1 -- 172.21.2.116:0/711399193 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe760017020 con 0x7fe7640abce0 2021-11-21T21:55:12.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe756ffd700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe744053660 0x7fe744055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe756ffd700 1 -- 172.21.2.116:0/711399193 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe76006b100 con 0x7fe7640abce0 2021-11-21T21:55:12.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe769751700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe744053660 0x7fe744055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 Processor -- start 2021-11-21T21:55:12.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- start start 2021-11-21T21:55:12.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x13ebec0 0x14222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x14227c0 0x141ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x141efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x1348d50 con 0x1422c40 2021-11-21T21:55:12.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x135c4b0 con 0x14227c0 2021-11-21T21:55:12.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x133cc20 con 0x13ebec0 2021-11-21T21:55:12.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x141efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x141efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46264/0 (socket says 172.21.2.116:46264) 2021-11-21T21:55:12.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 -- 172.21.2.116:0/1767255613 learned_addr learned my addr 172.21.2.116:0/1767255613 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:12.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 -- 172.21.2.116:0/1767255613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x13ebec0 msgr2=0x14222d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:12.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 --2- 172.21.2.116:0/1767255613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x13ebec0 0x14222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 -- 172.21.2.116:0/1767255613 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x14227c0 msgr2=0x141ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:12.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 --2- 172.21.2.116:0/1767255613 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x14227c0 0x141ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 -- 172.21.2.116:0/1767255613 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1394cb0 con 0x1422c40 2021-11-21T21:55:12.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76a753700 1 --2- 172.21.2.116:0/1767255613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x141efc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x1363360 tx=0x7fe75c016000).ready entity=mon.0 client_cookie=e32f594be6652d7e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:12.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe769751700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe744053660 0x7fe744055b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fe758006950 tx=0x7fe758006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:12.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1767255613 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe75c006ce0 con 0x1422c40 2021-11-21T21:55:12.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1767255613 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00bdb0 con 0x1422c40 2021-11-21T21:55:12.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe76c1b6700 1 -- 172.21.2.116:0/1767255613 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe75c020660 con 0x1422c40 2021-11-21T21:55:12.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- 172.21.2.116:0/1767255613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 msgr2=0x141efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1767255613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x141efc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x1363360 tx=0x7fe75c016000).stop 2021-11-21T21:55:12.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- 172.21.2.116:0/1767255613 shutdown_connections 2021-11-21T21:55:12.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1767255613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x141efc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:12.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1767255613 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x14227c0 0x141ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1767255613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x13ebec0 0x14222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- 172.21.2.116:0/1767255613 >> 172.21.2.116:0/1767255613 conn(0x13e9630 msgr2=0x13ea2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:12.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- 172.21.2.116:0/1767255613 shutdown_connections 2021-11-21T21:55:12.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- 172.21.2.116:0/1767255613 wait complete. 2021-11-21T21:55:12.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- start start 2021-11-21T21:55:12.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 Processor -- start 2021-11-21T21:55:12.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x141db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 0x141c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 0x141cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x141e060 con 0x14227c0 2021-11-21T21:55:12.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x141e2f0 con 0x1422c40 2021-11-21T21:55:12.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x1426170 con 0x1427160 2021-11-21T21:55:12.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe7763fa740 10 client.0 ll_register_callbacks cb 0x13502a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:55:12.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe769f52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x141db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe769f52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x141db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46268/0 (socket says 172.21.2.116:46268) 2021-11-21T21:55:12.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.154+0000 7fe769f52700 1 -- 172.21.2.116:0/752983602 learned_addr learned my addr 172.21.2.116:0/752983602 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:12.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769f52700 1 -- 172.21.2.116:0/752983602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 msgr2=0x141cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:12.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769f52700 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 0x141cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769f52700 1 -- 172.21.2.116:0/752983602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 msgr2=0x141c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:12.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769f52700 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 0x141c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769f52700 1 -- 172.21.2.116:0/752983602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7640af5a0 con 0x14227c0 2021-11-21T21:55:12.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769f52700 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x141db70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe76006b310 tx=0x7fe760012000).ready entity=mon.0 client_cookie=787a694c77465970 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:12.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:55:12.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 1 -- 172.21.2.116:0/752983602 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe760044d70 con 0x14227c0 2021-11-21T21:55:12.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 1 -- 172.21.2.116:0/752983602 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe760044ed0 con 0x14227c0 2021-11-21T21:55:12.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1352570 con 0x14227c0 2021-11-21T21:55:12.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 1 -- 172.21.2.116:0/752983602 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe760045090 con 0x14227c0 2021-11-21T21:55:12.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 10 client.5124 fetch_fsmap learned FSMap version 11 2021-11-21T21:55:12.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x134be50 con 0x14227c0 2021-11-21T21:55:12.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 1 -- 172.21.2.116:0/752983602 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1352570 con 0x14227c0 2021-11-21T21:55:12.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 1 -- 172.21.2.116:0/752983602 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 11) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fe760069830 con 0x14227c0 2021-11-21T21:55:12.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 10 client.5124 fetch_fsmap finished waiting for FSMap version 11 2021-11-21T21:55:12.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 10 client.5124 resolve_mds: resolved ID 'c' to GID 4878 2021-11-21T21:55:12.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 20 client.5124 populate_metadata read hostname 'gibba016' 2021-11-21T21:55:12.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x1424f60 0x14326c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 4 client.5124 mds_command: new command op to 4878 tid=0[{"prefix": "get_command_descriptions"}] 2021-11-21T21:55:12.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 --> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x134a8e0 con 0x1424f60 2021-11-21T21:55:12.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769751700 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x1424f60 0x14326c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 10 client.5124 ms_handle_connect on v2:172.21.2.116:6834/2051796411 2021-11-21T21:55:12.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe769751700 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x1424f60 0x14326c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1b474d21df150f76 server_cookie=a235f5a3004e1db2 in_seq=0 out_seq=0 2021-11-21T21:55:12.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 1 -- 172.21.2.116:0/752983602 <== mds.0 v2:172.21.2.116:6834/2051796411 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x134a8e0 con 0x1424f60 2021-11-21T21:55:12.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 10 client.5124 handle_command_reply: tid=0 2021-11-21T21:55:12.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 client.5124 shutdown 2021-11-21T21:55:12.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 msgr2=0x141db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x141db70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe76006b310 tx=0x7fe760012000).stop 2021-11-21T21:55:12.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 shutdown_connections 2021-11-21T21:55:12.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x1424f60 0x14326c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:12.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 0x141cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 0x141c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe740ff9700 0 client.5124 ms_handle_reset on v2:172.21.2.116:6834/2051796411 2021-11-21T21:55:12.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 --2- 172.21.2.116:0/752983602 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x141db70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:12.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 >> 172.21.2.116:0/752983602 conn(0x13ebec0 msgr2=0x141f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:12.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 shutdown_connections 2021-11-21T21:55:12.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 1 -- 172.21.2.116:0/752983602 wait complete. 2021-11-21T21:55:12.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.158+0000 7fe7763fa740 20 client.5124 trim_cache size 0 max 16384 2021-11-21T21:55:12.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 Processor -- start 2021-11-21T21:55:12.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- start start 2021-11-21T21:55:12.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x142a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 0x141c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 0x13e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x1424840 con 0x14227c0 2021-11-21T21:55:12.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x13b4a40 con 0x1422c40 2021-11-21T21:55:12.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x13eb8f0 con 0x1427160 2021-11-21T21:55:12.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x142a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x142a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46278/0 (socket says 172.21.2.116:46278) 2021-11-21T21:55:12.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 -- 172.21.2.116:0/1327815245 learned_addr learned my addr 172.21.2.116:0/1327815245 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:12.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 -- 172.21.2.116:0/1327815245 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 msgr2=0x13e9c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:55:12.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 --2- 172.21.2.116:0/1327815245 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 0x13e9c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 -- 172.21.2.116:0/1327815245 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 msgr2=0x141c260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:55:12.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 --2- 172.21.2.116:0/1327815245 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 0x141c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 -- 172.21.2.116:0/1327815245 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1354750 con 0x14227c0 2021-11-21T21:55:12.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769f52700 1 --2- 172.21.2.116:0/1327815245 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x142a820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x1359b30 tx=0x7fe76006b050).ready entity=mon.0 client_cookie=538ac668a9a62fff server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:12.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe742ffd700 1 -- 172.21.2.116:0/1327815245 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe760045db0 con 0x14227c0 2021-11-21T21:55:12.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe742ffd700 1 -- 172.21.2.116:0/1327815245 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760041e10 con 0x14227c0 2021-11-21T21:55:12.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe742ffd700 1 -- 172.21.2.116:0/1327815245 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7600699d0 con 0x14227c0 2021-11-21T21:55:12.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- 172.21.2.116:0/1327815245 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 msgr2=0x142a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:12.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1327815245 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x142a820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x1359b30 tx=0x7fe76006b050).stop 2021-11-21T21:55:12.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- 172.21.2.116:0/1327815245 shutdown_connections 2021-11-21T21:55:12.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1327815245 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1427160 0x13e9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1327815245 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1422c40 0x141c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1327815245 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x14227c0 0x142a820 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:12.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- 172.21.2.116:0/1327815245 >> 172.21.2.116:0/1327815245 conn(0x13ebec0 msgr2=0x1427640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:12.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- 172.21.2.116:0/1327815245 shutdown_connections 2021-11-21T21:55:12.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- 172.21.2.116:0/1327815245 wait complete. 2021-11-21T21:55:12.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- start start 2021-11-21T21:55:12.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 Processor -- start 2021-11-21T21:55:12.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x14227c0 0x142b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x14823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1427160 0x14828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x141ca50 con 0x1422c40 2021-11-21T21:55:12.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x141d000 con 0x1427160 2021-11-21T21:55:12.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x141f820 con 0x14227c0 2021-11-21T21:55:12.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe7763fa740 10 client.0 ll_register_callbacks cb 0x141e250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:55:12.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.166+0000 7fe769751700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x14823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x14823e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46282/0 (socket says 172.21.2.116:46282) 2021-11-21T21:55:12.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 -- 172.21.2.116:0/1603925514 learned_addr learned my addr 172.21.2.116:0/1603925514 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:12.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 -- 172.21.2.116:0/1603925514 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x14227c0 msgr2=0x142b530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:55:12.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x14227c0 0x142b530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 -- 172.21.2.116:0/1603925514 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1427160 msgr2=0x14828d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:55:12.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1427160 0x14828d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:12.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 -- 172.21.2.116:0/1603925514 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x134be50 con 0x1422c40 2021-11-21T21:55:12.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769751700 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x14823e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fe758013000 tx=0x7fe758011000).ready entity=mon.0 client_cookie=9ebffb57211edc2d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:12.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe741ffb700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:55:12.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe741ffb700 1 -- 172.21.2.116:0/1603925514 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe758013860 con 0x1422c40 2021-11-21T21:55:12.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe741ffb700 1 -- 172.21.2.116:0/1603925514 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe75801a030 con 0x1422c40 2021-11-21T21:55:12.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe75c00bdb0 con 0x1422c40 2021-11-21T21:55:12.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe741ffb700 1 -- 172.21.2.116:0/1603925514 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7580139c0 con 0x1422c40 2021-11-21T21:55:12.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 10 client.5130 fetch_fsmap learned FSMap version 11 2021-11-21T21:55:12.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x1351140 con 0x1422c40 2021-11-21T21:55:12.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe741ffb700 1 -- 172.21.2.116:0/1603925514 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00bdb0 con 0x1422c40 2021-11-21T21:55:12.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe741ffb700 1 -- 172.21.2.116:0/1603925514 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 11) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fe758013b20 con 0x1422c40 2021-11-21T21:55:12.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 10 client.5130 fetch_fsmap finished waiting for FSMap version 11 2021-11-21T21:55:12.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 10 client.5130 resolve_mds: resolved ID 'c' to GID 4878 2021-11-21T21:55:12.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 20 client.5130 populate_metadata read hostname 'gibba016' 2021-11-21T21:55:12.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x142ba20 0x14317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:12.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 4 client.5130 mds_command: new command op to 4878 tid=0[{"prefix": "cache drop"}] 2021-11-21T21:55:12.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 --> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1240590 con 0x142ba20 2021-11-21T21:55:12.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769f52700 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x142ba20 0x14317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:12.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe769f52700 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x142ba20 0x14317c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d3333447c0915df1 server_cookie=75bbaafa90bf5ceb in_seq=0 out_seq=0 2021-11-21T21:55:12.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:12.170+0000 7fe741ffb700 10 client.5130 ms_handle_connect on v2:172.21.2.116:6834/2051796411 2021-11-21T21:55:33.546 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:33.550 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:33.556 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:33.561 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:55:56.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe741ffb700 1 -- 172.21.2.116:0/1603925514 <== mds.0 v2:172.21.2.116:6834/2051796411 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x1240590 con 0x142ba20 2021-11-21T21:55:56.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe741ffb700 10 client.5130 handle_command_reply: tid=0 2021-11-21T21:55:56.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 client.5130 shutdown 2021-11-21T21:55:56.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 msgr2=0x14823e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:56.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x14823e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fe758013000 tx=0x7fe758011000).stop 2021-11-21T21:55:56.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 shutdown_connections 2021-11-21T21:55:56.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:6834/2051796411,v1:172.21.2.116:6835/2051796411] conn(0x142ba20 0x14317c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:56.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1427160 0x14828d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1422c40 0x14823e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:56.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 --2- 172.21.2.116:0/1603925514 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x14227c0 0x142b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 >> 172.21.2.116:0/1603925514 conn(0x13ebec0 msgr2=0x1427640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:56.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe741ffb700 0 client.5130 ms_handle_reset on v2:172.21.2.116:6834/2051796411 2021-11-21T21:55:56.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 shutdown_connections 2021-11-21T21:55:56.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 1 -- 172.21.2.116:0/1603925514 wait complete. 2021-11-21T21:55:56.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.484+0000 7fe7763fa740 20 client.5130 trim_cache size 0 max 16384 2021-11-21T21:55:56.491 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:55:56.492 INFO:teuthology.orchestra.run.gibba016.stdout: "client_recall": { 2021-11-21T21:55:56.492 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": 0, 2021-11-21T21:55:56.492 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "(0) Success", 2021-11-21T21:55:56.493 INFO:teuthology.orchestra.run.gibba016.stdout: "recalled": 920 2021-11-21T21:55:56.493 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.493 INFO:teuthology.orchestra.run.gibba016.stdout: "flush_journal": { 2021-11-21T21:55:56.494 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": 0, 2021-11-21T21:55:56.494 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "" 2021-11-21T21:55:56.494 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.494 INFO:teuthology.orchestra.run.gibba016.stdout: "trim_cache": { 2021-11-21T21:55:56.495 INFO:teuthology.orchestra.run.gibba016.stdout: "trimmed": 900 2021-11-21T21:55:56.495 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.495 INFO:teuthology.orchestra.run.gibba016.stdout: "cache": { 2021-11-21T21:55:56.495 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": { 2021-11-21T21:55:56.495 INFO:teuthology.orchestra.run.gibba016.stdout: "items": 2489, 2021-11-21T21:55:56.496 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes": 431632 2021-11-21T21:55:56.496 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:56.496 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.497 INFO:teuthology.orchestra.run.gibba016.stdout: "duration": 44.313649101999999 2021-11-21T21:55:56.497 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:55:56.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe744053660 msgr2=0x7fe744055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:56.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe744053660 0x7fe744055b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fe758006950 tx=0x7fe758006a80).stop 2021-11-21T21:55:56.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640abce0 msgr2=0x7fe764146ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:56.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640abce0 0x7fe764146ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fe760002f10 tx=0x7fe760006d70).stop 2021-11-21T21:55:56.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 shutdown_connections 2021-11-21T21:55:56.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe764144810 0x7fe7640b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7640ac600 0x7fe7640b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe744053660 0x7fe744055b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:56.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 --2- 172.21.2.116:0/711399193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7640abce0 0x7fe764146ad0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:56.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 >> 172.21.2.116:0/711399193 conn(0x7fe76405bd20 msgr2=0x7fe76405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:56.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 shutdown_connections 2021-11-21T21:55:56.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.488+0000 7fe76c1b6700 1 -- 172.21.2.116:0/711399193 wait complete. 2021-11-21T21:55:56.502 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:55:56.503 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:55:56.512 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:55:56.526 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:55:56.541 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:55:56.556 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:55:56.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:55:56.557 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:55:56.557 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:55:56.562 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:55:56.562 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:55:56.563 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:55:56.563 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:55:56.564 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:55:56.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14058]: fuse finished with error 0 and tester_r 0 2021-11-21T21:55:56.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:55:56.572 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:55:56.573 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:55:56.617 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T21:55:56.617 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:55:56.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:55:56.656 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:55:56.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:55:56.663 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:55:56.663 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:55:56.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:55:56.680 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:55:56.680 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:55:56.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:55:56.683 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:55:56.732 INFO:teuthology.orchestra.run.gibba036.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:55:56.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:55:56.735 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:55:56.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 Processor -- start 2021-11-21T21:55:56.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- start start 2021-11-21T21:55:56.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f51f40a6aa0 0x7f51f414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:56.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 0x7f51f41470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:56.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f40a6e80 con 0x7f51f41475d0 2021-11-21T21:55:56.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f40a8bc0 con 0x7f51f40a6aa0 2021-11-21T21:55:56.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f414a7b0 con 0x7f51f40a86b0 2021-11-21T21:55:56.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f51f40a6aa0 0x7f51f414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:56.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f51f40a6aa0 0x7f51f414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46290/0 (socket says 172.21.2.116:46290) 2021-11-21T21:55:56.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 -- 172.21.2.116:0/3435364678 learned_addr learned my addr 172.21.2.116:0/3435364678 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:56.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 -- 172.21.2.116:0/3435364678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 msgr2=0x7f51f41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:56.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 --2- 172.21.2.116:0/3435364678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 0x7f51f41470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 -- 172.21.2.116:0/3435364678 >> v1:172.21.2.117:6789/0 conn(0x7f51f41475d0 legacy=0x7f51f41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:56.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 -- 172.21.2.116:0/3435364678 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f414aa10 con 0x7f51f40a6aa0 2021-11-21T21:55:56.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fa129700 1 --2- 172.21.2.116:0/3435364678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f40a6aa0 0x7f51f414b150 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f51f4061ab0 tx=0x7f51e8006eb0).ready entity=mon.0 client_cookie=73579ff9ae32a92d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:56.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51f9127700 1 -- 172.21.2.116:0/3435364678 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51e800d570 con 0x7f51f40a6aa0 2021-11-21T21:55:56.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51f9127700 1 -- 172.21.2.116:0/3435364678 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e8000e90 con 0x7f51f40a6aa0 2021-11-21T21:55:56.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51f9127700 1 -- 172.21.2.116:0/3435364678 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51e800d8c0 con 0x7f51f40a6aa0 2021-11-21T21:55:56.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- 172.21.2.116:0/3435364678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f40a6aa0 msgr2=0x7f51f414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:56.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 --2- 172.21.2.116:0/3435364678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f40a6aa0 0x7f51f414b150 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f51f4061ab0 tx=0x7f51e8006eb0).stop 2021-11-21T21:55:56.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- 172.21.2.116:0/3435364678 shutdown_connections 2021-11-21T21:55:56.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 --2- 172.21.2.116:0/3435364678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 0x7f51f41470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 --2- 172.21.2.116:0/3435364678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f40a6aa0 0x7f51f414b150 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:56.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- 172.21.2.116:0/3435364678 >> 172.21.2.116:0/3435364678 conn(0x7f51f405bd20 msgr2=0x7f51f405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:56.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- 172.21.2.116:0/3435364678 shutdown_connections 2021-11-21T21:55:56.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- 172.21.2.116:0/3435364678 wait complete. 2021-11-21T21:55:56.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 Processor -- start 2021-11-21T21:55:56.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.860+0000 7f51fc38d700 1 -- start start 2021-11-21T21:55:56.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f40a6aa0 0x7f51f4142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:56.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 0x7f51f4156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:56.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f41475d0 0x7f51f4064690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:56.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41565a0 con 0x7f51f41475d0 2021-11-21T21:55:56.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4065240 con 0x7f51f40a6aa0 2021-11-21T21:55:56.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4064fa0 con 0x7f51f40a86b0 2021-11-21T21:55:56.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f41475d0 0x7f51f4064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:56.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f41475d0 0x7f51f4064690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46300/0 (socket says 172.21.2.116:46300) 2021-11-21T21:55:56.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 -- 172.21.2.116:0/237963981 learned_addr learned my addr 172.21.2.116:0/237963981 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:56.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 -- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 msgr2=0x7f51f4156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:56.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 0x7f51f4156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 -- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f40a6aa0 msgr2=0x7f51f4142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:56.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f40a6aa0 0x7f51f4142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:56.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 -- 172.21.2.116:0/237963981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e800b1e0 con 0x7f51f41475d0 2021-11-21T21:55:56.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa92a700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f41475d0 0x7f51f4064690 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f51ec003290 tx=0x7f51ec0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:56.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51e77fe700 1 -- 172.21.2.116:0/237963981 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ec00f510 con 0x7f51f41475d0 2021-11-21T21:55:56.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51e77fe700 1 -- 172.21.2.116:0/237963981 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00f670 con 0x7f51f41475d0 2021-11-21T21:55:56.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51e77fe700 1 -- 172.21.2.116:0/237963981 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ec00f7e0 con 0x7f51f41475d0 2021-11-21T21:55:56.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f4156790 con 0x7f51f41475d0 2021-11-21T21:55:56.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f4064b80 con 0x7f51f41475d0 2021-11-21T21:55:56.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4156920 con 0x7f51f41475d0 2021-11-21T21:55:56.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51e77fe700 1 -- 172.21.2.116:0/237963981 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f51ec00f070 con 0x7f51f41475d0 2021-11-21T21:55:56.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51e77fe700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51d40536a0 0x7f51d4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:56.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51e77fe700 1 -- 172.21.2.116:0/237963981 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f51ec06b4d0 con 0x7f51f41475d0 2021-11-21T21:55:56.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa129700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51d40536a0 0x7f51d4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:56.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.864+0000 7f51fa129700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51d40536a0 0x7f51d4055b60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f51f4061ab0 tx=0x7f51e8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:56.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.868+0000 7f51e77fe700 1 -- 172.21.2.116:0/237963981 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51ec042630 con 0x7f51f41475d0 2021-11-21T21:55:56.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.948+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f51f40573d0 con 0x7f51f41475d0 2021-11-21T21:55:56.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51e77fe700 1 -- 172.21.2.116:0/237963981 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v33) v1 ==== 81+0+33645 (secure 0 0 0) 0x7f51ec045c50 con 0x7f51f41475d0 2021-11-21T21:55:56.952 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:55:56.952 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:55:56.953 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 33, 2021-11-21T21:55:56.953 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:55:56.953 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:55:56.953 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:54:53.370027+0000", 2021-11-21T21:55:56.954 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:55:56.954 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:55:56.954 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:55:56.954 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:55:56.954 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:55:56.955 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:55:56.955 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:55:56.955 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:55:56.955 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:55:56.956 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:55:56.956 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:55:56.956 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:55:56.956 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:55:56.956 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:55:56.957 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:55:56.957 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 6, 2021-11-21T21:55:56.957 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:55:56.957 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:55:56.957 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:55:56.958 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:55:56.958 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:55:56.958 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:56.958 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:55:56.959 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:55:56.960 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:55:56.960 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:55:56.960 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:55:56.960 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:55:56.961 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:55:56.961 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:55:56.961 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:55:56.961 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:55:56.961 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:55:56.962 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:55:56.962 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:55:56.962 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:55:56.962 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:55:56.963 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:55:56.963 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:55:56.963 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:55:56.963 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:55:56.963 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:55:56.964 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:55:56.964 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:55:56.964 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:55:56.964 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.965 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:55:56.965 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:55:56.965 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:55:56.966 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:55:56.966 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:55:56.967 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:55:56.967 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:55:56.967 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:55:56.967 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:55:56.967 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:55:56.968 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:55:56.968 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:55:56.968 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:55:56.968 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:55:56.969 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:55:56.969 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:55:56.969 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:55:56.969 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:55:56.970 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:55:56.970 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:55:56.970 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:55:56.970 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:55:56.970 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:55:56.971 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:55:56.971 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:55:56.971 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:55:56.971 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:55:56.972 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.972 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:55:56.972 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:55:56.972 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:55:56.972 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:55:56.973 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:55:56.973 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:55:56.973 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:55:56.973 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:55:56.974 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:55:56.974 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:55:56.974 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:55:56.974 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:55:56.974 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:55:56.975 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.975 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:55:56.975 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:55:56.975 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:56.976 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.976 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:56.977 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:55:56.977 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:55:56.977 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:55:56.978 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:55:56.978 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:55:56.978 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:55:56.979 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:55:56.979 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:55:56.979 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:55:56.979 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:55:56.980 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:55:56.980 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:55:56.980 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:55:56.980 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:55:56.980 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:55:56.981 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:55:56.981 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:55:56.981 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:55:56.981 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:55:56.982 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:55:56.982 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:55:56.982 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:55:56.982 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:55:56.983 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.983 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:55:56.983 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:55:56.983 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:55:56.984 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:55:56.984 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:55:56.984 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:55:56.984 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:55:56.984 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:55:56.985 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:55:56.985 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:55:56.985 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:55:56.985 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:55:56.985 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:55:56.986 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:55:56.986 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:55:56.987 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:55:56.987 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:55:56.987 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:55:56.988 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:55:56.988 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:55:56.988 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:55:56.988 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:55:56.989 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:55:56.989 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:55:56.989 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:55:56.989 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:55:56.990 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:55:56.990 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.990 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:55:56.990 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:55:56.991 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:55:56.991 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:55:56.991 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:55:56.991 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:55:56.991 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:55:56.992 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:55:56.992 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:55:56.992 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:55:56.993 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:55:56.993 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:55:56.993 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:55:56.994 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:55:56.994 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:56.994 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:56.994 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:56.995 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 5, 2021-11-21T21:55:56.995 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T21:55:56.995 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:49.278376+0000", 2021-11-21T21:55:56.995 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:55:56.995 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:55:56.996 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:55:56.996 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:55:56.996 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:55:56.996 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:55:56.997 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:55:56.997 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:55:56.997 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:55:56.997 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:55:56.997 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:55:56.998 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:55:56.998 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:55:56.998 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:55:56.998 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:55:56.999 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:55:56.999 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:55:56.999 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:55:56.999 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:55:57.000 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:55:57.000 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.000 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "32", 2021-11-21T21:55:57.000 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:55:57.000 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:55:57.001 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:55:57.001 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:55:57.001 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:55:57.001 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:55:57.002 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:55:57.002 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:55:57.002 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:55:57.002 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:55:57.002 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:55:57.003 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:55:57.003 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:55:57.004 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:55:57.004 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:55:57.004 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:55:57.004 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:55:57.005 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:55:57.005 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:55:57.005 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:55:57.005 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:55:57.006 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:55:57.006 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:55:57.006 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:55:57.007 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:55:57.007 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:55:57.007 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.008 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:55:57.008 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:55:57.008 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:55:57.008 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:55:57.009 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:55:57.009 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:55:57.009 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:55:57.009 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:55:57.009 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:55:57.010 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:55:57.010 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:55:57.010 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:55:57.010 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T21:55:57.011 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T21:55:57.011 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T21:55:57.011 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.011 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:55:57.012 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:55:57.012 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T21:55:57.012 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.012 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.012 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.013 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.013 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 6, 2021-11-21T21:55:57.013 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T21:55:57.014 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:50.289265+0000", 2021-11-21T21:55:57.014 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:55:57.014 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:55:57.015 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:55:57.015 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:55:57.015 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:55:57.015 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:55:57.015 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:55:57.016 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:55:57.016 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:55:57.016 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:55:57.016 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:55:57.017 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:55:57.017 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:55:57.017 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:55:57.017 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:55:57.017 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:55:57.018 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:55:57.018 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:55:57.018 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:55:57.018 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:55:57.019 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "33", 2021-11-21T21:55:57.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:55:57.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:55:57.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:55:57.020 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:55:57.020 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:55:57.020 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:55:57.020 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:55:57.021 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:55:57.021 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:55:57.021 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:55:57.022 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:55:57.022 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:55:57.022 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:55:57.022 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:55:57.023 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:55:57.023 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:55:57.023 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:55:57.023 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:55:57.023 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:55:57.024 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:55:57.024 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:55:57.025 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:55:57.026 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:55:57.027 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:55:57.027 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:55:57.028 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:55:57.028 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.029 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:55:57.029 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:55:57.029 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:55:57.029 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:55:57.030 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:55:57.030 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:55:57.030 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:55:57.031 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:55:57.031 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:55:57.031 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:55:57.032 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:55:57.032 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:55:57.032 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:55:57.032 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:55:57.033 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T21:55:57.033 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.033 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.034 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.034 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:55:57.034 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:55:57.035 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.035 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:55:57.035 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:55:57.036 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.036 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.036 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.037 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.037 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.037 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.038 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.038 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:55:57.039 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.039 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.040 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.040 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.040 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.041 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.041 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:55:57.042 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.042 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.043 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.043 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.043 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:55:57.044 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.044 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.045 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.045 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.046 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.046 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.047 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.047 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.048 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:55:57.048 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.048 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.049 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.049 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.050 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:55:57.050 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.050 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.051 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.051 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.052 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.052 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.053 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.053 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.053 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:55:57.054 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.054 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.055 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.055 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.056 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:55:57.056 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.056 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.057 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.057 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.058 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.058 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.059 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.059 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.059 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:55:57.060 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.060 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.061 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.061 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.061 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:55:57.062 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:55:57.062 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.063 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.063 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.064 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:55:57.064 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:55:57.065 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:55:57.065 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:55:57.065 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.066 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.066 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.067 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.067 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.067 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.068 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:55:57.068 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:55:57.069 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.069 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.070 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.070 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.070 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.071 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.071 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.071 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 31, 2021-11-21T21:55:57.072 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.072 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.072 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.072 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.073 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.073 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.073 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:55:57.073 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.074 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.074 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.074 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.074 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:55:57.075 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.075 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.075 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.075 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.076 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.076 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.076 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.076 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.076 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:55:57.077 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.077 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.077 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.077 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.078 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:55:57.078 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.078 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.078 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.078 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.079 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.079 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.079 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.079 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.080 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:55:57.080 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.080 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.080 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.080 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.081 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:55:57.081 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.081 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.081 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.082 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.082 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.082 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.083 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.083 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.083 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:55:57.083 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.083 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.084 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.084 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.084 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:55:57.084 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:55:57.084 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.085 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.085 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.085 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:55:57.085 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:55:57.086 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:55:57.086 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:55:57.086 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.086 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.087 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.087 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.087 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.087 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.087 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:55:57.088 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:55:57.088 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.088 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.088 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.089 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.089 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.089 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.089 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.089 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 27, 2021-11-21T21:55:57.090 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.090 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.090 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.090 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.091 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.091 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.091 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:55:57.091 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.091 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.092 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.092 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.092 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:55:57.092 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.093 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.093 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.093 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.093 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.093 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.094 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.094 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.094 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:55:57.094 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.095 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.095 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.095 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.095 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:55:57.096 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.096 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.096 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.096 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.096 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.097 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.097 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.097 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.097 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:55:57.097 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.098 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.098 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.098 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.098 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:55:57.099 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.099 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.099 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.099 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.099 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.100 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.100 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.100 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.100 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:55:57.101 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.101 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.101 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.101 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.101 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:55:57.102 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:55:57.102 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.102 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.102 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.103 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:55:57.103 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:55:57.103 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:55:57.103 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:55:57.103 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.104 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.104 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.104 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.104 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.104 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.105 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:55:57.105 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:55:57.105 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.105 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.106 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.106 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.106 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.106 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.106 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.107 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:55:57.107 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.107 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.107 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.108 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.108 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.108 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.108 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:55:57.108 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.109 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.109 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.109 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.109 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:55:57.109 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.110 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.110 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.110 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.110 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.111 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.111 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.111 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.111 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:55:57.112 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.112 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.112 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.112 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.112 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:55:57.113 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.113 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.113 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.113 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.114 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.114 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.114 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.114 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.114 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:55:57.115 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.115 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.115 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.115 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.116 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:55:57.116 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.116 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.116 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.116 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.117 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.117 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.117 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.117 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.117 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:55:57.118 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.118 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.118 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.118 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.119 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:55:57.119 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:55:57.119 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.119 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.119 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.120 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:55:57.120 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:55:57.120 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:55:57.121 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:55:57.121 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.121 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.121 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.121 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.122 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.122 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.122 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:55:57.122 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:55:57.123 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.123 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.123 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.123 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.123 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.124 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.124 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.124 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:55:57.124 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.124 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.125 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.125 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.125 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.125 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.126 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:55:57.126 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.126 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.126 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.126 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.127 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:55:57.127 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.127 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.127 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.127 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.128 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.128 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.128 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.128 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.129 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:55:57.129 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.129 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.129 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.129 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.130 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:55:57.130 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.130 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.130 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.131 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.131 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.131 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.131 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.131 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.132 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:55:57.132 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.132 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.132 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.132 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.133 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:55:57.133 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.133 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.133 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.134 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.134 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.134 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.134 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.134 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.135 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:55:57.135 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.135 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.135 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.136 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.136 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:55:57.136 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:55:57.136 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.137 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.137 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.137 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:55:57.137 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:55:57.138 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:55:57.138 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:55:57.138 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.138 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.138 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.139 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.139 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.139 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.139 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:55:57.140 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:55:57.140 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.140 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.140 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.140 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.141 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.141 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.141 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.141 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:55:57.142 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.142 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.142 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.142 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.142 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.143 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.143 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:55:57.143 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.143 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.143 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.144 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.144 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:55:57.144 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.144 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.145 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.145 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.145 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.145 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.145 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.146 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.146 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:55:57.146 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.146 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.147 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.147 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.147 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:55:57.147 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.147 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.148 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.148 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.148 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.148 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.149 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.149 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.149 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:55:57.149 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.150 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.150 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.150 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.150 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:55:57.151 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.151 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.151 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.151 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.151 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.152 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.152 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.152 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.152 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:55:57.153 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.153 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.153 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.153 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.153 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:55:57.154 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:55:57.154 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.154 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.154 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.154 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:55:57.155 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:55:57.155 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:55:57.155 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:55:57.155 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.156 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.156 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.156 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.156 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.156 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.157 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:55:57.157 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:55:57.157 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.157 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.157 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.158 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.158 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.158 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.158 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.159 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 27, 2021-11-21T21:55:57.159 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.159 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.159 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.159 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.160 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.160 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.160 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:55:57.160 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.161 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.161 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.161 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.161 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:55:57.161 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.162 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.162 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.162 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.162 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.162 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.163 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.163 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.163 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:55:57.163 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.164 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.164 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.164 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.164 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:55:57.164 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.165 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.165 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.165 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.165 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.165 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.166 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.166 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.166 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:55:57.166 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.167 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.167 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.167 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.167 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:55:57.167 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.168 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.168 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.168 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.168 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.169 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.169 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.169 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.169 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:55:57.169 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.170 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.170 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.170 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.170 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:55:57.170 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:55:57.171 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.171 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.171 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.171 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:55:57.172 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:55:57.172 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:55:57.172 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:55:57.172 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.172 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.173 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.173 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.173 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.173 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.173 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:55:57.174 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:55:57.174 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:55:57.174 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:55:57.174 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:55:57.175 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:55:57.175 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:55:57.175 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:55:57.175 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:55:57.175 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:55:57.176 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:55:57.176 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:55:57.176 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:55:57.176 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.177 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.177 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.177 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:55:57.177 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.177 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.178 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.178 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.178 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:55:57.178 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.178 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.179 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.179 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.179 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:55:57.179 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.180 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.180 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.180 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:55:57.180 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.180 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.181 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.181 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.181 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:55:57.181 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.182 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.182 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.182 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.182 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:55:57.182 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.183 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.183 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.183 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:55:57.183 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.184 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.184 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.184 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.184 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:55:57.184 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.185 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.185 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.185 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.185 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:55:57.186 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:55:57.186 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.186 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:55:57.186 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:55:57.186 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.187 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.187 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.187 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:55:57.187 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:55:57.188 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:55:57.188 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.188 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.188 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.188 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:55:57.189 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:55:57.189 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:55:57.189 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:55:57.189 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:55:57.189 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:55:57.190 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:55:57.190 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:55:57.190 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.190 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:55:57.191 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:55:57.191 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.191 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:55:57.191 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.191 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.192 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.192 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.192 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.192 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:55:57.193 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.193 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.193 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.193 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:55:57.193 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.194 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.194 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.194 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.194 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.195 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:55:57.195 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.195 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.195 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.195 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:55:57.196 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.196 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.196 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.196 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.196 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.197 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:55:57.197 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.197 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.197 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.198 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:55:57.198 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.198 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.198 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.198 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.199 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.199 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:55:57.199 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.199 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.200 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.200 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:55:57.200 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.200 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.200 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.201 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.201 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.201 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:55:57.201 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.202 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.202 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.202 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:55:57.202 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.202 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.203 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.203 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.203 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.203 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:55:57.204 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.204 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.204 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.204 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:55:57.204 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.205 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.205 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.205 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.205 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.205 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:55:57.206 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.206 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.206 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:55:57.206 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:55:57.207 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:55:57.207 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:55:57.207 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:55:57.207 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:55:57.208 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:55:57.208 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:55:57.208 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:55:57.208 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.208 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:55:57.209 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:55:57.209 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:55:57.209 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:55:57.209 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:55:57.210 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T21:55:57.210 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:55:57.210 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:55:57.210 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:55:57.210 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:55:57.211 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:55:57.211 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:55:57.211 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:55:57.211 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:55:57.211 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:55:57.212 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:55:57.212 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:55:57.212 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:55:57.212 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:55:57.213 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:55:57.213 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:55:57.213 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:55:57.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51d40536a0 msgr2=0x7f51d4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51d40536a0 0x7f51d4055b60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f51f4061ab0 tx=0x7f51e8015000).stop 2021-11-21T21:55:57.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f41475d0 msgr2=0x7f51f4064690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f41475d0 0x7f51f4064690 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f51ec003290 tx=0x7f51ec0090f0).stop 2021-11-21T21:55:57.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 shutdown_connections 2021-11-21T21:55:57.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51d40536a0 0x7f51d4055b60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:57.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f41475d0 0x7f51f4064690 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:57.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f40a86b0 0x7f51f4156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 --2- 172.21.2.116:0/237963981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f40a6aa0 0x7f51f4142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 >> 172.21.2.116:0/237963981 conn(0x7f51f405bd20 msgr2=0x7f51f4148850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:57.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 shutdown_connections 2021-11-21T21:55:57.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:56.952+0000 7f51fc38d700 1 -- 172.21.2.116:0/237963981 wait complete. 2021-11-21T21:55:57.218 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:55:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7253fff700 1 Processor -- start 2021-11-21T21:55:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7253fff700 1 -- start start 2021-11-21T21:55:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7253fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72540b20a0 0x7f72540b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7253fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7254147a70 0x7f7254143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7253fff700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7254062330 con 0x7f72540b30f0 2021-11-21T21:55:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7253fff700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540b2620 con 0x7f7254147a70 2021-11-21T21:55:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7253fff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7254147330 con 0x7f72540b20a0 2021-11-21T21:55:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7252ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f72540b30f0 0x7f725405f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58260/0 (socket says 172.21.2.116:58260) 2021-11-21T21:55:57.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7252ffd700 1 -- 172.21.2.116:0/2435937826 learned_addr learned my addr 172.21.2.116:0/2435937826 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:55:57.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640734822 0 0) 0x7f7254062330 con 0x7f72540b30f0 2021-11-21T21:55:57.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.272+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7238003260 con 0x7f72540b30f0 2021-11-21T21:55:57.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1261241154 0 0) 0x7f7238003260 con 0x7f72540b30f0 2021-11-21T21:55:57.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72540b20a0 msgr2=0x7f72540b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:57.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72540b20a0 0x7f72540b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 >> v2:172.21.2.117:3300/0 conn(0x7f7254147a70 msgr2=0x7f7254143cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:57.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 --2- 172.21.2.116:0/2435937826 >> v2:172.21.2.117:3300/0 conn(0x7f7254147a70 0x7f7254143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254147100 con 0x7f72540b30f0 2021-11-21T21:55:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f72480017e0 con 0x7f72540b30f0 2021-11-21T21:55:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 >> v1:172.21.2.116:6789/0 conn(0x7f72540b30f0 legacy=0x7f725405f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72380037d0 0x7f7238003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f7238004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72540b20a0 0x7f723800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540b2620 con 0x7f7238004500 2021-11-21T21:55:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7254147330 con 0x7f72380037d0 2021-11-21T21:55:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7238005330 con 0x7f72540b20a0 2021-11-21T21:55:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72527fc700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f7238004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72527fc700 1 -- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72540b20a0 msgr2=0x7f723800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72527fc700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f72540b20a0 0x7f723800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72527fc700 1 -- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72380037d0 msgr2=0x7f7238003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72527fc700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f72380037d0 0x7f7238003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72527fc700 1 -- 172.21.2.116:0/2435937826 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7238003260 con 0x7f7238004500 2021-11-21T21:55:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72527fc700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f7238004980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f7244002f70 tx=0x7f7244006ca0).ready entity=mon.0 client_cookie=96f09adc40cdff9c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7244012410 con 0x7f7238004500 2021-11-21T21:55:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7244012570 con 0x7f7238004500 2021-11-21T21:55:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7251ffb700 1 -- 172.21.2.116:0/2435937826 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72440128d0 con 0x7f7238004500 2021-11-21T21:55:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- 172.21.2.116:0/2435937826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 msgr2=0x7f7238004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 --2- 172.21.2.116:0/2435937826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f7238004980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f7244002f70 tx=0x7f7244006ca0).stop 2021-11-21T21:55:57.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f72537fe700 1 -- 172.21.2.116:0/2435937826 reap_dead start 2021-11-21T21:55:57.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- 172.21.2.116:0/2435937826 shutdown_connections 2021-11-21T21:55:57.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- 172.21.2.116:0/2435937826 >> 172.21.2.116:0/2435937826 conn(0x7f72540b44c0 msgr2=0x7f725405bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:57.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- 172.21.2.116:0/2435937826 shutdown_connections 2021-11-21T21:55:57.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- 172.21.2.116:0/2435937826 wait complete. 2021-11-21T21:55:57.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 Processor -- start 2021-11-21T21:55:57.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- start start 2021-11-21T21:55:57.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f725414cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7254098580 0x7f725409b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7254098a00 0x7f725409b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540ab1a0 con 0x7f7238004500 2021-11-21T21:55:57.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540ab440 con 0x7f7254098580 2021-11-21T21:55:57.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540ab6e0 con 0x7f7254098a00 2021-11-21T21:55:57.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f725414cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:57.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f725414cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46316/0 (socket says 172.21.2.116:46316) 2021-11-21T21:55:57.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 -- 172.21.2.116:0/1706250765 learned_addr learned my addr 172.21.2.116:0/1706250765 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:57.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 -- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7254098a00 msgr2=0x7f725409b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7254098a00 0x7f725409b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 -- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7254098580 msgr2=0x7f725409b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7254098580 0x7f725409b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 -- 172.21.2.116:0/1706250765 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254099540 con 0x7f7238004500 2021-11-21T21:55:57.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7252ffd700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f725414cc10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f72480014f0 tx=0x7f7254146ff0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:57.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7243fff700 1 -- 172.21.2.116:0/1706250765 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72480054e0 con 0x7f7238004500 2021-11-21T21:55:57.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7243fff700 1 -- 172.21.2.116:0/1706250765 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7248005640 con 0x7f7238004500 2021-11-21T21:55:57.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.276+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725409bcd0 con 0x7f7238004500 2021-11-21T21:55:57.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f7243fff700 1 -- 172.21.2.116:0/1706250765 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72480059a0 con 0x7f7238004500 2021-11-21T21:55:57.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7254156710 con 0x7f7238004500 2021-11-21T21:55:57.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72540573d0 con 0x7f7238004500 2021-11-21T21:55:57.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f7243fff700 1 -- 172.21.2.116:0/1706250765 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f724801a060 con 0x7f7238004500 2021-11-21T21:55:57.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f7243fff700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7230053610 0x7f7230055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f7243fff700 1 -- 172.21.2.116:0/1706250765 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f724806b4a0 con 0x7f7238004500 2021-11-21T21:55:57.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f72527fc700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7230053610 0x7f7230055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:57.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f72527fc700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7230053610 0x7f7230055ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f7244002f70 tx=0x7f7244015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:57.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.280+0000 7f7243fff700 1 -- 172.21.2.116:0/1706250765 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7248042610 con 0x7f7238004500 2021-11-21T21:55:57.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.388+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7254099b80 con 0x7f7238004500 2021-11-21T21:55:57.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7243fff700 1 -- 172.21.2.116:0/1706250765 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2480 (secure 0 0 0) 0x7f7248018070 con 0x7f7238004500 2021-11-21T21:55:57.393 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:55:57.393 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:54:52.356582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2051796411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2051796411},{"type":"v1","addr":"172.21.2.116:6835","nonce":2051796411}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:55:57.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7230053610 msgr2=0x7f7230055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7230053610 0x7f7230055ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f7244002f70 tx=0x7f7244015000).stop 2021-11-21T21:55:57.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 msgr2=0x7f725414cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f725414cc10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f72480014f0 tx=0x7f7254146ff0).stop 2021-11-21T21:55:57.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 shutdown_connections 2021-11-21T21:55:57.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7254098a00 0x7f725409b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7254098580 0x7f725409b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7230053610 0x7f7230055ad0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:57.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 --2- 172.21.2.116:0/1706250765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7238004500 0x7f725414cc10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:57.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 >> 172.21.2.116:0/1706250765 conn(0x7f72540b44c0 msgr2=0x7f725405c660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:57.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 shutdown_connections 2021-11-21T21:55:57.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.392+0000 7f7253fff700 1 -- 172.21.2.116:0/1706250765 wait complete. 2021-11-21T21:55:57.397 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 11 2021-11-21T21:55:57.401 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T21:55:57.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 Processor -- start 2021-11-21T21:55:57.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- start start 2021-11-21T21:55:57.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7effc80abce0 0x7effc81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc8147e80 con 0x7effc80ac600 2021-11-21T21:55:57.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc80af5a0 con 0x7effc8144810 2021-11-21T21:55:57.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc80b1490 con 0x7effc80abce0 2021-11-21T21:55:57.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcdbf8700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7effc80ac600 0x7effc80aca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58284/0 (socket says 172.21.2.116:58284) 2021-11-21T21:55:57.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcdbf8700 1 -- 172.21.2.116:0/996725699 learned_addr learned my addr 172.21.2.116:0/996725699 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:55:57.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effccbf6700 1 --2- 172.21.2.116:0/996725699 >> v2:172.21.2.116:3300/0 conn(0x7effc80abce0 0x7effc81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:57.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effccbf6700 1 -- 172.21.2.116:0/996725699 >> v1:172.21.2.117:6789/0 conn(0x7effc8144810 legacy=0x7effc80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:55:57.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effccbf6700 1 -- 172.21.2.116:0/996725699 >> v1:172.21.2.116:6789/0 conn(0x7effc80ac600 legacy=0x7effc80aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effccbf6700 1 -- 172.21.2.116:0/996725699 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc80b1fd0 con 0x7effc80abce0 2021-11-21T21:55:57.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effccbf6700 1 --2- 172.21.2.116:0/996725699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc81478c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7effc80b4e00 tx=0x7effb8006df0).ready entity=mon.0 client_cookie=fee76fd49d242a2d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:57.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effb7fff700 1 -- 172.21.2.116:0/996725699 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb800d5c0 con 0x7effc80abce0 2021-11-21T21:55:57.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effb7fff700 1 -- 172.21.2.116:0/996725699 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8000e90 con 0x7effc80abce0 2021-11-21T21:55:57.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effb7fff700 1 -- 172.21.2.116:0/996725699 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb800d910 con 0x7effc80abce0 2021-11-21T21:55:57.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- 172.21.2.116:0/996725699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 msgr2=0x7effc81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:57.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 --2- 172.21.2.116:0/996725699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc81478c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7effc80b4e00 tx=0x7effb8006df0).stop 2021-11-21T21:55:57.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- 172.21.2.116:0/996725699 shutdown_connections 2021-11-21T21:55:57.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 --2- 172.21.2.116:0/996725699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc81478c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:57.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- 172.21.2.116:0/996725699 >> 172.21.2.116:0/996725699 conn(0x7effc805bd20 msgr2=0x7effc805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:57.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- 172.21.2.116:0/996725699 shutdown_connections 2021-11-21T21:55:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.452+0000 7effcf65b700 1 -- 172.21.2.116:0/996725699 wait complete. 2021-11-21T21:55:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 Processor -- start 2021-11-21T21:55:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 -- start start 2021-11-21T21:55:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc8145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7effc80ac600 0x7effc8146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7effc8144810 0x7effc80b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8146ff0 con 0x7effc80abce0 2021-11-21T21:55:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8157890 con 0x7effc80ac600 2021-11-21T21:55:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc8145a80 con 0x7effc8144810 2021-11-21T21:55:57.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc8145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:57.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc8145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46334/0 (socket says 172.21.2.116:46334) 2021-11-21T21:55:57.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 -- 172.21.2.116:0/380876393 learned_addr learned my addr 172.21.2.116:0/380876393 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:57.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 -- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7effc8144810 msgr2=0x7effc80b2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:55:57.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7effc8144810 0x7effc80b2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 -- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7effc80ac600 msgr2=0x7effc8146b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:55:57.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7effc80ac600 0x7effc8146b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:57.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 -- 172.21.2.116:0/380876393 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb800b180 con 0x7effc80abce0 2021-11-21T21:55:57.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcd3f7700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc8145340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7effc4002ef0 tx=0x7effc4006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:57.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effb67fc700 1 -- 172.21.2.116:0/380876393 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc4009ea0 con 0x7effc80abce0 2021-11-21T21:55:57.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effb67fc700 1 -- 172.21.2.116:0/380876393 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4002320 con 0x7effc80abce0 2021-11-21T21:55:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc8145d20 con 0x7effc80abce0 2021-11-21T21:55:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effb67fc700 1 -- 172.21.2.116:0/380876393 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc4002680 con 0x7effc80abce0 2021-11-21T21:55:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc8147420 con 0x7effc80abce0 2021-11-21T21:55:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc805cb80 con 0x7effc80abce0 2021-11-21T21:55:57.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effb67fc700 1 -- 172.21.2.116:0/380876393 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7effc400f030 con 0x7effc80abce0 2021-11-21T21:55:57.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effb67fc700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effa8053600 0x7effa8055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:57.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effb67fc700 1 -- 172.21.2.116:0/380876393 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7effc406b780 con 0x7effc80abce0 2021-11-21T21:55:57.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.456+0000 7effccbf6700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effa8053600 0x7effa8055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:57.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.460+0000 7effccbf6700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effa8053600 0x7effa8055ac0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7effc80b4e00 tx=0x7effb8012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:57.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.460+0000 7effb67fc700 1 -- 172.21.2.116:0/380876393 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7effc40427d0 con 0x7effc80abce0 2021-11-21T21:55:57.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:57.568+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7effc8145eb0 con 0x7effc80abce0 2021-11-21T21:55:58.027 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T21:55:58.028+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T21:55:58.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.036+0000 7effb67fc700 1 -- 172.21.2.116:0/380876393 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v12) v1 ==== 161+0+0 (secure 0 0 0) 0x7effc4045df0 con 0x7effc80abce0 2021-11-21T21:55:58.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effa8053600 msgr2=0x7effa8055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effa8053600 0x7effa8055ac0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7effc80b4e00 tx=0x7effb8012000).stop 2021-11-21T21:55:58.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 msgr2=0x7effc8145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc8145340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7effc4002ef0 tx=0x7effc4006d50).stop 2021-11-21T21:55:58.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 shutdown_connections 2021-11-21T21:55:58.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7effc8144810 0x7effc80b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:58.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7effc80ac600 0x7effc8146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:58.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effa8053600 0x7effa8055ac0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 --2- 172.21.2.116:0/380876393 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7effc80abce0 0x7effc8145340 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 >> 172.21.2.116:0/380876393 conn(0x7effc805bd20 msgr2=0x7effc80af780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:58.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 shutdown_connections 2021-11-21T21:55:58.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.040+0000 7effcf65b700 1 -- 172.21.2.116:0/380876393 wait complete. 2021-11-21T21:55:58.044 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T21:55:58.046 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:55:58.057 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:55:58.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 Processor -- start 2021-11-21T21:55:58.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- start start 2021-11-21T21:55:58.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6df80a6aa0 0x7f6df814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df8062410 0x7f6df81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df80a6e80 con 0x7f6df80a86b0 2021-11-21T21:55:58.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df80a8bc0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df814a7b0 con 0x7f6df8062410 2021-11-21T21:55:58.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6df80a6aa0 0x7f6df814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6df80a6aa0 0x7f6df814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46350/0 (socket says 172.21.2.116:46350) 2021-11-21T21:55:58.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 -- 172.21.2.116:0/3461780603 learned_addr learned my addr 172.21.2.116:0/3461780603 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:58.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df6d9d700 1 --2- 172.21.2.116:0/3461780603 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df8062410 0x7f6df81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 -- 172.21.2.116:0/3461780603 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df8062410 msgr2=0x7f6df81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:58.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 --2- 172.21.2.116:0/3461780603 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df8062410 0x7f6df81443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 -- 172.21.2.116:0/3461780603 >> v1:172.21.2.117:6789/0 conn(0x7f6df80a86b0 legacy=0x7f6df81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:58.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 -- 172.21.2.116:0/3461780603 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df814aa10 con 0x7f6df80a6aa0 2021-11-21T21:55:58.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df6d9d700 1 --2- 172.21.2.116:0/3461780603 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df8062410 0x7f6df81443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:55:58.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df659c700 1 --2- 172.21.2.116:0/3461780603 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df814b150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f6df8061ab0 tx=0x7f6de8006eb0).ready entity=mon.0 client_cookie=7c6468035642aa0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:58.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df559a700 1 -- 172.21.2.116:0/3461780603 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6de800d570 con 0x7f6df80a6aa0 2021-11-21T21:55:58.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df559a700 1 -- 172.21.2.116:0/3461780603 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8000e90 con 0x7f6df80a6aa0 2021-11-21T21:55:58.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6df559a700 1 -- 172.21.2.116:0/3461780603 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6de800d8c0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/3461780603 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 msgr2=0x7f6df814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/3461780603 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df814b150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f6df8061ab0 tx=0x7f6de8006eb0).stop 2021-11-21T21:55:58.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/3461780603 shutdown_connections 2021-11-21T21:55:58.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/3461780603 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df8062410 0x7f6df81443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/3461780603 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df814b150 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/3461780603 >> 172.21.2.116:0/3461780603 conn(0x7f6df805bd20 msgr2=0x7f6df805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:58.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/3461780603 shutdown_connections 2021-11-21T21:55:58.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/3461780603 wait complete. 2021-11-21T21:55:58.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 Processor -- start 2021-11-21T21:55:58.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.100+0000 7f6dfcc0b700 1 -- start start 2021-11-21T21:55:58.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6df8062410 0x7f6df8064780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df8064c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df80a86b0 0x7f6df8156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8065160 con 0x7f6df80a6aa0 2021-11-21T21:55:58.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80a98c0 con 0x7f6df8062410 2021-11-21T21:55:58.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80ab5f0 con 0x7f6df80a86b0 2021-11-21T21:55:58.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df8064c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df8064c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46358/0 (socket says 172.21.2.116:46358) 2021-11-21T21:55:58.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 -- 172.21.2.116:0/850048694 learned_addr learned my addr 172.21.2.116:0/850048694 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df6d9d700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df80a86b0 0x7f6df8156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df659c700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6df8062410 0x7f6df8064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 -- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df80a86b0 msgr2=0x7f6df8156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df80a86b0 0x7f6df8156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 -- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6df8062410 msgr2=0x7f6df8064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6df8062410 0x7f6df8064780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 -- 172.21.2.116:0/850048694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de800b1e0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df659c700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6df8062410 0x7f6df8064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:55:58.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df5d9b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df8064c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f6de0002f30 tx=0x7f6de0006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:58.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6def7fe700 1 -- 172.21.2.116:0/850048694 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6de0002320 con 0x7f6df80a6aa0 2021-11-21T21:55:58.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6def7fe700 1 -- 172.21.2.116:0/850048694 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de0002480 con 0x7f6df80a6aa0 2021-11-21T21:55:58.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df80ab350 con 0x7f6df80a6aa0 2021-11-21T21:55:58.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6def7fe700 1 -- 172.21.2.116:0/850048694 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6de00027e0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df80aa090 con 0x7f6df80a6aa0 2021-11-21T21:55:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df80573d0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6def7fe700 1 -- 172.21.2.116:0/850048694 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6de000f030 con 0x7f6df80a6aa0 2021-11-21T21:55:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6def7fe700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ddc053520 0x7f6ddc0559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6def7fe700 1 -- 172.21.2.116:0/850048694 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f6de006b5c0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df659c700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ddc053520 0x7f6ddc0559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.104+0000 7f6df659c700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ddc053520 0x7f6ddc0559e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f6de8017000 tx=0x7f6de8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:58.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.108+0000 7f6def7fe700 1 -- 172.21.2.116:0/850048694 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6de00426d0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.216+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6df805db70 con 0x7f6df80a6aa0 2021-11-21T21:55:58.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.216+0000 7f6def7fe700 1 -- 172.21.2.116:0/850048694 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2119 (secure 0 0 0) 0x7f6de0045cf0 con 0x7f6df80a6aa0 2021-11-21T21:55:58.219 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:55:58.219 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:54:51.352804+0000","modified":"2021-11-21T21:55:58.029697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-11-21T21:55:58.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ddc053520 msgr2=0x7f6ddc0559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ddc053520 0x7f6ddc0559e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f6de8017000 tx=0x7f6de8015000).stop 2021-11-21T21:55:58.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 msgr2=0x7f6df8064c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df8064c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f6de0002f30 tx=0x7f6de0006d70).stop 2021-11-21T21:55:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 shutdown_connections 2021-11-21T21:55:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6df80a86b0 0x7f6df8156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ddc053520 0x7f6ddc0559e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6df80a6aa0 0x7f6df8064c70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 --2- 172.21.2.116:0/850048694 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6df8062410 0x7f6df8064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 >> 172.21.2.116:0/850048694 conn(0x7f6df805bd20 msgr2=0x7f6df8148650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:58.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 shutdown_connections 2021-11-21T21:55:58.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.220+0000 7f6dfcc0b700 1 -- 172.21.2.116:0/850048694 wait complete. 2021-11-21T21:55:58.223 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 12 2021-11-21T21:55:58.226 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T21:55:58.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4ba759700 1 Processor -- start 2021-11-21T21:55:58.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4ba759700 1 -- start start 2021-11-21T21:55:58.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4ba759700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff4b41478c0 0x7ff4b40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4ba759700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b4147e80 con 0x7ff4b4144810 2021-11-21T21:55:58.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4ba759700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40af5a0 con 0x7ff4b40abce0 2021-11-21T21:55:58.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4ba759700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40b1490 con 0x7ff4b41478c0 2021-11-21T21:55:58.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4b8cf6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff4b41478c0 0x7ff4b40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4b3fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff4b4144810 0x7ff4b40b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58320/0 (socket says 172.21.2.116:58320) 2021-11-21T21:55:58.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.276+0000 7ff4b3fff700 1 -- 172.21.2.116:0/952642898 learned_addr learned my addr 172.21.2.116:0/952642898 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:55:58.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 -- 172.21.2.116:0/952642898 >> v1:172.21.2.117:6789/0 conn(0x7ff4b40abce0 legacy=0x7ff4b40ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:55:58.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 -- 172.21.2.116:0/952642898 >> v1:172.21.2.116:6789/0 conn(0x7ff4b4144810 legacy=0x7ff4b40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 -- 172.21.2.116:0/952642898 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40b1fd0 con 0x7ff4b41478c0 2021-11-21T21:55:58.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 --2- 172.21.2.116:0/952642898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b40acd10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7ff4b40b4e00 tx=0x7ff4ac0091d0).ready entity=mon.0 client_cookie=bf69c338f37b6b02 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:58.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b2ffd700 1 -- 172.21.2.116:0/952642898 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4ac009ea0 con 0x7ff4b41478c0 2021-11-21T21:55:58.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b2ffd700 1 -- 172.21.2.116:0/952642898 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00b860 con 0x7ff4b41478c0 2021-11-21T21:55:58.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b2ffd700 1 -- 172.21.2.116:0/952642898 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4ac00b9d0 con 0x7ff4b41478c0 2021-11-21T21:55:58.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/952642898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 msgr2=0x7ff4b40acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 --2- 172.21.2.116:0/952642898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b40acd10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7ff4b40b4e00 tx=0x7ff4ac0091d0).stop 2021-11-21T21:55:58.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/952642898 shutdown_connections 2021-11-21T21:55:58.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 --2- 172.21.2.116:0/952642898 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b40acd10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:58.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/952642898 >> 172.21.2.116:0/952642898 conn(0x7ff4b405bd20 msgr2=0x7ff4b405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:58.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/952642898 shutdown_connections 2021-11-21T21:55:58.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/952642898 wait complete. 2021-11-21T21:55:58.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 Processor -- start 2021-11-21T21:55:58.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- start start 2021-11-21T21:55:58.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4b40abce0 0x7ff4b4156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4b4144810 0x7ff4b413f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b4146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b41467e0 con 0x7ff4b41478c0 2021-11-21T21:55:58.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4143680 con 0x7ff4b4144810 2021-11-21T21:55:58.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4156c10 con 0x7ff4b40abce0 2021-11-21T21:55:58.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b4146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b4146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46374/0 (socket says 172.21.2.116:46374) 2021-11-21T21:55:58.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 -- 172.21.2.116:0/879354182 learned_addr learned my addr 172.21.2.116:0/879354182 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:58.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 -- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4b40abce0 msgr2=0x7ff4b4156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4b40abce0 0x7ff4b4156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 -- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4b4144810 msgr2=0x7ff4b413f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4b4144810 0x7ff4b413f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 -- 172.21.2.116:0/879354182 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac009470 con 0x7ff4b41478c0 2021-11-21T21:55:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b8cf6700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b4146eb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff4b40b4e00 tx=0x7ff4ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:58.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b17fa700 1 -- 172.21.2.116:0/879354182 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4ac009ea0 con 0x7ff4b41478c0 2021-11-21T21:55:58.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b17fa700 1 -- 172.21.2.116:0/879354182 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac01f5e0 con 0x7ff4b41478c0 2021-11-21T21:55:58.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b17fa700 1 -- 172.21.2.116:0/879354182 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4ac01f940 con 0x7ff4b41478c0 2021-11-21T21:55:58.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b4146450 con 0x7ff4b41478c0 2021-11-21T21:55:58.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b413f590 con 0x7ff4b41478c0 2021-11-21T21:55:58.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40b2c50 con 0x7ff4b41478c0 2021-11-21T21:55:58.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.280+0000 7ff4b17fa700 1 -- 172.21.2.116:0/879354182 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff4ac026030 con 0x7ff4b41478c0 2021-11-21T21:55:58.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.284+0000 7ff4b17fa700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4980536d0 0x7ff498055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:58.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.284+0000 7ff4b17fa700 1 -- 172.21.2.116:0/879354182 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6155+0+0 (secure 0 0 0) 0x7ff4ac06c770 con 0x7ff4b41478c0 2021-11-21T21:55:58.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.284+0000 7ff4b3fff700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4980536d0 0x7ff498055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:58.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.284+0000 7ff4b3fff700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4980536d0 0x7ff498055b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff4b4062500 tx=0x7ff4a4007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:58.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.284+0000 7ff4b17fa700 1 -- 172.21.2.116:0/879354182 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4b40b2c50 con 0x7ff4b41478c0 2021-11-21T21:55:58.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:58.392+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4b40b2c50 con 0x7ff4b41478c0 2021-11-21T21:55:59.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4b17fa700 1 -- 172.21.2.116:0/879354182 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff4b40b2c50 con 0x7ff4b41478c0 2021-11-21T21:55:59.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4980536d0 msgr2=0x7ff498055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:59.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4980536d0 0x7ff498055b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff4b4062500 tx=0x7ff4a4007800).stop 2021-11-21T21:55:59.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 msgr2=0x7ff4b4146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:59.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b4146eb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff4b40b4e00 tx=0x7ff4ac016000).stop 2021-11-21T21:55:59.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 shutdown_connections 2021-11-21T21:55:59.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4980536d0 0x7ff498055b90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:59.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4b41478c0 0x7ff4b4146eb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:59.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4b4144810 0x7ff4b413f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:59.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 --2- 172.21.2.116:0/879354182 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4b40abce0 0x7ff4b4156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:59.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 >> 172.21.2.116:0/879354182 conn(0x7ff4b405bd20 msgr2=0x7ff4b405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:59.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 shutdown_connections 2021-11-21T21:55:59.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.052+0000 7ff4ba759700 1 -- 172.21.2.116:0/879354182 wait complete. 2021-11-21T21:55:59.060 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T21:55:59.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f7174d6b700 1 Processor -- start 2021-11-21T21:55:59.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f7174d6b700 1 -- start start 2021-11-21T21:55:59.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f7174d6b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 0x7f7170147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:59.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f7174d6b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7170143e60 0x7f71701442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:59.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f7174d6b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71700b1940 con 0x7f71700b11c0 2021-11-21T21:55:59.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f7174d6b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71700b15a0 con 0x7f7170143e60 2021-11-21T21:55:59.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f7174d6b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700b1b70 con 0x7f71700a8670 2021-11-21T21:55:59.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716e59c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f71700b11c0 0x7f7170147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58336/0 (socket says 172.21.2.116:58336) 2021-11-21T21:55:59.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716e59c700 1 -- 172.21.2.116:0/2960860649 learned_addr learned my addr 172.21.2.116:0/2960860649 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:55:59.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716ed9d700 1 --2- 172.21.2.116:0/2960860649 >> v2:172.21.2.116:3300/0 conn(0x7f7170143e60 0x7f71701442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:59.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716d59a700 1 -- 172.21.2.116:0/2960860649 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274169709 0 0) 0x7f71700b1940 con 0x7f71700b11c0 2021-11-21T21:55:59.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716d59a700 1 -- 172.21.2.116:0/2960860649 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7154003260 con 0x7f71700b11c0 2021-11-21T21:55:59.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716ed9d700 1 -- 172.21.2.116:0/2960860649 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 msgr2=0x7f7170147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:55:59.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716ed9d700 1 --2- 172.21.2.116:0/2960860649 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 0x7f7170147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:59.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716ed9d700 1 -- 172.21.2.116:0/2960860649 >> v1:172.21.2.116:6789/0 conn(0x7f71700b11c0 legacy=0x7f7170147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:59.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716ed9d700 1 -- 172.21.2.116:0/2960860649 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700b1e10 con 0x7f7170143e60 2021-11-21T21:55:59.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716ed9d700 1 --2- 172.21.2.116:0/2960860649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7170143e60 0x7f71701442f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f7160001110 tx=0x7f71680091d0).ready entity=mon.0 client_cookie=2a104114fa013d84 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:59.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716d59a700 1 -- 172.21.2.116:0/2960860649 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f716800b680 con 0x7f7170143e60 2021-11-21T21:55:59.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716d59a700 1 -- 172.21.2.116:0/2960860649 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800b7e0 con 0x7f7170143e60 2021-11-21T21:55:59.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.116+0000 7f716d59a700 1 -- 172.21.2.116:0/2960860649 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f716800b990 con 0x7f7170143e60 2021-11-21T21:55:59.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2960860649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7170143e60 msgr2=0x7f71701442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:59.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2960860649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7170143e60 0x7f71701442f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f7160001110 tx=0x7f71680091d0).stop 2021-11-21T21:55:59.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2960860649 shutdown_connections 2021-11-21T21:55:59.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2960860649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7170143e60 0x7f71701442f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:55:59.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2960860649 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 0x7f7170147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:59.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2960860649 >> 172.21.2.116:0/2960860649 conn(0x7f717005bd20 msgr2=0x7f717005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:55:59.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2960860649 shutdown_connections 2021-11-21T21:55:59.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2960860649 wait complete. 2021-11-21T21:55:59.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 Processor -- start 2021-11-21T21:55:59.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- start start 2021-11-21T21:55:59.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 0x7f71700651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:59.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71700b11c0 0x7f71700656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:59.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7170143e60 0x7f7170065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:59.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700660c0 con 0x7f71700b11c0 2021-11-21T21:55:59.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717014bef0 con 0x7f7170143e60 2021-11-21T21:55:59.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717014c0e0 con 0x7f71700a8670 2021-11-21T21:55:59.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71700b11c0 0x7f71700656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:59.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71700b11c0 0x7f71700656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46388/0 (socket says 172.21.2.116:46388) 2021-11-21T21:55:59.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 -- 172.21.2.116:0/2423667777 learned_addr learned my addr 172.21.2.116:0/2423667777 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:55:59.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 -- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 msgr2=0x7f71700651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:59.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 0x7f71700651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:59.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 -- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7170143e60 msgr2=0x7f7170065bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:55:59.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7170143e60 0x7f7170065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:55:59.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 -- 172.21.2.116:0/2423667777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168009470 con 0x7f71700b11c0 2021-11-21T21:55:59.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f716dd9b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71700b11c0 0x7f71700656e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f7164002f70 tx=0x7f7164006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:59.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f715f7fe700 1 -- 172.21.2.116:0/2423667777 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7164013460 con 0x7f71700b11c0 2021-11-21T21:55:59.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f715f7fe700 1 -- 172.21.2.116:0/2423667777 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640135c0 con 0x7f71700b11c0 2021-11-21T21:55:59.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f715f7fe700 1 -- 172.21.2.116:0/2423667777 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7164013920 con 0x7f71700b11c0 2021-11-21T21:55:59.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717014c380 con 0x7f71700b11c0 2021-11-21T21:55:59.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7170067630 con 0x7f71700b11c0 2021-11-21T21:55:59.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71700b20b0 con 0x7f71700b11c0 2021-11-21T21:55:59.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f715f7fe700 1 -- 172.21.2.116:0/2423667777 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f716401a060 con 0x7f71700b11c0 2021-11-21T21:55:59.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f715f7fe700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f714c053660 0x7f714c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:55:59.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.120+0000 7f715f7fe700 1 -- 172.21.2.116:0/2423667777 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f716406b500 con 0x7f71700b11c0 2021-11-21T21:55:59.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.124+0000 7f716e59c700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f714c053660 0x7f714c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:55:59.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.124+0000 7f716e59c700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f714c053660 0x7f714c055b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f7160000bc0 tx=0x7f71600076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:55:59.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.124+0000 7f715f7fe700 1 -- 172.21.2.116:0/2423667777 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7164042500 con 0x7f71700b11c0 2021-11-21T21:55:59.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:55:59.208+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71700573d0 con 0x7f71700b11c0 2021-11-21T21:56:00.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f715f7fe700 1 -- 172.21.2.116:0/2423667777 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 185+0+0 (secure 0 0 0) 0x7f7164018030 con 0x7f71700b11c0 2021-11-21T21:56:00.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f714c053660 msgr2=0x7f714c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:00.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f714c053660 0x7f714c055b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f7160000bc0 tx=0x7f71600076c0).stop 2021-11-21T21:56:00.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71700b11c0 msgr2=0x7f71700656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:00.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71700b11c0 0x7f71700656e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f7164002f70 tx=0x7f7164006ec0).stop 2021-11-21T21:56:00.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 shutdown_connections 2021-11-21T21:56:00.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7170143e60 0x7f7170065bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:00.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f714c053660 0x7f714c055b20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:00.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71700b11c0 0x7f71700656e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:00.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 --2- 172.21.2.116:0/2423667777 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f71700a8670 0x7f71700651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:00.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.060+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 >> 172.21.2.116:0/2423667777 conn(0x7f717005bd20 msgr2=0x7f7170146290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:00.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.064+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 shutdown_connections 2021-11-21T21:56:00.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.064+0000 7f7174d6b700 1 -- 172.21.2.116:0/2423667777 wait complete. 2021-11-21T21:56:00.067 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T21:56:00.069 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T21:56:00.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5bacee7700 1 Processor -- start 2021-11-21T21:56:00.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5bacee7700 1 -- start start 2021-11-21T21:56:00.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5bacee7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba8147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:00.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5bacee7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5ba8143e60 0x7f5ba81442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:00.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5bacee7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba80b1940 con 0x7f5ba80b11c0 2021-11-21T21:56:00.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5bacee7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80b15a0 con 0x7f5ba8143e60 2021-11-21T21:56:00.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5bacee7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80b1b70 con 0x7f5ba80a8670 2021-11-21T21:56:00.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba6d9d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5ba8143e60 0x7f5ba81442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:00.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba5d9b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba8147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:00.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba659c700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f5ba80b11c0 0x7f5ba8147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:57262/0 (socket says 172.21.2.116:57262) 2021-11-21T21:56:00.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba659c700 1 -- 172.21.2.116:0/3165118185 learned_addr learned my addr 172.21.2.116:0/3165118185 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:00.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba6d9d700 1 -- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 msgr2=0x7f5ba8147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:00.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba6d9d700 1 --2- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba8147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:00.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba6d9d700 1 -- 172.21.2.116:0/3165118185 >> v1:172.21.2.117:6789/0 conn(0x7f5ba80b11c0 legacy=0x7f5ba8147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:00.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba6d9d700 1 -- 172.21.2.116:0/3165118185 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80b1e10 con 0x7f5ba8143e60 2021-11-21T21:56:00.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.120+0000 7f5ba5d9b700 1 --2- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba8147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:56:00.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba6d9d700 1 --2- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 0x7f5ba81442f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f5ba00032f0 tx=0x7f5ba00091d0).ready entity=mon.1 client_cookie=5d7f6e463c455a62 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:00.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba559a700 1 -- 172.21.2.116:0/3165118185 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba000b6c0 con 0x7f5ba8143e60 2021-11-21T21:56:00.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba559a700 1 -- 172.21.2.116:0/3165118185 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba000b820 con 0x7f5ba8143e60 2021-11-21T21:56:00.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba559a700 1 -- 172.21.2.116:0/3165118185 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba000b990 con 0x7f5ba8143e60 2021-11-21T21:56:00.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 msgr2=0x7f5ba81442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:00.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 0x7f5ba81442f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f5ba00032f0 tx=0x7f5ba00091d0).stop 2021-11-21T21:56:00.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- 172.21.2.116:0/3165118185 shutdown_connections 2021-11-21T21:56:00.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 0x7f5ba81442f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:00.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3165118185 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba8147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:00.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- 172.21.2.116:0/3165118185 >> 172.21.2.116:0/3165118185 conn(0x7f5ba805bd20 msgr2=0x7f5ba805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:00.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- 172.21.2.116:0/3165118185 shutdown_connections 2021-11-21T21:56:00.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- 172.21.2.116:0/3165118185 wait complete. 2021-11-21T21:56:00.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 Processor -- start 2021-11-21T21:56:00.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- start start 2021-11-21T21:56:00.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba814cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:00.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ba80b11c0 0x7f5ba8063850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:00.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 0x7f5ba80b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:00.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba815a250 con 0x7f5ba80b11c0 2021-11-21T21:56:00.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba814a720 con 0x7f5ba8143e60 2021-11-21T21:56:00.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba815a470 con 0x7f5ba80a8670 2021-11-21T21:56:00.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ba80b11c0 0x7f5ba8063850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:00.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ba80b11c0 0x7f5ba8063850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46404/0 (socket says 172.21.2.116:46404) 2021-11-21T21:56:00.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 -- 172.21.2.116:0/3496541375 learned_addr learned my addr 172.21.2.116:0/3496541375 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:00.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba6d9d700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 0x7f5ba80b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:00.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba659c700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba814cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:00.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 -- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 msgr2=0x7f5ba814cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:00.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba814cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:00.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 -- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 msgr2=0x7f5ba80b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:00.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 0x7f5ba80b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:00.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 -- 172.21.2.116:0/3496541375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c00b180 con 0x7f5ba80b11c0 2021-11-21T21:56:00.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5ba5d9b700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ba80b11c0 0x7f5ba8063850 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f5b9c00c000 tx=0x7f5b9c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:00.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5b977fe700 1 -- 172.21.2.116:0/3496541375 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9c00c890 con 0x7f5ba80b11c0 2021-11-21T21:56:00.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5b977fe700 1 -- 172.21.2.116:0/3496541375 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c00c9f0 con 0x7f5ba80b11c0 2021-11-21T21:56:00.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba0009470 con 0x7f5ba80b11c0 2021-11-21T21:56:00.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba814a950 con 0x7f5ba80b11c0 2021-11-21T21:56:00.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.124+0000 7f5b977fe700 1 -- 172.21.2.116:0/3496541375 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9c00cd50 con 0x7f5ba80b11c0 2021-11-21T21:56:00.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.128+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba814ab10 con 0x7f5ba80b11c0 2021-11-21T21:56:00.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.128+0000 7f5b977fe700 1 -- 172.21.2.116:0/3496541375 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5b9c01c460 con 0x7f5ba80b11c0 2021-11-21T21:56:00.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.128+0000 7f5b977fe700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b84053640 0x7f5b84055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:00.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.128+0000 7f5b977fe700 1 -- 172.21.2.116:0/3496541375 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f5b9c06bed0 con 0x7f5ba80b11c0 2021-11-21T21:56:00.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.128+0000 7f5ba659c700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b84053640 0x7f5b84055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:00.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.128+0000 7f5ba659c700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b84053640 0x7f5b84055b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5b9800b000 tx=0x7f5b98009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:00.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.128+0000 7f5b977fe700 1 -- 172.21.2.116:0/3496541375 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b9c043020 con 0x7f5ba80b11c0 2021-11-21T21:56:00.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:00.212+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5ba80b20b0 con 0x7f5ba80b11c0 2021-11-21T21:56:01.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5b977fe700 1 -- 172.21.2.116:0/3496541375 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5b9c046640 con 0x7f5ba80b11c0 2021-11-21T21:56:01.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b84053640 msgr2=0x7f5b84055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b84053640 0x7f5b84055b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5b9800b000 tx=0x7f5b98009040).stop 2021-11-21T21:56:01.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ba80b11c0 msgr2=0x7f5ba8063850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ba80b11c0 0x7f5ba8063850 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f5b9c00c000 tx=0x7f5b9c006df0).stop 2021-11-21T21:56:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 shutdown_connections 2021-11-21T21:56:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ba8143e60 0x7f5ba80b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b84053640 0x7f5b84055b00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ba80b11c0 0x7f5ba8063850 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 --2- 172.21.2.116:0/3496541375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ba80a8670 0x7f5ba814cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 >> 172.21.2.116:0/3496541375 conn(0x7f5ba805bd20 msgr2=0x7f5ba8144270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 shutdown_connections 2021-11-21T21:56:01.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.072+0000 7f5bacee7700 1 -- 172.21.2.116:0/3496541375 wait complete. 2021-11-21T21:56:01.078 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T21:56:01.080 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T21:56:01.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 Processor -- start 2021-11-21T21:56:01.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- start start 2021-11-21T21:56:01.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 0x7fa6f80b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa6f80b3690 0x7fa6f813a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80b3c50 con 0x7fa6f80adc10 2021-11-21T21:56:01.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80b3ef0 con 0x7fa6f80b3690 2021-11-21T21:56:01.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b4190 con 0x7fa6f8144a10 2021-11-21T21:56:01.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa6f80b3690 0x7fa6f813a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa6f80b3690 0x7fa6f813a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46410/0 (socket says 172.21.2.116:46410) 2021-11-21T21:56:01.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 -- 172.21.2.116:0/1759261935 learned_addr learned my addr 172.21.2.116:0/1759261935 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:01.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 -- 172.21.2.116:0/1759261935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 msgr2=0x7fa6f80b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:01.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 --2- 172.21.2.116:0/1759261935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 0x7fa6f80b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:01.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 -- 172.21.2.116:0/1759261935 >> v1:172.21.2.117:6789/0 conn(0x7fa6f80adc10 legacy=0x7fa6f8141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:01.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 -- 172.21.2.116:0/1759261935 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8140b80 con 0x7fa6f80b3690 2021-11-21T21:56:01.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fe5ce700 1 --2- 172.21.2.116:0/1759261935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f813a540 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fa6f80b7000 tx=0x7fa6ec006eb0).ready entity=mon.0 client_cookie=e4d8a4da02fef1e4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:01.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fcdcb700 1 -- 172.21.2.116:0/1759261935 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6ec00d570 con 0x7fa6f80b3690 2021-11-21T21:56:01.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fcdcb700 1 -- 172.21.2.116:0/1759261935 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec000e90 con 0x7fa6f80b3690 2021-11-21T21:56:01.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fcdcb700 1 -- 172.21.2.116:0/1759261935 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6ec00d8c0 con 0x7fa6f80b3690 2021-11-21T21:56:01.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- 172.21.2.116:0/1759261935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 msgr2=0x7fa6f813a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- 172.21.2.116:0/1759261935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f813a540 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fa6f80b7000 tx=0x7fa6ec006eb0).stop 2021-11-21T21:56:01.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- 172.21.2.116:0/1759261935 shutdown_connections 2021-11-21T21:56:01.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- 172.21.2.116:0/1759261935 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f813a540 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- 172.21.2.116:0/1759261935 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 0x7fa6f80b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:01.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- 172.21.2.116:0/1759261935 >> 172.21.2.116:0/1759261935 conn(0x7fa6f805bd20 msgr2=0x7fa6f805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:01.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- 172.21.2.116:0/1759261935 shutdown_connections 2021-11-21T21:56:01.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- 172.21.2.116:0/1759261935 wait complete. 2021-11-21T21:56:01.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 Processor -- start 2021-11-21T21:56:01.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- start start 2021-11-21T21:56:01.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa6f80adc10 0x7fa6f80b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 0x7fa6f8153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b0600 con 0x7fa6f80b3690 2021-11-21T21:56:01.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b0830 con 0x7fa6f80adc10 2021-11-21T21:56:01.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b0ad0 con 0x7fa6f8144a10 2021-11-21T21:56:01.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f80afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46416/0 (socket says 172.21.2.116:46416) 2021-11-21T21:56:01.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 -- 172.21.2.116:0/176916316 learned_addr learned my addr 172.21.2.116:0/176916316 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:01.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 -- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 msgr2=0x7fa6f8153760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:01.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 0x7fa6f8153760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:01.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 -- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa6f80adc10 msgr2=0x7fa6f80b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa6f80adc10 0x7fa6f80b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:01.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 -- 172.21.2.116:0/176916316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec00b1e0 con 0x7fa6f80b3690 2021-11-21T21:56:01.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6fd5cc700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f80afbb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fa6e8002f70 tx=0x7fa6e8006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:01.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6f6ffd700 1 -- 172.21.2.116:0/176916316 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6e8013460 con 0x7fa6f80b3690 2021-11-21T21:56:01.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6f6ffd700 1 -- 172.21.2.116:0/176916316 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e80135c0 con 0x7fa6f80b3690 2021-11-21T21:56:01.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f81532c0 con 0x7fa6f80b3690 2021-11-21T21:56:01.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa6f6ffd700 1 -- 172.21.2.116:0/176916316 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6e8013920 con 0x7fa6f80b3690 2021-11-21T21:56:01.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.140+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f80b12a0 con 0x7fa6f80b3690 2021-11-21T21:56:01.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.144+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f81530d0 con 0x7fa6f80b3690 2021-11-21T21:56:01.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.144+0000 7fa6f6ffd700 1 -- 172.21.2.116:0/176916316 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa6e800f070 con 0x7fa6f80b3690 2021-11-21T21:56:01.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.144+0000 7fa6f6ffd700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa6d8053520 0x7fa6d80559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.144+0000 7fa6f6ffd700 1 -- 172.21.2.116:0/176916316 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa6e806b050 con 0x7fa6f80b3690 2021-11-21T21:56:01.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.144+0000 7fa6fddcd700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa6d8053520 0x7fa6d80559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.144+0000 7fa6fddcd700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa6d8053520 0x7fa6d80559e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fa6f0003310 tx=0x7fa6f0006d30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:01.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.144+0000 7fa6f6ffd700 1 -- 172.21.2.116:0/176916316 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa6e8042380 con 0x7fa6f80b3690 2021-11-21T21:56:01.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.228+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa6f8153450 con 0x7fa6f80b3690 2021-11-21T21:56:01.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.232+0000 7fa6f6ffd700 1 -- 172.21.2.116:0/176916316 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa6e80459a0 con 0x7fa6f80b3690 2021-11-21T21:56:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa6d8053520 msgr2=0x7fa6d80559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa6d8053520 0x7fa6d80559e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fa6f0003310 tx=0x7fa6f0006d30).stop 2021-11-21T21:56:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 msgr2=0x7fa6f80afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f80afbb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fa6e8002f70 tx=0x7fa6e8006e20).stop 2021-11-21T21:56:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 shutdown_connections 2021-11-21T21:56:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa6f8144a10 0x7fa6f8153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa6d8053520 0x7fa6d80559e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa6f80b3690 0x7fa6f80afbb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 --2- 172.21.2.116:0/176916316 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa6f80adc10 0x7fa6f80b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:01.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 >> 172.21.2.116:0/176916316 conn(0x7fa6f805bd20 msgr2=0x7fa6f8143f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:01.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 shutdown_connections 2021-11-21T21:56:01.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.236+0000 7fa700031700 1 -- 172.21.2.116:0/176916316 wait complete. 2021-11-21T21:56:01.242 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2021-11-21T21:56:01.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12f1ac9700 1 Processor -- start 2021-11-21T21:56:01.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12f1ac9700 1 -- start start 2021-11-21T21:56:01.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12f1ac9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12f1ac9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12ec0aed90 0x7f12ec13c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12f1ac9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec0b00d0 con 0x7f12ec0af360 2021-11-21T21:56:01.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12f1ac9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ec0adba0 con 0x7f12ec0aed90 2021-11-21T21:56:01.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12f1ac9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0af7a0 con 0x7f12ec0b24a0 2021-11-21T21:56:01.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12eaffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f12ec0af360 0x7f12ec0ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58380/0 (socket says 172.21.2.116:58380) 2021-11-21T21:56:01.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12eaffd700 1 -- 172.21.2.116:0/2415760242 learned_addr learned my addr 172.21.2.116:0/2415760242 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:01.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ebfff700 1 --2- 172.21.2.116:0/2415760242 >> v2:172.21.2.116:3300/0 conn(0x7f12ec0aed90 0x7f12ec13c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12eb7fe700 1 --2- 172.21.2.116:0/2415760242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ebfff700 1 -- 172.21.2.116:0/2415760242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 msgr2=0x7f12ec0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:01.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ebfff700 1 --2- 172.21.2.116:0/2415760242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec0b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ebfff700 1 -- 172.21.2.116:0/2415760242 >> v1:172.21.2.116:6789/0 conn(0x7f12ec0af360 legacy=0x7f12ec0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ebfff700 1 -- 172.21.2.116:0/2415760242 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0b2880 con 0x7f12ec0aed90 2021-11-21T21:56:01.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ebfff700 1 --2- 172.21.2.116:0/2415760242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0aed90 0x7f12ec13c210 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f12e4001490 tx=0x7f12dc006df0).ready entity=mon.0 client_cookie=c4f71682dff4a3bf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:01.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ea7fc700 1 -- 172.21.2.116:0/2415760242 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12dc00d4a0 con 0x7f12ec0aed90 2021-11-21T21:56:01.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ea7fc700 1 -- 172.21.2.116:0/2415760242 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc000e90 con 0x7f12ec0aed90 2021-11-21T21:56:01.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.292+0000 7f12ea7fc700 1 -- 172.21.2.116:0/2415760242 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12dc00d7f0 con 0x7f12ec0aed90 2021-11-21T21:56:01.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/2415760242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0aed90 msgr2=0x7f12ec13c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/2415760242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0aed90 0x7f12ec13c210 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f12e4001490 tx=0x7f12dc006df0).stop 2021-11-21T21:56:01.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/2415760242 shutdown_connections 2021-11-21T21:56:01.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/2415760242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0aed90 0x7f12ec13c210 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/2415760242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/2415760242 >> 172.21.2.116:0/2415760242 conn(0x7f12ec05fd00 msgr2=0x7f12ec0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:01.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/2415760242 shutdown_connections 2021-11-21T21:56:01.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/2415760242 wait complete. 2021-11-21T21:56:01.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 Processor -- start 2021-11-21T21:56:01.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- start start 2021-11-21T21:56:01.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12ec0aed90 0x7f12ec13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0af360 0x7f12ec0664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec065490 con 0x7f12ec0af360 2021-11-21T21:56:01.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec065730 con 0x7f12ec0aed90 2021-11-21T21:56:01.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec1556d0 con 0x7f12ec0b24a0 2021-11-21T21:56:01.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0af360 0x7f12ec0664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0af360 0x7f12ec0664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46430/0 (socket says 172.21.2.116:46430) 2021-11-21T21:56:01.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 -- 172.21.2.116:0/3678906179 learned_addr learned my addr 172.21.2.116:0/3678906179 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:01.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12ebfff700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eb7fe700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12ec0aed90 0x7f12ec13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 -- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 msgr2=0x7f12ec155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 -- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12ec0aed90 msgr2=0x7f12ec13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:01.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12ec0aed90 0x7f12ec13b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:01.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 -- 172.21.2.116:0/3678906179 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc00b180 con 0x7f12ec0af360 2021-11-21T21:56:01.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eaffd700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0af360 0x7f12ec0664c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f12e4001550 tx=0x7f12e4007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:01.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12e8ff9700 1 -- 172.21.2.116:0/3678906179 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e4005ba0 con 0x7f12ec0af360 2021-11-21T21:56:01.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12e8ff9700 1 -- 172.21.2.116:0/3678906179 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e4005d00 con 0x7f12ec0af360 2021-11-21T21:56:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec13b690 con 0x7f12ec0af360 2021-11-21T21:56:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12e8ff9700 1 -- 172.21.2.116:0/3678906179 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e4007300 con 0x7f12ec0af360 2021-11-21T21:56:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec065c80 con 0x7f12ec0af360 2021-11-21T21:56:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec132cf0 con 0x7f12ec0af360 2021-11-21T21:56:01.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12e8ff9700 1 -- 172.21.2.116:0/3678906179 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f12e4010030 con 0x7f12ec0af360 2021-11-21T21:56:01.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12e8ff9700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12cc053580 0x7f12cc055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:01.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12e8ff9700 1 -- 172.21.2.116:0/3678906179 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f12e406b610 con 0x7f12ec0af360 2021-11-21T21:56:01.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eb7fe700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12cc053580 0x7f12cc055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:01.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.296+0000 7f12eb7fe700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12cc053580 0x7f12cc055a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f12e000b000 tx=0x7f12e0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:01.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.300+0000 7f12e8ff9700 1 -- 172.21.2.116:0/3678906179 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12e40429f0 con 0x7f12ec0af360 2021-11-21T21:56:01.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:01.412+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f12ec0573d0 con 0x7f12ec0af360 2021-11-21T21:56:02.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12e8ff9700 1 -- 172.21.2.116:0/3678906179 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v81) v1 ==== 139+0+0 (secure 0 0 0) 0x7f12e4046010 con 0x7f12ec0af360 2021-11-21T21:56:02.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12cc053580 msgr2=0x7f12cc055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:02.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12cc053580 0x7f12cc055a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f12e000b000 tx=0x7f12e0006df0).stop 2021-11-21T21:56:02.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0af360 msgr2=0x7f12ec0664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:02.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0af360 0x7f12ec0664c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f12e4001550 tx=0x7f12e4007f40).stop 2021-11-21T21:56:02.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 shutdown_connections 2021-11-21T21:56:02.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12ec0b24a0 0x7f12ec155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:02.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12cc053580 0x7f12cc055a40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:02.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12ec0af360 0x7f12ec0664c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:02.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 --2- 172.21.2.116:0/3678906179 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12ec0aed90 0x7f12ec13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:02.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 >> 172.21.2.116:0/3678906179 conn(0x7f12ec05fd00 msgr2=0x7f12ec0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:02.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 shutdown_connections 2021-11-21T21:56:02.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.088+0000 7f12f1ac9700 1 -- 172.21.2.116:0/3678906179 wait complete. 2021-11-21T21:56:02.096 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-11-21T21:56:02.097 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2021-11-21T21:56:02.098 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2021-11-21T21:56:02.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.148+0000 7f0a5a192700 1 Processor -- start 2021-11-21T21:56:02.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.148+0000 7f0a5a192700 1 -- start start 2021-11-21T21:56:02.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a540a6aa0 0x7f0a5414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:02.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0a540a86b0 0x7f0a541470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:02.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a540a6e80 con 0x7f0a541475d0 2021-11-21T21:56:02.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540a8bc0 con 0x7f0a540a6aa0 2021-11-21T21:56:02.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5414a7b0 con 0x7f0a540a86b0 2021-11-21T21:56:02.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a540a6aa0 0x7f0a5414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:02.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a540a6aa0 0x7f0a5414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46438/0 (socket says 172.21.2.116:46438) 2021-11-21T21:56:02.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 -- 172.21.2.116:0/366683638 learned_addr learned my addr 172.21.2.116:0/366683638 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:02.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 -- 172.21.2.116:0/366683638 >> v2:172.21.2.117:3300/0 conn(0x7f0a540a86b0 msgr2=0x7f0a541470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:02.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 --2- 172.21.2.116:0/366683638 >> v2:172.21.2.117:3300/0 conn(0x7f0a540a86b0 0x7f0a541470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:02.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 -- 172.21.2.116:0/366683638 >> v1:172.21.2.117:6789/0 conn(0x7f0a541475d0 legacy=0x7f0a541443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:02.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 -- 172.21.2.116:0/366683638 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5414aa10 con 0x7f0a540a6aa0 2021-11-21T21:56:02.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 --2- 172.21.2.116:0/366683638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a540a6aa0 0x7f0a5414b150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f0a54061ab0 tx=0x7f0a3c006eb0).ready entity=mon.0 client_cookie=b5b0249827a7237e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:02.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a527fc700 1 -- 172.21.2.116:0/366683638 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a3c00d5c0 con 0x7f0a540a6aa0 2021-11-21T21:56:02.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a527fc700 1 -- 172.21.2.116:0/366683638 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c000e90 con 0x7f0a540a6aa0 2021-11-21T21:56:02.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a527fc700 1 -- 172.21.2.116:0/366683638 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a3c00d910 con 0x7f0a540a6aa0 2021-11-21T21:56:02.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- 172.21.2.116:0/366683638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a540a6aa0 msgr2=0x7f0a5414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:02.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- 172.21.2.116:0/366683638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a540a6aa0 0x7f0a5414b150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f0a54061ab0 tx=0x7f0a3c006eb0).stop 2021-11-21T21:56:02.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- 172.21.2.116:0/366683638 shutdown_connections 2021-11-21T21:56:02.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- 172.21.2.116:0/366683638 >> v2:172.21.2.117:3300/0 conn(0x7f0a540a86b0 0x7f0a541470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:02.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- 172.21.2.116:0/366683638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a540a6aa0 0x7f0a5414b150 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:02.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- 172.21.2.116:0/366683638 >> 172.21.2.116:0/366683638 conn(0x7f0a5405bd20 msgr2=0x7f0a5405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:02.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- 172.21.2.116:0/366683638 shutdown_connections 2021-11-21T21:56:02.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- 172.21.2.116:0/366683638 wait complete. 2021-11-21T21:56:02.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 Processor -- start 2021-11-21T21:56:02.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- start start 2021-11-21T21:56:02.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a540a6aa0 0x7f0a540a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:02.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a540a86b0 0x7f0a54064730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:02.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a541475d0 0x7f0a5415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:02.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54149c40 con 0x7f0a541475d0 2021-11-21T21:56:02.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54064c20 con 0x7f0a540a86b0 2021-11-21T21:56:02.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54159bf0 con 0x7f0a540a6aa0 2021-11-21T21:56:02.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a541475d0 0x7f0a5415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:02.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a541475d0 0x7f0a5415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46446/0 (socket says 172.21.2.116:46446) 2021-11-21T21:56:02.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 -- 172.21.2.116:0/1458209278 learned_addr learned my addr 172.21.2.116:0/1458209278 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:02.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a537fe700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a540a6aa0 0x7f0a540a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:02.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a52ffd700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a540a86b0 0x7f0a54064730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:02.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 -- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a540a6aa0 msgr2=0x7f0a540a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:02.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a540a6aa0 0x7f0a540a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:02.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 -- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a540a86b0 msgr2=0x7f0a54064730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:02.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a540a86b0 0x7f0a54064730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:02.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 -- 172.21.2.116:0/1458209278 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c00b1e0 con 0x7f0a541475d0 2021-11-21T21:56:02.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a52ffd700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a540a86b0 0x7f0a54064730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:56:02.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a53fff700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a541475d0 0x7f0a5415a2c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f0a54064680 tx=0x7f0a4c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:02.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a50ff9700 1 -- 172.21.2.116:0/1458209278 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4c00d4a0 con 0x7f0a541475d0 2021-11-21T21:56:02.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a50ff9700 1 -- 172.21.2.116:0/1458209278 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c00d600 con 0x7f0a541475d0 2021-11-21T21:56:02.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a54064e40 con 0x7f0a541475d0 2021-11-21T21:56:02.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a54159e20 con 0x7f0a541475d0 2021-11-21T21:56:02.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.152+0000 7f0a50ff9700 1 -- 172.21.2.116:0/1458209278 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4c00d960 con 0x7f0a541475d0 2021-11-21T21:56:02.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.156+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a540573d0 con 0x7f0a541475d0 2021-11-21T21:56:02.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.156+0000 7f0a50ff9700 1 -- 172.21.2.116:0/1458209278 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0a4c018030 con 0x7f0a541475d0 2021-11-21T21:56:02.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.156+0000 7f0a50ff9700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a380536f0 0x7f0a38055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:02.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.156+0000 7f0a50ff9700 1 -- 172.21.2.116:0/1458209278 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0a4c06b0b0 con 0x7f0a541475d0 2021-11-21T21:56:02.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.156+0000 7f0a537fe700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a380536f0 0x7f0a38055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:02.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.156+0000 7f0a537fe700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a380536f0 0x7f0a38055bb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f0a3c017000 tx=0x7f0a3c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:02.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.156+0000 7f0a50ff9700 1 -- 172.21.2.116:0/1458209278 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a4c042430 con 0x7f0a541475d0 2021-11-21T21:56:02.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:02.268+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f0a540b46e0 con 0x7f0a541475d0 2021-11-21T21:56:03.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a50ff9700 1 -- 172.21.2.116:0/1458209278 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v82) v1 ==== 147+0+0 (secure 0 0 0) 0x7f0a4c045a50 con 0x7f0a541475d0 2021-11-21T21:56:03.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a380536f0 msgr2=0x7f0a38055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a380536f0 0x7f0a38055bb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f0a3c017000 tx=0x7f0a3c015000).stop 2021-11-21T21:56:03.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a541475d0 msgr2=0x7f0a5415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a541475d0 0x7f0a5415a2c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f0a54064680 tx=0x7f0a4c0090f0).stop 2021-11-21T21:56:03.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 shutdown_connections 2021-11-21T21:56:03.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a380536f0 0x7f0a38055bb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a541475d0 0x7f0a5415a2c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a540a86b0 0x7f0a54064730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 --2- 172.21.2.116:0/1458209278 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a540a6aa0 0x7f0a540a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 >> 172.21.2.116:0/1458209278 conn(0x7f0a5405bd20 msgr2=0x7f0a54148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:03.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 shutdown_connections 2021-11-21T21:56:03.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.096+0000 7f0a5a192700 1 -- 172.21.2.116:0/1458209278 wait complete. 2021-11-21T21:56:03.104 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T21:56:03.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74ee33700 1 Processor -- start 2021-11-21T21:56:03.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74ee33700 1 -- start start 2021-11-21T21:56:03.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74ee33700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc748062e60 0x7fc748142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74ee33700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7480afae0 con 0x7fc7480ad470 2021-11-21T21:56:03.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74ee33700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7480b0f60 con 0x7fc7481426e0 2021-11-21T21:56:03.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74ee33700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc748063470 con 0x7fc748062e60 2021-11-21T21:56:03.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74d3d0700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc7480ad470 0x7fc7480ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58412/0 (socket says 172.21.2.116:58412) 2021-11-21T21:56:03.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc74d3d0700 1 -- 172.21.2.116:0/1468508941 learned_addr learned my addr 172.21.2.116:0/1468508941 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:03.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc747fff700 1 --2- 172.21.2.116:0/1468508941 >> v2:172.21.2.117:3300/0 conn(0x7fc748062e60 0x7fc748142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394455040 0 0) 0x7fc7480afae0 con 0x7fc7480ad470 2021-11-21T21:56:03.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.156+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc728004370 con 0x7fc7480ad470 2021-11-21T21:56:03.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597420901 0 0) 0x7fc728004370 con 0x7fc7480ad470 2021-11-21T21:56:03.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 >> v2:172.21.2.117:3300/0 conn(0x7fc748062e60 msgr2=0x7fc748142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:03.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 --2- 172.21.2.116:0/1468508941 >> v2:172.21.2.117:3300/0 conn(0x7fc748062e60 0x7fc748142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 >> v1:172.21.2.117:6789/0 conn(0x7fc7481426e0 legacy=0x7fc7480af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc748143050 con 0x7fc7480ad470 2021-11-21T21:56:03.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fc7400019e0 con 0x7fc7480ad470 2021-11-21T21:56:03.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 >> v1:172.21.2.116:6789/0 conn(0x7fc7480ad470 legacy=0x7fc7480ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280048b0 0x7fc728004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc728005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728005f80 0x7fc728003180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc748063470 con 0x7fc728005610 2021-11-21T21:56:03.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280037d0 con 0x7fc728005f80 2021-11-21T21:56:03.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc728003990 con 0x7fc7280048b0 2021-11-21T21:56:03.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc728005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280048b0 0x7fc728004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74d3d0700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728005f80 0x7fc728003180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 -- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280048b0 msgr2=0x7fc728004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:03.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280048b0 0x7fc728004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 -- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728005f80 msgr2=0x7fc728003180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:03.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728005f80 0x7fc728003180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74d3d0700 1 -- 172.21.2.116:0/1468508941 reap_dead start 2021-11-21T21:56:03.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 -- 172.21.2.116:0/1468508941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728004370 con 0x7fc728005610 2021-11-21T21:56:03.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc728005a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fc73800c000 tx=0x7fc738006df0).ready entity=mon.0 client_cookie=22ab6be8bdc5d191 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc73800e830 con 0x7fc728005610 2021-11-21T21:56:03.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73800e990 con 0x7fc728005610 2021-11-21T21:56:03.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc7477fe700 1 -- 172.21.2.116:0/1468508941 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc73800ecf0 con 0x7fc728005610 2021-11-21T21:56:03.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1468508941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 msgr2=0x7fc728005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc728005a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fc73800c000 tx=0x7fc738006df0).stop 2021-11-21T21:56:03.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1468508941 shutdown_connections 2021-11-21T21:56:03.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1468508941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc728005a90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1468508941 >> 172.21.2.116:0/1468508941 conn(0x7fc7480b5730 msgr2=0x7fc7480b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:03.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1468508941 shutdown_connections 2021-11-21T21:56:03.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1468508941 wait complete. 2021-11-21T21:56:03.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 Processor -- start 2021-11-21T21:56:03.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- start start 2021-11-21T21:56:03.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc748141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc748066f30 0x7fc74814dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc74814e0c0 0x7fc74813ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc748066250 con 0x7fc728005610 2021-11-21T21:56:03.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc748066480 con 0x7fc748066f30 2021-11-21T21:56:03.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc748066720 con 0x7fc74814e0c0 2021-11-21T21:56:03.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc748141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc748141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46464/0 (socket says 172.21.2.116:46464) 2021-11-21T21:56:03.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 -- 172.21.2.116:0/1573172289 learned_addr learned my addr 172.21.2.116:0/1573172289 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:03.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74d3d0700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc74814e0c0 0x7fc74813ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc747fff700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc748066f30 0x7fc74814dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 -- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc74814e0c0 msgr2=0x7fc74813ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc74814e0c0 0x7fc74813ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 -- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc748066f30 msgr2=0x7fc74814dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc748066f30 0x7fc74814dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 -- 172.21.2.116:0/1573172289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480669c0 con 0x7fc728005610 2021-11-21T21:56:03.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74cbcf700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc748141d40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fc73000d000 tx=0x7fc730007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc745ffb700 1 -- 172.21.2.116:0/1573172289 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc730014860 con 0x7fc728005610 2021-11-21T21:56:03.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc745ffb700 1 -- 172.21.2.116:0/1573172289 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7300149c0 con 0x7fc728005610 2021-11-21T21:56:03.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74813e210 con 0x7fc728005610 2021-11-21T21:56:03.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74814e540 con 0x7fc728005610 2021-11-21T21:56:03.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc745ffb700 1 -- 172.21.2.116:0/1573172289 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc730014d20 con 0x7fc728005610 2021-11-21T21:56:03.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc748066b50 con 0x7fc728005610 2021-11-21T21:56:03.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc745ffb700 1 -- 172.21.2.116:0/1573172289 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc73001c3c0 con 0x7fc728005610 2021-11-21T21:56:03.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.160+0000 7fc745ffb700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc72c053640 0x7fc72c055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.164+0000 7fc745ffb700 1 -- 172.21.2.116:0/1573172289 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc73006f850 con 0x7fc728005610 2021-11-21T21:56:03.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.164+0000 7fc747fff700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc72c053640 0x7fc72c055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.164+0000 7fc747fff700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc72c053640 0x7fc72c055b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc73801d000 tx=0x7fc738019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.164+0000 7fc745ffb700 1 -- 172.21.2.116:0/1573172289 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc730046c30 con 0x7fc728005610 2021-11-21T21:56:03.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.248+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc7480b1e00 con 0x7fc728005610 2021-11-21T21:56:03.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc745ffb700 1 -- 172.21.2.116:0/1573172289 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc73004a250 con 0x7fc728005610 2021-11-21T21:56:03.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc72c053640 msgr2=0x7fc72c055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc72c053640 0x7fc72c055b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc73801d000 tx=0x7fc738019000).stop 2021-11-21T21:56:03.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 msgr2=0x7fc748141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc748141d40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fc73000d000 tx=0x7fc730007a00).stop 2021-11-21T21:56:03.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 shutdown_connections 2021-11-21T21:56:03.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc74814e0c0 0x7fc74813ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc748066f30 0x7fc74814dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc72c053640 0x7fc72c055b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 --2- 172.21.2.116:0/1573172289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728005610 0x7fc748141d40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 >> 172.21.2.116:0/1573172289 conn(0x7fc7480b5730 msgr2=0x7fc7480b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:03.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 shutdown_connections 2021-11-21T21:56:03.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.252+0000 7fc74ee33700 1 -- 172.21.2.116:0/1573172289 wait complete. 2021-11-21T21:56:03.261 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:56:03.262 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:56:03.270 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:56:03.279 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:56:03.293 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:56:03.308 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:56:03.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:03.309 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:56:03.310 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:03.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:56:03.315 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T21:56:03.316 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T21:56:03.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:03.317 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:56:03.317 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:56:03.356 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:56:03.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:56:03.357 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:56:03.358 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:56:03.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 Processor -- start 2021-11-21T21:56:03.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- start start 2021-11-21T21:56:03.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f63f81443a0 0x7f63f8144810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f80b29b0 con 0x7f63f80b3320 2021-11-21T21:56:03.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f8067c30 con 0x7f63f8067850 2021-11-21T21:56:03.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f80ab720 con 0x7f63f81443a0 2021-11-21T21:56:03.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fdd69700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f63f80b3320 0x7f63f80ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58428/0 (socket says 172.21.2.116:58428) 2021-11-21T21:56:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fdd69700 1 -- 172.21.2.116:0/1888515969 learned_addr learned my addr 172.21.2.116:0/1888515969 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 --2- 172.21.2.116:0/1888515969 >> v2:172.21.2.116:3300/0 conn(0x7f63f81443a0 0x7f63f8144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 -- 172.21.2.116:0/1888515969 >> v1:172.21.2.117:6789/0 conn(0x7f63f8067850 legacy=0x7f63f80b7070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 -- 172.21.2.116:0/1888515969 >> v1:172.21.2.116:6789/0 conn(0x7f63f80b3320 legacy=0x7f63f80ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 -- 172.21.2.116:0/1888515969 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80b3780 con 0x7f63f81443a0 2021-11-21T21:56:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 --2- 172.21.2.116:0/1888515969 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f8144810 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f63f8064400 tx=0x7f63f00091d0).ready entity=mon.0 client_cookie=979c29e13c94e6be server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fd568700 1 -- 172.21.2.116:0/1888515969 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f0009ea0 con 0x7f63f81443a0 2021-11-21T21:56:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fd568700 1 -- 172.21.2.116:0/1888515969 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f000b830 con 0x7f63f81443a0 2021-11-21T21:56:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fd568700 1 -- 172.21.2.116:0/1888515969 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f000b9a0 con 0x7f63f81443a0 2021-11-21T21:56:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- 172.21.2.116:0/1888515969 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 msgr2=0x7f63f8144810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 --2- 172.21.2.116:0/1888515969 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f8144810 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f63f8064400 tx=0x7f63f00091d0).stop 2021-11-21T21:56:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- 172.21.2.116:0/1888515969 shutdown_connections 2021-11-21T21:56:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 --2- 172.21.2.116:0/1888515969 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f8144810 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- 172.21.2.116:0/1888515969 >> 172.21.2.116:0/1888515969 conn(0x7f63f8064d60 msgr2=0x7f63f8065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:03.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- 172.21.2.116:0/1888515969 shutdown_connections 2021-11-21T21:56:03.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- 172.21.2.116:0/1888515969 wait complete. 2021-11-21T21:56:03.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 Processor -- start 2021-11-21T21:56:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- start start 2021-11-21T21:56:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63f8067850 0x7f63f80630a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63f80b3320 0x7f63f80af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80b0220 con 0x7f63f81443a0 2021-11-21T21:56:03.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80af410 con 0x7f63f8067850 2021-11-21T21:56:03.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f64007ce700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80a2020 con 0x7f63f80b3320 2021-11-21T21:56:03.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f80afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46482/0 (socket says 172.21.2.116:46482) 2021-11-21T21:56:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 -- 172.21.2.116:0/4160691867 learned_addr learned my addr 172.21.2.116:0/4160691867 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 -- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63f80b3320 msgr2=0x7f63f80af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63f80b3320 0x7f63f80af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:03.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 -- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63f8067850 msgr2=0x7f63f80630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63f8067850 0x7f63f80630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.416+0000 7f63fed6b700 1 -- 172.21.2.116:0/4160691867 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0009470 con 0x7f63f81443a0 2021-11-21T21:56:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63fed6b700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f80afd30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f63f8064400 tx=0x7f63f0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63eb7fe700 1 -- 172.21.2.116:0/4160691867 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f0009ea0 con 0x7f63f81443a0 2021-11-21T21:56:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63eb7fe700 1 -- 172.21.2.116:0/4160691867 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f001f5e0 con 0x7f63f81443a0 2021-11-21T21:56:03.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f80a22c0 con 0x7f63f81443a0 2021-11-21T21:56:03.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f809e920 con 0x7f63f81443a0 2021-11-21T21:56:03.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63eb7fe700 1 -- 172.21.2.116:0/4160691867 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f001f940 con 0x7f63f81443a0 2021-11-21T21:56:03.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f80b7910 con 0x7f63f81443a0 2021-11-21T21:56:03.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63eb7fe700 1 -- 172.21.2.116:0/4160691867 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f63f0026030 con 0x7f63f81443a0 2021-11-21T21:56:03.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63eb7fe700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63d80536d0 0x7f63d8055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63eb7fe700 1 -- 172.21.2.116:0/4160691867 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f63f006c320 con 0x7f63f81443a0 2021-11-21T21:56:03.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63fe56a700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63d80536d0 0x7f63d8055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63fe56a700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63d80536d0 0x7f63d8055b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f63ec002fb0 tx=0x7f63ec0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.420+0000 7f63eb7fe700 1 -- 172.21.2.116:0/4160691867 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f63f0043760 con 0x7f63f81443a0 2021-11-21T21:56:03.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.504+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f63f80573d0 con 0x7f63f81443a0 2021-11-21T21:56:03.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.504+0000 7f63eb7fe700 1 -- 172.21.2.116:0/4160691867 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v36) v1 ==== 81+0+28827 (secure 0 0 0) 0x7f63f0015090 con 0x7f63f81443a0 2021-11-21T21:56:03.507 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:03.507 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:56:03.508 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 36, 2021-11-21T21:56:03.508 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:56:03.508 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:56:03.508 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:56:01.062329+0000", 2021-11-21T21:56:03.508 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:56:03.509 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:56:03.509 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:56:03.509 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:56:03.509 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:56:03.510 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:56:03.510 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:56:03.510 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:56:03.510 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:56:03.510 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:56:03.511 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:56:03.511 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:56:03.511 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:56:03.511 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:56:03.512 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:56:03.512 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 6, 2021-11-21T21:56:03.512 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:56:03.512 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:56:03.512 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:56:03.513 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:56:03.513 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:56:03.513 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.513 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:56:03.513 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:56:03.514 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:56:03.514 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:56:03.514 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:56:03.514 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:56:03.515 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:56:03.515 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:56:03.515 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:56:03.515 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:56:03.515 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:56:03.516 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:56:03.516 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:56:03.516 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:56:03.516 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:56:03.516 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:56:03.517 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:56:03.517 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:56:03.517 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:56:03.518 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:56:03.518 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:56:03.518 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:56:03.518 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:56:03.519 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.519 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:56:03.519 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:56:03.519 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:56:03.519 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:56:03.520 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:56:03.520 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:56:03.520 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:56:03.520 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:56:03.520 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:56:03.521 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:56:03.521 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:56:03.521 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:56:03.521 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:56:03.521 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:56:03.522 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:56:03.522 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:56:03.522 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:56:03.522 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:56:03.526 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:56:03.527 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:56:03.527 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:56:03.527 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:56:03.527 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:56:03.528 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:56:03.528 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:56:03.528 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:56:03.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:56:03.529 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.529 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:56:03.529 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:56:03.529 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:56:03.529 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:56:03.530 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:56:03.530 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:56:03.530 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:56:03.530 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:56:03.531 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:56:03.531 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:56:03.531 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:56:03.531 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:56:03.532 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:56:03.532 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.532 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:56:03.532 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:56:03.532 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.533 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.533 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.533 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:56:03.533 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:56:03.534 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:56:03.534 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:56:03.534 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:56:03.534 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:56:03.534 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:56:03.535 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:56:03.535 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:56:03.535 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:56:03.535 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:56:03.536 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:56:03.536 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:56:03.536 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:56:03.536 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:56:03.536 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:56:03.537 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:56:03.537 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:56:03.537 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:56:03.537 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:56:03.538 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:56:03.538 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:56:03.538 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:56:03.538 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.538 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:56:03.539 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:56:03.539 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:56:03.539 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:56:03.539 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:56:03.540 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:56:03.540 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:56:03.540 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:56:03.541 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:56:03.541 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:56:03.541 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:56:03.541 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:56:03.541 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:56:03.542 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:56:03.542 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:56:03.542 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:56:03.542 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:56:03.543 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:56:03.543 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:56:03.543 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:56:03.543 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:56:03.544 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:56:03.544 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:56:03.544 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:56:03.544 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:56:03.545 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:56:03.545 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:56:03.545 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.545 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:56:03.545 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:56:03.546 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:56:03.546 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:56:03.546 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:56:03.546 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:56:03.547 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:56:03.547 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:56:03.547 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:56:03.547 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:56:03.547 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:56:03.548 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:56:03.548 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:56:03.548 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:56:03.548 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.549 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.549 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:56:03.549 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:56:03.549 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.550 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:56:03.550 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:56:03.550 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.551 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.551 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.551 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.551 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.551 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.552 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.552 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:56:03.552 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.552 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.553 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.553 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.553 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.553 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.553 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:56:03.554 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.554 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.554 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.554 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.554 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:56:03.555 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.555 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.555 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.555 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.556 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.556 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.556 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.556 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.557 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:56:03.557 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.557 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.557 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.557 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.558 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:56:03.558 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.558 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.558 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.559 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.559 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.559 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.559 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.560 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.560 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:56:03.560 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.560 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.560 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.561 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.561 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:56:03.561 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.561 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.561 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.562 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.562 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.562 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.562 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.563 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.563 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:56:03.563 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.563 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.563 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.564 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.564 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:56:03.564 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:56:03.564 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.565 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.565 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.565 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:56:03.565 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:56:03.566 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:56:03.566 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:56:03.566 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.566 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.566 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.567 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.567 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.567 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:56:03.568 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:56:03.568 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.568 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.568 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.568 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.569 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.569 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.569 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.569 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 31, 2021-11-21T21:56:03.570 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.570 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.570 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.570 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.571 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.571 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.571 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:56:03.571 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.572 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.572 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.572 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.572 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:56:03.572 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.573 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.573 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.573 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.573 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.573 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.574 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.574 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.574 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:56:03.574 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.575 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.575 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.575 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.575 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:56:03.575 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.576 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.576 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.576 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.576 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.577 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.577 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.577 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.577 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:56:03.577 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.578 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.578 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.578 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.578 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:56:03.578 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.579 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.579 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.579 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.579 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.580 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.580 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.580 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.581 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:56:03.581 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.581 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.582 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.582 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.582 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:56:03.582 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:56:03.582 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.583 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.583 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.583 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:56:03.583 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:56:03.584 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:56:03.584 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:56:03.584 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.584 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.584 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.585 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.585 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.585 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.585 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:56:03.586 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:56:03.586 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.586 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.586 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.586 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.587 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.587 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.587 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.587 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 27, 2021-11-21T21:56:03.588 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.588 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.588 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.588 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.588 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.589 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.589 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:56:03.589 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.589 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.589 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.590 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.590 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:56:03.590 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.590 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.591 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.591 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.591 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.591 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.591 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.592 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.592 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:56:03.592 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.592 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.593 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.593 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.593 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:56:03.593 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.593 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.594 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.594 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.594 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.594 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.595 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.595 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.595 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:56:03.595 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.595 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.596 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.596 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.596 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:56:03.597 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.597 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.597 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.597 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.597 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.598 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.598 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.598 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.598 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:56:03.599 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.599 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.599 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.599 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.599 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:56:03.600 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:56:03.600 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.600 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.600 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.601 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:56:03.601 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:56:03.601 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:56:03.601 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:56:03.601 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.602 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.602 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.602 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.602 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.603 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.603 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:56:03.603 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:56:03.603 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.603 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.604 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.604 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.604 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.604 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.604 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.605 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:56:03.605 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.605 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.605 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.606 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.606 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.606 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.606 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:56:03.606 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.607 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.607 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.607 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.607 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:56:03.607 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.608 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.608 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.608 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.608 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.609 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.609 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.609 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.609 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:56:03.609 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.610 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.610 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.610 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.610 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:56:03.611 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.611 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.611 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.611 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.611 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.612 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.612 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.612 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.612 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:56:03.612 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.613 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.613 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.613 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.613 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:56:03.614 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.614 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.614 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.614 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.614 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.615 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.615 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.615 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.615 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:56:03.615 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.616 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.616 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.616 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.616 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:56:03.617 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:56:03.617 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.617 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.617 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.617 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:56:03.618 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:56:03.618 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:56:03.618 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:56:03.618 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.619 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.619 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.619 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.619 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.619 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.620 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:56:03.620 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:56:03.620 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.620 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.621 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.621 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.621 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.621 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.621 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.622 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:56:03.622 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.622 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.622 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.622 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.623 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.623 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.623 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:56:03.623 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.624 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.624 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.624 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.624 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:56:03.624 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.625 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.625 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.625 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.625 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.626 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.626 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.626 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.626 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:56:03.626 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.627 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.627 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.628 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.628 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:56:03.628 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.628 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.629 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.629 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.629 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.629 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.629 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.630 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.630 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:56:03.630 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.630 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.631 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.631 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.631 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:56:03.631 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.631 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.632 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.632 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.632 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.632 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.633 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.633 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.633 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:56:03.633 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.633 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.634 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.634 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.634 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:56:03.634 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:56:03.635 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.635 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.635 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.635 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:56:03.635 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:56:03.636 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:56:03.636 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:56:03.636 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.636 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.636 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.637 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.637 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.637 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.637 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:56:03.638 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:56:03.638 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.638 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.638 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.638 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.639 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.639 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.639 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.639 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:56:03.639 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.640 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.640 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.640 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.640 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.641 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.641 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:56:03.641 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.641 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.641 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.642 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.642 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:56:03.642 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.642 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.642 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.643 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.643 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.643 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.643 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.644 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.644 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:56:03.644 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.644 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.644 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.645 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.645 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:56:03.645 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.645 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.645 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.646 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.646 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.646 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.646 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.647 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.647 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:56:03.647 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.647 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.647 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.648 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.648 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:56:03.648 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.648 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.648 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.649 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.649 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.649 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.649 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.650 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.650 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:56:03.650 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.650 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.651 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.651 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.651 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:56:03.651 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:56:03.652 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.652 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.652 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.652 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:56:03.652 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:56:03.653 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:56:03.653 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:56:03.653 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.653 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.654 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.654 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.654 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.654 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.655 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:56:03.655 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:56:03.655 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.655 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.655 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.656 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.656 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.656 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.656 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.656 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 27, 2021-11-21T21:56:03.657 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.657 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.657 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.657 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.658 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.658 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.658 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:56:03.658 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.658 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.659 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.659 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.659 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:56:03.659 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.660 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.660 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.660 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.660 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.660 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.661 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.661 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.661 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:56:03.661 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.662 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.662 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.662 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.662 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:56:03.662 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.663 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.663 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.663 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.663 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.663 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.664 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.664 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.664 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:56:03.664 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.665 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.665 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.665 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.665 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:56:03.666 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.666 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.666 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.666 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.666 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.667 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.667 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.667 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.667 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:56:03.667 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.668 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.668 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.668 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.668 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:56:03.669 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:56:03.669 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.669 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.669 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.669 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:56:03.670 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:56:03.670 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:56:03.670 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:56:03.670 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.670 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.671 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.671 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.671 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.671 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.672 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:56:03.672 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:56:03.672 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:56:03.672 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:56:03.672 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:56:03.673 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:56:03.673 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:56:03.673 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:56:03.673 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:56:03.673 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 30, 2021-11-21T21:56:03.674 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:56:03.674 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:56:03.674 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:56:03.674 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.675 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.675 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.675 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:56:03.675 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.675 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.676 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.676 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.676 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:56:03.676 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.677 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.677 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.677 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.677 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:56:03.677 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.678 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.678 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.678 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:56:03.678 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.678 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.679 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.679 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.679 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:56:03.679 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.679 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.680 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.680 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.680 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:56:03.680 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.681 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.681 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.681 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:56:03.681 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.681 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.682 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.682 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.682 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:56:03.682 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.683 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.683 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.683 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.683 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:56:03.683 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:56:03.684 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.684 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:56:03.684 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:56:03.684 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.685 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.685 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.685 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:56:03.685 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:56:03.685 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:56:03.686 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.686 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.686 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.686 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:56:03.687 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:56:03.687 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:56:03.687 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:56:03.687 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:56:03.687 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:56:03.688 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:56:03.688 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:03.688 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.688 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:56:03.688 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:56:03.689 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.689 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:56:03.689 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.689 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.689 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.690 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.690 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.690 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:56:03.690 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.691 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.691 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.691 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:56:03.691 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.691 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.692 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.692 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.692 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.692 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:56:03.693 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.693 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.693 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.693 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:56:03.693 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.694 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.694 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.694 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.694 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.694 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:56:03.695 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.695 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.695 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.695 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:56:03.696 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.696 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.696 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.696 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.696 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.697 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:56:03.697 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.697 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.697 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.698 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:56:03.698 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.698 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.698 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.698 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.699 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.699 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:56:03.699 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.699 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.699 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.700 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:56:03.700 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.700 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.700 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.701 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.701 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.701 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:56:03.701 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.701 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.702 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.702 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:56:03.702 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.702 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.703 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.703 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.703 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.703 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:56:03.703 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.704 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.704 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:03.704 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:56:03.704 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:56:03.705 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:56:03.705 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:56:03.705 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:56:03.705 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:56:03.705 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:56:03.706 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:56:03.706 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.706 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:56:03.706 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:56:03.707 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:56:03.707 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:56:03.707 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:56:03.707 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T21:56:03.707 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/2051796411": "2021-11-22T21:55:57.571693+0000", 2021-11-21T21:56:03.708 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/2051796411": "2021-11-22T21:55:57.571693+0000" 2021-11-21T21:56:03.708 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.708 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:56:03.708 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:56:03.709 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:56:03.709 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:56:03.709 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:56:03.709 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:56:03.709 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:56:03.710 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:56:03.710 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:03.710 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:03.710 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:56:03.711 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:56:03.711 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:56:03.711 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:56:03.711 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:56:03.711 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:56:03.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63d80536d0 msgr2=0x7f63d8055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63d80536d0 0x7f63d8055b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f63ec002fb0 tx=0x7f63ec0069b0).stop 2021-11-21T21:56:03.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 msgr2=0x7f63f80afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f80afd30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f63f8064400 tx=0x7f63f0016000).stop 2021-11-21T21:56:03.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 shutdown_connections 2021-11-21T21:56:03.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f63d80536d0 0x7f63d8055b90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f63f81443a0 0x7f63f80afd30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f63f80b3320 0x7f63f80af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:03.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 --2- 172.21.2.116:0/4160691867 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f63f8067850 0x7f63f80630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:03.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 >> 172.21.2.116:0/4160691867 conn(0x7f63f8064d60 msgr2=0x7f63f80adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:03.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 shutdown_connections 2021-11-21T21:56:03.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.508+0000 7f64007ce700 1 -- 172.21.2.116:0/4160691867 wait complete. 2021-11-21T21:56:03.716 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:03.717 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:03.731 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:03.743 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:03.750 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:03.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 Processor -- start 2021-11-21T21:56:03.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- start start 2021-11-21T21:56:03.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f96ec0b7070 0x7f96ec0ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec0b29b0 con 0x7f96ec1443a0 2021-11-21T21:56:03.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec067c30 con 0x7f96ec0b7070 2021-11-21T21:56:03.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0b7910 con 0x7f96ec067850 2021-11-21T21:56:03.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eaffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f96ec0b7070 0x7f96ec0ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eb7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eaffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f96ec0b7070 0x7f96ec0ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:47656/0 (socket says 172.21.2.116:47656) 2021-11-21T21:56:03.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eb7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec0b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:49830/0 (socket says 172.21.2.116:49830) 2021-11-21T21:56:03.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eaffd700 1 -- 172.21.2.116:0/4082475054 learned_addr learned my addr 172.21.2.116:0/4082475054 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:03.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96ebfff700 1 --1- 172.21.2.116:0/4082475054 >> v1:172.21.2.117:6789/0 conn(0x7f96ec1443a0 0x7f96ec144830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:57356/0 (socket says 172.21.2.116:57356) 2021-11-21T21:56:03.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eb7fe700 1 -- 172.21.2.116:0/4082475054 >> v2:172.21.2.117:3300/0 conn(0x7f96ec0b7070 msgr2=0x7f96ec0ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:03.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eb7fe700 1 --2- 172.21.2.116:0/4082475054 >> v2:172.21.2.117:3300/0 conn(0x7f96ec0b7070 0x7f96ec0ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eb7fe700 1 -- 172.21.2.116:0/4082475054 >> v1:172.21.2.117:6789/0 conn(0x7f96ec1443a0 legacy=0x7f96ec144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eb7fe700 1 -- 172.21.2.116:0/4082475054 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec0b2be0 con 0x7f96ec067850 2021-11-21T21:56:03.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96eb7fe700 1 --2- 172.21.2.116:0/4082475054 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec0b3320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f96dc002f70 tx=0x7f96dc006df0).ready entity=mon.2 client_cookie=d5045feb505ca791 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96ea7fc700 1 -- 172.21.2.116:0/4082475054 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96dc00d570 con 0x7f96ec067850 2021-11-21T21:56:03.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96ea7fc700 1 -- 172.21.2.116:0/4082475054 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc000e90 con 0x7f96ec067850 2021-11-21T21:56:03.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96ea7fc700 1 -- 172.21.2.116:0/4082475054 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96dc00d8c0 con 0x7f96ec067850 2021-11-21T21:56:03.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- 172.21.2.116:0/4082475054 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 msgr2=0x7f96ec0b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 --2- 172.21.2.116:0/4082475054 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec0b3320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f96dc002f70 tx=0x7f96dc006df0).stop 2021-11-21T21:56:03.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- 172.21.2.116:0/4082475054 shutdown_connections 2021-11-21T21:56:03.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 --2- 172.21.2.116:0/4082475054 >> v2:172.21.2.117:3300/0 conn(0x7f96ec0b7070 0x7f96ec0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 --2- 172.21.2.116:0/4082475054 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec0b3320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- 172.21.2.116:0/4082475054 >> 172.21.2.116:0/4082475054 conn(0x7f96ec064d60 msgr2=0x7f96ec065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:03.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- 172.21.2.116:0/4082475054 shutdown_connections 2021-11-21T21:56:03.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.776+0000 7f96f1c50700 1 -- 172.21.2.116:0/4082475054 wait complete. 2021-11-21T21:56:03.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 Processor -- start 2021-11-21T21:56:03.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 -- start start 2021-11-21T21:56:03.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96ec0b7070 0x7f96ec143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96ec1443a0 0x7f96ec1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec148770 con 0x7f96ec1443a0 2021-11-21T21:56:03.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec147270 con 0x7f96ec0b7070 2021-11-21T21:56:03.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec147510 con 0x7f96ec067850 2021-11-21T21:56:03.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96ec1443a0 0x7f96ec1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96ec1443a0 0x7f96ec1494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46492/0 (socket says 172.21.2.116:46492) 2021-11-21T21:56:03.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 -- 172.21.2.116:0/470722258 learned_addr learned my addr 172.21.2.116:0/470722258 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:03.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96eaffd700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96ec0b7070 0x7f96ec143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96eb7fe700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 -- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 msgr2=0x7f96ec145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 -- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96ec0b7070 msgr2=0x7f96ec143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96ec0b7070 0x7f96ec143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 -- 172.21.2.116:0/470722258 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc00b180 con 0x7f96ec1443a0 2021-11-21T21:56:03.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96ebfff700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96ec1443a0 0x7f96ec1494c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f96ec066f90 tx=0x7f96e4007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96e8ff9700 1 -- 172.21.2.116:0/470722258 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e4005ae0 con 0x7f96ec1443a0 2021-11-21T21:56:03.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96e8ff9700 1 -- 172.21.2.116:0/470722258 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4005c40 con 0x7f96ec1443a0 2021-11-21T21:56:03.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec1477b0 con 0x7f96ec1443a0 2021-11-21T21:56:03.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec0aa610 con 0x7f96ec1443a0 2021-11-21T21:56:03.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96e8ff9700 1 -- 172.21.2.116:0/470722258 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e40070d0 con 0x7f96ec1443a0 2021-11-21T21:56:03.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec0573d0 con 0x7f96ec1443a0 2021-11-21T21:56:03.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96e8ff9700 1 -- 172.21.2.116:0/470722258 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f96e4010070 con 0x7f96ec1443a0 2021-11-21T21:56:03.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96e8ff9700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96cc053610 0x7f96cc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:03.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96e8ff9700 1 -- 172.21.2.116:0/470722258 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f96e406af90 con 0x7f96ec1443a0 2021-11-21T21:56:03.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96eb7fe700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96cc053610 0x7f96cc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:03.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.780+0000 7f96eb7fe700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96cc053610 0x7f96cc055ad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f96dc01a000 tx=0x7f96dc014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:03.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.784+0000 7f96e8ff9700 1 -- 172.21.2.116:0/470722258 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96e40422c0 con 0x7f96ec1443a0 2021-11-21T21:56:03.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96ec068740 con 0x7f96ec1443a0 2021-11-21T21:56:03.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96e8ff9700 1 -- 172.21.2.116:0/470722258 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+1551 (secure 0 0 0) 0x7f96e40458e0 con 0x7f96ec1443a0 2021-11-21T21:56:03.892 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:03.893 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":5157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2140793945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2140793945},{"type":"v1","addr":"172.21.2.116:6835","nonce":2140793945}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13}],"filesystems":[]} 2021-11-21T21:56:03.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96cc053610 msgr2=0x7f96cc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96cc053610 0x7f96cc055ad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f96dc01a000 tx=0x7f96dc014000).stop 2021-11-21T21:56:03.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96ec1443a0 msgr2=0x7f96ec1494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:03.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96ec1443a0 0x7f96ec1494c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f96ec066f90 tx=0x7f96e4007f00).stop 2021-11-21T21:56:03.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 shutdown_connections 2021-11-21T21:56:03.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f96cc053610 0x7f96cc055ad0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96ec1443a0 0x7f96ec1494c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f96ec0b7070 0x7f96ec143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 --2- 172.21.2.116:0/470722258 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96ec067850 0x7f96ec145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:03.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 >> 172.21.2.116:0/470722258 conn(0x7f96ec064d60 msgr2=0x7f96ec14ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:03.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 shutdown_connections 2021-11-21T21:56:03.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:03.892+0000 7f96f1c50700 1 -- 172.21.2.116:0/470722258 wait complete. 2021-11-21T21:56:03.897 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 13 2021-11-21T21:56:03.900 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:04.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f98324d0700 1 Processor -- start 2021-11-21T21:56:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f98324d0700 1 -- start start 2021-11-21T21:56:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f98324d0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f982c062410 0x7f982c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f98324d0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982c0a6e80 con 0x7f982c0a6aa0 2021-11-21T21:56:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f98324d0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982c14a7b0 con 0x7f982c0a86b0 2021-11-21T21:56:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f98324d0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982c0a8bc0 con 0x7f982c062410 2021-11-21T21:56:04.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f982bfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f982c0a6aa0 0x7f982c1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58458/0 (socket says 172.21.2.116:58458) 2021-11-21T21:56:04.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f982bfff700 1 -- 172.21.2.116:0/2782913964 learned_addr learned my addr 172.21.2.116:0/2782913964 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:04.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896043627 0 0) 0x7f982c0a6e80 con 0x7f982c0a6aa0 2021-11-21T21:56:04.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f980c003260 con 0x7f982c0a6aa0 2021-11-21T21:56:04.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f9830a6d700 1 --2- 172.21.2.116:0/2782913964 >> v2:172.21.2.117:3300/0 conn(0x7f982c062410 0x7f982c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.004+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 617130534 0 0) 0x7f980c003260 con 0x7f982c0a6aa0 2021-11-21T21:56:04.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 >> v2:172.21.2.117:3300/0 conn(0x7f982c062410 msgr2=0x7f982c144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 --2- 172.21.2.116:0/2782913964 >> v2:172.21.2.117:3300/0 conn(0x7f982c062410 0x7f982c144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 >> v1:172.21.2.117:6789/0 conn(0x7f982c0a86b0 legacy=0x7f982c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c14aa10 con 0x7f982c0a6aa0 2021-11-21T21:56:04.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9814001680 con 0x7f982c0a6aa0 2021-11-21T21:56:04.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 >> v1:172.21.2.116:6789/0 conn(0x7f982c0a6aa0 legacy=0x7f982c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f980c003820 0x7f980c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f980c004500 0x7f980c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f980c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0a8bc0 con 0x7f980c004f70 2021-11-21T21:56:04.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c00be50 con 0x7f980c003820 2021-11-21T21:56:04.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c00c010 con 0x7f980c004500 2021-11-21T21:56:04.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f980c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f980c003820 0x7f980c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982b7fe700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f980c004500 0x7f980c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 -- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f980c004500 msgr2=0x7f980c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f980c004500 0x7f980c004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 -- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f980c003820 msgr2=0x7f980c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f980c003820 0x7f980c003c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 -- 172.21.2.116:0/2782913964 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c003260 con 0x7f980c004f70 2021-11-21T21:56:04.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 -- 172.21.2.116:0/2782913964 reap_dead start 2021-11-21T21:56:04.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f9830a6d700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f980c00b750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f982400a000 tx=0x7f9824008040).ready entity=mon.0 client_cookie=21848678811f2e03 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:04.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982400c070 con 0x7f980c004f70 2021-11-21T21:56:04.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982400e810 con 0x7f980c004f70 2021-11-21T21:56:04.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982affd700 1 -- 172.21.2.116:0/2782913964 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982400eb70 con 0x7f980c004f70 2021-11-21T21:56:04.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- 172.21.2.116:0/2782913964 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 msgr2=0x7f980c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f980c00b750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f982400a000 tx=0x7f9824008040).stop 2021-11-21T21:56:04.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- 172.21.2.116:0/2782913964 shutdown_connections 2021-11-21T21:56:04.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 --2- 172.21.2.116:0/2782913964 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f980c00b750 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- 172.21.2.116:0/2782913964 >> 172.21.2.116:0/2782913964 conn(0x7f982c05bd20 msgr2=0x7f982c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:04.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- 172.21.2.116:0/2782913964 shutdown_connections 2021-11-21T21:56:04.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- 172.21.2.116:0/2782913964 wait complete. 2021-11-21T21:56:04.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 Processor -- start 2021-11-21T21:56:04.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- start start 2021-11-21T21:56:04.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f982c13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f982c13eb60 0x7f982c13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f982c13b750 0x7f982c13db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c13f420 con 0x7f980c004f70 2021-11-21T21:56:04.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c13b050 con 0x7f982c13eb60 2021-11-21T21:56:04.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982c13cae0 con 0x7f982c13b750 2021-11-21T21:56:04.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f982c13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f982c13fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46514/0 (socket says 172.21.2.116:46514) 2021-11-21T21:56:04.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 -- 172.21.2.116:0/2881951474 learned_addr learned my addr 172.21.2.116:0/2881951474 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:04.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982b7fe700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f982c13eb60 0x7f982c13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 -- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f982c13b750 msgr2=0x7f982c13db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f982c13b750 0x7f982c13db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:04.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 -- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f982c13eb60 msgr2=0x7f982c13d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f982c13eb60 0x7f982c13d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 -- 172.21.2.116:0/2881951474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c13f180 con 0x7f980c004f70 2021-11-21T21:56:04.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f982bfff700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f982c13fb80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f9814008000 tx=0x7f9814009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:04.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98297fa700 1 -- 172.21.2.116:0/2881951474 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9814007170 con 0x7f980c004f70 2021-11-21T21:56:04.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98297fa700 1 -- 172.21.2.116:0/2881951474 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140072d0 con 0x7f980c004f70 2021-11-21T21:56:04.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982c13b2b0 con 0x7f980c004f70 2021-11-21T21:56:04.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98297fa700 1 -- 172.21.2.116:0/2881951474 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9814004920 con 0x7f980c004f70 2021-11-21T21:56:04.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.008+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982c13e760 con 0x7f980c004f70 2021-11-21T21:56:04.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.012+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982c13b440 con 0x7f980c004f70 2021-11-21T21:56:04.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.012+0000 7f98297fa700 1 -- 172.21.2.116:0/2881951474 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f981401b060 con 0x7f980c004f70 2021-11-21T21:56:04.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.012+0000 7f98297fa700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9810053610 0x7f9810055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.012+0000 7f98297fa700 1 -- 172.21.2.116:0/2881951474 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f981406b1b0 con 0x7f980c004f70 2021-11-21T21:56:04.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.012+0000 7f982b7fe700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9810053610 0x7f9810055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.012+0000 7f982b7fe700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9810053610 0x7f9810055ad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f981c009000 tx=0x7f981c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:04.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.012+0000 7f98297fa700 1 -- 172.21.2.116:0/2881951474 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98140425f0 con 0x7f980c004f70 2021-11-21T21:56:04.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f982c0573d0 con 0x7f980c004f70 2021-11-21T21:56:04.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98297fa700 1 -- 172.21.2.116:0/2881951474 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+1551 (secure 0 0 0) 0x7f9814019020 con 0x7f980c004f70 2021-11-21T21:56:04.124 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:04.124 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/4705115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":4705115},{"type":"v1","addr":"172.21.2.116:6833","nonce":4705115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2110528747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2110528747},{"type":"v1","addr":"172.21.2.117:6835","nonce":2110528747}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":5157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2140793945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2140793945},{"type":"v1","addr":"172.21.2.116:6835","nonce":2140793945}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13}],"filesystems":[]} 2021-11-21T21:56:04.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9810053610 msgr2=0x7f9810055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9810053610 0x7f9810055ad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f981c009000 tx=0x7f981c009040).stop 2021-11-21T21:56:04.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 msgr2=0x7f982c13fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f982c13fb80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f9814008000 tx=0x7f9814009040).stop 2021-11-21T21:56:04.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 shutdown_connections 2021-11-21T21:56:04.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f982c13b750 0x7f982c13db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:04.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f982c13eb60 0x7f982c13d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9810053610 0x7f9810055ad0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 --2- 172.21.2.116:0/2881951474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f980c004f70 0x7f982c13fb80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 >> 172.21.2.116:0/2881951474 conn(0x7f982c05bd20 msgr2=0x7f982c142200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:04.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 shutdown_connections 2021-11-21T21:56:04.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.124+0000 7f98324d0700 1 -- 172.21.2.116:0/2881951474 wait complete. 2021-11-21T21:56:04.128 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 13 2021-11-21T21:56:04.132 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T21:56:04.132 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T21:56:04.133 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T21:56:04.133 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:56:04.133 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T21:56:04.134 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T21:56:04.134 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T21:56:04.134 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:56:04.135 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T21:56:04.135 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T21:56:04.135 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T21:56:04.136 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:56:04.138 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T21:56:04.139 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T21:56:04.140 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T21:56:04.141 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T21:56:04.144 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T21:56:04.145 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T21:56:04.147 INFO:tasks.ceph.mds.b:Started 2021-11-21T21:56:04.147 DEBUG:teuthology.parallel:result is None 2021-11-21T21:56:04.183 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T21:56:04.184 INFO:tasks.ceph.mds.a:Started 2021-11-21T21:56:04.185 DEBUG:teuthology.parallel:result is None 2021-11-21T21:56:04.185 INFO:tasks.ceph.mds.c:Started 2021-11-21T21:56:04.186 DEBUG:teuthology.parallel:result is None 2021-11-21T21:56:04.186 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T21:56:04.218 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T21:56:04.219 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:56:04.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 Processor -- start 2021-11-21T21:56:04.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- start start 2021-11-21T21:56:04.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 0x7f03000b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f03000ac820 0x7f03001424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000b0500 con 0x7f03000b13b0 2021-11-21T21:56:04.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03000adc70 con 0x7f03000ac820 2021-11-21T21:56:04.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030013ec80 con 0x7f03000b5380 2021-11-21T21:56:04.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fdd9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f03000b13b0 0x7f030013c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58500/0 (socket says 172.21.2.116:58500) 2021-11-21T21:56:04.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fdd9b700 1 -- 172.21.2.116:0/280277291 learned_addr learned my addr 172.21.2.116:0/280277291 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:04.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336516417 0 0) 0x7f03000b0500 con 0x7f03000b13b0 2021-11-21T21:56:04.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e4003260 con 0x7f03000b13b0 2021-11-21T21:56:04.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fed9d700 1 --2- 172.21.2.116:0/280277291 >> v2:172.21.2.117:3300/0 conn(0x7f03000ac820 0x7f03001424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 0x7f03000b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2265318328 0 0) 0x7f02e4003260 con 0x7f03000b13b0 2021-11-21T21:56:04.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 msgr2=0x7f03000b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 0x7f03000b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 >> v2:172.21.2.117:3300/0 conn(0x7f03000ac820 msgr2=0x7f03001424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 --2- 172.21.2.116:0/280277291 >> v2:172.21.2.117:3300/0 conn(0x7f03000ac820 0x7f03001424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000b1b50 con 0x7f03000b13b0 2021-11-21T21:56:04.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f02f8004d60 con 0x7f03000b13b0 2021-11-21T21:56:04.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 >> v1:172.21.2.116:6789/0 conn(0x7f03000b13b0 legacy=0x7f030013c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f02e4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 0x7f02e4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02e40049f0 0x7f02e4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e400b750 con 0x7f02e4003800 2021-11-21T21:56:04.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e400b8c0 con 0x7f02e40049f0 2021-11-21T21:56:04.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e400ba80 con 0x7f03000b5380 2021-11-21T21:56:04.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f02e4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fed9d700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02e40049f0 0x7f02e4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fdd9b700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 0x7f02e4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 -- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 msgr2=0x7f02e4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f03000b5380 0x7f02e4004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 -- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02e40049f0 msgr2=0x7f02e4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:04.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02e40049f0 0x7f02e4004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 -- 172.21.2.116:0/280277291 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4003260 con 0x7f02e4003800 2021-11-21T21:56:04.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fe59c700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f02e4003c20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f02f400a000 tx=0x7f02f4008040).ready entity=mon.0 client_cookie=d3a79801f13e602c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:04.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02f4013670 con 0x7f02e4003800 2021-11-21T21:56:04.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f40137d0 con 0x7f02e4003800 2021-11-21T21:56:04.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fd59a700 1 -- 172.21.2.116:0/280277291 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02f4013b30 con 0x7f02e4003800 2021-11-21T21:56:04.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- 172.21.2.116:0/280277291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 msgr2=0x7f02e4003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 --2- 172.21.2.116:0/280277291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f02e4003c20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f02f400a000 tx=0x7f02f4008040).stop 2021-11-21T21:56:04.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f02fed9d700 1 -- 172.21.2.116:0/280277291 reap_dead start 2021-11-21T21:56:04.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- 172.21.2.116:0/280277291 shutdown_connections 2021-11-21T21:56:04.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- 172.21.2.116:0/280277291 >> 172.21.2.116:0/280277291 conn(0x7f0300061900 msgr2=0x7f0300069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:04.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- 172.21.2.116:0/280277291 shutdown_connections 2021-11-21T21:56:04.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.240+0000 7f0304e22700 1 -- 172.21.2.116:0/280277291 wait complete. 2021-11-21T21:56:04.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 Processor -- start 2021-11-21T21:56:04.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 -- start start 2021-11-21T21:56:04.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f03000b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0300067770 0x7f0300066d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0300067280 0x7f03000ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300066210 con 0x7f02e4003800 2021-11-21T21:56:04.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300066440 con 0x7f0300067280 2021-11-21T21:56:04.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000666e0 con 0x7f0300067770 2021-11-21T21:56:04.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f03000b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f03000b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46556/0 (socket says 172.21.2.116:46556) 2021-11-21T21:56:04.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 -- 172.21.2.116:0/2703242630 learned_addr learned my addr 172.21.2.116:0/2703242630 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:04.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fdd9b700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0300067770 0x7f0300066d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fed9d700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0300067280 0x7f03000ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 -- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0300067770 msgr2=0x7f0300066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0300067770 0x7f0300066d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 -- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0300067280 msgr2=0x7f03000ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:04.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0300067280 0x7f03000ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:04.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 -- 172.21.2.116:0/2703242630 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300065810 con 0x7f02e4003800 2021-11-21T21:56:04.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fe59c700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f03000b25e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f02f400a000 tx=0x7f02f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:04.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02ef7fe700 1 -- 172.21.2.116:0/2703242630 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02f4012c40 con 0x7f02e4003800 2021-11-21T21:56:04.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02ef7fe700 1 -- 172.21.2.116:0/2703242630 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f4000e90 con 0x7f02e4003800 2021-11-21T21:56:04.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0300065cb0 con 0x7f02e4003800 2021-11-21T21:56:04.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02ef7fe700 1 -- 172.21.2.116:0/2703242630 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02f401d410 con 0x7f02e4003800 2021-11-21T21:56:04.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03001510f0 con 0x7f02e4003800 2021-11-21T21:56:04.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0300065650 con 0x7f02e4003800 2021-11-21T21:56:04.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02ef7fe700 1 -- 172.21.2.116:0/2703242630 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f02f401d570 con 0x7f02e4003800 2021-11-21T21:56:04.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02ef7fe700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02dc053580 0x7f02dc055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:04.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02ef7fe700 1 -- 172.21.2.116:0/2703242630 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f02f406d120 con 0x7f02e4003800 2021-11-21T21:56:04.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fdd9b700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02dc053580 0x7f02dc055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:04.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.244+0000 7f02fdd9b700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02dc053580 0x7f02dc055a40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f02f800e000 tx=0x7f03001421d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:04.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.248+0000 7f02ef7fe700 1 -- 172.21.2.116:0/2703242630 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02f4044500 con 0x7f02e4003800 2021-11-21T21:56:04.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:04.340+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0300066210 con 0x7f02e4003800 2021-11-21T21:56:05.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f02ef7fe700 1 -- 172.21.2.116:0/2703242630 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v37) v1 ==== 97+0+0 (secure 0 0 0) 0x7f02f4047b20 con 0x7f02e4003800 2021-11-21T21:56:05.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02dc053580 msgr2=0x7f02dc055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02dc053580 0x7f02dc055a40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f02f800e000 tx=0x7f03001421d0).stop 2021-11-21T21:56:05.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 msgr2=0x7f03000b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f03000b25e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f02f400a000 tx=0x7f02f4008040).stop 2021-11-21T21:56:05.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 shutdown_connections 2021-11-21T21:56:05.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0300067280 0x7f03000ab460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0300067770 0x7f0300066d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02dc053580 0x7f02dc055a40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 --2- 172.21.2.116:0/2703242630 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02e4003800 0x7f03000b25e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 >> 172.21.2.116:0/2703242630 conn(0x7f0300061900 msgr2=0x7f03000af160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 shutdown_connections 2021-11-21T21:56:05.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.104+0000 7f0304e22700 1 -- 172.21.2.116:0/2703242630 wait complete. 2021-11-21T21:56:05.115 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T21:56:05.117 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T21:56:05.118 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T21:56:05.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbafc700700 1 Processor -- start 2021-11-21T21:56:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbafc700700 1 -- start start 2021-11-21T21:56:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbafc700700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf40a6aa0 0x7fbaf414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbafc700700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbaf4062410 0x7fbaf41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbafc700700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf40a6e80 con 0x7fbaf40a86b0 2021-11-21T21:56:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbafc700700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf40a8bc0 con 0x7fbaf4062410 2021-11-21T21:56:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbafc700700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf414a7b0 con 0x7fbaf40a6aa0 2021-11-21T21:56:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbaf9c9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbaf40a86b0 0x7fbaf41470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58520/0 (socket says 172.21.2.116:58520) 2021-11-21T21:56:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.168+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/3210129643 learned_addr learned my addr 172.21.2.116:0/3210129643 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918497285 0 0) 0x7fbaf40a6e80 con 0x7fbaf40a86b0 2021-11-21T21:56:05.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbae0003260 con 0x7fbaf40a86b0 2021-11-21T21:56:05.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148808696 0 0) 0x7fbae0003260 con 0x7fbaf40a86b0 2021-11-21T21:56:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf40a6aa0 msgr2=0x7fbaf414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf40a6aa0 0x7fbaf414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 >> v2:172.21.2.117:3300/0 conn(0x7fbaf4062410 msgr2=0x7fbaf41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafac9d700 1 --2- 172.21.2.116:0/3210129643 >> v2:172.21.2.117:3300/0 conn(0x7fbaf4062410 0x7fbaf41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 --2- 172.21.2.116:0/3210129643 >> v2:172.21.2.117:3300/0 conn(0x7fbaf4062410 0x7fbaf41443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf414aa10 con 0x7fbaf40a86b0 2021-11-21T21:56:05.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fbaec001a00 con 0x7fbaf40a86b0 2021-11-21T21:56:05.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafac9d700 1 --2- 172.21.2.116:0/3210129643 >> v2:172.21.2.117:3300/0 conn(0x7fbaf4062410 0x7fbaf41443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:56:05.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 >> v1:172.21.2.116:6789/0 conn(0x7fbaf40a86b0 legacy=0x7fbaf41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf40a6aa0 0x7fbae00037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbae0003ee0 0x7fbae0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0004bf0 0x7fbae000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf40a8bc0 con 0x7fbae0003ee0 2021-11-21T21:56:05.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf414a7b0 con 0x7fbae0004bf0 2021-11-21T21:56:05.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00053a0 con 0x7fbaf40a6aa0 2021-11-21T21:56:05.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbae0003ee0 0x7fbae0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafac9d700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0004bf0 0x7fbae000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafa49c700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf40a6aa0 0x7fbae00037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf40a6aa0 msgr2=0x7fbae00037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf40a6aa0 0x7fbae00037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0004bf0 msgr2=0x7fbae000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0004bf0 0x7fbae000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/3210129643 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae0003260 con 0x7fbae0003ee0 2021-11-21T21:56:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbae0003ee0 0x7fbae0004500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fbaec001710 tx=0x7fbaec002120).ready entity=mon.0 client_cookie=d769b8adde13d9bf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaec00cb70 con 0x7fbae0003ee0 2021-11-21T21:56:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec00ccd0 con 0x7fbae0003ee0 2021-11-21T21:56:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf949a700 1 -- 172.21.2.116:0/3210129643 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaec004b50 con 0x7fbae0003ee0 2021-11-21T21:56:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- 172.21.2.116:0/3210129643 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbae0003ee0 msgr2=0x7fbae0004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 --2- 172.21.2.116:0/3210129643 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbae0003ee0 0x7fbae0004500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fbaec001710 tx=0x7fbaec002120).stop 2021-11-21T21:56:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafac9d700 1 -- 172.21.2.116:0/3210129643 reap_dead start 2021-11-21T21:56:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- 172.21.2.116:0/3210129643 shutdown_connections 2021-11-21T21:56:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- 172.21.2.116:0/3210129643 >> 172.21.2.116:0/3210129643 conn(0x7fbaf405bd20 msgr2=0x7fbaf405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- 172.21.2.116:0/3210129643 shutdown_connections 2021-11-21T21:56:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- 172.21.2.116:0/3210129643 wait complete. 2021-11-21T21:56:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 Processor -- start 2021-11-21T21:56:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- start start 2021-11-21T21:56:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0003ee0 0x7fbaf41464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf40ab710 0x7fbaf4156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf4156e00 0x7fbaf40aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4064420 con 0x7fbaf40ab710 2021-11-21T21:56:05.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf41563b0 con 0x7fbae0003ee0 2021-11-21T21:56:05.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4065760 con 0x7fbaf4156e00 2021-11-21T21:56:05.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf40ab710 0x7fbaf4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf40ab710 0x7fbaf4156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46576/0 (socket says 172.21.2.116:46576) 2021-11-21T21:56:05.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/1947021128 learned_addr learned my addr 172.21.2.116:0/1947021128 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf4156e00 msgr2=0x7fbaf40aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf4156e00 0x7fbaf40aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0003ee0 msgr2=0x7fbaf41464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0003ee0 0x7fbaf41464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 -- 172.21.2.116:0/1947021128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4065a00 con 0x7fbaf40ab710 2021-11-21T21:56:05.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf9c9b700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf40ab710 0x7fbaf4156910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fbaec001710 tx=0x7fbaec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf37fe700 1 -- 172.21.2.116:0/1947021128 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaec002120 con 0x7fbaf40ab710 2021-11-21T21:56:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf37fe700 1 -- 172.21.2.116:0/1947021128 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec007380 con 0x7fbaf40ab710 2021-11-21T21:56:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf4065ca0 con 0x7fbaf40ab710 2021-11-21T21:56:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbaf37fe700 1 -- 172.21.2.116:0/1947021128 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaec0076e0 con 0x7fbaf40ab710 2021-11-21T21:56:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.172+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf4146e70 con 0x7fbaf40ab710 2021-11-21T21:56:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.176+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf40aa260 con 0x7fbaf40ab710 2021-11-21T21:56:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.176+0000 7fbaf37fe700 1 -- 172.21.2.116:0/1947021128 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbaec01d070 con 0x7fbaf40ab710 2021-11-21T21:56:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.176+0000 7fbaf37fe700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad4053720 0x7fbad4055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.176+0000 7fbaf37fe700 1 -- 172.21.2.116:0/1947021128 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbaec06c6a0 con 0x7fbaf40ab710 2021-11-21T21:56:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.176+0000 7fbafa49c700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad4053720 0x7fbad4055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.176+0000 7fbafa49c700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad4053720 0x7fbad4055be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fbae800a000 tx=0x7fbae8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.176+0000 7fbaf37fe700 1 -- 172.21.2.116:0/1947021128 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbaec043a20 con 0x7fbaf40ab710 2021-11-21T21:56:05.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbaf40573d0 con 0x7fbaf40ab710 2021-11-21T21:56:05.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbaf37fe700 1 -- 172.21.2.116:0/1947021128 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fbaec02e0b0 con 0x7fbaf40ab710 2021-11-21T21:56:05.288 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:05.288 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:56:05.289 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T21:56:05.289 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.289 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T21:56:05.289 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T21:56:05.290 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.290 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:56:05.290 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.290 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.290 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.291 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.291 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.291 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.291 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T21:56:05.292 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T21:56:05.292 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.292 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:56:05.292 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.292 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.293 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.293 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.293 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.293 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.293 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T21:56:05.294 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T21:56:05.294 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.294 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:56:05.294 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.295 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.295 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.295 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.295 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.295 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.296 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T21:56:05.296 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T21:56:05.296 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.296 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.297 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.297 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.297 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.297 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.298 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.298 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T21:56:05.298 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T21:56:05.299 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.299 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.299 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.299 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.299 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.300 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.300 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.300 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T21:56:05.300 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T21:56:05.301 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.301 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.301 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.301 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.302 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.302 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.302 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.302 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T21:56:05.302 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T21:56:05.303 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.303 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.303 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.303 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.304 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.304 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.304 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.304 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T21:56:05.305 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T21:56:05.305 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.305 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.305 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.306 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.306 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.306 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.306 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.306 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T21:56:05.307 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T21:56:05.307 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.307 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.307 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.308 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.308 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.308 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.308 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.309 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T21:56:05.309 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T21:56:05.309 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.309 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.309 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.310 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.310 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.310 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.310 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.311 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T21:56:05.311 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T21:56:05.311 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.312 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.313 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.313 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.313 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.313 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.314 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.314 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T21:56:05.314 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T21:56:05.314 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.315 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:56:05.315 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:56:05.315 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:56:05.315 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.315 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.316 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.316 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T21:56:05.316 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T21:56:05.316 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.316 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:56:05.317 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:56:05.317 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:56:05.317 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.317 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.318 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.318 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T21:56:05.318 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T21:56:05.318 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.318 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:56:05.319 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:56:05.319 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:56:05.320 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.321 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.321 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.322 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.322 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T21:56:05.322 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T21:56:05.323 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.323 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:56:05.323 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:56:05.323 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.323 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.324 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.324 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.324 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T21:56:05.324 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T21:56:05.325 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:56:05.325 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:56:05.325 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:56:05.325 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:56:05.326 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.326 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.326 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:56:05.326 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:56:05.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad4053720 msgr2=0x7fbad4055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad4053720 0x7fbad4055be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fbae800a000 tx=0x7fbae8008040).stop 2021-11-21T21:56:05.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf40ab710 msgr2=0x7fbaf4156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf40ab710 0x7fbaf4156910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fbaec001710 tx=0x7fbaec015000).stop 2021-11-21T21:56:05.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 shutdown_connections 2021-11-21T21:56:05.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf4156e00 0x7fbaf40aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad4053720 0x7fbad4055be0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf40ab710 0x7fbaf4156910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 --2- 172.21.2.116:0/1947021128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbae0003ee0 0x7fbaf41464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 >> 172.21.2.116:0/1947021128 conn(0x7fbaf405bd20 msgr2=0x7fbaf4143800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 shutdown_connections 2021-11-21T21:56:05.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.288+0000 7fbafc700700 1 -- 172.21.2.116:0/1947021128 wait complete. 2021-11-21T21:56:05.333 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:56:05.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57aae2a700 1 Processor -- start 2021-11-21T21:56:05.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57aae2a700 1 -- start start 2021-11-21T21:56:05.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57aae2a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f57a40a6aa0 0x7f57a414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57aae2a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f57a40a86b0 0x7f57a41470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57aae2a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a40a6e80 con 0x7f57a41475d0 2021-11-21T21:56:05.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57aae2a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a40a8bc0 con 0x7f57a40a6aa0 2021-11-21T21:56:05.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57aae2a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a414a7b0 con 0x7f57a40a86b0 2021-11-21T21:56:05.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f57a40a6aa0 0x7f57a414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f57a40a6aa0 0x7f57a414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46584/0 (socket says 172.21.2.116:46584) 2021-11-21T21:56:05.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 -- 172.21.2.116:0/1250550127 learned_addr learned my addr 172.21.2.116:0/1250550127 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 -- 172.21.2.116:0/1250550127 >> v2:172.21.2.117:3300/0 conn(0x7f57a40a86b0 msgr2=0x7f57a41470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:05.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 --2- 172.21.2.116:0/1250550127 >> v2:172.21.2.117:3300/0 conn(0x7f57a40a86b0 0x7f57a41470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 -- 172.21.2.116:0/1250550127 >> v1:172.21.2.116:6789/0 conn(0x7f57a41475d0 legacy=0x7f57a41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 -- 172.21.2.116:0/1250550127 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a414aa10 con 0x7f57a40a6aa0 2021-11-21T21:56:05.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a8bc6700 1 --2- 172.21.2.116:0/1250550127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a6aa0 0x7f57a414b150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f579c001490 tx=0x7f5794006eb0).ready entity=mon.0 client_cookie=2ddc8fed1f6893d9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a37fe700 1 -- 172.21.2.116:0/1250550127 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f579400d5c0 con 0x7f57a40a6aa0 2021-11-21T21:56:05.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a37fe700 1 -- 172.21.2.116:0/1250550127 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794000e90 con 0x7f57a40a6aa0 2021-11-21T21:56:05.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.384+0000 7f57a37fe700 1 -- 172.21.2.116:0/1250550127 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f579400d950 con 0x7f57a40a6aa0 2021-11-21T21:56:05.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/1250550127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a6aa0 msgr2=0x7f57a414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 --2- 172.21.2.116:0/1250550127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a6aa0 0x7f57a414b150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f579c001490 tx=0x7f5794006eb0).stop 2021-11-21T21:56:05.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/1250550127 shutdown_connections 2021-11-21T21:56:05.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 --2- 172.21.2.116:0/1250550127 >> v2:172.21.2.117:3300/0 conn(0x7f57a40a86b0 0x7f57a41470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 --2- 172.21.2.116:0/1250550127 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a6aa0 0x7f57a414b150 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/1250550127 >> 172.21.2.116:0/1250550127 conn(0x7f57a405bd20 msgr2=0x7f57a405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/1250550127 shutdown_connections 2021-11-21T21:56:05.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/1250550127 wait complete. 2021-11-21T21:56:05.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 Processor -- start 2021-11-21T21:56:05.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- start start 2021-11-21T21:56:05.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57a40a6aa0 0x7f57a4156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a86b0 0x7f57a4156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57a41475d0 0x7f57a4064810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40abae0 con 0x7f57a40a86b0 2021-11-21T21:56:05.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41572d0 con 0x7f57a41475d0 2021-11-21T21:56:05.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4157570 con 0x7f57a40a6aa0 2021-11-21T21:56:05.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a86b0 0x7f57a4156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a86b0 0x7f57a4156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46590/0 (socket says 172.21.2.116:46590) 2021-11-21T21:56:05.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 -- 172.21.2.116:0/2837290145 learned_addr learned my addr 172.21.2.116:0/2837290145 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 -- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57a40a6aa0 msgr2=0x7f57a4156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57a40a6aa0 0x7f57a4156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 -- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57a41475d0 msgr2=0x7f57a4064810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57a41475d0 0x7f57a4064810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 -- 172.21.2.116:0/2837290145 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579400b1e0 con 0x7f57a40a86b0 2021-11-21T21:56:05.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a3fff700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a86b0 0x7f57a4156b10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f5798002fb0 tx=0x7f5798006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a1ffb700 1 -- 172.21.2.116:0/2837290145 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5798009e00 con 0x7f57a40a86b0 2021-11-21T21:56:05.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a1ffb700 1 -- 172.21.2.116:0/2837290145 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798013460 con 0x7f57a40a86b0 2021-11-21T21:56:05.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a1ffb700 1 -- 172.21.2.116:0/2837290145 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57980137c0 con 0x7f57a40a86b0 2021-11-21T21:56:05.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a40a9190 con 0x7f57a40a86b0 2021-11-21T21:56:05.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a4146aa0 con 0x7f57a40a86b0 2021-11-21T21:56:05.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a40573d0 con 0x7f57a40a86b0 2021-11-21T21:56:05.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a1ffb700 1 -- 172.21.2.116:0/2837290145 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f579800f070 con 0x7f57a40a86b0 2021-11-21T21:56:05.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a1ffb700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5784053600 0x7f5784055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.388+0000 7f57a1ffb700 1 -- 172.21.2.116:0/2837290145 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f579806af80 con 0x7f57a40a86b0 2021-11-21T21:56:05.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.392+0000 7f57a8bc6700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5784053600 0x7f5784055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.392+0000 7f57a8bc6700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5784053600 0x7f5784055ac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f579c001490 tx=0x7f5794015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.392+0000 7f57a1ffb700 1 -- 172.21.2.116:0/2837290145 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5798042410 con 0x7f57a40a86b0 2021-11-21T21:56:05.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a414a410 con 0x7f57a40a86b0 2021-11-21T21:56:05.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57a1ffb700 1 -- 172.21.2.116:0/2837290145 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13181 (secure 0 0 0) 0x7f5798045a30 con 0x7f57a40a86b0 2021-11-21T21:56:05.477 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:05.477 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":37,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:56:05.096318+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:56:05.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5784053600 msgr2=0x7f5784055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5784053600 0x7f5784055ac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f579c001490 tx=0x7f5794015000).stop 2021-11-21T21:56:05.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a86b0 msgr2=0x7f57a4156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a86b0 0x7f57a4156b10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f5798002fb0 tx=0x7f5798006df0).stop 2021-11-21T21:56:05.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 shutdown_connections 2021-11-21T21:56:05.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57a41475d0 0x7f57a4064810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5784053600 0x7f5784055ac0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57a40a86b0 0x7f57a4156b10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 --2- 172.21.2.116:0/2837290145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57a40a6aa0 0x7f57a4156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 >> 172.21.2.116:0/2837290145 conn(0x7f57a405bd20 msgr2=0x7f57a4148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 shutdown_connections 2021-11-21T21:56:05.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.476+0000 7f57aae2a700 1 -- 172.21.2.116:0/2837290145 wait complete. 2021-11-21T21:56:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T21:56:05.485 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:56:05.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7fceac700 1 Processor -- start 2021-11-21T21:56:05.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7fceac700 1 -- start start 2021-11-21T21:56:05.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7fceac700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7f8144810 0x7fd7f81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7fceac700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd7f80ac600 0x7fd7f80aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7fceac700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f8147e80 con 0x7fd7f80abce0 2021-11-21T21:56:05.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7fceac700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80b1490 con 0x7fd7f8144810 2021-11-21T21:56:05.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7fceac700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80b1fd0 con 0x7fd7f80ac600 2021-11-21T21:56:05.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7f8144810 0x7fd7f81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7f8144810 0x7fd7f81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46602/0 (socket says 172.21.2.116:46602) 2021-11-21T21:56:05.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 -- 172.21.2.116:0/3755753318 learned_addr learned my addr 172.21.2.116:0/3755753318 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 -- 172.21.2.116:0/3755753318 >> v2:172.21.2.117:3300/0 conn(0x7fd7f80ac600 msgr2=0x7fd7f80aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:05.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 --2- 172.21.2.116:0/3755753318 >> v2:172.21.2.117:3300/0 conn(0x7fd7f80ac600 0x7fd7f80aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 -- 172.21.2.116:0/3755753318 >> v1:172.21.2.117:6789/0 conn(0x7fd7f80abce0 legacy=0x7fd7f80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:05.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 -- 172.21.2.116:0/3755753318 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80af5a0 con 0x7fd7f8144810 2021-11-21T21:56:05.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f659c700 1 --2- 172.21.2.116:0/3755753318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f8144810 0x7fd7f81478c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fd7f80b4e00 tx=0x7fd7e0006eb0).ready entity=mon.0 client_cookie=c0865c3c680c301 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.536+0000 7fd7f559a700 1 -- 172.21.2.116:0/3755753318 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e000d5c0 con 0x7fd7f8144810 2021-11-21T21:56:05.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f559a700 1 -- 172.21.2.116:0/3755753318 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e0000e90 con 0x7fd7f8144810 2021-11-21T21:56:05.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f559a700 1 -- 172.21.2.116:0/3755753318 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e000d910 con 0x7fd7f8144810 2021-11-21T21:56:05.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/3755753318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f8144810 msgr2=0x7fd7f81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 --2- 172.21.2.116:0/3755753318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f8144810 0x7fd7f81478c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fd7f80b4e00 tx=0x7fd7e0006eb0).stop 2021-11-21T21:56:05.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/3755753318 shutdown_connections 2021-11-21T21:56:05.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 --2- 172.21.2.116:0/3755753318 >> v2:172.21.2.117:3300/0 conn(0x7fd7f80ac600 0x7fd7f80aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 --2- 172.21.2.116:0/3755753318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f8144810 0x7fd7f81478c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/3755753318 >> 172.21.2.116:0/3755753318 conn(0x7fd7f805bd20 msgr2=0x7fd7f805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/3755753318 shutdown_connections 2021-11-21T21:56:05.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/3755753318 wait complete. 2021-11-21T21:56:05.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 Processor -- start 2021-11-21T21:56:05.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- start start 2021-11-21T21:56:05.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7f80abce0 0x7fd7f80b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f80ac600 0x7fd7f8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7f8144810 0x7fd7f80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81435d0 con 0x7fd7f80ac600 2021-11-21T21:56:05.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8143870 con 0x7fd7f80abce0 2021-11-21T21:56:05.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80b3be0 con 0x7fd7f8144810 2021-11-21T21:56:05.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f80ac600 0x7fd7f8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f80ac600 0x7fd7f8159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46606/0 (socket says 172.21.2.116:46606) 2021-11-21T21:56:05.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 -- 172.21.2.116:0/1809834660 learned_addr learned my addr 172.21.2.116:0/1809834660 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 -- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7f8144810 msgr2=0x7fd7f80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7f8144810 0x7fd7f80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 -- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7f80abce0 msgr2=0x7fd7f80b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7f80abce0 0x7fd7f80b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 -- 172.21.2.116:0/1809834660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e000b1e0 con 0x7fd7f80ac600 2021-11-21T21:56:05.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7f5d9b700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f80ac600 0x7fd7f8159c00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fd7e8002fb0 tx=0x7fd7e8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7ef7fe700 1 -- 172.21.2.116:0/1809834660 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e8013460 con 0x7fd7f80ac600 2021-11-21T21:56:05.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7ef7fe700 1 -- 172.21.2.116:0/1809834660 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e80135c0 con 0x7fd7f80ac600 2021-11-21T21:56:05.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f80b3e80 con 0x7fd7f80ac600 2021-11-21T21:56:05.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f81429a0 con 0x7fd7f80ac600 2021-11-21T21:56:05.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7ef7fe700 1 -- 172.21.2.116:0/1809834660 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e8013920 con 0x7fd7f80ac600 2021-11-21T21:56:05.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f80b4010 con 0x7fd7f80ac600 2021-11-21T21:56:05.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.540+0000 7fd7ef7fe700 1 -- 172.21.2.116:0/1809834660 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd7e800f070 con 0x7fd7f80ac600 2021-11-21T21:56:05.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.544+0000 7fd7ef7fe700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7dc053690 0x7fd7dc055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.544+0000 7fd7ef7fe700 1 -- 172.21.2.116:0/1809834660 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd7e806b1d0 con 0x7fd7f80ac600 2021-11-21T21:56:05.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.544+0000 7fd7f659c700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7dc053690 0x7fd7dc055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.544+0000 7fd7f659c700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7dc053690 0x7fd7dc055b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd7f80b4e00 tx=0x7fd7e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.544+0000 7fd7ef7fe700 1 -- 172.21.2.116:0/1809834660 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7e8042550 con 0x7fd7f80ac600 2021-11-21T21:56:05.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.624+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd7f80573d0 con 0x7fd7f80ac600 2021-11-21T21:56:05.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.624+0000 7fd7ef7fe700 1 -- 172.21.2.116:0/1809834660 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v37) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd7e8045b70 con 0x7fd7f80ac600 2021-11-21T21:56:05.628 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:56:05.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7dc053690 msgr2=0x7fd7dc055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7dc053690 0x7fd7dc055b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd7f80b4e00 tx=0x7fd7e0015000).stop 2021-11-21T21:56:05.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f80ac600 msgr2=0x7fd7f8159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f80ac600 0x7fd7f8159c00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fd7e8002fb0 tx=0x7fd7e8006df0).stop 2021-11-21T21:56:05.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 shutdown_connections 2021-11-21T21:56:05.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7f8144810 0x7fd7f80b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7dc053690 0x7fd7dc055b50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7f80ac600 0x7fd7f8159c00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 --2- 172.21.2.116:0/1809834660 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7f80abce0 0x7fd7f80b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 >> 172.21.2.116:0/1809834660 conn(0x7fd7f805bd20 msgr2=0x7fd7f814ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 shutdown_connections 2021-11-21T21:56:05.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.628+0000 7fd7fceac700 1 -- 172.21.2.116:0/1809834660 wait complete. 2021-11-21T21:56:05.635 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:56:05.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 Processor -- start 2021-11-21T21:56:05.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- start start 2021-11-21T21:56:05.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c144810 0x7f6b1c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6b1c0ac600 0x7f6b1c0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1c147e80 con 0x7f6b1c0abce0 2021-11-21T21:56:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b1490 con 0x7f6b1c0ac600 2021-11-21T21:56:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b1fd0 con 0x7f6b1c144810 2021-11-21T21:56:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1b7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f6b1c0abce0 0x7f6b1c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58568/0 (socket says 172.21.2.116:58568) 2021-11-21T21:56:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1b7fe700 1 -- 172.21.2.116:0/3044997324 learned_addr learned my addr 172.21.2.116:0/3044997324 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:05.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829959682 0 0) 0x7f6b1c147e80 con 0x7f6b1c0abce0 2021-11-21T21:56:05.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6afc003260 con 0x7f6b1c0abce0 2021-11-21T21:56:05.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2302890959 0 0) 0x7f6afc003260 con 0x7f6b1c0abce0 2021-11-21T21:56:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c144810 msgr2=0x7f6b1c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c144810 0x7f6b1c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 >> v2:172.21.2.117:3300/0 conn(0x7f6b1c0ac600 msgr2=0x7f6b1c0aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 --2- 172.21.2.116:0/3044997324 >> v2:172.21.2.117:3300/0 conn(0x7f6b1c0ac600 0x7f6b1c0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0af5a0 con 0x7f6b1c0abce0 2021-11-21T21:56:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f6b04001660 con 0x7f6b1c0abce0 2021-11-21T21:56:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 >> v1:172.21.2.116:6789/0 conn(0x7f6b1c0abce0 legacy=0x7f6b1c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6afc0037d0 0x7f6afc003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c144810 0x7f6afc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6afc004bf0 0x7f6afc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b1490 con 0x7f6afc0037d0 2021-11-21T21:56:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b1fd0 con 0x7f6afc004bf0 2021-11-21T21:56:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc005300 con 0x7f6b1c144810 2021-11-21T21:56:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1bfff700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6afc0037d0 0x7f6afc003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1bfff700 1 -- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c144810 msgr2=0x7f6afc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1bfff700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c144810 0x7f6afc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1bfff700 1 -- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6afc004bf0 msgr2=0x7f6afc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1bfff700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6afc004bf0 0x7f6afc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1bfff700 1 -- 172.21.2.116:0/3044997324 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc003260 con 0x7f6afc0037d0 2021-11-21T21:56:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1bfff700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6afc0037d0 0x7f6afc003bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f6b0c002f70 tx=0x7f6b0c006ca0).ready entity=mon.0 client_cookie=397d6c834cc966c8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b0c012410 con 0x7f6afc0037d0 2021-11-21T21:56:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c012570 con 0x7f6afc0037d0 2021-11-21T21:56:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b1affd700 1 -- 172.21.2.116:0/3044997324 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b0c0128d0 con 0x7f6afc0037d0 2021-11-21T21:56:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- 172.21.2.116:0/3044997324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6afc0037d0 msgr2=0x7f6afc003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 --2- 172.21.2.116:0/3044997324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6afc0037d0 0x7f6afc003bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f6b0c002f70 tx=0x7f6b0c006ca0).stop 2021-11-21T21:56:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b208e6700 1 -- 172.21.2.116:0/3044997324 reap_dead start 2021-11-21T21:56:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- 172.21.2.116:0/3044997324 shutdown_connections 2021-11-21T21:56:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- 172.21.2.116:0/3044997324 >> 172.21.2.116:0/3044997324 conn(0x7f6b1c05bd20 msgr2=0x7f6b1c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- 172.21.2.116:0/3044997324 shutdown_connections 2021-11-21T21:56:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.688+0000 7f6b22349700 1 -- 172.21.2.116:0/3044997324 wait complete. 2021-11-21T21:56:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 Processor -- start 2021-11-21T21:56:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 -- start start 2021-11-21T21:56:05.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6afc0037d0 0x7f6b1c13f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b1c156a70 0x7f6b1c0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c066d70 0x7f6b1c067200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c063360 con 0x7f6b1c156a70 2021-11-21T21:56:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c146810 con 0x7f6afc0037d0 2021-11-21T21:56:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b39c0 con 0x7f6b1c066d70 2021-11-21T21:56:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b1c156a70 0x7f6b1c0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b1c156a70 0x7f6b1c0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46626/0 (socket says 172.21.2.116:46626) 2021-11-21T21:56:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 -- 172.21.2.116:0/1754962332 learned_addr learned my addr 172.21.2.116:0/1754962332 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 -- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c066d70 msgr2=0x7f6b1c067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c066d70 0x7f6b1c067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 -- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6afc0037d0 msgr2=0x7f6b1c13f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6afc0037d0 0x7f6b1c13f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 -- 172.21.2.116:0/1754962332 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c146a40 con 0x7f6b1c156a70 2021-11-21T21:56:05.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1b7fe700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b1c156a70 0x7f6b1c0b3d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f6b04001370 tx=0x7f6b04001a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b197fa700 1 -- 172.21.2.116:0/1754962332 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b04007170 con 0x7f6b1c156a70 2021-11-21T21:56:05.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b197fa700 1 -- 172.21.2.116:0/1754962332 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b040072d0 con 0x7f6b1c156a70 2021-11-21T21:56:05.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c0b4270 con 0x7f6b1c156a70 2021-11-21T21:56:05.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c064830 con 0x7f6b1c156a70 2021-11-21T21:56:05.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b197fa700 1 -- 172.21.2.116:0/1754962332 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b04004920 con 0x7f6b1c156a70 2021-11-21T21:56:05.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c0b2e30 con 0x7f6b1c156a70 2021-11-21T21:56:05.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b197fa700 1 -- 172.21.2.116:0/1754962332 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6b04004a80 con 0x7f6b1c156a70 2021-11-21T21:56:05.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b197fa700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b000536d0 0x7f6b00055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b197fa700 1 -- 172.21.2.116:0/1754962332 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6b0406b3e0 con 0x7f6b1c156a70 2021-11-21T21:56:05.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.692+0000 7f6b1bfff700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b000536d0 0x7f6b00055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.696+0000 7f6b1bfff700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b000536d0 0x7f6b00055b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f6b0c002f70 tx=0x7f6b0c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.696+0000 7f6b197fa700 1 -- 172.21.2.116:0/1754962332 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b04042700 con 0x7f6b1c156a70 2021-11-21T21:56:05.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6b1c0af060 con 0x7f6b1c156a70 2021-11-21T21:56:05.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b197fa700 1 -- 172.21.2.116:0/1754962332 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6b04045d20 con 0x7f6b1c156a70 2021-11-21T21:56:05.781 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:56:05.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b000536d0 msgr2=0x7f6b00055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b000536d0 0x7f6b00055b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f6b0c002f70 tx=0x7f6b0c015000).stop 2021-11-21T21:56:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b1c156a70 msgr2=0x7f6b1c0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b1c156a70 0x7f6b1c0b3d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f6b04001370 tx=0x7f6b04001a90).stop 2021-11-21T21:56:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 shutdown_connections 2021-11-21T21:56:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b1c066d70 0x7f6b1c067200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b000536d0 0x7f6b00055b90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b1c156a70 0x7f6b1c0b3d80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 --2- 172.21.2.116:0/1754962332 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6afc0037d0 0x7f6b1c13f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 >> 172.21.2.116:0/1754962332 conn(0x7f6b1c05bd20 msgr2=0x7f6b1c147020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 shutdown_connections 2021-11-21T21:56:05.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.780+0000 7f6b22349700 1 -- 172.21.2.116:0/1754962332 wait complete. 2021-11-21T21:56:05.790 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T21:56:05.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.840+0000 7fd7a7ea7700 1 Processor -- start 2021-11-21T21:56:05.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.840+0000 7fd7a7ea7700 1 -- start start 2021-11-21T21:56:05.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd7a00af0e0 0x7fd7a0067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7a00b0d50 0x7fd7a013dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00af4c0 con 0x7fd7a005cc60 2021-11-21T21:56:05.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a0067a80 con 0x7fd7a00b0d50 2021-11-21T21:56:05.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a005c280 con 0x7fd7a00af0e0 2021-11-21T21:56:05.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7a00b0d50 0x7fd7a013dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7a00b0d50 0x7fd7a013dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46632/0 (socket says 172.21.2.116:46632) 2021-11-21T21:56:05.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 -- 172.21.2.116:0/2928132256 learned_addr learned my addr 172.21.2.116:0/2928132256 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 --1- 172.21.2.116:0/2928132256 >> v1:172.21.2.116:6789/0 conn(0x7fd7a005cc60 0x7fd7a01408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58588/0 (socket says 172.21.2.116:58588) 2021-11-21T21:56:05.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 -- 172.21.2.116:0/2928132256 >> v2:172.21.2.117:3300/0 conn(0x7fd7a00af0e0 msgr2=0x7fd7a0067590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:05.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 --2- 172.21.2.116:0/2928132256 >> v2:172.21.2.117:3300/0 conn(0x7fd7a00af0e0 0x7fd7a0067590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 -- 172.21.2.116:0/2928132256 >> v1:172.21.2.116:6789/0 conn(0x7fd7a005cc60 legacy=0x7fd7a01408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 -- 172.21.2.116:0/2928132256 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a005c520 con 0x7fd7a00b0d50 2021-11-21T21:56:05.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a6444700 1 --2- 172.21.2.116:0/2928132256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00b0d50 0x7fd7a013dc10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fd79c001110 tx=0x7fd798009290).ready entity=mon.0 client_cookie=1a035b463cadd237 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a4c41700 1 -- 172.21.2.116:0/2928132256 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79800b730 con 0x7fd7a00b0d50 2021-11-21T21:56:05.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a4c41700 1 -- 172.21.2.116:0/2928132256 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79800b890 con 0x7fd7a00b0d50 2021-11-21T21:56:05.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a4c41700 1 -- 172.21.2.116:0/2928132256 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79800ba40 con 0x7fd7a00b0d50 2021-11-21T21:56:05.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/2928132256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00b0d50 msgr2=0x7fd7a013dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/2928132256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00b0d50 0x7fd7a013dc10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fd79c001110 tx=0x7fd798009290).stop 2021-11-21T21:56:05.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/2928132256 shutdown_connections 2021-11-21T21:56:05.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/2928132256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00b0d50 0x7fd7a013dc10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/2928132256 >> v2:172.21.2.117:3300/0 conn(0x7fd7a00af0e0 0x7fd7a0067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/2928132256 >> 172.21.2.116:0/2928132256 conn(0x7fd7a005fd00 msgr2=0x7fd7a00600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/2928132256 shutdown_connections 2021-11-21T21:56:05.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/2928132256 wait complete. 2021-11-21T21:56:05.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 Processor -- start 2021-11-21T21:56:05.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- start start 2021-11-21T21:56:05.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a005cc60 0x7fd7a00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00af0e0 0x7fd7a0066170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7a00b0d50 0x7fd7a0142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00652b0 con 0x7fd7a00af0e0 2021-11-21T21:56:05.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0065f30 con 0x7fd7a00b0d50 2021-11-21T21:56:05.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0064d70 con 0x7fd7a005cc60 2021-11-21T21:56:05.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00af0e0 0x7fd7a0066170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00af0e0 0x7fd7a0066170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46638/0 (socket says 172.21.2.116:46638) 2021-11-21T21:56:05.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 -- 172.21.2.116:0/934825962 learned_addr learned my addr 172.21.2.116:0/934825962 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:05.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 -- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a005cc60 msgr2=0x7fd7a00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a005cc60 0x7fd7a00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 -- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7a00b0d50 msgr2=0x7fd7a0142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7a00b0d50 0x7fd7a0142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 -- 172.21.2.116:0/934825962 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798009530 con 0x7fd7a00af0e0 2021-11-21T21:56:05.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a5442700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00af0e0 0x7fd7a0066170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fd79c0011d0 tx=0x7fd79c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd78effd700 1 -- 172.21.2.116:0/934825962 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79c0072e0 con 0x7fd7a00af0e0 2021-11-21T21:56:05.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd78effd700 1 -- 172.21.2.116:0/934825962 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c005550 con 0x7fd7a00af0e0 2021-11-21T21:56:05.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a0065010 con 0x7fd7a00af0e0 2021-11-21T21:56:05.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd78effd700 1 -- 172.21.2.116:0/934825962 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79c0058b0 con 0x7fd7a00af0e0 2021-11-21T21:56:05.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.844+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a0065bf0 con 0x7fd7a00af0e0 2021-11-21T21:56:05.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.848+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a005c7c0 con 0x7fd7a00af0e0 2021-11-21T21:56:05.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.848+0000 7fd78effd700 1 -- 172.21.2.116:0/934825962 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd79c010070 con 0x7fd7a00af0e0 2021-11-21T21:56:05.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.848+0000 7fd78effd700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7800534f0 0x7fd7800559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:05.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.848+0000 7fd78effd700 1 -- 172.21.2.116:0/934825962 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd79c06af10 con 0x7fd7a00af0e0 2021-11-21T21:56:05.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.848+0000 7fd7a5c43700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7800534f0 0x7fd7800559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:05.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.848+0000 7fd7a5c43700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7800534f0 0x7fd7800559b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fd790002fb0 tx=0x7fd7900069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:05.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.848+0000 7fd78effd700 1 -- 172.21.2.116:0/934825962 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd79c042290 con 0x7fd7a00af0e0 2021-11-21T21:56:05.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd7a00573d0 con 0x7fd7a00af0e0 2021-11-21T21:56:05.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd78effd700 1 -- 172.21.2.116:0/934825962 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+143 (secure 0 0 0) 0x7fd79c0458b0 con 0x7fd7a00af0e0 2021-11-21T21:56:05.936 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:05.936 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T21:56:05.936 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.937 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T21:56:05.937 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T21:56:05.937 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:05.937 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:56:05.938 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T21:56:05.938 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T21:56:05.938 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:05.938 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T21:56:05.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7800534f0 msgr2=0x7fd7800559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7800534f0 0x7fd7800559b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fd790002fb0 tx=0x7fd7900069b0).stop 2021-11-21T21:56:05.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00af0e0 msgr2=0x7fd7a0066170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:05.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00af0e0 0x7fd7a0066170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fd79c0011d0 tx=0x7fd79c007c00).stop 2021-11-21T21:56:05.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 shutdown_connections 2021-11-21T21:56:05.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7a00b0d50 0x7fd7a0142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7800534f0 0x7fd7800559b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7a00af0e0 0x7fd7a0066170 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:05.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 --2- 172.21.2.116:0/934825962 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7a005cc60 0x7fd7a00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:05.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 >> 172.21.2.116:0/934825962 conn(0x7fd7a005fd00 msgr2=0x7fd7a01354f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:05.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 shutdown_connections 2021-11-21T21:56:05.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:05.932+0000 7fd7a7ea7700 1 -- 172.21.2.116:0/934825962 wait complete. 2021-11-21T21:56:05.943 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T21:56:05.944 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T21:56:06.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 Processor -- start 2021-11-21T21:56:06.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- start start 2021-11-21T21:56:06.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff9b8144810 0x7ff9b81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff9b80ac600 0x7ff9b80aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b8147e80 con 0x7ff9b80abce0 2021-11-21T21:56:06.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b80b1490 con 0x7ff9b8144810 2021-11-21T21:56:06.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b80b1fd0 con 0x7ff9b80ac600 2021-11-21T21:56:06.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff9b8144810 0x7ff9b81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff9b8144810 0x7ff9b81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46644/0 (socket says 172.21.2.116:46644) 2021-11-21T21:56:06.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 -- 172.21.2.116:0/883964881 learned_addr learned my addr 172.21.2.116:0/883964881 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:06.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 -- 172.21.2.116:0/883964881 >> v2:172.21.2.117:3300/0 conn(0x7ff9b80ac600 msgr2=0x7ff9b80aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:06.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 --2- 172.21.2.116:0/883964881 >> v2:172.21.2.117:3300/0 conn(0x7ff9b80ac600 0x7ff9b80aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:06.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 -- 172.21.2.116:0/883964881 >> v1:172.21.2.117:6789/0 conn(0x7ff9b80abce0 legacy=0x7ff9b80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:06.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 -- 172.21.2.116:0/883964881 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80af5a0 con 0x7ff9b8144810 2021-11-21T21:56:06.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b77fe700 1 --2- 172.21.2.116:0/883964881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b8144810 0x7ff9b81478c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7ff9b80b4e00 tx=0x7ff9a8006eb0).ready entity=mon.0 client_cookie=2cfcbc5cd572bb54 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:06.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b67fc700 1 -- 172.21.2.116:0/883964881 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9a800d5c0 con 0x7ff9b8144810 2021-11-21T21:56:06.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b67fc700 1 -- 172.21.2.116:0/883964881 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a8000e90 con 0x7ff9b8144810 2021-11-21T21:56:06.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9b67fc700 1 -- 172.21.2.116:0/883964881 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9a800d910 con 0x7ff9b8144810 2021-11-21T21:56:06.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- 172.21.2.116:0/883964881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b8144810 msgr2=0x7ff9b81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/883964881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b8144810 0x7ff9b81478c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7ff9b80b4e00 tx=0x7ff9a8006eb0).stop 2021-11-21T21:56:06.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- 172.21.2.116:0/883964881 shutdown_connections 2021-11-21T21:56:06.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/883964881 >> v2:172.21.2.117:3300/0 conn(0x7ff9b80ac600 0x7ff9b80aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:06.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/883964881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b8144810 0x7ff9b81478c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- 172.21.2.116:0/883964881 >> 172.21.2.116:0/883964881 conn(0x7ff9b805bd20 msgr2=0x7ff9b805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:06.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- 172.21.2.116:0/883964881 shutdown_connections 2021-11-21T21:56:06.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- 172.21.2.116:0/883964881 wait complete. 2021-11-21T21:56:06.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 Processor -- start 2021-11-21T21:56:06.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- start start 2021-11-21T21:56:06.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b80abce0 0x7ff9b8143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b80ac600 0x7ff9b813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b8144810 0x7ff9b813a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8148e80 con 0x7ff9b80ac600 2021-11-21T21:56:06.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8159e30 con 0x7ff9b8144810 2021-11-21T21:56:06.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.000+0000 7ff9bdca9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80b30b0 con 0x7ff9b80abce0 2021-11-21T21:56:06.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b80ac600 0x7ff9b813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b80ac600 0x7ff9b813a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46652/0 (socket says 172.21.2.116:46652) 2021-11-21T21:56:06.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 -- 172.21.2.116:0/297694274 learned_addr learned my addr 172.21.2.116:0/297694274 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:06.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b77fe700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b80abce0 0x7ff9b8143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 -- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b80abce0 msgr2=0x7ff9b8143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b7fff700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b8144810 0x7ff9b813a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b80abce0 0x7ff9b8143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 -- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b8144810 msgr2=0x7ff9b813a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b8144810 0x7ff9b813a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 -- 172.21.2.116:0/297694274 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a800b1e0 con 0x7ff9b80ac600 2021-11-21T21:56:06.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b77fe700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b80abce0 0x7ff9b8143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:56:06.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b7fff700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b8144810 0x7ff9b813a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:56:06.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b6ffd700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b80ac600 0x7ff9b813a4d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7ff9a0002fb0 tx=0x7ff9a0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:06.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b4ff9700 1 -- 172.21.2.116:0/297694274 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9a0013460 con 0x7ff9b80ac600 2021-11-21T21:56:06.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b4ff9700 1 -- 172.21.2.116:0/297694274 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a00135c0 con 0x7ff9b80ac600 2021-11-21T21:56:06.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b4ff9700 1 -- 172.21.2.116:0/297694274 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9a0013920 con 0x7ff9b80ac600 2021-11-21T21:56:06.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b8146bf0 con 0x7ff9b80ac600 2021-11-21T21:56:06.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b81463e0 con 0x7ff9b80ac600 2021-11-21T21:56:06.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b80b32b0 con 0x7ff9b80ac600 2021-11-21T21:56:06.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b4ff9700 1 -- 172.21.2.116:0/297694274 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff9a000f070 con 0x7ff9b80ac600 2021-11-21T21:56:06.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b4ff9700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff99c053660 0x7ff99c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b4ff9700 1 -- 172.21.2.116:0/297694274 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff9a006b1e0 con 0x7ff9b80ac600 2021-11-21T21:56:06.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b77fe700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff99c053660 0x7ff99c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.004+0000 7ff9b77fe700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff99c053660 0x7ff99c055b20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7ff9a8017000 tx=0x7ff9a8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:06.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.008+0000 7ff9b4ff9700 1 -- 172.21.2.116:0/297694274 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff9b80b32b0 con 0x7ff9b80ac600 2021-11-21T21:56:06.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.088+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7ff9b80b3270 con 0x7ff9b80ac600 2021-11-21T21:56:06.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.748+0000 7ff9b4ff9700 1 -- 172.21.2.116:0/297694274 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v38) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff9a0042560 con 0x7ff9b80ac600 2021-11-21T21:56:06.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff99c053660 msgr2=0x7ff99c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff99c053660 0x7ff99c055b20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7ff9a8017000 tx=0x7ff9a8015000).stop 2021-11-21T21:56:06.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b80ac600 msgr2=0x7ff9b813a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b80ac600 0x7ff9b813a4d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7ff9a0002fb0 tx=0x7ff9a0006df0).stop 2021-11-21T21:56:06.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 shutdown_connections 2021-11-21T21:56:06.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b8144810 0x7ff9b813a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff99c053660 0x7ff99c055b20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b80ac600 0x7ff9b813a4d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 --2- 172.21.2.116:0/297694274 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b80abce0 0x7ff9b8143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 >> 172.21.2.116:0/297694274 conn(0x7ff9b805bd20 msgr2=0x7ff9b81400c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:06.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 shutdown_connections 2021-11-21T21:56:06.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.752+0000 7ff9bdca9700 1 -- 172.21.2.116:0/297694274 wait complete. 2021-11-21T21:56:06.754 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T21:56:06.758 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T21:56:06.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8ca0d700 1 Processor -- start 2021-11-21T21:56:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8ca0d700 1 -- start start 2021-11-21T21:56:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8ca0d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c88144810 0x7f0c881478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8ca0d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0c880abce0 0x7f0c880b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8ca0d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c88147e80 con 0x7f0c880b1db0 2021-11-21T21:56:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8ca0d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880b1490 con 0x7f0c880abce0 2021-11-21T21:56:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8ca0d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880af060 con 0x7f0c88144810 2021-11-21T21:56:06.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0c880abce0 0x7f0c880b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0c880abce0 0x7f0c880b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46662/0 (socket says 172.21.2.116:46662) 2021-11-21T21:56:06.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 -- 172.21.2.116:0/1762333672 learned_addr learned my addr 172.21.2.116:0/1762333672 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:06.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 -- 172.21.2.116:0/1762333672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c88144810 msgr2=0x7f0c881478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:06.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 --2- 172.21.2.116:0/1762333672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c88144810 0x7f0c881478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:06.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 -- 172.21.2.116:0/1762333672 >> v1:172.21.2.117:6789/0 conn(0x7f0c880b1db0 legacy=0x7f0c880ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:06.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 -- 172.21.2.116:0/1762333672 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880af5a0 con 0x7f0c880abce0 2021-11-21T21:56:06.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c85d9b700 1 --2- 172.21.2.116:0/1762333672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c880abce0 0x7f0c880b18c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f0c880b2a30 tx=0x7f0c7c006eb0).ready entity=mon.0 client_cookie=17a66dcd004bfe2b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:06.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.808+0000 7f0c8559a700 1 -- 172.21.2.116:0/1762333672 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7c00d570 con 0x7f0c880abce0 2021-11-21T21:56:06.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8559a700 1 -- 172.21.2.116:0/1762333672 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c000e90 con 0x7f0c880abce0 2021-11-21T21:56:06.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8559a700 1 -- 172.21.2.116:0/1762333672 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7c00d8c0 con 0x7f0c880abce0 2021-11-21T21:56:06.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/1762333672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c880abce0 msgr2=0x7f0c880b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/1762333672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c880abce0 0x7f0c880b18c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f0c880b2a30 tx=0x7f0c7c006eb0).stop 2021-11-21T21:56:06.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/1762333672 shutdown_connections 2021-11-21T21:56:06.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/1762333672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c880abce0 0x7f0c880b18c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:06.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/1762333672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c88144810 0x7f0c881478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:06.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/1762333672 >> 172.21.2.116:0/1762333672 conn(0x7f0c8805bd20 msgr2=0x7f0c8805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:06.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/1762333672 shutdown_connections 2021-11-21T21:56:06.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/1762333672 wait complete. 2021-11-21T21:56:06.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 Processor -- start 2021-11-21T21:56:06.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- start start 2021-11-21T21:56:06.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c880abce0 0x7f0c88159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c88144810 0x7f0c88145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c880670c0 0x7f0c8813ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8814ad20 con 0x7f0c88144810 2021-11-21T21:56:06.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880667b0 con 0x7f0c880abce0 2021-11-21T21:56:06.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880669c0 con 0x7f0c880670c0 2021-11-21T21:56:06.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c88144810 0x7f0c88145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c88144810 0x7f0c88145b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46668/0 (socket says 172.21.2.116:46668) 2021-11-21T21:56:06.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 -- 172.21.2.116:0/577006099 learned_addr learned my addr 172.21.2.116:0/577006099 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:06.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 -- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c880670c0 msgr2=0x7f0c8813ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c880670c0 0x7f0c8813ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:06.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 -- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c880abce0 msgr2=0x7f0c88159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:06.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c880abce0 0x7f0c88159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:06.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 -- 172.21.2.116:0/577006099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c00b1e0 con 0x7f0c88144810 2021-11-21T21:56:06.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c85d9b700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c88144810 0x7f0c88145b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f0c880b2a30 tx=0x7f0c7c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:06.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c777fe700 1 -- 172.21.2.116:0/577006099 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7c006b80 con 0x7f0c88144810 2021-11-21T21:56:06.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c777fe700 1 -- 172.21.2.116:0/577006099 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00d8f0 con 0x7f0c88144810 2021-11-21T21:56:06.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8814af50 con 0x7f0c88144810 2021-11-21T21:56:06.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c777fe700 1 -- 172.21.2.116:0/577006099 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7c01e460 con 0x7f0c88144810 2021-11-21T21:56:06.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c88149e20 con 0x7f0c88144810 2021-11-21T21:56:06.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.812+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c880af300 con 0x7f0c88144810 2021-11-21T21:56:06.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.816+0000 7f0c777fe700 1 -- 172.21.2.116:0/577006099 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0c7c025030 con 0x7f0c88144810 2021-11-21T21:56:06.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.816+0000 7f0c777fe700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c64053610 0x7f0c64055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:06.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.816+0000 7f0c777fe700 1 -- 172.21.2.116:0/577006099 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f0c7c06cb00 con 0x7f0c88144810 2021-11-21T21:56:06.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.816+0000 7f0c8659c700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c64053610 0x7f0c64055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:06.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.816+0000 7f0c8659c700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c64053610 0x7f0c64055ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0c78002f70 tx=0x7f0c780069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:06.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.816+0000 7f0c777fe700 1 -- 172.21.2.116:0/577006099 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0c7c043e30 con 0x7f0c88144810 2021-11-21T21:56:06.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:06.900+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f0c880573d0 con 0x7f0c88144810 2021-11-21T21:56:07.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c777fe700 1 -- 172.21.2.116:0/577006099 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v39) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0c7c023070 con 0x7f0c88144810 2021-11-21T21:56:07.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c64053610 msgr2=0x7f0c64055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:07.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c64053610 0x7f0c64055ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0c78002f70 tx=0x7f0c780069e0).stop 2021-11-21T21:56:07.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c88144810 msgr2=0x7f0c88145b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:07.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c88144810 0x7f0c88145b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f0c880b2a30 tx=0x7f0c7c015000).stop 2021-11-21T21:56:07.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 shutdown_connections 2021-11-21T21:56:07.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c880670c0 0x7f0c8813ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c64053610 0x7f0c64055ad0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:07.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c88144810 0x7f0c88145b20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:07.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 --2- 172.21.2.116:0/577006099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c880abce0 0x7f0c88159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 >> 172.21.2.116:0/577006099 conn(0x7f0c8805bd20 msgr2=0x7f0c880af780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:07.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 shutdown_connections 2021-11-21T21:56:07.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.756+0000 7f0c8ca0d700 1 -- 172.21.2.116:0/577006099 wait complete. 2021-11-21T21:56:07.760 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T21:56:07.763 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T21:56:07.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86cd1e2700 1 Processor -- start 2021-11-21T21:56:07.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86cd1e2700 1 -- start start 2021-11-21T21:56:07.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86cd1e2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f86c80ae490 0x7f86c80a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:07.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86cd1e2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f86c80b2040 0x7f86c80b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86cd1e2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c80aff10 con 0x7f86c805f8d0 2021-11-21T21:56:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86cd1e2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c80af3b0 con 0x7f86c80ae490 2021-11-21T21:56:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86cd1e2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c80b0140 con 0x7f86c80b2040 2021-11-21T21:56:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f86c80ae490 0x7f86c80a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f86c80ae490 0x7f86c80a9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46676/0 (socket says 172.21.2.116:46676) 2021-11-21T21:56:07.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 -- 172.21.2.116:0/3972889386 learned_addr learned my addr 172.21.2.116:0/3972889386 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:07.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 -- 172.21.2.116:0/3972889386 >> v2:172.21.2.117:3300/0 conn(0x7f86c80b2040 msgr2=0x7f86c80b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:07.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 --2- 172.21.2.116:0/3972889386 >> v2:172.21.2.117:3300/0 conn(0x7f86c80b2040 0x7f86c80b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 -- 172.21.2.116:0/3972889386 >> v1:172.21.2.117:6789/0 conn(0x7f86c805f8d0 legacy=0x7f86c80a9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:07.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 -- 172.21.2.116:0/3972889386 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c80b03e0 con 0x7f86c80ae490 2021-11-21T21:56:07.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c77fe700 1 --2- 172.21.2.116:0/3972889386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c80a9710 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f86c80b69a0 tx=0x7f86bc006eb0).ready entity=mon.0 client_cookie=fb33d94ba53287c7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:07.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c6ffd700 1 -- 172.21.2.116:0/3972889386 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc00d5c0 con 0x7f86c80ae490 2021-11-21T21:56:07.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c6ffd700 1 -- 172.21.2.116:0/3972889386 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc000e90 con 0x7f86c80ae490 2021-11-21T21:56:07.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.820+0000 7f86c6ffd700 1 -- 172.21.2.116:0/3972889386 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc00d910 con 0x7f86c80ae490 2021-11-21T21:56:07.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/3972889386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 msgr2=0x7f86c80a9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:07.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/3972889386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c80a9710 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f86c80b69a0 tx=0x7f86bc006eb0).stop 2021-11-21T21:56:07.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/3972889386 shutdown_connections 2021-11-21T21:56:07.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/3972889386 >> v2:172.21.2.117:3300/0 conn(0x7f86c80b2040 0x7f86c80b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/3972889386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c80a9710 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:07.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/3972889386 >> 172.21.2.116:0/3972889386 conn(0x7f86c80b5e60 msgr2=0x7f86c80b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:07.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/3972889386 shutdown_connections 2021-11-21T21:56:07.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/3972889386 wait complete. 2021-11-21T21:56:07.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 Processor -- start 2021-11-21T21:56:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- start start 2021-11-21T21:56:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86c805f8d0 0x7f86c813f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c813e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86c80b2040 0x7f86c8140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c813eae0 con 0x7f86c80ae490 2021-11-21T21:56:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c8067b10 con 0x7f86c805f8d0 2021-11-21T21:56:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c813ed10 con 0x7f86c80b2040 2021-11-21T21:56:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c813e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c813e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46680/0 (socket says 172.21.2.116:46680) 2021-11-21T21:56:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 -- 172.21.2.116:0/616202756 learned_addr learned my addr 172.21.2.116:0/616202756 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 -- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86c80b2040 msgr2=0x7f86c8140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86c80b2040 0x7f86c8140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 -- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86c805f8d0 msgr2=0x7f86c813f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:07.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86c805f8d0 0x7f86c813f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 -- 172.21.2.116:0/616202756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc00b1e0 con 0x7f86c80ae490 2021-11-21T21:56:07.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c77fe700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c813e5f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f86c80b69a0 tx=0x7f86bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:07.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c57fa700 1 -- 172.21.2.116:0/616202756 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc002f70 con 0x7f86c80ae490 2021-11-21T21:56:07.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c57fa700 1 -- 172.21.2.116:0/616202756 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00d940 con 0x7f86c80ae490 2021-11-21T21:56:07.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c813efb0 con 0x7f86c80ae490 2021-11-21T21:56:07.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c80676e0 con 0x7f86c80ae490 2021-11-21T21:56:07.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c57fa700 1 -- 172.21.2.116:0/616202756 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc011e90 con 0x7f86c80ae490 2021-11-21T21:56:07.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c813f140 con 0x7f86c80ae490 2021-11-21T21:56:07.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c57fa700 1 -- 172.21.2.116:0/616202756 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f86bc025030 con 0x7f86c80ae490 2021-11-21T21:56:07.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.824+0000 7f86c57fa700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f86a8053660 0x7f86a8055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:07.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.828+0000 7f86c57fa700 1 -- 172.21.2.116:0/616202756 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f86bc06cda0 con 0x7f86c80ae490 2021-11-21T21:56:07.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.828+0000 7f86c7fff700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f86a8053660 0x7f86a8055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:07.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.828+0000 7f86c7fff700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f86a8053660 0x7f86a8055b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f86b8002f90 tx=0x7f86b80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:07.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.828+0000 7f86c57fa700 1 -- 172.21.2.116:0/616202756 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86bc043e60 con 0x7f86c80ae490 2021-11-21T21:56:07.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.940+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f86c80a9ec0 con 0x7f86c80ae490 2021-11-21T21:56:07.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.944+0000 7f86c57fa700 1 -- 172.21.2.116:0/616202756 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v14) v1 ==== 187+0+0 (secure 0 0 0) 0x7f86bc0230d0 con 0x7f86c80ae490 2021-11-21T21:56:07.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f86a8053660 msgr2=0x7f86a8055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:07.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f86a8053660 0x7f86a8055b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f86b8002f90 tx=0x7f86b80069b0).stop 2021-11-21T21:56:07.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 msgr2=0x7f86c813e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:07.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c813e5f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f86c80b69a0 tx=0x7f86bc015000).stop 2021-11-21T21:56:07.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 shutdown_connections 2021-11-21T21:56:07.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86c80b2040 0x7f86c8140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f86a8053660 0x7f86a8055b20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:07.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86c80ae490 0x7f86c813e5f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:07.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 --2- 172.21.2.116:0/616202756 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86c805f8d0 0x7f86c813f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:07.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 >> 172.21.2.116:0/616202756 conn(0x7f86c80b5e60 msgr2=0x7f86c80b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:07.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 shutdown_connections 2021-11-21T21:56:07.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:07.948+0000 7f86cd1e2700 1 -- 172.21.2.116:0/616202756 wait complete. 2021-11-21T21:56:07.955 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 7 and data pool 8 2021-11-21T21:56:07.957 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:56:08.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e372a700 1 Processor -- start 2021-11-21T21:56:08.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e372a700 1 -- start start 2021-11-21T21:56:08.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e372a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb0dc0a6aa0 0x7fb0dc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e372a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0a86b0 0x7fb0dc1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e372a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc0a6e80 con 0x7fb0dc1475d0 2021-11-21T21:56:08.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e372a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc0a8bc0 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e372a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc14a7b0 con 0x7fb0dc0a86b0 2021-11-21T21:56:08.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e1cc7700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fb0dc1475d0 0x7fb0dc1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:57556/0 (socket says 172.21.2.116:57556) 2021-11-21T21:56:08.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e14c6700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb0dc0a6aa0 0x7fb0dc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e1cc7700 1 -- 172.21.2.116:0/520118404 learned_addr learned my addr 172.21.2.116:0/520118404 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:08.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e0cc5700 1 --2- 172.21.2.116:0/520118404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0a86b0 0x7fb0dc1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0cbfff700 1 -- 172.21.2.116:0/520118404 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910044066 0 0) 0x7fb0dc0a6e80 con 0x7fb0dc1475d0 2021-11-21T21:56:08.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e0cc5700 1 -- 172.21.2.116:0/520118404 >> v2:172.21.2.117:3300/0 conn(0x7fb0dc0a6aa0 msgr2=0x7fb0dc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:08.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e0cc5700 1 --2- 172.21.2.116:0/520118404 >> v2:172.21.2.117:3300/0 conn(0x7fb0dc0a6aa0 0x7fb0dc14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e0cc5700 1 -- 172.21.2.116:0/520118404 >> v1:172.21.2.117:6789/0 conn(0x7fb0dc1475d0 legacy=0x7fb0dc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e0cc5700 1 -- 172.21.2.116:0/520118404 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc14aa10 con 0x7fb0dc0a86b0 2021-11-21T21:56:08.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.008+0000 7fb0e0cc5700 1 --2- 172.21.2.116:0/520118404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0a86b0 0x7fb0dc1470e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb0cc002f70 tx=0x7fb0cc006df0).ready entity=mon.2 client_cookie=d192daccba6d8db8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0cbfff700 1 -- 172.21.2.116:0/520118404 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0cc00d570 con 0x7fb0dc0a86b0 2021-11-21T21:56:08.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0cbfff700 1 -- 172.21.2.116:0/520118404 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc000e90 con 0x7fb0dc0a86b0 2021-11-21T21:56:08.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0cbfff700 1 -- 172.21.2.116:0/520118404 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0cc00d8c0 con 0x7fb0dc0a86b0 2021-11-21T21:56:08.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/520118404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0a86b0 msgr2=0x7fb0dc1470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 --2- 172.21.2.116:0/520118404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0a86b0 0x7fb0dc1470e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb0cc002f70 tx=0x7fb0cc006df0).stop 2021-11-21T21:56:08.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/520118404 shutdown_connections 2021-11-21T21:56:08.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 --2- 172.21.2.116:0/520118404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc0a86b0 0x7fb0dc1470e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 --2- 172.21.2.116:0/520118404 >> v2:172.21.2.117:3300/0 conn(0x7fb0dc0a6aa0 0x7fb0dc14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/520118404 >> 172.21.2.116:0/520118404 conn(0x7fb0dc05bd20 msgr2=0x7fb0dc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:08.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/520118404 shutdown_connections 2021-11-21T21:56:08.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/520118404 wait complete. 2021-11-21T21:56:08.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 Processor -- start 2021-11-21T21:56:08.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- start start 2021-11-21T21:56:08.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0a6aa0 0x7fb0dc14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0a86b0 0x7fb0dc063af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc1475d0 0x7fb0dc145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1457b0 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1465d0 con 0x7fb0dc0a86b0 2021-11-21T21:56:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc063fe0 con 0x7fb0dc1475d0 2021-11-21T21:56:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0a6aa0 0x7fb0dc14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0a6aa0 0x7fb0dc14d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46692/0 (socket says 172.21.2.116:46692) 2021-11-21T21:56:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 -- 172.21.2.116:0/3554571697 learned_addr learned my addr 172.21.2.116:0/3554571697 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:08.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e0cc5700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0a86b0 0x7fb0dc063af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e1cc7700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc1475d0 0x7fb0dc145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 -- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc1475d0 msgr2=0x7fb0dc145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc1475d0 0x7fb0dc145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 -- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0a86b0 msgr2=0x7fb0dc063af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0a86b0 0x7fb0dc063af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 -- 172.21.2.116:0/3554571697 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc00b180 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e14c6700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0a6aa0 0x7fb0dc14d080 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fb0d800c000 tx=0x7fb0d8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0ca7fc700 1 -- 172.21.2.116:0/3554571697 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0d8010780 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0ca7fc700 1 -- 172.21.2.116:0/3554571697 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80108e0 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0ca7fc700 1 -- 172.21.2.116:0/3554571697 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0d8010c40 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc063850 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc146800 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.012+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc146c40 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.016+0000 7fb0ca7fc700 1 -- 172.21.2.116:0/3554571697 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb0d801c460 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.016+0000 7fb0ca7fc700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0bc0535d0 0x7fb0bc055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.016+0000 7fb0ca7fc700 1 -- 172.21.2.116:0/3554571697 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb0d806bdf0 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.016+0000 7fb0e0cc5700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0bc0535d0 0x7fb0bc055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.016+0000 7fb0e0cc5700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0bc0535d0 0x7fb0bc055a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb0cc01a000 tx=0x7fb0cc014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.016+0000 7fb0ca7fc700 1 -- 172.21.2.116:0/3554571697 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb0d8042de0 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.100+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0dc05db70 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0ca7fc700 1 -- 172.21.2.116:0/3554571697 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16064 (secure 0 0 0) 0x7fb0d8046400 con 0x7fb0dc0a6aa0 2021-11-21T21:56:08.104 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:08.104 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":40,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:56:07.942281+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:56:06.102882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-11-21T21:56:06.914261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:56:08.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0bc0535d0 msgr2=0x7fb0bc055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0bc0535d0 0x7fb0bc055a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb0cc01a000 tx=0x7fb0cc014000).stop 2021-11-21T21:56:08.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0a6aa0 msgr2=0x7fb0dc14d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0a6aa0 0x7fb0dc14d080 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fb0d800c000 tx=0x7fb0d8006df0).stop 2021-11-21T21:56:08.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 shutdown_connections 2021-11-21T21:56:08.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0dc1475d0 0x7fb0dc145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0dc0a86b0 0x7fb0dc063af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0bc0535d0 0x7fb0bc055a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 --2- 172.21.2.116:0/3554571697 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0dc0a6aa0 0x7fb0dc14d080 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 >> 172.21.2.116:0/3554571697 conn(0x7fb0dc05bd20 msgr2=0x7fb0dc142e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:08.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 shutdown_connections 2021-11-21T21:56:08.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.104+0000 7fb0e372a700 1 -- 172.21.2.116:0/3554571697 wait complete. 2021-11-21T21:56:08.111 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:56:08.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54bbe2700 1 Processor -- start 2021-11-21T21:56:08.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54bbe2700 1 -- start start 2021-11-21T21:56:08.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54bbe2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd5440a86b0 0x7fd54414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54bbe2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5440a6e80 con 0x7fd5440a6aa0 2021-11-21T21:56:08.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54bbe2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54414a7b0 con 0x7fd544062410 2021-11-21T21:56:08.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54bbe2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5440a8bc0 con 0x7fd5440a86b0 2021-11-21T21:56:08.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54997e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd5440a6aa0 0x7fd5441470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58656/0 (socket says 172.21.2.116:58656) 2021-11-21T21:56:08.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54997e700 1 -- 172.21.2.116:0/2509548888 learned_addr learned my addr 172.21.2.116:0/2509548888 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:08.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54917d700 1 --2- 172.21.2.116:0/2509548888 >> v2:172.21.2.116:3300/0 conn(0x7fd5440a86b0 0x7fd54414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54897c700 1 -- 172.21.2.116:0/2509548888 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373684418 0 0) 0x7fd5440a6e80 con 0x7fd5440a6aa0 2021-11-21T21:56:08.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.160+0000 7fd54897c700 1 -- 172.21.2.116:0/2509548888 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c003260 con 0x7fd5440a6aa0 2021-11-21T21:56:08.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 -- 172.21.2.116:0/2509548888 >> v1:172.21.2.117:6789/0 conn(0x7fd544062410 legacy=0x7fd5441443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:08.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 -- 172.21.2.116:0/2509548888 >> v1:172.21.2.116:6789/0 conn(0x7fd5440a6aa0 legacy=0x7fd5441470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 -- 172.21.2.116:0/2509548888 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54414aa10 con 0x7fd5440a86b0 2021-11-21T21:56:08.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 --2- 172.21.2.116:0/2509548888 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a86b0 0x7fd54414b150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fd544061b50 tx=0x7fd540006df0).ready entity=mon.0 client_cookie=650399830a9a18ba server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54897c700 1 -- 172.21.2.116:0/2509548888 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd54000d5c0 con 0x7fd5440a86b0 2021-11-21T21:56:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54897c700 1 -- 172.21.2.116:0/2509548888 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd540000e90 con 0x7fd5440a86b0 2021-11-21T21:56:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54897c700 1 -- 172.21.2.116:0/2509548888 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd54000d910 con 0x7fd5440a86b0 2021-11-21T21:56:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/2509548888 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a86b0 msgr2=0x7fd54414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/2509548888 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a86b0 0x7fd54414b150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fd544061b50 tx=0x7fd540006df0).stop 2021-11-21T21:56:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/2509548888 shutdown_connections 2021-11-21T21:56:08.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/2509548888 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a86b0 0x7fd54414b150 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/2509548888 >> 172.21.2.116:0/2509548888 conn(0x7fd54405bd20 msgr2=0x7fd54405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/2509548888 shutdown_connections 2021-11-21T21:56:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/2509548888 wait complete. 2021-11-21T21:56:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 Processor -- start 2021-11-21T21:56:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- start start 2021-11-21T21:56:08.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd544062410 0x7fd54413fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a6aa0 0x7fd5441456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5440a86b0 0x7fd544147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd544140100 con 0x7fd5440a6aa0 2021-11-21T21:56:08.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd544146bf0 con 0x7fd544062410 2021-11-21T21:56:08.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd544146e90 con 0x7fd5440a86b0 2021-11-21T21:56:08.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a6aa0 0x7fd5441456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a6aa0 0x7fd5441456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46710/0 (socket says 172.21.2.116:46710) 2021-11-21T21:56:08.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 -- 172.21.2.116:0/3372014709 learned_addr learned my addr 172.21.2.116:0/3372014709 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:08.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 -- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5440a86b0 msgr2=0x7fd544147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5440a86b0 0x7fd544147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 -- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd544062410 msgr2=0x7fd54413fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd544062410 0x7fd54413fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 -- 172.21.2.116:0/3372014709 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54000b180 con 0x7fd5440a6aa0 2021-11-21T21:56:08.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54917d700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a6aa0 0x7fd5441456e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fd544061b50 tx=0x7fd540014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd536ffd700 1 -- 172.21.2.116:0/3372014709 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd540002f70 con 0x7fd5440a6aa0 2021-11-21T21:56:08.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd536ffd700 1 -- 172.21.2.116:0/3372014709 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54001e400 con 0x7fd5440a6aa0 2021-11-21T21:56:08.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd544147130 con 0x7fd5440a6aa0 2021-11-21T21:56:08.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd536ffd700 1 -- 172.21.2.116:0/3372014709 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd54001e760 con 0x7fd5440a6aa0 2021-11-21T21:56:08.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd544145100 con 0x7fd5440a6aa0 2021-11-21T21:56:08.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.164+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd544147f00 con 0x7fd5440a6aa0 2021-11-21T21:56:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.168+0000 7fd536ffd700 1 -- 172.21.2.116:0/3372014709 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd540025030 con 0x7fd5440a6aa0 2021-11-21T21:56:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.168+0000 7fd536ffd700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd524053690 0x7fd524055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.168+0000 7fd536ffd700 1 -- 172.21.2.116:0/3372014709 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd54006c880 con 0x7fd5440a6aa0 2021-11-21T21:56:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.168+0000 7fd54997e700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd524053690 0x7fd524055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.168+0000 7fd54997e700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd524053690 0x7fd524055b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fd538001110 tx=0x7fd5380076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.168+0000 7fd536ffd700 1 -- 172.21.2.116:0/3372014709 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5400438d0 con 0x7fd5440a6aa0 2021-11-21T21:56:08.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.256+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5440573d0 con 0x7fd5440a6aa0 2021-11-21T21:56:08.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.256+0000 7fd536ffd700 1 -- 172.21.2.116:0/3372014709 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16193 (secure 0 0 0) 0x7fd540023090 con 0x7fd5440a6aa0 2021-11-21T21:56:08.259 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:08.259 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":43,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:56:08.228292+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:56:06.102882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-11-21T21:56:06.914261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.116:6835/2140793945":"2021-11-22T21:56:08.223482+0000","172.21.2.116:6834/2140793945":"2021-11-22T21:56:08.223482+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:56:08.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd524053690 msgr2=0x7fd524055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd524053690 0x7fd524055b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fd538001110 tx=0x7fd5380076c0).stop 2021-11-21T21:56:08.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a6aa0 msgr2=0x7fd5441456e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a6aa0 0x7fd5441456e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fd544061b50 tx=0x7fd540014000).stop 2021-11-21T21:56:08.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 shutdown_connections 2021-11-21T21:56:08.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5440a86b0 0x7fd544147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd524053690 0x7fd524055b50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5440a6aa0 0x7fd5441456e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 --2- 172.21.2.116:0/3372014709 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd544062410 0x7fd54413fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 >> 172.21.2.116:0/3372014709 conn(0x7fd54405bd20 msgr2=0x7fd5441441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:08.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 shutdown_connections 2021-11-21T21:56:08.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.260+0000 7fd54bbe2700 1 -- 172.21.2.116:0/3372014709 wait complete. 2021-11-21T21:56:08.267 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T21:56:08.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83c4a52700 1 Processor -- start 2021-11-21T21:56:08.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83c4a52700 1 -- start start 2021-11-21T21:56:08.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83c4a52700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f83c00b3690 0x7f83c00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83c4a52700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c00b3c50 con 0x7f83c0144a10 2021-11-21T21:56:08.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83c4a52700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c00b3ef0 con 0x7f83c00adc10 2021-11-21T21:56:08.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83c4a52700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c00b4190 con 0x7f83c00b3690 2021-11-21T21:56:08.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bdd9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f83c0144a10 0x7f83c013a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58674/0 (socket says 172.21.2.116:58674) 2021-11-21T21:56:08.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bdd9b700 1 -- 172.21.2.116:0/1416923256 learned_addr learned my addr 172.21.2.116:0/1416923256 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:08.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718211545 0 0) 0x7f83c00b3c50 con 0x7f83c0144a10 2021-11-21T21:56:08.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a0003260 con 0x7f83c0144a10 2021-11-21T21:56:08.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750118730 0 0) 0x7f83a0003260 con 0x7f83c0144a10 2021-11-21T21:56:08.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 >> v2:172.21.2.117:3300/0 conn(0x7f83c00b3690 msgr2=0x7f83c00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:08.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 --2- 172.21.2.116:0/1416923256 >> v2:172.21.2.117:3300/0 conn(0x7f83c00b3690 0x7f83c00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 >> v1:172.21.2.117:6789/0 conn(0x7f83c00adc10 legacy=0x7f83c0141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:08.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0140b80 con 0x7f83c0144a10 2021-11-21T21:56:08.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f83a8001660 con 0x7f83c0144a10 2021-11-21T21:56:08.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 >> v1:172.21.2.116:6789/0 conn(0x7f83c0144a10 legacy=0x7f83c013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83a0003820 0x7f83a0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83a0004500 0x7f83a0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83a0005050 0x7f83a000fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00b4190 con 0x7f83a0003820 2021-11-21T21:56:08.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a0010350 con 0x7f83a0005050 2021-11-21T21:56:08.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a0010510 con 0x7f83a0004500 2021-11-21T21:56:08.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83be59c700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83a0003820 0x7f83a0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83be59c700 1 -- 172.21.2.116:0/1416923256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83a0004500 msgr2=0x7f83a0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:08.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83be59c700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83a0004500 0x7f83a0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83be59c700 1 -- 172.21.2.116:0/1416923256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83a0005050 msgr2=0x7f83a000fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:08.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83be59c700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83a0005050 0x7f83a000fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83be59c700 1 -- 172.21.2.116:0/1416923256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0003260 con 0x7f83a0003820 2021-11-21T21:56:08.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83bed9d700 1 -- 172.21.2.116:0/1416923256 reap_dead start 2021-11-21T21:56:08.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.320+0000 7f83be59c700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83a0003820 0x7f83a0003c20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f83b0000f70 tx=0x7f83b00079e0).ready entity=mon.0 client_cookie=f12ba3478b2e92af server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b0007cf0 con 0x7f83a0003820 2021-11-21T21:56:08.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0014410 con 0x7f83a0003820 2021-11-21T21:56:08.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bd59a700 1 -- 172.21.2.116:0/1416923256 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b0014770 con 0x7f83a0003820 2021-11-21T21:56:08.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/1416923256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83a0003820 msgr2=0x7f83a0003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83a0003820 0x7f83a0003c20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f83b0000f70 tx=0x7f83b00079e0).stop 2021-11-21T21:56:08.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/1416923256 shutdown_connections 2021-11-21T21:56:08.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 --2- 172.21.2.116:0/1416923256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83a0003820 0x7f83a0003c20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/1416923256 >> 172.21.2.116:0/1416923256 conn(0x7f83c005bd20 msgr2=0x7f83c005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:08.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/1416923256 shutdown_connections 2021-11-21T21:56:08.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/1416923256 wait complete. 2021-11-21T21:56:08.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 Processor -- start 2021-11-21T21:56:08.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- start start 2021-11-21T21:56:08.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83a0003820 0x7f83c00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83c01102a0 0x7f83c00b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83c00b5850 0x7f83c013d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00affc0 con 0x7f83c00b5850 2021-11-21T21:56:08.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c0112460 con 0x7f83a0003820 2021-11-21T21:56:08.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00afa80 con 0x7f83c01102a0 2021-11-21T21:56:08.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83c00b5850 0x7f83c013d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83c00b5850 0x7f83c013d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46730/0 (socket says 172.21.2.116:46730) 2021-11-21T21:56:08.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 -- 172.21.2.116:0/3561905184 learned_addr learned my addr 172.21.2.116:0/3561905184 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:08.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 -- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83c01102a0 msgr2=0x7f83c00b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83c01102a0 0x7f83c00b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 -- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83a0003820 msgr2=0x7f83c00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83a0003820 0x7f83c00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 -- 172.21.2.116:0/3561905184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c00afd20 con 0x7f83c00b5850 2021-11-21T21:56:08.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83bed9d700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83c00b5850 0x7f83c013d950 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f83c0060150 tx=0x7f83b800a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83b77fe700 1 -- 172.21.2.116:0/3561905184 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b8005a20 con 0x7f83c00b5850 2021-11-21T21:56:08.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83b77fe700 1 -- 172.21.2.116:0/3561905184 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8005b80 con 0x7f83c00b5850 2021-11-21T21:56:08.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83b77fe700 1 -- 172.21.2.116:0/3561905184 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b8007070 con 0x7f83c00b5850 2021-11-21T21:56:08.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c00aee50 con 0x7f83c00b5850 2021-11-21T21:56:08.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c00af2f0 con 0x7f83c00b5850 2021-11-21T21:56:08.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c00aefe0 con 0x7f83c00b5850 2021-11-21T21:56:08.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83b77fe700 1 -- 172.21.2.116:0/3561905184 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f83b801d070 con 0x7f83c00b5850 2021-11-21T21:56:08.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83b77fe700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83a40536a0 0x7f83a4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:08.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83b77fe700 1 -- 172.21.2.116:0/3561905184 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f83b806ac60 con 0x7f83c00b5850 2021-11-21T21:56:08.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.324+0000 7f83be59c700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83a40536a0 0x7f83a4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:08.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.328+0000 7f83be59c700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83a40536a0 0x7f83a4055b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f83b0000f70 tx=0x7f83b0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:08.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.328+0000 7f83b77fe700 1 -- 172.21.2.116:0/3561905184 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83b8041c60 con 0x7f83c00b5850 2021-11-21T21:56:08.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.440+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f83c00573d0 con 0x7f83c00b5850 2021-11-21T21:56:08.944 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T21:56:08.944+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T21:56:08.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.952+0000 7f83b77fe700 1 -- 172.21.2.116:0/3561905184 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f83b8045280 con 0x7f83c00b5850 2021-11-21T21:56:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83a40536a0 msgr2=0x7f83a4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83a40536a0 0x7f83a4055b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f83b0000f70 tx=0x7f83b0015000).stop 2021-11-21T21:56:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83c00b5850 msgr2=0x7f83c013d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83c00b5850 0x7f83c013d950 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f83c0060150 tx=0x7f83b800a060).stop 2021-11-21T21:56:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 shutdown_connections 2021-11-21T21:56:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83a40536a0 0x7f83a4055b60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83c00b5850 0x7f83c013d950 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83c01102a0 0x7f83c00b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 --2- 172.21.2.116:0/3561905184 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83a0003820 0x7f83c00b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 >> 172.21.2.116:0/3561905184 conn(0x7f83c005bd20 msgr2=0x7f83c00673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 shutdown_connections 2021-11-21T21:56:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:08.956+0000 7f83c4a52700 1 -- 172.21.2.116:0/3561905184 wait complete. 2021-11-21T21:56:08.964 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:09.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06bd705700 1 Processor -- start 2021-11-21T21:56:09.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06bd705700 1 -- start start 2021-11-21T21:56:09.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06bd705700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f06b805ced0 0x7f06b805fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06bd705700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 0x7f06b8099320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06bd705700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b814abc0 con 0x7f06b814a730 2021-11-21T21:56:09.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06bd705700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b80b3de0 con 0x7f06b805ced0 2021-11-21T21:56:09.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06bd705700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80608c0 con 0x7f06b8098ef0 2021-11-21T21:56:09.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b67fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f06b814a730 0x7f06b80b4240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58706/0 (socket says 172.21.2.116:58706) 2021-11-21T21:56:09.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b67fc700 1 -- 172.21.2.116:0/1653205286 learned_addr learned my addr 172.21.2.116:0/1653205286 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:09.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405469563 0 0) 0x7f06b814abc0 con 0x7f06b814a730 2021-11-21T21:56:09.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0698003260 con 0x7f06b814a730 2021-11-21T21:56:09.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/1653205286 >> v2:172.21.2.117:3300/0 conn(0x7f06b805ced0 0x7f06b805fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4163949356 0 0) 0x7f0698003260 con 0x7f06b814a730 2021-11-21T21:56:09.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 msgr2=0x7f06b8099320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.012+0000 7f06b5ffb700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 0x7f06b8099320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 >> v2:172.21.2.117:3300/0 conn(0x7f06b805ced0 msgr2=0x7f06b805fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 --2- 172.21.2.116:0/1653205286 >> v2:172.21.2.117:3300/0 conn(0x7f06b805ced0 0x7f06b805fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b80b4950 con 0x7f06b814a730 2021-11-21T21:56:09.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f06a0001660 con 0x7f06b814a730 2021-11-21T21:56:09.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 >> v1:172.21.2.116:6789/0 conn(0x7f06b814a730 legacy=0x7f06b80b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0698003750 0x7f0698003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0698004500 0x7f0698004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 0x7f0698004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80b3de0 con 0x7f0698003750 2021-11-21T21:56:09.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80608c0 con 0x7f0698004500 2021-11-21T21:56:09.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069800b8b0 con 0x7f06b8098ef0 2021-11-21T21:56:09.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0698003750 0x7f0698003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b67fc700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0698004500 0x7f0698004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 0x7f0698004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 -- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 msgr2=0x7f0698004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 0x7f0698004ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 -- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0698004500 msgr2=0x7f0698004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0698004500 0x7f0698004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 -- 172.21.2.116:0/1653205286 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0698003260 con 0x7f0698003750 2021-11-21T21:56:09.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06b8098ef0 0x7f0698004ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:56:09.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0698003750 0x7f0698003b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f06a800a000 tx=0x7f06a8008040).ready entity=mon.0 client_cookie=f74e4df20e892dc3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a8013670 con 0x7f0698003750 2021-11-21T21:56:09.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a80137d0 con 0x7f0698003750 2021-11-21T21:56:09.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b5ffb700 1 -- 172.21.2.116:0/1653205286 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a8013b30 con 0x7f0698003750 2021-11-21T21:56:09.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- 172.21.2.116:0/1653205286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0698003750 msgr2=0x7f0698003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 --2- 172.21.2.116:0/1653205286 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0698003750 0x7f0698003b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f06a800a000 tx=0x7f06a8008040).stop 2021-11-21T21:56:09.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 -- 172.21.2.116:0/1653205286 reap_dead start 2021-11-21T21:56:09.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- 172.21.2.116:0/1653205286 shutdown_connections 2021-11-21T21:56:09.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- 172.21.2.116:0/1653205286 >> 172.21.2.116:0/1653205286 conn(0x7f06b80b9da0 msgr2=0x7f06b80b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- 172.21.2.116:0/1653205286 shutdown_connections 2021-11-21T21:56:09.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- 172.21.2.116:0/1653205286 wait complete. 2021-11-21T21:56:09.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 Processor -- start 2021-11-21T21:56:09.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- start start 2021-11-21T21:56:09.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0698003750 0x7f06b80b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b809eaf0 0x7f06b809ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b8159fc0 0x7f06b815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80b3040 con 0x7f06b8159fc0 2021-11-21T21:56:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80b1dd0 con 0x7f06b809eaf0 2021-11-21T21:56:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80b3270 con 0x7f0698003750 2021-11-21T21:56:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b8159fc0 0x7f06b815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b8159fc0 0x7f06b815a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46764/0 (socket says 172.21.2.116:46764) 2021-11-21T21:56:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 -- 172.21.2.116:0/2585758313 learned_addr learned my addr 172.21.2.116:0/2585758313 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0698003750 0x7f06b80b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b67fc700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b809eaf0 0x7f06b809ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 -- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0698003750 msgr2=0x7f06b80b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0698003750 0x7f06b80b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 -- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b809eaf0 msgr2=0x7f06b809ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b809eaf0 0x7f06b809ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 -- 172.21.2.116:0/2585758313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b809c360 con 0x7f06b8159fc0 2021-11-21T21:56:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06b77fe700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b8159fc0 0x7f06b815a470 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f06a00017c0 tx=0x7f06b0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06affff700 1 -- 172.21.2.116:0/2585758313 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b000ceb0 con 0x7f06b8159fc0 2021-11-21T21:56:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06affff700 1 -- 172.21.2.116:0/2585758313 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b000f4e0 con 0x7f06b8159fc0 2021-11-21T21:56:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06affff700 1 -- 172.21.2.116:0/2585758313 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b000f840 con 0x7f06b8159fc0 2021-11-21T21:56:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b80b2da0 con 0x7f06b8159fc0 2021-11-21T21:56:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.016+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b809c600 con 0x7f06b8159fc0 2021-11-21T21:56:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.020+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b8060380 con 0x7f06b8159fc0 2021-11-21T21:56:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.020+0000 7f06affff700 1 -- 172.21.2.116:0/2585758313 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f06b0019030 con 0x7f06b8159fc0 2021-11-21T21:56:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.020+0000 7f06affff700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f069c0534f0 0x7f069c0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.020+0000 7f06affff700 1 -- 172.21.2.116:0/2585758313 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f06b006b420 con 0x7f06b8159fc0 2021-11-21T21:56:09.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.020+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f069c0534f0 0x7f069c0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.020+0000 7f06b6ffd700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f069c0534f0 0x7f069c0559b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f06a8019000 tx=0x7f06a8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.020+0000 7f06affff700 1 -- 172.21.2.116:0/2585758313 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06b0042530 con 0x7f06b8159fc0 2021-11-21T21:56:09.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06b80573d0 con 0x7f06b8159fc0 2021-11-21T21:56:09.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06affff700 1 -- 172.21.2.116:0/2585758313 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2494 (secure 0 0 0) 0x7f06b0045b50 con 0x7f06b8159fc0 2021-11-21T21:56:09.134 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:09.134 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:08.956930+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:creating","state_seq":2,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:09.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f069c0534f0 msgr2=0x7f069c0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f069c0534f0 0x7f069c0559b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f06a8019000 tx=0x7f06a8008040).stop 2021-11-21T21:56:09.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b8159fc0 msgr2=0x7f06b815a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b8159fc0 0x7f06b815a470 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f06a00017c0 tx=0x7f06b0008040).stop 2021-11-21T21:56:09.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 shutdown_connections 2021-11-21T21:56:09.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f069c0534f0 0x7f069c0559b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06b8159fc0 0x7f06b815a470 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06b809eaf0 0x7f06b809ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 --2- 172.21.2.116:0/2585758313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0698003750 0x7f06b80b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 >> 172.21.2.116:0/2585758313 conn(0x7f06b80b9da0 msgr2=0x7f06b805bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 shutdown_connections 2021-11-21T21:56:09.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.132+0000 7f06bd705700 1 -- 172.21.2.116:0/2585758313 wait complete. 2021-11-21T21:56:09.139 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 16 2021-11-21T21:56:09.141 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:56:09.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.188+0000 7f99198de700 1 Processor -- start 2021-11-21T21:56:09.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.188+0000 7f99198de700 1 -- start start 2021-11-21T21:56:09.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140ac690 0x7f99140adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99140a8650 con 0x7f9914143200 2021-11-21T21:56:09.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99140aca70 con 0x7f99140b2350 2021-11-21T21:56:09.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99140acd30 con 0x7f99140ac690 2021-11-21T21:56:09.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99137fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9914143200 0x7f99141436b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58724/0 (socket says 172.21.2.116:58724) 2021-11-21T21:56:09.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99137fe700 1 -- 172.21.2.116:0/122724294 learned_addr learned my addr 172.21.2.116:0/122724294 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:09.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601751788 0 0) 0x7f99140a8650 con 0x7f9914143200 2021-11-21T21:56:09.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f4003260 con 0x7f9914143200 2021-11-21T21:56:09.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140ac690 0x7f99140adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4029691625 0 0) 0x7f98f4003260 con 0x7f9914143200 2021-11-21T21:56:09.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140ac690 msgr2=0x7f99140adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140ac690 0x7f99140adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 >> v1:172.21.2.117:6789/0 conn(0x7f99140b2350 legacy=0x7f99140b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140ae4e0 con 0x7f9914143200 2021-11-21T21:56:09.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f990c0019e0 con 0x7f9914143200 2021-11-21T21:56:09.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 >> v1:172.21.2.116:6789/0 conn(0x7f9914143200 legacy=0x7f99141436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f98f4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98f4004500 0x7f98f4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140ac690 0x7f98f4004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140acd30 con 0x7f98f4003800 2021-11-21T21:56:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f400b8e0 con 0x7f98f4004500 2021-11-21T21:56:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f400baa0 con 0x7f99140ac690 2021-11-21T21:56:09.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f98f4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 -- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140ac690 msgr2=0x7f98f4004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140ac690 0x7f98f4004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 -- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98f4004500 msgr2=0x7f98f4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99127fc700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98f4004500 0x7f98f4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98f4004500 0x7f98f4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 -- 172.21.2.116:0/122724294 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140ae4e0 con 0x7f98f4003800 2021-11-21T21:56:09.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99127fc700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98f4004500 0x7f98f4004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:56:09.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9912ffd700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f98f4003c20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f990400a000 tx=0x7f9904008040).ready entity=mon.0 client_cookie=226a1fda40fa699d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9904013670 con 0x7f98f4003800 2021-11-21T21:56:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99040137d0 con 0x7f98f4003800 2021-11-21T21:56:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f9911ffb700 1 -- 172.21.2.116:0/122724294 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9904013b30 con 0x7f98f4003800 2021-11-21T21:56:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- 172.21.2.116:0/122724294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 msgr2=0x7f98f4003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 --2- 172.21.2.116:0/122724294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f98f4003c20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f990400a000 tx=0x7f9904008040).stop 2021-11-21T21:56:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99137fe700 1 -- 172.21.2.116:0/122724294 reap_dead start 2021-11-21T21:56:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- 172.21.2.116:0/122724294 shutdown_connections 2021-11-21T21:56:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- 172.21.2.116:0/122724294 >> 172.21.2.116:0/122724294 conn(0x7f991405bd20 msgr2=0x7f991405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- 172.21.2.116:0/122724294 shutdown_connections 2021-11-21T21:56:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 -- 172.21.2.116:0/122724294 wait complete. 2021-11-21T21:56:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.192+0000 7f99198de700 1 Processor -- start 2021-11-21T21:56:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 -- start start 2021-11-21T21:56:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f9914146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140aa5f0 0x7f99141559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9914155eb0 0x7f99141524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140aa3d0 con 0x7f98f4003800 2021-11-21T21:56:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9914152070 con 0x7f9914155eb0 2021-11-21T21:56:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9914139ed0 con 0x7f99140aa5f0 2021-11-21T21:56:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f9914146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f9914146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46780/0 (socket says 172.21.2.116:46780) 2021-11-21T21:56:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 -- 172.21.2.116:0/4050495173 learned_addr learned my addr 172.21.2.116:0/4050495173 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 -- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140aa5f0 msgr2=0x7f99141559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140aa5f0 0x7f99141559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 -- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9914155eb0 msgr2=0x7f99141524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99137fe700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9914155eb0 0x7f99141524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9914155eb0 0x7f99141524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 -- 172.21.2.116:0/4050495173 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991413a170 con 0x7f98f4003800 2021-11-21T21:56:09.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f9912ffd700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f9914146310 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f990400a000 tx=0x7f9904008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f990bfff700 1 -- 172.21.2.116:0/4050495173 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9904012d40 con 0x7f98f4003800 2021-11-21T21:56:09.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f990bfff700 1 -- 172.21.2.116:0/4050495173 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9904000e90 con 0x7f98f4003800 2021-11-21T21:56:09.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991413a410 con 0x7f98f4003800 2021-11-21T21:56:09.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f990bfff700 1 -- 172.21.2.116:0/4050495173 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9904012ea0 con 0x7f98f4003800 2021-11-21T21:56:09.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9914146ae0 con 0x7f98f4003800 2021-11-21T21:56:09.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99141564d0 con 0x7f98f4003800 2021-11-21T21:56:09.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f990bfff700 1 -- 172.21.2.116:0/4050495173 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f990401d600 con 0x7f98f4003800 2021-11-21T21:56:09.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f990bfff700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98f8053520 0x7f98f80559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f990bfff700 1 -- 172.21.2.116:0/4050495173 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f990406d380 con 0x7f98f4003800 2021-11-21T21:56:09.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99127fc700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98f8053520 0x7f98f80559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.196+0000 7f99127fc700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98f8053520 0x7f98f80559e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f98fc008000 tx=0x7f98fc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.200+0000 7f990bfff700 1 -- 172.21.2.116:0/4050495173 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9904044430 con 0x7f98f4003800 2021-11-21T21:56:09.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.280+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99140a8880 con 0x7f98f4003800 2021-11-21T21:56:09.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f990bfff700 1 -- 172.21.2.116:0/4050495173 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16167 (secure 0 0 0) 0x7f990400b120 con 0x7f98f4003800 2021-11-21T21:56:09.283 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:09.284 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:56:09.235532+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:56:06.102882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-11-21T21:56:06.914261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.116:6835/2140793945":"2021-11-22T21:56:08.223482+0000","172.21.2.116:6834/2140793945":"2021-11-22T21:56:08.223482+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:56:09.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98f8053520 msgr2=0x7f98f80559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98f8053520 0x7f98f80559e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f98fc008000 tx=0x7f98fc009040).stop 2021-11-21T21:56:09.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 msgr2=0x7f9914146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f9914146310 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f990400a000 tx=0x7f9904008040).stop 2021-11-21T21:56:09.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 shutdown_connections 2021-11-21T21:56:09.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9914155eb0 0x7f99141524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99140aa5f0 0x7f99141559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98f8053520 0x7f98f80559e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 --2- 172.21.2.116:0/4050495173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98f4003800 0x7f9914146310 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 >> 172.21.2.116:0/4050495173 conn(0x7f991405bd20 msgr2=0x7f99140b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 shutdown_connections 2021-11-21T21:56:09.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.284+0000 7f99198de700 1 -- 172.21.2.116:0/4050495173 wait complete. 2021-11-21T21:56:09.291 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:56:09.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.344+0000 7fc71ad75700 1 Processor -- start 2021-11-21T21:56:09.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.344+0000 7fc71ad75700 1 -- start start 2021-11-21T21:56:09.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc7140affc0 0x7fc71405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7140af7e0 0x7fc71413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc714143f70 con 0x7fc7140ad780 2021-11-21T21:56:09.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140adb60 con 0x7fc7140affc0 2021-11-21T21:56:09.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc714143160 con 0x7fc7140af7e0 2021-11-21T21:56:09.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc718b11700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc7140ad780 0x7fc714143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58744/0 (socket says 172.21.2.116:58744) 2021-11-21T21:56:09.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc718b11700 1 -- 172.21.2.116:0/3742209985 learned_addr learned my addr 172.21.2.116:0/3742209985 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:09.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227065842 0 0) 0x7fc714143f70 con 0x7fc7140ad780 2021-11-21T21:56:09.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f4003260 con 0x7fc7140ad780 2021-11-21T21:56:09.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792022346 0 0) 0x7fc6f4003260 con 0x7fc7140ad780 2021-11-21T21:56:09.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7140af7e0 msgr2=0x7fc71413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7140af7e0 0x7fc71413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 >> v2:172.21.2.117:3300/0 conn(0x7fc7140affc0 msgr2=0x7fc71405f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 --2- 172.21.2.116:0/3742209985 >> v2:172.21.2.117:3300/0 conn(0x7fc7140affc0 0x7fc71405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140b0ce0 con 0x7fc7140ad780 2021-11-21T21:56:09.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fc704001660 con 0x7fc7140ad780 2021-11-21T21:56:09.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 >> v1:172.21.2.116:6789/0 conn(0x7fc7140ad780 legacy=0x7fc714143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6f40037d0 0x7fc6f4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7140af7e0 0x7fc6f4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc6f4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140adb60 con 0x7fc6f40049f0 2021-11-21T21:56:09.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714143160 con 0x7fc6f40037d0 2021-11-21T21:56:09.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f400fe90 con 0x7fc7140af7e0 2021-11-21T21:56:09.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc6f4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 -- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7140af7e0 msgr2=0x7fc6f4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7140af7e0 0x7fc6f4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 -- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6f40037d0 msgr2=0x7fc6f4003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6f40037d0 0x7fc6f4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 -- 172.21.2.116:0/3742209985 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f4003260 con 0x7fc6f40049f0 2021-11-21T21:56:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc6f4004e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fc70c0032f0 tx=0x7fc70c006f90).ready entity=mon.0 client_cookie=c010152dbf7fbd26 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70c00b460 con 0x7fc6f40049f0 2021-11-21T21:56:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c00b5c0 con 0x7fc6f40049f0 2021-11-21T21:56:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc7137fe700 1 -- 172.21.2.116:0/3742209985 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70c00b730 con 0x7fc6f40049f0 2021-11-21T21:56:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- 172.21.2.116:0/3742209985 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 msgr2=0x7fc6f4004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 --2- 172.21.2.116:0/3742209985 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc6f4004e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fc70c0032f0 tx=0x7fc70c006f90).stop 2021-11-21T21:56:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc719312700 1 -- 172.21.2.116:0/3742209985 reap_dead start 2021-11-21T21:56:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- 172.21.2.116:0/3742209985 shutdown_connections 2021-11-21T21:56:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- 172.21.2.116:0/3742209985 >> 172.21.2.116:0/3742209985 conn(0x7fc7140b6000 msgr2=0x7fc7140b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- 172.21.2.116:0/3742209985 shutdown_connections 2021-11-21T21:56:09.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- 172.21.2.116:0/3742209985 wait complete. 2021-11-21T21:56:09.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 Processor -- start 2021-11-21T21:56:09.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- start start 2021-11-21T21:56:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc7141428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7140ab340 0x7fc7140648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc714064dc0 0x7fc71413ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71413cff0 con 0x7fc6f40049f0 2021-11-21T21:56:09.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71413c810 con 0x7fc7140ab340 2021-11-21T21:56:09.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc71ad75700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71413cab0 con 0x7fc714064dc0 2021-11-21T21:56:09.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc718b11700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc7141428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc718b11700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc7141428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46800/0 (socket says 172.21.2.116:46800) 2021-11-21T21:56:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.348+0000 7fc718b11700 1 -- 172.21.2.116:0/2369352269 learned_addr learned my addr 172.21.2.116:0/2369352269 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc718b11700 1 -- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc714064dc0 msgr2=0x7fc71413ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc718b11700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc714064dc0 0x7fc71413ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc718b11700 1 -- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7140ab340 msgr2=0x7fc7140648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc718b11700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7140ab340 0x7fc7140648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc718b11700 1 -- 172.21.2.116:0/2369352269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71413cd50 con 0x7fc6f40049f0 2021-11-21T21:56:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc718b11700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc7141428f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc704001370 tx=0x7fc704001a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc711ffb700 1 -- 172.21.2.116:0/2369352269 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc704007170 con 0x7fc6f40049f0 2021-11-21T21:56:09.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc711ffb700 1 -- 172.21.2.116:0/2369352269 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7040072d0 con 0x7fc6f40049f0 2021-11-21T21:56:09.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc711ffb700 1 -- 172.21.2.116:0/2369352269 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc704004920 con 0x7fc6f40049f0 2021-11-21T21:56:09.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7140accd0 con 0x7fc6f40049f0 2021-11-21T21:56:09.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71413b4b0 con 0x7fc6f40049f0 2021-11-21T21:56:09.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7140b0a40 con 0x7fc6f40049f0 2021-11-21T21:56:09.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc711ffb700 1 -- 172.21.2.116:0/2369352269 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc704004a80 con 0x7fc6f40049f0 2021-11-21T21:56:09.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc711ffb700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6f8053540 0x7fc6f8055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc711ffb700 1 -- 172.21.2.116:0/2369352269 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fc70406b670 con 0x7fc6f40049f0 2021-11-21T21:56:09.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc713fff700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6f8053540 0x7fc6f8055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc713fff700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6f8053540 0x7fc6f8055a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc6fc002f70 tx=0x7fc6fc006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.352+0000 7fc711ffb700 1 -- 172.21.2.116:0/2369352269 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc704042780 con 0x7fc6f40049f0 2021-11-21T21:56:09.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.464+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc7140638e0 con 0x7fc6f40049f0 2021-11-21T21:56:09.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.468+0000 7fc711ffb700 1 -- 172.21.2.116:0/2369352269 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc704045da0 con 0x7fc6f40049f0 2021-11-21T21:56:09.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6f8053540 msgr2=0x7fc6f8055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6f8053540 0x7fc6f8055a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc6fc002f70 tx=0x7fc6fc006a90).stop 2021-11-21T21:56:09.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 msgr2=0x7fc7141428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc7141428f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc704001370 tx=0x7fc704001a60).stop 2021-11-21T21:56:09.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 shutdown_connections 2021-11-21T21:56:09.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc714064dc0 0x7fc71413ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7140ab340 0x7fc7140648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6f8053540 0x7fc6f8055a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 --2- 172.21.2.116:0/2369352269 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6f40049f0 0x7fc7141428f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 >> 172.21.2.116:0/2369352269 conn(0x7fc7140b6000 msgr2=0x7fc7140b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 shutdown_connections 2021-11-21T21:56:09.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.472+0000 7fc71ad75700 1 -- 172.21.2.116:0/2369352269 wait complete. 2021-11-21T21:56:09.474 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T21:56:09.478 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:56:09.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 Processor -- start 2021-11-21T21:56:09.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- start start 2021-11-21T21:56:09.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe5400ad780 0x7fe54005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe5400af7e0 0x7fe54013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe540143f70 con 0x7fe5400affc0 2021-11-21T21:56:09.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5400adb60 con 0x7fe5400af7e0 2021-11-21T21:56:09.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe540143160 con 0x7fe5400ad780 2021-11-21T21:56:09.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe5400af7e0 0x7fe54013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe5400af7e0 0x7fe54013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46808/0 (socket says 172.21.2.116:46808) 2021-11-21T21:56:09.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 -- 172.21.2.116:0/1569176867 learned_addr learned my addr 172.21.2.116:0/1569176867 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:09.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 -- 172.21.2.116:0/1569176867 >> v2:172.21.2.117:3300/0 conn(0x7fe5400ad780 msgr2=0x7fe54005f730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:09.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 --2- 172.21.2.116:0/1569176867 >> v2:172.21.2.117:3300/0 conn(0x7fe5400ad780 0x7fe54005f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 -- 172.21.2.116:0/1569176867 >> v1:172.21.2.117:6789/0 conn(0x7fe5400affc0 legacy=0x7fe540143860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:09.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 -- 172.21.2.116:0/1569176867 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400b0ce0 con 0x7fe5400af7e0 2021-11-21T21:56:09.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe54570f700 1 --2- 172.21.2.116:0/1569176867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400af7e0 0x7fe54013fe70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fe540068820 tx=0x7fe53800b530).ready entity=mon.0 client_cookie=f0bf201f64b4baa server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe53f7fe700 1 -- 172.21.2.116:0/1569176867 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53800be70 con 0x7fe5400af7e0 2021-11-21T21:56:09.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe53f7fe700 1 -- 172.21.2.116:0/1569176867 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538010470 con 0x7fe5400af7e0 2021-11-21T21:56:09.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe53f7fe700 1 -- 172.21.2.116:0/1569176867 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5380107d0 con 0x7fe5400af7e0 2021-11-21T21:56:09.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- 172.21.2.116:0/1569176867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400af7e0 msgr2=0x7fe54013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 --2- 172.21.2.116:0/1569176867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400af7e0 0x7fe54013fe70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fe540068820 tx=0x7fe53800b530).stop 2021-11-21T21:56:09.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- 172.21.2.116:0/1569176867 shutdown_connections 2021-11-21T21:56:09.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 --2- 172.21.2.116:0/1569176867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400af7e0 0x7fe54013fe70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 --2- 172.21.2.116:0/1569176867 >> v2:172.21.2.117:3300/0 conn(0x7fe5400ad780 0x7fe54005f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- 172.21.2.116:0/1569176867 >> 172.21.2.116:0/1569176867 conn(0x7fe5400b6000 msgr2=0x7fe5400b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- 172.21.2.116:0/1569176867 shutdown_connections 2021-11-21T21:56:09.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.532+0000 7fe547172700 1 -- 172.21.2.116:0/1569176867 wait complete. 2021-11-21T21:56:09.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 Processor -- start 2021-11-21T21:56:09.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 -- start start 2021-11-21T21:56:09.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5400ad780 0x7fe54014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5400af7e0 0x7fe54013db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400affc0 0x7fe5400aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54014ee90 con 0x7fe5400affc0 2021-11-21T21:56:09.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe540151790 con 0x7fe5400ad780 2021-11-21T21:56:09.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54014dfc0 con 0x7fe5400af7e0 2021-11-21T21:56:09.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400affc0 0x7fe5400aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400affc0 0x7fe5400aaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46818/0 (socket says 172.21.2.116:46818) 2021-11-21T21:56:09.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 -- 172.21.2.116:0/1796258341 learned_addr learned my addr 172.21.2.116:0/1796258341 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:09.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 -- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5400af7e0 msgr2=0x7fe54013db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5400af7e0 0x7fe54013db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 -- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5400ad780 msgr2=0x7fe54014e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5400ad780 0x7fe54014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 -- 172.21.2.116:0/1796258341 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53800b7d0 con 0x7fe5400affc0 2021-11-21T21:56:09.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe54570f700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400affc0 0x7fe5400aaf30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fe540068820 tx=0x7fe53800b530).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe53dffb700 1 -- 172.21.2.116:0/1796258341 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53801d070 con 0x7fe5400affc0 2021-11-21T21:56:09.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe53dffb700 1 -- 172.21.2.116:0/1796258341 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538018950 con 0x7fe5400affc0 2021-11-21T21:56:09.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54014e260 con 0x7fe5400affc0 2021-11-21T21:56:09.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe53dffb700 1 -- 172.21.2.116:0/1796258341 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe538018cb0 con 0x7fe5400affc0 2021-11-21T21:56:09.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54013e050 con 0x7fe5400affc0 2021-11-21T21:56:09.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5400b0a40 con 0x7fe5400affc0 2021-11-21T21:56:09.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe53dffb700 1 -- 172.21.2.116:0/1796258341 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe53800f3d0 con 0x7fe5400affc0 2021-11-21T21:56:09.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe53dffb700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5240535d0 0x7fe524055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe53dffb700 1 -- 172.21.2.116:0/1796258341 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fe53806d460 con 0x7fe5400affc0 2021-11-21T21:56:09.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe544f0e700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5240535d0 0x7fe524055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.536+0000 7fe544f0e700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5240535d0 0x7fe524055a90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe530002fb0 tx=0x7fe5300069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.540+0000 7fe53dffb700 1 -- 172.21.2.116:0/1796258341 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5380444b0 con 0x7fe5400affc0 2021-11-21T21:56:09.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.648+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe5400573d0 con 0x7fe5400affc0 2021-11-21T21:56:09.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe53dffb700 1 -- 172.21.2.116:0/1796258341 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe53800b630 con 0x7fe5400affc0 2021-11-21T21:56:09.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5240535d0 msgr2=0x7fe524055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5240535d0 0x7fe524055a90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe530002fb0 tx=0x7fe5300069b0).stop 2021-11-21T21:56:09.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400affc0 msgr2=0x7fe5400aaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400affc0 0x7fe5400aaf30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fe540068820 tx=0x7fe53800b530).stop 2021-11-21T21:56:09.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 shutdown_connections 2021-11-21T21:56:09.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5240535d0 0x7fe524055a90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5400affc0 0x7fe5400aaf30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5400af7e0 0x7fe54013db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 --2- 172.21.2.116:0/1796258341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5400ad780 0x7fe54014e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:09.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 >> 172.21.2.116:0/1796258341 conn(0x7fe5400b6000 msgr2=0x7fe54013b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 shutdown_connections 2021-11-21T21:56:09.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.656+0000 7fe547172700 1 -- 172.21.2.116:0/1796258341 wait complete. 2021-11-21T21:56:09.663 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T21:56:09.663 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:09.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.716+0000 7f862c481700 1 Processor -- start 2021-11-21T21:56:09.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.716+0000 7f862c481700 1 -- start start 2021-11-21T21:56:09.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.716+0000 7f862c481700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f86240b24a0 0x7f86240b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.716+0000 7f862c481700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86240ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.716+0000 7f862c481700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86240b00d0 con 0x7f86240aeb70 2021-11-21T21:56:09.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.716+0000 7f862c481700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86240adba0 con 0x7f86240b24a0 2021-11-21T21:56:09.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.716+0000 7f862c481700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86240af7a0 con 0x7f86240af360 2021-11-21T21:56:09.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f8629a1c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86240ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862a21d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f86240b24a0 0x7f86240b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862aa1e700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f86240aeb70 0x7f8624069e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:57692/0 (socket says 172.21.2.116:57692) 2021-11-21T21:56:09.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862aa1e700 1 -- 172.21.2.116:0/163123445 learned_addr learned my addr 172.21.2.116:0/163123445 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:09.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f8629a1c700 1 -- 172.21.2.116:0/163123445 >> v2:172.21.2.117:3300/0 conn(0x7f86240b24a0 msgr2=0x7f86240b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:09.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f8629a1c700 1 --2- 172.21.2.116:0/163123445 >> v2:172.21.2.117:3300/0 conn(0x7f86240b24a0 0x7f86240b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f8629a1c700 1 -- 172.21.2.116:0/163123445 >> v1:172.21.2.117:6789/0 conn(0x7f86240aeb70 legacy=0x7f8624069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f8629a1c700 1 -- 172.21.2.116:0/163123445 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86240b2880 con 0x7f86240af360 2021-11-21T21:56:09.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862a21d700 1 --2- 172.21.2.116:0/163123445 >> v2:172.21.2.117:3300/0 conn(0x7f86240b24a0 0x7f86240b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:56:09.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f8629a1c700 1 --2- 172.21.2.116:0/163123445 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86240ae680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8614002f70 tx=0x7f8614006df0).ready entity=mon.2 client_cookie=84bcbafcf23fd7d5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862921b700 1 -- 172.21.2.116:0/163123445 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f861400d570 con 0x7f86240af360 2021-11-21T21:56:09.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862921b700 1 -- 172.21.2.116:0/163123445 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8614000e90 con 0x7f86240af360 2021-11-21T21:56:09.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862921b700 1 -- 172.21.2.116:0/163123445 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f861400d8c0 con 0x7f86240af360 2021-11-21T21:56:09.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 -- 172.21.2.116:0/163123445 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 msgr2=0x7f86240ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 --2- 172.21.2.116:0/163123445 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86240ae680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8614002f70 tx=0x7f8614006df0).stop 2021-11-21T21:56:09.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 -- 172.21.2.116:0/163123445 shutdown_connections 2021-11-21T21:56:09.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 --2- 172.21.2.116:0/163123445 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86240ae680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 --2- 172.21.2.116:0/163123445 >> v2:172.21.2.117:3300/0 conn(0x7f86240b24a0 0x7f86240b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 -- 172.21.2.116:0/163123445 >> 172.21.2.116:0/163123445 conn(0x7f862405fd00 msgr2=0x7f86240600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 -- 172.21.2.116:0/163123445 shutdown_connections 2021-11-21T21:56:09.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 -- 172.21.2.116:0/163123445 wait complete. 2021-11-21T21:56:09.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 Processor -- start 2021-11-21T21:56:09.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.720+0000 7f862c481700 1 -- start start 2021-11-21T21:56:09.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86240aeb70 0x7f8624141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86241598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86240b24a0 0x7f8624159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86241432a0 con 0x7f86240aeb70 2021-11-21T21:56:09.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862413e1e0 con 0x7f86240b24a0 2021-11-21T21:56:09.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862413e480 con 0x7f86240af360 2021-11-21T21:56:09.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86240aeb70 0x7f8624141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86240aeb70 0x7f8624141750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46828/0 (socket says 172.21.2.116:46828) 2021-11-21T21:56:09.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 -- 172.21.2.116:0/3137867595 learned_addr learned my addr 172.21.2.116:0/3137867595 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:09.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f8629a1c700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86241598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862aa1e700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86240b24a0 0x7f8624159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 -- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 msgr2=0x7f86241598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86241598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 -- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86240b24a0 msgr2=0x7f8624159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86240b24a0 0x7f8624159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 -- 172.21.2.116:0/3137867595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862000b180 con 0x7f86240aeb70 2021-11-21T21:56:09.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f8629a1c700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86241598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:56:09.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862a21d700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86240aeb70 0x7f8624141750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f862000c000 tx=0x7f8620006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f86137fe700 1 -- 172.21.2.116:0/3137867595 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f862000c860 con 0x7f86240aeb70 2021-11-21T21:56:09.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f86137fe700 1 -- 172.21.2.116:0/3137867595 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862000c9c0 con 0x7f86240aeb70 2021-11-21T21:56:09.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f86137fe700 1 -- 172.21.2.116:0/3137867595 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f862000cd20 con 0x7f86240aeb70 2021-11-21T21:56:09.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861400b180 con 0x7f86240aeb70 2021-11-21T21:56:09.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8624064930 con 0x7f86240aeb70 2021-11-21T21:56:09.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86240573d0 con 0x7f86240aeb70 2021-11-21T21:56:09.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f86137fe700 1 -- 172.21.2.116:0/3137867595 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f862001c460 con 0x7f86240aeb70 2021-11-21T21:56:09.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f86137fe700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8604053640 0x7f8604055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:09.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f86137fe700 1 -- 172.21.2.116:0/3137867595 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f862006bf60 con 0x7f86240aeb70 2021-11-21T21:56:09.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f8629a1c700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8604053640 0x7f8604055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:09.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.724+0000 7f8629a1c700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8604053640 0x7f8604055b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8614018000 tx=0x7f8614014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:09.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.728+0000 7f86137fe700 1 -- 172.21.2.116:0/3137867595 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8620043010 con 0x7f86240aeb70 2021-11-21T21:56:09.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f862405f2d0 con 0x7f86240aeb70 2021-11-21T21:56:09.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f86137fe700 1 -- 172.21.2.116:0/3137867595 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2494 (secure 0 0 0) 0x7f8620046630 con 0x7f86240aeb70 2021-11-21T21:56:09.837 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:09.837 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:08.956930+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:creating","state_seq":2,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:09.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8604053640 msgr2=0x7f8604055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8604053640 0x7f8604055b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8614018000 tx=0x7f8614014000).stop 2021-11-21T21:56:09.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86240aeb70 msgr2=0x7f8624141750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:09.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86240aeb70 0x7f8624141750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f862000c000 tx=0x7f8620006df0).stop 2021-11-21T21:56:09.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 shutdown_connections 2021-11-21T21:56:09.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f86240b24a0 0x7f8624159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f86240af360 0x7f86241598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8604053640 0x7f8604055b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 --2- 172.21.2.116:0/3137867595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f86240aeb70 0x7f8624141750 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:09.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 >> 172.21.2.116:0/3137867595 conn(0x7f862405fd00 msgr2=0x7f862413c730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:09.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 shutdown_connections 2021-11-21T21:56:09.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:09.836+0000 7f862c481700 1 -- 172.21.2.116:0/3137867595 wait complete. 2021-11-21T21:56:09.841 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 16 2021-11-21T21:56:10.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:56:07.942419+0000', 'modified': '2021-11-21T21:56:08.956930+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5247': {'gid': 5247, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 2, 'addr': '172.21.2.116:6835/2465449969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2465449969}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2465449969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:56:10.147 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5247:up:creating 2021-11-21T21:56:11.148 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:11.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 Processor -- start 2021-11-21T21:56:11.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- start start 2021-11-21T21:56:11.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f057c0b5630 0x7f057c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c097ec0 0x7f057c146e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f057c147300 0x7f057c099650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c144b40 con 0x7f057c0b5630 2021-11-21T21:56:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c14a560 con 0x7f057c147300 2021-11-21T21:56:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c14a800 con 0x7f057c097ec0 2021-11-21T21:56:11.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f057c0b5630 0x7f057c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f057c0b5630 0x7f057c14af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46836/0 (socket says 172.21.2.116:46836) 2021-11-21T21:56:11.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 -- 172.21.2.116:0/2857860321 learned_addr learned my addr 172.21.2.116:0/2857860321 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:11.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 -- 172.21.2.116:0/2857860321 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c097ec0 msgr2=0x7f057c146e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:11.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 --2- 172.21.2.116:0/2857860321 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c097ec0 0x7f057c146e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 -- 172.21.2.116:0/2857860321 >> v2:172.21.2.117:3300/0 conn(0x7f057c147300 msgr2=0x7f057c099650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 --2- 172.21.2.116:0/2857860321 >> v2:172.21.2.117:3300/0 conn(0x7f057c147300 0x7f057c099650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 -- 172.21.2.116:0/2857860321 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c14aaa0 con 0x7f057c0b5630 2021-11-21T21:56:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0580b19700 1 --2- 172.21.2.116:0/2857860321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c0b5630 0x7f057c14af40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f056c002f70 tx=0x7f056c006eb0).ready entity=mon.0 client_cookie=7686cbfe3d5bf4e4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f057b7fe700 1 -- 172.21.2.116:0/2857860321 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f056c00d570 con 0x7f057c0b5630 2021-11-21T21:56:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f057b7fe700 1 -- 172.21.2.116:0/2857860321 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c000e90 con 0x7f057c0b5630 2021-11-21T21:56:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f057b7fe700 1 -- 172.21.2.116:0/2857860321 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f056c00d900 con 0x7f057c0b5630 2021-11-21T21:56:11.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- 172.21.2.116:0/2857860321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c0b5630 msgr2=0x7f057c14af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2857860321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c0b5630 0x7f057c14af40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f056c002f70 tx=0x7f056c006eb0).stop 2021-11-21T21:56:11.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- 172.21.2.116:0/2857860321 shutdown_connections 2021-11-21T21:56:11.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2857860321 >> v2:172.21.2.117:3300/0 conn(0x7f057c147300 0x7f057c099650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2857860321 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c097ec0 0x7f057c146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2857860321 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c0b5630 0x7f057c14af40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- 172.21.2.116:0/2857860321 >> 172.21.2.116:0/2857860321 conn(0x7f057c05c7f0 msgr2=0x7f057c0b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:11.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- 172.21.2.116:0/2857860321 shutdown_connections 2021-11-21T21:56:11.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.204+0000 7f0581b1b700 1 -- 172.21.2.116:0/2857860321 wait complete. 2021-11-21T21:56:11.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 Processor -- start 2021-11-21T21:56:11.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 -- start start 2021-11-21T21:56:11.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c097ec0 0x7f057c146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f057c0b5630 0x7f057c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c147300 0x7f057c159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c062fb0 con 0x7f057c097ec0 2021-11-21T21:56:11.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0655a0 con 0x7f057c0b5630 2021-11-21T21:56:11.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c065840 con 0x7f057c147300 2021-11-21T21:56:11.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c097ec0 0x7f057c146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c097ec0 0x7f057c146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46842/0 (socket says 172.21.2.116:46842) 2021-11-21T21:56:11.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 -- 172.21.2.116:0/2895600019 learned_addr learned my addr 172.21.2.116:0/2895600019 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:11.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 -- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c147300 msgr2=0x7f057c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c147300 0x7f057c159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 -- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f057c0b5630 msgr2=0x7f057c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f057c0b5630 0x7f057c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 -- 172.21.2.116:0/2895600019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c00b1e0 con 0x7f057c097ec0 2021-11-21T21:56:11.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0580b19700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c097ec0 0x7f057c146af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f056c002f70 tx=0x7f056c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0579ffb700 1 -- 172.21.2.116:0/2895600019 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f056c0119d0 con 0x7f057c097ec0 2021-11-21T21:56:11.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0579ffb700 1 -- 172.21.2.116:0/2895600019 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c011b30 con 0x7f057c097ec0 2021-11-21T21:56:11.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c065ae0 con 0x7f057c097ec0 2021-11-21T21:56:11.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0579ffb700 1 -- 172.21.2.116:0/2895600019 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f056c011e90 con 0x7f057c097ec0 2021-11-21T21:56:11.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c066020 con 0x7f057c097ec0 2021-11-21T21:56:11.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0579ffb700 1 -- 172.21.2.116:0/2895600019 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f056c025030 con 0x7f057c097ec0 2021-11-21T21:56:11.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c0573d0 con 0x7f057c097ec0 2021-11-21T21:56:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0579ffb700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05600536f0 0x7f0560055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f0579ffb700 1 -- 172.21.2.116:0/2895600019 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f056c06cf20 con 0x7f057c097ec0 2021-11-21T21:56:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.208+0000 7f057bfff700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05600536f0 0x7f0560055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.212+0000 7f057bfff700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05600536f0 0x7f0560055bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f0564002fb0 tx=0x7f05640069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.212+0000 7f0579ffb700 1 -- 172.21.2.116:0/2895600019 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f056c043f20 con 0x7f057c097ec0 2021-11-21T21:56:11.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f057c159710 con 0x7f057c097ec0 2021-11-21T21:56:11.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0579ffb700 1 -- 172.21.2.116:0/2895600019 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f056c023020 con 0x7f057c097ec0 2021-11-21T21:56:11.320 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:11.320 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:09.961260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:11.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05600536f0 msgr2=0x7f0560055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05600536f0 0x7f0560055bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f0564002fb0 tx=0x7f05640069b0).stop 2021-11-21T21:56:11.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c097ec0 msgr2=0x7f057c146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c097ec0 0x7f057c146af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f056c002f70 tx=0x7f056c015000).stop 2021-11-21T21:56:11.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 shutdown_connections 2021-11-21T21:56:11.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f057c147300 0x7f057c159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f057c0b5630 0x7f057c1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05600536f0 0x7f0560055bb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 --2- 172.21.2.116:0/2895600019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f057c097ec0 0x7f057c146af0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 >> 172.21.2.116:0/2895600019 conn(0x7f057c05c7f0 msgr2=0x7f057c143f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:11.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 shutdown_connections 2021-11-21T21:56:11.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.320+0000 7f0581b1b700 1 -- 172.21.2.116:0/2895600019 wait complete. 2021-11-21T21:56:11.326 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 17 2021-11-21T21:56:11.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:56:07.942419+0000', 'modified': '2021-11-21T21:56:09.961260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5247': {'gid': 5247, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2465449969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2465449969}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2465449969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:56:11.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:56:11.644 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:56:11.712 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5247,"want_state":"up:active","state":"up:active","rank_uptime":2.74966782,"mdsmap_epoch":17,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":7.4879447429999999} 2021-11-21T21:56:11.712 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5247, "mdsmap_epoch": 17, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 2.74966782, "state": "up:active", "uptime": 7.487944743, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:56:11.713 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:56:11.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aaa0f5700 1 Processor -- start 2021-11-21T21:56:11.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aaa0f5700 1 -- start start 2021-11-21T21:56:11.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aaa0f5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8aa4062410 0x7f8aa4144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aaa0f5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa40a6e80 con 0x7f8aa40a6aa0 2021-11-21T21:56:11.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aaa0f5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa414a7b0 con 0x7f8aa40a86b0 2021-11-21T21:56:11.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aaa0f5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa40a8bc0 con 0x7f8aa4062410 2021-11-21T21:56:11.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aa37fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8aa40a6aa0 0x7f8aa41470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58806/0 (socket says 172.21.2.116:58806) 2021-11-21T21:56:11.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aa37fe700 1 -- 172.21.2.116:0/3910757825 learned_addr learned my addr 172.21.2.116:0/3910757825 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:11.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.764+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52530035 0 0) 0x7f8aa40a6e80 con 0x7f8aa40a6aa0 2021-11-21T21:56:11.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a84003260 con 0x7f8aa40a6aa0 2021-11-21T21:56:11.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2127290996 0 0) 0x7f8a84003260 con 0x7f8aa40a6aa0 2021-11-21T21:56:11.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 >> v2:172.21.2.117:3300/0 conn(0x7f8aa4062410 msgr2=0x7f8aa4144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:11.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 --2- 172.21.2.116:0/3910757825 >> v2:172.21.2.117:3300/0 conn(0x7f8aa4062410 0x7f8aa4144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 >> v1:172.21.2.117:6789/0 conn(0x7f8aa40a86b0 legacy=0x7f8aa41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:11.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa414aa10 con 0x7f8aa40a6aa0 2021-11-21T21:56:11.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8a94001680 con 0x7f8aa40a6aa0 2021-11-21T21:56:11.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 >> v1:172.21.2.116:6789/0 conn(0x7f8aa40a6aa0 legacy=0x7f8aa41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8a84003820 0x7f8a84003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8a84004500 0x7f8a84004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8a84005050 0x7f8a8400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa40a8bc0 con 0x7f8a84004500 2021-11-21T21:56:11.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8400bf40 con 0x7f8a84003820 2021-11-21T21:56:11.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8400c100 con 0x7f8a84005050 2021-11-21T21:56:11.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa2ffd700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8a84004500 0x7f8a84004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa2ffd700 1 -- 172.21.2.116:0/3910757825 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8a84005050 msgr2=0x7f8a8400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:11.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa2ffd700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8a84005050 0x7f8a8400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa2ffd700 1 -- 172.21.2.116:0/3910757825 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8a84003820 msgr2=0x7f8a84003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:11.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa2ffd700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8a84003820 0x7f8a84003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa2ffd700 1 -- 172.21.2.116:0/3910757825 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a84003260 con 0x7f8a84004500 2021-11-21T21:56:11.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 -- 172.21.2.116:0/3910757825 reap_dead start 2021-11-21T21:56:11.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa2ffd700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8a84004500 0x7f8a84004980 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8a8c000f70 tx=0x7f8a8c0079e0).ready entity=mon.0 client_cookie=f1b2be95553561c6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8c007cf0 con 0x7f8a84004500 2021-11-21T21:56:11.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c014410 con 0x7f8a84004500 2021-11-21T21:56:11.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa27fc700 1 -- 172.21.2.116:0/3910757825 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8c014770 con 0x7f8a84004500 2021-11-21T21:56:11.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/3910757825 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8a84004500 msgr2=0x7f8a84004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8a84004500 0x7f8a84004980 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8a8c000f70 tx=0x7f8a8c0079e0).stop 2021-11-21T21:56:11.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/3910757825 shutdown_connections 2021-11-21T21:56:11.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/3910757825 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8a84004500 0x7f8a84004980 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/3910757825 >> 172.21.2.116:0/3910757825 conn(0x7f8aa405bd20 msgr2=0x7f8aa405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:11.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/3910757825 shutdown_connections 2021-11-21T21:56:11.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/3910757825 wait complete. 2021-11-21T21:56:11.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 Processor -- start 2021-11-21T21:56:11.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- start start 2021-11-21T21:56:11.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8a84004500 0x7f8aa4064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8aa4064be0 0x7f8aa4159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8aa40a9b90 0x7f8aa4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4066ab0 con 0x7f8aa40a9b90 2021-11-21T21:56:11.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4065080 con 0x7f8a84004500 2021-11-21T21:56:11.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aaa0f5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa40a9450 con 0x7f8aa4064be0 2021-11-21T21:56:11.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8aa40a9b90 0x7f8aa4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8aa40a9b90 0x7f8aa4159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46866/0 (socket says 172.21.2.116:46866) 2021-11-21T21:56:11.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 -- 172.21.2.116:0/2783540853 learned_addr learned my addr 172.21.2.116:0/2783540853 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:11.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 -- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8aa4064be0 msgr2=0x7f8aa4159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8aa4064be0 0x7f8aa4159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 -- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8a84004500 msgr2=0x7f8aa4064400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8a84004500 0x7f8aa4064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 -- 172.21.2.116:0/2783540853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40a96f0 con 0x7f8aa40a9b90 2021-11-21T21:56:11.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa3fff700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8aa40a9b90 0x7f8aa4159d50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f8a94001110 tx=0x7f8a9c00a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.768+0000 7f8aa0ff9700 1 -- 172.21.2.116:0/2783540853 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a9c00ce40 con 0x7f8aa40a9b90 2021-11-21T21:56:11.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa0ff9700 1 -- 172.21.2.116:0/2783540853 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c005c30 con 0x7f8aa40a9b90 2021-11-21T21:56:11.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa0ff9700 1 -- 172.21.2.116:0/2783540853 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a9c007070 con 0x7f8aa40a9b90 2021-11-21T21:56:11.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa414d440 con 0x7f8aa40a9b90 2021-11-21T21:56:11.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa4065b40 con 0x7f8aa40a9b90 2021-11-21T21:56:11.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa40a9880 con 0x7f8aa40a9b90 2021-11-21T21:56:11.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa0ff9700 1 -- 172.21.2.116:0/2783540853 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8a9c01d070 con 0x7f8aa40a9b90 2021-11-21T21:56:11.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa0ff9700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8a88053580 0x7f8a88055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa0ff9700 1 -- 172.21.2.116:0/2783540853 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8a9c06b020 con 0x7f8aa40a9b90 2021-11-21T21:56:11.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa37fe700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8a88053580 0x7f8a88055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa37fe700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8a88053580 0x7f8a88055a40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8aa4061b50 tx=0x7f8aa40a8520).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.772+0000 7f8aa0ff9700 1 -- 172.21.2.116:0/2783540853 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a9c042130 con 0x7f8aa40a9b90 2021-11-21T21:56:11.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.856+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8aa40573d0 con 0x7f8aa40a9b90 2021-11-21T21:56:11.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.856+0000 7f8aa0ff9700 1 -- 172.21.2.116:0/2783540853 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16167 (secure 0 0 0) 0x7f8a9c045750 con 0x7f8aa40a9b90 2021-11-21T21:56:11.860 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:11.861 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:56:09.235532+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:56:06.102882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-11-21T21:56:06.914261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.116:6835/2140793945":"2021-11-22T21:56:08.223482+0000","172.21.2.116:6834/2140793945":"2021-11-22T21:56:08.223482+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:56:11.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8a88053580 msgr2=0x7f8a88055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8a88053580 0x7f8a88055a40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8aa4061b50 tx=0x7f8aa40a8520).stop 2021-11-21T21:56:11.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8aa40a9b90 msgr2=0x7f8aa4159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8aa40a9b90 0x7f8aa4159d50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f8a94001110 tx=0x7f8a9c00a060).stop 2021-11-21T21:56:11.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 shutdown_connections 2021-11-21T21:56:11.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8a88053580 0x7f8a88055a40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8aa40a9b90 0x7f8aa4159d50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8aa4064be0 0x7f8aa4159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 --2- 172.21.2.116:0/2783540853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8a84004500 0x7f8aa4064400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 >> 172.21.2.116:0/2783540853 conn(0x7f8aa405bd20 msgr2=0x7f8aa4149020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:11.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 shutdown_connections 2021-11-21T21:56:11.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.860+0000 7f8aaa0f5700 1 -- 172.21.2.116:0/2783540853 wait complete. 2021-11-21T21:56:11.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:56:06.102882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:56:06.914261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:56:11.874 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:56:11.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.968+0000 7f33381dc700 1 Processor -- start 2021-11-21T21:56:11.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.968+0000 7f33381dc700 1 -- start start 2021-11-21T21:56:11.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f33300abce0 0x7f33301478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300ac600 0x7f33300aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3330147e80 con 0x7f3330144810 2021-11-21T21:56:11.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33300b1490 con 0x7f33300abce0 2021-11-21T21:56:11.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33300b1fd0 con 0x7f33300ac600 2021-11-21T21:56:11.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f33300abce0 0x7f33301478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f33300abce0 0x7f33301478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:48038/0 (socket says 172.21.2.116:48038) 2021-11-21T21:56:11.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 -- 172.21.2.116:0/3645204740 learned_addr learned my addr 172.21.2.116:0/3645204740 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:11.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3336779700 1 --2- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300ac600 0x7f33300aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 -- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300ac600 msgr2=0x7f33300aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:11.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 --2- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300ac600 0x7f33300aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 -- 172.21.2.116:0/3645204740 >> v1:172.21.2.117:6789/0 conn(0x7f3330144810 legacy=0x7f33300b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 -- 172.21.2.116:0/3645204740 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33300af5a0 con 0x7f33300abce0 2021-11-21T21:56:11.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3335777700 1 --2- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300abce0 0x7f33301478c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f33280032f0 tx=0x7f3328009290).ready entity=mon.1 client_cookie=d038710b122cb440 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3334f76700 1 -- 172.21.2.116:0/3645204740 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f332800b8a0 con 0x7f33300abce0 2021-11-21T21:56:11.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3334f76700 1 -- 172.21.2.116:0/3645204740 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800ba00 con 0x7f33300abce0 2021-11-21T21:56:11.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f3334f76700 1 -- 172.21.2.116:0/3645204740 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f332800bb70 con 0x7f33300abce0 2021-11-21T21:56:11.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300abce0 msgr2=0x7f33301478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 --2- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300abce0 0x7f33301478c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f33280032f0 tx=0x7f3328009290).stop 2021-11-21T21:56:11.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- 172.21.2.116:0/3645204740 shutdown_connections 2021-11-21T21:56:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 --2- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300ac600 0x7f33300aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 --2- 172.21.2.116:0/3645204740 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300abce0 0x7f33301478c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- 172.21.2.116:0/3645204740 >> 172.21.2.116:0/3645204740 conn(0x7f333005bd20 msgr2=0x7f333005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:11.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- 172.21.2.116:0/3645204740 shutdown_connections 2021-11-21T21:56:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.972+0000 7f33381dc700 1 -- 172.21.2.116:0/3645204740 wait complete. 2021-11-21T21:56:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 Processor -- start 2021-11-21T21:56:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 -- start start 2021-11-21T21:56:11.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300abce0 0x7f333013f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300ac600 0x7f3330145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3330144810 0x7f33301459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33301468f0 con 0x7f3330144810 2021-11-21T21:56:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3330146b20 con 0x7f33300ac600 2021-11-21T21:56:11.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3330146dc0 con 0x7f33300abce0 2021-11-21T21:56:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3330144810 0x7f33301459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3330144810 0x7f33301459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46876/0 (socket says 172.21.2.116:46876) 2021-11-21T21:56:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 -- 172.21.2.116:0/3223223478 learned_addr learned my addr 172.21.2.116:0/3223223478 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 -- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300abce0 msgr2=0x7f333013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300abce0 0x7f333013f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 -- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300ac600 msgr2=0x7f3330145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300ac600 0x7f3330145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 -- 172.21.2.116:0/3223223478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328009530 con 0x7f3330144810 2021-11-21T21:56:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3336779700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3330144810 0x7f33301459f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f332c00c000 tx=0x7f332c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f331effd700 1 -- 172.21.2.116:0/3223223478 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f332c010780 con 0x7f3330144810 2021-11-21T21:56:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f331effd700 1 -- 172.21.2.116:0/3223223478 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c0108e0 con 0x7f3330144810 2021-11-21T21:56:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3330144dc0 con 0x7f3330144810 2021-11-21T21:56:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f331effd700 1 -- 172.21.2.116:0/3223223478 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f332c010c40 con 0x7f3330144810 2021-11-21T21:56:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33300b3890 con 0x7f3330144810 2021-11-21T21:56:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3330157930 con 0x7f3330144810 2021-11-21T21:56:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f331effd700 1 -- 172.21.2.116:0/3223223478 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f332c01c460 con 0x7f3330144810 2021-11-21T21:56:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f331effd700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3310053610 0x7f3310055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f331effd700 1 -- 172.21.2.116:0/3223223478 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f332c06bd10 con 0x7f3330144810 2021-11-21T21:56:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.976+0000 7f3335f78700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3310053610 0x7f3310055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.980+0000 7f3335f78700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3310053610 0x7f3310055ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3330062500 tx=0x7f3320007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:11.980+0000 7f331effd700 1 -- 172.21.2.116:0/3223223478 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3330157930 con 0x7f3330144810 2021-11-21T21:56:12.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3330157930 con 0x7f3330144810 2021-11-21T21:56:12.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f331effd700 1 -- 172.21.2.116:0/3223223478 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3330157930 con 0x7f3330144810 2021-11-21T21:56:12.064 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:56:12.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3310053610 msgr2=0x7f3310055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3310053610 0x7f3310055ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3330062500 tx=0x7f3320007810).stop 2021-11-21T21:56:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3330144810 msgr2=0x7f33301459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3330144810 0x7f33301459f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f332c00c000 tx=0x7f332c006df0).stop 2021-11-21T21:56:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 shutdown_connections 2021-11-21T21:56:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3310053610 0x7f3310055ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3330144810 0x7f33301459f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33300ac600 0x7f3330145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 --2- 172.21.2.116:0/3223223478 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33300abce0 0x7f333013f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 >> 172.21.2.116:0/3223223478 conn(0x7f333005bd20 msgr2=0x7f33301409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 shutdown_connections 2021-11-21T21:56:12.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.064+0000 7f33381dc700 1 -- 172.21.2.116:0/3223223478 wait complete. 2021-11-21T21:56:12.072 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:56:12.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.124+0000 7f45a8244700 1 Processor -- start 2021-11-21T21:56:12.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.124+0000 7f45a8244700 1 -- start start 2021-11-21T21:56:12.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.124+0000 7f45a8244700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f45a0144810 0x7f45a01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.124+0000 7f45a8244700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.124+0000 7f45a8244700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a0147e80 con 0x7f45a00abce0 2021-11-21T21:56:12.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.124+0000 7f45a8244700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a00b1490 con 0x7f45a0144810 2021-11-21T21:56:12.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.124+0000 7f45a8244700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b1fd0 con 0x7f45a00ac600 2021-11-21T21:56:12.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f45a0144810 0x7f45a01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a67e1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a00aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f45a0144810 0x7f45a01478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:48052/0 (socket says 172.21.2.116:48052) 2021-11-21T21:56:12.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 -- 172.21.2.116:0/3856868754 learned_addr learned my addr 172.21.2.116:0/3856868754 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a67e1700 1 -- 172.21.2.116:0/3856868754 >> v2:172.21.2.117:3300/0 conn(0x7f45a0144810 msgr2=0x7f45a01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:12.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a67e1700 1 --2- 172.21.2.116:0/3856868754 >> v2:172.21.2.117:3300/0 conn(0x7f45a0144810 0x7f45a01478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a67e1700 1 -- 172.21.2.116:0/3856868754 >> v1:172.21.2.117:6789/0 conn(0x7f45a00abce0 legacy=0x7f45a00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a67e1700 1 -- 172.21.2.116:0/3856868754 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a00af5a0 con 0x7f45a00ac600 2021-11-21T21:56:12.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a67e1700 1 --2- 172.21.2.116:0/3856868754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a00aca90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f45980032f0 tx=0x7f45980091d0).ready entity=mon.2 client_cookie=5feb8df043ba8202 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a4fde700 1 -- 172.21.2.116:0/3856868754 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459800b6c0 con 0x7f45a00ac600 2021-11-21T21:56:12.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a4fde700 1 -- 172.21.2.116:0/3856868754 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459800b820 con 0x7f45a00ac600 2021-11-21T21:56:12.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a4fde700 1 -- 172.21.2.116:0/3856868754 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459800b990 con 0x7f45a00ac600 2021-11-21T21:56:12.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- 172.21.2.116:0/3856868754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 msgr2=0x7f45a00aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 --2- 172.21.2.116:0/3856868754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a00aca90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f45980032f0 tx=0x7f45980091d0).stop 2021-11-21T21:56:12.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- 172.21.2.116:0/3856868754 shutdown_connections 2021-11-21T21:56:12.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 --2- 172.21.2.116:0/3856868754 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a00aca90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 --2- 172.21.2.116:0/3856868754 >> v2:172.21.2.117:3300/0 conn(0x7f45a0144810 0x7f45a01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- 172.21.2.116:0/3856868754 >> 172.21.2.116:0/3856868754 conn(0x7f45a005bd20 msgr2=0x7f45a005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- 172.21.2.116:0/3856868754 shutdown_connections 2021-11-21T21:56:12.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- 172.21.2.116:0/3856868754 wait complete. 2021-11-21T21:56:12.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 Processor -- start 2021-11-21T21:56:12.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- start start 2021-11-21T21:56:12.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45a00abce0 0x7f45a01435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a0159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45a0144810 0x7f45a015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b4820 con 0x7f45a00abce0 2021-11-21T21:56:12.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b3f70 con 0x7f45a0144810 2021-11-21T21:56:12.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a8244700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0149230 con 0x7f45a00ac600 2021-11-21T21:56:12.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45a00abce0 0x7f45a01435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45a00abce0 0x7f45a01435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46892/0 (socket says 172.21.2.116:46892) 2021-11-21T21:56:12.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 -- 172.21.2.116:0/3237771970 learned_addr learned my addr 172.21.2.116:0/3237771970 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 -- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 msgr2=0x7f45a0159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a0159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 -- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45a0144810 msgr2=0x7f45a015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45a0144810 0x7f45a015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 -- 172.21.2.116:0/3237771970 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598009470 con 0x7f45a00abce0 2021-11-21T21:56:12.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f45a5fe0700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45a00abce0 0x7f45a01435c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f459c00c000 tx=0x7f459c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f4592ffd700 1 -- 172.21.2.116:0/3237771970 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459c0136e0 con 0x7f45a00abce0 2021-11-21T21:56:12.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f4592ffd700 1 -- 172.21.2.116:0/3237771970 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c013840 con 0x7f45a00abce0 2021-11-21T21:56:12.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.128+0000 7f4592ffd700 1 -- 172.21.2.116:0/3237771970 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459c013ba0 con 0x7f45a00abce0 2021-11-21T21:56:12.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a00b4510 con 0x7f45a00abce0 2021-11-21T21:56:12.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a00b2e30 con 0x7f45a00abce0 2021-11-21T21:56:12.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a01493f0 con 0x7f45a00abce0 2021-11-21T21:56:12.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f4592ffd700 1 -- 172.21.2.116:0/3237771970 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f459c023070 con 0x7f45a00abce0 2021-11-21T21:56:12.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f4592ffd700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4580053580 0x7f4580055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f4592ffd700 1 -- 172.21.2.116:0/3237771970 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f459c06bd10 con 0x7f45a00abce0 2021-11-21T21:56:12.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f45a57df700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4580053580 0x7f4580055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f45a57df700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4580053580 0x7f4580055a40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f45a00b4e00 tx=0x7f4594007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.132+0000 7f4592ffd700 1 -- 172.21.2.116:0/3237771970 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f459c042e20 con 0x7f45a00abce0 2021-11-21T21:56:12.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f45a00af060 con 0x7f45a00abce0 2021-11-21T21:56:12.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f4592ffd700 1 -- 172.21.2.116:0/3237771970 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f459c046440 con 0x7f45a00abce0 2021-11-21T21:56:12.218 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:56:12.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4580053580 msgr2=0x7f4580055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4580053580 0x7f4580055a40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f45a00b4e00 tx=0x7f4594007810).stop 2021-11-21T21:56:12.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45a00abce0 msgr2=0x7f45a01435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45a00abce0 0x7f45a01435c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f459c00c000 tx=0x7f459c008040).stop 2021-11-21T21:56:12.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 shutdown_connections 2021-11-21T21:56:12.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45a0144810 0x7f45a015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45a00ac600 0x7f45a0159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4580053580 0x7f4580055a40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 --2- 172.21.2.116:0/3237771970 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45a00abce0 0x7f45a01435c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.216+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 >> 172.21.2.116:0/3237771970 conn(0x7f45a005bd20 msgr2=0x7f45a00b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.220+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 shutdown_connections 2021-11-21T21:56:12.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.220+0000 7f45a8244700 1 -- 172.21.2.116:0/3237771970 wait complete. 2021-11-21T21:56:12.225 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:56:12.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f85da700 1 Processor -- start 2021-11-21T21:56:12.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f85da700 1 -- start start 2021-11-21T21:56:12.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f85da700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc7f0144810 0x7fc7f01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f85da700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc7f00ac600 0x7fc7f00aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f85da700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f0147e80 con 0x7fc7f00abce0 2021-11-21T21:56:12.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f85da700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00b1490 con 0x7fc7f00ac600 2021-11-21T21:56:12.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f85da700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00b1fd0 con 0x7fc7f0144810 2021-11-21T21:56:12.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc7f00ac600 0x7fc7f00aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc7f00ac600 0x7fc7f00aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46898/0 (socket says 172.21.2.116:46898) 2021-11-21T21:56:12.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 -- 172.21.2.116:0/2151278215 learned_addr learned my addr 172.21.2.116:0/2151278215 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 -- 172.21.2.116:0/2151278215 >> v2:172.21.2.117:3300/0 conn(0x7fc7f0144810 msgr2=0x7fc7f01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:12.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 --2- 172.21.2.116:0/2151278215 >> v2:172.21.2.117:3300/0 conn(0x7fc7f0144810 0x7fc7f01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 -- 172.21.2.116:0/2151278215 >> v1:172.21.2.117:6789/0 conn(0x7fc7f00abce0 legacy=0x7fc7f00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:12.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 -- 172.21.2.116:0/2151278215 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00af5a0 con 0x7fc7f00ac600 2021-11-21T21:56:12.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f6b77700 1 --2- 172.21.2.116:0/2151278215 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f00ac600 0x7fc7f00aca50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fc7f00b4e00 tx=0x7fc7e8009290).ready entity=mon.0 client_cookie=31f6200f6391c25 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f5374700 1 -- 172.21.2.116:0/2151278215 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e800b830 con 0x7fc7f00ac600 2021-11-21T21:56:12.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f5374700 1 -- 172.21.2.116:0/2151278215 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e800b990 con 0x7fc7f00ac600 2021-11-21T21:56:12.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.276+0000 7fc7f5374700 1 -- 172.21.2.116:0/2151278215 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e800bb40 con 0x7fc7f00ac600 2021-11-21T21:56:12.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/2151278215 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f00ac600 msgr2=0x7fc7f00aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 --2- 172.21.2.116:0/2151278215 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f00ac600 0x7fc7f00aca50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fc7f00b4e00 tx=0x7fc7e8009290).stop 2021-11-21T21:56:12.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/2151278215 shutdown_connections 2021-11-21T21:56:12.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 --2- 172.21.2.116:0/2151278215 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f00ac600 0x7fc7f00aca50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 --2- 172.21.2.116:0/2151278215 >> v2:172.21.2.117:3300/0 conn(0x7fc7f0144810 0x7fc7f01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/2151278215 >> 172.21.2.116:0/2151278215 conn(0x7fc7f005bd20 msgr2=0x7fc7f005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/2151278215 shutdown_connections 2021-11-21T21:56:12.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/2151278215 wait complete. 2021-11-21T21:56:12.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 Processor -- start 2021-11-21T21:56:12.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- start start 2021-11-21T21:56:12.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f00abce0 0x7fc7f0145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7f00ac600 0x7fc7f013c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0144810 0x7fc7f013cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f014acf0 con 0x7fc7f0144810 2021-11-21T21:56:12.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b3be0 con 0x7fc7f00ac600 2021-11-21T21:56:12.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b42b0 con 0x7fc7f00abce0 2021-11-21T21:56:12.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0144810 0x7fc7f013cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0144810 0x7fc7f013cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46908/0 (socket says 172.21.2.116:46908) 2021-11-21T21:56:12.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 -- 172.21.2.116:0/949512323 learned_addr learned my addr 172.21.2.116:0/949512323 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 -- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f00abce0 msgr2=0x7fc7f0145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f00abce0 0x7fc7f0145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 -- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7f00ac600 msgr2=0x7fc7f013c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7f00ac600 0x7fc7f013c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 -- 172.21.2.116:0/949512323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e8009530 con 0x7fc7f0144810 2021-11-21T21:56:12.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f6b77700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0144810 0x7fc7f013cb00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc7f0156660 tx=0x7fc7e8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7e37fe700 1 -- 172.21.2.116:0/949512323 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e8021630 con 0x7fc7f0144810 2021-11-21T21:56:12.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f014a1f0 con 0x7fc7f0144810 2021-11-21T21:56:12.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f00674d0 con 0x7fc7f0144810 2021-11-21T21:56:12.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7e37fe700 1 -- 172.21.2.116:0/949512323 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e8021790 con 0x7fc7f0144810 2021-11-21T21:56:12.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7e37fe700 1 -- 172.21.2.116:0/949512323 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e8021af0 con 0x7fc7f0144810 2021-11-21T21:56:12.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f00af060 con 0x7fc7f0144810 2021-11-21T21:56:12.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7e37fe700 1 -- 172.21.2.116:0/949512323 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc7e8027020 con 0x7fc7f0144810 2021-11-21T21:56:12.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7e37fe700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7d0053580 0x7fc7d0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.280+0000 7fc7e37fe700 1 -- 172.21.2.116:0/949512323 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fc7e806d380 con 0x7fc7f0144810 2021-11-21T21:56:12.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.284+0000 7fc7f6376700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7d0053580 0x7fc7d0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.284+0000 7fc7f6376700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7d0053580 0x7fc7d0055a40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fc7ec002f70 tx=0x7fc7ec0069f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.284+0000 7fc7e37fe700 1 -- 172.21.2.116:0/949512323 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc7e80443c0 con 0x7fc7f0144810 2021-11-21T21:56:12.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7f00573d0 con 0x7fc7f0144810 2021-11-21T21:56:12.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7e37fe700 1 -- 172.21.2.116:0/949512323 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc7e8015020 con 0x7fc7f0144810 2021-11-21T21:56:12.368 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:56:12.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7d0053580 msgr2=0x7fc7d0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7d0053580 0x7fc7d0055a40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fc7ec002f70 tx=0x7fc7ec0069f0).stop 2021-11-21T21:56:12.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0144810 msgr2=0x7fc7f013cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0144810 0x7fc7f013cb00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc7f0156660 tx=0x7fc7e8016000).stop 2021-11-21T21:56:12.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 shutdown_connections 2021-11-21T21:56:12.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7d0053580 0x7fc7d0055a40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0144810 0x7fc7f013cb00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7f00ac600 0x7fc7f013c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 --2- 172.21.2.116:0/949512323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f00abce0 0x7fc7f0145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 >> 172.21.2.116:0/949512323 conn(0x7fc7f005bd20 msgr2=0x7fc7f00b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 shutdown_connections 2021-11-21T21:56:12.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.368+0000 7fc7f85da700 1 -- 172.21.2.116:0/949512323 wait complete. 2021-11-21T21:56:12.375 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:56:12.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 Processor -- start 2021-11-21T21:56:12.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- start start 2021-11-21T21:56:12.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080b6770 0x7fdb080b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdb080ad580 0x7fdb080ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb080b0070 con 0x7fdb080b1270 2021-11-21T21:56:12.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080b05c0 con 0x7fdb080ad580 2021-11-21T21:56:12.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080b2230 con 0x7fdb080b6770 2021-11-21T21:56:12.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdb080ad580 0x7fdb080ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdb080ad580 0x7fdb080ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46916/0 (socket says 172.21.2.116:46916) 2021-11-21T21:56:12.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 -- 172.21.2.116:0/3646940537 learned_addr learned my addr 172.21.2.116:0/3646940537 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 -- 172.21.2.116:0/3646940537 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080b6770 msgr2=0x7fdb080b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:12.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 --2- 172.21.2.116:0/3646940537 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080b6770 0x7fdb080b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 -- 172.21.2.116:0/3646940537 >> v1:172.21.2.117:6789/0 conn(0x7fdb080b1270 legacy=0x7fdb081394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:12.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 -- 172.21.2.116:0/3646940537 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080b24d0 con 0x7fdb080ad580 2021-11-21T21:56:12.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0e360700 1 --2- 172.21.2.116:0/3646940537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080ad580 0x7fdb080ad9d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fdb08060900 tx=0x7fdb0000b530).ready entity=mon.0 client_cookie=9c18ece760715d14 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0cb5d700 1 -- 172.21.2.116:0/3646940537 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb0000be70 con 0x7fdb080ad580 2021-11-21T21:56:12.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0cb5d700 1 -- 172.21.2.116:0/3646940537 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb00010470 con 0x7fdb080ad580 2021-11-21T21:56:12.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0cb5d700 1 -- 172.21.2.116:0/3646940537 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb00010810 con 0x7fdb080ad580 2021-11-21T21:56:12.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/3646940537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080ad580 msgr2=0x7fdb080ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/3646940537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080ad580 0x7fdb080ad9d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fdb08060900 tx=0x7fdb0000b530).stop 2021-11-21T21:56:12.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/3646940537 shutdown_connections 2021-11-21T21:56:12.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/3646940537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080ad580 0x7fdb080ad9d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/3646940537 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080b6770 0x7fdb080b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/3646940537 >> 172.21.2.116:0/3646940537 conn(0x7fdb08062040 msgr2=0x7fdb080b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/3646940537 shutdown_connections 2021-11-21T21:56:12.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/3646940537 wait complete. 2021-11-21T21:56:12.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 Processor -- start 2021-11-21T21:56:12.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.428+0000 7fdb0fdc3700 1 -- start start 2021-11-21T21:56:12.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080ad580 0x7fdb0813c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080b1270 0x7fdb08063fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb080b6770 0x7fdb08141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080663f0 con 0x7fdb080b1270 2021-11-21T21:56:12.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb08142370 con 0x7fdb080b6770 2021-11-21T21:56:12.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb08065a10 con 0x7fdb080ad580 2021-11-21T21:56:12.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080b1270 0x7fdb08063fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080b1270 0x7fdb08063fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46920/0 (socket says 172.21.2.116:46920) 2021-11-21T21:56:12.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 -- 172.21.2.116:0/1659734201 learned_addr learned my addr 172.21.2.116:0/1659734201 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 -- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080ad580 msgr2=0x7fdb0813c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080ad580 0x7fdb0813c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 -- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb080b6770 msgr2=0x7fdb08141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb080b6770 0x7fdb08141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 -- 172.21.2.116:0/1659734201 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0000b7d0 con 0x7fdb080b1270 2021-11-21T21:56:12.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0d35e700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080b1270 0x7fdb08063fa0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fdb04002f90 tx=0x7fdb04006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdaf6ffd700 1 -- 172.21.2.116:0/1659734201 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb04009e00 con 0x7fdb080b1270 2021-11-21T21:56:12.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdaf6ffd700 1 -- 172.21.2.116:0/1659734201 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb04013460 con 0x7fdb080b1270 2021-11-21T21:56:12.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdaf6ffd700 1 -- 172.21.2.116:0/1659734201 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb040137c0 con 0x7fdb080b1270 2021-11-21T21:56:12.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb08065cb0 con 0x7fdb080b1270 2021-11-21T21:56:12.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb08137930 con 0x7fdb080b1270 2021-11-21T21:56:12.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb080573d0 con 0x7fdb080b1270 2021-11-21T21:56:12.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdaf6ffd700 1 -- 172.21.2.116:0/1659734201 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdb0400f070 con 0x7fdb080b1270 2021-11-21T21:56:12.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdaf6ffd700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdae80534f0 0x7fdae80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdaf6ffd700 1 -- 172.21.2.116:0/1659734201 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fdb0406b070 con 0x7fdb080b1270 2021-11-21T21:56:12.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.432+0000 7fdb0db5f700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdae80534f0 0x7fdae80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.436+0000 7fdb0db5f700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdae80534f0 0x7fdae80559b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fdaf8002f70 tx=0x7fdaf80069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.436+0000 7fdaf6ffd700 1 -- 172.21.2.116:0/1659734201 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdb04042180 con 0x7fdb080b1270 2021-11-21T21:56:12.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.516+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb08068320 con 0x7fdb080b1270 2021-11-21T21:56:12.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.516+0000 7fdaf6ffd700 1 -- 172.21.2.116:0/1659734201 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdb040457a0 con 0x7fdb080b1270 2021-11-21T21:56:12.519 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:56:12.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdae80534f0 msgr2=0x7fdae80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdae80534f0 0x7fdae80559b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fdaf8002f70 tx=0x7fdaf80069e0).stop 2021-11-21T21:56:12.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080b1270 msgr2=0x7fdb08063fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080b1270 0x7fdb08063fa0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fdb04002f90 tx=0x7fdb04006de0).stop 2021-11-21T21:56:12.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 shutdown_connections 2021-11-21T21:56:12.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb080b6770 0x7fdb08141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdae80534f0 0x7fdae80559b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb080b1270 0x7fdb08063fa0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 --2- 172.21.2.116:0/1659734201 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb080ad580 0x7fdb0813c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 >> 172.21.2.116:0/1659734201 conn(0x7fdb08062040 msgr2=0x7fdb08138d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 shutdown_connections 2021-11-21T21:56:12.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.520+0000 7fdb0fdc3700 1 -- 172.21.2.116:0/1659734201 wait complete. 2021-11-21T21:56:12.528 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:56:12.529 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:12.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 Processor -- start 2021-11-21T21:56:12.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- start start 2021-11-21T21:56:12.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a6aa0 0x7f0be814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0be8062410 0x7f0be81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80a6e80 con 0x7f0be80a86b0 2021-11-21T21:56:12.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be80a8bc0 con 0x7f0be8062410 2021-11-21T21:56:12.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be814a7b0 con 0x7f0be80a6aa0 2021-11-21T21:56:12.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0be8062410 0x7f0be81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0be8062410 0x7f0be81443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46928/0 (socket says 172.21.2.116:46928) 2021-11-21T21:56:12.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 -- 172.21.2.116:0/983790619 learned_addr learned my addr 172.21.2.116:0/983790619 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 -- 172.21.2.116:0/983790619 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a6aa0 msgr2=0x7f0be814b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:12.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 --2- 172.21.2.116:0/983790619 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a6aa0 0x7f0be814b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 -- 172.21.2.116:0/983790619 >> v1:172.21.2.117:6789/0 conn(0x7f0be80a86b0 legacy=0x7f0be81470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:12.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 -- 172.21.2.116:0/983790619 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be814aa10 con 0x7f0be8062410 2021-11-21T21:56:12.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0bed396700 1 --2- 172.21.2.116:0/983790619 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be81443f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f0be8061ab0 tx=0x7f0be0009290).ready entity=mon.0 client_cookie=4617b6a5e552cad5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0be77fe700 1 -- 172.21.2.116:0/983790619 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0be000b830 con 0x7f0be8062410 2021-11-21T21:56:12.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0be77fe700 1 -- 172.21.2.116:0/983790619 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be000b990 con 0x7f0be8062410 2021-11-21T21:56:12.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0be77fe700 1 -- 172.21.2.116:0/983790619 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0be000bb40 con 0x7f0be8062410 2021-11-21T21:56:12.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- 172.21.2.116:0/983790619 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 msgr2=0x7f0be81443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- 172.21.2.116:0/983790619 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be81443f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f0be8061ab0 tx=0x7f0be0009290).stop 2021-11-21T21:56:12.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- 172.21.2.116:0/983790619 shutdown_connections 2021-11-21T21:56:12.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- 172.21.2.116:0/983790619 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be81443f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- 172.21.2.116:0/983790619 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a6aa0 0x7f0be814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- 172.21.2.116:0/983790619 >> 172.21.2.116:0/983790619 conn(0x7f0be805bd20 msgr2=0x7f0be805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- 172.21.2.116:0/983790619 shutdown_connections 2021-11-21T21:56:12.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- 172.21.2.116:0/983790619 wait complete. 2021-11-21T21:56:12.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 Processor -- start 2021-11-21T21:56:12.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- start start 2021-11-21T21:56:12.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be80aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0be80a6aa0 0x7f0be80aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a86b0 0x7f0be80650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80a9c00 con 0x7f0be8062410 2021-11-21T21:56:12.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80aa9b0 con 0x7f0be80a6aa0 2021-11-21T21:56:12.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0beedf9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80a9e60 con 0x7f0be80a86b0 2021-11-21T21:56:12.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0becb95700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be80aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0becb95700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be80aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46932/0 (socket says 172.21.2.116:46932) 2021-11-21T21:56:12.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.580+0000 7f0becb95700 1 -- 172.21.2.116:0/2953069642 learned_addr learned my addr 172.21.2.116:0/2953069642 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:12.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0becb95700 1 -- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a86b0 msgr2=0x7f0be80650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0becb95700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a86b0 0x7f0be80650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0becb95700 1 -- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0be80a6aa0 msgr2=0x7f0be80aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0becb95700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0be80a6aa0 0x7f0be80aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0becb95700 1 -- 172.21.2.116:0/2953069642 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0009530 con 0x7f0be8062410 2021-11-21T21:56:12.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0becb95700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be80aa4c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f0bd0002fb0 tx=0x7f0bd0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be5ffb700 1 -- 172.21.2.116:0/2953069642 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bd0013460 con 0x7f0be8062410 2021-11-21T21:56:12.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be5ffb700 1 -- 172.21.2.116:0/2953069642 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd00135c0 con 0x7f0be8062410 2021-11-21T21:56:12.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be5ffb700 1 -- 172.21.2.116:0/2953069642 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bd0013920 con 0x7f0be8062410 2021-11-21T21:56:12.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be8157110 con 0x7f0be8062410 2021-11-21T21:56:12.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be8156e00 con 0x7f0be8062410 2021-11-21T21:56:12.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be80573d0 con 0x7f0be8062410 2021-11-21T21:56:12.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be5ffb700 1 -- 172.21.2.116:0/2953069642 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0bd000f070 con 0x7f0be8062410 2021-11-21T21:56:12.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be5ffb700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0bcc053610 0x7f0bcc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:12.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be5ffb700 1 -- 172.21.2.116:0/2953069642 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f0bd006b1c0 con 0x7f0be8062410 2021-11-21T21:56:12.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be7fff700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0bcc053610 0x7f0bcc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:12.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.584+0000 7f0be7fff700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0bcc053610 0x7f0bcc055ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0bd8002ef0 tx=0x7f0bd8006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:12.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.588+0000 7f0be5ffb700 1 -- 172.21.2.116:0/2953069642 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0bd00421c0 con 0x7f0be8062410 2021-11-21T21:56:12.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.692+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0be805db70 con 0x7f0be8062410 2021-11-21T21:56:12.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0be5ffb700 1 -- 172.21.2.116:0/2953069642 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f0bd0018030 con 0x7f0be8062410 2021-11-21T21:56:12.695 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:12.696 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:09.961260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:12.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0bcc053610 msgr2=0x7f0bcc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0bcc053610 0x7f0bcc055ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0bd8002ef0 tx=0x7f0bd8006910).stop 2021-11-21T21:56:12.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 msgr2=0x7f0be80aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:12.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be80aa4c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f0bd0002fb0 tx=0x7f0bd0006df0).stop 2021-11-21T21:56:12.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 shutdown_connections 2021-11-21T21:56:12.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0be80a86b0 0x7f0be80650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0be80a6aa0 0x7f0be80aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:12.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0bcc053610 0x7f0bcc055ad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 --2- 172.21.2.116:0/2953069642 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0be8062410 0x7f0be80aa4c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:12.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 >> 172.21.2.116:0/2953069642 conn(0x7f0be805bd20 msgr2=0x7f0be8143630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:12.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 shutdown_connections 2021-11-21T21:56:12.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:12.696+0000 7f0beedf9700 1 -- 172.21.2.116:0/2953069642 wait complete. 2021-11-21T21:56:12.700 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 17 2021-11-21T21:56:13.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:56:07.942419+0000', 'modified': '2021-11-21T21:56:09.961260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5247': {'gid': 5247, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2465449969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2465449969}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2465449969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:56:13.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:56:13.007 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:56:13.129 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5247,"want_state":"up:active","state":"up:active","rank_uptime":4.1673936310000004,"mdsmap_epoch":17,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":8.9056716250000001} 2021-11-21T21:56:13.130 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5247, "mdsmap_epoch": 17, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 4.167393631, "state": "up:active", "uptime": 8.905671625, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:56:13.130 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:56:13.131 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T21:56:13.131 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T21:56:13.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:13.132 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:13.143 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:56:13.195 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:56:13.196 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:56:13.203 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:56:13.204 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:56:13.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:13.206 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:56:13.253 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:56:13.253 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:13.256 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:56:13.266 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:56:13.267 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:56:13.272 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:56:13.272 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:56:13.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:13.274 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:56:13.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T21:56:13.296+0000 7f36fdb22080 -1 init, newargv = 0x56155af77b90 newargc=9 2021-11-21T21:56:13.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14170]: starting ceph client 2021-11-21T21:56:13.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14170]: starting fuse 2021-11-21T21:56:14.379 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:56:14.391 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:56:14.391 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:56:14.438 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:56:14.439 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:56:14.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:14.441 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:56:14.447 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T21:56:14.447 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:56:14.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:14.448 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T21:56:14.449 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T21:56:14.449 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T21:56:14.449 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:56:14.450 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T21:56:14.450 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:14.451 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T21:56:14.451 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T21:56:14.451 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T21:56:14.452 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:14.452 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T21:56:14.453 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T21:56:14.453 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T21:56:14.453 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T21:56:14.454 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:14.454 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T21:56:14.455 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:56:14.455 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:56:14.456 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T21:56:14.456 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:56:14.457 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:14.457 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T21:56:14.459 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:56:14.488 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14170.asok 2021-11-21T21:56:14.490 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14170.asok 2021-11-21T21:56:14.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:14.491 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14170.asok status 2021-11-21T21:56:14.597 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T21:56:14.598 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T21:56:14.598 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:56:14.598 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:56:14.598 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T21:56:14.598 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T21:56:14.599 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T21:56:14.599 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14170", 2021-11-21T21:56:14.599 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T21:56:14.599 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:14.600 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T21:56:14.600 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T21:56:14.600 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5394, 2021-11-21T21:56:14.600 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T21:56:14.601 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T21:56:14.601 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T21:56:14.601 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5394 2021-11-21T21:56:14.601 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:14.601 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:56:14.602 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:56:14.602 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:56:14.602 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3920444899 2021-11-21T21:56:14.602 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T21:56:14.603 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:14.603 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:56:14.603 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:56:14.603 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:56:14.603 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3920444899 2021-11-21T21:56:14.604 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:14.604 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5394 172.21.2.119:0/3920444899", 2021-11-21T21:56:14.604 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3920444899", 2021-11-21T21:56:14.604 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T21:56:14.605 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 17, 2021-11-21T21:56:14.605 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 44, 2021-11-21T21:56:14.605 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:56:14.605 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T21:56:14.606 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T21:56:14.606 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T21:56:14.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:14.608 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:56:14.609 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:14.619 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:56:14.619 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:56:14.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:14.620 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:14.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:14.679 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:56:14.680 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:14.687 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:56:14.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:56:14.688 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T21:56:14.689 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-11-21T21:56:14.689 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-11-21T21:56:14.689 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:56:14.690 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:14.690 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-11-21T21:56:14.691 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-11-21T21:56:14.692 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:14.692 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-11-21T21:56:14.693 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-11-21T21:56:14.693 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:14.693 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-11-21T21:56:14.694 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-11-21T21:56:14.694 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-11-21T21:56:14.695 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-11-21T21:56:14.695 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-11-21T21:56:14.695 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-11-21T21:56:14.696 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-11-21T21:56:14.696 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:56:29.234 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:29.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.345+0000 7f7c53997700 1 Processor -- start 2021-11-21T21:56:29.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.345+0000 7f7c53997700 1 -- start start 2021-11-21T21:56:29.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7c4c0a6aa0 0x7f7c4c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7c4c0a86b0 0x7f7c4c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4c0a6e80 con 0x7f7c4c1475d0 2021-11-21T21:56:29.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4c0a8bc0 con 0x7f7c4c0a6aa0 2021-11-21T21:56:29.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4c14a7b0 con 0x7f7c4c0a86b0 2021-11-21T21:56:29.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51733700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7c4c0a6aa0 0x7f7c4c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7c4c1475d0 0x7f7c4c1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58900/0 (socket says 172.21.2.116:58900) 2021-11-21T21:56:29.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51733700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7c4c0a6aa0 0x7f7c4c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46942/0 (socket says 172.21.2.116:46942) 2021-11-21T21:56:29.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 -- 172.21.2.116:0/1190953348 learned_addr learned my addr 172.21.2.116:0/1190953348 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:29.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51733700 1 -- 172.21.2.116:0/1190953348 >> v2:172.21.2.117:3300/0 conn(0x7f7c4c0a86b0 msgr2=0x7f7c4c1470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:29.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51733700 1 --2- 172.21.2.116:0/1190953348 >> v2:172.21.2.117:3300/0 conn(0x7f7c4c0a86b0 0x7f7c4c1470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51733700 1 -- 172.21.2.116:0/1190953348 >> v1:172.21.2.116:6789/0 conn(0x7f7c4c1475d0 legacy=0x7f7c4c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51733700 1 -- 172.21.2.116:0/1190953348 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c14aa10 con 0x7f7c4c0a6aa0 2021-11-21T21:56:29.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51733700 1 --2- 172.21.2.116:0/1190953348 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c0a6aa0 0x7f7c4c14b150 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7c40002f70 tx=0x7f7c40006df0).ready entity=mon.0 client_cookie=ecf522826f88866b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c3ffff700 1 -- 172.21.2.116:0/1190953348 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c4000d5c0 con 0x7f7c4c0a6aa0 2021-11-21T21:56:29.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c3ffff700 1 -- 172.21.2.116:0/1190953348 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c40000e90 con 0x7f7c4c0a6aa0 2021-11-21T21:56:29.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c3ffff700 1 -- 172.21.2.116:0/1190953348 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c4000d950 con 0x7f7c4c0a6aa0 2021-11-21T21:56:29.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- 172.21.2.116:0/1190953348 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c0a6aa0 msgr2=0x7f7c4c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- 172.21.2.116:0/1190953348 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c0a6aa0 0x7f7c4c14b150 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7c40002f70 tx=0x7f7c40006df0).stop 2021-11-21T21:56:29.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- 172.21.2.116:0/1190953348 shutdown_connections 2021-11-21T21:56:29.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- 172.21.2.116:0/1190953348 >> v2:172.21.2.117:3300/0 conn(0x7f7c4c0a86b0 0x7f7c4c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- 172.21.2.116:0/1190953348 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c0a6aa0 0x7f7c4c14b150 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- 172.21.2.116:0/1190953348 >> 172.21.2.116:0/1190953348 conn(0x7f7c4c05bd20 msgr2=0x7f7c4c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:29.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- 172.21.2.116:0/1190953348 shutdown_connections 2021-11-21T21:56:29.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- 172.21.2.116:0/1190953348 wait complete. 2021-11-21T21:56:29.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 Processor -- start 2021-11-21T21:56:29.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- start start 2021-11-21T21:56:29.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c4c0a6aa0 0x7f7c4c0a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c4c0a86b0 0x7f7c4c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c1475d0 0x7f7c4c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0ab490 con 0x7f7c4c1475d0 2021-11-21T21:56:29.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0ab1f0 con 0x7f7c4c0a6aa0 2021-11-21T21:56:29.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0a9490 con 0x7f7c4c0a86b0 2021-11-21T21:56:29.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c1475d0 0x7f7c4c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c1475d0 0x7f7c4c153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46950/0 (socket says 172.21.2.116:46950) 2021-11-21T21:56:29.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 -- 172.21.2.116:0/1409434373 learned_addr learned my addr 172.21.2.116:0/1409434373 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:29.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 -- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c4c0a86b0 msgr2=0x7f7c4c156550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:29.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c4c0a86b0 0x7f7c4c156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 -- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c4c0a6aa0 msgr2=0x7f7c4c0a9bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:29.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c4c0a6aa0 0x7f7c4c0a9bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 -- 172.21.2.116:0/1409434373 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4000b180 con 0x7f7c4c1475d0 2021-11-21T21:56:29.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c51f34700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c1475d0 0x7f7c4c153710 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f7c4c061ab0 tx=0x7f7c44007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c3e7fc700 1 -- 172.21.2.116:0/1409434373 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c44005b80 con 0x7f7c4c1475d0 2021-11-21T21:56:29.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c3e7fc700 1 -- 172.21.2.116:0/1409434373 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c44005ce0 con 0x7f7c4c1475d0 2021-11-21T21:56:29.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c0a9730 con 0x7f7c4c1475d0 2021-11-21T21:56:29.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c156a40 con 0x7f7c4c1475d0 2021-11-21T21:56:29.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.349+0000 7f7c3e7fc700 1 -- 172.21.2.116:0/1409434373 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c440072c0 con 0x7f7c4c1475d0 2021-11-21T21:56:29.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.353+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c0aba00 con 0x7f7c4c1475d0 2021-11-21T21:56:29.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.353+0000 7f7c3e7fc700 1 -- 172.21.2.116:0/1409434373 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7c44010070 con 0x7f7c4c1475d0 2021-11-21T21:56:29.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.353+0000 7f7c3e7fc700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.353+0000 7f7c3e7fc700 1 -- 172.21.2.116:0/1409434373 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f7c4406b2d0 con 0x7f7c4c1475d0 2021-11-21T21:56:29.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.353+0000 7f7c51733700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.353+0000 7f7c51733700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f7c40002f70 tx=0x7f7c40014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.353+0000 7f7c3e7fc700 1 -- 172.21.2.116:0/1409434373 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c4c0aba00 con 0x7f7c4c1475d0 2021-11-21T21:56:29.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c4c0a98c0 con 0x7f7c4c1475d0 2021-11-21T21:56:29.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c3e7fc700 1 -- 172.21.2.116:0/1409434373 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f7c44042270 con 0x7f7c4c1475d0 2021-11-21T21:56:29.461 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:29.461 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:09.961260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:29.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 msgr2=0x7f7c34055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f7c40002f70 tx=0x7f7c40014000).stop 2021-11-21T21:56:29.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c1475d0 msgr2=0x7f7c4c153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c1475d0 0x7f7c4c153710 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f7c4c061ab0 tx=0x7f7c44007f30).stop 2021-11-21T21:56:29.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 shutdown_connections 2021-11-21T21:56:29.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c4c1475d0 0x7f7c4c153710 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c4c0a86b0 0x7f7c4c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 --2- 172.21.2.116:0/1409434373 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c4c0a6aa0 0x7f7c4c0a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 >> 172.21.2.116:0/1409434373 conn(0x7f7c4c05bd20 msgr2=0x7f7c4c148650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:29.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 shutdown_connections 2021-11-21T21:56:29.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.461+0000 7f7c53997700 1 -- 172.21.2.116:0/1409434373 wait complete. 2021-11-21T21:56:29.467 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 17 2021-11-21T21:56:29.468 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2021-11-21T21:56:29.540 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-11-21T21:56:29.540 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-11-21T21:56:29.541 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:29.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5cda57700 1 Processor -- start 2021-11-21T21:56:29.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5cda57700 1 -- start start 2021-11-21T21:56:29.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5cda57700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa5c8062410 0x7fa5c8144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5cda57700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c80a6e80 con 0x7fa5c80a6aa0 2021-11-21T21:56:29.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5cda57700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c814a7b0 con 0x7fa5c80a86b0 2021-11-21T21:56:29.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5cda57700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80a8bc0 con 0x7fa5c8062410 2021-11-21T21:56:29.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5c6ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa5c80a6aa0 0x7fa5c81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:58912/0 (socket says 172.21.2.116:58912) 2021-11-21T21:56:29.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.601+0000 7fa5c6ffd700 1 -- 172.21.2.116:0/1590354275 learned_addr learned my addr 172.21.2.116:0/1590354275 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:29.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451354503 0 0) 0x7fa5c80a6e80 con 0x7fa5c80a6aa0 2021-11-21T21:56:29.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a8003260 con 0x7fa5c80a6aa0 2021-11-21T21:56:29.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 284833177 0 0) 0x7fa5a8003260 con 0x7fa5c80a6aa0 2021-11-21T21:56:29.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 >> v2:172.21.2.117:3300/0 conn(0x7fa5c8062410 msgr2=0x7fa5c8144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:29.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 --2- 172.21.2.116:0/1590354275 >> v2:172.21.2.117:3300/0 conn(0x7fa5c8062410 0x7fa5c8144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 >> v1:172.21.2.117:6789/0 conn(0x7fa5c80a86b0 legacy=0x7fa5c81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:29.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c814aa10 con 0x7fa5c80a6aa0 2021-11-21T21:56:29.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fa5b8001680 con 0x7fa5c80a6aa0 2021-11-21T21:56:29.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 >> v1:172.21.2.116:6789/0 conn(0x7fa5c80a6aa0 legacy=0x7fa5c81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa5a8003820 0x7fa5a8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5a8004500 0x7fa5a8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa5a8005050 0x7fa5a800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80a8bc0 con 0x7fa5a8004500 2021-11-21T21:56:29.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a800bf40 con 0x7fa5a8005050 2021-11-21T21:56:29.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a800c100 con 0x7fa5a8003820 2021-11-21T21:56:29.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5a8004500 0x7fa5a8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 -- 172.21.2.116:0/1590354275 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa5a8003820 msgr2=0x7fa5a8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:29.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa5a8003820 0x7fa5a8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 -- 172.21.2.116:0/1590354275 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa5a8005050 msgr2=0x7fa5a800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:29.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa5a8005050 0x7fa5a800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 -- 172.21.2.116:0/1590354275 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8003260 con 0x7fa5a8004500 2021-11-21T21:56:29.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c77fe700 1 -- 172.21.2.116:0/1590354275 reap_dead start 2021-11-21T21:56:29.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5a8004500 0x7fa5a8004980 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fa5b0000f70 tx=0x7fa5b00079e0).ready entity=mon.0 client_cookie=5d1516b532b8d70b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5b0007cf0 con 0x7fa5a8004500 2021-11-21T21:56:29.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b0014410 con 0x7fa5a8004500 2021-11-21T21:56:29.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c5ffb700 1 -- 172.21.2.116:0/1590354275 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5b0014770 con 0x7fa5a8004500 2021-11-21T21:56:29.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- 172.21.2.116:0/1590354275 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5a8004500 msgr2=0x7fa5a8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5a8004500 0x7fa5a8004980 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fa5b0000f70 tx=0x7fa5b00079e0).stop 2021-11-21T21:56:29.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- 172.21.2.116:0/1590354275 shutdown_connections 2021-11-21T21:56:29.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1590354275 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5a8004500 0x7fa5a8004980 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- 172.21.2.116:0/1590354275 >> 172.21.2.116:0/1590354275 conn(0x7fa5c805bd20 msgr2=0x7fa5c805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:29.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- 172.21.2.116:0/1590354275 shutdown_connections 2021-11-21T21:56:29.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- 172.21.2.116:0/1590354275 wait complete. 2021-11-21T21:56:29.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 Processor -- start 2021-11-21T21:56:29.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- start start 2021-11-21T21:56:29.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa5a8004500 0x7fa5c814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5c80654a0 0x7fa5c8159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa5c8159e50 0x7fa5c80b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8065e10 con 0x7fa5c80654a0 2021-11-21T21:56:29.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80660b0 con 0x7fa5a8004500 2021-11-21T21:56:29.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8066350 con 0x7fa5c8159e50 2021-11-21T21:56:29.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5c80654a0 0x7fa5c8159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5c80654a0 0x7fa5c8159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46970/0 (socket says 172.21.2.116:46970) 2021-11-21T21:56:29.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 -- 172.21.2.116:0/1786688240 learned_addr learned my addr 172.21.2.116:0/1786688240 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:29.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 -- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa5c8159e50 msgr2=0x7fa5c80b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa5c8159e50 0x7fa5c80b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 -- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa5a8004500 msgr2=0x7fa5c814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa5a8004500 0x7fa5c814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 -- 172.21.2.116:0/1786688240 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80665f0 con 0x7fa5c80654a0 2021-11-21T21:56:29.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5c67fc700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5c80654a0 0x7fa5c8159960 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fa5b0000f70 tx=0x7fa5b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5bffff700 1 -- 172.21.2.116:0/1786688240 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5b0013840 con 0x7fa5c80654a0 2021-11-21T21:56:29.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5bffff700 1 -- 172.21.2.116:0/1786688240 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b00139a0 con 0x7fa5c80654a0 2021-11-21T21:56:29.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c8066a90 con 0x7fa5c80654a0 2021-11-21T21:56:29.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5bffff700 1 -- 172.21.2.116:0/1786688240 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5b0013d00 con 0x7fa5c80654a0 2021-11-21T21:56:29.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.605+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c814c480 con 0x7fa5c80654a0 2021-11-21T21:56:29.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.609+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8066780 con 0x7fa5c80654a0 2021-11-21T21:56:29.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.609+0000 7fa5bffff700 1 -- 172.21.2.116:0/1786688240 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa5b002b3f0 con 0x7fa5c80654a0 2021-11-21T21:56:29.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.609+0000 7fa5bffff700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5ac0536d0 0x7fa5ac055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.609+0000 7fa5bffff700 1 -- 172.21.2.116:0/1786688240 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fa5b001c040 con 0x7fa5c80654a0 2021-11-21T21:56:29.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.609+0000 7fa5c6ffd700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5ac0536d0 0x7fa5ac055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.609+0000 7fa5c6ffd700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5ac0536d0 0x7fa5ac055b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa5c8061b50 tx=0x7fa5c80a8520).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.609+0000 7fa5bffff700 1 -- 172.21.2.116:0/1786688240 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa5b0043fa0 con 0x7fa5c80654a0 2021-11-21T21:56:29.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5c80573d0 con 0x7fa5c80654a0 2021-11-21T21:56:29.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5bffff700 1 -- 172.21.2.116:0/1786688240 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fa5b00309d0 con 0x7fa5c80654a0 2021-11-21T21:56:29.721 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:29.721 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:09.961260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:29.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5ac0536d0 msgr2=0x7fa5ac055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5ac0536d0 0x7fa5ac055b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa5c8061b50 tx=0x7fa5c80a8520).stop 2021-11-21T21:56:29.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5c80654a0 msgr2=0x7fa5c8159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5c80654a0 0x7fa5c8159960 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fa5b0000f70 tx=0x7fa5b0015000).stop 2021-11-21T21:56:29.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 shutdown_connections 2021-11-21T21:56:29.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa5c8159e50 0x7fa5c80b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5ac0536d0 0x7fa5ac055b90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa5c80654a0 0x7fa5c8159960 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 --2- 172.21.2.116:0/1786688240 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa5a8004500 0x7fa5c814b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 >> 172.21.2.116:0/1786688240 conn(0x7fa5c805bd20 msgr2=0x7fa5c81441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:29.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 shutdown_connections 2021-11-21T21:56:29.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.721+0000 7fa5cda57700 1 -- 172.21.2.116:0/1786688240 wait complete. 2021-11-21T21:56:29.725 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 17 2021-11-21T21:56:29.728 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2021-11-21T21:56:29.785 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-11-21T21:56:29.786 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-11-21T21:56:29.787 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:56:29.803 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:29.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.853+0000 7fcc3055c700 1 Processor -- start 2021-11-21T21:56:29.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.853+0000 7fcc3055c700 1 -- start start 2021-11-21T21:56:29.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcc280a6aa0 0x7fcc2814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcc280a86b0 0x7fcc281470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 0x7fcc281443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280a8bc0 con 0x7fcc280a6aa0 2021-11-21T21:56:29.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280a6e80 con 0x7fcc280a86b0 2021-11-21T21:56:29.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2814a7b0 con 0x7fcc281475d0 2021-11-21T21:56:29.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcc280a6aa0 0x7fcc2814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcc280a6aa0 0x7fcc2814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46976/0 (socket says 172.21.2.116:46976) 2021-11-21T21:56:29.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/2938773902 learned_addr learned my addr 172.21.2.116:0/2938773902 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:29.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/2938773902 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 msgr2=0x7fcc281443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:29.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- 172.21.2.116:0/2938773902 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 0x7fcc281443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/2938773902 >> v2:172.21.2.117:3300/0 conn(0x7fcc280a86b0 msgr2=0x7fcc281470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:29.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- 172.21.2.116:0/2938773902 >> v2:172.21.2.117:3300/0 conn(0x7fcc280a86b0 0x7fcc281470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/2938773902 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2814aa10 con 0x7fcc280a6aa0 2021-11-21T21:56:29.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- 172.21.2.116:0/2938773902 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc2814b150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fcc1c002f70 tx=0x7fcc1c006eb0).ready entity=mon.0 client_cookie=8bb1d241e000e7ea server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2d2f6700 1 -- 172.21.2.116:0/2938773902 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1c00d570 con 0x7fcc280a6aa0 2021-11-21T21:56:29.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2d2f6700 1 -- 172.21.2.116:0/2938773902 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c000e90 con 0x7fcc280a6aa0 2021-11-21T21:56:29.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2d2f6700 1 -- 172.21.2.116:0/2938773902 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1c00d900 con 0x7fcc280a6aa0 2021-11-21T21:56:29.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- 172.21.2.116:0/2938773902 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 msgr2=0x7fcc2814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- 172.21.2.116:0/2938773902 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc2814b150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fcc1c002f70 tx=0x7fcc1c006eb0).stop 2021-11-21T21:56:29.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- 172.21.2.116:0/2938773902 shutdown_connections 2021-11-21T21:56:29.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- 172.21.2.116:0/2938773902 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 0x7fcc281443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- 172.21.2.116:0/2938773902 >> v2:172.21.2.117:3300/0 conn(0x7fcc280a86b0 0x7fcc281470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- 172.21.2.116:0/2938773902 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc2814b150 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- 172.21.2.116:0/2938773902 >> 172.21.2.116:0/2938773902 conn(0x7fcc2805bd20 msgr2=0x7fcc2805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:29.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- 172.21.2.116:0/2938773902 shutdown_connections 2021-11-21T21:56:29.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- 172.21.2.116:0/2938773902 wait complete. 2021-11-21T21:56:29.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 Processor -- start 2021-11-21T21:56:29.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- start start 2021-11-21T21:56:29.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc28064990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc280a86b0 0x7fcc280aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 0x7fcc280ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28145090 con 0x7fcc280a6aa0 2021-11-21T21:56:29.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280a8e70 con 0x7fcc280a86b0 2021-11-21T21:56:29.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc3055c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28156d70 con 0x7fcc281475d0 2021-11-21T21:56:29.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc28064990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc28064990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46982/0 (socket says 172.21.2.116:46982) 2021-11-21T21:56:29.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/1696142184 learned_addr learned my addr 172.21.2.116:0/1696142184 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:29.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 msgr2=0x7fcc280ab360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:29.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 0x7fcc280ab360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc280a86b0 msgr2=0x7fcc280aae70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:29.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc280a86b0 0x7fcc280aae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 -- 172.21.2.116:0/1696142184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c00b1e0 con 0x7fcc280a6aa0 2021-11-21T21:56:29.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.857+0000 7fcc2e2f8700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc28064990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fcc1c002f70 tx=0x7fcc1c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc1b7fe700 1 -- 172.21.2.116:0/1696142184 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1c011690 con 0x7fcc280a6aa0 2021-11-21T21:56:29.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc1b7fe700 1 -- 172.21.2.116:0/1696142184 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c0117f0 con 0x7fcc280a6aa0 2021-11-21T21:56:29.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc28157000 con 0x7fcc280a6aa0 2021-11-21T21:56:29.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc1b7fe700 1 -- 172.21.2.116:0/1696142184 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1c011b50 con 0x7fcc280a6aa0 2021-11-21T21:56:29.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc281572a0 con 0x7fcc280a6aa0 2021-11-21T21:56:29.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc280573d0 con 0x7fcc280a6aa0 2021-11-21T21:56:29.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc1b7fe700 1 -- 172.21.2.116:0/1696142184 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcc1c025030 con 0x7fcc280a6aa0 2021-11-21T21:56:29.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc1b7fe700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc080535b0 0x7fcc08055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:29.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc1b7fe700 1 -- 172.21.2.116:0/1696142184 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fcc1c06cc60 con 0x7fcc280a6aa0 2021-11-21T21:56:29.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc2daf7700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc080535b0 0x7fcc08055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:29.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc2daf7700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc080535b0 0x7fcc08055a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fcc24002f70 tx=0x7fcc24006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:29.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.861+0000 7fcc1b7fe700 1 -- 172.21.2.116:0/1696142184 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc1c043cd0 con 0x7fcc280a6aa0 2021-11-21T21:56:29.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.969+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc2805db70 con 0x7fcc280a6aa0 2021-11-21T21:56:29.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.969+0000 7fcc1b7fe700 1 -- 172.21.2.116:0/1696142184 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fcc1c023070 con 0x7fcc280a6aa0 2021-11-21T21:56:29.971 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:29.971 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:09.961260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:29.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc080535b0 msgr2=0x7fcc08055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc080535b0 0x7fcc08055a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fcc24002f70 tx=0x7fcc24006a00).stop 2021-11-21T21:56:29.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 msgr2=0x7fcc28064990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:29.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc28064990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fcc1c002f70 tx=0x7fcc1c015000).stop 2021-11-21T21:56:29.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 shutdown_connections 2021-11-21T21:56:29.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcc281475d0 0x7fcc280ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcc280a86b0 0x7fcc280aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:29.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcc080535b0 0x7fcc08055a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 --2- 172.21.2.116:0/1696142184 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcc280a6aa0 0x7fcc28064990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:29.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 >> 172.21.2.116:0/1696142184 conn(0x7fcc2805bd20 msgr2=0x7fcc28148440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:29.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 shutdown_connections 2021-11-21T21:56:29.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:29.973+0000 7fcc3055c700 1 -- 172.21.2.116:0/1696142184 wait complete. 2021-11-21T21:56:29.976 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 17 2021-11-21T21:56:29.978 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 2021-11-21T21:56:30.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.029+0000 7f8dfdb43700 1 Processor -- start 2021-11-21T21:56:30.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.029+0000 7f8dfdb43700 1 -- start start 2021-11-21T21:56:30.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.029+0000 7f8dfdb43700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df80abce0 0x7f8df81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.029+0000 7f8dfdb43700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8df80ac600 0x7f8df80acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.029+0000 7f8dfdb43700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df8147e80 con 0x7f8df8144810 2021-11-21T21:56:30.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.029+0000 7f8dfdb43700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df80b1490 con 0x7f8df80ac600 2021-11-21T21:56:30.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.029+0000 7f8dfdb43700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80b1fd0 con 0x7f8df80abce0 2021-11-21T21:56:30.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8df80ac600 0x7f8df80acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8df80ac600 0x7f8df80acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:48158/0 (socket says 172.21.2.116:48158) 2021-11-21T21:56:30.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 -- 172.21.2.116:0/2519821875 learned_addr learned my addr 172.21.2.116:0/2519821875 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:30.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 --1- 172.21.2.116:0/2519821875 >> v1:172.21.2.117:6789/0 conn(0x7f8df8144810 0x7f8df80b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:57856/0 (socket says 172.21.2.116:57856) 2021-11-21T21:56:30.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df80abce0 0x7f8df81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 -- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df80abce0 msgr2=0x7f8df81478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 --2- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df80abce0 0x7f8df81478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 -- 172.21.2.116:0/2519821875 >> v1:172.21.2.117:6789/0 conn(0x7f8df8144810 legacy=0x7f8df80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:30.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 -- 172.21.2.116:0/2519821875 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df80af5a0 con 0x7f8df80ac600 2021-11-21T21:56:30.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df80abce0 0x7f8df81478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:56:30.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 --2- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 0x7f8df80acab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8df00032f0 tx=0x7f8df0009290).ready entity=mon.1 client_cookie=e8fc69ad830e15c6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:30.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df67fc700 1 -- 172.21.2.116:0/2519821875 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df000b8a0 con 0x7f8df80ac600 2021-11-21T21:56:30.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df67fc700 1 -- 172.21.2.116:0/2519821875 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df000ba00 con 0x7f8df80ac600 2021-11-21T21:56:30.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df67fc700 1 -- 172.21.2.116:0/2519821875 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df000bb70 con 0x7f8df80ac600 2021-11-21T21:56:30.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 msgr2=0x7f8df80acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:30.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 0x7f8df80acab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8df00032f0 tx=0x7f8df0009290).stop 2021-11-21T21:56:30.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2519821875 shutdown_connections 2021-11-21T21:56:30.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 0x7f8df80acab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/2519821875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df80abce0 0x7f8df81478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2519821875 >> 172.21.2.116:0/2519821875 conn(0x7f8df805bd20 msgr2=0x7f8df805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:30.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2519821875 shutdown_connections 2021-11-21T21:56:30.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2519821875 wait complete. 2021-11-21T21:56:30.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 Processor -- start 2021-11-21T21:56:30.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- start start 2021-11-21T21:56:30.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8df80abce0 0x7f8df813c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 0x7f8df80b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df8144810 0x7f8df8142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81573a0 con 0x7f8df80abce0 2021-11-21T21:56:30.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8142090 con 0x7f8df80ac600 2021-11-21T21:56:30.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8142330 con 0x7f8df8144810 2021-11-21T21:56:30.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8df80abce0 0x7f8df813c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8df80abce0 0x7f8df813c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:46994/0 (socket says 172.21.2.116:46994) 2021-11-21T21:56:30.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 -- 172.21.2.116:0/4074008375 learned_addr learned my addr 172.21.2.116:0/4074008375 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:30.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 0x7f8df80b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df7fff700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df8144810 0x7f8df8142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 -- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df8144810 msgr2=0x7f8df8142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:30.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df8144810 0x7f8df8142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 -- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 msgr2=0x7f8df80b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:30.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 0x7f8df80b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 -- 172.21.2.116:0/4074008375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de000b180 con 0x7f8df80abce0 2021-11-21T21:56:30.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df77fe700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8df80abce0 0x7f8df813c2b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f8df80b2a30 tx=0x7f8de8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:30.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df4ff9700 1 -- 172.21.2.116:0/4074008375 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de80070d0 con 0x7f8df80abce0 2021-11-21T21:56:30.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df4ff9700 1 -- 172.21.2.116:0/4074008375 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de8007230 con 0x7f8df80abce0 2021-11-21T21:56:30.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8df4ff9700 1 -- 172.21.2.116:0/4074008375 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de8005550 con 0x7f8df80abce0 2021-11-21T21:56:30.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.033+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df0009530 con 0x7f8df80abce0 2021-11-21T21:56:30.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df8142f60 con 0x7f8df80abce0 2021-11-21T21:56:30.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df4ff9700 1 -- 172.21.2.116:0/4074008375 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8de8010070 con 0x7f8df80abce0 2021-11-21T21:56:30.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df4ff9700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8ddc0536a0 0x7f8ddc055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df4ff9700 1 -- 172.21.2.116:0/4074008375 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8de806af30 con 0x7f8df80abce0 2021-11-21T21:56:30.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 Processor -- start 2021-11-21T21:56:30.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- start start 2021-11-21T21:56:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8ddc0536a0 0x7f8ddc055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x23ccec0 0x24032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x23ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x23fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x2329d50 con 0x24037c0 2021-11-21T21:56:30.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x233d4b0 con 0x2403c40 2021-11-21T21:56:30.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x231dc20 con 0x23ccec0 2021-11-21T21:56:30.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x23ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x23ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47004/0 (socket says 172.21.2.116:47004) 2021-11-21T21:56:30.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 -- 172.21.2.116:0/2000385597 learned_addr learned my addr 172.21.2.116:0/2000385597 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:30.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 -- 172.21.2.116:0/2000385597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x23ccec0 msgr2=0x24032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/2000385597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x23ccec0 0x24032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 -- 172.21.2.116:0/2000385597 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 msgr2=0x23fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/2000385597 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x23fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 -- 172.21.2.116:0/2000385597 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2375cb0 con 0x24037c0 2021-11-21T21:56:30.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/2000385597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x23ffad0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x2344360 tx=0x7f8de001d000).ready entity=mon.0 client_cookie=68bf7429bfca842f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:30.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2000385597 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de001d070 con 0x24037c0 2021-11-21T21:56:30.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2000385597 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0011860 con 0x24037c0 2021-11-21T21:56:30.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8dfdb43700 1 -- 172.21.2.116:0/2000385597 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de0015ea0 con 0x24037c0 2021-11-21T21:56:30.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- 172.21.2.116:0/2000385597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 msgr2=0x23ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:30.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8ddc0536a0 0x7f8ddc055b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8de000a000 tx=0x7f8de0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:30.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2000385597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x23ffad0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x2344360 tx=0x7f8de001d000).stop 2021-11-21T21:56:30.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- 172.21.2.116:0/2000385597 shutdown_connections 2021-11-21T21:56:30.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2000385597 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x23fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2000385597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x23ffad0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2000385597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x23ccec0 0x24032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- 172.21.2.116:0/2000385597 >> 172.21.2.116:0/2000385597 conn(0x23ca630 msgr2=0x23cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:30.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- 172.21.2.116:0/2000385597 shutdown_connections 2021-11-21T21:56:30.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- 172.21.2.116:0/2000385597 wait complete. 2021-11-21T21:56:30.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- start start 2021-11-21T21:56:30.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 Processor -- start 2021-11-21T21:56:30.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x24037c0 0x23f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x2402660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x2405980 0x23fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x2402b50 con 0x2405980 2021-11-21T21:56:30.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x2402d70 con 0x2403c40 2021-11-21T21:56:30.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x23f14a0 con 0x24037c0 2021-11-21T21:56:30.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8e07d87740 10 client.0 ll_register_callbacks cb 0x23312a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:56:30.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x2405980 0x23fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x2405980 0x23fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47010/0 (socket says 172.21.2.116:47010) 2021-11-21T21:56:30.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.037+0000 7f8df7fff700 1 -- 172.21.2.116:0/2029609323 learned_addr learned my addr 172.21.2.116:0/2029609323 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:30.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x2402660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df7fff700 1 -- 172.21.2.116:0/2029609323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x24037c0 msgr2=0x23f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df7fff700 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x24037c0 0x23f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df7fff700 1 -- 172.21.2.116:0/2029609323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 msgr2=0x2402660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df7fff700 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x2402660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df7fff700 1 -- 172.21.2.116:0/2029609323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de0015620 con 0x2405980 2021-11-21T21:56:30.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df7fff700 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x2405980 0x23fe5b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f8df001b000 tx=0x7f8df0016000).ready entity=mon.0 client_cookie=66a4c3127b748c61 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:30.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:56:30.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 1 -- 172.21.2.116:0/2029609323 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df000b8a0 con 0x2405980 2021-11-21T21:56:30.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2333570 con 0x2405980 2021-11-21T21:56:30.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 1 -- 172.21.2.116:0/2029609323 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8df0006d40 con 0x2405980 2021-11-21T21:56:30.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 1 -- 172.21.2.116:0/2029609323 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df0011d90 con 0x2405980 2021-11-21T21:56:30.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 10 client.5424 fetch_fsmap learned FSMap version 17 2021-11-21T21:56:30.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x232ce50 con 0x2405980 2021-11-21T21:56:30.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 1 -- 172.21.2.116:0/2029609323 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2333570 con 0x2405980 2021-11-21T21:56:30.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 1 -- 172.21.2.116:0/2029609323 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 17) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f8df002abc0 con 0x2405980 2021-11-21T21:56:30.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 10 client.5424 fetch_fsmap finished waiting for FSMap version 17 2021-11-21T21:56:30.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 10 client.5424 resolve_mds: resolved ID 'c' to GID 5247 2021-11-21T21:56:30.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 20 client.5424 populate_metadata read hostname 'gibba016' 2021-11-21T21:56:30.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x24156d0 0x2415ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 4 client.5424 mds_command: new command op to 5247 tid=0[{"prefix": "get_command_descriptions"}] 2021-11-21T21:56:30.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 --> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x232b8e0 con 0x24156d0 2021-11-21T21:56:30.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df77fe700 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x24156d0 0x2415ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 10 client.5424 ms_handle_connect on v2:172.21.2.116:6834/2465449969 2021-11-21T21:56:30.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8df77fe700 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x24156d0 0x2415ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e786e5cb74994ef3 server_cookie=48d9d1b6054811ef in_seq=0 out_seq=0 2021-11-21T21:56:30.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 1 -- 172.21.2.116:0/2029609323 <== mds.0 v2:172.21.2.116:6834/2465449969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x232b8e0 con 0x24156d0 2021-11-21T21:56:30.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 10 client.5424 handle_command_reply: tid=0 2021-11-21T21:56:30.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 client.5424 shutdown 2021-11-21T21:56:30.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x2405980 msgr2=0x23fe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:30.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x2405980 0x23fe5b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f8df001b000 tx=0x7f8df0016000).stop 2021-11-21T21:56:30.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 shutdown_connections 2021-11-21T21:56:30.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x24156d0 0x2415ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x2405980 0x23fe5b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8dd67fc700 0 client.5424 ms_handle_reset on v2:172.21.2.116:6834/2465449969 2021-11-21T21:56:30.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x2402660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2029609323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x24037c0 0x23f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 >> 172.21.2.116:0/2029609323 conn(0x23ccec0 msgr2=0x24002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:30.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 shutdown_connections 2021-11-21T21:56:30.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 1 -- 172.21.2.116:0/2029609323 wait complete. 2021-11-21T21:56:30.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.041+0000 7f8e07d87740 20 client.5424 trim_cache size 0 max 16384 2021-11-21T21:56:30.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 Processor -- start 2021-11-21T21:56:30.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- start start 2021-11-21T21:56:30.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x2402360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2403c40 0x2405d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2405980 0x2400f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x240abd0 con 0x24037c0 2021-11-21T21:56:30.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x230f070 con 0x2405980 2021-11-21T21:56:30.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x2395a40 con 0x2403c40 2021-11-21T21:56:30.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x2402360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x2402360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47016/0 (socket says 172.21.2.116:47016) 2021-11-21T21:56:30.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 -- 172.21.2.116:0/2461490870 learned_addr learned my addr 172.21.2.116:0/2461490870 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:30.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 -- 172.21.2.116:0/2461490870 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2403c40 msgr2=0x2405d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 --2- 172.21.2.116:0/2461490870 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2403c40 0x2405d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 -- 172.21.2.116:0/2461490870 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2405980 msgr2=0x2400f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:30.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 --2- 172.21.2.116:0/2461490870 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2405980 0x2400f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 -- 172.21.2.116:0/2461490870 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23f21e0 con 0x24037c0 2021-11-21T21:56:30.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 --2- 172.21.2.116:0/2461490870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x2402360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8de8014000 tx=0x7f8de8009000).ready entity=mon.0 client_cookie=9d0e4a2cb618e4a7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:30.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8decff9700 1 -- 172.21.2.116:0/2461490870 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de804a070 con 0x24037c0 2021-11-21T21:56:30.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8decff9700 1 -- 172.21.2.116:0/2461490870 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de804b040 con 0x24037c0 2021-11-21T21:56:30.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8decff9700 1 -- 172.21.2.116:0/2461490870 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de804e280 con 0x24037c0 2021-11-21T21:56:30.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- 172.21.2.116:0/2461490870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 msgr2=0x2402360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:30.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2461490870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x2402360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8de8014000 tx=0x7f8de8009000).stop 2021-11-21T21:56:30.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- 172.21.2.116:0/2461490870 shutdown_connections 2021-11-21T21:56:30.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2461490870 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2405980 0x2400f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2461490870 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2403c40 0x2405d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- 172.21.2.116:0/2461490870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x2402360 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:30.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- 172.21.2.116:0/2461490870 >> 172.21.2.116:0/2461490870 conn(0x23ccec0 msgr2=0x23cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:30.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- 172.21.2.116:0/2461490870 shutdown_connections 2021-11-21T21:56:30.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- 172.21.2.116:0/2461490870 wait complete. 2021-11-21T21:56:30.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- start start 2021-11-21T21:56:30.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 Processor -- start 2021-11-21T21:56:30.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x24666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x2466bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2405980 0x23ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x24072b0 con 0x24037c0 2021-11-21T21:56:30.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x246a810 con 0x2403c40 2021-11-21T21:56:30.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x2406c40 con 0x2405980 2021-11-21T21:56:30.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8e07d87740 10 client.0 ll_register_callbacks cb 0x2402cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:56:30.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x24666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x24666e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47024/0 (socket says 172.21.2.116:47024) 2021-11-21T21:56:30.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.049+0000 7f8df77fe700 1 -- 172.21.2.116:0/848840170 learned_addr learned my addr 172.21.2.116:0/848840170 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:30.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df77fe700 1 -- 172.21.2.116:0/848840170 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2405980 msgr2=0x23ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df77fe700 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2405980 0x23ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df77fe700 1 -- 172.21.2.116:0/848840170 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 msgr2=0x2466bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:30.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df77fe700 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x2466bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:30.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df77fe700 1 -- 172.21.2.116:0/848840170 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df8142f60 con 0x24037c0 2021-11-21T21:56:30.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df77fe700 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x24666e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f8de8014000 tx=0x7f8de8047030).ready entity=mon.0 client_cookie=67ca72489901d52a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:30.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8dd77fe700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:56:30.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8dd77fe700 1 -- 172.21.2.116:0/848840170 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de8045340 con 0x24037c0 2021-11-21T21:56:30.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8de0011860 con 0x24037c0 2021-11-21T21:56:30.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8dd77fe700 1 -- 172.21.2.116:0/848840170 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8de8041a50 con 0x24037c0 2021-11-21T21:56:30.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8dd77fe700 1 -- 172.21.2.116:0/848840170 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8de8041c10 con 0x24037c0 2021-11-21T21:56:30.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 10 client.5430 fetch_fsmap learned FSMap version 17 2021-11-21T21:56:30.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x2403630 con 0x24037c0 2021-11-21T21:56:30.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8dd77fe700 1 -- 172.21.2.116:0/848840170 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0011860 con 0x24037c0 2021-11-21T21:56:30.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8dd77fe700 1 -- 172.21.2.116:0/848840170 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 17) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f8de8044bc0 con 0x24037c0 2021-11-21T21:56:30.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 10 client.5430 fetch_fsmap finished waiting for FSMap version 17 2021-11-21T21:56:30.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 10 client.5430 resolve_mds: resolved ID 'c' to GID 5247 2021-11-21T21:56:30.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 20 client.5430 populate_metadata read hostname 'gibba016' 2021-11-21T21:56:30.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x2412a40 0x23ff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:30.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 4 client.5430 mds_command: new command op to 5247 tid=0[{"prefix": "cache drop"}] 2021-11-21T21:56:30.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 --> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x2221590 con 0x2412a40 2021-11-21T21:56:30.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x2412a40 0x23ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:30.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8df6ffd700 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x2412a40 0x23ff690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c6f0ef924c4052c1 server_cookie=3b1e4c0a97ac5c1f in_seq=0 out_seq=0 2021-11-21T21:56:30.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:30.053+0000 7f8dd77fe700 10 client.5430 ms_handle_connect on v2:172.21.2.116:6834/2465449969 2021-11-21T21:56:33.796 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:33.799 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:33.802 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:33.805 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:56:41.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dd77fe700 1 -- 172.21.2.116:0/848840170 <== mds.0 v2:172.21.2.116:6834/2465449969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+380 (crc 0 0 0) 0x2221590 con 0x2412a40 2021-11-21T21:56:41.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dd77fe700 10 client.5430 handle_command_reply: tid=0 2021-11-21T21:56:41.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 client.5430 shutdown 2021-11-21T21:56:41.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 msgr2=0x24666e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x24666e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f8de8014000 tx=0x7f8de8047030).stop 2021-11-21T21:56:41.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 shutdown_connections 2021-11-21T21:56:41.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:6834/2465449969,v1:172.21.2.116:6835/2465449969] conn(0x2412a40 0x23ff690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x2405980 0x23ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dd77fe700 0 client.5430 ms_handle_reset on v2:172.21.2.116:6834/2465449969 2021-11-21T21:56:41.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x2403c40 0x2466bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 --2- 172.21.2.116:0/848840170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x24037c0 0x24666e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 >> 172.21.2.116:0/848840170 conn(0x23ccec0 msgr2=0x23cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 shutdown_connections 2021-11-21T21:56:41.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 1 -- 172.21.2.116:0/848840170 wait complete. 2021-11-21T21:56:41.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8e07d87740 20 client.5430 trim_cache size 0 max 16384 2021-11-21T21:56:41.139 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:56:41.139 INFO:teuthology.orchestra.run.gibba016.stdout: "client_recall": { 2021-11-21T21:56:41.140 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": 0, 2021-11-21T21:56:41.140 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "(0) Success", 2021-11-21T21:56:41.140 INFO:teuthology.orchestra.run.gibba016.stdout: "recalled": 220 2021-11-21T21:56:41.141 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:41.141 INFO:teuthology.orchestra.run.gibba016.stdout: "flush_journal": { 2021-11-21T21:56:41.141 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": 0, 2021-11-21T21:56:41.141 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "" 2021-11-21T21:56:41.141 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:41.142 INFO:teuthology.orchestra.run.gibba016.stdout: "trim_cache": { 2021-11-21T21:56:41.142 INFO:teuthology.orchestra.run.gibba016.stdout: "trimmed": 0 2021-11-21T21:56:41.142 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:41.142 INFO:teuthology.orchestra.run.gibba016.stdout: "cache": { 2021-11-21T21:56:41.142 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": { 2021-11-21T21:56:41.143 INFO:teuthology.orchestra.run.gibba016.stdout: "items": 19930, 2021-11-21T21:56:41.143 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes": 3095840 2021-11-21T21:56:41.143 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:56:41.143 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:56:41.143 INFO:teuthology.orchestra.run.gibba016.stdout: "duration": 11.081920252 2021-11-21T21:56:41.144 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:56:41.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8ddc0536a0 msgr2=0x7f8ddc055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8ddc0536a0 0x7f8ddc055b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8de000a000 tx=0x7f8de0006df0).stop 2021-11-21T21:56:41.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8df80abce0 msgr2=0x7f8df813c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8df80abce0 0x7f8df813c2b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f8df80b2a30 tx=0x7f8de8007c40).stop 2021-11-21T21:56:41.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 shutdown_connections 2021-11-21T21:56:41.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8df8144810 0x7f8df8142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8df80ac600 0x7f8df80b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8ddc0536a0 0x7f8ddc055b60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 --2- 172.21.2.116:0/4074008375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8df80abce0 0x7f8df813c2b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.133+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 >> 172.21.2.116:0/4074008375 conn(0x7f8df805bd20 msgr2=0x7f8df813ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.137+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 shutdown_connections 2021-11-21T21:56:41.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.137+0000 7f8dfdb43700 1 -- 172.21.2.116:0/4074008375 wait complete. 2021-11-21T21:56:41.149 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-11-21T21:56:41.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:41.150 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:56:41.150 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:41.160 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-11-21T21:56:41.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:56:41.162 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-11-21T21:56:41.162 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:56:41.163 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:56:41.163 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:41.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:41.180 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:56:41.180 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:41.185 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T21:56:41.185 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:56:41.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:56:41.186 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:41.232 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:56:41.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.345+0000 7f77d37f7700 1 Processor -- start 2021-11-21T21:56:41.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.345+0000 7f77d37f7700 1 -- start start 2021-11-21T21:56:41.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 0x7f77cc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f77cc062410 0x7f77cc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc0a6e80 con 0x7f77cc0a86b0 2021-11-21T21:56:41.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc0a8bc0 con 0x7f77cc062410 2021-11-21T21:56:41.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc14a7b0 con 0x7f77cc0a6aa0 2021-11-21T21:56:41.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f77cc062410 0x7f77cc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f77cc062410 0x7f77cc1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47032/0 (socket says 172.21.2.116:47032) 2021-11-21T21:56:41.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/1719309467 learned_addr learned my addr 172.21.2.116:0/1719309467 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/1719309467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 msgr2=0x7f77cc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- 172.21.2.116:0/1719309467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 0x7f77cc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/1719309467 >> v1:172.21.2.116:6789/0 conn(0x7f77cc0a86b0 legacy=0x7f77cc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/1719309467 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc14aa10 con 0x7f77cc062410 2021-11-21T21:56:41.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- 172.21.2.116:0/1719309467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc062410 0x7f77cc1443f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f77c0001110 tx=0x7f77c4009290).ready entity=mon.0 client_cookie=ff5bc006c3ccef18 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77bffff700 1 -- 172.21.2.116:0/1719309467 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77c400b760 con 0x7f77cc062410 2021-11-21T21:56:41.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77bffff700 1 -- 172.21.2.116:0/1719309467 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c400b8c0 con 0x7f77cc062410 2021-11-21T21:56:41.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77bffff700 1 -- 172.21.2.116:0/1719309467 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77c400ba30 con 0x7f77cc062410 2021-11-21T21:56:41.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- 172.21.2.116:0/1719309467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc062410 msgr2=0x7f77cc1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- 172.21.2.116:0/1719309467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc062410 0x7f77cc1443f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f77c0001110 tx=0x7f77c4009290).stop 2021-11-21T21:56:41.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- 172.21.2.116:0/1719309467 shutdown_connections 2021-11-21T21:56:41.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- 172.21.2.116:0/1719309467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc062410 0x7f77cc1443f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- 172.21.2.116:0/1719309467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 0x7f77cc14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- 172.21.2.116:0/1719309467 >> 172.21.2.116:0/1719309467 conn(0x7f77cc05bd20 msgr2=0x7f77cc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- 172.21.2.116:0/1719309467 shutdown_connections 2021-11-21T21:56:41.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- 172.21.2.116:0/1719309467 wait complete. 2021-11-21T21:56:41.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 Processor -- start 2021-11-21T21:56:41.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- start start 2021-11-21T21:56:41.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77cc062410 0x7f77cc064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 0x7f77cc066d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc0a86b0 0x7f77cc067280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0a94b0 con 0x7f77cc0a86b0 2021-11-21T21:56:41.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0a9b40 con 0x7f77cc062410 2021-11-21T21:56:41.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d37f7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc15a210 con 0x7f77cc0a6aa0 2021-11-21T21:56:41.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc0a86b0 0x7f77cc067280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc0a86b0 0x7f77cc067280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47040/0 (socket says 172.21.2.116:47040) 2021-11-21T21:56:41.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/3170262664 learned_addr learned my addr 172.21.2.116:0/3170262664 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 msgr2=0x7f77cc066d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:41.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 0x7f77cc066d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77cc062410 msgr2=0x7f77cc064ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:41.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77cc062410 0x7f77cc064ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.349+0000 7f77d1d94700 1 -- 172.21.2.116:0/3170262664 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4009530 con 0x7f77cc0a86b0 2021-11-21T21:56:41.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77d1d94700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc0a86b0 0x7f77cc067280 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f77cc064c00 tx=0x7f77c4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77be7fc700 1 -- 172.21.2.116:0/3170262664 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77c400b760 con 0x7f77cc0a86b0 2021-11-21T21:56:41.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77be7fc700 1 -- 172.21.2.116:0/3170262664 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4011e70 con 0x7f77cc0a86b0 2021-11-21T21:56:41.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc15a4b0 con 0x7f77cc0a86b0 2021-11-21T21:56:41.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77be7fc700 1 -- 172.21.2.116:0/3170262664 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77c4020620 con 0x7f77cc0a86b0 2021-11-21T21:56:41.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc0654e0 con 0x7f77cc0a86b0 2021-11-21T21:56:41.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc15a640 con 0x7f77cc0a86b0 2021-11-21T21:56:41.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77be7fc700 1 -- 172.21.2.116:0/3170262664 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f77c400f070 con 0x7f77cc0a86b0 2021-11-21T21:56:41.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77be7fc700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77ac053660 0x7f77ac055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77be7fc700 1 -- 172.21.2.116:0/3170262664 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f77c406cb90 con 0x7f77cc0a86b0 2021-11-21T21:56:41.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77d1593700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77ac053660 0x7f77ac055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.353+0000 7f77d1593700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77ac053660 0x7f77ac055b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f77c8002f70 tx=0x7f77c8006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.357+0000 7f77be7fc700 1 -- 172.21.2.116:0/3170262664 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77c4043b90 con 0x7f77cc0a86b0 2021-11-21T21:56:41.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77cc15a640 con 0x7f77cc0a86b0 2021-11-21T21:56:41.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77be7fc700 1 -- 172.21.2.116:0/3170262664 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16167 (secure 0 0 0) 0x7f77c4015070 con 0x7f77cc0a86b0 2021-11-21T21:56:41.438 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:41.438 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:56:09.235532+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:56:06.102882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-11-21T21:56:06.914261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.116:6835/2140793945":"2021-11-22T21:56:08.223482+0000","172.21.2.116:6834/2140793945":"2021-11-22T21:56:08.223482+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:56:41.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77ac053660 msgr2=0x7f77ac055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77ac053660 0x7f77ac055b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f77c8002f70 tx=0x7f77c8006a00).stop 2021-11-21T21:56:41.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc0a86b0 msgr2=0x7f77cc067280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc0a86b0 0x7f77cc067280 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f77cc064c00 tx=0x7f77c4016000).stop 2021-11-21T21:56:41.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 shutdown_connections 2021-11-21T21:56:41.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77ac053660 0x7f77ac055b20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77cc0a86b0 0x7f77cc067280 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f77cc0a6aa0 0x7f77cc066d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 --2- 172.21.2.116:0/3170262664 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77cc062410 0x7f77cc064ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 >> 172.21.2.116:0/3170262664 conn(0x7f77cc05bd20 msgr2=0x7f77cc1435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 shutdown_connections 2021-11-21T21:56:41.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.437+0000 7f77d37f7700 1 -- 172.21.2.116:0/3170262664 wait complete. 2021-11-21T21:56:41.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:56:06.102882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:56:06.914261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:56:41.445 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:56:41.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 Processor -- start 2021-11-21T21:56:41.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 -- start start 2021-11-21T21:56:41.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4aa8144810 0x7f4aa81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f4aa80abce0 0x7f4aa80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80b1db0 0x7f4aa80ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa8147e80 con 0x7f4aa8144810 2021-11-21T21:56:41.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa80af5a0 con 0x7f4aa80abce0 2021-11-21T21:56:41.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aafe06700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80b1490 con 0x7f4aa80b1db0 2021-11-21T21:56:41.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4aa8144810 0x7f4aa81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4aa8144810 0x7f4aa81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47044/0 (socket says 172.21.2.116:47044) 2021-11-21T21:56:41.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 -- 172.21.2.116:0/292715044 learned_addr learned my addr 172.21.2.116:0/292715044 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 -- 172.21.2.116:0/292715044 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80b1db0 msgr2=0x7f4aa80ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:41.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 --2- 172.21.2.116:0/292715044 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80b1db0 0x7f4aa80ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 -- 172.21.2.116:0/292715044 >> v2:172.21.2.117:3300/0 conn(0x7f4aa80abce0 msgr2=0x7f4aa80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:56:41.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 --2- 172.21.2.116:0/292715044 >> v2:172.21.2.117:3300/0 conn(0x7f4aa80abce0 0x7f4aa80b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 -- 172.21.2.116:0/292715044 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80af060 con 0x7f4aa8144810 2021-11-21T21:56:41.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aadba2700 1 --2- 172.21.2.116:0/292715044 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144810 0x7f4aa81478c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f4a9c002f70 tx=0x7f4a9c006eb0).ready entity=mon.0 client_cookie=d4ccc7094d0ed004 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aacba0700 1 -- 172.21.2.116:0/292715044 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a9c00d570 con 0x7f4aa8144810 2021-11-21T21:56:41.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aacba0700 1 -- 172.21.2.116:0/292715044 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c000e90 con 0x7f4aa8144810 2021-11-21T21:56:41.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.537+0000 7f4aacba0700 1 -- 172.21.2.116:0/292715044 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a9c00d900 con 0x7f4aa8144810 2021-11-21T21:56:41.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/292715044 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144810 msgr2=0x7f4aa81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 --2- 172.21.2.116:0/292715044 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144810 0x7f4aa81478c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f4a9c002f70 tx=0x7f4a9c006eb0).stop 2021-11-21T21:56:41.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/292715044 shutdown_connections 2021-11-21T21:56:41.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 --2- 172.21.2.116:0/292715044 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80b1db0 0x7f4aa80ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 --2- 172.21.2.116:0/292715044 >> v2:172.21.2.117:3300/0 conn(0x7f4aa80abce0 0x7f4aa80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 --2- 172.21.2.116:0/292715044 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144810 0x7f4aa81478c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/292715044 >> 172.21.2.116:0/292715044 conn(0x7f4aa805bd20 msgr2=0x7f4aa805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/292715044 shutdown_connections 2021-11-21T21:56:41.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/292715044 wait complete. 2021-11-21T21:56:41.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 Processor -- start 2021-11-21T21:56:41.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- start start 2021-11-21T21:56:41.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80abce0 0x7f4aa8145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4aa8144810 0x7f4aa8145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144d20 0x7f4aa815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa8143c50 con 0x7f4aa8144d20 2021-11-21T21:56:41.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa8143470 con 0x7f4aa8144810 2021-11-21T21:56:41.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa8143710 con 0x7f4aa80abce0 2021-11-21T21:56:41.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144d20 0x7f4aa815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144d20 0x7f4aa815a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47054/0 (socket says 172.21.2.116:47054) 2021-11-21T21:56:41.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 -- 172.21.2.116:0/2957832326 learned_addr learned my addr 172.21.2.116:0/2957832326 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 -- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80abce0 msgr2=0x7f4aa8145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80abce0 0x7f4aa8145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 -- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4aa8144810 msgr2=0x7f4aa8145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4aa8144810 0x7f4aa8145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 -- 172.21.2.116:0/2957832326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c00b1e0 con 0x7f4aa8144d20 2021-11-21T21:56:41.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aae3a3700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144d20 0x7f4aa815a0d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f4aa0003330 tx=0x7f4aa00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4a9affd700 1 -- 172.21.2.116:0/2957832326 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4aa0001520 con 0x7f4aa8144d20 2021-11-21T21:56:41.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4a9affd700 1 -- 172.21.2.116:0/2957832326 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0001680 con 0x7f4aa8144d20 2021-11-21T21:56:41.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa81439b0 con 0x7f4aa8144d20 2021-11-21T21:56:41.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa80b49c0 con 0x7f4aa8144d20 2021-11-21T21:56:41.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4a9affd700 1 -- 172.21.2.116:0/2957832326 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4aa00019e0 con 0x7f4aa8144d20 2021-11-21T21:56:41.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a84003410 con 0x7f4aa8144d20 2021-11-21T21:56:41.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.541+0000 7f4a9affd700 1 -- 172.21.2.116:0/2957832326 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4aa0018030 con 0x7f4aa8144d20 2021-11-21T21:56:41.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.545+0000 7f4a9affd700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a88053720 0x7f4a88055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.545+0000 7f4a9affd700 1 -- 172.21.2.116:0/2957832326 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f4aa006b430 con 0x7f4aa8144d20 2021-11-21T21:56:41.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.545+0000 7f4aadba2700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a88053720 0x7f4a88055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.545+0000 7f4aadba2700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a88053720 0x7f4a88055be0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f4a9c002f70 tx=0x7f4a9c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.545+0000 7f4a9affd700 1 -- 172.21.2.116:0/2957832326 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4aa00424e0 con 0x7f4aa8144d20 2021-11-21T21:56:41.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.625+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4a84003c40 con 0x7f4aa8144d20 2021-11-21T21:56:41.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.625+0000 7f4a9affd700 1 -- 172.21.2.116:0/2957832326 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4aa0016020 con 0x7f4aa8144d20 2021-11-21T21:56:41.627 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:56:41.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a88053720 msgr2=0x7f4a88055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a88053720 0x7f4a88055be0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f4a9c002f70 tx=0x7f4a9c015000).stop 2021-11-21T21:56:41.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144d20 msgr2=0x7f4aa815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144d20 0x7f4aa815a0d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f4aa0003330 tx=0x7f4aa00090f0).stop 2021-11-21T21:56:41.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 shutdown_connections 2021-11-21T21:56:41.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4a88053720 0x7f4a88055be0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4aa8144d20 0x7f4aa815a0d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4aa8144810 0x7f4aa8145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 --2- 172.21.2.116:0/2957832326 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4aa80abce0 0x7f4aa8145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 >> 172.21.2.116:0/2957832326 conn(0x7f4aa805bd20 msgr2=0x7f4aa80b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 shutdown_connections 2021-11-21T21:56:41.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.629+0000 7f4aafe06700 1 -- 172.21.2.116:0/2957832326 wait complete. 2021-11-21T21:56:41.634 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:56:41.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.685+0000 7f5c17478700 1 Processor -- start 2021-11-21T21:56:41.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.685+0000 7f5c17478700 1 -- start start 2021-11-21T21:56:41.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5c100b1790 0x7f5c10063bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c10140930 con 0x7f5c10067890 2021-11-21T21:56:41.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c100b1080 con 0x7f5c100b0ca0 2021-11-21T21:56:41.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100b1c00 con 0x7f5c100b1790 2021-11-21T21:56:41.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c14a13700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5c10067890 0x7f5c100634b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59014/0 (socket says 172.21.2.116:59014) 2021-11-21T21:56:41.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c14a13700 1 -- 172.21.2.116:0/1168792931 learned_addr learned my addr 172.21.2.116:0/1168792931 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:41.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895237057 0 0) 0x7f5c10140930 con 0x7f5c10067890 2021-11-21T21:56:41.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf8003260 con 0x7f5c10067890 2021-11-21T21:56:41.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 --2- 172.21.2.116:0/1168792931 >> v2:172.21.2.117:3300/0 conn(0x7f5c100b1790 0x7f5c10063bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004610257 0 0) 0x7f5bf8003260 con 0x7f5c10067890 2021-11-21T21:56:41.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 >> v2:172.21.2.117:3300/0 conn(0x7f5c100b1790 msgr2=0x7f5c10063bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 --2- 172.21.2.116:0/1168792931 >> v2:172.21.2.117:3300/0 conn(0x7f5c100b1790 0x7f5c10063bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 >> v1:172.21.2.117:6789/0 conn(0x7f5c100b0ca0 legacy=0x7f5c10140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100640b0 con 0x7f5c10067890 2021-11-21T21:56:41.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f5c080019e0 con 0x7f5c10067890 2021-11-21T21:56:41.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 >> v1:172.21.2.116:6789/0 conn(0x7f5c10067890 legacy=0x7f5c100634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5bf8003820 0x7f5bf8003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5bf8004500 0x7f5bf8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5bf8004f90 0x7f5bf800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100b1c00 con 0x7f5bf8004f90 2021-11-21T21:56:41.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf800be80 con 0x7f5bf8003820 2021-11-21T21:56:41.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf800c040 con 0x7f5bf8004500 2021-11-21T21:56:41.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5bf8004f90 0x7f5bf800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 -- 172.21.2.116:0/1168792931 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5bf8004500 msgr2=0x7f5bf8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5bf8004500 0x7f5bf8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 -- 172.21.2.116:0/1168792931 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5bf8003820 msgr2=0x7f5bf8003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5bf8003820 0x7f5bf8003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 -- 172.21.2.116:0/1168792931 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf8003260 con 0x7f5bf8004f90 2021-11-21T21:56:41.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 -- 172.21.2.116:0/1168792931 reap_dead start 2021-11-21T21:56:41.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c15a15700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5bf8004f90 0x7f5bf800b750 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5c0c00a000 tx=0x7f5c0c008040).ready entity=mon.0 client_cookie=e7ba1d8e38eff4e0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0c013620 con 0x7f5bf8004f90 2021-11-21T21:56:41.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c013780 con 0x7f5bf8004f90 2021-11-21T21:56:41.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5bfffff700 1 -- 172.21.2.116:0/1168792931 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0c013ae0 con 0x7f5bf8004f90 2021-11-21T21:56:41.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- 172.21.2.116:0/1168792931 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5bf8004f90 msgr2=0x7f5bf800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5bf8004f90 0x7f5bf800b750 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5c0c00a000 tx=0x7f5c0c008040).stop 2021-11-21T21:56:41.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- 172.21.2.116:0/1168792931 shutdown_connections 2021-11-21T21:56:41.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 --2- 172.21.2.116:0/1168792931 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5bf8004f90 0x7f5bf800b750 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- 172.21.2.116:0/1168792931 >> 172.21.2.116:0/1168792931 conn(0x7f5c1005bd20 msgr2=0x7f5c1005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- 172.21.2.116:0/1168792931 shutdown_connections 2021-11-21T21:56:41.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- 172.21.2.116:0/1168792931 wait complete. 2021-11-21T21:56:41.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 Processor -- start 2021-11-21T21:56:41.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 -- start start 2021-11-21T21:56:41.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5bf8004f90 0x7f5c10141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c10139920 0x7f5c1013a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.689+0000 7f5c17478700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c100b2ab0 0x7f5c1012b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c17478700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10141410 con 0x7f5c100b2ab0 2021-11-21T21:56:41.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c17478700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100b2540 con 0x7f5c10139920 2021-11-21T21:56:41.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c17478700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100ad2c0 con 0x7f5bf8004f90 2021-11-21T21:56:41.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c100b2ab0 0x7f5c1012b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c100b2ab0 0x7f5c1012b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47072/0 (socket says 172.21.2.116:47072) 2021-11-21T21:56:41.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 -- 172.21.2.116:0/826057625 learned_addr learned my addr 172.21.2.116:0/826057625 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c14a13700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c10139920 0x7f5c1013a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 -- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5bf8004f90 msgr2=0x7f5c10141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5bf8004f90 0x7f5c10141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 -- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c10139920 msgr2=0x7f5c1013a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c10139920 0x7f5c1013a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 -- 172.21.2.116:0/826057625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10141640 con 0x7f5c100b2ab0 2021-11-21T21:56:41.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c14a13700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c10139920 0x7f5c1013a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:56:41.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15a15700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c100b2ab0 0x7f5c1012b410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f5c0c00a000 tx=0x7f5c0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5bfe7fc700 1 -- 172.21.2.116:0/826057625 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0c012900 con 0x7f5c100b2ab0 2021-11-21T21:56:41.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5bfe7fc700 1 -- 172.21.2.116:0/826057625 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c000e90 con 0x7f5c100b2ab0 2021-11-21T21:56:41.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c100b4950 con 0x7f5c100b2ab0 2021-11-21T21:56:41.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5bfe7fc700 1 -- 172.21.2.116:0/826057625 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0c012c50 con 0x7f5c100b2ab0 2021-11-21T21:56:41.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1013a800 con 0x7f5c100b2ab0 2021-11-21T21:56:41.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c10142900 con 0x7f5c100b2ab0 2021-11-21T21:56:41.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5bfe7fc700 1 -- 172.21.2.116:0/826057625 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5c0c01b040 con 0x7f5c100b2ab0 2021-11-21T21:56:41.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5bfe7fc700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5bf00536f0 0x7f5bf0055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5bfe7fc700 1 -- 172.21.2.116:0/826057625 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f5c0c06c440 con 0x7f5c100b2ab0 2021-11-21T21:56:41.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15214700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5bf00536f0 0x7f5bf0055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.693+0000 7f5c15214700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5bf00536f0 0x7f5bf0055bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5c00001fa0 tx=0x7f5c000077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.697+0000 7f5bfe7fc700 1 -- 172.21.2.116:0/826057625 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c0c043440 con 0x7f5c100b2ab0 2021-11-21T21:56:41.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c1013f7b0 con 0x7f5c100b2ab0 2021-11-21T21:56:41.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5bfe7fc700 1 -- 172.21.2.116:0/826057625 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c0c046a60 con 0x7f5c100b2ab0 2021-11-21T21:56:41.781 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:56:41.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5bf00536f0 msgr2=0x7f5bf0055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5bf00536f0 0x7f5bf0055bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5c00001fa0 tx=0x7f5c000077f0).stop 2021-11-21T21:56:41.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c100b2ab0 msgr2=0x7f5c1012b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c100b2ab0 0x7f5c1012b410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f5c0c00a000 tx=0x7f5c0c016000).stop 2021-11-21T21:56:41.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 shutdown_connections 2021-11-21T21:56:41.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5bf00536f0 0x7f5bf0055bb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c100b2ab0 0x7f5c1012b410 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c10139920 0x7f5c1013a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 --2- 172.21.2.116:0/826057625 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5bf8004f90 0x7f5c10141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 >> 172.21.2.116:0/826057625 conn(0x7f5c1005bd20 msgr2=0x7f5c100ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 shutdown_connections 2021-11-21T21:56:41.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.781+0000 7f5c17478700 1 -- 172.21.2.116:0/826057625 wait complete. 2021-11-21T21:56:41.788 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:56:41.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.841+0000 7f12f6151700 1 Processor -- start 2021-11-21T21:56:41.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.841+0000 7f12f6151700 1 -- start start 2021-11-21T21:56:41.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f12f00b0ca0 0x7f12f00b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12f00ac5a0 0x7f12f00634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f0140930 con 0x7f12f0067890 2021-11-21T21:56:41.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f00b1080 con 0x7f12f00ac5a0 2021-11-21T21:56:41.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f00acd20 con 0x7f12f00b0ca0 2021-11-21T21:56:41.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12f00ac5a0 0x7f12f00634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12f00ac5a0 0x7f12f00634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47082/0 (socket says 172.21.2.116:47082) 2021-11-21T21:56:41.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 -- 172.21.2.116:0/706609881 learned_addr learned my addr 172.21.2.116:0/706609881 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 -- 172.21.2.116:0/706609881 >> v2:172.21.2.117:3300/0 conn(0x7f12f00b0ca0 msgr2=0x7f12f00b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 --2- 172.21.2.116:0/706609881 >> v2:172.21.2.117:3300/0 conn(0x7f12f00b0ca0 0x7f12f00b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 -- 172.21.2.116:0/706609881 >> v1:172.21.2.116:6789/0 conn(0x7f12f0067890 legacy=0x7f12f0140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 -- 172.21.2.116:0/706609881 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f00acfc0 con 0x7f12f00ac5a0 2021-11-21T21:56:41.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12effff700 1 --2- 172.21.2.116:0/706609881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f00634b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f12e80032f0 tx=0x7f12e800b530).ready entity=mon.0 client_cookie=2fcd28e9cd3e45f9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12ee7fc700 1 -- 172.21.2.116:0/706609881 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e800be70 con 0x7f12f00ac5a0 2021-11-21T21:56:41.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12ee7fc700 1 -- 172.21.2.116:0/706609881 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e8010470 con 0x7f12f00ac5a0 2021-11-21T21:56:41.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12ee7fc700 1 -- 172.21.2.116:0/706609881 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e80107d0 con 0x7f12f00ac5a0 2021-11-21T21:56:41.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- 172.21.2.116:0/706609881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 msgr2=0x7f12f00634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- 172.21.2.116:0/706609881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f00634b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f12e80032f0 tx=0x7f12e800b530).stop 2021-11-21T21:56:41.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- 172.21.2.116:0/706609881 shutdown_connections 2021-11-21T21:56:41.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- 172.21.2.116:0/706609881 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f00634b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- 172.21.2.116:0/706609881 >> v2:172.21.2.117:3300/0 conn(0x7f12f00b0ca0 0x7f12f00b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- 172.21.2.116:0/706609881 >> 172.21.2.116:0/706609881 conn(0x7f12f005bd20 msgr2=0x7f12f005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- 172.21.2.116:0/706609881 shutdown_connections 2021-11-21T21:56:41.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- 172.21.2.116:0/706609881 wait complete. 2021-11-21T21:56:41.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 Processor -- start 2021-11-21T21:56:41.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- start start 2021-11-21T21:56:41.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12f0067890 0x7f12f00b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f0137760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12f00b0ca0 0x7f12f0137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f00668a0 con 0x7f12f00ac5a0 2021-11-21T21:56:41.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0065ec0 con 0x7f12f00b0ca0 2021-11-21T21:56:41.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12f6151700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0066160 con 0x7f12f0067890 2021-11-21T21:56:41.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f0137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f0137760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47088/0 (socket says 172.21.2.116:47088) 2021-11-21T21:56:41.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 -- 172.21.2.116:0/2669635421 learned_addr learned my addr 172.21.2.116:0/2669635421 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 -- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12f0067890 msgr2=0x7f12f00b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12f0067890 0x7f12f00b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 -- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12f00b0ca0 msgr2=0x7f12f0137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12f00b0ca0 0x7f12f0137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 -- 172.21.2.116:0/2669635421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e800b7d0 con 0x7f12f00ac5a0 2021-11-21T21:56:41.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12eeffd700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f0137760 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f12f0062f30 tx=0x7f12d8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12ecff9700 1 -- 172.21.2.116:0/2669635421 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d800ce30 con 0x7f12f00ac5a0 2021-11-21T21:56:41.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.845+0000 7f12ecff9700 1 -- 172.21.2.116:0/2669635421 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d800f470 con 0x7f12f00ac5a0 2021-11-21T21:56:41.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f0066400 con 0x7f12f00ac5a0 2021-11-21T21:56:41.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12ecff9700 1 -- 172.21.2.116:0/2669635421 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d800f7d0 con 0x7f12f00ac5a0 2021-11-21T21:56:41.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f013c4d0 con 0x7f12f00ac5a0 2021-11-21T21:56:41.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f0066590 con 0x7f12f00ac5a0 2021-11-21T21:56:41.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12ecff9700 1 -- 172.21.2.116:0/2669635421 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f12d8019030 con 0x7f12f00ac5a0 2021-11-21T21:56:41.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12ecff9700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12d4053640 0x7f12d4055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12ecff9700 1 -- 172.21.2.116:0/2669635421 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f12d806ae10 con 0x7f12f00ac5a0 2021-11-21T21:56:41.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12ef7fe700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12d4053640 0x7f12d4055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12ef7fe700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12d4053640 0x7f12d4055b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f12e0002f70 tx=0x7f12e00069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:41.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.849+0000 7f12ecff9700 1 -- 172.21.2.116:0/2669635421 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12d8041f20 con 0x7f12f00ac5a0 2021-11-21T21:56:41.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12f00573d0 con 0x7f12f00ac5a0 2021-11-21T21:56:41.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12ecff9700 1 -- 172.21.2.116:0/2669635421 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7f12d8045540 con 0x7f12f00ac5a0 2021-11-21T21:56:41.933 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:56:41.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12d4053640 msgr2=0x7f12d4055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12d4053640 0x7f12d4055b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f12e0002f70 tx=0x7f12e00069e0).stop 2021-11-21T21:56:41.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 msgr2=0x7f12f0137760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:41.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f0137760 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f12f0062f30 tx=0x7f12d8007c50).stop 2021-11-21T21:56:41.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 shutdown_connections 2021-11-21T21:56:41.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12f00b0ca0 0x7f12f0137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12d4053640 0x7f12d4055b00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12f00ac5a0 0x7f12f0137760 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 --2- 172.21.2.116:0/2669635421 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12f0067890 0x7f12f00b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 >> 172.21.2.116:0/2669635421 conn(0x7f12f005bd20 msgr2=0x7f12f013d870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 shutdown_connections 2021-11-21T21:56:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.933+0000 7f12f6151700 1 -- 172.21.2.116:0/2669635421 wait complete. 2021-11-21T21:56:41.940 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:56:41.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 Processor -- start 2021-11-21T21:56:41.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- start start 2021-11-21T21:56:41.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff80abce0 0x7f7ff81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7ff80ac600 0x7f7ff80acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:41.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff8147e80 con 0x7f7ff8144810 2021-11-21T21:56:41.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80b1490 con 0x7f7ff80ac600 2021-11-21T21:56:41.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80b1fd0 con 0x7f7ff80abce0 2021-11-21T21:56:41.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7ff80ac600 0x7f7ff80acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:41.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7ff80ac600 0x7f7ff80acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47092/0 (socket says 172.21.2.116:47092) 2021-11-21T21:56:41.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 -- 172.21.2.116:0/3501797409 learned_addr learned my addr 172.21.2.116:0/3501797409 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:41.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 -- 172.21.2.116:0/3501797409 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff80abce0 msgr2=0x7f7ff81478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 --2- 172.21.2.116:0/3501797409 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff80abce0 0x7f7ff81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:41.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 -- 172.21.2.116:0/3501797409 >> v1:172.21.2.117:6789/0 conn(0x7f7ff8144810 legacy=0x7f7ff80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:41.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 -- 172.21.2.116:0/3501797409 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80af5a0 con 0x7f7ff80ac600 2021-11-21T21:56:42.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff77fe700 1 --2- 172.21.2.116:0/3501797409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80ac600 0x7f7ff80acab0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f7ff80b2a30 tx=0x7f7ff0009290).ready entity=mon.0 client_cookie=d47506f6cc67d2d1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:42.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff5ffb700 1 -- 172.21.2.116:0/3501797409 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff000b860 con 0x7f7ff80ac600 2021-11-21T21:56:42.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff5ffb700 1 -- 172.21.2.116:0/3501797409 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff000b9c0 con 0x7f7ff80ac600 2021-11-21T21:56:42.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ff5ffb700 1 -- 172.21.2.116:0/3501797409 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff000bb70 con 0x7f7ff80ac600 2021-11-21T21:56:42.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/3501797409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80ac600 msgr2=0x7f7ff80acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/3501797409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80ac600 0x7f7ff80acab0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f7ff80b2a30 tx=0x7f7ff0009290).stop 2021-11-21T21:56:42.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/3501797409 shutdown_connections 2021-11-21T21:56:42.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/3501797409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80ac600 0x7f7ff80acab0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:42.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/3501797409 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff80abce0 0x7f7ff81478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/3501797409 >> 172.21.2.116:0/3501797409 conn(0x7f7ff805bd20 msgr2=0x7f7ff805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:42.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/3501797409 shutdown_connections 2021-11-21T21:56:42.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.993+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/3501797409 wait complete. 2021-11-21T21:56:42.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 Processor -- start 2021-11-21T21:56:42.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 -- start start 2021-11-21T21:56:42.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80abce0 0x7f7ff8156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ff80ac600 0x7f7ff81572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff8144810 0x7f7ff80b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8145b50 con 0x7f7ff80abce0 2021-11-21T21:56:42.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff81471c0 con 0x7f7ff80ac600 2021-11-21T21:56:42.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8147460 con 0x7f7ff8144810 2021-11-21T21:56:42.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80abce0 0x7f7ff8156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:42.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80abce0 0x7f7ff8156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47098/0 (socket says 172.21.2.116:47098) 2021-11-21T21:56:42.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 -- 172.21.2.116:0/379707748 learned_addr learned my addr 172.21.2.116:0/379707748 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:42.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 -- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff8144810 msgr2=0x7f7ff80b2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:42.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff8144810 0x7f7ff80b2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 -- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ff80ac600 msgr2=0x7f7ff81572d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:56:42.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ff80ac600 0x7f7ff81572d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 -- 172.21.2.116:0/379707748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0009530 con 0x7f7ff80abce0 2021-11-21T21:56:42.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff6ffd700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80abce0 0x7f7ff8156ba0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f7fe8002f10 tx=0x7f7fe8006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:42.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7feffff700 1 -- 172.21.2.116:0/379707748 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe800f510 con 0x7f7ff80abce0 2021-11-21T21:56:42.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7feffff700 1 -- 172.21.2.116:0/379707748 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe800f670 con 0x7f7ff80abce0 2021-11-21T21:56:42.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff8145840 con 0x7f7ff80abce0 2021-11-21T21:56:42.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7feffff700 1 -- 172.21.2.116:0/379707748 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe800f7e0 con 0x7f7ff80abce0 2021-11-21T21:56:42.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff8144f40 con 0x7f7ff80abce0 2021-11-21T21:56:42.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff8145d50 con 0x7f7ff80abce0 2021-11-21T21:56:42.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7feffff700 1 -- 172.21.2.116:0/379707748 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7fe8017020 con 0x7f7ff80abce0 2021-11-21T21:56:42.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7feffff700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fdc0534f0 0x7f7fdc0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7feffff700 1 -- 172.21.2.116:0/379707748 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f7fe806af40 con 0x7f7ff80abce0 2021-11-21T21:56:42.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff67fc700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fdc0534f0 0x7f7fdc0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:42.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:41.997+0000 7f7ff67fc700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fdc0534f0 0x7f7fdc0559b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f7fe0002f70 tx=0x7f7fe0006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:42.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.001+0000 7f7feffff700 1 -- 172.21.2.116:0/379707748 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7fe8042050 con 0x7f7ff80abce0 2021-11-21T21:56:42.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ff8144f40 con 0x7f7ff80abce0 2021-11-21T21:56:42.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7feffff700 1 -- 172.21.2.116:0/379707748 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7ff8144f40 con 0x7f7ff80abce0 2021-11-21T21:56:42.084 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:56:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fdc0534f0 msgr2=0x7f7fdc0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fdc0534f0 0x7f7fdc0559b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f7fe0002f70 tx=0x7f7fe0006a00).stop 2021-11-21T21:56:42.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80abce0 msgr2=0x7f7ff8156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80abce0 0x7f7ff8156ba0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f7fe8002f10 tx=0x7f7fe8006d70).stop 2021-11-21T21:56:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 shutdown_connections 2021-11-21T21:56:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ff8144810 0x7f7ff80b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ff80ac600 0x7f7ff81572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7fdc0534f0 0x7f7fdc0559b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 --2- 172.21.2.116:0/379707748 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7ff80abce0 0x7f7ff8156ba0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 >> 172.21.2.116:0/379707748 conn(0x7f7ff805bd20 msgr2=0x7f7ff805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:42.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 shutdown_connections 2021-11-21T21:56:42.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.085+0000 7f7ffd7a6700 1 -- 172.21.2.116:0/379707748 wait complete. 2021-11-21T21:56:42.091 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:56:42.092 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:56:42.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.141+0000 7f25dae58700 1 Processor -- start 2021-11-21T21:56:42.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.141+0000 7f25dae58700 1 -- start start 2021-11-21T21:56:42.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d40a86b0 0x7f25d414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d40a6e80 con 0x7f25d40a6aa0 2021-11-21T21:56:42.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d414a7b0 con 0x7f25d4062410 2021-11-21T21:56:42.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40a8bc0 con 0x7f25d40a86b0 2021-11-21T21:56:42.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f25d40a6aa0 0x7f25d41470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59062/0 (socket says 172.21.2.116:59062) 2021-11-21T21:56:42.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2956990618 learned_addr learned my addr 172.21.2.116:0/2956990618 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:56:42.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515941721 0 0) 0x7f25d40a6e80 con 0x7f25d40a6aa0 2021-11-21T21:56:42.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b4003260 con 0x7f25d40a6aa0 2021-11-21T21:56:42.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d3fff700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d40a86b0 0x7f25d414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:42.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3278328783 0 0) 0x7f25b4003260 con 0x7f25d40a6aa0 2021-11-21T21:56:42.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d40a86b0 msgr2=0x7f25d414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:42.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d40a86b0 0x7f25d414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:42.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 >> v1:172.21.2.117:6789/0 conn(0x7f25d4062410 legacy=0x7f25d41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d414aa10 con 0x7f25d40a6aa0 2021-11-21T21:56:42.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f25c4001680 con 0x7f25d40a6aa0 2021-11-21T21:56:42.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 >> v1:172.21.2.116:6789/0 conn(0x7f25d40a6aa0 legacy=0x7f25d41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25b4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d40a86b0 0x7f25b4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25b40049f0 0x7f25b4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40a8bc0 con 0x7f25b4003780 2021-11-21T21:56:42.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b400b800 con 0x7f25b40049f0 2021-11-21T21:56:42.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b400b990 con 0x7f25d40a86b0 2021-11-21T21:56:42.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25b4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:42.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d40a86b0 msgr2=0x7f25b4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:42.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d40a86b0 0x7f25b4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25b40049f0 msgr2=0x7f25b4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:56:42.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25b40049f0 0x7f25b4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2956990618 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4003260 con 0x7f25b4003780 2021-11-21T21:56:42.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d8bf4700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25b4003b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f25d4061b50 tx=0x7f25c4001e10).ready entity=mon.0 client_cookie=1ab2c6b5335cae01 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:42.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25c4007170 con 0x7f25b4003780 2021-11-21T21:56:42.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c40072d0 con 0x7f25b4003780 2021-11-21T21:56:42.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d37fe700 1 -- 172.21.2.116:0/2956990618 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25c400c8f0 con 0x7f25b4003780 2021-11-21T21:56:42.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- 172.21.2.116:0/2956990618 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 msgr2=0x7f25b4003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 --2- 172.21.2.116:0/2956990618 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25b4003b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f25d4061b50 tx=0x7f25c4001e10).stop 2021-11-21T21:56:42.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25d93f5700 1 -- 172.21.2.116:0/2956990618 reap_dead start 2021-11-21T21:56:42.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- 172.21.2.116:0/2956990618 shutdown_connections 2021-11-21T21:56:42.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- 172.21.2.116:0/2956990618 >> 172.21.2.116:0/2956990618 conn(0x7f25d405bd20 msgr2=0x7f25d405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:42.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- 172.21.2.116:0/2956990618 shutdown_connections 2021-11-21T21:56:42.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- 172.21.2.116:0/2956990618 wait complete. 2021-11-21T21:56:42.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 Processor -- start 2021-11-21T21:56:42.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.145+0000 7f25dae58700 1 -- start start 2021-11-21T21:56:42.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25d40ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25d41435a0 0x7f25d4146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d4063850 0x7f25d40a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4156f20 con 0x7f25b4003780 2021-11-21T21:56:42.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40aaeb0 con 0x7f25d41435a0 2021-11-21T21:56:42.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41465f0 con 0x7f25d4063850 2021-11-21T21:56:42.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25d40ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:42.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25d40ab4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47118/0 (socket says 172.21.2.116:47118) 2021-11-21T21:56:42.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2855985492 learned_addr learned my addr 172.21.2.116:0/2855985492 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:56:42.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d4063850 msgr2=0x7f25d40a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d4063850 0x7f25d40a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25d41435a0 msgr2=0x7f25d4146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25d41435a0 0x7f25d4146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 -- 172.21.2.116:0/2855985492 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d40aac10 con 0x7f25b4003780 2021-11-21T21:56:42.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d8bf4700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25d40ab4d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f25d4061b50 tx=0x7f25c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:42.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d1ffb700 1 -- 172.21.2.116:0/2855985492 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25c4007170 con 0x7f25b4003780 2021-11-21T21:56:42.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d1ffb700 1 -- 172.21.2.116:0/2855985492 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c4004920 con 0x7f25b4003780 2021-11-21T21:56:42.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d1ffb700 1 -- 172.21.2.116:0/2855985492 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25c4004c80 con 0x7f25b4003780 2021-11-21T21:56:42.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d4157180 con 0x7f25b4003780 2021-11-21T21:56:42.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d40aa0d0 con 0x7f25b4003780 2021-11-21T21:56:42.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d4157310 con 0x7f25b4003780 2021-11-21T21:56:42.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d1ffb700 1 -- 172.21.2.116:0/2855985492 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f25c401d070 con 0x7f25b4003780 2021-11-21T21:56:42.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d1ffb700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25b80536d0 0x7f25b8055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:56:42.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d1ffb700 1 -- 172.21.2.116:0/2855985492 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f25c406c520 con 0x7f25b4003780 2021-11-21T21:56:42.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d3fff700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25b80536d0 0x7f25b8055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:56:42.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.149+0000 7f25d3fff700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25b80536d0 0x7f25b8055b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f25bc00a000 tx=0x7f25bc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:56:42.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.153+0000 7f25d1ffb700 1 -- 172.21.2.116:0/2855985492 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f25c4043670 con 0x7f25b4003780 2021-11-21T21:56:42.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25d40573d0 con 0x7f25b4003780 2021-11-21T21:56:42.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25d1ffb700 1 -- 172.21.2.116:0/2855985492 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f25c4046c50 con 0x7f25b4003780 2021-11-21T21:56:42.261 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:56:42.261 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:09.961260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:56:42.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25b80536d0 msgr2=0x7f25b8055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25b80536d0 0x7f25b8055b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f25bc00a000 tx=0x7f25bc008040).stop 2021-11-21T21:56:42.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 msgr2=0x7f25d40ab4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:56:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25d40ab4d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f25d4061b50 tx=0x7f25c4015000).stop 2021-11-21T21:56:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 shutdown_connections 2021-11-21T21:56:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25d4063850 0x7f25d40a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25d41435a0 0x7f25d4146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:56:42.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25b80536d0 0x7f25b8055b90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:42.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 --2- 172.21.2.116:0/2855985492 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25b4003780 0x7f25d40ab4d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:56:42.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 >> 172.21.2.116:0/2855985492 conn(0x7f25d405bd20 msgr2=0x7f25d4149020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:56:42.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 shutdown_connections 2021-11-21T21:56:42.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:56:42.261+0000 7f25dae58700 1 -- 172.21.2.116:0/2855985492 wait complete. 2021-11-21T21:56:42.265 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 17 2021-11-21T21:56:42.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:56:07.942419+0000', 'modified': '2021-11-21T21:56:09.961260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5247': {'gid': 5247, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2465449969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2465449969}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2465449969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:56:42.571 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:56:42.571 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:56:42.636 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5247,"want_state":"up:active","state":"up:active","rank_uptime":33.673966810000003,"mdsmap_epoch":17,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":38.4122451} 2021-11-21T21:56:42.637 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5247, "mdsmap_epoch": 17, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 33.67396681, "state": "up:active", "uptime": 38.4122451, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:56:42.637 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:56:42.638 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T21:56:42.638 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T21:56:42.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:42.640 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:42.651 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:56:42.702 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:56:42.703 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:56:42.709 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:56:42.709 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:56:42.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:42.711 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:56:42.715 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:56:42.716 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:42.762 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:56:42.771 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:56:42.771 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:56:42.776 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:56:42.777 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:56:42.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:42.778 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:56:42.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T21:56:42.801+0000 7fc2c5e0c080 -1 init, newargv = 0x5607c1764c10 newargc=9 2021-11-21T21:56:42.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14251]: starting ceph client 2021-11-21T21:56:42.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14251]: starting fuse 2021-11-21T21:56:43.883 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:56:43.896 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:56:43.896 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:56:43.947 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:56:43.948 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:56:43.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:43.949 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:56:43.955 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T21:56:43.956 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:56:43.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:43.957 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T21:56:43.957 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T21:56:43.957 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T21:56:43.958 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:56:43.958 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T21:56:43.958 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:43.959 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T21:56:43.959 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T21:56:43.960 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T21:56:43.960 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:43.960 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T21:56:43.961 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T21:56:43.961 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T21:56:43.962 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T21:56:43.962 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:43.962 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T21:56:43.963 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:56:43.964 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:56:43.964 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T21:56:43.965 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:56:43.965 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:56:43.966 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T21:56:43.967 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:56:44.035 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14251.asok 2021-11-21T21:56:44.037 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14251.asok 2021-11-21T21:56:44.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:44.038 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14251.asok status 2021-11-21T21:56:44.086 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T21:56:44.086 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T21:56:44.087 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:56:44.087 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:56:44.087 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T21:56:44.087 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T21:56:44.088 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T21:56:44.088 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14251", 2021-11-21T21:56:44.088 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T21:56:44.088 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:44.089 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T21:56:44.089 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T21:56:44.089 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5481, 2021-11-21T21:56:44.089 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T21:56:44.090 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T21:56:44.090 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T21:56:44.090 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5481 2021-11-21T21:56:44.090 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:44.090 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:56:44.091 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:56:44.091 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:56:44.091 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2861923144 2021-11-21T21:56:44.091 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T21:56:44.092 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:44.092 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:56:44.092 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:56:44.092 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:56:44.092 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2861923144 2021-11-21T21:56:44.093 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:56:44.093 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5481 172.21.2.119:0/2861923144", 2021-11-21T21:56:44.093 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/2861923144", 2021-11-21T21:56:44.093 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T21:56:44.093 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 17, 2021-11-21T21:56:44.094 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 44, 2021-11-21T21:56:44.094 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:56:44.094 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T21:56:44.095 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T21:56:44.095 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T21:56:44.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:44.097 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:56:44.097 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:56:44.103 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:56:44.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:56:44.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:56:44.104 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:03.864 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:03.868 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:03.871 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:03.875 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:33.933 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:33.937 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:33.941 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:33.945 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:57:34.229 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:57:34.230 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:34.243 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:34.257 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:34.286 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:34.315 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:57:34.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:34.316 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:57:34.316 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:34.328 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:57:34.329 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:57:34.329 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:57:34.330 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:57:34.330 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:34.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14251]: fuse finished with error 0 and tester_r 0 2021-11-21T21:57:34.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:34.387 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:57:34.387 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:34.392 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T21:57:34.392 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:57:34.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:57:34.503 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:34.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:34.549 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:57:34.549 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:57:34.559 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:57:34.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:57:34.561 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:57:34.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:57:34.562 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:57:34.571 INFO:teuthology.orchestra.run.gibba036.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:57:34.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:57:34.573 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:57:34.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 Processor -- start 2021-11-21T21:57:34.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- start start 2021-11-21T21:57:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f75000b5630 0x7f750014af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750014a560 con 0x7f7500099650 2021-11-21T21:57:34.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750014a800 con 0x7f7500097ec0 2021-11-21T21:57:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7500144b40 con 0x7f75000b5630 2021-11-21T21:57:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f7504bf1700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7500099650 0x7f7500099ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59086/0 (socket says 172.21.2.116:59086) 2021-11-21T21:57:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f7504bf1700 1 -- 172.21.2.116:0/1250605513 learned_addr learned my addr 172.21.2.116:0/1250605513 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524617219 0 0) 0x7f750014a560 con 0x7f7500099650 2021-11-21T21:57:34.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec003260 con 0x7f7500099650 2021-11-21T21:57:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 --2- 172.21.2.116:0/1250605513 >> v2:172.21.2.117:3300/0 conn(0x7f75000b5630 0x7f750014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3193807196 0 0) 0x7f74ec003260 con 0x7f7500099650 2021-11-21T21:57:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 >> v2:172.21.2.117:3300/0 conn(0x7f75000b5630 msgr2=0x7f750014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:34.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 --2- 172.21.2.116:0/1250605513 >> v2:172.21.2.117:3300/0 conn(0x7f75000b5630 0x7f750014af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 >> v1:172.21.2.117:6789/0 conn(0x7f7500097ec0 legacy=0x7f7500146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750014aaa0 con 0x7f7500099650 2021-11-21T21:57:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f74f80019e0 con 0x7f7500099650 2021-11-21T21:57:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 >> v1:172.21.2.116:6789/0 conn(0x7f7500099650 legacy=0x7f7500099ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:34.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f74ec003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74ec004500 0x7f74ec004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74ec004f90 0x7f74ec00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500144b40 con 0x7f74ec003820 2021-11-21T21:57:34.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec00be80 con 0x7f74ec004500 2021-11-21T21:57:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec00c040 con 0x7f74ec004f90 2021-11-21T21:57:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f74ec003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 -- 172.21.2.116:0/1250605513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74ec004f90 msgr2=0x7f74ec00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74ec004f90 0x7f74ec00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:34.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 -- 172.21.2.116:0/1250605513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74ec004500 msgr2=0x7f74ec004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74ec004500 0x7f74ec004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f7504bf1700 1 -- 172.21.2.116:0/1250605513 reap_dead start 2021-11-21T21:57:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 -- 172.21.2.116:0/1250605513 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec003260 con 0x7f74ec003820 2021-11-21T21:57:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74fffff700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f74ec003c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f74f000a000 tx=0x7f74f0008040).ready entity=mon.0 client_cookie=785109e13f7d0358 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:34.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74f0013670 con 0x7f74ec003820 2021-11-21T21:57:34.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f00137d0 con 0x7f74ec003820 2021-11-21T21:57:34.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f74feffd700 1 -- 172.21.2.116:0/1250605513 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74f0013b30 con 0x7f74ec003820 2021-11-21T21:57:34.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- 172.21.2.116:0/1250605513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 msgr2=0x7f74ec003c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:34.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f74ec003c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f74f000a000 tx=0x7f74f0008040).stop 2021-11-21T21:57:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- 172.21.2.116:0/1250605513 shutdown_connections 2021-11-21T21:57:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 --2- 172.21.2.116:0/1250605513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f74ec003c40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- 172.21.2.116:0/1250605513 >> 172.21.2.116:0/1250605513 conn(0x7f750005c7f0 msgr2=0x7f75000b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- 172.21.2.116:0/1250605513 shutdown_connections 2021-11-21T21:57:34.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.628+0000 7f75053f2700 1 -- 172.21.2.116:0/1250605513 wait complete. 2021-11-21T21:57:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 Processor -- start 2021-11-21T21:57:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 -- start start 2021-11-21T21:57:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f750005fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75000631c0 0x7f7500143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7500143e10 0x7f75001598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500063860 con 0x7f74ec003820 2021-11-21T21:57:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500063620 con 0x7f7500143e10 2021-11-21T21:57:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7500063be0 con 0x7f75000631c0 2021-11-21T21:57:34.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f750005fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f750005fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47138/0 (socket says 172.21.2.116:47138) 2021-11-21T21:57:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 -- 172.21.2.116:0/167587853 learned_addr learned my addr 172.21.2.116:0/167587853 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 -- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75000631c0 msgr2=0x7f7500143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75000631c0 0x7f7500143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 -- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7500143e10 msgr2=0x7f75001598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:34.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7500143e10 0x7f75001598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:34.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 -- 172.21.2.116:0/167587853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500063e80 con 0x7f74ec003820 2021-11-21T21:57:34.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fffff700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f750005fa70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f74f000a000 tx=0x7f74f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:34.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fd7fa700 1 -- 172.21.2.116:0/167587853 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74f0012d40 con 0x7f74ec003820 2021-11-21T21:57:34.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fd7fa700 1 -- 172.21.2.116:0/167587853 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f0000e90 con 0x7f74ec003820 2021-11-21T21:57:34.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7500067920 con 0x7f74ec003820 2021-11-21T21:57:34.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fd7fa700 1 -- 172.21.2.116:0/167587853 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74f0012ea0 con 0x7f74ec003820 2021-11-21T21:57:34.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7500064890 con 0x7f74ec003820 2021-11-21T21:57:34.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7500063860 con 0x7f74ec003820 2021-11-21T21:57:34.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fd7fa700 1 -- 172.21.2.116:0/167587853 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f74f001d600 con 0x7f74ec003820 2021-11-21T21:57:34.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fd7fa700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74e0053690 0x7f74e0055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:34.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74fd7fa700 1 -- 172.21.2.116:0/167587853 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f74f006d130 con 0x7f74ec003820 2021-11-21T21:57:34.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74ff7fe700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74e0053690 0x7f74e0055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:34.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.632+0000 7f74ff7fe700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74e0053690 0x7f74e0055b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f74f4001fa0 tx=0x7f74f40077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:34.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.636+0000 7f74fd7fa700 1 -- 172.21.2.116:0/167587853 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74f0044260 con 0x7f74ec003820 2021-11-21T21:57:34.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7500159730 con 0x7f74ec003820 2021-11-21T21:57:34.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f74fd7fa700 1 -- 172.21.2.116:0/167587853 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v44) v1 ==== 81+0+33799 (secure 0 0 0) 0x7f74f000b120 con 0x7f74ec003820 2021-11-21T21:57:34.719 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:34.719 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:57:34.719 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 44, 2021-11-21T21:57:34.720 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:57:34.720 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:57:34.720 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:56:09.235532+0000", 2021-11-21T21:57:34.720 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:57:34.721 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:57:34.721 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:57:34.721 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:57:34.721 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:57:34.722 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:57:34.722 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:57:34.722 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:57:34.722 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:57:34.722 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:34.723 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:57:34.723 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:57:34.723 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:57:34.723 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:57:34.724 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:57:34.724 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 8, 2021-11-21T21:57:34.724 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:57:34.724 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:57:34.724 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:57:34.725 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:57:34.725 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:57:34.725 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.725 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:57:34.726 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:57:34.726 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:57:34.726 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:57:34.726 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:57:34.726 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:57:34.727 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:57:34.727 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:57:34.727 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:57:34.727 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:57:34.728 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:57:34.728 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:57:34.728 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:57:34.728 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:57:34.728 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:57:34.729 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:57:34.729 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:57:34.729 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:57:34.729 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:57:34.730 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:57:34.730 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:57:34.730 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:57:34.730 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:57:34.730 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.731 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:57:34.731 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:57:34.731 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:57:34.731 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:57:34.731 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:57:34.732 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:57:34.732 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:57:34.732 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:57:34.732 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:57:34.733 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:57:34.733 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:57:34.733 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:57:34.733 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:57:34.733 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:57:34.734 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:57:34.734 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:57:34.734 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:57:34.734 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:57:34.735 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:57:34.735 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:57:34.735 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:57:34.735 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:57:34.735 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:57:34.736 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:57:34.736 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:57:34.736 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:57:34.736 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:57:34.737 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.737 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:57:34.737 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:57:34.737 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:57:34.737 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:57:34.738 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:57:34.738 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:57:34.738 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:57:34.738 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:57:34.738 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:57:34.739 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:57:34.739 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:57:34.739 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:57:34.739 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:57:34.740 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.740 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:57:34.740 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:57:34.740 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.740 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.741 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.741 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:57:34.741 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:57:34.741 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:57:34.742 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:57:34.742 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:57:34.742 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:57:34.742 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:57:34.742 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:57:34.743 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:57:34.743 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:57:34.743 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:57:34.743 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:57:34.744 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:57:34.744 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:57:34.744 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:57:34.744 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:57:34.744 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:57:34.745 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:57:34.745 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:57:34.745 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:57:34.745 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:57:34.745 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:57:34.746 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:57:34.746 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.746 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:57:34.746 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:57:34.747 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:57:34.747 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:57:34.747 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:57:34.747 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:57:34.747 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:57:34.748 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:57:34.748 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:57:34.748 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:57:34.748 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:57:34.749 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:57:34.749 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:57:34.749 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:57:34.749 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:57:34.749 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:57:34.750 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:57:34.750 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:57:34.750 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:57:34.750 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:57:34.751 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:57:34.751 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:57:34.751 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:57:34.751 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:57:34.751 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:57:34.752 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:57:34.752 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:57:34.752 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.752 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:57:34.752 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:57:34.753 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:57:34.753 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:57:34.753 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:57:34.753 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:57:34.754 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:57:34.754 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:57:34.754 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:57:34.754 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:57:34.754 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:57:34.755 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:57:34.755 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:57:34.755 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:57:34.755 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.756 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.756 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.756 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 7, 2021-11-21T21:57:34.756 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T21:57:34.756 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:56:06.102882+0000", 2021-11-21T21:57:34.757 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:57:34.757 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:57:34.757 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:57:34.757 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:57:34.757 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:57:34.758 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:57:34.758 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:57:34.758 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:57:34.758 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:57:34.759 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:57:34.759 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:57:34.759 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:57:34.759 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:57:34.759 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:57:34.760 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:57:34.760 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:57:34.760 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:57:34.760 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:57:34.761 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:57:34.761 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:57:34.761 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.761 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "44", 2021-11-21T21:57:34.761 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:57:34.762 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:57:34.762 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:57:34.762 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:57:34.762 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:57:34.762 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:57:34.763 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:57:34.763 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:57:34.763 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:57:34.763 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:57:34.764 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:57:34.764 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:57:34.764 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:57:34.764 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:57:34.764 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:57:34.765 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:57:34.765 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:57:34.765 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:57:34.765 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:57:34.766 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:57:34.766 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:57:34.766 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:57:34.766 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:57:34.766 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:57:34.767 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:57:34.767 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:57:34.767 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.767 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:57:34.768 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:57:34.768 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:57:34.768 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:57:34.768 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:57:34.768 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:57:34.769 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:57:34.769 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:57:34.769 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:57:34.769 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:57:34.769 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:57:34.770 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:57:34.770 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T21:57:34.770 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T21:57:34.770 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T21:57:34.771 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.771 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:57:34.771 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:57:34.771 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T21:57:34.771 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.772 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.772 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.772 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.772 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 8, 2021-11-21T21:57:34.773 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T21:57:34.773 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:56:06.914261+0000", 2021-11-21T21:57:34.773 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:57:34.773 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:57:34.773 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:57:34.774 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:57:34.774 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:57:34.774 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:57:34.774 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:57:34.775 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:57:34.775 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:57:34.775 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:57:34.775 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:57:34.775 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:57:34.776 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:57:34.776 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:57:34.776 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:57:34.776 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:57:34.777 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:57:34.777 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:57:34.777 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:57:34.777 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:57:34.777 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.778 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "44", 2021-11-21T21:57:34.778 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:57:34.778 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:57:34.778 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:57:34.778 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:57:34.779 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:57:34.779 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:57:34.779 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:57:34.779 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:57:34.780 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:57:34.780 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:57:34.780 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:57:34.780 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:57:34.781 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:57:34.781 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:57:34.781 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:57:34.781 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:57:34.781 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:57:34.782 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:57:34.782 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:57:34.782 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:57:34.782 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:57:34.783 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:57:34.783 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:57:34.783 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:57:34.783 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:57:34.783 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:57:34.784 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.784 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:57:34.784 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:57:34.784 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:57:34.785 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:57:34.785 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:57:34.785 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:57:34.785 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:57:34.785 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:57:34.786 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:57:34.786 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:57:34.786 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:57:34.786 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:57:34.786 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:57:34.787 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:57:34.787 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T21:57:34.787 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.787 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.788 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.788 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:34.788 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:57:34.788 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.788 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:57:34.789 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:57:34.789 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.789 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.789 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.790 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.790 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.790 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.790 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.790 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:34.791 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.791 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.791 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.791 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.791 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.792 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.792 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:57:34.792 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.792 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.793 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.793 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.793 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:57:34.793 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.793 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.794 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.794 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.794 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.794 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.794 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.795 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.795 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:57:34.795 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.795 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.796 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.796 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.796 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:57:34.796 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.796 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.797 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.797 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.797 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.797 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.797 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.798 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.798 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:57:34.798 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.798 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.799 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.799 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.799 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:57:34.799 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.799 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.800 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.800 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.800 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.800 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.801 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.801 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.801 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:57:34.801 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.801 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.802 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.802 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.802 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:57:34.802 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:34.802 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.803 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.803 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.803 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:57:34.803 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:57:34.804 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:57:34.804 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:57:34.804 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.804 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.804 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.805 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.805 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.805 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.805 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:57:34.805 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:57:34.806 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.806 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.806 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.806 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.807 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.807 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.807 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.807 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 41, 2021-11-21T21:57:34.807 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.808 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.808 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.808 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.808 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.808 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.809 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:57:34.809 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.809 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.809 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.810 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.810 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:57:34.810 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.810 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.810 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.811 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.811 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.811 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.811 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.811 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.812 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:57:34.812 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.812 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.812 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.813 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.813 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:57:34.813 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.813 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.813 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.814 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.814 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.814 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.814 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.815 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.815 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:57:34.815 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.815 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.815 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.816 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.816 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:57:34.816 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.816 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.816 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.817 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.817 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.817 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.817 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.818 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.818 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:57:34.818 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.818 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.818 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.819 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.819 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:57:34.819 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:34.819 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.819 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.820 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.820 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:57:34.820 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:57:34.820 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:57:34.821 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:57:34.821 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.821 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.821 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.821 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.822 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.822 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.822 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:57:34.822 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:57:34.822 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.823 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.823 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.823 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.823 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.824 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.824 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.824 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 27, 2021-11-21T21:57:34.824 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.824 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.825 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.825 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.825 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.825 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.825 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:57:34.826 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.826 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.826 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.826 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.827 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:57:34.827 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.827 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.827 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.827 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.828 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.828 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.828 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.828 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.828 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:57:34.829 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.829 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.829 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.829 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.830 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:57:34.830 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.830 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.830 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.830 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.831 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.831 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.831 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.831 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.832 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:57:34.832 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.832 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.832 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.832 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.833 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:57:34.833 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.833 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.833 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.833 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.834 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.834 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.834 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.834 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.835 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:57:34.835 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.835 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.835 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.835 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.836 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:57:34.836 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:34.836 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.836 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.836 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.837 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:57:34.837 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:57:34.837 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:57:34.837 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:57:34.838 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.838 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.838 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.838 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.838 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.839 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.839 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:57:34.839 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:57:34.839 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.840 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.840 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.840 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.840 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.840 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.841 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.841 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:34.841 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.841 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.841 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.842 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.842 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.842 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.842 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:57:34.843 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.843 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.843 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.843 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.843 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:57:34.844 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.844 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.844 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.844 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.845 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.845 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.845 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.845 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.845 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:57:34.846 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.846 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.846 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.846 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.846 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:57:34.847 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.847 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.847 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.847 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.848 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.848 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.848 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.848 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.848 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:57:34.849 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.849 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.849 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.849 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.849 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:57:34.850 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.850 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.850 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.850 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.851 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.851 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.851 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.851 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.851 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:57:34.852 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.852 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.852 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.852 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.853 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:57:34.853 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:34.853 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.853 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.853 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.854 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:57:34.854 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:57:34.854 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:57:34.854 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:57:34.855 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.855 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.855 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.855 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.855 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.856 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.856 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:57:34.856 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:57:34.856 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.857 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.857 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.857 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.857 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.857 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.858 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.858 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:34.858 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.858 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.859 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.859 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.859 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.859 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.859 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:57:34.860 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.860 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.860 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.860 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.860 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:57:34.861 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.861 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.861 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.861 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.862 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.862 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.862 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.862 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.862 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:57:34.863 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.863 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.863 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.863 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.864 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:57:34.864 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.864 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.864 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.864 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.865 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.865 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.865 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.865 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.866 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:57:34.866 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.866 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.866 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.866 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.867 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:57:34.867 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.867 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.867 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.867 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.868 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.868 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.868 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.868 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.869 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:57:34.869 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.869 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.869 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.869 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.870 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:57:34.870 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:34.870 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.870 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.870 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.871 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:57:34.871 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:57:34.871 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:57:34.871 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:57:34.872 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.872 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.872 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.872 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.872 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.873 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.873 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:57:34.873 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:57:34.873 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.873 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.874 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.874 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.874 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.874 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.875 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.875 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:34.875 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.875 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.875 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.876 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.876 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.876 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.876 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:57:34.876 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.877 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.877 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.877 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.877 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:57:34.878 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.878 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.878 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.878 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.878 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.879 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.879 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.879 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.879 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:57:34.880 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.880 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.880 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.880 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.880 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:57:34.881 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.881 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.881 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.881 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.881 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.882 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.882 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.882 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.882 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:57:34.883 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.883 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.883 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.883 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.883 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:57:34.884 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.884 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.884 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.884 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.885 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.885 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.885 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.885 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.886 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:57:34.886 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.886 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.886 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.887 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.887 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:57:34.887 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:34.887 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.887 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.888 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.888 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:57:34.888 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:57:34.888 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:57:34.889 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:57:34.889 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.889 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.889 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.889 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.890 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.890 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.890 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:57:34.890 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:57:34.891 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.891 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.891 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.891 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.891 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.892 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.892 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.892 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:34.892 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.892 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.893 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.893 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.893 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.893 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.894 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:57:34.894 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.894 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.894 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.894 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.895 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:57:34.895 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.895 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.895 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.895 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.896 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.896 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.896 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.896 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.897 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:57:34.897 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.897 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.897 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.897 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.898 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:57:34.898 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.898 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.898 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.898 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.899 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.899 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.899 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.899 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.900 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:57:34.900 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.900 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.900 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.900 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.901 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:57:34.901 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.901 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.901 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.901 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.902 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.902 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.902 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.902 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.903 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:57:34.903 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.903 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.903 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.903 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.904 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:57:34.904 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:34.904 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.904 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.904 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.905 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:57:34.905 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:57:34.905 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:57:34.906 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:57:34.906 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.906 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.906 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.907 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.907 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.907 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.908 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:57:34.908 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:57:34.908 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:34.908 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:34.909 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:34.909 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:34.909 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:34.909 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:34.910 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:34.910 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 38, 2021-11-21T21:57:34.910 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:34.910 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:34.911 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:34.911 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.911 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.911 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.912 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:57:34.912 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.912 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.912 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.912 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.913 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:57:34.913 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.913 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.913 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.914 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.914 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:34.914 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.914 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.914 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.915 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:57:34.915 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.915 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.915 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.916 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.916 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:57:34.916 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.916 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.916 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.917 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.917 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:34.917 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.917 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.918 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.918 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:57:34.918 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.918 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.919 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.919 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.919 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:57:34.919 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.920 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.920 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.920 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.920 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:34.921 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:34.921 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.921 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:34.922 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:57:34.922 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.922 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.922 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.923 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:34.923 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:57:34.923 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:34.923 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.923 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.924 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.924 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:57:34.924 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:57:34.924 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:57:34.925 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:57:34.925 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:34.925 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:34.925 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:34.925 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:34.926 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.926 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:34.926 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:57:34.926 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.926 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:57:34.927 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.927 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.927 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.927 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.928 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.928 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:57:34.928 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.928 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.928 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.929 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:57:34.929 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.929 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.929 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.929 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.930 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.930 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:57:34.930 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.930 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.931 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.931 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:57:34.931 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.931 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.932 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.932 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.932 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.932 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:57:34.932 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.933 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.933 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.933 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:57:34.933 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.934 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.934 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.934 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.934 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.934 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:57:34.935 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.935 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.935 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.935 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:57:34.935 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.936 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.936 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.936 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.936 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.937 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:57:34.937 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.937 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.937 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.937 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:57:34.938 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.938 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.938 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.938 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.939 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.939 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:57:34.939 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.939 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.939 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.940 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:57:34.940 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.940 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.940 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.941 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.941 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.941 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:57:34.941 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.941 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.942 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:34.942 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:57:34.942 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:34.942 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:34.943 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:34.943 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:34.943 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:34.943 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:57:34.943 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:34.944 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.944 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:34.944 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:57:34.944 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:57:34.944 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:57:34.945 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:57:34.945 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T21:57:34.945 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/2140793945": "2021-11-22T21:56:08.223482+0000", 2021-11-21T21:57:34.945 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/2140793945": "2021-11-22T21:56:08.223482+0000" 2021-11-21T21:57:34.945 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.946 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:57:34.946 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:57:34.946 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:57:34.946 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:57:34.947 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:57:34.947 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:57:34.947 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:57:34.947 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:57:34.947 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:34.948 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:34.948 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:57:34.948 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:57:34.948 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:57:34.949 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:57:34.949 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:57:34.949 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:57:34.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74e0053690 msgr2=0x7f74e0055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:34.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74e0053690 0x7f74e0055b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f74f4001fa0 tx=0x7f74f40077f0).stop 2021-11-21T21:57:34.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 msgr2=0x7f750005fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:34.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f750005fa70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f74f000a000 tx=0x7f74f0008040).stop 2021-11-21T21:57:34.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 shutdown_connections 2021-11-21T21:57:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7500143e10 0x7f75001598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75000631c0 0x7f7500143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74e0053690 0x7f74e0055b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 --2- 172.21.2.116:0/167587853 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74ec003820 0x7f750005fa70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 >> 172.21.2.116:0/167587853 conn(0x7f750005c7f0 msgr2=0x7f7500148210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:34.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 shutdown_connections 2021-11-21T21:57:34.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:34.716+0000 7f75053f2700 1 -- 172.21.2.116:0/167587853 wait complete. 2021-11-21T21:57:34.954 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:57:35.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17f4c1700 1 Processor -- start 2021-11-21T21:57:35.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17f4c1700 1 -- start start 2021-11-21T21:57:35.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17f4c1700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd178097ec0 0x7fd17814af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17f4c1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd178099650 0x7fd178099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17f4c1700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17814a560 con 0x7fd1780b5630 2021-11-21T21:57:35.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17f4c1700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd178144b40 con 0x7fd178099650 2021-11-21T21:57:35.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17f4c1700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17814a800 con 0x7fd178097ec0 2021-11-21T21:57:35.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd178099650 0x7fd178099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd178099650 0x7fd178099ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47150/0 (socket says 172.21.2.116:47150) 2021-11-21T21:57:35.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 -- 172.21.2.116:0/2131147533 learned_addr learned my addr 172.21.2.116:0/2131147533 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 -- 172.21.2.116:0/2131147533 >> v2:172.21.2.117:3300/0 conn(0x7fd178097ec0 msgr2=0x7fd17814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17dcbe700 1 --2- 172.21.2.116:0/2131147533 >> v2:172.21.2.117:3300/0 conn(0x7fd178097ec0 0x7fd17814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 --2- 172.21.2.116:0/2131147533 >> v2:172.21.2.117:3300/0 conn(0x7fd178097ec0 0x7fd17814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 -- 172.21.2.116:0/2131147533 >> v1:172.21.2.117:6789/0 conn(0x7fd1780b5630 legacy=0x7fd178146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 -- 172.21.2.116:0/2131147533 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17814aaa0 con 0x7fd178099650 2021-11-21T21:57:35.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17dcbe700 1 --2- 172.21.2.116:0/2131147533 >> v2:172.21.2.117:3300/0 conn(0x7fd178097ec0 0x7fd17814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:57:35.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17ecc0700 1 --2- 172.21.2.116:0/2131147533 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd178099ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fd17809afa0 tx=0x7fd170009290).ready entity=mon.0 client_cookie=dc6b8577bc646f46 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:35.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17d4bd700 1 -- 172.21.2.116:0/2131147533 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd17000b830 con 0x7fd178099650 2021-11-21T21:57:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17d4bd700 1 -- 172.21.2.116:0/2131147533 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17000b990 con 0x7fd178099650 2021-11-21T21:57:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.008+0000 7fd17d4bd700 1 -- 172.21.2.116:0/2131147533 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd17000bb40 con 0x7fd178099650 2021-11-21T21:57:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/2131147533 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 msgr2=0x7fd178099ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/2131147533 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd178099ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fd17809afa0 tx=0x7fd170009290).stop 2021-11-21T21:57:35.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/2131147533 shutdown_connections 2021-11-21T21:57:35.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/2131147533 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd178099ae0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/2131147533 >> v2:172.21.2.117:3300/0 conn(0x7fd178097ec0 0x7fd17814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/2131147533 >> 172.21.2.116:0/2131147533 conn(0x7fd17805c7f0 msgr2=0x7fd1780b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:35.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/2131147533 shutdown_connections 2021-11-21T21:57:35.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/2131147533 wait complete. 2021-11-21T21:57:35.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 Processor -- start 2021-11-21T21:57:35.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- start start 2021-11-21T21:57:35.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd178097ec0 0x7fd1781489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd1780669c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1780b5630 0x7fd178066eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178066710 con 0x7fd178099650 2021-11-21T21:57:35.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780650e0 con 0x7fd178097ec0 2021-11-21T21:57:35.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd178065380 con 0x7fd1780b5630 2021-11-21T21:57:35.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd1780669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd1780669c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47154/0 (socket says 172.21.2.116:47154) 2021-11-21T21:57:35.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 -- 172.21.2.116:0/1587292564 learned_addr learned my addr 172.21.2.116:0/1587292564 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:35.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17e4bf700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd178097ec0 0x7fd1781489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 -- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1780b5630 msgr2=0x7fd178066eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1780b5630 0x7fd178066eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 -- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd178097ec0 msgr2=0x7fd1781489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd178097ec0 0x7fd1781489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 -- 172.21.2.116:0/1587292564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170009530 con 0x7fd178099650 2021-11-21T21:57:35.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17dcbe700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd1780669c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fd174007000 tx=0x7fd174008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:35.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd16b7fe700 1 -- 172.21.2.116:0/1587292564 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd174010580 con 0x7fd178099650 2021-11-21T21:57:35.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd16b7fe700 1 -- 172.21.2.116:0/1587292564 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1740106e0 con 0x7fd178099650 2021-11-21T21:57:35.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178065620 con 0x7fd178099650 2021-11-21T21:57:35.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178065b40 con 0x7fd178099650 2021-11-21T21:57:35.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd16b7fe700 1 -- 172.21.2.116:0/1587292564 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd174010a40 con 0x7fd178099650 2021-11-21T21:57:35.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1780657b0 con 0x7fd178099650 2021-11-21T21:57:35.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd16b7fe700 1 -- 172.21.2.116:0/1587292564 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd174010ba0 con 0x7fd178099650 2021-11-21T21:57:35.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.012+0000 7fd16b7fe700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd158053520 0x7fd1580559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.016+0000 7fd16b7fe700 1 -- 172.21.2.116:0/1587292564 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fd17406bad0 con 0x7fd178099650 2021-11-21T21:57:35.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.016+0000 7fd17e4bf700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd158053520 0x7fd1580559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.016+0000 7fd17e4bf700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd158053520 0x7fd1580559e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fd16c00a000 tx=0x7fd16c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:35.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.016+0000 7fd16b7fe700 1 -- 172.21.2.116:0/1587292564 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd174042be0 con 0x7fd178099650 2021-11-21T21:57:35.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd178147520 con 0x7fd178099650 2021-11-21T21:57:35.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd16b7fe700 1 -- 172.21.2.116:0/1587292564 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fd174046200 con 0x7fd178099650 2021-11-21T21:57:35.125 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:35.125 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:56:09.961260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2465449969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2465449969},{"type":"v1","addr":"172.21.2.116:6835","nonce":2465449969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:57:35.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd158053520 msgr2=0x7fd1580559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd158053520 0x7fd1580559e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fd16c00a000 tx=0x7fd16c008040).stop 2021-11-21T21:57:35.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 msgr2=0x7fd1780669c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd1780669c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fd174007000 tx=0x7fd174008040).stop 2021-11-21T21:57:35.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 shutdown_connections 2021-11-21T21:57:35.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1780b5630 0x7fd178066eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd158053520 0x7fd1580559e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd178099650 0x7fd1780669c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 --2- 172.21.2.116:0/1587292564 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd178097ec0 0x7fd1781489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 >> 172.21.2.116:0/1587292564 conn(0x7fd17805c7f0 msgr2=0x7fd178143ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:35.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 shutdown_connections 2021-11-21T21:57:35.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.124+0000 7fd17f4c1700 1 -- 172.21.2.116:0/1587292564 wait complete. 2021-11-21T21:57:35.129 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 17 2021-11-21T21:57:35.132 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T21:57:35.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b81c2700 1 Processor -- start 2021-11-21T21:57:35.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b81c2700 1 -- start start 2021-11-21T21:57:35.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b81c2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0144810 0x7f88b01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b81c2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f88b00ac600 0x7f88b00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b81c2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b0147e80 con 0x7f88b00abce0 2021-11-21T21:57:35.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b81c2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b00b1490 con 0x7f88b00ac600 2021-11-21T21:57:35.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b81c2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b1fd0 con 0x7f88b0144810 2021-11-21T21:57:35.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b575d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f88b00abce0 0x7f88b00b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59120/0 (socket says 172.21.2.116:59120) 2021-11-21T21:57:35.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b575d700 1 -- 172.21.2.116:0/3179663988 learned_addr learned my addr 172.21.2.116:0/3179663988 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:35.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496439506 0 0) 0x7f88b0147e80 con 0x7f88b00abce0 2021-11-21T21:57:35.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8898003260 con 0x7f88b00abce0 2021-11-21T21:57:35.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 995976984 0 0) 0x7f8898003260 con 0x7f88b00abce0 2021-11-21T21:57:35.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0144810 msgr2=0x7f88b01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b4f5c700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0144810 0x7f88b01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 >> v2:172.21.2.117:3300/0 conn(0x7f88b00ac600 msgr2=0x7f88b00aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.180+0000 7f88b4f5c700 1 --2- 172.21.2.116:0/3179663988 >> v2:172.21.2.117:3300/0 conn(0x7f88b00ac600 0x7f88b00aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00af5a0 con 0x7f88b00abce0 2021-11-21T21:57:35.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f88ac001660 con 0x7f88b00abce0 2021-11-21T21:57:35.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 >> v1:172.21.2.116:6789/0 conn(0x7f88b00abce0 legacy=0x7f88b00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8898003750 0x7f8898003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0144810 0x7f8898004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898004b10 0x7f8898004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b1490 con 0x7f8898003750 2021-11-21T21:57:35.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b1fd0 con 0x7f8898004b10 2021-11-21T21:57:35.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889800b990 con 0x7f88b0144810 2021-11-21T21:57:35.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b5f5e700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8898003750 0x7f8898003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b675f700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898004b10 0x7f8898004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0144810 0x7f8898004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b5f5e700 1 -- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0144810 msgr2=0x7f8898004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b5f5e700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0144810 0x7f8898004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b5f5e700 1 -- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898004b10 msgr2=0x7f8898004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b5f5e700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898004b10 0x7f8898004f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b5f5e700 1 -- 172.21.2.116:0/3179663988 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898003260 con 0x7f8898003750 2021-11-21T21:57:35.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b675f700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898004b10 0x7f8898004f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:57:35.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b5f5e700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8898003750 0x7f8898003b70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f88a4002f70 tx=0x7f88a4006ca0).ready entity=mon.0 client_cookie=4d6850c5417d7df1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:35.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a4013410 con 0x7f8898003750 2021-11-21T21:57:35.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4013570 con 0x7f8898003750 2021-11-21T21:57:35.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b4f5c700 1 -- 172.21.2.116:0/3179663988 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a40138d0 con 0x7f8898003750 2021-11-21T21:57:35.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- 172.21.2.116:0/3179663988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8898003750 msgr2=0x7f8898003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 --2- 172.21.2.116:0/3179663988 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8898003750 0x7f8898003b70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f88a4002f70 tx=0x7f88a4006ca0).stop 2021-11-21T21:57:35.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b675f700 1 -- 172.21.2.116:0/3179663988 reap_dead start 2021-11-21T21:57:35.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- 172.21.2.116:0/3179663988 shutdown_connections 2021-11-21T21:57:35.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- 172.21.2.116:0/3179663988 >> 172.21.2.116:0/3179663988 conn(0x7f88b005bd20 msgr2=0x7f88b005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:35.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- 172.21.2.116:0/3179663988 shutdown_connections 2021-11-21T21:57:35.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- 172.21.2.116:0/3179663988 wait complete. 2021-11-21T21:57:35.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 Processor -- start 2021-11-21T21:57:35.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- start start 2021-11-21T21:57:35.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898003750 0x7f88b0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88b015a760 0x7f88b0156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0157350 0x7f88b00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01453e0 con 0x7f88b015a760 2021-11-21T21:57:35.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0144ea0 con 0x7f8898003750 2021-11-21T21:57:35.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0145140 con 0x7f88b0157350 2021-11-21T21:57:35.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88b015a760 0x7f88b0156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88b015a760 0x7f88b0156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47176/0 (socket says 172.21.2.116:47176) 2021-11-21T21:57:35.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 -- 172.21.2.116:0/2270793025 learned_addr learned my addr 172.21.2.116:0/2270793025 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:35.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 -- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0157350 msgr2=0x7f88b00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0157350 0x7f88b00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 -- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898003750 msgr2=0x7f88b0143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898003750 0x7f88b0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 -- 172.21.2.116:0/2270793025 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b01569c0 con 0x7f88b015a760 2021-11-21T21:57:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b575d700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88b015a760 0x7f88b0156e60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f88ac008000 tx=0x7f88ac009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88a2ffd700 1 -- 172.21.2.116:0/2270793025 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88ac007170 con 0x7f88b015a760 2021-11-21T21:57:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88a2ffd700 1 -- 172.21.2.116:0/2270793025 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac0072d0 con 0x7f88b015a760 2021-11-21T21:57:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88a2ffd700 1 -- 172.21.2.116:0/2270793025 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88ac004920 con 0x7f88b015a760 2021-11-21T21:57:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.184+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b015abe0 con 0x7f88b015a760 2021-11-21T21:57:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b01457d0 con 0x7f88b015a760 2021-11-21T21:57:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0156b50 con 0x7f88b015a760 2021-11-21T21:57:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88a2ffd700 1 -- 172.21.2.116:0/2270793025 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f88ac004a80 con 0x7f88b015a760 2021-11-21T21:57:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88a2ffd700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88900536a0 0x7f8890055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88a2ffd700 1 -- 172.21.2.116:0/2270793025 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f88ac06c010 con 0x7f88b015a760 2021-11-21T21:57:35.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88b5f5e700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88900536a0 0x7f8890055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88b5f5e700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88900536a0 0x7f8890055b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f88a4002f70 tx=0x7f88a4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:35.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.188+0000 7f88a2ffd700 1 -- 172.21.2.116:0/2270793025 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f88ac042f30 con 0x7f88b015a760 2021-11-21T21:57:35.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.300+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f88b00af060 con 0x7f88b015a760 2021-11-21T21:57:35.904 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T21:57:35.900+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T21:57:35.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.912+0000 7f88a2ffd700 1 -- 172.21.2.116:0/2270793025 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f88ac046550 con 0x7f88b015a760 2021-11-21T21:57:35.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88900536a0 msgr2=0x7f8890055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88900536a0 0x7f8890055b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f88a4002f70 tx=0x7f88a4014000).stop 2021-11-21T21:57:35.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88b015a760 msgr2=0x7f88b0156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88b015a760 0x7f88b0156e60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f88ac008000 tx=0x7f88ac009040).stop 2021-11-21T21:57:35.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 shutdown_connections 2021-11-21T21:57:35.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88b0157350 0x7f88b00b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f88900536a0 0x7f8890055b60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88b015a760 0x7f88b0156e60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 --2- 172.21.2.116:0/2270793025 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8898003750 0x7f88b0143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 >> 172.21.2.116:0/2270793025 conn(0x7f88b005bd20 msgr2=0x7f88b01400f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:35.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 shutdown_connections 2021-11-21T21:57:35.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.916+0000 7f88b81c2700 1 -- 172.21.2.116:0/2270793025 wait complete. 2021-11-21T21:57:35.920 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T21:57:35.924 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:57:35.935 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:57:35.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.972+0000 7fec59bef700 1 Processor -- start 2021-11-21T21:57:35.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.972+0000 7fec59bef700 1 -- start start 2021-11-21T21:57:35.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fec540a86b0 0x7fec5414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec540a6e80 con 0x7fec540a6aa0 2021-11-21T21:57:35.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec5414a7b0 con 0x7fec54062410 2021-11-21T21:57:35.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec540a8bc0 con 0x7fec540a86b0 2021-11-21T21:57:35.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec537fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fec540a6aa0 0x7fec541470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59146/0 (socket says 172.21.2.116:59146) 2021-11-21T21:57:35.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec537fe700 1 -- 172.21.2.116:0/2549020741 learned_addr learned my addr 172.21.2.116:0/2549020741 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:35.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057666082 0 0) 0x7fec540a6e80 con 0x7fec540a6aa0 2021-11-21T21:57:35.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec40003260 con 0x7fec540a6aa0 2021-11-21T21:57:35.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec52ffd700 1 --2- 172.21.2.116:0/2549020741 >> v2:172.21.2.117:3300/0 conn(0x7fec540a86b0 0x7fec5414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3340600739 0 0) 0x7fec40003260 con 0x7fec540a6aa0 2021-11-21T21:57:35.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 >> v2:172.21.2.117:3300/0 conn(0x7fec540a86b0 msgr2=0x7fec5414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 --2- 172.21.2.116:0/2549020741 >> v2:172.21.2.117:3300/0 conn(0x7fec540a86b0 0x7fec5414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 >> v1:172.21.2.117:6789/0 conn(0x7fec54062410 legacy=0x7fec541443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5414aa10 con 0x7fec540a6aa0 2021-11-21T21:57:35.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fec44001680 con 0x7fec540a6aa0 2021-11-21T21:57:35.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 >> v1:172.21.2.116:6789/0 conn(0x7fec540a6aa0 legacy=0x7fec541470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec40003820 0x7fec40003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec40004500 0x7fec40004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec4000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec540a8bc0 con 0x7fec40004f70 2021-11-21T21:57:35.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4000be50 con 0x7fec40004500 2021-11-21T21:57:35.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4000c010 con 0x7fec40003820 2021-11-21T21:57:35.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec4000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec537fe700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec40003820 0x7fec40003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 -- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec40003820 msgr2=0x7fec40003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec40003820 0x7fec40003c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec52ffd700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec40004500 0x7fec40004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 -- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec40004500 msgr2=0x7fec40004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:35.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec40004500 0x7fec40004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 -- 172.21.2.116:0/2549020741 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40003260 con 0x7fec40004f70 2021-11-21T21:57:35.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 -- 172.21.2.116:0/2549020741 reap_dead start 2021-11-21T21:57:35.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec53fff700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec4000b750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fec4c002320 tx=0x7fec4c007d80).ready entity=mon.0 client_cookie=c9148e9ea4efb410 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:35.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec4c002510 con 0x7fec40004f70 2021-11-21T21:57:35.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c006890 con 0x7fec40004f70 2021-11-21T21:57:35.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec527fc700 1 -- 172.21.2.116:0/2549020741 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec4c006bf0 con 0x7fec40004f70 2021-11-21T21:57:35.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- 172.21.2.116:0/2549020741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 msgr2=0x7fec4000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec4000b750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fec4c002320 tx=0x7fec4c007d80).stop 2021-11-21T21:57:35.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- 172.21.2.116:0/2549020741 shutdown_connections 2021-11-21T21:57:35.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 --2- 172.21.2.116:0/2549020741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec4000b750 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:35.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- 172.21.2.116:0/2549020741 >> 172.21.2.116:0/2549020741 conn(0x7fec5405bd20 msgr2=0x7fec5405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:35.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- 172.21.2.116:0/2549020741 shutdown_connections 2021-11-21T21:57:35.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- 172.21.2.116:0/2549020741 wait complete. 2021-11-21T21:57:35.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 Processor -- start 2021-11-21T21:57:35.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- start start 2021-11-21T21:57:35.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec54147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec54065bb0 0x7fec540ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec54066d50 0x7fec54067200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:35.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec540676f0 con 0x7fec40004f70 2021-11-21T21:57:35.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec54066050 con 0x7fec54065bb0 2021-11-21T21:57:35.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.976+0000 7fec59bef700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec540662f0 con 0x7fec54066d50 2021-11-21T21:57:35.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec54147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:35.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec54147440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47202/0 (socket says 172.21.2.116:47202) 2021-11-21T21:57:35.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 -- 172.21.2.116:0/1565674755 learned_addr learned my addr 172.21.2.116:0/1565674755 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:35.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 -- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec54066d50 msgr2=0x7fec54067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:35.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec54066d50 0x7fec54067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:35.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 -- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec54065bb0 msgr2=0x7fec540ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec54065bb0 0x7fec540ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 -- 172.21.2.116:0/1565674755 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec54066590 con 0x7fec40004f70 2021-11-21T21:57:36.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec537fe700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec54147440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fec44008000 tx=0x7fec44009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:36.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec50ff9700 1 -- 172.21.2.116:0/1565674755 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec44007170 con 0x7fec40004f70 2021-11-21T21:57:36.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec50ff9700 1 -- 172.21.2.116:0/1565674755 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec440072d0 con 0x7fec40004f70 2021-11-21T21:57:36.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec50ff9700 1 -- 172.21.2.116:0/1565674755 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec44004920 con 0x7fec40004f70 2021-11-21T21:57:36.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec54066890 con 0x7fec40004f70 2021-11-21T21:57:36.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec54159530 con 0x7fec40004f70 2021-11-21T21:57:36.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec540573d0 con 0x7fec40004f70 2021-11-21T21:57:36.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec50ff9700 1 -- 172.21.2.116:0/1565674755 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fec44004a80 con 0x7fec40004f70 2021-11-21T21:57:36.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec50ff9700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec34053580 0x7fec34055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec50ff9700 1 -- 172.21.2.116:0/1565674755 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6241+0+0 (secure 0 0 0) 0x7fec4406bac0 con 0x7fec40004f70 2021-11-21T21:57:36.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec52ffd700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec34053580 0x7fec34055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:36.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.980+0000 7fec52ffd700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec34053580 0x7fec34055a40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fec48008000 tx=0x7fec48008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:36.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:35.984+0000 7fec50ff9700 1 -- 172.21.2.116:0/1565674755 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec44042b70 con 0x7fec40004f70 2021-11-21T21:57:36.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec54066a20 con 0x7fec40004f70 2021-11-21T21:57:36.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec50ff9700 1 -- 172.21.2.116:0/1565674755 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2130 (secure 0 0 0) 0x7fec44046190 con 0x7fec40004f70 2021-11-21T21:57:36.094 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:36.094 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:56:07.942419+0000","modified":"2021-11-21T21:57:35.904970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-11-21T21:57:36.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec34053580 msgr2=0x7fec34055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec34053580 0x7fec34055a40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fec48008000 tx=0x7fec48008040).stop 2021-11-21T21:57:36.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 msgr2=0x7fec54147440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec54147440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fec44008000 tx=0x7fec44009040).stop 2021-11-21T21:57:36.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 shutdown_connections 2021-11-21T21:57:36.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec54066d50 0x7fec54067200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec54065bb0 0x7fec540ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec34053580 0x7fec34055a40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:36.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 --2- 172.21.2.116:0/1565674755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec40004f70 0x7fec54147440 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:36.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 >> 172.21.2.116:0/1565674755 conn(0x7fec5405bd20 msgr2=0x7fec54148f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:36.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 shutdown_connections 2021-11-21T21:57:36.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.092+0000 7fec59bef700 1 -- 172.21.2.116:0/1565674755 wait complete. 2021-11-21T21:57:36.098 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 18 2021-11-21T21:57:36.100 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T21:57:36.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f1880e32700 1 Processor -- start 2021-11-21T21:57:36.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f1880e32700 1 -- start start 2021-11-21T21:57:36.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f1880e32700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c0abce0 0x7f187c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f1880e32700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c147e80 con 0x7f187c144810 2021-11-21T21:57:36.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f1880e32700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c0af5a0 con 0x7f187c0ac600 2021-11-21T21:57:36.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f1880e32700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0b1490 con 0x7f187c0abce0 2021-11-21T21:57:36.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187a59c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f187c144810 0x7f187c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59166/0 (socket says 172.21.2.116:59166) 2021-11-21T21:57:36.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187a59c700 1 -- 172.21.2.116:0/517973501 learned_addr learned my addr 172.21.2.116:0/517973501 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:36.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310357702 0 0) 0x7f187c147e80 con 0x7f187c144810 2021-11-21T21:57:36.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f185c003260 con 0x7f187c144810 2021-11-21T21:57:36.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 966646913 0 0) 0x7f185c003260 con 0x7f187c144810 2021-11-21T21:57:36.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c0abce0 msgr2=0x7f187c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:36.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187959a700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c0abce0 0x7f187c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.152+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 >> v1:172.21.2.117:6789/0 conn(0x7f187c0ac600 legacy=0x7f187c0acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:36.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0b1fd0 con 0x7f187c144810 2021-11-21T21:57:36.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f186c001660 con 0x7f187c144810 2021-11-21T21:57:36.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 >> v1:172.21.2.116:6789/0 conn(0x7f187c144810 legacy=0x7f187c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c0abce0 0x7f185c003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c003e20 0x7f185c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f185c004b10 0x7f185c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0b1490 con 0x7f185c004b10 2021-11-21T21:57:36.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c00b960 con 0x7f185c003e20 2021-11-21T21:57:36.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c00bb20 con 0x7f187c0abce0 2021-11-21T21:57:36.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f185c004b10 0x7f185c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:36.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c0abce0 msgr2=0x7f185c003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:36.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c0abce0 0x7f185c003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c003e20 msgr2=0x7f185c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:36.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c003e20 0x7f185c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/517973501 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c003260 con 0x7f185c004b10 2021-11-21T21:57:36.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f185c004b10 0x7f185c004f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f1874002320 tx=0x7f1874007d80).ready entity=mon.0 client_cookie=e54ec76098cfc0be server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:36.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1874002510 con 0x7f185c004b10 2021-11-21T21:57:36.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1874006890 con 0x7f185c004b10 2021-11-21T21:57:36.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187959a700 1 -- 172.21.2.116:0/517973501 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1874006bf0 con 0x7f185c004b10 2021-11-21T21:57:36.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/517973501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f185c004b10 msgr2=0x7f185c004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 --2- 172.21.2.116:0/517973501 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f185c004b10 0x7f185c004f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f1874002320 tx=0x7f1874007d80).stop 2021-11-21T21:57:36.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/517973501 reap_dead start 2021-11-21T21:57:36.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/517973501 shutdown_connections 2021-11-21T21:57:36.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/517973501 >> 172.21.2.116:0/517973501 conn(0x7f187c05bd20 msgr2=0x7f187c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:36.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/517973501 shutdown_connections 2021-11-21T21:57:36.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/517973501 wait complete. 2021-11-21T21:57:36.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 Processor -- start 2021-11-21T21:57:36.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- start start 2021-11-21T21:57:36.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c004b10 0x7f187c14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c13e100 0x7f187c13e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f187c1566d0 0x7f187c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0b4bd0 con 0x7f187c1566d0 2021-11-21T21:57:36.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c140ad0 con 0x7f185c004b10 2021-11-21T21:57:36.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c146370 con 0x7f187c13e100 2021-11-21T21:57:36.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f187c1566d0 0x7f187c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:36.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f187c1566d0 0x7f187c156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47224/0 (socket says 172.21.2.116:47224) 2021-11-21T21:57:36.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/3817183398 learned_addr learned my addr 172.21.2.116:0/3817183398 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:36.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187a59c700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c004b10 0x7f187c14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:36.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c13e100 msgr2=0x7f187c13e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c13e100 0x7f187c13e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c004b10 msgr2=0x7f187c14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c004b10 0x7f187c14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:36.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 -- 172.21.2.116:0/3817183398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0b41f0 con 0x7f187c1566d0 2021-11-21T21:57:36.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f187ad9d700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f187c1566d0 0x7f187c156b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f187c0b4ab0 tx=0x7f1874016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:36.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f18737fe700 1 -- 172.21.2.116:0/3817183398 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1874002510 con 0x7f187c1566d0 2021-11-21T21:57:36.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f18737fe700 1 -- 172.21.2.116:0/3817183398 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1874013810 con 0x7f187c1566d0 2021-11-21T21:57:36.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c0b4490 con 0x7f187c1566d0 2021-11-21T21:57:36.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f18737fe700 1 -- 172.21.2.116:0/3817183398 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1874013b70 con 0x7f187c1566d0 2021-11-21T21:57:36.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c13ea70 con 0x7f187c1566d0 2021-11-21T21:57:36.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.156+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c0af060 con 0x7f187c1566d0 2021-11-21T21:57:36.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.160+0000 7f18737fe700 1 -- 172.21.2.116:0/3817183398 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1874026030 con 0x7f187c1566d0 2021-11-21T21:57:36.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.160+0000 7f18737fe700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1860053640 0x7f1860055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.160+0000 7f18737fe700 1 -- 172.21.2.116:0/3817183398 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6241+0+0 (secure 0 0 0) 0x7f187406cb80 con 0x7f187c1566d0 2021-11-21T21:57:36.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.160+0000 7f187a59c700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1860053640 0x7f1860055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:36.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.160+0000 7f187a59c700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1860053640 0x7f1860055b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f186c00b000 tx=0x7f187c146e80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:36.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.160+0000 7f18737fe700 1 -- 172.21.2.116:0/3817183398 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1874043c30 con 0x7f187c1566d0 2021-11-21T21:57:36.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.272+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f187c0573d0 con 0x7f187c1566d0 2021-11-21T21:57:36.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f18737fe700 1 -- 172.21.2.116:0/3817183398 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1874024020 con 0x7f187c1566d0 2021-11-21T21:57:36.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1860053640 msgr2=0x7f1860055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1860053640 0x7f1860055b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f186c00b000 tx=0x7f187c146e80).stop 2021-11-21T21:57:36.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f187c1566d0 msgr2=0x7f187c156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:36.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f187c1566d0 0x7f187c156b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f187c0b4ab0 tx=0x7f1874016000).stop 2021-11-21T21:57:36.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 shutdown_connections 2021-11-21T21:57:36.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1860053640 0x7f1860055b00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:36.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f187c1566d0 0x7f187c156b50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:36.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f187c13e100 0x7f187c13e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 --2- 172.21.2.116:0/3817183398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f185c004b10 0x7f187c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:36.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 >> 172.21.2.116:0/3817183398 conn(0x7f187c05bd20 msgr2=0x7f187c148e20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:36.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 shutdown_connections 2021-11-21T21:57:36.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.924+0000 7f1880e32700 1 -- 172.21.2.116:0/3817183398 wait complete. 2021-11-21T21:57:36.933 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T21:57:36.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2889f22700 1 Processor -- start 2021-11-21T21:57:36.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2889f22700 1 -- start start 2021-11-21T21:57:36.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2889f22700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f28840a8670 0x7f2884147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2889f22700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2884143e60 0x7f28841442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:36.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2889f22700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840b1940 con 0x7f28840b11c0 2021-11-21T21:57:36.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2889f22700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28840b15a0 con 0x7f28840a8670 2021-11-21T21:57:36.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2889f22700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28840b1b70 con 0x7f2884143e60 2021-11-21T21:57:36.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f28840a8670 0x7f2884147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:36.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f28840a8670 0x7f2884147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47234/0 (socket says 172.21.2.116:47234) 2021-11-21T21:57:36.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 -- 172.21.2.116:0/1706487671 learned_addr learned my addr 172.21.2.116:0/1706487671 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:36.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 -- 172.21.2.116:0/1706487671 >> v2:172.21.2.117:3300/0 conn(0x7f2884143e60 msgr2=0x7f28841442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:36.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 --2- 172.21.2.116:0/1706487671 >> v2:172.21.2.117:3300/0 conn(0x7f2884143e60 0x7f28841442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:36.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 -- 172.21.2.116:0/1706487671 >> v1:172.21.2.117:6789/0 conn(0x7f28840b11c0 legacy=0x7f2884147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:37.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 -- 172.21.2.116:0/1706487671 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840b1e10 con 0x7f28840a8670 2021-11-21T21:57:37.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f2882ffd700 1 --2- 172.21.2.116:0/1706487671 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840a8670 0x7f2884147940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f2884147180 tx=0x7f2874006eb0).ready entity=mon.0 client_cookie=dcb6bee350850b26 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:37.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f28827fc700 1 -- 172.21.2.116:0/1706487671 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287400d5c0 con 0x7f28840a8670 2021-11-21T21:57:37.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f28827fc700 1 -- 172.21.2.116:0/1706487671 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874000e90 con 0x7f28840a8670 2021-11-21T21:57:37.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.992+0000 7f28827fc700 1 -- 172.21.2.116:0/1706487671 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287400d910 con 0x7f28840a8670 2021-11-21T21:57:37.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/1706487671 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840a8670 msgr2=0x7f2884147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:37.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 --2- 172.21.2.116:0/1706487671 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840a8670 0x7f2884147940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f2884147180 tx=0x7f2874006eb0).stop 2021-11-21T21:57:37.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/1706487671 shutdown_connections 2021-11-21T21:57:37.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 --2- 172.21.2.116:0/1706487671 >> v2:172.21.2.117:3300/0 conn(0x7f2884143e60 0x7f28841442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:37.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 --2- 172.21.2.116:0/1706487671 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840a8670 0x7f2884147940 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:37.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/1706487671 >> 172.21.2.116:0/1706487671 conn(0x7f288405bd20 msgr2=0x7f288405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:37.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/1706487671 shutdown_connections 2021-11-21T21:57:37.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/1706487671 wait complete. 2021-11-21T21:57:37.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 Processor -- start 2021-11-21T21:57:37.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- start start 2021-11-21T21:57:37.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28840a8670 0x7f28841468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:37.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840b11c0 0x7f2884141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:37.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2884143e60 0x7f28841415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:37.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2884065550 con 0x7f28840b11c0 2021-11-21T21:57:37.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2884066b70 con 0x7f2884143e60 2021-11-21T21:57:37.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2884066190 con 0x7f28840a8670 2021-11-21T21:57:37.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840b11c0 0x7f2884141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:37.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840b11c0 0x7f2884141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47238/0 (socket says 172.21.2.116:47238) 2021-11-21T21:57:37.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 -- 172.21.2.116:0/917460980 learned_addr learned my addr 172.21.2.116:0/917460980 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:37.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2883fff700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2884143e60 0x7f28841415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:37.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 -- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28840a8670 msgr2=0x7f28841468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:37.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28840a8670 0x7f28841468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:37.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 -- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2884143e60 msgr2=0x7f28841415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:37.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2884143e60 0x7f28841415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:37.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 -- 172.21.2.116:0/917460980 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287400b1e0 con 0x7f28840b11c0 2021-11-21T21:57:37.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2882ffd700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840b11c0 0x7f2884141100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f2884147180 tx=0x7f2874015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:37.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2880ff9700 1 -- 172.21.2.116:0/917460980 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2874006be0 con 0x7f28840b11c0 2021-11-21T21:57:37.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2880ff9700 1 -- 172.21.2.116:0/917460980 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874002f50 con 0x7f28840b11c0 2021-11-21T21:57:37.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2884066430 con 0x7f28840b11c0 2021-11-21T21:57:37.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2880ff9700 1 -- 172.21.2.116:0/917460980 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2874011ce0 con 0x7f28840b11c0 2021-11-21T21:57:37.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2884141ae0 con 0x7f28840b11c0 2021-11-21T21:57:37.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288414a0b0 con 0x7f28840b11c0 2021-11-21T21:57:37.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2880ff9700 1 -- 172.21.2.116:0/917460980 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2874025030 con 0x7f28840b11c0 2021-11-21T21:57:37.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2880ff9700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2868053640 0x7f2868055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:37.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:36.996+0000 7f2880ff9700 1 -- 172.21.2.116:0/917460980 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6241+0+0 (secure 0 0 0) 0x7f287406cd20 con 0x7f28840b11c0 2021-11-21T21:57:37.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.000+0000 7f28837fe700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2868053640 0x7f2868055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:37.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.000+0000 7f28837fe700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2868053640 0x7f2868055b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f286c002f90 tx=0x7f286c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:37.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.000+0000 7f2880ff9700 1 -- 172.21.2.116:0/917460980 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2874043cc0 con 0x7f28840b11c0 2021-11-21T21:57:37.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.084+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28840b20b0 con 0x7f28840b11c0 2021-11-21T21:57:37.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2880ff9700 1 -- 172.21.2.116:0/917460980 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2874023020 con 0x7f28840b11c0 2021-11-21T21:57:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2868053640 msgr2=0x7f2868055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2868053640 0x7f2868055b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f286c002f90 tx=0x7f286c0069b0).stop 2021-11-21T21:57:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840b11c0 msgr2=0x7f2884141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:37.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840b11c0 0x7f2884141100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f2884147180 tx=0x7f2874015000).stop 2021-11-21T21:57:37.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 shutdown_connections 2021-11-21T21:57:37.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2884143e60 0x7f28841415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:37.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2868053640 0x7f2868055b00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:37.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28840b11c0 0x7f2884141100 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:37.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 --2- 172.21.2.116:0/917460980 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28840a8670 0x7f28841468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:37.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 >> 172.21.2.116:0/917460980 conn(0x7f288405bd20 msgr2=0x7f288414c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:37.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 shutdown_connections 2021-11-21T21:57:37.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:37.944+0000 7f2889f22700 1 -- 172.21.2.116:0/917460980 wait complete. 2021-11-21T21:57:37.949 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T21:57:37.953 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T21:57:38.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.004+0000 7fb8b5723700 1 Processor -- start 2021-11-21T21:57:38.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.004+0000 7fb8b5723700 1 -- start start 2021-11-21T21:57:38.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.004+0000 7fb8b5723700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00ad780 0x7fb8b005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:38.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.004+0000 7fb8b5723700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb8b00af7e0 0x7fb8b013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:38.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.004+0000 7fb8b5723700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b0143f70 con 0x7fb8b00affc0 2021-11-21T21:57:38.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.004+0000 7fb8b5723700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b00adb60 con 0x7fb8b00af7e0 2021-11-21T21:57:38.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.004+0000 7fb8b5723700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0143160 con 0x7fb8b00ad780 2021-11-21T21:57:38.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00ad780 0x7fb8b005f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:38.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8af7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb8b00af7e0 0x7fb8b013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:38.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8ae7fc700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fb8b00affc0 0x7fb8b0143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:58112/0 (socket says 172.21.2.116:58112) 2021-11-21T21:57:38.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8ae7fc700 1 -- 172.21.2.116:0/1503366140 learned_addr learned my addr 172.21.2.116:0/1503366140 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:38.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 -- 172.21.2.116:0/1503366140 >> v2:172.21.2.117:3300/0 conn(0x7fb8b00af7e0 msgr2=0x7fb8b013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:38.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- 172.21.2.116:0/1503366140 >> v2:172.21.2.117:3300/0 conn(0x7fb8b00af7e0 0x7fb8b013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:38.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 -- 172.21.2.116:0/1503366140 >> v1:172.21.2.117:6789/0 conn(0x7fb8b00affc0 legacy=0x7fb8b0143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:38.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 -- 172.21.2.116:0/1503366140 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b00b0ce0 con 0x7fb8b00ad780 2021-11-21T21:57:38.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8af7fe700 1 --2- 172.21.2.116:0/1503366140 >> v2:172.21.2.117:3300/0 conn(0x7fb8b00af7e0 0x7fb8b013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:57:38.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- 172.21.2.116:0/1503366140 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00ad780 0x7fb8b005f730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb8a4002f70 tx=0x7fb8a4006e80).ready entity=mon.2 client_cookie=49a08a0e6f0ea5d0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:38.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8adffb700 1 -- 172.21.2.116:0/1503366140 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a400b7d0 con 0x7fb8b00ad780 2021-11-21T21:57:38.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8adffb700 1 -- 172.21.2.116:0/1503366140 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a400b930 con 0x7fb8b00ad780 2021-11-21T21:57:38.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8adffb700 1 -- 172.21.2.116:0/1503366140 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a400bc90 con 0x7fb8b00ad780 2021-11-21T21:57:38.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- 172.21.2.116:0/1503366140 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00ad780 msgr2=0x7fb8b005f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:38.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 --2- 172.21.2.116:0/1503366140 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00ad780 0x7fb8b005f730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb8a4002f70 tx=0x7fb8a4006e80).stop 2021-11-21T21:57:38.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- 172.21.2.116:0/1503366140 shutdown_connections 2021-11-21T21:57:38.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 --2- 172.21.2.116:0/1503366140 >> v2:172.21.2.117:3300/0 conn(0x7fb8b00af7e0 0x7fb8b013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:38.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 --2- 172.21.2.116:0/1503366140 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00ad780 0x7fb8b005f730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:38.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- 172.21.2.116:0/1503366140 >> 172.21.2.116:0/1503366140 conn(0x7fb8b00b6000 msgr2=0x7fb8b00b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:38.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- 172.21.2.116:0/1503366140 shutdown_connections 2021-11-21T21:57:38.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- 172.21.2.116:0/1503366140 wait complete. 2021-11-21T21:57:38.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 Processor -- start 2021-11-21T21:57:38.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- start start 2021-11-21T21:57:38.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb8b00ad780 0x7fb8b009df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:38.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b00af7e0 0x7fb8b00b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:38.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00affc0 0x7fb8b01512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:38.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00671b0 con 0x7fb8b00ad780 2021-11-21T21:57:38.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b009e680 con 0x7fb8b00af7e0 2021-11-21T21:57:38.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b009e8e0 con 0x7fb8b00affc0 2021-11-21T21:57:38.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb8b00ad780 0x7fb8b009df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:38.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb8b00ad780 0x7fb8b009df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47250/0 (socket says 172.21.2.116:47250) 2021-11-21T21:57:38.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 -- 172.21.2.116:0/2272405987 learned_addr learned my addr 172.21.2.116:0/2272405987 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:38.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 -- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00affc0 msgr2=0x7fb8b01512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:38.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00affc0 0x7fb8b01512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:38.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 -- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b00af7e0 msgr2=0x7fb8b00b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:38.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b00af7e0 0x7fb8b00b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:38.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 -- 172.21.2.116:0/2272405987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8009470 con 0x7fb8b00ad780 2021-11-21T21:57:38.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8aeffd700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb8b00ad780 0x7fb8b009df50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fb8a4002f70 tx=0x7fb8a4006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:38.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb89ffff700 1 -- 172.21.2.116:0/2272405987 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a4020070 con 0x7fb8b00ad780 2021-11-21T21:57:38.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb89ffff700 1 -- 172.21.2.116:0/2272405987 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a4000e90 con 0x7fb8b00ad780 2021-11-21T21:57:38.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a400d410 con 0x7fb8b00ad780 2021-11-21T21:57:38.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb89ffff700 1 -- 172.21.2.116:0/2272405987 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a4019800 con 0x7fb8b00ad780 2021-11-21T21:57:38.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.008+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b0067680 con 0x7fb8b00ad780 2021-11-21T21:57:38.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.012+0000 7fb89ffff700 1 -- 172.21.2.116:0/2272405987 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb8a402c070 con 0x7fb8b00ad780 2021-11-21T21:57:38.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.012+0000 7fb89ffff700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb88c0536a0 0x7fb88c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:38.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.012+0000 7fb89ffff700 1 -- 172.21.2.116:0/2272405987 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5803+0+0 (secure 0 0 0) 0x7fb8a406f1c0 con 0x7fb8b00ad780 2021-11-21T21:57:38.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.012+0000 7fb8ae7fc700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb88c0536a0 0x7fb88c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:38.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.012+0000 7fb8ae7fc700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb88c0536a0 0x7fb88c055b60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb8b0068820 tx=0x7fb8a0007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:38.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.016+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b0067840 con 0x7fb8b00ad780 2021-11-21T21:57:38.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.016+0000 7fb89ffff700 1 -- 172.21.2.116:0/2272405987 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8a4014080 con 0x7fb8b00ad780 2021-11-21T21:57:38.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.100+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8b00573d0 con 0x7fb8b00ad780 2021-11-21T21:57:38.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb89ffff700 1 -- 172.21.2.116:0/2272405987 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 173+0+0 (secure 0 0 0) 0x7fb8a4049dc0 con 0x7fb8b00ad780 2021-11-21T21:57:38.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb88c0536a0 msgr2=0x7fb88c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:38.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb88c0536a0 0x7fb88c055b60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb8b0068820 tx=0x7fb8a0007820).stop 2021-11-21T21:57:38.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb8b00ad780 msgr2=0x7fb8b009df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:38.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb8b00ad780 0x7fb8b009df50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fb8a4002f70 tx=0x7fb8a4006e80).stop 2021-11-21T21:57:38.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 shutdown_connections 2021-11-21T21:57:38.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb8b00affc0 0x7fb8b01512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:38.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb8b00af7e0 0x7fb8b00b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:38.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb88c0536a0 0x7fb88c055b60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:38.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 --2- 172.21.2.116:0/2272405987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb8b00ad780 0x7fb8b009df50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:38.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 >> 172.21.2.116:0/2272405987 conn(0x7fb8b00b6000 msgr2=0x7fb8b00b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:38.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 shutdown_connections 2021-11-21T21:57:38.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:38.956+0000 7fb8b5723700 1 -- 172.21.2.116:0/2272405987 wait complete. 2021-11-21T21:57:38.962 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T21:57:38.965 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T21:57:39.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a5669e700 1 Processor -- start 2021-11-21T21:57:39.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a5669e700 1 -- start start 2021-11-21T21:57:39.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a5669e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7a50144810 0x7f7a501478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a5669e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a5669e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a50147e80 con 0x7f7a500b1db0 2021-11-21T21:57:39.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a5669e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500b1490 con 0x7f7a50144810 2021-11-21T21:57:39.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a5669e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500af060 con 0x7f7a500abce0 2021-11-21T21:57:39.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a54c3b700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f7a500b1db0 0x7f7a500ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:58126/0 (socket says 172.21.2.116:58126) 2021-11-21T21:57:39.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a54c3b700 1 -- 172.21.2.116:0/888382722 learned_addr learned my addr 172.21.2.116:0/888382722 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:39.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4ffff700 1 --2- 172.21.2.116:0/888382722 >> v2:172.21.2.117:3300/0 conn(0x7f7a50144810 0x7f7a501478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4f7fe700 1 --2- 172.21.2.116:0/888382722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4f7fe700 1 -- 172.21.2.116:0/888382722 >> v2:172.21.2.117:3300/0 conn(0x7f7a50144810 msgr2=0x7f7a501478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:39.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4f7fe700 1 --2- 172.21.2.116:0/888382722 >> v2:172.21.2.117:3300/0 conn(0x7f7a50144810 0x7f7a501478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4f7fe700 1 -- 172.21.2.116:0/888382722 >> v1:172.21.2.117:6789/0 conn(0x7f7a500b1db0 legacy=0x7f7a500ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4f7fe700 1 -- 172.21.2.116:0/888382722 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500af5a0 con 0x7f7a500abce0 2021-11-21T21:57:39.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4f7fe700 1 --2- 172.21.2.116:0/888382722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b18c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7a44002f70 tx=0x7f7a44006df0).ready entity=mon.2 client_cookie=5a597902e6cae724 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:39.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4effd700 1 -- 172.21.2.116:0/888382722 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4400d570 con 0x7f7a500abce0 2021-11-21T21:57:39.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4effd700 1 -- 172.21.2.116:0/888382722 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a44000e90 con 0x7f7a500abce0 2021-11-21T21:57:39.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.016+0000 7f7a4effd700 1 -- 172.21.2.116:0/888382722 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4400d8c0 con 0x7f7a500abce0 2021-11-21T21:57:39.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/888382722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 msgr2=0x7f7a500b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 --2- 172.21.2.116:0/888382722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b18c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7a44002f70 tx=0x7f7a44006df0).stop 2021-11-21T21:57:39.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/888382722 shutdown_connections 2021-11-21T21:57:39.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 --2- 172.21.2.116:0/888382722 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b18c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 --2- 172.21.2.116:0/888382722 >> v2:172.21.2.117:3300/0 conn(0x7f7a50144810 0x7f7a501478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/888382722 >> 172.21.2.116:0/888382722 conn(0x7f7a5005bd20 msgr2=0x7f7a5005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:39.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/888382722 shutdown_connections 2021-11-21T21:57:39.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/888382722 wait complete. 2021-11-21T21:57:39.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 Processor -- start 2021-11-21T21:57:39.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- start start 2021-11-21T21:57:39.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a50144810 0x7f7a50156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a5013c8c0 0x7f7a501573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5013cce0 con 0x7f7a50144810 2021-11-21T21:57:39.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5013c0a0 con 0x7f7a5013c8c0 2021-11-21T21:57:39.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50145160 con 0x7f7a500abce0 2021-11-21T21:57:39.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a50144810 0x7f7a50156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a50144810 0x7f7a50156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47266/0 (socket says 172.21.2.116:47266) 2021-11-21T21:57:39.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 -- 172.21.2.116:0/190831273 learned_addr learned my addr 172.21.2.116:0/190831273 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:39.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a54c3b700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a5013c8c0 0x7f7a501573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4ffff700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 -- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 msgr2=0x7f7a500b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 -- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a5013c8c0 msgr2=0x7f7a501573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a5013c8c0 0x7f7a501573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 -- 172.21.2.116:0/190831273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4400b180 con 0x7f7a50144810 2021-11-21T21:57:39.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a54c3b700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a5013c8c0 0x7f7a501573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:57:39.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4f7fe700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a50144810 0x7f7a50156ec0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f7a5013d1a0 tx=0x7f7a44014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:39.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4d7fa700 1 -- 172.21.2.116:0/190831273 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a44013e50 con 0x7f7a50144810 2021-11-21T21:57:39.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4d7fa700 1 -- 172.21.2.116:0/190831273 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4401e400 con 0x7f7a50144810 2021-11-21T21:57:39.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5013ced0 con 0x7f7a50144810 2021-11-21T21:57:39.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a50156a20 con 0x7f7a50144810 2021-11-21T21:57:39.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4d7fa700 1 -- 172.21.2.116:0/190831273 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4401e760 con 0x7f7a50144810 2021-11-21T21:57:39.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a500af300 con 0x7f7a50144810 2021-11-21T21:57:39.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4d7fa700 1 -- 172.21.2.116:0/190831273 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7a44025030 con 0x7f7a50144810 2021-11-21T21:57:39.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4d7fa700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a30053640 0x7f7a30055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.020+0000 7f7a4d7fa700 1 -- 172.21.2.116:0/190831273 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7a4406c580 con 0x7f7a50144810 2021-11-21T21:57:39.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.024+0000 7f7a4ffff700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a30053640 0x7f7a30055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.024+0000 7f7a4ffff700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a30053640 0x7f7a30055b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f7a4000b000 tx=0x7f7a40008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:39.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.024+0000 7f7a4d7fa700 1 -- 172.21.2.116:0/190831273 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a44043920 con 0x7f7a50144810 2021-11-21T21:57:39.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.108+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7a5005caf0 con 0x7f7a50144810 2021-11-21T21:57:39.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a4d7fa700 1 -- 172.21.2.116:0/190831273 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a440230a0 con 0x7f7a50144810 2021-11-21T21:57:39.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a30053640 msgr2=0x7f7a30055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a30053640 0x7f7a30055b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f7a4000b000 tx=0x7f7a40008040).stop 2021-11-21T21:57:39.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a50144810 msgr2=0x7f7a50156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a50144810 0x7f7a50156ec0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f7a5013d1a0 tx=0x7f7a44014000).stop 2021-11-21T21:57:39.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 shutdown_connections 2021-11-21T21:57:39.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a5013c8c0 0x7f7a501573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a30053640 0x7f7a30055b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a50144810 0x7f7a50156ec0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 --2- 172.21.2.116:0/190831273 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a500abce0 0x7f7a500b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 >> 172.21.2.116:0/190831273 conn(0x7f7a5005bd20 msgr2=0x7f7a500b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:39.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 shutdown_connections 2021-11-21T21:57:39.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.116+0000 7f7a5669e700 1 -- 172.21.2.116:0/190831273 wait complete. 2021-11-21T21:57:39.124 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2021-11-21T21:57:39.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.172+0000 7f46f5b4e700 1 Processor -- start 2021-11-21T21:57:39.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.172+0000 7f46f5b4e700 1 -- start start 2021-11-21T21:57:39.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f46f0147ca0 0x7f46f00ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f005c440 con 0x7f46f005c060 2021-11-21T21:57:39.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f00aeee0 con 0x7f46f005cb00 2021-11-21T21:57:39.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f00b17d0 con 0x7f46f0147ca0 2021-11-21T21:57:39.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f46f005c060 0x7f46f0147590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59228/0 (socket says 172.21.2.116:59228) 2021-11-21T21:57:39.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 -- 172.21.2.116:0/1855287091 learned_addr learned my addr 172.21.2.116:0/1855287091 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:39.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693552909 0 0) 0x7f46f005c440 con 0x7f46f005c060 2021-11-21T21:57:39.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d0003260 con 0x7f46f005c060 2021-11-21T21:57:39.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46eeffd700 1 --2- 172.21.2.116:0/1855287091 >> v2:172.21.2.117:3300/0 conn(0x7f46f0147ca0 0x7f46f00ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905432125 0 0) 0x7f46d0003260 con 0x7f46f005c060 2021-11-21T21:57:39.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 >> v2:172.21.2.117:3300/0 conn(0x7f46f0147ca0 msgr2=0x7f46f00ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:39.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 --2- 172.21.2.116:0/1855287091 >> v2:172.21.2.117:3300/0 conn(0x7f46f0147ca0 0x7f46f00ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 >> v1:172.21.2.117:6789/0 conn(0x7f46f005cb00 legacy=0x7f46f00ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:39.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0067a50 con 0x7f46f005c060 2021-11-21T21:57:39.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f46d8001660 con 0x7f46f005c060 2021-11-21T21:57:39.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 >> v1:172.21.2.116:6789/0 conn(0x7f46f005c060 legacy=0x7f46f0147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46d0003750 0x7f46d0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46d0004500 0x7f46d0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0004ec0 0x7f46d000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00b17d0 con 0x7f46d0003750 2021-11-21T21:57:39.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d000bda0 con 0x7f46d0004ec0 2021-11-21T21:57:39.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d000bf60 con 0x7f46d0004500 2021-11-21T21:57:39.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46d0003750 0x7f46d0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 -- 172.21.2.116:0/1855287091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46d0004500 msgr2=0x7f46d0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:39.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46d0004500 0x7f46d0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:39.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 -- 172.21.2.116:0/1855287091 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0004ec0 msgr2=0x7f46d000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:39.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0004ec0 0x7f46d000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:39.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 -- 172.21.2.116:0/1855287091 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d0003260 con 0x7f46d0003750 2021-11-21T21:57:39.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 -- 172.21.2.116:0/1855287091 reap_dead start 2021-11-21T21:57:39.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46d0003750 0x7f46d0003b70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f46f005f060 tx=0x7f46d8001e30).ready entity=mon.0 client_cookie=89307c8af7c17972 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:39.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46d80071a0 con 0x7f46d0003750 2021-11-21T21:57:39.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8007300 con 0x7f46d0003750 2021-11-21T21:57:39.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ee7fc700 1 -- 172.21.2.116:0/1855287091 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46d800c8f0 con 0x7f46d0003750 2021-11-21T21:57:39.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1855287091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46d0003750 msgr2=0x7f46d0003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46d0003750 0x7f46d0003b70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f46f005f060 tx=0x7f46d8001e30).stop 2021-11-21T21:57:39.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1855287091 shutdown_connections 2021-11-21T21:57:39.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1855287091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46d0003750 0x7f46d0003b70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1855287091 >> 172.21.2.116:0/1855287091 conn(0x7f46f00b4f10 msgr2=0x7f46f00b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:39.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1855287091 shutdown_connections 2021-11-21T21:57:39.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1855287091 wait complete. 2021-11-21T21:57:39.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 Processor -- start 2021-11-21T21:57:39.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- start start 2021-11-21T21:57:39.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0003750 0x7f46f00b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46f014a7b0 0x7f46f0156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46f0159ff0 0x7f46f0066330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f0065bf0 con 0x7f46f0159ff0 2021-11-21T21:57:39.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f0157350 con 0x7f46d0003750 2021-11-21T21:57:39.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46f5b4e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00671b0 con 0x7f46f014a7b0 2021-11-21T21:57:39.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46f0159ff0 0x7f46f0066330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46f0159ff0 0x7f46f0066330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47288/0 (socket says 172.21.2.116:47288) 2021-11-21T21:57:39.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 -- 172.21.2.116:0/1868533421 learned_addr learned my addr 172.21.2.116:0/1868533421 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:39.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ef7fe700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0003750 0x7f46f00b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46eeffd700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46f014a7b0 0x7f46f0156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 -- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46f014a7b0 msgr2=0x7f46f0156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46f014a7b0 0x7f46f0156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 -- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0003750 msgr2=0x7f46f00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0003750 0x7f46f00b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 -- 172.21.2.116:0/1868533421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0067450 con 0x7f46f0159ff0 2021-11-21T21:57:39.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46effff700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46f0159ff0 0x7f46f0066330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f46f00b2ab0 tx=0x7f46e8007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:39.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.176+0000 7f46ecff9700 1 -- 172.21.2.116:0/1868533421 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e8005880 con 0x7f46f0159ff0 2021-11-21T21:57:39.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ecff9700 1 -- 172.21.2.116:0/1868533421 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e80059e0 con 0x7f46f0159ff0 2021-11-21T21:57:39.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ecff9700 1 -- 172.21.2.116:0/1868533421 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e8005d40 con 0x7f46f0159ff0 2021-11-21T21:57:39.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f01598b0 con 0x7f46f0159ff0 2021-11-21T21:57:39.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f01497f0 con 0x7f46f0159ff0 2021-11-21T21:57:39.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f00b0df0 con 0x7f46f0159ff0 2021-11-21T21:57:39.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ecff9700 1 -- 172.21.2.116:0/1868533421 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f46e801c070 con 0x7f46f0159ff0 2021-11-21T21:57:39.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ecff9700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46d4053580 0x7f46d4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:39.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ecff9700 1 -- 172.21.2.116:0/1868533421 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f46e806a8e0 con 0x7f46f0159ff0 2021-11-21T21:57:39.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ef7fe700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46d4053580 0x7f46d4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:39.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ef7fe700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46d4053580 0x7f46d4055a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f46d8018000 tx=0x7f46d8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:39.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.180+0000 7f46ecff9700 1 -- 172.21.2.116:0/1868533421 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46e8041cd0 con 0x7f46f0159ff0 2021-11-21T21:57:39.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.296+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f46f00573d0 con 0x7f46f0159ff0 2021-11-21T21:57:39.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46ecff9700 1 -- 172.21.2.116:0/1868533421 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v141) v1 ==== 144+0+0 (secure 0 0 0) 0x7f46e80452f0 con 0x7f46f0159ff0 2021-11-21T21:57:39.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46d4053580 msgr2=0x7f46d4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46d4053580 0x7f46d4055a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f46d8018000 tx=0x7f46d8015000).stop 2021-11-21T21:57:39.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46f0159ff0 msgr2=0x7f46f0066330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:39.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46f0159ff0 0x7f46f0066330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f46f00b2ab0 tx=0x7f46e8007da0).stop 2021-11-21T21:57:39.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 shutdown_connections 2021-11-21T21:57:39.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46d4053580 0x7f46d4055a40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46f0159ff0 0x7f46f0066330 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46f014a7b0 0x7f46f0156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 --2- 172.21.2.116:0/1868533421 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46d0003750 0x7f46f00b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:39.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 >> 172.21.2.116:0/1868533421 conn(0x7f46f00b4f10 msgr2=0x7f46f0144530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:39.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 shutdown_connections 2021-11-21T21:57:39.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:39.960+0000 7f46f5b4e700 1 -- 172.21.2.116:0/1868533421 wait complete. 2021-11-21T21:57:39.970 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-11-21T21:57:39.970 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2021-11-21T21:57:39.971 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2021-11-21T21:57:40.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 Processor -- start 2021-11-21T21:57:40.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- start start 2021-11-21T21:57:40.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfcc0aff40 0x7fbfcc05fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:40.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbfcc142dc0 0x7fbfcc0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:40.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfcc13d3f0 con 0x7fbfcc05f730 2021-11-21T21:57:40.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfcc143540 con 0x7fbfcc0aff40 2021-11-21T21:57:40.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfcc1437e0 con 0x7fbfcc142dc0 2021-11-21T21:57:40.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfcc0aff40 0x7fbfcc05fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:40.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfcc0aff40 0x7fbfcc05fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47292/0 (socket says 172.21.2.116:47292) 2021-11-21T21:57:40.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/555796826 learned_addr learned my addr 172.21.2.116:0/555796826 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:40.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/555796826 >> v2:172.21.2.117:3300/0 conn(0x7fbfcc142dc0 msgr2=0x7fbfcc0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:40.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 --2- 172.21.2.116:0/555796826 >> v2:172.21.2.117:3300/0 conn(0x7fbfcc142dc0 0x7fbfcc0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:40.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/555796826 >> v1:172.21.2.117:6789/0 conn(0x7fbfcc05f730 legacy=0x7fbfcc13cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:40.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/555796826 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc143a80 con 0x7fbfcc0aff40 2021-11-21T21:57:40.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcb7fe700 1 --2- 172.21.2.116:0/555796826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc05fb10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbfcc0b6a60 tx=0x7fbfc0006f40).ready entity=mon.0 client_cookie=36dc2e7e0770a5fe server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:40.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcaffd700 1 -- 172.21.2.116:0/555796826 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc000dc30 con 0x7fbfcc0aff40 2021-11-21T21:57:40.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcaffd700 1 -- 172.21.2.116:0/555796826 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc0000e90 con 0x7fbfcc0aff40 2021-11-21T21:57:40.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfcaffd700 1 -- 172.21.2.116:0/555796826 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc000be90 con 0x7fbfcc0aff40 2021-11-21T21:57:40.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- 172.21.2.116:0/555796826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 msgr2=0x7fbfcc05fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:40.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 --2- 172.21.2.116:0/555796826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc05fb10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbfcc0b6a60 tx=0x7fbfc0006f40).stop 2021-11-21T21:57:40.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- 172.21.2.116:0/555796826 shutdown_connections 2021-11-21T21:57:40.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 --2- 172.21.2.116:0/555796826 >> v2:172.21.2.117:3300/0 conn(0x7fbfcc142dc0 0x7fbfcc0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:40.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 --2- 172.21.2.116:0/555796826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc05fb10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:40.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- 172.21.2.116:0/555796826 >> 172.21.2.116:0/555796826 conn(0x7fbfcc0b5ea0 msgr2=0x7fbfcc0b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:40.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- 172.21.2.116:0/555796826 shutdown_connections 2021-11-21T21:57:40.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.020+0000 7fbfd105e700 1 -- 172.21.2.116:0/555796826 wait complete. 2021-11-21T21:57:40.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 Processor -- start 2021-11-21T21:57:40.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 -- start start 2021-11-21T21:57:40.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfcc05f730 0x7fbfcc063da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:40.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:40.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfcc142dc0 0x7fbfcc13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:40.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc0678d0 con 0x7fbfcc0aff40 2021-11-21T21:57:40.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc140400 con 0x7fbfcc05f730 2021-11-21T21:57:40.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc0655c0 con 0x7fbfcc142dc0 2021-11-21T21:57:40.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:40.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc13fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47300/0 (socket says 172.21.2.116:47300) 2021-11-21T21:57:40.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/2161829294 learned_addr learned my addr 172.21.2.116:0/2161829294 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:40.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfcc142dc0 msgr2=0x7fbfcc13ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:40.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfcc142dc0 0x7fbfcc13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:40.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfcc05f730 msgr2=0x7fbfcc063da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:40.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfcc05f730 0x7fbfcc063da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:40.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 -- 172.21.2.116:0/2161829294 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc000b460 con 0x7fbfcc0aff40 2021-11-21T21:57:40.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfcb7fe700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc13fa20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbfcc140540 tx=0x7fbfcc0afe60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:40.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfc97fa700 1 -- 172.21.2.116:0/2161829294 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc0010d20 con 0x7fbfcc0aff40 2021-11-21T21:57:40.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfc97fa700 1 -- 172.21.2.116:0/2161829294 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc0010e80 con 0x7fbfcc0aff40 2021-11-21T21:57:40.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfc97fa700 1 -- 172.21.2.116:0/2161829294 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc0024640 con 0x7fbfcc0aff40 2021-11-21T21:57:40.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfcc13ec80 con 0x7fbfcc0aff40 2021-11-21T21:57:40.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc065f30 con 0x7fbfcc0aff40 2021-11-21T21:57:40.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfcc13ea40 con 0x7fbfcc0aff40 2021-11-21T21:57:40.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfc97fa700 1 -- 172.21.2.116:0/2161829294 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbfc002d070 con 0x7fbfcc0aff40 2021-11-21T21:57:40.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfc97fa700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfac0534f0 0x7fbfac0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:40.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.024+0000 7fbfc97fa700 1 -- 172.21.2.116:0/2161829294 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fbfc006f020 con 0x7fbfcc0aff40 2021-11-21T21:57:40.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.028+0000 7fbfcbfff700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfac0534f0 0x7fbfac0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:40.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.028+0000 7fbfcbfff700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfac0534f0 0x7fbfac0559b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fbfbc002f90 tx=0x7fbfbc0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:40.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.028+0000 7fbfc97fa700 1 -- 172.21.2.116:0/2161829294 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfcc13ea40 con 0x7fbfcc0aff40 2021-11-21T21:57:40.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.140+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7fbfcc13ea40 con 0x7fbfcc0aff40 2021-11-21T21:57:40.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfc97fa700 1 -- 172.21.2.116:0/2161829294 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v142) v1 ==== 155+0+0 (secure 0 0 0) 0x7fbfcc13ea40 con 0x7fbfcc0aff40 2021-11-21T21:57:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfac0534f0 msgr2=0x7fbfac0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfac0534f0 0x7fbfac0559b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fbfbc002f90 tx=0x7fbfbc0069b0).stop 2021-11-21T21:57:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 msgr2=0x7fbfcc13fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc13fa20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbfcc140540 tx=0x7fbfcc0afe60).stop 2021-11-21T21:57:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 shutdown_connections 2021-11-21T21:57:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfcc142dc0 0x7fbfcc13ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbfac0534f0 0x7fbfac0559b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfcc0aff40 0x7fbfcc13fa20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 --2- 172.21.2.116:0/2161829294 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfcc05f730 0x7fbfcc063da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 >> 172.21.2.116:0/2161829294 conn(0x7fbfcc0b5ea0 msgr2=0x7fbfcc0ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 shutdown_connections 2021-11-21T21:57:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:40.968+0000 7fbfd105e700 1 -- 172.21.2.116:0/2161829294 wait complete. 2021-11-21T21:57:40.977 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T21:57:41.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.028+0000 7f6714a7e700 1 Processor -- start 2021-11-21T21:57:41.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.028+0000 7f6714a7e700 1 -- start start 2021-11-21T21:57:41.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 0x7f671005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f67100af620 0x7f6710142f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f67100afa80 0x7f6710143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67100b06c0 con 0x7f67100afa80 2021-11-21T21:57:41.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67100adf20 con 0x7f67100af620 2021-11-21T21:57:41.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67100b2470 con 0x7f67100b02e0 2021-11-21T21:57:41.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f67100afa80 0x7f6710143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f67100afa80 0x7f6710143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47310/0 (socket says 172.21.2.116:47310) 2021-11-21T21:57:41.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 -- 172.21.2.116:0/1657648302 learned_addr learned my addr 172.21.2.116:0/1657648302 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 -- 172.21.2.116:0/1657648302 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 msgr2=0x7f671005f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:41.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 --2- 172.21.2.116:0/1657648302 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 0x7f671005f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 -- 172.21.2.116:0/1657648302 >> v2:172.21.2.117:3300/0 conn(0x7f67100af620 msgr2=0x7f6710142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:41.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 --2- 172.21.2.116:0/1657648302 >> v2:172.21.2.117:3300/0 conn(0x7f67100af620 0x7f6710142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 -- 172.21.2.116:0/1657648302 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100b26d0 con 0x7f67100afa80 2021-11-21T21:57:41.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670ed9d700 1 --2- 172.21.2.116:0/1657648302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710143460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f67080032f0 tx=0x7f6708009290).ready entity=mon.0 client_cookie=a595d55f09820180 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670d59a700 1 -- 172.21.2.116:0/1657648302 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670800b870 con 0x7f67100afa80 2021-11-21T21:57:41.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670d59a700 1 -- 172.21.2.116:0/1657648302 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670800b9d0 con 0x7f67100afa80 2021-11-21T21:57:41.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670d59a700 1 -- 172.21.2.116:0/1657648302 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670800bb40 con 0x7f67100afa80 2021-11-21T21:57:41.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- 172.21.2.116:0/1657648302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 msgr2=0x7f6710143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- 172.21.2.116:0/1657648302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710143460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f67080032f0 tx=0x7f6708009290).stop 2021-11-21T21:57:41.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- 172.21.2.116:0/1657648302 shutdown_connections 2021-11-21T21:57:41.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- 172.21.2.116:0/1657648302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710143460 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- 172.21.2.116:0/1657648302 >> v2:172.21.2.117:3300/0 conn(0x7f67100af620 0x7f6710142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- 172.21.2.116:0/1657648302 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 0x7f671005f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- 172.21.2.116:0/1657648302 >> 172.21.2.116:0/1657648302 conn(0x7f67100b5ff0 msgr2=0x7f67100b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- 172.21.2.116:0/1657648302 shutdown_connections 2021-11-21T21:57:41.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- 172.21.2.116:0/1657648302 wait complete. 2021-11-21T21:57:41.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 Processor -- start 2021-11-21T21:57:41.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- start start 2021-11-21T21:57:41.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f67100af620 0x7f67100663a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 0x7f6710146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710148d60 con 0x7f67100afa80 2021-11-21T21:57:41.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710148f90 con 0x7f67100af620 2021-11-21T21:57:41.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6710149230 con 0x7f67100b02e0 2021-11-21T21:57:41.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47316/0 (socket says 172.21.2.116:47316) 2021-11-21T21:57:41.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 -- 172.21.2.116:0/2162733367 learned_addr learned my addr 172.21.2.116:0/2162733367 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 -- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 msgr2=0x7f6710146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 0x7f6710146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 -- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f67100af620 msgr2=0x7f67100663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f67100af620 0x7f67100663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 -- 172.21.2.116:0/2162733367 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6708009530 con 0x7f67100afa80 2021-11-21T21:57:41.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f670dd9b700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710146920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f6700002f70 tx=0x7f6700006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f67077fe700 1 -- 172.21.2.116:0/2162733367 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6700013460 con 0x7f67100afa80 2021-11-21T21:57:41.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f67077fe700 1 -- 172.21.2.116:0/2162733367 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67000135c0 con 0x7f67100afa80 2021-11-21T21:57:41.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f67077fe700 1 -- 172.21.2.116:0/2162733367 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6700013920 con 0x7f67100afa80 2021-11-21T21:57:41.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67101494d0 con 0x7f67100afa80 2021-11-21T21:57:41.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.032+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6710146620 con 0x7f67100afa80 2021-11-21T21:57:41.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.036+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67100b0da0 con 0x7f67100afa80 2021-11-21T21:57:41.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.036+0000 7f67077fe700 1 -- 172.21.2.116:0/2162733367 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f670000f070 con 0x7f67100afa80 2021-11-21T21:57:41.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.036+0000 7f67077fe700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f66f4053580 0x7f66f4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.036+0000 7f67077fe700 1 -- 172.21.2.116:0/2162733367 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f670006b0b0 con 0x7f67100afa80 2021-11-21T21:57:41.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.036+0000 7f670e59c700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f66f4053580 0x7f66f4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.036+0000 7f670e59c700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f66f4053580 0x7f66f4055a40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f66f8002f70 tx=0x7f66f8006960).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.036+0000 7f67077fe700 1 -- 172.21.2.116:0/2162733367 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6700042390 con 0x7f67100afa80 2021-11-21T21:57:41.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.120+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f67100573d0 con 0x7f67100afa80 2021-11-21T21:57:41.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f67077fe700 1 -- 172.21.2.116:0/2162733367 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f67000459b0 con 0x7f67100afa80 2021-11-21T21:57:41.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f66f4053580 msgr2=0x7f66f4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f66f4053580 0x7f66f4055a40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f66f8002f70 tx=0x7f66f8006960).stop 2021-11-21T21:57:41.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 msgr2=0x7f6710146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710146920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f6700002f70 tx=0x7f6700006ec0).stop 2021-11-21T21:57:41.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 shutdown_connections 2021-11-21T21:57:41.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f67100b02e0 0x7f6710146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f66f4053580 0x7f66f4055a40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f67100afa80 0x7f6710146920 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 --2- 172.21.2.116:0/2162733367 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f67100af620 0x7f67100663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 >> 172.21.2.116:0/2162733367 conn(0x7f67100b5ff0 msgr2=0x7f67100b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 shutdown_connections 2021-11-21T21:57:41.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.128+0000 7f6714a7e700 1 -- 172.21.2.116:0/2162733367 wait complete. 2021-11-21T21:57:41.135 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:57:41.137 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:41.145 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:41.154 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:41.166 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:57:41.181 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:57:41.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:41.183 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:57:41.183 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:41.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:57:41.188 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T21:57:41.188 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T21:57:41.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:41.189 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:57:41.190 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:57:41.227 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:57:41.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:57:41.229 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:57:41.229 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:57:41.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 Processor -- start 2021-11-21T21:57:41.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- start start 2021-11-21T21:57:41.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fca8c0a86b0 0x7fca8c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c0a6e80 con 0x7fca8c062410 2021-11-21T21:57:41.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c14a7b0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0a8bc0 con 0x7fca8c0a86b0 2021-11-21T21:57:41.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fca8c0a86b0 0x7fca8c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fca8c0a86b0 0x7fca8c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47324/0 (socket says 172.21.2.116:47324) 2021-11-21T21:57:41.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/62686033 learned_addr learned my addr 172.21.2.116:0/62686033 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca90832700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fca8c062410 0x7fca8c1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59282/0 (socket says 172.21.2.116:59282) 2021-11-21T21:57:41.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/62686033 >> v1:172.21.2.117:6789/0 conn(0x7fca8c0a6aa0 legacy=0x7fca8c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:41.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/62686033 >> v1:172.21.2.116:6789/0 conn(0x7fca8c062410 legacy=0x7fca8c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/62686033 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c14aa10 con 0x7fca8c0a86b0 2021-11-21T21:57:41.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- 172.21.2.116:0/62686033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a86b0 0x7fca8c14b150 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fca8c061b50 tx=0x7fca80006eb0).ready entity=mon.0 client_cookie=2e63bf0b7398ceb5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8affd700 1 -- 172.21.2.116:0/62686033 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca8000d610 con 0x7fca8c0a86b0 2021-11-21T21:57:41.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8affd700 1 -- 172.21.2.116:0/62686033 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80000e90 con 0x7fca8c0a86b0 2021-11-21T21:57:41.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8affd700 1 -- 172.21.2.116:0/62686033 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca8000d960 con 0x7fca8c0a86b0 2021-11-21T21:57:41.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- 172.21.2.116:0/62686033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a86b0 msgr2=0x7fca8c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 --2- 172.21.2.116:0/62686033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a86b0 0x7fca8c14b150 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fca8c061b50 tx=0x7fca80006eb0).stop 2021-11-21T21:57:41.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- 172.21.2.116:0/62686033 shutdown_connections 2021-11-21T21:57:41.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 --2- 172.21.2.116:0/62686033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a86b0 0x7fca8c14b150 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- 172.21.2.116:0/62686033 >> 172.21.2.116:0/62686033 conn(0x7fca8c05bd20 msgr2=0x7fca8c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- 172.21.2.116:0/62686033 shutdown_connections 2021-11-21T21:57:41.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- 172.21.2.116:0/62686033 wait complete. 2021-11-21T21:57:41.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 Processor -- start 2021-11-21T21:57:41.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- start start 2021-11-21T21:57:41.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca8c062410 0x7fca8c0648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a6aa0 0x7fca8c146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca8c0a86b0 0x7fca8c146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c066ed0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c065420 con 0x7fca8c0a86b0 2021-11-21T21:57:41.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b0e10 con 0x7fca8c062410 2021-11-21T21:57:41.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a6aa0 0x7fca8c146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a6aa0 0x7fca8c146600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47330/0 (socket says 172.21.2.116:47330) 2021-11-21T21:57:41.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/872926689 learned_addr learned my addr 172.21.2.116:0/872926689 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca8c062410 msgr2=0x7fca8c0648c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:41.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca8c062410 0x7fca8c0648c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca8c0a86b0 msgr2=0x7fca8c146af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:41.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca8c0a86b0 0x7fca8c146af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 -- 172.21.2.116:0/872926689 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8000b1e0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca8b7fe700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a6aa0 0x7fca8c146600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fca8c063e20 tx=0x7fca80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca897fa700 1 -- 172.21.2.116:0/872926689 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca800118f0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca897fa700 1 -- 172.21.2.116:0/872926689 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80011a50 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c0b08d0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca897fa700 1 -- 172.21.2.116:0/872926689 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca80011db0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.292+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c0655f0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.296+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c1462f0 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.296+0000 7fca897fa700 1 -- 172.21.2.116:0/872926689 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fca80025030 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.296+0000 7fca897fa700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca6c053660 0x7fca6c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.296+0000 7fca897fa700 1 -- 172.21.2.116:0/872926689 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fca8006cb30 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.296+0000 7fca8bfff700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca6c053660 0x7fca6c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.296+0000 7fca8bfff700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca6c053660 0x7fca6c055b20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fca7c002f90 tx=0x7fca7c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.296+0000 7fca897fa700 1 -- 172.21.2.116:0/872926689 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca80043e10 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.380+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fca8c05db70 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca897fa700 1 -- 172.21.2.116:0/872926689 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v47) v1 ==== 81+0+28977 (secure 0 0 0) 0x7fca80023020 con 0x7fca8c0a6aa0 2021-11-21T21:57:41.384 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:41.385 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:57:41.385 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 47, 2021-11-21T21:57:41.385 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:57:41.385 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:57:41.385 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:57:38.944838+0000", 2021-11-21T21:57:41.386 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:57:41.386 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:57:41.386 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:57:41.386 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:57:41.387 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:57:41.387 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:57:41.387 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:57:41.387 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:57:41.388 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:57:41.388 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:41.388 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:57:41.388 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:57:41.388 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:57:41.389 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:57:41.389 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:57:41.389 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 8, 2021-11-21T21:57:41.389 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:57:41.390 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:57:41.390 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:57:41.390 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:57:41.390 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:57:41.390 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.391 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:57:41.391 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:57:41.391 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:57:41.391 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:57:41.392 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:57:41.392 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:57:41.392 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:57:41.392 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:57:41.392 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:57:41.393 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:57:41.393 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:57:41.393 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:57:41.393 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:57:41.394 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:57:41.394 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:57:41.394 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:57:41.394 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:57:41.394 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:57:41.395 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:57:41.395 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:57:41.395 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:57:41.395 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:57:41.396 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:57:41.396 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.396 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:57:41.396 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:57:41.397 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:57:41.397 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:57:41.397 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:57:41.397 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:57:41.398 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:57:41.398 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:57:41.398 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:57:41.398 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:57:41.398 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:57:41.399 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:57:41.399 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:57:41.399 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:57:41.399 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:57:41.400 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:57:41.400 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:57:41.400 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:57:41.400 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:57:41.400 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:57:41.401 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:57:41.401 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:57:41.401 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:57:41.401 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:57:41.401 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:57:41.402 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:57:41.402 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:57:41.402 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.402 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:57:41.403 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:57:41.403 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:57:41.403 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:57:41.403 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:57:41.404 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:57:41.404 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:57:41.404 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:57:41.404 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:57:41.404 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:57:41.405 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:57:41.405 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:57:41.405 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:57:41.405 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.405 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:57:41.406 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:57:41.406 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.406 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.406 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.407 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:57:41.407 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:57:41.407 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:57:41.407 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:57:41.407 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:57:41.408 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:57:41.408 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:57:41.408 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:57:41.408 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:57:41.408 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:57:41.409 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:57:41.409 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:57:41.409 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:57:41.409 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:57:41.410 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:57:41.410 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:57:41.410 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:57:41.410 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:57:41.410 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:57:41.411 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:57:41.411 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:57:41.411 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:57:41.411 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:57:41.412 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.412 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:57:41.412 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:57:41.412 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:57:41.412 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:57:41.413 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:57:41.413 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:57:41.413 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:57:41.413 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:57:41.414 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:57:41.414 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:57:41.414 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:57:41.414 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:57:41.415 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:57:41.415 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:57:41.415 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:57:41.415 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:57:41.415 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:57:41.416 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:57:41.416 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:57:41.416 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:57:41.416 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:57:41.417 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:57:41.417 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:57:41.417 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:57:41.417 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:57:41.417 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:57:41.418 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:57:41.418 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.418 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:57:41.418 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:57:41.419 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:57:41.419 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:57:41.419 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:57:41.419 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:57:41.419 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:57:41.420 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:57:41.420 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:57:41.420 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:57:41.420 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:57:41.421 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:57:41.421 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:57:41.421 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:57:41.421 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.421 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.422 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:41.422 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:57:41.422 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.422 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:57:41.423 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:57:41.423 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.423 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.423 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.423 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.424 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.424 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.424 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.425 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:41.425 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.425 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.425 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.425 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.426 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.426 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.426 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:57:41.426 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.427 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.427 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.427 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:57:41.427 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.428 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.428 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.428 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.428 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.428 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.429 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.429 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.429 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:57:41.429 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.430 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.430 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.430 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.430 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:57:41.430 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.431 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.431 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.431 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.431 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.432 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.432 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:57:41.432 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.433 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.433 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.433 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:57:41.433 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.434 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.434 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.434 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.434 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.435 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.435 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.435 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.435 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:57:41.436 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.436 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.436 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.436 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.436 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:57:41.437 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:57:41.437 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.437 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.437 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.438 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:57:41.438 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:57:41.438 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:57:41.438 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:57:41.438 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.439 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.439 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.439 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.439 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.440 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.440 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:57:41.440 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:57:41.440 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.440 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.441 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.441 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.441 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.441 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.442 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.442 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 41, 2021-11-21T21:57:41.442 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.442 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.442 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.443 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.443 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.443 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.443 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:57:41.443 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.444 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.444 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.444 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.444 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:57:41.445 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.445 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.445 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.445 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.445 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.446 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.446 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.446 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.446 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:57:41.446 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.447 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.447 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.447 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.447 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:57:41.448 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.448 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.448 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.448 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.448 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.449 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.449 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.449 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:57:41.449 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.450 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.450 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.450 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.450 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:57:41.451 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.451 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.451 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.451 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.451 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.452 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.452 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.452 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:57:41.453 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.453 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.453 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.453 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.453 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:57:41.454 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:57:41.454 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.454 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.454 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.455 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:57:41.455 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:57:41.455 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:57:41.455 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:57:41.455 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.456 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.456 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.456 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.456 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.456 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.457 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:57:41.457 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:57:41.457 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.457 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.458 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.458 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.458 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.458 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.458 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.459 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 27, 2021-11-21T21:57:41.459 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.459 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.459 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.460 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.460 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.460 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.460 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:57:41.460 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.461 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.461 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.461 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.461 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:57:41.461 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.462 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.462 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.462 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.462 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.463 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.463 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.463 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.463 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:57:41.463 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.464 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.464 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.464 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.464 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:57:41.464 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.465 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.465 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.465 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.465 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.465 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.466 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.466 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.466 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:57:41.466 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.467 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.467 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.467 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.467 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:57:41.467 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.468 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.468 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.468 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.468 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.469 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.469 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.469 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.469 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:57:41.469 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.470 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.470 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.470 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.470 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:57:41.470 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:57:41.471 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.471 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.471 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.471 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:57:41.472 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:57:41.472 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:57:41.472 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:57:41.472 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.472 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.473 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.473 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.473 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.473 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.473 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:57:41.474 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:57:41.474 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.474 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.474 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.475 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.475 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.475 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.475 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.475 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:41.476 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.476 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.476 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.476 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.476 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.477 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.477 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:57:41.477 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.477 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.478 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.478 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:57:41.478 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.478 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.479 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.479 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.479 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.479 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.480 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.480 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:57:41.480 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.480 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.481 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.481 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.481 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:57:41.481 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.481 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.482 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.482 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.482 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.482 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.482 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.483 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.483 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:57:41.483 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.483 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.484 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.484 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.484 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:57:41.484 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.484 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.485 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.485 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.485 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.485 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.485 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.486 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.486 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:57:41.486 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.486 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.487 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.487 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.487 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:57:41.487 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:57:41.487 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.488 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.488 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.488 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:57:41.488 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:57:41.489 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:57:41.489 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:57:41.489 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.489 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.489 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.490 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.490 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.490 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.490 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:57:41.490 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:57:41.491 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.491 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.491 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.491 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.492 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.492 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.492 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.492 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:41.492 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.493 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.493 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.493 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.493 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.494 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.494 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:57:41.494 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.494 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.494 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.495 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.495 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:57:41.495 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.495 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.495 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.496 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.496 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.496 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.496 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.497 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.497 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:57:41.497 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.497 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.497 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.498 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.498 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:57:41.498 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.498 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.499 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.499 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.499 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.499 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.500 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.500 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:57:41.500 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.500 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.501 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.501 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.501 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:57:41.501 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.501 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.502 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.502 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.502 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.502 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.503 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.503 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.503 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:57:41.503 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.503 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.504 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.504 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.504 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:57:41.504 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:57:41.505 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.505 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.505 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.505 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:57:41.505 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:57:41.506 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:57:41.506 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:57:41.506 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.506 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.506 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.507 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.507 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.507 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.507 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:57:41.508 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:57:41.508 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.508 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.508 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.508 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.509 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.509 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.509 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.509 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:41.509 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.510 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.510 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.510 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.511 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.511 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:57:41.511 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.511 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.512 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.512 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.512 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:57:41.512 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.512 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.513 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.513 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.513 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.513 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.513 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.514 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.514 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:57:41.514 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.514 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.515 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.515 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.515 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:57:41.515 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.515 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.516 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.516 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.516 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.516 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.516 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.517 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.517 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:57:41.517 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.517 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.518 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.518 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.518 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:57:41.518 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.518 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.519 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.519 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.519 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.519 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.519 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.520 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.520 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:57:41.520 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.520 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.521 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.521 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.521 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:57:41.521 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:57:41.521 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.522 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.522 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.522 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:57:41.522 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:57:41.522 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:57:41.523 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:57:41.523 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.523 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.523 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.524 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.524 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.524 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.524 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:57:41.524 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:57:41.525 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.525 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.525 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.525 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.526 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.526 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.526 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.526 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 39, 2021-11-21T21:57:41.527 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.527 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.527 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.528 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:57:41.528 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.528 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.529 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.529 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.529 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:57:41.529 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.529 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.530 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.530 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.530 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.530 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.531 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.531 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.531 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:57:41.531 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.531 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.532 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.532 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.532 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:57:41.532 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.532 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.533 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.533 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.533 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.533 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.534 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.534 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.534 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:57:41.534 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.534 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.535 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.535 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:57:41.535 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.536 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.536 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.536 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.536 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.536 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.537 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.537 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.537 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:57:41.537 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.537 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.538 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.538 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.538 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:57:41.538 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:57:41.539 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.539 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.539 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.539 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:57:41.539 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:57:41.540 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:57:41.540 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:57:41.540 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.540 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.540 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.541 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.541 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.541 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.541 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:57:41.542 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:57:41.542 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:57:41.542 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:57:41.542 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:57:41.542 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:57:41.543 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:57:41.543 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:57:41.543 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:57:41.543 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 38, 2021-11-21T21:57:41.543 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:57:41.544 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:57:41.544 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:57:41.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.544 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.545 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.545 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:57:41.545 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.545 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.546 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.546 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:57:41.546 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.546 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.547 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.547 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.547 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:57:41.547 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.547 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.548 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.548 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:57:41.548 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.548 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.548 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.549 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.549 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:57:41.549 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.549 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.549 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.550 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.550 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:57:41.550 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.550 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.551 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.551 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:57:41.551 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.551 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.551 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.552 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.552 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:57:41.552 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.552 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.553 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.553 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.553 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:57:41.553 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:57:41.553 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.554 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:57:41.554 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:57:41.554 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.554 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.554 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.555 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:57:41.555 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:57:41.555 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:57:41.555 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.556 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.556 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.556 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:57:41.556 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:57:41.556 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:57:41.557 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:57:41.557 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:57:41.557 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:57:41.557 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:57:41.557 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:41.558 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.558 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:41.558 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:57:41.558 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.559 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:57:41.559 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.559 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.559 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.559 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.560 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.560 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:57:41.560 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.560 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.561 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.561 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:57:41.561 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.561 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.561 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.562 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.562 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.562 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:57:41.562 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.563 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.563 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.563 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:57:41.563 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.563 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.564 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.564 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.564 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.564 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:57:41.564 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.565 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.565 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.565 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:57:41.565 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.566 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.566 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.566 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.566 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.566 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:57:41.567 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.567 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.567 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:57:41.567 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.568 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.568 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.568 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.568 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.569 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:57:41.569 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.569 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.569 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.569 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:57:41.570 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.570 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.570 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.570 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.570 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.571 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:57:41.571 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.571 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.571 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.572 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:57:41.572 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.572 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.572 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.572 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.573 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.573 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:57:41.573 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.573 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.573 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:41.574 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:57:41.574 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:57:41.574 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:57:41.574 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:57:41.575 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:57:41.575 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:57:41.575 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:57:41.575 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:57:41.575 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.576 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:57:41.576 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:57:41.576 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:57:41.576 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:57:41.576 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:57:41.577 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T21:57:41.577 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/2465449969": "2021-11-22T21:57:35.302734+0000", 2021-11-21T21:57:41.577 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/2465449969": "2021-11-22T21:57:35.302734+0000", 2021-11-21T21:57:41.577 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/2140793945": "2021-11-22T21:56:08.223482+0000", 2021-11-21T21:57:41.577 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/2140793945": "2021-11-22T21:56:08.223482+0000" 2021-11-21T21:57:41.578 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.578 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:57:41.578 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:57:41.578 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:57:41.579 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:57:41.579 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:57:41.579 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:57:41.579 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:57:41.580 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:57:41.580 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:41.580 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:41.580 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:57:41.580 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:57:41.581 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:57:41.581 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:57:41.581 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:57:41.581 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:57:41.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca6c053660 msgr2=0x7fca6c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca6c053660 0x7fca6c055b20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fca7c002f90 tx=0x7fca7c006930).stop 2021-11-21T21:57:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a6aa0 msgr2=0x7fca8c146600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a6aa0 0x7fca8c146600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fca8c063e20 tx=0x7fca80015000).stop 2021-11-21T21:57:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 shutdown_connections 2021-11-21T21:57:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca8c0a86b0 0x7fca8c146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca6c053660 0x7fca6c055b20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca8c0a6aa0 0x7fca8c146600 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 --2- 172.21.2.116:0/872926689 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca8c062410 0x7fca8c0648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 >> 172.21.2.116:0/872926689 conn(0x7fca8c05bd20 msgr2=0x7fca8c1426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 shutdown_connections 2021-11-21T21:57:41.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.384+0000 7fca92295700 1 -- 172.21.2.116:0/872926689 wait complete. 2021-11-21T21:57:41.586 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:57:41.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87dc867700 1 Processor -- start 2021-11-21T21:57:41.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87dc867700 1 -- start start 2021-11-21T21:57:41.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87dc867700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f87d80a86b0 0x7f87d814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87dc867700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f87d8062410 0x7f87d81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87dc867700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d80a6e80 con 0x7f87d80a6aa0 2021-11-21T21:57:41.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87dc867700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d80a8bc0 con 0x7f87d80a86b0 2021-11-21T21:57:41.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87dc867700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d814a7b0 con 0x7f87d8062410 2021-11-21T21:57:41.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f87d80a86b0 0x7f87d814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f87d80a86b0 0x7f87d814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47338/0 (socket says 172.21.2.116:47338) 2021-11-21T21:57:41.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 -- 172.21.2.116:0/1532320930 learned_addr learned my addr 172.21.2.116:0/1532320930 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 -- 172.21.2.116:0/1532320930 >> v2:172.21.2.117:3300/0 conn(0x7f87d8062410 msgr2=0x7f87d81443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:41.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 --2- 172.21.2.116:0/1532320930 >> v2:172.21.2.117:3300/0 conn(0x7f87d8062410 0x7f87d81443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 -- 172.21.2.116:0/1532320930 >> v1:172.21.2.117:6789/0 conn(0x7f87d80a6aa0 legacy=0x7f87d81470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:41.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 -- 172.21.2.116:0/1532320930 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d814aa10 con 0x7f87d80a86b0 2021-11-21T21:57:41.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d5d9b700 1 --2- 172.21.2.116:0/1532320930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d814b150 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f87d8061ab0 tx=0x7f87c8006eb0).ready entity=mon.0 client_cookie=17d983fc336ab69b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d559a700 1 -- 172.21.2.116:0/1532320930 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c800d5c0 con 0x7f87d80a86b0 2021-11-21T21:57:41.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d559a700 1 -- 172.21.2.116:0/1532320930 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8000e90 con 0x7f87d80a86b0 2021-11-21T21:57:41.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.640+0000 7f87d559a700 1 -- 172.21.2.116:0/1532320930 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c800d910 con 0x7f87d80a86b0 2021-11-21T21:57:41.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/1532320930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 msgr2=0x7f87d814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 --2- 172.21.2.116:0/1532320930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d814b150 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f87d8061ab0 tx=0x7f87c8006eb0).stop 2021-11-21T21:57:41.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/1532320930 shutdown_connections 2021-11-21T21:57:41.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 --2- 172.21.2.116:0/1532320930 >> v2:172.21.2.117:3300/0 conn(0x7f87d8062410 0x7f87d81443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 --2- 172.21.2.116:0/1532320930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d814b150 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/1532320930 >> 172.21.2.116:0/1532320930 conn(0x7f87d805bd20 msgr2=0x7f87d805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/1532320930 shutdown_connections 2021-11-21T21:57:41.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/1532320930 wait complete. 2021-11-21T21:57:41.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 Processor -- start 2021-11-21T21:57:41.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- start start 2021-11-21T21:57:41.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87d8062410 0x7f87d8065080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87d80a6aa0 0x7f87d8065570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d80aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8065a60 con 0x7f87d80a86b0 2021-11-21T21:57:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8157340 con 0x7f87d80a6aa0 2021-11-21T21:57:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d81570a0 con 0x7f87d8062410 2021-11-21T21:57:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d80aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d80aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47342/0 (socket says 172.21.2.116:47342) 2021-11-21T21:57:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 -- 172.21.2.116:0/528720379 learned_addr learned my addr 172.21.2.116:0/528720379 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 -- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87d8062410 msgr2=0x7f87d8065080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87d8062410 0x7f87d8065080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 -- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87d80a6aa0 msgr2=0x7f87d8065570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:41.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87d80a6aa0 0x7f87d8065570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 -- 172.21.2.116:0/528720379 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c800b1e0 con 0x7f87d80a86b0 2021-11-21T21:57:41.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87d6d9d700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d80aab70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f87d0003330 tx=0x7f87d00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87cf7fe700 1 -- 172.21.2.116:0/528720379 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87d0001520 con 0x7f87d80a86b0 2021-11-21T21:57:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87cf7fe700 1 -- 172.21.2.116:0/528720379 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0001680 con 0x7f87d80a86b0 2021-11-21T21:57:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d8156d90 con 0x7f87d80a86b0 2021-11-21T21:57:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d80aa030 con 0x7f87d80a86b0 2021-11-21T21:57:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87cf7fe700 1 -- 172.21.2.116:0/528720379 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87d00019e0 con 0x7f87d80a86b0 2021-11-21T21:57:41.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d80573d0 con 0x7f87d80a86b0 2021-11-21T21:57:41.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87cf7fe700 1 -- 172.21.2.116:0/528720379 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f87d0018030 con 0x7f87d80a86b0 2021-11-21T21:57:41.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87cf7fe700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87bc053720 0x7f87bc055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.644+0000 7f87cf7fe700 1 -- 172.21.2.116:0/528720379 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f87d006b1a0 con 0x7f87d80a86b0 2021-11-21T21:57:41.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.648+0000 7f87d659c700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87bc053720 0x7f87bc055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.648+0000 7f87d659c700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87bc053720 0x7f87bc055be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f87c0002fb0 tx=0x7f87c0006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.648+0000 7f87cf7fe700 1 -- 172.21.2.116:0/528720379 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87d0042530 con 0x7f87d80a86b0 2021-11-21T21:57:41.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87d805db70 con 0x7f87d80a86b0 2021-11-21T21:57:41.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87cf7fe700 1 -- 172.21.2.116:0/528720379 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f87d0045b50 con 0x7f87d80a86b0 2021-11-21T21:57:41.758 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:41.758 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/3302572168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3302572168},{"type":"v1","addr":"172.21.2.116:6835","nonce":3302572168}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[]} 2021-11-21T21:57:41.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87bc053720 msgr2=0x7f87bc055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87bc053720 0x7f87bc055be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f87c0002fb0 tx=0x7f87c0006930).stop 2021-11-21T21:57:41.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 msgr2=0x7f87d80aab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d80aab70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f87d0003330 tx=0x7f87d00091a0).stop 2021-11-21T21:57:41.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 shutdown_connections 2021-11-21T21:57:41.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87bc053720 0x7f87bc055be0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87d80a86b0 0x7f87d80aab70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87d80a6aa0 0x7f87d8065570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 --2- 172.21.2.116:0/528720379 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87d8062410 0x7f87d8065080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 >> 172.21.2.116:0/528720379 conn(0x7f87d805bd20 msgr2=0x7f87d81437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 shutdown_connections 2021-11-21T21:57:41.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.756+0000 7f87dc867700 1 -- 172.21.2.116:0/528720379 wait complete. 2021-11-21T21:57:41.762 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 20 2021-11-21T21:57:41.765 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:57:41.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.813+0000 7f57969ee700 1 Processor -- start 2021-11-21T21:57:41.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.813+0000 7f57969ee700 1 -- start start 2021-11-21T21:57:41.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f57900a6aa0 0x7f579014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f57900a86b0 0x7f57901470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57901475d0 0x7f57901443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900a8bc0 con 0x7f57900a86b0 2021-11-21T21:57:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900a6e80 con 0x7f57900a6aa0 2021-11-21T21:57:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579014a7b0 con 0x7f57901475d0 2021-11-21T21:57:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f57900a86b0 0x7f57901470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f57900a86b0 0x7f57901470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47354/0 (socket says 172.21.2.116:47354) 2021-11-21T21:57:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/1736126661 learned_addr learned my addr 172.21.2.116:0/1736126661 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/1736126661 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57901475d0 msgr2=0x7f57901443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- 172.21.2.116:0/1736126661 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57901475d0 0x7f57901443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/1736126661 >> v2:172.21.2.117:3300/0 conn(0x7f57900a6aa0 msgr2=0x7f579014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- 172.21.2.116:0/1736126661 >> v2:172.21.2.117:3300/0 conn(0x7f57900a6aa0 0x7f579014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/1736126661 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579014aa10 con 0x7f57900a86b0 2021-11-21T21:57:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- 172.21.2.116:0/1736126661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57901470e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f57880032f0 tx=0x7f5788009290).ready entity=mon.0 client_cookie=d5addb433b7857af server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578effd700 1 -- 172.21.2.116:0/1736126661 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f578800b870 con 0x7f57900a86b0 2021-11-21T21:57:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578effd700 1 -- 172.21.2.116:0/1736126661 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800b9d0 con 0x7f57900a86b0 2021-11-21T21:57:41.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578effd700 1 -- 172.21.2.116:0/1736126661 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f578800bb40 con 0x7f57900a86b0 2021-11-21T21:57:41.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- 172.21.2.116:0/1736126661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 msgr2=0x7f57901470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- 172.21.2.116:0/1736126661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57901470e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f57880032f0 tx=0x7f5788009290).stop 2021-11-21T21:57:41.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- 172.21.2.116:0/1736126661 shutdown_connections 2021-11-21T21:57:41.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- 172.21.2.116:0/1736126661 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57901475d0 0x7f57901443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- 172.21.2.116:0/1736126661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57901470e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- 172.21.2.116:0/1736126661 >> v2:172.21.2.117:3300/0 conn(0x7f57900a6aa0 0x7f579014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- 172.21.2.116:0/1736126661 >> 172.21.2.116:0/1736126661 conn(0x7f579005bd20 msgr2=0x7f579005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- 172.21.2.116:0/1736126661 shutdown_connections 2021-11-21T21:57:41.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- 172.21.2.116:0/1736126661 wait complete. 2021-11-21T21:57:41.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 Processor -- start 2021-11-21T21:57:41.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- start start 2021-11-21T21:57:41.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57900a6aa0 0x7f57900b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57900a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57901475d0 0x7f57900a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900aa5f0 con 0x7f57900a86b0 2021-11-21T21:57:41.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790147050 con 0x7f57901475d0 2021-11-21T21:57:41.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57901472f0 con 0x7f57900a6aa0 2021-11-21T21:57:41.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57900a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57900a8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47356/0 (socket says 172.21.2.116:47356) 2021-11-21T21:57:41.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/3221738742 learned_addr learned my addr 172.21.2.116:0/3221738742 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:41.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57900a6aa0 msgr2=0x7f57900b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57900a6aa0 0x7f57900b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57901475d0 msgr2=0x7f57900a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57901475d0 0x7f57900a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 -- 172.21.2.116:0/3221738742 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788009530 con 0x7f57900a86b0 2021-11-21T21:57:41.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578f7fe700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57900a8fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f57880032f0 tx=0x7f5788016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578d7fa700 1 -- 172.21.2.116:0/3221738742 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f578800b870 con 0x7f57900a86b0 2021-11-21T21:57:41.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578d7fa700 1 -- 172.21.2.116:0/3221738742 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800bdb0 con 0x7f57900a86b0 2021-11-21T21:57:41.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790145e60 con 0x7f57900a86b0 2021-11-21T21:57:41.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f578d7fa700 1 -- 172.21.2.116:0/3221738742 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5788020620 con 0x7f57900a86b0 2021-11-21T21:57:41.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.817+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57900b4b70 con 0x7f57900a86b0 2021-11-21T21:57:41.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.821+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57900573d0 con 0x7f57900a86b0 2021-11-21T21:57:41.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.821+0000 7f578d7fa700 1 -- 172.21.2.116:0/3221738742 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5788026020 con 0x7f57900a86b0 2021-11-21T21:57:41.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.821+0000 7f578d7fa700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5774053610 0x7f5774055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:41.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.821+0000 7f578d7fa700 1 -- 172.21.2.116:0/3221738742 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f578806c770 con 0x7f57900a86b0 2021-11-21T21:57:41.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.821+0000 7f578ffff700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5774053610 0x7f5774055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:41.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.821+0000 7f578ffff700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5774053610 0x7f5774055ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f5778002f70 tx=0x7f57780069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:41.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.821+0000 7f578d7fa700 1 -- 172.21.2.116:0/3221738742 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5788043b60 con 0x7f57900a86b0 2021-11-21T21:57:41.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5790068320 con 0x7f57900a86b0 2021-11-21T21:57:41.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f578d7fa700 1 -- 172.21.2.116:0/3221738742 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f5788015070 con 0x7f57900a86b0 2021-11-21T21:57:41.934 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:41.934 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1614044972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1614044972},{"type":"v1","addr":"172.21.2.117:6835","nonce":1614044972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/1902081621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1902081621},{"type":"v1","addr":"172.21.2.116:6833","nonce":1902081621}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/3302572168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3302572168},{"type":"v1","addr":"172.21.2.116:6835","nonce":3302572168}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[]} 2021-11-21T21:57:41.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5774053610 msgr2=0x7f5774055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5774053610 0x7f5774055ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f5778002f70 tx=0x7f57780069e0).stop 2021-11-21T21:57:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 msgr2=0x7f57900a8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57900a8fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f57880032f0 tx=0x7f5788016000).stop 2021-11-21T21:57:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 shutdown_connections 2021-11-21T21:57:41.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57901475d0 0x7f57900a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5774053610 0x7f5774055ad0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57900a86b0 0x7f57900a8fc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:41.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 --2- 172.21.2.116:0/3221738742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f57900a6aa0 0x7f57900b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:41.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 >> 172.21.2.116:0/3221738742 conn(0x7f579005bd20 msgr2=0x7f5790148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:41.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 shutdown_connections 2021-11-21T21:57:41.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:41.933+0000 7f57969ee700 1 -- 172.21.2.116:0/3221738742 wait complete. 2021-11-21T21:57:41.938 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 20 2021-11-21T21:57:41.941 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T21:57:41.942 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T21:57:41.942 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T21:57:41.942 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:57:41.943 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T21:57:41.943 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T21:57:41.943 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T21:57:41.944 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:57:41.944 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T21:57:41.944 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T21:57:41.945 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T21:57:41.945 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:57:41.948 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T21:57:41.949 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T21:57:41.950 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T21:57:41.950 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T21:57:41.952 INFO:tasks.ceph.mds.a:Started 2021-11-21T21:57:41.952 DEBUG:teuthology.parallel:result is None 2021-11-21T21:57:41.953 INFO:tasks.ceph.mds.c:Started 2021-11-21T21:57:41.954 DEBUG:teuthology.parallel:result is None 2021-11-21T21:57:41.956 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T21:57:41.957 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T21:57:41.959 INFO:tasks.ceph.mds.b:Started 2021-11-21T21:57:41.959 DEBUG:teuthology.parallel:result is None 2021-11-21T21:57:41.959 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T21:57:41.985 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T21:57:41.986 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:57:41.993 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T21:57:42.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.013+0000 7f2198b24700 1 Processor -- start 2021-11-21T21:57:42.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.013+0000 7f2198b24700 1 -- start start 2021-11-21T21:57:42.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.013+0000 7f2198b24700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f21940b6b50 0x7f21940adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:42.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.013+0000 7f2198b24700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21940ae180 con 0x7f21940b1270 2021-11-21T21:57:42.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.013+0000 7f2198b24700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21940b0070 con 0x7f21940b6770 2021-11-21T21:57:42.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.013+0000 7f2198b24700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21940b05c0 con 0x7f21940b6b50 2021-11-21T21:57:42.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f21940b6b50 0x7f21940adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:42.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2191d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f21940b1270 0x7f21940ad580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59346/0 (socket says 172.21.2.116:59346) 2021-11-21T21:57:42.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2191d9b700 1 -- 172.21.2.116:0/2780166205 learned_addr learned my addr 172.21.2.116:0/2780166205 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:42.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 -- 172.21.2.116:0/2780166205 >> v1:172.21.2.117:6789/0 conn(0x7f21940b6770 legacy=0x7f21941394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:42.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 -- 172.21.2.116:0/2780166205 >> v1:172.21.2.116:6789/0 conn(0x7f21940b1270 legacy=0x7f21940ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:42.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 -- 172.21.2.116:0/2780166205 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940b2230 con 0x7f21940b6b50 2021-11-21T21:57:42.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 --2- 172.21.2.116:0/2780166205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f21940adc90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f21940609a0 tx=0x7f218c0091d0).ready entity=mon.0 client_cookie=18b8e50ac2a73131 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:42.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f219159a700 1 -- 172.21.2.116:0/2780166205 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f218c009ea0 con 0x7f21940b6b50 2021-11-21T21:57:42.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f219159a700 1 -- 172.21.2.116:0/2780166205 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c00b860 con 0x7f21940b6b50 2021-11-21T21:57:42.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f219159a700 1 -- 172.21.2.116:0/2780166205 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f218c00b9d0 con 0x7f21940b6b50 2021-11-21T21:57:42.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/2780166205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 msgr2=0x7f21940adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:42.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 --2- 172.21.2.116:0/2780166205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f21940adc90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f21940609a0 tx=0x7f218c0091d0).stop 2021-11-21T21:57:42.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/2780166205 shutdown_connections 2021-11-21T21:57:42.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 --2- 172.21.2.116:0/2780166205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f21940adc90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:42.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/2780166205 >> 172.21.2.116:0/2780166205 conn(0x7f2194062040 msgr2=0x7f21940b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:42.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/2780166205 shutdown_connections 2021-11-21T21:57:42.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/2780166205 wait complete. 2021-11-21T21:57:42.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 Processor -- start 2021-11-21T21:57:42.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- start start 2021-11-21T21:57:42.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21940b1270 0x7f219405c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:42.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21940b6770 0x7f2194150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:42.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f2194065e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:42.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219413b2a0 con 0x7f21940b6b50 2021-11-21T21:57:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219405cae0 con 0x7f21940b6770 2021-11-21T21:57:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2194067640 con 0x7f21940b1270 2021-11-21T21:57:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f2194065e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f2194065e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47398/0 (socket says 172.21.2.116:47398) 2021-11-21T21:57:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 -- 172.21.2.116:0/1708388834 learned_addr learned my addr 172.21.2.116:0/1708388834 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2191d9b700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21940b6770 0x7f2194150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f219259c700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21940b1270 0x7f219405c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 -- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21940b1270 msgr2=0x7f219405c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21940b1270 0x7f219405c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 -- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21940b6770 msgr2=0x7f2194150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21940b6770 0x7f2194150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 -- 172.21.2.116:0/1708388834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c009470 con 0x7f21940b6b50 2021-11-21T21:57:42.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2192d9d700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f2194065e70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f21940609a0 tx=0x7f218c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:42.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f218b7fe700 1 -- 172.21.2.116:0/1708388834 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f218c009ea0 con 0x7f21940b6b50 2021-11-21T21:57:42.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f218b7fe700 1 -- 172.21.2.116:0/1708388834 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c0032f0 con 0x7f21940b6b50 2021-11-21T21:57:42.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219414ff20 con 0x7f21940b6b50 2021-11-21T21:57:42.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f218b7fe700 1 -- 172.21.2.116:0/1708388834 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f218c01f5e0 con 0x7f21940b6b50 2021-11-21T21:57:42.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2194150460 con 0x7f21940b6b50 2021-11-21T21:57:42.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.017+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2194150620 con 0x7f21940b6b50 2021-11-21T21:57:42.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.021+0000 7f218b7fe700 1 -- 172.21.2.116:0/1708388834 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f218c026030 con 0x7f21940b6b50 2021-11-21T21:57:42.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.021+0000 7f218b7fe700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2178053610 0x7f2178055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:42.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.021+0000 7f218b7fe700 1 -- 172.21.2.116:0/1708388834 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f218c06c310 con 0x7f21940b6b50 2021-11-21T21:57:42.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.021+0000 7f219259c700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2178053610 0x7f2178055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:42.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.021+0000 7f219259c700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2178053610 0x7f2178055ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f218400d000 tx=0x7f2184007b70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:42.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.021+0000 7f218b7fe700 1 -- 172.21.2.116:0/1708388834 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f218c047130 con 0x7f21940b6b50 2021-11-21T21:57:42.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:42.117+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f21940573d0 con 0x7f21940b6b50 2021-11-21T21:57:43.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.009+0000 7f218b7fe700 1 -- 172.21.2.116:0/1708388834 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f218c015080 con 0x7f21940b6b50 2021-11-21T21:57:43.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2178053610 msgr2=0x7f2178055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:43.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2178053610 0x7f2178055ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f218400d000 tx=0x7f2184007b70).stop 2021-11-21T21:57:43.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 msgr2=0x7f2194065e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:43.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f2194065e70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f21940609a0 tx=0x7f218c016000).stop 2021-11-21T21:57:43.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 shutdown_connections 2021-11-21T21:57:43.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2178053610 0x7f2178055ad0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f21940b6b50 0x7f2194065e70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f21940b6770 0x7f2194150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 --2- 172.21.2.116:0/1708388834 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f21940b1270 0x7f219405c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.013+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 >> 172.21.2.116:0/1708388834 conn(0x7f2194062040 msgr2=0x7f219413be10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:43.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.017+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 shutdown_connections 2021-11-21T21:57:43.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.017+0000 7f2198b24700 1 -- 172.21.2.116:0/1708388834 wait complete. 2021-11-21T21:57:43.636 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T21:57:43.638 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T21:57:43.639 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T21:57:43.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d8987700 1 Processor -- start 2021-11-21T21:57:43.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d8987700 1 -- start start 2021-11-21T21:57:43.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d8987700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d40a86b0 0x7fa1d414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d8987700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa1d4062410 0x7fa1d41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d8987700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d40a6e80 con 0x7fa1d40a6aa0 2021-11-21T21:57:43.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d8987700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d40a8bc0 con 0x7fa1d4062410 2021-11-21T21:57:43.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d8987700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d414a7b0 con 0x7fa1d40a86b0 2021-11-21T21:57:43.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa1d4062410 0x7fa1d41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:43.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa1d4062410 0x7fa1d41443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47410/0 (socket says 172.21.2.116:47410) 2021-11-21T21:57:43.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/1935425522 learned_addr learned my addr 172.21.2.116:0/1935425522 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:43.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/1935425522 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d40a86b0 msgr2=0x7fa1d414b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:43.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 --2- 172.21.2.116:0/1935425522 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d40a86b0 0x7fa1d414b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:43.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/1935425522 >> v1:172.21.2.117:6789/0 conn(0x7fa1d40a6aa0 legacy=0x7fa1d41470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:43.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/1935425522 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d414aa10 con 0x7fa1d4062410 2021-11-21T21:57:43.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d2d9d700 1 --2- 172.21.2.116:0/1935425522 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d4062410 0x7fa1d41443f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fa1d4061ab0 tx=0x7fa1cc009290).ready entity=mon.0 client_cookie=92970131a5450eb7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:43.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d159a700 1 -- 172.21.2.116:0/1935425522 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1cc00b830 con 0x7fa1d4062410 2021-11-21T21:57:43.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d159a700 1 -- 172.21.2.116:0/1935425522 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc00b990 con 0x7fa1d4062410 2021-11-21T21:57:43.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.729+0000 7fa1d159a700 1 -- 172.21.2.116:0/1935425522 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1cc00bb40 con 0x7fa1d4062410 2021-11-21T21:57:43.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/1935425522 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d4062410 msgr2=0x7fa1d41443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:43.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 --2- 172.21.2.116:0/1935425522 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d4062410 0x7fa1d41443f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fa1d4061ab0 tx=0x7fa1cc009290).stop 2021-11-21T21:57:43.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/1935425522 shutdown_connections 2021-11-21T21:57:43.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 --2- 172.21.2.116:0/1935425522 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d4062410 0x7fa1d41443f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 --2- 172.21.2.116:0/1935425522 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d40a86b0 0x7fa1d414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:43.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/1935425522 >> 172.21.2.116:0/1935425522 conn(0x7fa1d405bd20 msgr2=0x7fa1d405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:43.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/1935425522 shutdown_connections 2021-11-21T21:57:43.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/1935425522 wait complete. 2021-11-21T21:57:43.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 Processor -- start 2021-11-21T21:57:43.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- start start 2021-11-21T21:57:43.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d4062410 0x7fa1d40a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1d40a6aa0 0x7fa1d4063850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d40a86b0 0x7fa1d4065f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4159c10 con 0x7fa1d40a86b0 2021-11-21T21:57:43.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4159e40 con 0x7fa1d40a6aa0 2021-11-21T21:57:43.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d415a0e0 con 0x7fa1d4062410 2021-11-21T21:57:43.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d40a86b0 0x7fa1d4065f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:43.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d40a86b0 0x7fa1d4065f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47416/0 (socket says 172.21.2.116:47416) 2021-11-21T21:57:43.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/3343258088 learned_addr learned my addr 172.21.2.116:0/3343258088 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:43.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d4062410 msgr2=0x7fa1d40a9ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:43.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d4062410 0x7fa1d40a9ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:43.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1d40a6aa0 msgr2=0x7fa1d4063850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:43.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1d40a6aa0 0x7fa1d4063850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:43.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 -- 172.21.2.116:0/3343258088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc009530 con 0x7fa1d40a86b0 2021-11-21T21:57:43.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d2d9d700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d40a86b0 0x7fa1d4065f70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fa1d4061ab0 tx=0x7fa1cc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:43.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1cb7fe700 1 -- 172.21.2.116:0/3343258088 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1cc00b830 con 0x7fa1d40a86b0 2021-11-21T21:57:43.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1cb7fe700 1 -- 172.21.2.116:0/3343258088 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc0107a0 con 0x7fa1d40a86b0 2021-11-21T21:57:43.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1cb7fe700 1 -- 172.21.2.116:0/3343258088 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1cc010b00 con 0x7fa1d40a86b0 2021-11-21T21:57:43.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d415a380 con 0x7fa1d40a86b0 2021-11-21T21:57:43.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d40aa1c0 con 0x7fa1d40a86b0 2021-11-21T21:57:43.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d415a510 con 0x7fa1d40a86b0 2021-11-21T21:57:43.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1cb7fe700 1 -- 172.21.2.116:0/3343258088 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa1cc00f070 con 0x7fa1d40a86b0 2021-11-21T21:57:43.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1cb7fe700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1b80536f0 0x7fa1b8055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.733+0000 7fa1cb7fe700 1 -- 172.21.2.116:0/3343258088 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa1cc06cfd0 con 0x7fa1d40a86b0 2021-11-21T21:57:43.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.737+0000 7fa1d259c700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1b80536f0 0x7fa1b8055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:43.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.737+0000 7fa1d259c700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1b80536f0 0x7fa1b8055bb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa1c4002fb0 tx=0x7fa1c4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:43.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.737+0000 7fa1cb7fe700 1 -- 172.21.2.116:0/3343258088 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa1cc044350 con 0x7fa1d40a86b0 2021-11-21T21:57:43.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.845+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa1d40a7ea0 con 0x7fa1d40a86b0 2021-11-21T21:57:43.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.845+0000 7fa1cb7fe700 1 -- 172.21.2.116:0/3343258088 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fa1cc015070 con 0x7fa1d40a86b0 2021-11-21T21:57:43.847 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:43.848 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:57:43.848 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T21:57:43.848 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.848 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T21:57:43.849 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T21:57:43.849 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.849 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:57:43.849 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.850 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.850 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.850 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.850 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.850 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.851 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T21:57:43.851 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T21:57:43.851 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.851 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:57:43.852 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.852 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.852 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.852 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.852 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.853 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.853 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T21:57:43.853 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T21:57:43.853 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.854 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:57:43.854 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.854 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.854 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.854 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.855 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.855 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.855 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T21:57:43.855 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T21:57:43.855 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.856 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.856 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.856 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.856 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.857 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.857 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.857 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T21:57:43.857 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T21:57:43.857 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.858 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.858 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.858 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.858 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.859 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.859 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.859 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T21:57:43.859 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T21:57:43.859 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.860 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.860 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.860 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.860 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.860 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.861 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.861 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T21:57:43.861 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T21:57:43.861 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.862 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.862 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.862 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.862 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.862 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.863 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.863 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T21:57:43.863 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T21:57:43.863 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.864 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.864 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.864 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.864 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.864 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.865 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.865 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T21:57:43.865 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T21:57:43.865 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.865 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.866 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.866 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.866 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.866 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.867 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.867 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T21:57:43.867 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T21:57:43.867 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.867 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.868 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.868 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.868 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.868 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.868 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.869 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T21:57:43.869 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T21:57:43.869 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.869 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.870 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.870 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.870 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.870 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.871 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.871 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T21:57:43.871 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T21:57:43.871 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.871 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:57:43.872 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:57:43.872 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:57:43.872 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.872 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.872 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.873 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T21:57:43.873 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T21:57:43.873 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.873 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:57:43.873 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:57:43.874 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:57:43.874 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.874 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.874 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.875 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T21:57:43.875 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T21:57:43.875 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.875 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:57:43.875 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:57:43.876 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:57:43.876 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.876 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.876 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.877 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.877 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T21:57:43.877 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T21:57:43.877 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.877 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:57:43.878 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:57:43.878 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.878 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.878 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:43.878 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:43.879 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T21:57:43.879 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T21:57:43.879 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:57:43.879 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:57:43.880 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:57:43.880 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:57:43.880 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.880 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:43.880 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:57:43.881 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:57:43.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1b80536f0 msgr2=0x7fa1b8055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:43.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1b80536f0 0x7fa1b8055bb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa1c4002fb0 tx=0x7fa1c4006930).stop 2021-11-21T21:57:43.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d40a86b0 msgr2=0x7fa1d4065f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:43.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d40a86b0 0x7fa1d4065f70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fa1d4061ab0 tx=0x7fa1cc016000).stop 2021-11-21T21:57:43.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 shutdown_connections 2021-11-21T21:57:43.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1b80536f0 0x7fa1b8055bb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1d40a86b0 0x7fa1d4065f70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1d40a6aa0 0x7fa1d4063850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:43.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 --2- 172.21.2.116:0/3343258088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1d4062410 0x7fa1d40a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:43.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 >> 172.21.2.116:0/3343258088 conn(0x7fa1d405bd20 msgr2=0x7fa1d40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:43.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 shutdown_connections 2021-11-21T21:57:43.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.849+0000 7fa1d8987700 1 -- 172.21.2.116:0/3343258088 wait complete. 2021-11-21T21:57:43.885 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:57:43.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 Processor -- start 2021-11-21T21:57:43.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- start start 2021-11-21T21:57:43.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a6aa0 0x7fc61014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc610062410 0x7fc6101443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:43.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6100a6e80 con 0x7fc6100a86b0 2021-11-21T21:57:43.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6100a8bc0 con 0x7fc610062410 2021-11-21T21:57:43.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61014a7b0 con 0x7fc6100a6aa0 2021-11-21T21:57:43.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc614c2e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc6100a86b0 0x7fc6101470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59378/0 (socket says 172.21.2.116:59378) 2021-11-21T21:57:43.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc614c2e700 1 -- 172.21.2.116:0/1751158117 learned_addr learned my addr 172.21.2.116:0/1751158117 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:43.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc615c30700 1 --2- 172.21.2.116:0/1751158117 >> v2:172.21.2.116:3300/0 conn(0x7fc610062410 0x7fc6101443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:43.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc61542f700 1 --2- 172.21.2.116:0/1751158117 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a6aa0 0x7fc61014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:43.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc615c30700 1 -- 172.21.2.116:0/1751158117 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a6aa0 msgr2=0x7fc61014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:43.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc615c30700 1 --2- 172.21.2.116:0/1751158117 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a6aa0 0x7fc61014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc615c30700 1 -- 172.21.2.116:0/1751158117 >> v1:172.21.2.116:6789/0 conn(0x7fc6100a86b0 legacy=0x7fc6101470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:43.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc615c30700 1 -- 172.21.2.116:0/1751158117 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61014aa10 con 0x7fc610062410 2021-11-21T21:57:43.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc615c30700 1 --2- 172.21.2.116:0/1751158117 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc6101443f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fc6080032f0 tx=0x7fc6080091d0).ready entity=mon.0 client_cookie=2744d48d92aa84fb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:43.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc5fffff700 1 -- 172.21.2.116:0/1751158117 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc608009ea0 con 0x7fc610062410 2021-11-21T21:57:43.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc5fffff700 1 -- 172.21.2.116:0/1751158117 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60800b8a0 con 0x7fc610062410 2021-11-21T21:57:43.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc5fffff700 1 -- 172.21.2.116:0/1751158117 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc60800ba10 con 0x7fc610062410 2021-11-21T21:57:43.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- 172.21.2.116:0/1751158117 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 msgr2=0x7fc6101443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:43.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 --2- 172.21.2.116:0/1751158117 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc6101443f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fc6080032f0 tx=0x7fc6080091d0).stop 2021-11-21T21:57:43.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- 172.21.2.116:0/1751158117 shutdown_connections 2021-11-21T21:57:43.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 --2- 172.21.2.116:0/1751158117 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc6101443f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 --2- 172.21.2.116:0/1751158117 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a6aa0 0x7fc61014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:43.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- 172.21.2.116:0/1751158117 >> 172.21.2.116:0/1751158117 conn(0x7fc61005bd20 msgr2=0x7fc61005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:43.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- 172.21.2.116:0/1751158117 shutdown_connections 2021-11-21T21:57:43.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- 172.21.2.116:0/1751158117 wait complete. 2021-11-21T21:57:43.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 Processor -- start 2021-11-21T21:57:43.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.989+0000 7fc617693700 1 -- start start 2021-11-21T21:57:44.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc610143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6100a6aa0 0x7fc6100a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a86b0 0x7fc6100aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100653f0 con 0x7fc610062410 2021-11-21T21:57:44.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610156480 con 0x7fc6100a6aa0 2021-11-21T21:57:44.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc610156720 con 0x7fc6100a86b0 2021-11-21T21:57:44.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc610143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc610143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47430/0 (socket says 172.21.2.116:47430) 2021-11-21T21:57:44.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 -- 172.21.2.116:0/2296482038 learned_addr learned my addr 172.21.2.116:0/2296482038 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:44.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc615c30700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a86b0 0x7fc6100aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc614c2e700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6100a6aa0 0x7fc6100a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 -- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a86b0 msgr2=0x7fc6100aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a86b0 0x7fc6100aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 -- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6100a6aa0 msgr2=0x7fc6100a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6100a6aa0 0x7fc6100a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 -- 172.21.2.116:0/2296482038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc608009470 con 0x7fc610062410 2021-11-21T21:57:44.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc61542f700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc610143990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fc60c0030b0 tx=0x7fc60c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc5fe7fc700 1 -- 172.21.2.116:0/2296482038 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc60c011780 con 0x7fc610062410 2021-11-21T21:57:44.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc5fe7fc700 1 -- 172.21.2.116:0/2296482038 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60c0118e0 con 0x7fc610062410 2021-11-21T21:57:44.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6101569c0 con 0x7fc610062410 2021-11-21T21:57:44.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc5fe7fc700 1 -- 172.21.2.116:0/2296482038 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc60c011c40 con 0x7fc610062410 2021-11-21T21:57:44.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc610147440 con 0x7fc610062410 2021-11-21T21:57:44.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6100573d0 con 0x7fc610062410 2021-11-21T21:57:44.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc5fe7fc700 1 -- 172.21.2.116:0/2296482038 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc60c012420 con 0x7fc610062410 2021-11-21T21:57:44.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc5fe7fc700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5f00534f0 0x7fc5f00559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc5fe7fc700 1 -- 172.21.2.116:0/2296482038 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc60c06b500 con 0x7fc610062410 2021-11-21T21:57:44.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc614c2e700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5f00534f0 0x7fc5f00559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.993+0000 7fc614c2e700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5f00534f0 0x7fc5f00559b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc60000b000 tx=0x7fc600009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:43.997+0000 7fc5fe7fc700 1 -- 172.21.2.116:0/2296482038 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc60c042930 con 0x7fc610062410 2021-11-21T21:57:44.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc610068320 con 0x7fc610062410 2021-11-21T21:57:44.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc5fe7fc700 1 -- 172.21.2.116:0/2296482038 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13181 (secure 0 0 0) 0x7fc60c045f50 con 0x7fc610062410 2021-11-21T21:57:44.082 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:44.082 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":51,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:57:43.001086+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:57:44.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5f00534f0 msgr2=0x7fc5f00559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5f00534f0 0x7fc5f00559b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc60000b000 tx=0x7fc600009040).stop 2021-11-21T21:57:44.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 msgr2=0x7fc610143990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc610143990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fc60c0030b0 tx=0x7fc60c008040).stop 2021-11-21T21:57:44.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 shutdown_connections 2021-11-21T21:57:44.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6100a86b0 0x7fc6100aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6100a6aa0 0x7fc6100a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc5f00534f0 0x7fc5f00559b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 --2- 172.21.2.116:0/2296482038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc610062410 0x7fc610143990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 >> 172.21.2.116:0/2296482038 conn(0x7fc61005bd20 msgr2=0x7fc6101441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 shutdown_connections 2021-11-21T21:57:44.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.081+0000 7fc617693700 1 -- 172.21.2.116:0/2296482038 wait complete. 2021-11-21T21:57:44.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T21:57:44.090 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:57:44.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f7593b3b700 1 Processor -- start 2021-11-21T21:57:44.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f7593b3b700 1 -- start start 2021-11-21T21:57:44.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f7593b3b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f758c0abce0 0x7f758c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f7593b3b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f758c0ac600 0x7f758c0aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f7593b3b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c147e80 con 0x7f758c144810 2021-11-21T21:57:44.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f7593b3b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0b1490 con 0x7f758c0ac600 2021-11-21T21:57:44.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f7593b3b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0b1fd0 con 0x7f758c0abce0 2021-11-21T21:57:44.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f75920d8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f758c0ac600 0x7f758c0aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f75918d7700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f758c144810 0x7f758c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59390/0 (socket says 172.21.2.116:59390) 2021-11-21T21:57:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f75920d8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f758c0ac600 0x7f758c0aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47434/0 (socket says 172.21.2.116:47434) 2021-11-21T21:57:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.141+0000 7f75918d7700 1 -- 172.21.2.116:0/1384737612 learned_addr learned my addr 172.21.2.116:0/1384737612 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:44.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75920d8700 1 -- 172.21.2.116:0/1384737612 >> v2:172.21.2.117:3300/0 conn(0x7f758c0abce0 msgr2=0x7f758c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:44.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75920d8700 1 --2- 172.21.2.116:0/1384737612 >> v2:172.21.2.117:3300/0 conn(0x7f758c0abce0 0x7f758c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75920d8700 1 -- 172.21.2.116:0/1384737612 >> v1:172.21.2.116:6789/0 conn(0x7f758c144810 legacy=0x7f758c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75920d8700 1 -- 172.21.2.116:0/1384737612 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0af5a0 con 0x7f758c0ac600 2021-11-21T21:57:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75920d8700 1 --2- 172.21.2.116:0/1384737612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c0aca50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f75840032f0 tx=0x7f75840091d0).ready entity=mon.0 client_cookie=fe6db1dbadbc0fd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75908d5700 1 -- 172.21.2.116:0/1384737612 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7584009ea0 con 0x7f758c0ac600 2021-11-21T21:57:44.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75908d5700 1 -- 172.21.2.116:0/1384737612 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400b870 con 0x7f758c0ac600 2021-11-21T21:57:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75908d5700 1 -- 172.21.2.116:0/1384737612 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f758400b9e0 con 0x7f758c0ac600 2021-11-21T21:57:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- 172.21.2.116:0/1384737612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 msgr2=0x7f758c0aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 --2- 172.21.2.116:0/1384737612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c0aca50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f75840032f0 tx=0x7f75840091d0).stop 2021-11-21T21:57:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- 172.21.2.116:0/1384737612 shutdown_connections 2021-11-21T21:57:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 --2- 172.21.2.116:0/1384737612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c0aca50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 --2- 172.21.2.116:0/1384737612 >> v2:172.21.2.117:3300/0 conn(0x7f758c0abce0 0x7f758c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- 172.21.2.116:0/1384737612 >> 172.21.2.116:0/1384737612 conn(0x7f758c05bd20 msgr2=0x7f758c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- 172.21.2.116:0/1384737612 shutdown_connections 2021-11-21T21:57:44.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- 172.21.2.116:0/1384737612 wait complete. 2021-11-21T21:57:44.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 Processor -- start 2021-11-21T21:57:44.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- start start 2021-11-21T21:57:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f758c0abce0 0x7f758c143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c144810 0x7f758c156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c159ba0 con 0x7f758c0ac600 2021-11-21T21:57:44.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15aa50 con 0x7f758c0abce0 2021-11-21T21:57:44.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c159dd0 con 0x7f758c144810 2021-11-21T21:57:44.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47444/0 (socket says 172.21.2.116:47444) 2021-11-21T21:57:44.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 -- 172.21.2.116:0/293692189 learned_addr learned my addr 172.21.2.116:0/293692189 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:44.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 -- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c144810 msgr2=0x7f758c156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c144810 0x7f758c156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 -- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f758c0abce0 msgr2=0x7f758c143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f758c0abce0 0x7f758c143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 -- 172.21.2.116:0/293692189 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009470 con 0x7f758c0ac600 2021-11-21T21:57:44.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f75910d6700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c156650 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f7588002f70 tx=0x7f7588006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f757effd700 1 -- 172.21.2.116:0/293692189 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7588013460 con 0x7f758c0ac600 2021-11-21T21:57:44.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f757effd700 1 -- 172.21.2.116:0/293692189 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75880135c0 con 0x7f758c0ac600 2021-11-21T21:57:44.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c15a070 con 0x7f758c0ac600 2021-11-21T21:57:44.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f757effd700 1 -- 172.21.2.116:0/293692189 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7588013920 con 0x7f758c0ac600 2021-11-21T21:57:44.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.145+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c15a5b0 con 0x7f758c0ac600 2021-11-21T21:57:44.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.149+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c15a770 con 0x7f758c0ac600 2021-11-21T21:57:44.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.149+0000 7f757effd700 1 -- 172.21.2.116:0/293692189 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f758801a060 con 0x7f758c0ac600 2021-11-21T21:57:44.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.149+0000 7f757effd700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f756c053640 0x7f756c055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.149+0000 7f757effd700 1 -- 172.21.2.116:0/293692189 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f758806b040 con 0x7f758c0ac600 2021-11-21T21:57:44.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.149+0000 7f75918d7700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f756c053640 0x7f756c055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.149+0000 7f75918d7700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f756c053640 0x7f756c055b00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f758c062500 tx=0x7f7580007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.149+0000 7f757effd700 1 -- 172.21.2.116:0/293692189 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f75880424d0 con 0x7f758c0ac600 2021-11-21T21:57:44.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f758c0af060 con 0x7f758c0ac600 2021-11-21T21:57:44.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f757effd700 1 -- 172.21.2.116:0/293692189 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v51) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7588045af0 con 0x7f758c0ac600 2021-11-21T21:57:44.234 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:57:44.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f756c053640 msgr2=0x7f756c055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f756c053640 0x7f756c055b00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f758c062500 tx=0x7f7580007820).stop 2021-11-21T21:57:44.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 msgr2=0x7f758c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c156650 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f7588002f70 tx=0x7f7588006e90).stop 2021-11-21T21:57:44.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 shutdown_connections 2021-11-21T21:57:44.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c144810 0x7f758c156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f756c053640 0x7f756c055b00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0ac600 0x7f758c156650 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 --2- 172.21.2.116:0/293692189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f758c0abce0 0x7f758c143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 >> 172.21.2.116:0/293692189 conn(0x7f758c05bd20 msgr2=0x7f758c1494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 shutdown_connections 2021-11-21T21:57:44.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.233+0000 7f7593b3b700 1 -- 172.21.2.116:0/293692189 wait complete. 2021-11-21T21:57:44.240 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:57:44.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 Processor -- start 2021-11-21T21:57:44.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- start start 2021-11-21T21:57:44.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd538144810 0x7fd5381478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd5380abce0 0x7fd5380b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd538147e80 con 0x7fd5380b1db0 2021-11-21T21:57:44.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5380b1490 con 0x7fd5380abce0 2021-11-21T21:57:44.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380af060 con 0x7fd538144810 2021-11-21T21:57:44.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd5380abce0 0x7fd5380b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd5380abce0 0x7fd5380b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47450/0 (socket says 172.21.2.116:47450) 2021-11-21T21:57:44.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 -- 172.21.2.116:0/4154501761 learned_addr learned my addr 172.21.2.116:0/4154501761 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:44.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 -- 172.21.2.116:0/4154501761 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd538144810 msgr2=0x7fd5381478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:44.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 --2- 172.21.2.116:0/4154501761 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd538144810 0x7fd5381478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 -- 172.21.2.116:0/4154501761 >> v1:172.21.2.117:6789/0 conn(0x7fd5380b1db0 legacy=0x7fd5380ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:44.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 -- 172.21.2.116:0/4154501761 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380af5a0 con 0x7fd5380abce0 2021-11-21T21:57:44.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd537fff700 1 --2- 172.21.2.116:0/4154501761 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5380abce0 0x7fd5380b18c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fd5380b2a30 tx=0x7fd528006eb0).ready entity=mon.0 client_cookie=49e8995de3b8c195 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd5377fe700 1 -- 172.21.2.116:0/4154501761 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd52800d570 con 0x7fd5380abce0 2021-11-21T21:57:44.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd5377fe700 1 -- 172.21.2.116:0/4154501761 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd528000e90 con 0x7fd5380abce0 2021-11-21T21:57:44.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd5377fe700 1 -- 172.21.2.116:0/4154501761 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd52800d8c0 con 0x7fd5380abce0 2021-11-21T21:57:44.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- 172.21.2.116:0/4154501761 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5380abce0 msgr2=0x7fd5380b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 --2- 172.21.2.116:0/4154501761 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5380abce0 0x7fd5380b18c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fd5380b2a30 tx=0x7fd528006eb0).stop 2021-11-21T21:57:44.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- 172.21.2.116:0/4154501761 shutdown_connections 2021-11-21T21:57:44.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 --2- 172.21.2.116:0/4154501761 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5380abce0 0x7fd5380b18c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 --2- 172.21.2.116:0/4154501761 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd538144810 0x7fd5381478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- 172.21.2.116:0/4154501761 >> 172.21.2.116:0/4154501761 conn(0x7fd53805bd20 msgr2=0x7fd53805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- 172.21.2.116:0/4154501761 shutdown_connections 2021-11-21T21:57:44.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- 172.21.2.116:0/4154501761 wait complete. 2021-11-21T21:57:44.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 Processor -- start 2021-11-21T21:57:44.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.293+0000 7fd53f0da700 1 -- start start 2021-11-21T21:57:44.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5380abce0 0x7fd53813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd538144810 0x7fd53814a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd538156700 0x7fd538144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53814a310 con 0x7fd538156700 2021-11-21T21:57:44.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd538146360 con 0x7fd538144810 2021-11-21T21:57:44.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53814a4f0 con 0x7fd5380abce0 2021-11-21T21:57:44.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd538156700 0x7fd538144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd538156700 0x7fd538144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47452/0 (socket says 172.21.2.116:47452) 2021-11-21T21:57:44.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 -- 172.21.2.116:0/3560134129 learned_addr learned my addr 172.21.2.116:0/3560134129 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 -- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5380abce0 msgr2=0x7fd53813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5380abce0 0x7fd53813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 -- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd538144810 msgr2=0x7fd53814a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd538144810 0x7fd53814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 -- 172.21.2.116:0/3560134129 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52800b1e0 con 0x7fd538156700 2021-11-21T21:57:44.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53d677700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd538156700 0x7fd538144d20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fd530003310 tx=0x7fd530009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd535ffb700 1 -- 172.21.2.116:0/3560134129 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd530001500 con 0x7fd538156700 2021-11-21T21:57:44.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd535ffb700 1 -- 172.21.2.116:0/3560134129 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530001660 con 0x7fd538156700 2021-11-21T21:57:44.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd535ffb700 1 -- 172.21.2.116:0/3560134129 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5300019c0 con 0x7fd538156700 2021-11-21T21:57:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53814a0b0 con 0x7fd538156700 2021-11-21T21:57:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd538146990 con 0x7fd538156700 2021-11-21T21:57:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5380b2c50 con 0x7fd538156700 2021-11-21T21:57:44.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd535ffb700 1 -- 172.21.2.116:0/3560134129 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd530018030 con 0x7fd538156700 2021-11-21T21:57:44.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd535ffb700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd51c053610 0x7fd51c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd535ffb700 1 -- 172.21.2.116:0/3560134129 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd53006afb0 con 0x7fd538156700 2021-11-21T21:57:44.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53ce76700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd51c053610 0x7fd51c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.297+0000 7fd53ce76700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd51c053610 0x7fd51c055ad0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fd520002f70 tx=0x7fd5200069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.301+0000 7fd535ffb700 1 -- 172.21.2.116:0/3560134129 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5300424f0 con 0x7fd538156700 2021-11-21T21:57:44.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd5380af300 con 0x7fd538156700 2021-11-21T21:57:44.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd535ffb700 1 -- 172.21.2.116:0/3560134129 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd530016030 con 0x7fd538156700 2021-11-21T21:57:44.389 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:57:44.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd51c053610 msgr2=0x7fd51c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd51c053610 0x7fd51c055ad0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fd520002f70 tx=0x7fd5200069e0).stop 2021-11-21T21:57:44.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd538156700 msgr2=0x7fd538144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd538156700 0x7fd538144d20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fd530003310 tx=0x7fd530009120).stop 2021-11-21T21:57:44.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 shutdown_connections 2021-11-21T21:57:44.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd51c053610 0x7fd51c055ad0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd538156700 0x7fd538144d20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd538144810 0x7fd53814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 --2- 172.21.2.116:0/3560134129 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5380abce0 0x7fd53813cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 >> 172.21.2.116:0/3560134129 conn(0x7fd53805bd20 msgr2=0x7fd5380b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 shutdown_connections 2021-11-21T21:57:44.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.389+0000 7fd53f0da700 1 -- 172.21.2.116:0/3560134129 wait complete. 2021-11-21T21:57:44.396 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T21:57:44.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.445+0000 7f0ccbfe3700 1 Processor -- start 2021-11-21T21:57:44.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.445+0000 7f0ccbfe3700 1 -- start start 2021-11-21T21:57:44.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0cc40b24a0 0x7f0cc40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0cc40aed90 0x7f0cc413c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40b00d0 con 0x7f0cc40af360 2021-11-21T21:57:44.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc40adba0 con 0x7f0cc40b24a0 2021-11-21T21:57:44.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc40af7a0 con 0x7f0cc40aed90 2021-11-21T21:57:44.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0cc40af360 0x7f0cc40ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59416/0 (socket says 172.21.2.116:59416) 2021-11-21T21:57:44.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 -- 172.21.2.116:0/4045713240 learned_addr learned my addr 172.21.2.116:0/4045713240 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:44.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc9d7f700 1 --2- 172.21.2.116:0/4045713240 >> v2:172.21.2.116:3300/0 conn(0x7f0cc40b24a0 0x7f0cc40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc8d7d700 1 -- 172.21.2.116:0/4045713240 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083761720 0 0) 0x7f0cc40b00d0 con 0x7f0cc40af360 2021-11-21T21:57:44.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc8d7d700 1 -- 172.21.2.116:0/4045713240 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cac003260 con 0x7f0cc40af360 2021-11-21T21:57:44.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc9d7f700 1 -- 172.21.2.116:0/4045713240 >> v2:172.21.2.117:3300/0 conn(0x7f0cc40aed90 msgr2=0x7f0cc413c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:44.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc9d7f700 1 --2- 172.21.2.116:0/4045713240 >> v2:172.21.2.117:3300/0 conn(0x7f0cc40aed90 0x7f0cc413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc9d7f700 1 -- 172.21.2.116:0/4045713240 >> v1:172.21.2.116:6789/0 conn(0x7f0cc40af360 legacy=0x7f0cc40ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc9d7f700 1 -- 172.21.2.116:0/4045713240 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40b2880 con 0x7f0cc40b24a0 2021-11-21T21:57:44.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc9d7f700 1 --2- 172.21.2.116:0/4045713240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40b24a0 0x7f0cc40b1a40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f0cb4002f70 tx=0x7f0cb4007f00).ready entity=mon.0 client_cookie=c06aa966338a0fe server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc8d7d700 1 -- 172.21.2.116:0/4045713240 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cb4007610 con 0x7f0cc40b24a0 2021-11-21T21:57:44.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc8d7d700 1 -- 172.21.2.116:0/4045713240 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb40056f0 con 0x7f0cc40b24a0 2021-11-21T21:57:44.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc8d7d700 1 -- 172.21.2.116:0/4045713240 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cb4005a50 con 0x7f0cc40b24a0 2021-11-21T21:57:44.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/4045713240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40b24a0 msgr2=0x7f0cc40b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/4045713240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40b24a0 0x7f0cc40b1a40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f0cb4002f70 tx=0x7f0cb4007f00).stop 2021-11-21T21:57:44.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/4045713240 shutdown_connections 2021-11-21T21:57:44.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/4045713240 >> v2:172.21.2.117:3300/0 conn(0x7f0cc40aed90 0x7f0cc413c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/4045713240 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40b24a0 0x7f0cc40b1a40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/4045713240 >> 172.21.2.116:0/4045713240 conn(0x7f0cc405fd00 msgr2=0x7f0cc40600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/4045713240 shutdown_connections 2021-11-21T21:57:44.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/4045713240 wait complete. 2021-11-21T21:57:44.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 Processor -- start 2021-11-21T21:57:44.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- start start 2021-11-21T21:57:44.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0cc40aed90 0x7f0cc414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40af360 0x7f0cc4065430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0cc40b24a0 0x7f0cc4066f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4110130 con 0x7f0cc40af360 2021-11-21T21:57:44.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41103d0 con 0x7f0cc40aed90 2021-11-21T21:57:44.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4110670 con 0x7f0cc40b24a0 2021-11-21T21:57:44.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40af360 0x7f0cc4065430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40af360 0x7f0cc4065430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47466/0 (socket says 172.21.2.116:47466) 2021-11-21T21:57:44.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 -- 172.21.2.116:0/3518296864 learned_addr learned my addr 172.21.2.116:0/3518296864 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:44.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 -- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0cc40b24a0 msgr2=0x7f0cc4066f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0cc40b24a0 0x7f0cc4066f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 -- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0cc40aed90 msgr2=0x7f0cc414d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0cc40aed90 0x7f0cc414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 -- 172.21.2.116:0/3518296864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb400c200 con 0x7f0cc40af360 2021-11-21T21:57:44.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cc957e700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40af360 0x7f0cc4065430 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0cc0001110 tx=0x7f0cc0007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cb2ffd700 1 -- 172.21.2.116:0/3518296864 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cc00072e0 con 0x7f0cc40af360 2021-11-21T21:57:44.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cb2ffd700 1 -- 172.21.2.116:0/3518296864 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc0007440 con 0x7f0cc40af360 2021-11-21T21:57:44.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc410fdb0 con 0x7f0cc40af360 2021-11-21T21:57:44.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0cb2ffd700 1 -- 172.21.2.116:0/3518296864 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cc0005800 con 0x7f0cc40af360 2021-11-21T21:57:44.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.449+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc4066960 con 0x7f0cc40af360 2021-11-21T21:57:44.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.453+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc4110830 con 0x7f0cc40af360 2021-11-21T21:57:44.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.453+0000 7f0cb2ffd700 1 -- 172.21.2.116:0/3518296864 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0cc0010070 con 0x7f0cc40af360 2021-11-21T21:57:44.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.453+0000 7f0cb2ffd700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ca40534f0 0x7f0ca40559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.453+0000 7f0cb2ffd700 1 -- 172.21.2.116:0/3518296864 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0cc006ad90 con 0x7f0cc40af360 2021-11-21T21:57:44.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.453+0000 7f0cc9d7f700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ca40534f0 0x7f0ca40559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.453+0000 7f0cc9d7f700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ca40534f0 0x7f0ca40559b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f0cb4002f70 tx=0x7f0cb4017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.453+0000 7f0cb2ffd700 1 -- 172.21.2.116:0/3518296864 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0cc00421e0 con 0x7f0cc40af360 2021-11-21T21:57:44.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.537+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0cc40573d0 con 0x7f0cc40af360 2021-11-21T21:57:44.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.537+0000 7f0cb2ffd700 1 -- 172.21.2.116:0/3518296864 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0cc0018070 con 0x7f0cc40af360 2021-11-21T21:57:44.539 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:44.540 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T21:57:44.540 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:44.540 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T21:57:44.540 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T21:57:44.541 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:57:44.541 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:57:44.541 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T21:57:44.541 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T21:57:44.541 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:57:44.542 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T21:57:44.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ca40534f0 msgr2=0x7f0ca40559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ca40534f0 0x7f0ca40559b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f0cb4002f70 tx=0x7f0cb4017000).stop 2021-11-21T21:57:44.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40af360 msgr2=0x7f0cc4065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40af360 0x7f0cc4065430 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0cc0001110 tx=0x7f0cc0007ad0).stop 2021-11-21T21:57:44.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 shutdown_connections 2021-11-21T21:57:44.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0cc40b24a0 0x7f0cc4066f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ca40534f0 0x7f0ca40559b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0cc40af360 0x7f0cc4065430 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 --2- 172.21.2.116:0/3518296864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0cc40aed90 0x7f0cc414d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 >> 172.21.2.116:0/3518296864 conn(0x7f0cc405fd00 msgr2=0x7f0cc413d790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 shutdown_connections 2021-11-21T21:57:44.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.541+0000 7f0ccbfe3700 1 -- 172.21.2.116:0/3518296864 wait complete. 2021-11-21T21:57:44.546 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T21:57:44.547 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T21:57:44.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 Processor -- start 2021-11-21T21:57:44.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- start start 2021-11-21T21:57:44.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc05ced0 0x7f45cc0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45cc061060 0x7f45cc09a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45cc09b180 con 0x7f45cc09a7f0 2021-11-21T21:57:44.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45cc09b3e0 con 0x7f45cc061060 2021-11-21T21:57:44.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc09b680 con 0x7f45cc05ced0 2021-11-21T21:57:44.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45cc061060 0x7f45cc09a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45cc061060 0x7f45cc09a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47476/0 (socket says 172.21.2.116:47476) 2021-11-21T21:57:44.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/2899839346 learned_addr learned my addr 172.21.2.116:0/2899839346 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:44.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/2899839346 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc05ced0 msgr2=0x7f45cc0acb80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:44.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- 172.21.2.116:0/2899839346 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc05ced0 0x7f45cc0acb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/2899839346 >> v1:172.21.2.117:6789/0 conn(0x7f45cc09a7f0 legacy=0x7f45cc146070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:44.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/2899839346 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc09b920 con 0x7f45cc061060 2021-11-21T21:57:44.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- 172.21.2.116:0/2899839346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc09a300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f45cc0991e0 tx=0x7f45c0006eb0).ready entity=mon.0 client_cookie=527637a854a8ab64 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45bffff700 1 -- 172.21.2.116:0/2899839346 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c000d570 con 0x7f45cc061060 2021-11-21T21:57:44.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45bffff700 1 -- 172.21.2.116:0/2899839346 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c0000e90 con 0x7f45cc061060 2021-11-21T21:57:44.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45bffff700 1 -- 172.21.2.116:0/2899839346 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c000d900 con 0x7f45cc061060 2021-11-21T21:57:44.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- 172.21.2.116:0/2899839346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 msgr2=0x7f45cc09a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:44.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- 172.21.2.116:0/2899839346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc09a300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f45cc0991e0 tx=0x7f45c0006eb0).stop 2021-11-21T21:57:44.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- 172.21.2.116:0/2899839346 shutdown_connections 2021-11-21T21:57:44.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- 172.21.2.116:0/2899839346 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc09a300 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:44.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- 172.21.2.116:0/2899839346 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc05ced0 0x7f45cc0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- 172.21.2.116:0/2899839346 >> 172.21.2.116:0/2899839346 conn(0x7f45cc0b5de0 msgr2=0x7f45cc0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:44.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- 172.21.2.116:0/2899839346 shutdown_connections 2021-11-21T21:57:44.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- 172.21.2.116:0/2899839346 wait complete. 2021-11-21T21:57:44.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 Processor -- start 2021-11-21T21:57:44.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- start start 2021-11-21T21:57:44.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45cc05ced0 0x7f45cc063f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc0b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc09a7f0 0x7f45cc0649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc064ec0 con 0x7f45cc061060 2021-11-21T21:57:44.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc0aff50 con 0x7f45cc05ced0 2021-11-21T21:57:44.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc0b01f0 con 0x7f45cc09a7f0 2021-11-21T21:57:44.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc0b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc0b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47478/0 (socket says 172.21.2.116:47478) 2021-11-21T21:57:44.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/1797344588 learned_addr learned my addr 172.21.2.116:0/1797344588 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:44.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc09a7f0 msgr2=0x7f45cc0649d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:44.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc09a7f0 0x7f45cc0649d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45cc05ced0 msgr2=0x7f45cc063f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:44.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45cc05ced0 0x7f45cc063f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:44.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 -- 172.21.2.116:0/1797344588 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c000b1e0 con 0x7f45cc061060 2021-11-21T21:57:44.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d08c1700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc0b80a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f45cc0991e0 tx=0x7f45c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45be7fc700 1 -- 172.21.2.116:0/1797344588 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c0002f50 con 0x7f45cc061060 2021-11-21T21:57:44.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45be7fc700 1 -- 172.21.2.116:0/1797344588 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c0011ce0 con 0x7f45cc061060 2021-11-21T21:57:44.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc065100 con 0x7f45cc061060 2021-11-21T21:57:44.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45be7fc700 1 -- 172.21.2.116:0/1797344588 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c001e650 con 0x7f45cc061060 2021-11-21T21:57:44.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.601+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc159a50 con 0x7f45cc061060 2021-11-21T21:57:44.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.605+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc0573d0 con 0x7f45cc061060 2021-11-21T21:57:44.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.605+0000 7f45be7fc700 1 -- 172.21.2.116:0/1797344588 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f45c0025030 con 0x7f45cc061060 2021-11-21T21:57:44.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.605+0000 7f45be7fc700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45ac0534f0 0x7f45ac0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:44.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.605+0000 7f45be7fc700 1 -- 172.21.2.116:0/1797344588 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f45c006cc10 con 0x7f45cc061060 2021-11-21T21:57:44.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.605+0000 7f45d10c2700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45ac0534f0 0x7f45ac0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:44.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.605+0000 7f45d10c2700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45ac0534f0 0x7f45ac0559b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f45c8002fb0 tx=0x7f45c8006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:44.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.605+0000 7f45be7fc700 1 -- 172.21.2.116:0/1797344588 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45c0043f90 con 0x7f45cc061060 2021-11-21T21:57:44.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:44.689+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f45cc0658e0 con 0x7f45cc061060 2021-11-21T21:57:45.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45be7fc700 1 -- 172.21.2.116:0/1797344588 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v52) v1 ==== 133+0+0 (secure 0 0 0) 0x7f45c0023070 con 0x7f45cc061060 2021-11-21T21:57:45.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45ac0534f0 msgr2=0x7f45ac0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:45.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45ac0534f0 0x7f45ac0559b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f45c8002fb0 tx=0x7f45c8006930).stop 2021-11-21T21:57:45.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 msgr2=0x7f45cc0b80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:45.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc0b80a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f45cc0991e0 tx=0x7f45c0015000).stop 2021-11-21T21:57:45.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 shutdown_connections 2021-11-21T21:57:45.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45cc09a7f0 0x7f45cc0649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:45.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45ac0534f0 0x7f45ac0559b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:45.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45cc061060 0x7f45cc0b80a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 --2- 172.21.2.116:0/1797344588 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45cc05ced0 0x7f45cc063f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 >> 172.21.2.116:0/1797344588 conn(0x7f45cc0b5de0 msgr2=0x7f45cc1466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 shutdown_connections 2021-11-21T21:57:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.021+0000 7f45d3326700 1 -- 172.21.2.116:0/1797344588 wait complete. 2021-11-21T21:57:45.025 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T21:57:45.028 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T21:57:45.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f247b8d4700 1 Processor -- start 2021-11-21T21:57:45.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f247b8d4700 1 -- start start 2021-11-21T21:57:45.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f247b8d4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f24740b5380 0x7f24740b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:45.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f247b8d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24740b13b0 0x7f247413c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:45.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f247b8d4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24740b0500 con 0x7f247413c6b0 2021-11-21T21:57:45.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f247b8d4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24740adc70 con 0x7f24740b5380 2021-11-21T21:57:45.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f247b8d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247413ec80 con 0x7f24740b13b0 2021-11-21T21:57:45.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f24740b5380 0x7f24740b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:45.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f24740b5380 0x7f24740b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47486/0 (socket says 172.21.2.116:47486) 2021-11-21T21:57:45.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 -- 172.21.2.116:0/2212413129 learned_addr learned my addr 172.21.2.116:0/2212413129 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:45.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 -- 172.21.2.116:0/2212413129 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24740b13b0 msgr2=0x7f247413c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:45.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 --2- 172.21.2.116:0/2212413129 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24740b13b0 0x7f247413c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:45.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 -- 172.21.2.116:0/2212413129 >> v1:172.21.2.117:6789/0 conn(0x7f247413c6b0 legacy=0x7f24741424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:45.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 -- 172.21.2.116:0/2212413129 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740b1b50 con 0x7f24740b5380 2021-11-21T21:57:45.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2479670700 1 --2- 172.21.2.116:0/2212413129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b5380 0x7f24740b5760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f24740af960 tx=0x7f2470006f40).ready entity=mon.0 client_cookie=efd8d7194ac4856 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:45.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2467fff700 1 -- 172.21.2.116:0/2212413129 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2470010070 con 0x7f24740b5380 2021-11-21T21:57:45.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.081+0000 7f2467fff700 1 -- 172.21.2.116:0/2212413129 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470000e90 con 0x7f24740b5380 2021-11-21T21:57:45.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2467fff700 1 -- 172.21.2.116:0/2212413129 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2470008860 con 0x7f24740b5380 2021-11-21T21:57:45.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2212413129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b5380 msgr2=0x7f24740b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:45.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2212413129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b5380 0x7f24740b5760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f24740af960 tx=0x7f2470006f40).stop 2021-11-21T21:57:45.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2212413129 shutdown_connections 2021-11-21T21:57:45.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2212413129 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24740b13b0 0x7f247413c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:45.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2212413129 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b5380 0x7f24740b5760 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:45.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2212413129 >> 172.21.2.116:0/2212413129 conn(0x7f2474061900 msgr2=0x7f2474069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:45.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2212413129 shutdown_connections 2021-11-21T21:57:45.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2212413129 wait complete. 2021-11-21T21:57:45.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 Processor -- start 2021-11-21T21:57:45.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- start start 2021-11-21T21:57:45.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b13b0 0x7f2474141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:45.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f24740b5380 0x7f247413d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:45.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f247413c6b0 0x7f247413a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:45.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2474141f40 con 0x7f24740b13b0 2021-11-21T21:57:45.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2474141c20 con 0x7f24740b5380 2021-11-21T21:57:45.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2474140cb0 con 0x7f247413c6b0 2021-11-21T21:57:45.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b13b0 0x7f2474141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:45.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b13b0 0x7f2474141690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47492/0 (socket says 172.21.2.116:47492) 2021-11-21T21:57:45.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 -- 172.21.2.116:0/2242412319 learned_addr learned my addr 172.21.2.116:0/2242412319 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:45.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 -- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f247413c6b0 msgr2=0x7f247413a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:45.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f247413c6b0 0x7f247413a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:45.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 -- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f24740b5380 msgr2=0x7f247413d7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:45.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f24740b5380 0x7f247413d7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:45.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 -- 172.21.2.116:0/2242412319 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470008240 con 0x7f24740b13b0 2021-11-21T21:57:45.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f2479670700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b13b0 0x7f2474141690 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f24740af960 tx=0x7f24740b5310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:45.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f24667fc700 1 -- 172.21.2.116:0/2242412319 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2470015850 con 0x7f24740b13b0 2021-11-21T21:57:45.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f24667fc700 1 -- 172.21.2.116:0/2242412319 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24700159b0 con 0x7f24740b13b0 2021-11-21T21:57:45.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2474140f50 con 0x7f24740b13b0 2021-11-21T21:57:45.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f24667fc700 1 -- 172.21.2.116:0/2242412319 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2470015d10 con 0x7f24740b13b0 2021-11-21T21:57:45.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247413e260 con 0x7f24740b13b0 2021-11-21T21:57:45.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24741413f0 con 0x7f24740b13b0 2021-11-21T21:57:45.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f24667fc700 1 -- 172.21.2.116:0/2242412319 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f247001e480 con 0x7f24740b13b0 2021-11-21T21:57:45.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f24667fc700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24540535d0 0x7f2454055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:45.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.085+0000 7f24667fc700 1 -- 172.21.2.116:0/2242412319 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f247006cef0 con 0x7f24740b13b0 2021-11-21T21:57:45.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.089+0000 7f2478e6f700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24540535d0 0x7f2454055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:45.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.089+0000 7f2478e6f700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24540535d0 0x7f2454055a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f2468002f70 tx=0x7f2468006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:45.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.089+0000 7f24667fc700 1 -- 172.21.2.116:0/2242412319 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24741413f0 con 0x7f24740b13b0 2021-11-21T21:57:45.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:45.173+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f24741413f0 con 0x7f24740b13b0 2021-11-21T21:57:46.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.033+0000 7f24667fc700 1 -- 172.21.2.116:0/2242412319 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7f24741413f0 con 0x7f24740b13b0 2021-11-21T21:57:46.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24540535d0 msgr2=0x7f2454055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24540535d0 0x7f2454055a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f2468002f70 tx=0x7f2468006a00).stop 2021-11-21T21:57:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b13b0 msgr2=0x7f2474141690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b13b0 0x7f2474141690 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f24740af960 tx=0x7f24740b5310).stop 2021-11-21T21:57:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 shutdown_connections 2021-11-21T21:57:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f247413c6b0 0x7f247413a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f24740b5380 0x7f247413d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24540535d0 0x7f2454055a90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 --2- 172.21.2.116:0/2242412319 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24740b13b0 0x7f2474141690 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 >> 172.21.2.116:0/2242412319 conn(0x7f2474061900 msgr2=0x7f24740b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 shutdown_connections 2021-11-21T21:57:46.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.037+0000 7f247b8d4700 1 -- 172.21.2.116:0/2242412319 wait complete. 2021-11-21T21:57:46.039 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T21:57:46.042 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T21:57:46.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9dfa6d8700 1 Processor -- start 2021-11-21T21:57:46.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9dfa6d8700 1 -- start start 2021-11-21T21:57:46.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9dfa6d8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 0x7f9df4147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9dfa6d8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9df4143e60 0x7f9df41442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9dfa6d8700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df40b1940 con 0x7f9df40a8670 2021-11-21T21:57:46.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9dfa6d8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df40b15a0 con 0x7f9df4143e60 2021-11-21T21:57:46.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9dfa6d8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40b1b70 con 0x7f9df40b11c0 2021-11-21T21:57:46.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df37fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9df40a8670 0x7f9df4147720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59458/0 (socket says 172.21.2.116:59458) 2021-11-21T21:57:46.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df37fe700 1 -- 172.21.2.116:0/60207779 learned_addr learned my addr 172.21.2.116:0/60207779 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:46.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497134335 0 0) 0x7f9df40b1940 con 0x7f9df40a8670 2021-11-21T21:57:46.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd4003260 con 0x7f9df40a8670 2021-11-21T21:57:46.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 --2- 172.21.2.116:0/60207779 >> v2:172.21.2.117:3300/0 conn(0x7f9df4143e60 0x7f9df41442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df3fff700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 0x7f9df4147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2102350907 0 0) 0x7f9dd4003260 con 0x7f9df40a8670 2021-11-21T21:57:46.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 msgr2=0x7f9df4147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:46.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 0x7f9df4147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 >> v2:172.21.2.117:3300/0 conn(0x7f9df4143e60 msgr2=0x7f9df41442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:46.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 --2- 172.21.2.116:0/60207779 >> v2:172.21.2.117:3300/0 conn(0x7f9df4143e60 0x7f9df41442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df40b1e10 con 0x7f9df40a8670 2021-11-21T21:57:46.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9de4001660 con 0x7f9df40a8670 2021-11-21T21:57:46.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 >> v1:172.21.2.116:6789/0 conn(0x7f9df40a8670 legacy=0x7f9df4147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 0x7f9dd40037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd4003db0 0x7f9dd4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dd40049f0 0x7f9dd4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40b15a0 con 0x7f9dd40049f0 2021-11-21T21:57:46.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40b1b70 con 0x7f9dd4003db0 2021-11-21T21:57:46.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400b880 con 0x7f9df40b11c0 2021-11-21T21:57:46.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dd40049f0 0x7f9dd4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df3fff700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 0x7f9dd40037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df37fe700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd4003db0 0x7f9dd4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 -- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 msgr2=0x7f9dd40037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:46.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df40b11c0 0x7f9dd40037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 -- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd4003db0 msgr2=0x7f9dd4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:46.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd4003db0 0x7f9dd4004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 -- 172.21.2.116:0/60207779 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df40b1e10 con 0x7f9dd40049f0 2021-11-21T21:57:46.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df8c75700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dd40049f0 0x7f9dd4004e70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f9dec00a000 tx=0x7f9dec008040).ready entity=mon.0 client_cookie=ad020a840dfb54cf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dec00c070 con 0x7f9dd40049f0 2021-11-21T21:57:46.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec00e620 con 0x7f9dd40049f0 2021-11-21T21:57:46.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.097+0000 7f9df2ffd700 1 -- 172.21.2.116:0/60207779 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dec00e980 con 0x7f9dd40049f0 2021-11-21T21:57:46.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/60207779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dd40049f0 msgr2=0x7f9dd4004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 --2- 172.21.2.116:0/60207779 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dd40049f0 0x7f9dd4004e70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f9dec00a000 tx=0x7f9dec008040).stop 2021-11-21T21:57:46.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df8c75700 1 -- 172.21.2.116:0/60207779 reap_dead start 2021-11-21T21:57:46.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/60207779 shutdown_connections 2021-11-21T21:57:46.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/60207779 >> 172.21.2.116:0/60207779 conn(0x7f9df405bd20 msgr2=0x7f9df405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/60207779 shutdown_connections 2021-11-21T21:57:46.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/60207779 wait complete. 2021-11-21T21:57:46.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 Processor -- start 2021-11-21T21:57:46.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- start start 2021-11-21T21:57:46.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd40049f0 0x7f9df413c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9df4144ff0 0x7f9df4145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df414a290 0x7f9df413b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df414b340 con 0x7f9df4144ff0 2021-11-21T21:57:46.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df413edb0 con 0x7f9dd40049f0 2021-11-21T21:57:46.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df41460a0 con 0x7f9df414a290 2021-11-21T21:57:46.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9df4144ff0 0x7f9df4145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9df4144ff0 0x7f9df4145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47514/0 (socket says 172.21.2.116:47514) 2021-11-21T21:57:46.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 -- 172.21.2.116:0/3637599229 learned_addr learned my addr 172.21.2.116:0/3637599229 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:46.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 -- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df414a290 msgr2=0x7f9df413b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df414a290 0x7f9df413b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 -- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd40049f0 msgr2=0x7f9df413c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd40049f0 0x7f9df413c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 -- 172.21.2.116:0/3637599229 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df414b0a0 con 0x7f9df4144ff0 2021-11-21T21:57:46.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df37fe700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9df4144ff0 0x7f9df4145bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9df4146230 tx=0x7f9de4001570).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df17fa700 1 -- 172.21.2.116:0/3637599229 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9de4005430 con 0x7f9df4144ff0 2021-11-21T21:57:46.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df17fa700 1 -- 172.21.2.116:0/3637599229 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de4010040 con 0x7f9df4144ff0 2021-11-21T21:57:46.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df413f010 con 0x7f9df4144ff0 2021-11-21T21:57:46.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df414a710 con 0x7f9df4144ff0 2021-11-21T21:57:46.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df17fa700 1 -- 172.21.2.116:0/3637599229 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9de4004710 con 0x7f9df4144ff0 2021-11-21T21:57:46.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df40b20b0 con 0x7f9df4144ff0 2021-11-21T21:57:46.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df17fa700 1 -- 172.21.2.116:0/3637599229 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9de4004870 con 0x7f9df4144ff0 2021-11-21T21:57:46.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df17fa700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9dd80535d0 0x7f9dd8055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df17fa700 1 -- 172.21.2.116:0/3637599229 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f9de406b820 con 0x7f9df4144ff0 2021-11-21T21:57:46.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df3fff700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9dd80535d0 0x7f9dd8055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.101+0000 7f9df3fff700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9dd80535d0 0x7f9dd8055a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f9ddc00c000 tx=0x7f9ddc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.105+0000 7f9df17fa700 1 -- 172.21.2.116:0/3637599229 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9de40429f0 con 0x7f9df4144ff0 2021-11-21T21:57:46.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.217+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f9df40573d0 con 0x7f9df4144ff0 2021-11-21T21:57:46.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9df17fa700 1 -- 172.21.2.116:0/3637599229 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7f9de4046010 con 0x7f9df4144ff0 2021-11-21T21:57:46.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9dd80535d0 msgr2=0x7f9dd8055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9dd80535d0 0x7f9dd8055a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f9ddc00c000 tx=0x7f9ddc008040).stop 2021-11-21T21:57:46.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9df4144ff0 msgr2=0x7f9df4145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9df4144ff0 0x7f9df4145bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9df4146230 tx=0x7f9de4001570).stop 2021-11-21T21:57:46.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 shutdown_connections 2021-11-21T21:57:46.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9df414a290 0x7f9df413b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9dd80535d0 0x7f9dd8055a90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9df4144ff0 0x7f9df4145bb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 --2- 172.21.2.116:0/3637599229 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dd40049f0 0x7f9df413c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 >> 172.21.2.116:0/3637599229 conn(0x7f9df405bd20 msgr2=0x7f9df4144270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 shutdown_connections 2021-11-21T21:57:46.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.221+0000 7f9dfa6d8700 1 -- 172.21.2.116:0/3637599229 wait complete. 2021-11-21T21:57:46.226 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 9 and data pool 10 2021-11-21T21:57:46.230 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:57:46.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 Processor -- start 2021-11-21T21:57:46.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- start start 2021-11-21T21:57:46.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc0a6aa0 0x7f75dc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f75dc0a86b0 0x7f75dc1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0a6e80 con 0x7f75dc1475d0 2021-11-21T21:57:46.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc0a8bc0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc14a7b0 con 0x7f75dc0a6aa0 2021-11-21T21:57:46.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f75dc0a86b0 0x7f75dc1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f75dc0a86b0 0x7f75dc1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47534/0 (socket says 172.21.2.116:47534) 2021-11-21T21:57:46.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 -- 172.21.2.116:0/1572823157 learned_addr learned my addr 172.21.2.116:0/1572823157 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:46.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 -- 172.21.2.116:0/1572823157 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc0a6aa0 msgr2=0x7f75dc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:46.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 --2- 172.21.2.116:0/1572823157 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc0a6aa0 0x7f75dc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 -- 172.21.2.116:0/1572823157 >> v1:172.21.2.117:6789/0 conn(0x7f75dc1475d0 legacy=0x7f75dc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:46.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 -- 172.21.2.116:0/1572823157 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc14aa10 con 0x7f75dc0a86b0 2021-11-21T21:57:46.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e16ca700 1 --2- 172.21.2.116:0/1572823157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc1470e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f75dc061ab0 tx=0x7f75d0006eb0).ready entity=mon.0 client_cookie=264a25a602b37e8f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e0ec9700 1 -- 172.21.2.116:0/1572823157 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d000d570 con 0x7f75dc0a86b0 2021-11-21T21:57:46.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e0ec9700 1 -- 172.21.2.116:0/1572823157 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0000e90 con 0x7f75dc0a86b0 2021-11-21T21:57:46.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e0ec9700 1 -- 172.21.2.116:0/1572823157 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d000d8c0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- 172.21.2.116:0/1572823157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 msgr2=0x7f75dc1470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 --2- 172.21.2.116:0/1572823157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc1470e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f75dc061ab0 tx=0x7f75d0006eb0).stop 2021-11-21T21:57:46.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- 172.21.2.116:0/1572823157 shutdown_connections 2021-11-21T21:57:46.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 --2- 172.21.2.116:0/1572823157 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc1470e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 --2- 172.21.2.116:0/1572823157 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc0a6aa0 0x7f75dc14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- 172.21.2.116:0/1572823157 >> 172.21.2.116:0/1572823157 conn(0x7f75dc05bd20 msgr2=0x7f75dc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- 172.21.2.116:0/1572823157 shutdown_connections 2021-11-21T21:57:46.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.281+0000 7f75e412f700 1 -- 172.21.2.116:0/1572823157 wait complete. 2021-11-21T21:57:46.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 Processor -- start 2021-11-21T21:57:46.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 -- start start 2021-11-21T21:57:46.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75dc0a6aa0 0x7f75dc15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc1475d0 0x7f75dc158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1562b0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0666b0 con 0x7f75dc0a6aa0 2021-11-21T21:57:46.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc153f20 con 0x7f75dc1475d0 2021-11-21T21:57:46.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47538/0 (socket says 172.21.2.116:47538) 2021-11-21T21:57:46.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 -- 172.21.2.116:0/3794952365 learned_addr learned my addr 172.21.2.116:0/3794952365 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:46.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 -- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc1475d0 msgr2=0x7f75dc158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc1475d0 0x7f75dc158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 -- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75dc0a6aa0 msgr2=0x7f75dc15a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75dc0a6aa0 0x7f75dc15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 -- 172.21.2.116:0/3794952365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d000b1e0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e16ca700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc143860 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f75dc061ab0 tx=0x7f75d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75ceffd700 1 -- 172.21.2.116:0/3794952365 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d0006b80 con 0x7f75dc0a86b0 2021-11-21T21:57:46.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75ceffd700 1 -- 172.21.2.116:0/3794952365 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000d8f0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc0639d0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc067430 con 0x7f75dc0a86b0 2021-11-21T21:57:46.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75ceffd700 1 -- 172.21.2.116:0/3794952365 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d001e460 con 0x7f75dc0a86b0 2021-11-21T21:57:46.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b8003410 con 0x7f75dc0a86b0 2021-11-21T21:57:46.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75ceffd700 1 -- 172.21.2.116:0/3794952365 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f75d0025030 con 0x7f75dc0a86b0 2021-11-21T21:57:46.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75ceffd700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75bc0536f0 0x7f75bc055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75ceffd700 1 -- 172.21.2.116:0/3794952365 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f75d006cc50 con 0x7f75dc0a86b0 2021-11-21T21:57:46.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e1ecb700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75bc0536f0 0x7f75bc055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.285+0000 7f75e1ecb700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75bc0536f0 0x7f75bc055bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f75d8002f70 tx=0x7f75d8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.289+0000 7f75ceffd700 1 -- 172.21.2.116:0/3794952365 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f75d0043d50 con 0x7f75dc0a86b0 2021-11-21T21:57:46.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75b80036c0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75ceffd700 1 -- 172.21.2.116:0/3794952365 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16053 (secure 0 0 0) 0x7f75d0030ed0 con 0x7f75dc0a86b0 2021-11-21T21:57:46.374 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:46.374 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":54,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:57:46.218292+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:57:44.702609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-11-21T21:57:45.184798+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:57:46.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75bc0536f0 msgr2=0x7f75bc055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75bc0536f0 0x7f75bc055bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f75d8002f70 tx=0x7f75d8006a80).stop 2021-11-21T21:57:46.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 msgr2=0x7f75dc143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc143860 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f75dc061ab0 tx=0x7f75d0015000).stop 2021-11-21T21:57:46.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 shutdown_connections 2021-11-21T21:57:46.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75dc1475d0 0x7f75dc158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75bc0536f0 0x7f75bc055bb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f75dc0a86b0 0x7f75dc143860 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 --2- 172.21.2.116:0/3794952365 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75dc0a6aa0 0x7f75dc15a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 >> 172.21.2.116:0/3794952365 conn(0x7f75dc05bd20 msgr2=0x7f75dc13f290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 shutdown_connections 2021-11-21T21:57:46.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.373+0000 7f75e412f700 1 -- 172.21.2.116:0/3794952365 wait complete. 2021-11-21T21:57:46.381 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:57:46.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 Processor -- start 2021-11-21T21:57:46.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- start start 2021-11-21T21:57:46.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe9780b5660 0x7fe97813bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780b5a40 con 0x7fe9780aff60 2021-11-21T21:57:46.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780ae540 con 0x7fe9780b1660 2021-11-21T21:57:46.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780aef20 con 0x7fe9780b5660 2021-11-21T21:57:46.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97d58f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe9780b5660 0x7fe97813bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97d58f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe9780b5660 0x7fe97813bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47542/0 (socket says 172.21.2.116:47542) 2021-11-21T21:57:46.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97d58f700 1 -- 172.21.2.116:0/1015510427 learned_addr learned my addr 172.21.2.116:0/1015510427 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:46.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97dd90700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe9780aff60 0x7fe9781495b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59498/0 (socket says 172.21.2.116:59498) 2021-11-21T21:57:46.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97d58f700 1 -- 172.21.2.116:0/1015510427 >> v1:172.21.2.117:6789/0 conn(0x7fe9780b1660 legacy=0x7fe9780b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:46.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97d58f700 1 -- 172.21.2.116:0/1015510427 >> v1:172.21.2.116:6789/0 conn(0x7fe9780aff60 legacy=0x7fe9781495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97d58f700 1 -- 172.21.2.116:0/1015510427 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780ae7e0 con 0x7fe9780b5660 2021-11-21T21:57:46.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97d58f700 1 --2- 172.21.2.116:0/1015510427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780b5660 0x7fe97813bf10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fe978063a90 tx=0x7fe974006f40).ready entity=mon.0 client_cookie=cf5b8ee94a9c312f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe96bfff700 1 -- 172.21.2.116:0/1015510427 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe97400f070 con 0x7fe9780b5660 2021-11-21T21:57:46.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe96bfff700 1 -- 172.21.2.116:0/1015510427 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe974000e90 con 0x7fe9780b5660 2021-11-21T21:57:46.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe96bfff700 1 -- 172.21.2.116:0/1015510427 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe97400a0d0 con 0x7fe9780b5660 2021-11-21T21:57:46.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- 172.21.2.116:0/1015510427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780b5660 msgr2=0x7fe97813bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/1015510427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780b5660 0x7fe97813bf10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fe978063a90 tx=0x7fe974006f40).stop 2021-11-21T21:57:46.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- 172.21.2.116:0/1015510427 shutdown_connections 2021-11-21T21:57:46.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/1015510427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780b5660 0x7fe97813bf10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- 172.21.2.116:0/1015510427 >> 172.21.2.116:0/1015510427 conn(0x7fe97805f730 msgr2=0x7fe978063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- 172.21.2.116:0/1015510427 shutdown_connections 2021-11-21T21:57:46.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- 172.21.2.116:0/1015510427 wait complete. 2021-11-21T21:57:46.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 Processor -- start 2021-11-21T21:57:46.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- start start 2021-11-21T21:57:46.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780aff60 0x7fe9780a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9780b1660 0x7fe97813daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9780b5660 0x7fe97813dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978155a70 con 0x7fe9780aff60 2021-11-21T21:57:46.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978155d10 con 0x7fe9780b1660 2021-11-21T21:57:46.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.433+0000 7fe97f7f3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe978155fb0 con 0x7fe9780b5660 2021-11-21T21:57:46.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780aff60 0x7fe9780a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780aff60 0x7fe9780a9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47548/0 (socket says 172.21.2.116:47548) 2021-11-21T21:57:46.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 -- 172.21.2.116:0/3452561347 learned_addr learned my addr 172.21.2.116:0/3452561347 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:46.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 -- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9780b5660 msgr2=0x7fe97813dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9780b5660 0x7fe97813dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 -- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9780b1660 msgr2=0x7fe97813daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9780b1660 0x7fe97813daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 -- 172.21.2.116:0/3452561347 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97400abf0 con 0x7fe9780aff60 2021-11-21T21:57:46.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97d58f700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780aff60 0x7fe9780a9540 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe978063a90 tx=0x7fe974016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe96a7fc700 1 -- 172.21.2.116:0/3452561347 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe974018070 con 0x7fe9780aff60 2021-11-21T21:57:46.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe96a7fc700 1 -- 172.21.2.116:0/3452561347 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe974019040 con 0x7fe9780aff60 2021-11-21T21:57:46.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe978156250 con 0x7fe9780aff60 2021-11-21T21:57:46.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9780ac0c0 con 0x7fe9780aff60 2021-11-21T21:57:46.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe96a7fc700 1 -- 172.21.2.116:0/3452561347 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe974008040 con 0x7fe9780aff60 2021-11-21T21:57:46.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9780aea80 con 0x7fe9780aff60 2021-11-21T21:57:46.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe96a7fc700 1 -- 172.21.2.116:0/3452561347 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe974025030 con 0x7fe9780aff60 2021-11-21T21:57:46.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe96a7fc700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9580536f0 0x7fe958055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe96a7fc700 1 -- 172.21.2.116:0/3452561347 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe97406c6a0 con 0x7fe9780aff60 2021-11-21T21:57:46.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97cd8e700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9580536f0 0x7fe958055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.437+0000 7fe97cd8e700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9580536f0 0x7fe958055bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fe96c002f90 tx=0x7fe96c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.441+0000 7fe96a7fc700 1 -- 172.21.2.116:0/3452561347 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe974043800 con 0x7fe9780aff60 2021-11-21T21:57:46.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9781563e0 con 0x7fe9780aff60 2021-11-21T21:57:46.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe96a7fc700 1 -- 172.21.2.116:0/3452561347 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16053 (secure 0 0 0) 0x7fe974023020 con 0x7fe9780aff60 2021-11-21T21:57:46.522 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:46.523 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":54,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:57:46.218292+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:57:44.702609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-11-21T21:57:45.184798+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:57:46.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9580536f0 msgr2=0x7fe958055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9580536f0 0x7fe958055bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fe96c002f90 tx=0x7fe96c0069b0).stop 2021-11-21T21:57:46.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780aff60 msgr2=0x7fe9780a9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780aff60 0x7fe9780a9540 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe978063a90 tx=0x7fe974016000).stop 2021-11-21T21:57:46.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 shutdown_connections 2021-11-21T21:57:46.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9780b5660 0x7fe97813dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9780b1660 0x7fe97813daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9580536f0 0x7fe958055bb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 --2- 172.21.2.116:0/3452561347 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9780aff60 0x7fe9780a9540 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 >> 172.21.2.116:0/3452561347 conn(0x7fe97805f730 msgr2=0x7fe9780b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 shutdown_connections 2021-11-21T21:57:46.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.521+0000 7fe97f7f3700 1 -- 172.21.2.116:0/3452561347 wait complete. 2021-11-21T21:57:46.530 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T21:57:46.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 Processor -- start 2021-11-21T21:57:46.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 -- start start 2021-11-21T21:57:46.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1898144810 0x7f18981478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f18980abce0 0x7f18980b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f18980b1db0 0x7f18980ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1898147e80 con 0x7f18980abce0 2021-11-21T21:57:46.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18980af5a0 con 0x7f18980b1db0 2021-11-21T21:57:46.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f189d68a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b1490 con 0x7f1898144810 2021-11-21T21:57:46.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f18980abce0 0x7f18980b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f18980abce0 0x7f18980b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47560/0 (socket says 172.21.2.116:47560) 2021-11-21T21:57:46.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 -- 172.21.2.116:0/984768248 learned_addr learned my addr 172.21.2.116:0/984768248 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:46.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 -- 172.21.2.116:0/984768248 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1898144810 msgr2=0x7f18981478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:46.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 --2- 172.21.2.116:0/984768248 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1898144810 0x7f18981478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 -- 172.21.2.116:0/984768248 >> v2:172.21.2.117:3300/0 conn(0x7f18980b1db0 msgr2=0x7f18980ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:46.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 --2- 172.21.2.116:0/984768248 >> v2:172.21.2.117:3300/0 conn(0x7f18980b1db0 0x7f18980ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.581+0000 7f18967fc700 1 -- 172.21.2.116:0/984768248 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980af060 con 0x7f18980abce0 2021-11-21T21:57:46.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 --2- 172.21.2.116:0/984768248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f18980abce0 0x7f18980b18c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f1880002f70 tx=0x7f1880006eb0).ready entity=mon.0 client_cookie=57c8579f64c1533f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f1895ffb700 1 -- 172.21.2.116:0/984768248 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188000d570 con 0x7f18980abce0 2021-11-21T21:57:46.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f1895ffb700 1 -- 172.21.2.116:0/984768248 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1880000e90 con 0x7f18980abce0 2021-11-21T21:57:46.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f1895ffb700 1 -- 172.21.2.116:0/984768248 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188000d900 con 0x7f18980abce0 2021-11-21T21:57:46.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/984768248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f18980abce0 msgr2=0x7f18980b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:46.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 --2- 172.21.2.116:0/984768248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f18980abce0 0x7f18980b18c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f1880002f70 tx=0x7f1880006eb0).stop 2021-11-21T21:57:46.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/984768248 shutdown_connections 2021-11-21T21:57:46.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 --2- 172.21.2.116:0/984768248 >> v2:172.21.2.117:3300/0 conn(0x7f18980b1db0 0x7f18980ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 --2- 172.21.2.116:0/984768248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f18980abce0 0x7f18980b18c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:46.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 --2- 172.21.2.116:0/984768248 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1898144810 0x7f18981478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/984768248 >> 172.21.2.116:0/984768248 conn(0x7f189805bd20 msgr2=0x7f189805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:46.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/984768248 shutdown_connections 2021-11-21T21:57:46.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/984768248 wait complete. 2021-11-21T21:57:46.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 Processor -- start 2021-11-21T21:57:46.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- start start 2021-11-21T21:57:46.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f18980abce0 0x7f189813c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1898144810 0x7f1898156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1898146450 0x7f18981573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1898146e30 con 0x7f1898144810 2021-11-21T21:57:46.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981468f0 con 0x7f1898146450 2021-11-21T21:57:46.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1898146b90 con 0x7f18980abce0 2021-11-21T21:57:46.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1898144810 0x7f1898156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1898144810 0x7f1898156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47564/0 (socket says 172.21.2.116:47564) 2021-11-21T21:57:46.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 -- 172.21.2.116:0/4259315972 learned_addr learned my addr 172.21.2.116:0/4259315972 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:46.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 -- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f18980abce0 msgr2=0x7f189813c3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:46.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f18980abce0 0x7f189813c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 -- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1898146450 msgr2=0x7f18981573d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:57:46.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1898146450 0x7f18981573d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:46.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 -- 172.21.2.116:0/4259315972 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188000b1e0 con 0x7f1898144810 2021-11-21T21:57:46.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f18967fc700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1898144810 0x7f1898156ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f1880002f70 tx=0x7f1880015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f188ffff700 1 -- 172.21.2.116:0/4259315972 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18800119d0 con 0x7f1898144810 2021-11-21T21:57:46.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f188ffff700 1 -- 172.21.2.116:0/4259315972 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1880011b30 con 0x7f1898144810 2021-11-21T21:57:46.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1898156a40 con 0x7f1898144810 2021-11-21T21:57:46.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1898149ed0 con 0x7f1898144810 2021-11-21T21:57:46.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f188ffff700 1 -- 172.21.2.116:0/4259315972 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1880011e90 con 0x7f1898144810 2021-11-21T21:57:46.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1898156c00 con 0x7f1898144810 2021-11-21T21:57:46.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f188ffff700 1 -- 172.21.2.116:0/4259315972 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1880025030 con 0x7f1898144810 2021-11-21T21:57:46.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f188ffff700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f187c053690 0x7f187c055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:46.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.585+0000 7f188ffff700 1 -- 172.21.2.116:0/4259315972 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f188006cf10 con 0x7f1898144810 2021-11-21T21:57:46.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.589+0000 7f1896ffd700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f187c053690 0x7f187c055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:46.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.589+0000 7f1896ffd700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f187c053690 0x7f187c055b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1888000ed0 tx=0x7f18880068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:46.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.589+0000 7f188ffff700 1 -- 172.21.2.116:0/4259315972 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1880043f60 con 0x7f1898144810 2021-11-21T21:57:46.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:46.697+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f18980af300 con 0x7f1898144810 2021-11-21T21:57:47.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.241+0000 7f188ffff700 1 -- 172.21.2.116:0/4259315972 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1880023070 con 0x7f1898144810 2021-11-21T21:57:47.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f187c053690 msgr2=0x7f187c055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f187c053690 0x7f187c055b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1888000ed0 tx=0x7f18880068b0).stop 2021-11-21T21:57:47.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1898144810 msgr2=0x7f1898156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1898144810 0x7f1898156ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f1880002f70 tx=0x7f1880015000).stop 2021-11-21T21:57:47.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 shutdown_connections 2021-11-21T21:57:47.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1898146450 0x7f18981573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f187c053690 0x7f187c055b50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1898144810 0x7f1898156ee0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 --2- 172.21.2.116:0/4259315972 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f18980abce0 0x7f189813c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 >> 172.21.2.116:0/4259315972 conn(0x7f189805bd20 msgr2=0x7f1898147020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 shutdown_connections 2021-11-21T21:57:47.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.245+0000 7f189d68a700 1 -- 172.21.2.116:0/4259315972 wait complete. 2021-11-21T21:57:47.258 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:57:47.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 Processor -- start 2021-11-21T21:57:47.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- start start 2021-11-21T21:57:47.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fee38062410 0x7fee38144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee380a6e80 con 0x7fee380a6aa0 2021-11-21T21:57:47.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3814a7b0 con 0x7fee380a86b0 2021-11-21T21:57:47.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee380a8bc0 con 0x7fee38062410 2021-11-21T21:57:47.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3cc27700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fee38062410 0x7fee38144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3cc27700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fee38062410 0x7fee38144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47574/0 (socket says 172.21.2.116:47574) 2021-11-21T21:57:47.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3cc27700 1 -- 172.21.2.116:0/3464207474 learned_addr learned my addr 172.21.2.116:0/3464207474 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:47.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3cc27700 1 -- 172.21.2.116:0/3464207474 >> v1:172.21.2.117:6789/0 conn(0x7fee380a86b0 legacy=0x7fee381443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3cc27700 1 -- 172.21.2.116:0/3464207474 >> v1:172.21.2.116:6789/0 conn(0x7fee380a6aa0 legacy=0x7fee381470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3cc27700 1 -- 172.21.2.116:0/3464207474 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3814aa10 con 0x7fee38062410 2021-11-21T21:57:47.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3cc27700 1 --2- 172.21.2.116:0/3464207474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38144b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fee38061b50 tx=0x7fee30009290).ready entity=mon.0 client_cookie=fb62ffe05391ac64 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee36ffd700 1 -- 172.21.2.116:0/3464207474 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3000b860 con 0x7fee38062410 2021-11-21T21:57:47.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee36ffd700 1 -- 172.21.2.116:0/3464207474 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3000b9c0 con 0x7fee38062410 2021-11-21T21:57:47.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee36ffd700 1 -- 172.21.2.116:0/3464207474 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3000bb30 con 0x7fee38062410 2021-11-21T21:57:47.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- 172.21.2.116:0/3464207474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 msgr2=0x7fee38144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 --2- 172.21.2.116:0/3464207474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38144b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fee38061b50 tx=0x7fee30009290).stop 2021-11-21T21:57:47.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- 172.21.2.116:0/3464207474 shutdown_connections 2021-11-21T21:57:47.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 --2- 172.21.2.116:0/3464207474 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38144b00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- 172.21.2.116:0/3464207474 >> 172.21.2.116:0/3464207474 conn(0x7fee3805bd20 msgr2=0x7fee3805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- 172.21.2.116:0/3464207474 shutdown_connections 2021-11-21T21:57:47.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.309+0000 7fee3e68a700 1 -- 172.21.2.116:0/3464207474 wait complete. 2021-11-21T21:57:47.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 Processor -- start 2021-11-21T21:57:47.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 -- start start 2021-11-21T21:57:47.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38064aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee380a6aa0 0x7fee381536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee380a86b0 0x7fee38153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee381540a0 con 0x7fee38062410 2021-11-21T21:57:47.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38156930 con 0x7fee380a86b0 2021-11-21T21:57:47.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee38156bd0 con 0x7fee380a6aa0 2021-11-21T21:57:47.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38064aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38064aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47578/0 (socket says 172.21.2.116:47578) 2021-11-21T21:57:47.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 -- 172.21.2.116:0/1068093125 learned_addr learned my addr 172.21.2.116:0/1068093125 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:47.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 -- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee380a6aa0 msgr2=0x7fee381536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee380a6aa0 0x7fee381536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 -- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee380a86b0 msgr2=0x7fee38153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee380a86b0 0x7fee38153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 -- 172.21.2.116:0/1068093125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee30009530 con 0x7fee38062410 2021-11-21T21:57:47.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee37fff700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38064aa0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fee38061ab0 tx=0x7fee20007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee357fa700 1 -- 172.21.2.116:0/1068093125 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee2000ce00 con 0x7fee38062410 2021-11-21T21:57:47.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee357fa700 1 -- 172.21.2.116:0/1068093125 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20007240 con 0x7fee38062410 2021-11-21T21:57:47.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee357fa700 1 -- 172.21.2.116:0/1068093125 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee20006700 con 0x7fee38062410 2021-11-21T21:57:47.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee38156e70 con 0x7fee38062410 2021-11-21T21:57:47.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee38065010 con 0x7fee38062410 2021-11-21T21:57:47.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee380573d0 con 0x7fee38062410 2021-11-21T21:57:47.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee357fa700 1 -- 172.21.2.116:0/1068093125 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fee20010030 con 0x7fee38062410 2021-11-21T21:57:47.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee357fa700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee1c0535d0 0x7fee1c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee357fa700 1 -- 172.21.2.116:0/1068093125 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fee2006b710 con 0x7fee38062410 2021-11-21T21:57:47.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.313+0000 7fee377fe700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee1c0535d0 0x7fee1c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.317+0000 7fee377fe700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee1c0535d0 0x7fee1c055a90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fee28002f90 tx=0x7fee280069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.317+0000 7fee357fa700 1 -- 172.21.2.116:0/1068093125 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee20042700 con 0x7fee38062410 2021-11-21T21:57:47.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.425+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee3805db70 con 0x7fee38062410 2021-11-21T21:57:47.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.425+0000 7fee357fa700 1 -- 172.21.2.116:0/1068093125 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fee20045d20 con 0x7fee38062410 2021-11-21T21:57:47.427 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:47.428 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:57:47.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee1c0535d0 msgr2=0x7fee1c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee1c0535d0 0x7fee1c055a90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fee28002f90 tx=0x7fee280069b0).stop 2021-11-21T21:57:47.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 msgr2=0x7fee38064aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38064aa0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fee38061ab0 tx=0x7fee20007be0).stop 2021-11-21T21:57:47.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 shutdown_connections 2021-11-21T21:57:47.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee380a86b0 0x7fee38153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee380a6aa0 0x7fee381536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee1c0535d0 0x7fee1c055a90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 --2- 172.21.2.116:0/1068093125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee38062410 0x7fee38064aa0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 >> 172.21.2.116:0/1068093125 conn(0x7fee3805bd20 msgr2=0x7fee380b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 shutdown_connections 2021-11-21T21:57:47.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.429+0000 7fee3e68a700 1 -- 172.21.2.116:0/1068093125 wait complete. 2021-11-21T21:57:47.432 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:57:47.434 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:57:47.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 Processor -- start 2021-11-21T21:57:47.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- start start 2021-11-21T21:57:47.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f98640a86b0 0x7f986414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640a6e80 con 0x7f98640a6aa0 2021-11-21T21:57:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986414a7b0 con 0x7f9864062410 2021-11-21T21:57:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98640a8bc0 con 0x7f98640a86b0 2021-11-21T21:57:47.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9863fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f98640a86b0 0x7f986414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9863fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f98640a86b0 0x7f986414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47588/0 (socket says 172.21.2.116:47588) 2021-11-21T21:57:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9868d25700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f98640a6aa0 0x7f98641470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59540/0 (socket says 172.21.2.116:59540) 2021-11-21T21:57:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9863fff700 1 -- 172.21.2.116:0/1059569557 learned_addr learned my addr 172.21.2.116:0/1059569557 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9863fff700 1 -- 172.21.2.116:0/1059569557 >> v1:172.21.2.117:6789/0 conn(0x7f9864062410 legacy=0x7f98641443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9863fff700 1 -- 172.21.2.116:0/1059569557 >> v1:172.21.2.116:6789/0 conn(0x7f98640a6aa0 legacy=0x7f98641470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9863fff700 1 -- 172.21.2.116:0/1059569557 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986414aa10 con 0x7f98640a86b0 2021-11-21T21:57:47.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f9863fff700 1 --2- 172.21.2.116:0/1059569557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98640a86b0 0x7f986414b150 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f9864061b50 tx=0x7f984c006eb0).ready entity=mon.0 client_cookie=1e1d23cc1dbe5f28 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f98637fe700 1 -- 172.21.2.116:0/1059569557 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984c00d610 con 0x7f98640a86b0 2021-11-21T21:57:47.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f98637fe700 1 -- 172.21.2.116:0/1059569557 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c000e90 con 0x7f98640a86b0 2021-11-21T21:57:47.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f98637fe700 1 -- 172.21.2.116:0/1059569557 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984c00d960 con 0x7f98640a86b0 2021-11-21T21:57:47.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- 172.21.2.116:0/1059569557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98640a86b0 msgr2=0x7f986414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 --2- 172.21.2.116:0/1059569557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98640a86b0 0x7f986414b150 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f9864061b50 tx=0x7f984c006eb0).stop 2021-11-21T21:57:47.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- 172.21.2.116:0/1059569557 shutdown_connections 2021-11-21T21:57:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 --2- 172.21.2.116:0/1059569557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98640a86b0 0x7f986414b150 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- 172.21.2.116:0/1059569557 >> 172.21.2.116:0/1059569557 conn(0x7f986405bd20 msgr2=0x7f986405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- 172.21.2.116:0/1059569557 shutdown_connections 2021-11-21T21:57:47.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.485+0000 7f986af89700 1 -- 172.21.2.116:0/1059569557 wait complete. 2021-11-21T21:57:47.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 Processor -- start 2021-11-21T21:57:47.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 -- start start 2021-11-21T21:57:47.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9864062410 0x7f9864148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98640a6aa0 0x7f986415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98640a86b0 0x7f98640649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98641599b0 con 0x7f9864062410 2021-11-21T21:57:47.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9864067130 con 0x7f98640a6aa0 2021-11-21T21:57:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9864159be0 con 0x7f98640a86b0 2021-11-21T21:57:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9864062410 0x7f9864148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9864062410 0x7f9864148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47590/0 (socket says 172.21.2.116:47590) 2021-11-21T21:57:47.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 -- 172.21.2.116:0/1143153958 learned_addr learned my addr 172.21.2.116:0/1143153958 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 -- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98640a86b0 msgr2=0x7f98640649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98640a86b0 0x7f98640649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 -- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98640a6aa0 msgr2=0x7f986415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98640a6aa0 0x7f986415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 -- 172.21.2.116:0/1143153958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c00b1e0 con 0x7f9864062410 2021-11-21T21:57:47.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9868d25700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9864062410 0x7f9864148e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f9864061ab0 tx=0x7f9854007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9861ffb700 1 -- 172.21.2.116:0/1143153958 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f985400ce30 con 0x7f9864062410 2021-11-21T21:57:47.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9861ffb700 1 -- 172.21.2.116:0/1143153958 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9854007170 con 0x7f9864062410 2021-11-21T21:57:47.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9861ffb700 1 -- 172.21.2.116:0/1143153958 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9854004730 con 0x7f9864062410 2021-11-21T21:57:47.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98641492f0 con 0x7f9864062410 2021-11-21T21:57:47.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98640673d0 con 0x7f9864062410 2021-11-21T21:57:47.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9864159da0 con 0x7f9864062410 2021-11-21T21:57:47.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9861ffb700 1 -- 172.21.2.116:0/1143153958 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f985401b070 con 0x7f9864062410 2021-11-21T21:57:47.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9861ffb700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9848053580 0x7f9848055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9861ffb700 1 -- 172.21.2.116:0/1143153958 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f985406aef0 con 0x7f9864062410 2021-11-21T21:57:47.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.489+0000 7f9863fff700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9848053580 0x7f9848055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.493+0000 7f9863fff700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9848053580 0x7f9848055a40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9864061b50 tx=0x7f984c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.493+0000 7f9861ffb700 1 -- 172.21.2.116:0/1143153958 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9854042050 con 0x7f9864062410 2021-11-21T21:57:47.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.573+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98640573d0 con 0x7f9864062410 2021-11-21T21:57:47.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.573+0000 7f9861ffb700 1 -- 172.21.2.116:0/1143153958 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16040 (secure 0 0 0) 0x7f9854045670 con 0x7f9864062410 2021-11-21T21:57:47.575 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:47.576 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":55,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:57:47.221698+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:57:44.702609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-11-21T21:57:45.184798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:57:47.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9848053580 msgr2=0x7f9848055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9848053580 0x7f9848055a40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9864061b50 tx=0x7f984c015000).stop 2021-11-21T21:57:47.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9864062410 msgr2=0x7f9864148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9864062410 0x7f9864148e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f9864061ab0 tx=0x7f9854007bf0).stop 2021-11-21T21:57:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 shutdown_connections 2021-11-21T21:57:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98640a86b0 0x7f98640649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98640a6aa0 0x7f986415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9848053580 0x7f9848055a40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 --2- 172.21.2.116:0/1143153958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9864062410 0x7f9864148e00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 >> 172.21.2.116:0/1143153958 conn(0x7f986405bd20 msgr2=0x7f9864142690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 shutdown_connections 2021-11-21T21:57:47.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.577+0000 7f986af89700 1 -- 172.21.2.116:0/1143153958 wait complete. 2021-11-21T21:57:47.583 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:57:47.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.637+0000 7fe9c8643700 1 Processor -- start 2021-11-21T21:57:47.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.637+0000 7fe9c8643700 1 -- start start 2021-11-21T21:57:47.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c0062610 0x7fe9c00adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c005cb80 con 0x7fe9c00b0d00 2021-11-21T21:57:47.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c0143ed0 con 0x7fe9c00b10e0 2021-11-21T21:57:47.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00a94c0 con 0x7fe9c0062610 2021-11-21T21:57:47.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c63df700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe9c00b0d00 0x7fe9c01430b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59554/0 (socket says 172.21.2.116:59554) 2021-11-21T21:57:47.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c63df700 1 -- 172.21.2.116:0/594365030 learned_addr learned my addr 172.21.2.116:0/594365030 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:47.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822404823 0 0) 0x7fe9c005cb80 con 0x7fe9c00b0d00 2021-11-21T21:57:47.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a8003260 con 0x7fe9c00b0d00 2021-11-21T21:57:47.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 114803546 0 0) 0x7fe9a8003260 con 0x7fe9c00b0d00 2021-11-21T21:57:47.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c0062610 msgr2=0x7fe9c00adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c0062610 0x7fe9c00adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 >> v1:172.21.2.117:6789/0 conn(0x7fe9c00b10e0 legacy=0x7fe9c01437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00af320 con 0x7fe9c00b0d00 2021-11-21T21:57:47.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe9b4001660 con 0x7fe9c00b0d00 2021-11-21T21:57:47.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 >> v1:172.21.2.116:6789/0 conn(0x7fe9c00b0d00 legacy=0x7fe9c01430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c0062610 0x7fe9a8003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9a8003ef0 0x7fe9a8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9a800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80052c0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a800bc40 con 0x7fe9a8003ef0 2021-11-21T21:57:47.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a800bd80 con 0x7fe9c0062610 2021-11-21T21:57:47.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9a800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c63df700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c0062610 0x7fe9a8003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 -- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c0062610 msgr2=0x7fe9a8003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c0062610 0x7fe9a8003800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 -- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9a8003ef0 msgr2=0x7fe9a8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c5bde700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9a8003ef0 0x7fe9a8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9a8003ef0 0x7fe9a8004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 -- 172.21.2.116:0/594365030 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8003260 con 0x7fe9a8004bf0 2021-11-21T21:57:47.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9a800b750 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fe9b80012f0 tx=0x7fe9b8007d60).ready entity=mon.0 client_cookie=683adf3a4140e7ec server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b80014c0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8006870 con 0x7fe9a8004bf0 2021-11-21T21:57:47.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c53dd700 1 -- 172.21.2.116:0/594365030 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b8006bd0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- 172.21.2.116:0/594365030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 msgr2=0x7fe9a800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 --2- 172.21.2.116:0/594365030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9a800b750 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fe9b80012f0 tx=0x7fe9b8007d60).stop 2021-11-21T21:57:47.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c6be0700 1 -- 172.21.2.116:0/594365030 reap_dead start 2021-11-21T21:57:47.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- 172.21.2.116:0/594365030 shutdown_connections 2021-11-21T21:57:47.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- 172.21.2.116:0/594365030 >> 172.21.2.116:0/594365030 conn(0x7fe9c005bd20 msgr2=0x7fe9c005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- 172.21.2.116:0/594365030 shutdown_connections 2021-11-21T21:57:47.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- 172.21.2.116:0/594365030 wait complete. 2021-11-21T21:57:47.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 Processor -- start 2021-11-21T21:57:47.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- start start 2021-11-21T21:57:47.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9c0128540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c00a9c50 0x7fe9c0133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9c0133d10 0x7fe9c0138a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00aa650 con 0x7fe9a8004bf0 2021-11-21T21:57:47.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c012af40 con 0x7fe9c0133d10 2021-11-21T21:57:47.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c8643700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c012a5b0 con 0x7fe9c00a9c50 2021-11-21T21:57:47.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c63df700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9c0128540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c63df700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9c0128540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47612/0 (socket says 172.21.2.116:47612) 2021-11-21T21:57:47.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c63df700 1 -- 172.21.2.116:0/3137427788 learned_addr learned my addr 172.21.2.116:0/3137427788 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:47.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.641+0000 7fe9c63df700 1 -- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c00a9c50 msgr2=0x7fe9c0133820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c63df700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c00a9c50 0x7fe9c0133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c63df700 1 -- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9c0133d10 msgr2=0x7fe9c0138a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c63df700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9c0133d10 0x7fe9c0138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c63df700 1 -- 172.21.2.116:0/3137427788 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0137ad0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c63df700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9c0128540 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fe9b400b000 tx=0x7fe9c00a8df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9b37fe700 1 -- 172.21.2.116:0/3137427788 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b4010680 con 0x7fe9a8004bf0 2021-11-21T21:57:47.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9b37fe700 1 -- 172.21.2.116:0/3137427788 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4010040 con 0x7fe9a8004bf0 2021-11-21T21:57:47.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9b37fe700 1 -- 172.21.2.116:0/3137427788 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b40109d0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c0137d70 con 0x7fe9a8004bf0 2021-11-21T21:57:47.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c00ab700 con 0x7fe9a8004bf0 2021-11-21T21:57:47.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c00af5c0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9b37fe700 1 -- 172.21.2.116:0/3137427788 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe9b4010b30 con 0x7fe9a8004bf0 2021-11-21T21:57:47.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9b37fe700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9a00534f0 0x7fe9a00559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9b37fe700 1 -- 172.21.2.116:0/3137427788 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe9b406b5f0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c5bde700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9a00534f0 0x7fe9a00559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9c5bde700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9a00534f0 0x7fe9a00559b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fe9bc007000 tx=0x7fe9bc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.645+0000 7fe9b37fe700 1 -- 172.21.2.116:0/3137427788 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe9b4042640 con 0x7fe9a8004bf0 2021-11-21T21:57:47.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.757+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe9c00573d0 con 0x7fe9a8004bf0 2021-11-21T21:57:47.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.765+0000 7fe9b37fe700 1 -- 172.21.2.116:0/3137427788 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe9b401a070 con 0x7fe9a8004bf0 2021-11-21T21:57:47.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9a00534f0 msgr2=0x7fe9a00559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9a00534f0 0x7fe9a00559b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fe9bc007000 tx=0x7fe9bc008040).stop 2021-11-21T21:57:47.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 msgr2=0x7fe9c0128540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9c0128540 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fe9b400b000 tx=0x7fe9c00a8df0).stop 2021-11-21T21:57:47.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 shutdown_connections 2021-11-21T21:57:47.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe9c0133d10 0x7fe9c0138a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9c00a9c50 0x7fe9c0133820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:47.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9a00534f0 0x7fe9a00559b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 --2- 172.21.2.116:0/3137427788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9a8004bf0 0x7fe9c0128540 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 >> 172.21.2.116:0/3137427788 conn(0x7fe9c005bd20 msgr2=0x7fe9c00b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 shutdown_connections 2021-11-21T21:57:47.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.769+0000 7fe9c8643700 1 -- 172.21.2.116:0/3137427788 wait complete. 2021-11-21T21:57:47.774 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T21:57:47.781 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:57:47.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 Processor -- start 2021-11-21T21:57:47.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- start start 2021-11-21T21:57:47.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c0a8670 0x7f732c147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f732c143e60 0x7f732c1442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0b1940 con 0x7f732c0b11c0 2021-11-21T21:57:47.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c0b15a0 con 0x7f732c143e60 2021-11-21T21:57:47.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0b1b70 con 0x7f732c0a8670 2021-11-21T21:57:47.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f732c143e60 0x7f732c1442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f732c143e60 0x7f732c1442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47620/0 (socket says 172.21.2.116:47620) 2021-11-21T21:57:47.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 -- 172.21.2.116:0/3762874641 learned_addr learned my addr 172.21.2.116:0/3762874641 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:47.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7330d32700 1 --2- 172.21.2.116:0/3762874641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c0a8670 0x7f732c147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 -- 172.21.2.116:0/3762874641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c0a8670 msgr2=0x7f732c147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 --2- 172.21.2.116:0/3762874641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c0a8670 0x7f732c147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 -- 172.21.2.116:0/3762874641 >> v1:172.21.2.117:6789/0 conn(0x7f732c0b11c0 legacy=0x7f732c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:47.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 -- 172.21.2.116:0/3762874641 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0b1e10 con 0x7f732c143e60 2021-11-21T21:57:47.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7330d32700 1 --2- 172.21.2.116:0/3762874641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c0a8670 0x7f732c147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:57:47.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7331d34700 1 --2- 172.21.2.116:0/3762874641 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c143e60 0x7f732c1442f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f732c0a8390 tx=0x7f7324009290).ready entity=mon.0 client_cookie=413c9feee15e8ff4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f731ffff700 1 -- 172.21.2.116:0/3762874641 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f732400b860 con 0x7f732c143e60 2021-11-21T21:57:47.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f731ffff700 1 -- 172.21.2.116:0/3762874641 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732400b9c0 con 0x7f732c143e60 2021-11-21T21:57:47.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f731ffff700 1 -- 172.21.2.116:0/3762874641 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f732400bb70 con 0x7f732c143e60 2021-11-21T21:57:47.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- 172.21.2.116:0/3762874641 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c143e60 msgr2=0x7f732c1442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 --2- 172.21.2.116:0/3762874641 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c143e60 0x7f732c1442f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f732c0a8390 tx=0x7f7324009290).stop 2021-11-21T21:57:47.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- 172.21.2.116:0/3762874641 shutdown_connections 2021-11-21T21:57:47.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 --2- 172.21.2.116:0/3762874641 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c143e60 0x7f732c1442f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 --2- 172.21.2.116:0/3762874641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c0a8670 0x7f732c147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- 172.21.2.116:0/3762874641 >> 172.21.2.116:0/3762874641 conn(0x7f732c05bd20 msgr2=0x7f732c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- 172.21.2.116:0/3762874641 shutdown_connections 2021-11-21T21:57:47.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- 172.21.2.116:0/3762874641 wait complete. 2021-11-21T21:57:47.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 Processor -- start 2021-11-21T21:57:47.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.841+0000 7f7333797700 1 -- start start 2021-11-21T21:57:47.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c0a8670 0x7f732c14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732c0b11c0 0x7f732c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c143e60 0x7f732c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c064ba0 con 0x7f732c0a8670 2021-11-21T21:57:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c064340 con 0x7f732c0b11c0 2021-11-21T21:57:47.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0b4950 con 0x7f732c143e60 2021-11-21T21:57:47.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c0a8670 0x7f732c14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c0a8670 0x7f732c14ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47626/0 (socket says 172.21.2.116:47626) 2021-11-21T21:57:47.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 -- 172.21.2.116:0/912811819 learned_addr learned my addr 172.21.2.116:0/912811819 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:47.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7330d32700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732c0b11c0 0x7f732c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331d34700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c143e60 0x7f732c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 -- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c143e60 msgr2=0x7f732c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c143e60 0x7f732c1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 -- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732c0b11c0 msgr2=0x7f732c1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732c0b11c0 0x7f732c1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 -- 172.21.2.116:0/912811819 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324009530 con 0x7f732c0a8670 2021-11-21T21:57:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7330d32700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732c0b11c0 0x7f732c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:57:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7331533700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c0a8670 0x7f732c14ac20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f7320001fa0 tx=0x7f7320007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f731e7fc700 1 -- 172.21.2.116:0/912811819 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f732000ce80 con 0x7f732c0a8670 2021-11-21T21:57:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f731e7fc700 1 -- 172.21.2.116:0/912811819 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7320007240 con 0x7f732c0a8670 2021-11-21T21:57:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c0b4c50 con 0x7f732c0a8670 2021-11-21T21:57:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c153f20 con 0x7f732c0a8670 2021-11-21T21:57:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f731e7fc700 1 -- 172.21.2.116:0/912811819 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7320006510 con 0x7f732c0a8670 2021-11-21T21:57:47.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c0b20b0 con 0x7f732c0a8670 2021-11-21T21:57:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f731e7fc700 1 -- 172.21.2.116:0/912811819 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7320010030 con 0x7f732c0a8670 2021-11-21T21:57:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f731e7fc700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f730c053640 0x7f730c055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f731e7fc700 1 -- 172.21.2.116:0/912811819 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f732006b4a0 con 0x7f732c0a8670 2021-11-21T21:57:47.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7330d32700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f730c053640 0x7f730c055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:47.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.845+0000 7f7330d32700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f730c053640 0x7f730c055b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f732800b000 tx=0x7f7328008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:47.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.849+0000 7f731e7fc700 1 -- 172.21.2.116:0/912811819 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7320042600 con 0x7f732c0a8670 2021-11-21T21:57:47.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.957+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f732c0573d0 con 0x7f732c0a8670 2021-11-21T21:57:47.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.965+0000 7f731e7fc700 1 -- 172.21.2.116:0/912811819 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7320045c20 con 0x7f732c0a8670 2021-11-21T21:57:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f730c053640 msgr2=0x7f730c055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f730c053640 0x7f730c055b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f732800b000 tx=0x7f7328008040).stop 2021-11-21T21:57:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c0a8670 msgr2=0x7f732c14ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c0a8670 0x7f732c14ac20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f7320001fa0 tx=0x7f7320007c30).stop 2021-11-21T21:57:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 shutdown_connections 2021-11-21T21:57:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f732c143e60 0x7f732c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732c0b11c0 0x7f732c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f730c053640 0x7f730c055b00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 --2- 172.21.2.116:0/912811819 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f732c0a8670 0x7f732c14ac20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:47.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 >> 172.21.2.116:0/912811819 conn(0x7f732c05bd20 msgr2=0x7f732c146120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:47.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 shutdown_connections 2021-11-21T21:57:47.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:47.969+0000 7f7333797700 1 -- 172.21.2.116:0/912811819 wait complete. 2021-11-21T21:57:47.972 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T21:57:47.981 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:57:48.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd359e700 1 Processor -- start 2021-11-21T21:57:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd359e700 1 -- start start 2021-11-21T21:57:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd359e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3cd40b5630 0x7f3cd414af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd359e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4099650 0x7f3cd4099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd359e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd414a560 con 0x7f3cd4097ec0 2021-11-21T21:57:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd359e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd4144b40 con 0x7f3cd40b5630 2021-11-21T21:57:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd359e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd414a800 con 0x7f3cd4099650 2021-11-21T21:57:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3cd40b5630 0x7f3cd414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3cd40b5630 0x7f3cd414af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47634/0 (socket says 172.21.2.116:47634) 2021-11-21T21:57:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 -- 172.21.2.116:0/1767879582 learned_addr learned my addr 172.21.2.116:0/1767879582 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 -- 172.21.2.116:0/1767879582 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4099650 msgr2=0x7f3cd4099ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 --2- 172.21.2.116:0/1767879582 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4099650 0x7f3cd4099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 -- 172.21.2.116:0/1767879582 >> v1:172.21.2.117:6789/0 conn(0x7f3cd4097ec0 legacy=0x7f3cd4146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 -- 172.21.2.116:0/1767879582 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd414aaa0 con 0x7f3cd40b5630 2021-11-21T21:57:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.033+0000 7f3cd259c700 1 --2- 172.21.2.116:0/1767879582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd40b5630 0x7f3cd414af40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3cd409afa0 tx=0x7f3cc4006eb0).ready entity=mon.0 client_cookie=18aa40c6405d2b3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd159a700 1 -- 172.21.2.116:0/1767879582 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc400d570 con 0x7f3cd40b5630 2021-11-21T21:57:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd159a700 1 -- 172.21.2.116:0/1767879582 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4000e90 con 0x7f3cd40b5630 2021-11-21T21:57:48.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd159a700 1 -- 172.21.2.116:0/1767879582 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc400d8c0 con 0x7f3cd40b5630 2021-11-21T21:57:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/1767879582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd40b5630 msgr2=0x7f3cd414af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 --2- 172.21.2.116:0/1767879582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd40b5630 0x7f3cd414af40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3cd409afa0 tx=0x7f3cc4006eb0).stop 2021-11-21T21:57:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/1767879582 shutdown_connections 2021-11-21T21:57:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 --2- 172.21.2.116:0/1767879582 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4099650 0x7f3cd4099ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 --2- 172.21.2.116:0/1767879582 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd40b5630 0x7f3cd414af40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/1767879582 >> 172.21.2.116:0/1767879582 conn(0x7f3cd405c7f0 msgr2=0x7f3cd40b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/1767879582 shutdown_connections 2021-11-21T21:57:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/1767879582 wait complete. 2021-11-21T21:57:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 Processor -- start 2021-11-21T21:57:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- start start 2021-11-21T21:57:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4097ec0 0x7f3cd4149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd4099650 0x7f3cd4156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3cd40b5630 0x7f3cd4157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4154130 con 0x7f3cd4099650 2021-11-21T21:57:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4063cc0 con 0x7f3cd40b5630 2021-11-21T21:57:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4156420 con 0x7f3cd4097ec0 2021-11-21T21:57:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd4099650 0x7f3cd4156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd4099650 0x7f3cd4156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47638/0 (socket says 172.21.2.116:47638) 2021-11-21T21:57:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 -- 172.21.2.116:0/2684676313 learned_addr learned my addr 172.21.2.116:0/2684676313 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 -- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4097ec0 msgr2=0x7f3cd4149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4097ec0 0x7f3cd4149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 -- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3cd40b5630 msgr2=0x7f3cd4157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3cd40b5630 0x7f3cd4157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 -- 172.21.2.116:0/2684676313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc400b1e0 con 0x7f3cd4099650 2021-11-21T21:57:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd1d9b700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd4099650 0x7f3cd4156b20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f3cc8002f90 tx=0x7f3cc8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cc37fe700 1 -- 172.21.2.116:0/2684676313 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc8009e70 con 0x7f3cd4099650 2021-11-21T21:57:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cc37fe700 1 -- 172.21.2.116:0/2684676313 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8013460 con 0x7f3cd4099650 2021-11-21T21:57:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cc37fe700 1 -- 172.21.2.116:0/2684676313 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc80137c0 con 0x7f3cd4099650 2021-11-21T21:57:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd415a4a0 con 0x7f3cd4099650 2021-11-21T21:57:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd4156680 con 0x7f3cd4099650 2021-11-21T21:57:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd4156840 con 0x7f3cd4099650 2021-11-21T21:57:48.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cc37fe700 1 -- 172.21.2.116:0/2684676313 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3cc800f070 con 0x7f3cd4099650 2021-11-21T21:57:48.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cc37fe700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3cb0053580 0x7f3cb0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.037+0000 7f3cc37fe700 1 -- 172.21.2.116:0/2684676313 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3cc806b1c0 con 0x7f3cd4099650 2021-11-21T21:57:48.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.041+0000 7f3cd259c700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3cb0053580 0x7f3cb0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.041+0000 7f3cd259c700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3cb0053580 0x7f3cb0055a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f3cd409afa0 tx=0x7f3cc4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.041+0000 7f3cc37fe700 1 -- 172.21.2.116:0/2684676313 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3cc8042210 con 0x7f3cd4099650 2021-11-21T21:57:48.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cd40573d0 con 0x7f3cd4099650 2021-11-21T21:57:48.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cc37fe700 1 -- 172.21.2.116:0/2684676313 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f3cc8045830 con 0x7f3cd4099650 2021-11-21T21:57:48.150 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:48.150 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:57:48.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3cb0053580 msgr2=0x7f3cb0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3cb0053580 0x7f3cb0055a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f3cd409afa0 tx=0x7f3cc4015000).stop 2021-11-21T21:57:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd4099650 msgr2=0x7f3cd4156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd4099650 0x7f3cd4156b20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f3cc8002f90 tx=0x7f3cc8006df0).stop 2021-11-21T21:57:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 shutdown_connections 2021-11-21T21:57:48.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3cd40b5630 0x7f3cd4157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3cb0053580 0x7f3cb0055a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3cd4099650 0x7f3cd4156b20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 --2- 172.21.2.116:0/2684676313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3cd4097ec0 0x7f3cd4149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 >> 172.21.2.116:0/2684676313 conn(0x7f3cd405c7f0 msgr2=0x7f3cd4146b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 shutdown_connections 2021-11-21T21:57:48.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.149+0000 7f3cd359e700 1 -- 172.21.2.116:0/2684676313 wait complete. 2021-11-21T21:57:48.154 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:57:48.459 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:57:46.218398+0000', 'modified': '2021-11-21T21:57:47.223289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5586}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5586': {'gid': 5586, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3159083787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3159083787}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3159083787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:57:48.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:57:48.461 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:57:48.526 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5586,"want_state":"up:active","state":"up:active","rank_uptime":2.3018685790000002,"mdsmap_epoch":23,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":6.5346787339999999} 2021-11-21T21:57:48.527 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5586, "mdsmap_epoch": 23, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 2.301868579, "state": "up:active", "uptime": 6.534678734, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:57:48.528 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:57:48.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.577+0000 7f5c99e68700 1 Processor -- start 2021-11-21T21:57:48.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.577+0000 7f5c99e68700 1 -- start start 2021-11-21T21:57:48.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.577+0000 7f5c99e68700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5c940a86b0 0x7f5c9414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.577+0000 7f5c99e68700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c941443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.577+0000 7f5c99e68700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c940a6e80 con 0x7f5c940a6aa0 2021-11-21T21:57:48.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.577+0000 7f5c99e68700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c940a8bc0 con 0x7f5c940a86b0 2021-11-21T21:57:48.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.577+0000 7f5c99e68700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9414a7b0 con 0x7f5c94062410 2021-11-21T21:57:48.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c93fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c941443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c92ffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5c940a86b0 0x7f5c9414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c937fe700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f5c940a6aa0 0x7f5c941470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:58512/0 (socket says 172.21.2.116:58512) 2021-11-21T21:57:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c937fe700 1 -- 172.21.2.116:0/2723137362 learned_addr learned my addr 172.21.2.116:0/2723137362 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c92ffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5c940a86b0 0x7f5c9414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:48816/0 (socket says 172.21.2.116:48816) 2021-11-21T21:57:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c93fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c941443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:50994/0 (socket says 172.21.2.116:50994) 2021-11-21T21:57:48.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c93fff700 1 -- 172.21.2.116:0/2723137362 >> v2:172.21.2.117:3300/0 conn(0x7f5c940a86b0 msgr2=0x7f5c9414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c93fff700 1 --2- 172.21.2.116:0/2723137362 >> v2:172.21.2.117:3300/0 conn(0x7f5c940a86b0 0x7f5c9414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c93fff700 1 -- 172.21.2.116:0/2723137362 >> v1:172.21.2.117:6789/0 conn(0x7f5c940a6aa0 legacy=0x7f5c941470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c93fff700 1 -- 172.21.2.116:0/2723137362 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9414aa10 con 0x7f5c94062410 2021-11-21T21:57:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c92ffd700 1 --2- 172.21.2.116:0/2723137362 >> v2:172.21.2.117:3300/0 conn(0x7f5c940a86b0 0x7f5c9414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:57:48.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c93fff700 1 --2- 172.21.2.116:0/2723137362 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c941443f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5c8c0032f0 tx=0x7f5c8c0091d0).ready entity=mon.2 client_cookie=d4afbf3c69aca7a4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c927fc700 1 -- 172.21.2.116:0/2723137362 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c8c00b6c0 con 0x7f5c94062410 2021-11-21T21:57:48.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c927fc700 1 -- 172.21.2.116:0/2723137362 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00b820 con 0x7f5c94062410 2021-11-21T21:57:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c927fc700 1 -- 172.21.2.116:0/2723137362 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c8c00b990 con 0x7f5c94062410 2021-11-21T21:57:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 -- 172.21.2.116:0/2723137362 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 msgr2=0x7f5c941443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 --2- 172.21.2.116:0/2723137362 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c941443f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5c8c0032f0 tx=0x7f5c8c0091d0).stop 2021-11-21T21:57:48.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 -- 172.21.2.116:0/2723137362 shutdown_connections 2021-11-21T21:57:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 --2- 172.21.2.116:0/2723137362 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c941443f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 --2- 172.21.2.116:0/2723137362 >> v2:172.21.2.117:3300/0 conn(0x7f5c940a86b0 0x7f5c9414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 -- 172.21.2.116:0/2723137362 >> 172.21.2.116:0/2723137362 conn(0x7f5c9405bd20 msgr2=0x7f5c9405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 -- 172.21.2.116:0/2723137362 shutdown_connections 2021-11-21T21:57:48.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.581+0000 7f5c99e68700 1 -- 172.21.2.116:0/2723137362 wait complete. 2021-11-21T21:57:48.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 Processor -- start 2021-11-21T21:57:48.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 -- start start 2021-11-21T21:57:48.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c94143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c940a6aa0 0x7f5c940a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c940a86b0 0x7f5c940a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9415ac40 con 0x7f5c940a6aa0 2021-11-21T21:57:48.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94063d30 con 0x7f5c940a86b0 2021-11-21T21:57:48.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94159d50 con 0x7f5c94062410 2021-11-21T21:57:48.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c940a6aa0 0x7f5c940a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c940a6aa0 0x7f5c940a8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47656/0 (socket says 172.21.2.116:47656) 2021-11-21T21:57:48.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 -- 172.21.2.116:0/1038704112 learned_addr learned my addr 172.21.2.116:0/1038704112 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:48.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 -- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 msgr2=0x7f5c94143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c94143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 -- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c940a86b0 msgr2=0x7f5c940a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c940a86b0 0x7f5c940a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 -- 172.21.2.116:0/1038704112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8400b180 con 0x7f5c940a6aa0 2021-11-21T21:57:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c92ffd700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c940a6aa0 0x7f5c940a8f70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f5c8400c000 tx=0x7f5c84006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c90ff9700 1 -- 172.21.2.116:0/1038704112 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c8400c860 con 0x7f5c940a6aa0 2021-11-21T21:57:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c90ff9700 1 -- 172.21.2.116:0/1038704112 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8400c9c0 con 0x7f5c940a6aa0 2021-11-21T21:57:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c009470 con 0x7f5c940a6aa0 2021-11-21T21:57:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c94063f60 con 0x7f5c940a6aa0 2021-11-21T21:57:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c90ff9700 1 -- 172.21.2.116:0/1038704112 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c8400cd20 con 0x7f5c940a6aa0 2021-11-21T21:57:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c940643d0 con 0x7f5c940a6aa0 2021-11-21T21:57:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c90ff9700 1 -- 172.21.2.116:0/1038704112 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5c8401c460 con 0x7f5c940a6aa0 2021-11-21T21:57:48.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c90ff9700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c78053690 0x7f5c78055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c90ff9700 1 -- 172.21.2.116:0/1038704112 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5c8406c020 con 0x7f5c940a6aa0 2021-11-21T21:57:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.585+0000 7f5c937fe700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c78053690 0x7f5c78055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.589+0000 7f5c937fe700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c78053690 0x7f5c78055b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f5c94061ab0 tx=0x7f5c7c007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.589+0000 7f5c90ff9700 1 -- 172.21.2.116:0/1038704112 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c84043010 con 0x7f5c940a6aa0 2021-11-21T21:57:48.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c940573d0 con 0x7f5c940a6aa0 2021-11-21T21:57:48.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c90ff9700 1 -- 172.21.2.116:0/1038704112 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16040 (secure 0 0 0) 0x7f5c84046630 con 0x7f5c940a6aa0 2021-11-21T21:57:48.672 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:48.673 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":55,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:57:47.221698+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:57:44.702609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-11-21T21:57:45.184798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:57:48.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c78053690 msgr2=0x7f5c78055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c78053690 0x7f5c78055b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f5c94061ab0 tx=0x7f5c7c007820).stop 2021-11-21T21:57:48.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c940a6aa0 msgr2=0x7f5c940a8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c940a6aa0 0x7f5c940a8f70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f5c8400c000 tx=0x7f5c84006df0).stop 2021-11-21T21:57:48.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 shutdown_connections 2021-11-21T21:57:48.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c940a86b0 0x7f5c940a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c78053690 0x7f5c78055b50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c940a6aa0 0x7f5c940a8f70 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 --2- 172.21.2.116:0/1038704112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c94062410 0x7f5c94143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 >> 172.21.2.116:0/1038704112 conn(0x7f5c9405bd20 msgr2=0x7f5c94149060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 shutdown_connections 2021-11-21T21:57:48.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.673+0000 7f5c99e68700 1 -- 172.21.2.116:0/1038704112 wait complete. 2021-11-21T21:57:48.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:57:44.702609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:57:45.184798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:57:48.680 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:57:48.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f353d87f700 1 Processor -- start 2021-11-21T21:57:48.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f353d87f700 1 -- start start 2021-11-21T21:57:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f353d87f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 0x7f3538142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f353d87f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3538062e60 0x7f35380af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f353d87f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35380b0f60 con 0x7f3538069e70 2021-11-21T21:57:48.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f353d87f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3538063470 con 0x7f3538062e60 2021-11-21T21:57:48.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f353d87f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3538143050 con 0x7f35381426e0 2021-11-21T21:57:48.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3538062e60 0x7f35380af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3538062e60 0x7f35380af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47664/0 (socket says 172.21.2.116:47664) 2021-11-21T21:57:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 -- 172.21.2.116:0/331539632 learned_addr learned my addr 172.21.2.116:0/331539632 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 -- 172.21.2.116:0/331539632 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 msgr2=0x7f3538142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:48.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 --2- 172.21.2.116:0/331539632 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 0x7f3538142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 -- 172.21.2.116:0/331539632 >> v1:172.21.2.116:6789/0 conn(0x7f3538069e70 legacy=0x7f35380ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 -- 172.21.2.116:0/331539632 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35380b1e00 con 0x7f3538062e60 2021-11-21T21:57:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f35367fc700 1 --2- 172.21.2.116:0/331539632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538062e60 0x7f35380af3d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f3528002f70 tx=0x7f3528006eb0).ready entity=mon.0 client_cookie=b78fe083af7d3ba3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f3535ffb700 1 -- 172.21.2.116:0/331539632 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f352800d5c0 con 0x7f3538062e60 2021-11-21T21:57:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f3535ffb700 1 -- 172.21.2.116:0/331539632 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3528000e90 con 0x7f3538062e60 2021-11-21T21:57:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.733+0000 7f3535ffb700 1 -- 172.21.2.116:0/331539632 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f352800d950 con 0x7f3538062e60 2021-11-21T21:57:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/331539632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538062e60 msgr2=0x7f35380af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 --2- 172.21.2.116:0/331539632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538062e60 0x7f35380af3d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f3528002f70 tx=0x7f3528006eb0).stop 2021-11-21T21:57:48.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/331539632 shutdown_connections 2021-11-21T21:57:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 --2- 172.21.2.116:0/331539632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538062e60 0x7f35380af3d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 --2- 172.21.2.116:0/331539632 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 0x7f3538142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/331539632 >> 172.21.2.116:0/331539632 conn(0x7f35380b5730 msgr2=0x7f35380b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/331539632 shutdown_connections 2021-11-21T21:57:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/331539632 wait complete. 2021-11-21T21:57:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 Processor -- start 2021-11-21T21:57:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- start start 2021-11-21T21:57:48.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538069e70 0x7f3538141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 0x7f353811c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f353813cb40 0x7f353813f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353813e030 con 0x7f3538069e70 2021-11-21T21:57:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353813fd40 con 0x7f353813cb40 2021-11-21T21:57:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353813e4c0 con 0x7f35381426e0 2021-11-21T21:57:48.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538069e70 0x7f3538141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538069e70 0x7f3538141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47668/0 (socket says 172.21.2.116:47668) 2021-11-21T21:57:48.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 -- 172.21.2.116:0/4180231773 learned_addr learned my addr 172.21.2.116:0/4180231773 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:48.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 -- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 msgr2=0x7f353811c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 0x7f353811c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 -- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f353813cb40 msgr2=0x7f353813f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f353813cb40 0x7f353813f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 -- 172.21.2.116:0/4180231773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352800b1e0 con 0x7f3538069e70 2021-11-21T21:57:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f3536ffd700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538069e70 0x7f3538141df0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f3520002ef0 tx=0x7f3520006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f352ffff700 1 -- 172.21.2.116:0/4180231773 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3520012410 con 0x7f3538069e70 2021-11-21T21:57:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f352ffff700 1 -- 172.21.2.116:0/4180231773 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3520012570 con 0x7f3538069e70 2021-11-21T21:57:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353813e220 con 0x7f3538069e70 2021-11-21T21:57:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f352ffff700 1 -- 172.21.2.116:0/4180231773 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35200128d0 con 0x7f3538069e70 2021-11-21T21:57:48.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353811cf80 con 0x7f3538069e70 2021-11-21T21:57:48.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3538141680 con 0x7f3538069e70 2021-11-21T21:57:48.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f352ffff700 1 -- 172.21.2.116:0/4180231773 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f352000f030 con 0x7f3538069e70 2021-11-21T21:57:48.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f352ffff700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f351c053580 0x7f351c055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.737+0000 7f352ffff700 1 -- 172.21.2.116:0/4180231773 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f352006b6d0 con 0x7f3538069e70 2021-11-21T21:57:48.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.741+0000 7f35367fc700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f351c053580 0x7f351c055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.741+0000 7f35367fc700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f351c053580 0x7f351c055a40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f3528002f70 tx=0x7f3528015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.741+0000 7f352ffff700 1 -- 172.21.2.116:0/4180231773 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3538141680 con 0x7f3538069e70 2021-11-21T21:57:48.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3538141680 con 0x7f3538069e70 2021-11-21T21:57:48.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f352ffff700 1 -- 172.21.2.116:0/4180231773 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3538141680 con 0x7f3538069e70 2021-11-21T21:57:48.825 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:57:48.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f351c053580 msgr2=0x7f351c055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f351c053580 0x7f351c055a40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f3528002f70 tx=0x7f3528015000).stop 2021-11-21T21:57:48.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538069e70 msgr2=0x7f3538141df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538069e70 0x7f3538141df0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f3520002ef0 tx=0x7f3520006d50).stop 2021-11-21T21:57:48.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 shutdown_connections 2021-11-21T21:57:48.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f353813cb40 0x7f353813f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35381426e0 0x7f353811c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f351c053580 0x7f351c055a40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 --2- 172.21.2.116:0/4180231773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3538069e70 0x7f3538141df0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 >> 172.21.2.116:0/4180231773 conn(0x7f35380b5730 msgr2=0x7f35380b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 shutdown_connections 2021-11-21T21:57:48.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.825+0000 7f353d87f700 1 -- 172.21.2.116:0/4180231773 wait complete. 2021-11-21T21:57:48.832 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:57:48.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 Processor -- start 2021-11-21T21:57:48.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- start start 2021-11-21T21:57:48.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc4740adc10 0x7fc4740b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc474144a10 0x7fc474141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4740b3c50 con 0x7fc474141510 2021-11-21T21:57:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4740b3ef0 con 0x7fc474144a10 2021-11-21T21:57:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4740b4190 con 0x7fc4740adc10 2021-11-21T21:57:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc474144a10 0x7fc474141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc474144a10 0x7fc474141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47678/0 (socket says 172.21.2.116:47678) 2021-11-21T21:57:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 -- 172.21.2.116:0/1318783133 learned_addr learned my addr 172.21.2.116:0/1318783133 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 -- 172.21.2.116:0/1318783133 >> v2:172.21.2.117:3300/0 conn(0x7fc4740adc10 msgr2=0x7fc4740b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 --2- 172.21.2.116:0/1318783133 >> v2:172.21.2.117:3300/0 conn(0x7fc4740adc10 0x7fc4740b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 -- 172.21.2.116:0/1318783133 >> v1:172.21.2.117:6789/0 conn(0x7fc474141510 legacy=0x7fc47413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 -- 172.21.2.116:0/1318783133 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474140b80 con 0x7fc474144a10 2021-11-21T21:57:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc478b7a700 1 --2- 172.21.2.116:0/1318783133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc474144a10 0x7fc474141020 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fc47405f430 tx=0x7fc464006eb0).ready entity=mon.0 client_cookie=75b3b0b0f95d6724 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc463fff700 1 -- 172.21.2.116:0/1318783133 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46400d5c0 con 0x7fc474144a10 2021-11-21T21:57:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc463fff700 1 -- 172.21.2.116:0/1318783133 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc464000e90 con 0x7fc474144a10 2021-11-21T21:57:48.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc463fff700 1 -- 172.21.2.116:0/1318783133 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46400d910 con 0x7fc474144a10 2021-11-21T21:57:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- 172.21.2.116:0/1318783133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc474144a10 msgr2=0x7fc474141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1318783133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc474144a10 0x7fc474141020 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fc47405f430 tx=0x7fc464006eb0).stop 2021-11-21T21:57:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- 172.21.2.116:0/1318783133 shutdown_connections 2021-11-21T21:57:48.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1318783133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc474144a10 0x7fc474141020 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1318783133 >> v2:172.21.2.117:3300/0 conn(0x7fc4740adc10 0x7fc4740b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- 172.21.2.116:0/1318783133 >> 172.21.2.116:0/1318783133 conn(0x7fc47405bd20 msgr2=0x7fc47405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- 172.21.2.116:0/1318783133 shutdown_connections 2021-11-21T21:57:48.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- 172.21.2.116:0/1318783133 wait complete. 2021-11-21T21:57:48.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 Processor -- start 2021-11-21T21:57:48.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- start start 2021-11-21T21:57:48.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc4740adc10 0x7fc4741108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc474141510 0x7fc474150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc474144a10 0x7fc47413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4740b1b90 con 0x7fc4740adc10 2021-11-21T21:57:48.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47414fb00 con 0x7fc474141510 2021-11-21T21:57:48.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47b5df700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc474113810 con 0x7fc474144a10 2021-11-21T21:57:48.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47937b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc4740adc10 0x7fc4741108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47937b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc4740adc10 0x7fc4741108a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47682/0 (socket says 172.21.2.116:47682) 2021-11-21T21:57:48.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.889+0000 7fc47937b700 1 -- 172.21.2.116:0/1775116649 learned_addr learned my addr 172.21.2.116:0/1775116649 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:48.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47937b700 1 -- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc474144a10 msgr2=0x7fc47413ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47937b700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc474144a10 0x7fc47413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47937b700 1 -- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc474141510 msgr2=0x7fc474150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47937b700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc474141510 0x7fc474150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47937b700 1 -- 172.21.2.116:0/1775116649 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46400b1e0 con 0x7fc4740adc10 2021-11-21T21:57:48.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47937b700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc4740adc10 0x7fc4741108a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fc470002f70 tx=0x7fc470006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc4627fc700 1 -- 172.21.2.116:0/1775116649 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc470013460 con 0x7fc4740adc10 2021-11-21T21:57:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc4627fc700 1 -- 172.21.2.116:0/1775116649 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4700135c0 con 0x7fc4740adc10 2021-11-21T21:57:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4740b18f0 con 0x7fc4740adc10 2021-11-21T21:57:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47414ff90 con 0x7fc4740adc10 2021-11-21T21:57:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc4627fc700 1 -- 172.21.2.116:0/1775116649 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc470013920 con 0x7fc4740adc10 2021-11-21T21:57:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4740573d0 con 0x7fc4740adc10 2021-11-21T21:57:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc4627fc700 1 -- 172.21.2.116:0/1775116649 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc47001a060 con 0x7fc4740adc10 2021-11-21T21:57:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc4627fc700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc454053540 0x7fc454055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc4627fc700 1 -- 172.21.2.116:0/1775116649 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc47006b4e0 con 0x7fc4740adc10 2021-11-21T21:57:48.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc478b7a700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc454053540 0x7fc454055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:48.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc478b7a700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc454053540 0x7fc454055a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fc47405f430 tx=0x7fc464015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:48.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.893+0000 7fc4627fc700 1 -- 172.21.2.116:0/1775116649 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc470042530 con 0x7fc4740adc10 2021-11-21T21:57:48.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc47405db30 con 0x7fc4740adc10 2021-11-21T21:57:48.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc4627fc700 1 -- 172.21.2.116:0/1775116649 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc470045b50 con 0x7fc4740adc10 2021-11-21T21:57:48.980 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:57:48.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc454053540 msgr2=0x7fc454055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc454053540 0x7fc454055a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fc47405f430 tx=0x7fc464015000).stop 2021-11-21T21:57:48.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc4740adc10 msgr2=0x7fc4741108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:48.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc4740adc10 0x7fc4741108a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fc470002f70 tx=0x7fc470006ec0).stop 2021-11-21T21:57:48.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 shutdown_connections 2021-11-21T21:57:48.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc474144a10 0x7fc47413ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc474141510 0x7fc474150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:48.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc454053540 0x7fc454055a00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 --2- 172.21.2.116:0/1775116649 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc4740adc10 0x7fc4741108a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:48.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 >> 172.21.2.116:0/1775116649 conn(0x7fc47405bd20 msgr2=0x7fc47413e710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:48.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 shutdown_connections 2021-11-21T21:57:48.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:48.977+0000 7fc47b5df700 1 -- 172.21.2.116:0/1775116649 wait complete. 2021-11-21T21:57:48.987 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:57:49.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 Processor -- start 2021-11-21T21:57:49.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- start start 2021-11-21T21:57:49.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 0x7f30001478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f30000abce0 0x7f30000b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f30000b1db0 0x7f30000ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3000147e80 con 0x7f30000b1db0 2021-11-21T21:57:49.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30000af5a0 con 0x7f30000abce0 2021-11-21T21:57:49.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30000b1490 con 0x7f3000144810 2021-11-21T21:57:49.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f30000b1db0 0x7f30000ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f30000b1db0 0x7f30000ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47692/0 (socket says 172.21.2.116:47692) 2021-11-21T21:57:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 -- 172.21.2.116:0/226141439 learned_addr learned my addr 172.21.2.116:0/226141439 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:49.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 -- 172.21.2.116:0/226141439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 msgr2=0x7f30001478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 --2- 172.21.2.116:0/226141439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 0x7f30001478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 -- 172.21.2.116:0/226141439 >> v2:172.21.2.117:3300/0 conn(0x7f30000abce0 msgr2=0x7f30000b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 --2- 172.21.2.116:0/226141439 >> v2:172.21.2.117:3300/0 conn(0x7f30000abce0 0x7f30000b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 -- 172.21.2.116:0/226141439 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30000af060 con 0x7f30000b1db0 2021-11-21T21:57:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2fff7fe700 1 --2- 172.21.2.116:0/226141439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000b1db0 0x7f30000ac600 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2ff80032f0 tx=0x7f2ff8009290).ready entity=mon.0 client_cookie=2016bd1de959673d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2ffdffb700 1 -- 172.21.2.116:0/226141439 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ff800b870 con 0x7f30000b1db0 2021-11-21T21:57:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2ffdffb700 1 -- 172.21.2.116:0/226141439 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff800b9d0 con 0x7f30000b1db0 2021-11-21T21:57:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f2ffdffb700 1 -- 172.21.2.116:0/226141439 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ff800bb80 con 0x7f30000b1db0 2021-11-21T21:57:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- 172.21.2.116:0/226141439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000b1db0 msgr2=0x7f30000ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 --2- 172.21.2.116:0/226141439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000b1db0 0x7f30000ac600 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2ff80032f0 tx=0x7f2ff8009290).stop 2021-11-21T21:57:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- 172.21.2.116:0/226141439 shutdown_connections 2021-11-21T21:57:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 --2- 172.21.2.116:0/226141439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000b1db0 0x7f30000ac600 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 --2- 172.21.2.116:0/226141439 >> v2:172.21.2.117:3300/0 conn(0x7f30000abce0 0x7f30000b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 --2- 172.21.2.116:0/226141439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 0x7f30001478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- 172.21.2.116:0/226141439 >> 172.21.2.116:0/226141439 conn(0x7f300005bd20 msgr2=0x7f300005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:49.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- 172.21.2.116:0/226141439 shutdown_connections 2021-11-21T21:57:49.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- 172.21.2.116:0/226141439 wait complete. 2021-11-21T21:57:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 Processor -- start 2021-11-21T21:57:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.037+0000 7f3005a1a700 1 -- start start 2021-11-21T21:57:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000abce0 0x7f3000143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 0x7f300013cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f300013d420 0x7f30001463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30001474a0 con 0x7f30000abce0 2021-11-21T21:57:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3000145350 con 0x7f300013d420 2021-11-21T21:57:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3000144d20 con 0x7f3000144810 2021-11-21T21:57:49.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000abce0 0x7f3000143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000abce0 0x7f3000143900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47696/0 (socket says 172.21.2.116:47696) 2021-11-21T21:57:49.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 -- 172.21.2.116:0/2408699554 learned_addr learned my addr 172.21.2.116:0/2408699554 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 -- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 msgr2=0x7f300013cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffe7fc700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 0x7f300013cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2fff7fe700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f300013d420 0x7f30001463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 0x7f300013cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 -- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f300013d420 msgr2=0x7f30001463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f300013d420 0x7f30001463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 -- 172.21.2.116:0/2408699554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8009530 con 0x7f30000abce0 2021-11-21T21:57:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffeffd700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000abce0 0x7f3000143900 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f30000632f0 tx=0x7f2fe8006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ff7fff700 1 -- 172.21.2.116:0/2408699554 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fe8012410 con 0x7f30000abce0 2021-11-21T21:57:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ff7fff700 1 -- 172.21.2.116:0/2408699554 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe8012570 con 0x7f30000abce0 2021-11-21T21:57:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300013d8b0 con 0x7f30000abce0 2021-11-21T21:57:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3000144f50 con 0x7f30000abce0 2021-11-21T21:57:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ff7fff700 1 -- 172.21.2.116:0/2408699554 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fe80128d0 con 0x7f30000abce0 2021-11-21T21:57:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3000145110 con 0x7f30000abce0 2021-11-21T21:57:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ff7fff700 1 -- 172.21.2.116:0/2408699554 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2fe800f030 con 0x7f30000abce0 2021-11-21T21:57:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ff7fff700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fe40536a0 0x7f2fe4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ff7fff700 1 -- 172.21.2.116:0/2408699554 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2fe806b570 con 0x7f30000abce0 2021-11-21T21:57:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.041+0000 7f2ffe7fc700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fe40536a0 0x7f2fe4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.049+0000 7f2ffe7fc700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fe40536a0 0x7f2fe4055b60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f2ff0007000 tx=0x7f2ff0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.049+0000 7f2ff7fff700 1 -- 172.21.2.116:0/2408699554 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fe80426d0 con 0x7f30000abce0 2021-11-21T21:57:49.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.125+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30000573d0 con 0x7f30000abce0 2021-11-21T21:57:49.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.125+0000 7f2ff7fff700 1 -- 172.21.2.116:0/2408699554 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2fe8045cf0 con 0x7f30000abce0 2021-11-21T21:57:49.131 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:57:49.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.125+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fe40536a0 msgr2=0x7f2fe4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.125+0000 7f3005a1a700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fe40536a0 0x7f2fe4055b60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f2ff0007000 tx=0x7f2ff0008040).stop 2021-11-21T21:57:49.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.125+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000abce0 msgr2=0x7f3000143900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.125+0000 7f3005a1a700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000abce0 0x7f3000143900 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f30000632f0 tx=0x7f2fe8006d70).stop 2021-11-21T21:57:49.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 shutdown_connections 2021-11-21T21:57:49.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f300013d420 0x7f30001463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3000144810 0x7f300013cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fe40536a0 0x7f2fe4055b60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 --2- 172.21.2.116:0/2408699554 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30000abce0 0x7f3000143900 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 >> 172.21.2.116:0/2408699554 conn(0x7f300005bd20 msgr2=0x7f30000af850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:49.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 shutdown_connections 2021-11-21T21:57:49.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.129+0000 7f3005a1a700 1 -- 172.21.2.116:0/2408699554 wait complete. 2021-11-21T21:57:49.138 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:57:49.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 Processor -- start 2021-11-21T21:57:49.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- start start 2021-11-21T21:57:49.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f84540abce0 0x7f84541478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84540ac600 0x7f84540aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8454147e80 con 0x7f8454144810 2021-11-21T21:57:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84540b1490 con 0x7f84540abce0 2021-11-21T21:57:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540b1fd0 con 0x7f84540ac600 2021-11-21T21:57:49.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8453fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8454144810 0x7f84540b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59658/0 (socket says 172.21.2.116:59658) 2021-11-21T21:57:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8453fff700 1 -- 172.21.2.116:0/3058527669 learned_addr learned my addr 172.21.2.116:0/3058527669 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:57:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069522287 0 0) 0x7f8454147e80 con 0x7f8454144810 2021-11-21T21:57:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8434003260 con 0x7f8454144810 2021-11-21T21:57:49.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f84537fe700 1 --2- 172.21.2.116:0/3058527669 >> v2:172.21.2.117:3300/0 conn(0x7f84540abce0 0x7f84541478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84540ac600 0x7f84540aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1185434573 0 0) 0x7f8434003260 con 0x7f8454144810 2021-11-21T21:57:49.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84540ac600 msgr2=0x7f84540aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:49.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84540ac600 0x7f84540aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 >> v2:172.21.2.117:3300/0 conn(0x7f84540abce0 msgr2=0x7f84541478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:49.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 --2- 172.21.2.116:0/3058527669 >> v2:172.21.2.117:3300/0 conn(0x7f84540abce0 0x7f84541478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540af5a0 con 0x7f8454144810 2021-11-21T21:57:49.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f843c001660 con 0x7f8454144810 2021-11-21T21:57:49.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 >> v1:172.21.2.116:6789/0 conn(0x7f8454144810 legacy=0x7f84540b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434003750 0x7f8434003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84540ac600 0x7f8434004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8434004a40 0x7f8434004ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540b1490 con 0x7f8434004a40 2021-11-21T21:57:49.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540b1fd0 con 0x7f8434003750 2021-11-21T21:57:49.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843400b910 con 0x7f84540ac600 2021-11-21T21:57:49.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f84537fe700 1 --2- 172.21.2.116:0/3058527669 >> v2:172.21.2.117:3300/0 conn(0x7f84540abce0 0x7f84541478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:57:49.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8434004a40 0x7f8434004ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8453fff700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434003750 0x7f8434003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 -- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84540ac600 msgr2=0x7f8434004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:49.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84540ac600 0x7f8434004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 -- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434003750 msgr2=0x7f8434003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:57:49.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434003750 0x7f8434003b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 -- 172.21.2.116:0/3058527669 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540af5a0 con 0x7f8434004a40 2021-11-21T21:57:49.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8434004a40 0x7f8434004ec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f844c007000 tx=0x7f844c008040).ready entity=mon.0 client_cookie=946e60f3638704af server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f844c00c640 con 0x7f8434004a40 2021-11-21T21:57:49.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c00c7a0 con 0x7f8434004a40 2021-11-21T21:57:49.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8452ffd700 1 -- 172.21.2.116:0/3058527669 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f844c00cb00 con 0x7f8434004a40 2021-11-21T21:57:49.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- 172.21.2.116:0/3058527669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8434004a40 msgr2=0x7f8434004ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 --2- 172.21.2.116:0/3058527669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8434004a40 0x7f8434004ec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f844c007000 tx=0x7f844c008040).stop 2021-11-21T21:57:49.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 -- 172.21.2.116:0/3058527669 reap_dead start 2021-11-21T21:57:49.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- 172.21.2.116:0/3058527669 shutdown_connections 2021-11-21T21:57:49.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- 172.21.2.116:0/3058527669 >> 172.21.2.116:0/3058527669 conn(0x7f845405bd20 msgr2=0x7f845405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:49.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- 172.21.2.116:0/3058527669 shutdown_connections 2021-11-21T21:57:49.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- 172.21.2.116:0/3058527669 wait complete. 2021-11-21T21:57:49.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 Processor -- start 2021-11-21T21:57:49.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- start start 2021-11-21T21:57:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434004a40 0x7f8454142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84541563a0 0x7f84541431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f845415a230 0x7f84540b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454143bb0 con 0x7f845415a230 2021-11-21T21:57:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454143da0 con 0x7f8434004a40 2021-11-21T21:57:49.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f845a27a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8454159af0 con 0x7f84541563a0 2021-11-21T21:57:49.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f845415a230 0x7f84540b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f845415a230 0x7f84540b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47718/0 (socket says 172.21.2.116:47718) 2021-11-21T21:57:49.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.189+0000 7f8458817700 1 -- 172.21.2.116:0/999675963 learned_addr learned my addr 172.21.2.116:0/999675963 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:49.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8458817700 1 -- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84541563a0 msgr2=0x7f84541431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8458817700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84541563a0 0x7f84541431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8458817700 1 -- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434004a40 msgr2=0x7f8454142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8458817700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434004a40 0x7f8454142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8458817700 1 -- 172.21.2.116:0/999675963 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454159d90 con 0x7f845415a230 2021-11-21T21:57:49.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8458817700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f845415a230 0x7f84540b2b60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f84540b4dc0 tx=0x7f844c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f84517fa700 1 -- 172.21.2.116:0/999675963 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f844c00c640 con 0x7f845415a230 2021-11-21T21:57:49.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f84517fa700 1 -- 172.21.2.116:0/999675963 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c020070 con 0x7f845415a230 2021-11-21T21:57:49.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84541436a0 con 0x7f845415a230 2021-11-21T21:57:49.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845413f910 con 0x7f845415a230 2021-11-21T21:57:49.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f84517fa700 1 -- 172.21.2.116:0/999675963 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f844c00a4a0 con 0x7f845415a230 2021-11-21T21:57:49.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c003410 con 0x7f845415a230 2021-11-21T21:57:49.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f84517fa700 1 -- 172.21.2.116:0/999675963 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f844c02c070 con 0x7f845415a230 2021-11-21T21:57:49.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f84517fa700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84380536a0 0x7f8438055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f84517fa700 1 -- 172.21.2.116:0/999675963 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f844c06cff0 con 0x7f845415a230 2021-11-21T21:57:49.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8453fff700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84380536a0 0x7f8438055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f8453fff700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84380536a0 0x7f8438055b60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f843c008000 tx=0x7f843c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.193+0000 7f84517fa700 1 -- 172.21.2.116:0/999675963 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f844c044150 con 0x7f845415a230 2021-11-21T21:57:49.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.277+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f842c003c40 con 0x7f845415a230 2021-11-21T21:57:49.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.277+0000 7f84517fa700 1 -- 172.21.2.116:0/999675963 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7f844c047770 con 0x7f845415a230 2021-11-21T21:57:49.285 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:57:49.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84380536a0 msgr2=0x7f8438055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84380536a0 0x7f8438055b60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f843c008000 tx=0x7f843c009040).stop 2021-11-21T21:57:49.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f845415a230 msgr2=0x7f84540b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f845415a230 0x7f84540b2b60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f84540b4dc0 tx=0x7f844c008040).stop 2021-11-21T21:57:49.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 shutdown_connections 2021-11-21T21:57:49.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84380536a0 0x7f8438055b60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f845415a230 0x7f84540b2b60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84541563a0 0x7f84541431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 --2- 172.21.2.116:0/999675963 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8434004a40 0x7f8454142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 >> 172.21.2.116:0/999675963 conn(0x7f845405bd20 msgr2=0x7f84540b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:49.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 shutdown_connections 2021-11-21T21:57:49.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.281+0000 7f845a27a700 1 -- 172.21.2.116:0/999675963 wait complete. 2021-11-21T21:57:49.291 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:57:49.291 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:57:49.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 Processor -- start 2021-11-21T21:57:49.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- start start 2021-11-21T21:57:49.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff72c0ad780 0x7ff72c05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff72c0af7e0 0x7ff72c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c143f70 con 0x7ff72c0affc0 2021-11-21T21:57:49.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c0adb60 con 0x7ff72c0ad780 2021-11-21T21:57:49.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c143160 con 0x7ff72c0af7e0 2021-11-21T21:57:49.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff72c0ad780 0x7ff72c05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff72c0ad780 0x7ff72c05f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47724/0 (socket says 172.21.2.116:47724) 2021-11-21T21:57:49.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/494149690 learned_addr learned my addr 172.21.2.116:0/494149690 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:49.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/494149690 >> v2:172.21.2.117:3300/0 conn(0x7ff72c0af7e0 msgr2=0x7ff72c13fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:57:49.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- 172.21.2.116:0/494149690 >> v2:172.21.2.117:3300/0 conn(0x7ff72c0af7e0 0x7ff72c13fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/494149690 >> v1:172.21.2.116:6789/0 conn(0x7ff72c0affc0 legacy=0x7ff72c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/494149690 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0b0ce0 con 0x7ff72c0ad780 2021-11-21T21:57:49.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- 172.21.2.116:0/494149690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c05f730 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7ff71c002f70 tx=0x7ff71c006f40).ready entity=mon.0 client_cookie=ffbf0b8db026d984 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72a7fc700 1 -- 172.21.2.116:0/494149690 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71c00dc70 con 0x7ff72c0ad780 2021-11-21T21:57:49.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72a7fc700 1 -- 172.21.2.116:0/494149690 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c000e90 con 0x7ff72c0ad780 2021-11-21T21:57:49.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72a7fc700 1 -- 172.21.2.116:0/494149690 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71c00be90 con 0x7ff72c0ad780 2021-11-21T21:57:49.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- 172.21.2.116:0/494149690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 msgr2=0x7ff72c05f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- 172.21.2.116:0/494149690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c05f730 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7ff71c002f70 tx=0x7ff71c006f40).stop 2021-11-21T21:57:49.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- 172.21.2.116:0/494149690 shutdown_connections 2021-11-21T21:57:49.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- 172.21.2.116:0/494149690 >> v2:172.21.2.117:3300/0 conn(0x7ff72c0af7e0 0x7ff72c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- 172.21.2.116:0/494149690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c05f730 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- 172.21.2.116:0/494149690 >> 172.21.2.116:0/494149690 conn(0x7ff72c0b6000 msgr2=0x7ff72c0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:49.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- 172.21.2.116:0/494149690 shutdown_connections 2021-11-21T21:57:49.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- 172.21.2.116:0/494149690 wait complete. 2021-11-21T21:57:49.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 Processor -- start 2021-11-21T21:57:49.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- start start 2021-11-21T21:57:49.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c13f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff72c0af7e0 0x7ff72c09e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff72c0affc0 0x7ff72c09ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c05bfe0 con 0x7ff72c0ad780 2021-11-21T21:57:49.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c14e8e0 con 0x7ff72c0affc0 2021-11-21T21:57:49.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c066b70 con 0x7ff72c0af7e0 2021-11-21T21:57:49.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c13f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c13f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47730/0 (socket says 172.21.2.116:47730) 2021-11-21T21:57:49.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/27437116 learned_addr learned my addr 172.21.2.116:0/27437116 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:57:49.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff72c0af7e0 msgr2=0x7ff72c09e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff72c0af7e0 0x7ff72c09e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff72c0affc0 msgr2=0x7ff72c09ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff72c0affc0 0x7ff72c09ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 -- 172.21.2.116:0/27437116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c00b460 con 0x7ff72c0ad780 2021-11-21T21:57:49.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff72b7fe700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c13f780 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7ff71c002f70 tx=0x7ff71c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff728ff9700 1 -- 172.21.2.116:0/27437116 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71c010e00 con 0x7ff72c0ad780 2021-11-21T21:57:49.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff728ff9700 1 -- 172.21.2.116:0/27437116 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c024450 con 0x7ff72c0ad780 2021-11-21T21:57:49.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c14eb50 con 0x7ff72c0ad780 2021-11-21T21:57:49.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c065170 con 0x7ff72c0ad780 2021-11-21T21:57:49.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.341+0000 7ff728ff9700 1 -- 172.21.2.116:0/27437116 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71c0247b0 con 0x7ff72c0ad780 2021-11-21T21:57:49.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.345+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c0ab130 con 0x7ff72c0ad780 2021-11-21T21:57:49.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.345+0000 7ff728ff9700 1 -- 172.21.2.116:0/27437116 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff71c021070 con 0x7ff72c0ad780 2021-11-21T21:57:49.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.345+0000 7ff728ff9700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff70c053540 0x7ff70c055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:57:49.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.345+0000 7ff728ff9700 1 -- 172.21.2.116:0/27437116 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff71c06f6c0 con 0x7ff72c0ad780 2021-11-21T21:57:49.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.345+0000 7ff72affd700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff70c053540 0x7ff70c055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:57:49.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.345+0000 7ff72affd700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff70c053540 0x7ff70c055a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7ff72c068820 tx=0x7ff7200077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:57:49.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.345+0000 7ff728ff9700 1 -- 172.21.2.116:0/27437116 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff71c046710 con 0x7ff72c0ad780 2021-11-21T21:57:49.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.453+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff72c0b0a40 con 0x7ff72c0ad780 2021-11-21T21:57:49.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.453+0000 7ff728ff9700 1 -- 172.21.2.116:0/27437116 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ff71c049d30 con 0x7ff72c0ad780 2021-11-21T21:57:49.459 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:57:49.459 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:57:49.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.453+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff70c053540 msgr2=0x7ff70c055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.453+0000 7ff732008700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff70c053540 0x7ff70c055a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7ff72c068820 tx=0x7ff7200077f0).stop 2021-11-21T21:57:49.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.453+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 msgr2=0x7ff72c13f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:57:49.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.453+0000 7ff732008700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c13f780 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7ff71c002f70 tx=0x7ff71c006f40).stop 2021-11-21T21:57:49.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 shutdown_connections 2021-11-21T21:57:49.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff72c0affc0 0x7ff72c09ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff72c0af7e0 0x7ff72c09e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:57:49.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff70c053540 0x7ff70c055a00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 --2- 172.21.2.116:0/27437116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff72c0ad780 0x7ff72c13f780 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:57:49.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 >> 172.21.2.116:0/27437116 conn(0x7ff72c0b6000 msgr2=0x7ff72c0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:57:49.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 shutdown_connections 2021-11-21T21:57:49.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:57:49.457+0000 7ff732008700 1 -- 172.21.2.116:0/27437116 wait complete. 2021-11-21T21:57:49.463 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:57:49.768 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:57:46.218398+0000', 'modified': '2021-11-21T21:57:47.223289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5586}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5586': {'gid': 5586, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3159083787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3159083787}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3159083787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:57:49.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:57:49.770 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:57:49.850 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5586,"want_state":"up:active","state":"up:active","rank_uptime":3.626100782,"mdsmap_epoch":23,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":7.8588956339999996} 2021-11-21T21:57:49.851 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5586, "mdsmap_epoch": 23, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 3.626100782, "state": "up:active", "uptime": 7.858895634, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:57:49.852 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:57:49.852 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T21:57:49.852 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T21:57:49.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:49.853 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:49.865 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:57:49.878 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:57:49.879 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:57:49.886 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:57:49.891 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:57:49.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:49.892 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:57:49.936 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:57:49.937 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:49.940 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:57:49.948 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:57:49.949 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:57:49.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T21:57:49.976+0000 7f74482e5080 -1 init, newargv = 0x5643dd0dfbf0 newargc=9 2021-11-21T21:57:49.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14335]: starting ceph client 2021-11-21T21:57:49.983 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:57:49.984 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:57:49.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:49.985 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:57:49.987 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14335]: starting fuse 2021-11-21T21:57:49.991 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T21:57:49.991 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:57:49.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:49.993 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T21:57:49.993 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T21:57:49.994 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T21:57:49.994 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:57:49.995 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T21:57:49.995 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:57:49.996 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T21:57:49.996 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T21:57:49.997 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T21:57:49.998 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:57:49.998 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T21:57:49.999 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T21:57:49.999 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T21:57:50.000 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T21:57:50.001 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:57:50.002 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T21:57:50.002 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:57:50.003 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:57:50.004 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T21:57:50.005 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:57:50.005 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:57:50.006 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T21:57:50.007 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:57:50.063 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14335.asok 2021-11-21T21:57:50.065 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14335.asok 2021-11-21T21:57:50.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:50.067 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14335.asok status 2021-11-21T21:57:50.119 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T21:57:50.119 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T21:57:50.119 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:57:50.120 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:57:50.120 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T21:57:50.120 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T21:57:50.120 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T21:57:50.121 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14335", 2021-11-21T21:57:50.121 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T21:57:50.121 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:57:50.121 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T21:57:50.121 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T21:57:50.122 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5763, 2021-11-21T21:57:50.122 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T21:57:50.122 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T21:57:50.122 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T21:57:50.123 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5763 2021-11-21T21:57:50.123 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:57:50.123 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:57:50.123 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:57:50.124 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:57:50.124 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 705809851 2021-11-21T21:57:50.124 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T21:57:50.124 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:57:50.124 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:57:50.125 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:57:50.125 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:57:50.125 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 705809851 2021-11-21T21:57:50.125 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:57:50.126 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5763 172.21.2.119:0/705809851", 2021-11-21T21:57:50.126 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/705809851", 2021-11-21T21:57:50.126 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T21:57:50.126 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 23, 2021-11-21T21:57:50.127 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 55, 2021-11-21T21:57:50.127 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:57:50.127 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T21:57:50.127 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T21:57:50.127 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T21:57:50.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:50.129 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:57:50.130 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:50.139 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:57:50.141 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:57:50.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:50.142 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:50.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:57:50.210 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:57:50.210 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:57:50.221 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:57:50.223 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:57:50.224 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T21:57:50.224 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-11-21T21:57:50.225 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-11-21T21:57:50.225 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:57:50.226 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:57:50.226 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-11-21T21:57:50.226 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-11-21T21:57:50.227 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:57:50.227 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-11-21T21:57:50.228 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-11-21T21:57:50.228 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:57:50.228 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-11-21T21:57:50.230 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-11-21T21:57:50.231 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-11-21T21:57:50.231 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-11-21T21:57:50.232 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-11-21T21:57:50.232 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-11-21T21:57:50.232 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-11-21T21:57:50.233 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:58:03.982 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:58:03.986 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:58:03.988 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:58:03.993 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:58:04.946 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:58:05.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f29000b7700 1 Processor -- start 2021-11-21T21:58:05.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f29000b7700 1 -- start start 2021-11-21T21:58:05.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f29000b7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f28f80a86b0 0x7f28f814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f29000b7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80a6e80 con 0x7f28f80a6aa0 2021-11-21T21:58:05.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f29000b7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f814a7b0 con 0x7f28f8062410 2021-11-21T21:58:05.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f29000b7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80a8bc0 con 0x7f28f80a86b0 2021-11-21T21:58:05.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fd652700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f28f80a86b0 0x7f28f814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fde53700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f28f80a6aa0 0x7f28f81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59692/0 (socket says 172.21.2.116:59692) 2021-11-21T21:58:05.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fde53700 1 -- 172.21.2.116:0/2166281937 learned_addr learned my addr 172.21.2.116:0/2166281937 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:58:05.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fd652700 1 -- 172.21.2.116:0/2166281937 >> v1:172.21.2.117:6789/0 conn(0x7f28f8062410 legacy=0x7f28f81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fd652700 1 -- 172.21.2.116:0/2166281937 >> v1:172.21.2.116:6789/0 conn(0x7f28f80a6aa0 legacy=0x7f28f81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fd652700 1 -- 172.21.2.116:0/2166281937 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f814aa10 con 0x7f28f80a86b0 2021-11-21T21:58:05.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fd652700 1 --2- 172.21.2.116:0/2166281937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f80a86b0 0x7f28f814b150 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f28f8061b50 tx=0x7f28f4006df0).ready entity=mon.0 client_cookie=eec0545d6ec80290 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fce51700 1 -- 172.21.2.116:0/2166281937 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28f400d5c0 con 0x7f28f80a86b0 2021-11-21T21:58:05.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fce51700 1 -- 172.21.2.116:0/2166281937 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4000e90 con 0x7f28f80a86b0 2021-11-21T21:58:05.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.001+0000 7f28fce51700 1 -- 172.21.2.116:0/2166281937 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28f400d910 con 0x7f28f80a86b0 2021-11-21T21:58:05.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/2166281937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f80a86b0 msgr2=0x7f28f814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 --2- 172.21.2.116:0/2166281937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f80a86b0 0x7f28f814b150 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f28f8061b50 tx=0x7f28f4006df0).stop 2021-11-21T21:58:05.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/2166281937 shutdown_connections 2021-11-21T21:58:05.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 --2- 172.21.2.116:0/2166281937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f80a86b0 0x7f28f814b150 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/2166281937 >> 172.21.2.116:0/2166281937 conn(0x7f28f805bd20 msgr2=0x7f28f805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/2166281937 shutdown_connections 2021-11-21T21:58:05.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/2166281937 wait complete. 2021-11-21T21:58:05.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 Processor -- start 2021-11-21T21:58:05.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- start start 2021-11-21T21:58:05.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f8062410 0x7f28f8148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28f80a6aa0 0x7f28f815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28f80a86b0 0x7f28f80649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f81599b0 con 0x7f28f8062410 2021-11-21T21:58:05.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8067130 con 0x7f28f80a6aa0 2021-11-21T21:58:05.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8159be0 con 0x7f28f80a86b0 2021-11-21T21:58:05.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f8062410 0x7f28f8148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f8062410 0x7f28f8148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47746/0 (socket says 172.21.2.116:47746) 2021-11-21T21:58:05.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 -- 172.21.2.116:0/845445339 learned_addr learned my addr 172.21.2.116:0/845445339 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 -- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28f80a86b0 msgr2=0x7f28f80649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28f80a86b0 0x7f28f80649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 -- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28f80a6aa0 msgr2=0x7f28f815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28f80a6aa0 0x7f28f815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 -- 172.21.2.116:0/845445339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f400b180 con 0x7f28f8062410 2021-11-21T21:58:05.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fde53700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f8062410 0x7f28f8148e00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f28f8061ab0 tx=0x7f28ec007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28eaffd700 1 -- 172.21.2.116:0/845445339 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28ec00ce30 con 0x7f28f8062410 2021-11-21T21:58:05.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28eaffd700 1 -- 172.21.2.116:0/845445339 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec0072e0 con 0x7f28f8062410 2021-11-21T21:58:05.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f81492f0 con 0x7f28f8062410 2021-11-21T21:58:05.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f80673d0 con 0x7f28f8062410 2021-11-21T21:58:05.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28eaffd700 1 -- 172.21.2.116:0/845445339 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28ec0057c0 con 0x7f28f8062410 2021-11-21T21:58:05.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f8159da0 con 0x7f28f8062410 2021-11-21T21:58:05.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28eaffd700 1 -- 172.21.2.116:0/845445339 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f28ec010070 con 0x7f28f8062410 2021-11-21T21:58:05.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28eaffd700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28d80535d0 0x7f28d8055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28eaffd700 1 -- 172.21.2.116:0/845445339 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f28ec06b150 con 0x7f28f8062410 2021-11-21T21:58:05.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.005+0000 7f28fd652700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28d80535d0 0x7f28d8055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.009+0000 7f28fd652700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28d80535d0 0x7f28d8055a90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f28f8061b50 tx=0x7f28f4012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.009+0000 7f28eaffd700 1 -- 172.21.2.116:0/845445339 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28ec0421a0 con 0x7f28f8062410 2021-11-21T21:58:05.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.117+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28f80573d0 con 0x7f28f8062410 2021-11-21T21:58:05.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.117+0000 7f28eaffd700 1 -- 172.21.2.116:0/845445339 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f28ec0457c0 con 0x7f28f8062410 2021-11-21T21:58:05.123 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:58:05.123 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:58:05.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28d80535d0 msgr2=0x7f28d8055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28d80535d0 0x7f28d8055a90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f28f8061b50 tx=0x7f28f4012000).stop 2021-11-21T21:58:05.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f8062410 msgr2=0x7f28f8148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f8062410 0x7f28f8148e00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f28f8061ab0 tx=0x7f28ec007c30).stop 2021-11-21T21:58:05.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 shutdown_connections 2021-11-21T21:58:05.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28f80a86b0 0x7f28f80649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28f80a6aa0 0x7f28f815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28d80535d0 0x7f28d8055a90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 --2- 172.21.2.116:0/845445339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28f8062410 0x7f28f8148e00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 >> 172.21.2.116:0/845445339 conn(0x7f28f805bd20 msgr2=0x7f28f8142690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 shutdown_connections 2021-11-21T21:58:05.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.121+0000 7f29000b7700 1 -- 172.21.2.116:0/845445339 wait complete. 2021-11-21T21:58:05.128 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:58:05.130 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2021-11-21T21:58:05.186 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-11-21T21:58:05.187 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-11-21T21:58:05.187 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:58:05.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.233+0000 7f22e2625700 1 Processor -- start 2021-11-21T21:58:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.233+0000 7f22e2625700 1 -- start start 2021-11-21T21:58:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.233+0000 7f22e2625700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f22dc0a86b0 0x7f22dc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.233+0000 7f22e2625700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc062410 0x7f22dc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.233+0000 7f22e2625700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc0a6e80 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.233+0000 7f22e2625700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc0a8bc0 con 0x7f22dc0a86b0 2021-11-21T21:58:05.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.233+0000 7f22e2625700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc14a7b0 con 0x7f22dc062410 2021-11-21T21:58:05.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e0bc2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc062410 0x7f22dc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22db7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f22dc0a86b0 0x7f22dc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22db7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f22dc0a86b0 0x7f22dc14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:48918/0 (socket says 172.21.2.116:48918) 2021-11-21T21:58:05.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22db7fe700 1 -- 172.21.2.116:0/2408436333 learned_addr learned my addr 172.21.2.116:0/2408436333 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22dbfff700 1 --1- 172.21.2.116:0/2408436333 >> v1:172.21.2.117:6789/0 conn(0x7f22dc0a6aa0 0x7f22dc1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:58618/0 (socket says 172.21.2.116:58618) 2021-11-21T21:58:05.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e0bc2700 1 -- 172.21.2.116:0/2408436333 >> v2:172.21.2.117:3300/0 conn(0x7f22dc0a86b0 msgr2=0x7f22dc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e0bc2700 1 --2- 172.21.2.116:0/2408436333 >> v2:172.21.2.117:3300/0 conn(0x7f22dc0a86b0 0x7f22dc14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e0bc2700 1 -- 172.21.2.116:0/2408436333 >> v1:172.21.2.117:6789/0 conn(0x7f22dc0a6aa0 legacy=0x7f22dc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e0bc2700 1 -- 172.21.2.116:0/2408436333 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc14aa10 con 0x7f22dc062410 2021-11-21T21:58:05.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e0bc2700 1 --2- 172.21.2.116:0/2408436333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc062410 0x7f22dc1443f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f22d40032f0 tx=0x7f22d40091d0).ready entity=mon.2 client_cookie=f2bcbf67c0855730 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22daffd700 1 -- 172.21.2.116:0/2408436333 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22d400b6c0 con 0x7f22dc062410 2021-11-21T21:58:05.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22daffd700 1 -- 172.21.2.116:0/2408436333 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d400b820 con 0x7f22dc062410 2021-11-21T21:58:05.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22daffd700 1 -- 172.21.2.116:0/2408436333 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22d400b990 con 0x7f22dc062410 2021-11-21T21:58:05.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 -- 172.21.2.116:0/2408436333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc062410 msgr2=0x7f22dc1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 --2- 172.21.2.116:0/2408436333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc062410 0x7f22dc1443f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f22d40032f0 tx=0x7f22d40091d0).stop 2021-11-21T21:58:05.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 -- 172.21.2.116:0/2408436333 shutdown_connections 2021-11-21T21:58:05.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 --2- 172.21.2.116:0/2408436333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc062410 0x7f22dc1443f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 --2- 172.21.2.116:0/2408436333 >> v2:172.21.2.117:3300/0 conn(0x7f22dc0a86b0 0x7f22dc14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 -- 172.21.2.116:0/2408436333 >> 172.21.2.116:0/2408436333 conn(0x7f22dc05bd20 msgr2=0x7f22dc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 -- 172.21.2.116:0/2408436333 shutdown_connections 2021-11-21T21:58:05.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.237+0000 7f22e2625700 1 -- 172.21.2.116:0/2408436333 wait complete. 2021-11-21T21:58:05.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 Processor -- start 2021-11-21T21:58:05.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 -- start start 2021-11-21T21:58:05.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f22dc062410 0x7f22dc142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f22dc0a6aa0 0x7f22dc15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc0a86b0 0x7f22dc156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc063d20 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc063fc0 con 0x7f22dc062410 2021-11-21T21:58:05.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1563e0 con 0x7f22dc0a86b0 2021-11-21T21:58:05.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f22dc0a6aa0 0x7f22dc15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f22dc0a6aa0 0x7f22dc15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47758/0 (socket says 172.21.2.116:47758) 2021-11-21T21:58:05.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 -- 172.21.2.116:0/3874521837 learned_addr learned my addr 172.21.2.116:0/3874521837 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e0bc2700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc0a86b0 0x7f22dc156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22dbfff700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f22dc062410 0x7f22dc142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 -- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc0a86b0 msgr2=0x7f22dc156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc0a86b0 0x7f22dc156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 -- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f22dc062410 msgr2=0x7f22dc142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f22dc062410 0x7f22dc142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 -- 172.21.2.116:0/3874521837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009470 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22db7fe700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f22dc0a6aa0 0x7f22dc15a370 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f22cc00c000 tx=0x7f22cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22d97fa700 1 -- 172.21.2.116:0/3874521837 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22cc0136e0 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22d97fa700 1 -- 172.21.2.116:0/3874521837 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc013840 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22d97fa700 1 -- 172.21.2.116:0/3874521837 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22cc013ba0 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc156680 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc063a50 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc156810 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22d97fa700 1 -- 172.21.2.116:0/3874521837 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f22cc023070 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22d97fa700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f22c0053660 0x7f22c0055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22d97fa700 1 -- 172.21.2.116:0/3874521837 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f22cc06bdf0 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.241+0000 7f22dbfff700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f22c0053660 0x7f22c0055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.245+0000 7f22dbfff700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f22c0053660 0x7f22c0055b20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f22c4008000 tx=0x7f22c4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.245+0000 7f22d97fa700 1 -- 172.21.2.116:0/3874521837 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f22cc042e40 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22dc0573d0 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22d97fa700 1 -- 172.21.2.116:0/3874521837 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f22cc046460 con 0x7f22dc0a6aa0 2021-11-21T21:58:05.358 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:58:05.358 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:58:05.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f22c0053660 msgr2=0x7f22c0055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f22c0053660 0x7f22c0055b20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f22c4008000 tx=0x7f22c4009040).stop 2021-11-21T21:58:05.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f22dc0a6aa0 msgr2=0x7f22dc15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f22dc0a6aa0 0x7f22dc15a370 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f22cc00c000 tx=0x7f22cc008040).stop 2021-11-21T21:58:05.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 shutdown_connections 2021-11-21T21:58:05.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f22dc0a86b0 0x7f22dc156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f22c0053660 0x7f22c0055b20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f22dc0a6aa0 0x7f22dc15a370 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 --2- 172.21.2.116:0/3874521837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f22dc062410 0x7f22dc142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 >> 172.21.2.116:0/3874521837 conn(0x7f22dc05bd20 msgr2=0x7f22dc0b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 shutdown_connections 2021-11-21T21:58:05.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.353+0000 7f22e2625700 1 -- 172.21.2.116:0/3874521837 wait complete. 2021-11-21T21:58:05.362 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:58:05.364 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2021-11-21T21:58:05.420 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-11-21T21:58:05.421 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-11-21T21:58:05.421 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:58:05.436 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:58:05.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76facc3700 1 Processor -- start 2021-11-21T21:58:05.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76facc3700 1 -- start start 2021-11-21T21:58:05.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76facc3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f76f40a6aa0 0x7f76f414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76facc3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f40a6e80 con 0x7f76f4062410 2021-11-21T21:58:05.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76facc3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f414a7b0 con 0x7f76f40a86b0 2021-11-21T21:58:05.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76facc3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f40a8bc0 con 0x7f76f40a6aa0 2021-11-21T21:58:05.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76f8a5f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f76f40a6aa0 0x7f76f414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76f9260700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f76f4062410 0x7f76f41443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59724/0 (socket says 172.21.2.116:59724) 2021-11-21T21:58:05.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.481+0000 7f76f9260700 1 -- 172.21.2.116:0/3531685804 learned_addr learned my addr 172.21.2.116:0/3531685804 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:58:05.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f37fe700 1 -- 172.21.2.116:0/3531685804 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042387795 0 0) 0x7f76f40a6e80 con 0x7f76f4062410 2021-11-21T21:58:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f37fe700 1 -- 172.21.2.116:0/3531685804 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76d4003260 con 0x7f76f4062410 2021-11-21T21:58:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f8a5f700 1 -- 172.21.2.116:0/3531685804 >> v1:172.21.2.117:6789/0 conn(0x7f76f40a86b0 legacy=0x7f76f41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f8a5f700 1 -- 172.21.2.116:0/3531685804 >> v1:172.21.2.116:6789/0 conn(0x7f76f4062410 legacy=0x7f76f41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f8a5f700 1 -- 172.21.2.116:0/3531685804 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f414aa10 con 0x7f76f40a6aa0 2021-11-21T21:58:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f8a5f700 1 --2- 172.21.2.116:0/3531685804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a6aa0 0x7f76f414b150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f76f4061b50 tx=0x7f76dc006df0).ready entity=mon.0 client_cookie=44995531caec41c5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f37fe700 1 -- 172.21.2.116:0/3531685804 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76dc00d5c0 con 0x7f76f40a6aa0 2021-11-21T21:58:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f37fe700 1 -- 172.21.2.116:0/3531685804 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76dc000e90 con 0x7f76f40a6aa0 2021-11-21T21:58:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f37fe700 1 -- 172.21.2.116:0/3531685804 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76dc00d910 con 0x7f76f40a6aa0 2021-11-21T21:58:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- 172.21.2.116:0/3531685804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a6aa0 msgr2=0x7f76f414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 --2- 172.21.2.116:0/3531685804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a6aa0 0x7f76f414b150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f76f4061b50 tx=0x7f76dc006df0).stop 2021-11-21T21:58:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- 172.21.2.116:0/3531685804 shutdown_connections 2021-11-21T21:58:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 --2- 172.21.2.116:0/3531685804 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a6aa0 0x7f76f414b150 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- 172.21.2.116:0/3531685804 >> 172.21.2.116:0/3531685804 conn(0x7f76f405bd20 msgr2=0x7f76f405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- 172.21.2.116:0/3531685804 shutdown_connections 2021-11-21T21:58:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- 172.21.2.116:0/3531685804 wait complete. 2021-11-21T21:58:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 Processor -- start 2021-11-21T21:58:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- start start 2021-11-21T21:58:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76f4062410 0x7f76f4067090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76f40a6aa0 0x7f76f4156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a86b0 0x7f76f4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4156d20 con 0x7f76f40a86b0 2021-11-21T21:58:05.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4065510 con 0x7f76f40a6aa0 2021-11-21T21:58:05.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f41476e0 con 0x7f76f4062410 2021-11-21T21:58:05.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a86b0 0x7f76f4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a86b0 0x7f76f4156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47774/0 (socket says 172.21.2.116:47774) 2021-11-21T21:58:05.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 -- 172.21.2.116:0/3293826990 learned_addr learned my addr 172.21.2.116:0/3293826990 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 -- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76f4062410 msgr2=0x7f76f4067090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76f4062410 0x7f76f4067090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 -- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76f40a6aa0 msgr2=0x7f76f4156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76f40a6aa0 0x7f76f4156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 -- 172.21.2.116:0/3293826990 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc00b180 con 0x7f76f40a86b0 2021-11-21T21:58:05.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f9260700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a86b0 0x7f76f4156830 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f76ec001490 tx=0x7f76ec007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f1ffb700 1 -- 172.21.2.116:0/3293826990 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76ec005ba0 con 0x7f76f40a86b0 2021-11-21T21:58:05.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f1ffb700 1 -- 172.21.2.116:0/3293826990 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec005d00 con 0x7f76f40a86b0 2021-11-21T21:58:05.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f4065740 con 0x7f76f40a86b0 2021-11-21T21:58:05.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76f1ffb700 1 -- 172.21.2.116:0/3293826990 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76ec0072c0 con 0x7f76f40a86b0 2021-11-21T21:58:05.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.485+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f40640e0 con 0x7f76f40a86b0 2021-11-21T21:58:05.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.489+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f40658d0 con 0x7f76f40a86b0 2021-11-21T21:58:05.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.489+0000 7f76f1ffb700 1 -- 172.21.2.116:0/3293826990 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f76ec010070 con 0x7f76f40a86b0 2021-11-21T21:58:05.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.489+0000 7f76f1ffb700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76d80534f0 0x7f76d80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.489+0000 7f76f1ffb700 1 -- 172.21.2.116:0/3293826990 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f76ec06b2b0 con 0x7f76f40a86b0 2021-11-21T21:58:05.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.489+0000 7f76f8a5f700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76d80534f0 0x7f76d80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.489+0000 7f76f8a5f700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76d80534f0 0x7f76d80559b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f76f4061b50 tx=0x7f76dc014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.489+0000 7f76f1ffb700 1 -- 172.21.2.116:0/3293826990 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76ec042390 con 0x7f76f40a86b0 2021-11-21T21:58:05.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76f405db70 con 0x7f76f40a86b0 2021-11-21T21:58:05.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76f1ffb700 1 -- 172.21.2.116:0/3293826990 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f76ec0459b0 con 0x7f76f40a86b0 2021-11-21T21:58:05.602 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:58:05.602 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:58:05.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76d80534f0 msgr2=0x7f76d80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76d80534f0 0x7f76d80559b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f76f4061b50 tx=0x7f76dc014000).stop 2021-11-21T21:58:05.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a86b0 msgr2=0x7f76f4156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a86b0 0x7f76f4156830 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f76ec001490 tx=0x7f76ec007d60).stop 2021-11-21T21:58:05.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 shutdown_connections 2021-11-21T21:58:05.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f76d80534f0 0x7f76d80559b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76f40a86b0 0x7f76f4156830 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76f40a6aa0 0x7f76f4156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 --2- 172.21.2.116:0/3293826990 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f76f4062410 0x7f76f4067090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 >> 172.21.2.116:0/3293826990 conn(0x7f76f405bd20 msgr2=0x7f76f4142670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 shutdown_connections 2021-11-21T21:58:05.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.597+0000 7f76facc3700 1 -- 172.21.2.116:0/3293826990 wait complete. 2021-11-21T21:58:05.606 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:58:05.608 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 5 2021-11-21T21:58:05.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351e299700 1 Processor -- start 2021-11-21T21:58:05.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351e299700 1 -- start start 2021-11-21T21:58:05.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351e299700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3518144810 0x7f35181478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351e299700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180ac600 0x7f35180aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351e299700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3518147e80 con 0x7f35180abce0 2021-11-21T21:58:05.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351e299700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35180b1490 con 0x7f3518144810 2021-11-21T21:58:05.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351e299700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35180b1fd0 con 0x7f35180ac600 2021-11-21T21:58:05.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351c836700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180ac600 0x7f35180aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f3517fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3518144810 0x7f35181478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f3517fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3518144810 0x7f35181478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:48946/0 (socket says 172.21.2.116:48946) 2021-11-21T21:58:05.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f351c836700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180ac600 0x7f35180aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:51124/0 (socket says 172.21.2.116:51124) 2021-11-21T21:58:05.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.657+0000 7f3517fff700 1 -- 172.21.2.116:0/3561124835 learned_addr learned my addr 172.21.2.116:0/3561124835 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3517fff700 1 -- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180ac600 msgr2=0x7f35180aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3517fff700 1 --2- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180ac600 0x7f35180aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3517fff700 1 -- 172.21.2.116:0/3561124835 >> v1:172.21.2.117:6789/0 conn(0x7f35180abce0 legacy=0x7f35180b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3517fff700 1 -- 172.21.2.116:0/3561124835 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180af5a0 con 0x7f3518144810 2021-11-21T21:58:05.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 --2- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180ac600 0x7f35180aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T21:58:05.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3517fff700 1 --2- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3518144810 0x7f35181478c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3500002f70 tx=0x7f3500006eb0).ready entity=mon.1 client_cookie=e7445ed8d64ff370 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3516ffd700 1 -- 172.21.2.116:0/3561124835 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f350000d570 con 0x7f3518144810 2021-11-21T21:58:05.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3516ffd700 1 -- 172.21.2.116:0/3561124835 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500000e90 con 0x7f3518144810 2021-11-21T21:58:05.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f3516ffd700 1 -- 172.21.2.116:0/3561124835 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f350000d900 con 0x7f3518144810 2021-11-21T21:58:05.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3518144810 msgr2=0x7f35181478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 --2- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3518144810 0x7f35181478c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3500002f70 tx=0x7f3500006eb0).stop 2021-11-21T21:58:05.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- 172.21.2.116:0/3561124835 shutdown_connections 2021-11-21T21:58:05.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 --2- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180ac600 0x7f35180aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 --2- 172.21.2.116:0/3561124835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3518144810 0x7f35181478c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- 172.21.2.116:0/3561124835 >> 172.21.2.116:0/3561124835 conn(0x7f351805bd20 msgr2=0x7f351805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- 172.21.2.116:0/3561124835 shutdown_connections 2021-11-21T21:58:05.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- 172.21.2.116:0/3561124835 wait complete. 2021-11-21T21:58:05.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 Processor -- start 2021-11-21T21:58:05.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- start start 2021-11-21T21:58:05.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180abce0 0x7f351814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35180ac600 0x7f3518159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3518144810 0x7f351815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518148ed0 con 0x7f3518144810 2021-11-21T21:58:05.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180b3030 con 0x7f35180ac600 2021-11-21T21:58:05.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351e299700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35180b32d0 con 0x7f35180abce0 2021-11-21T21:58:05.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3518144810 0x7f351815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3518144810 0x7f351815a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47788/0 (socket says 172.21.2.116:47788) 2021-11-21T21:58:05.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 -- 172.21.2.116:0/3010205940 learned_addr learned my addr 172.21.2.116:0/3010205940 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 -- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180abce0 msgr2=0x7f351814a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180abce0 0x7f351814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 -- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35180ac600 msgr2=0x7f3518159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35180ac600 0x7f3518159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 -- 172.21.2.116:0/3010205940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3510009470 con 0x7f3518144810 2021-11-21T21:58:05.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.661+0000 7f351c836700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3518144810 0x7f351815a200 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f351000c000 tx=0x7f35100091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35157fa700 1 -- 172.21.2.116:0/3010205940 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f351000c9d0 con 0x7f3518144810 2021-11-21T21:58:05.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35157fa700 1 -- 172.21.2.116:0/3010205940 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351000cb30 con 0x7f3518144810 2021-11-21T21:58:05.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350000b1e0 con 0x7f3518144810 2021-11-21T21:58:05.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35180b3810 con 0x7f3518144810 2021-11-21T21:58:05.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35157fa700 1 -- 172.21.2.116:0/3010205940 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f351000ce90 con 0x7f3518144810 2021-11-21T21:58:05.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35157fa700 1 -- 172.21.2.116:0/3010205940 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3510022070 con 0x7f3518144810 2021-11-21T21:58:05.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35157fa700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34fc053600 0x7f34fc055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35157fa700 1 -- 172.21.2.116:0/3010205940 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f351006c1b0 con 0x7f3518144810 2021-11-21T21:58:05.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 Processor -- start 2021-11-21T21:58:05.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- start start 2021-11-21T21:58:05.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f3517fff700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34fc053600 0x7f34fc055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10e8c90 0x10ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10b1090 0x10a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x10e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x101f7f0 con 0x10b1090 2021-11-21T21:58:05.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0xfffc70 con 0x10e11d0 2021-11-21T21:58:05.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x10e9070 con 0x10e8c90 2021-11-21T21:58:05.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10b1090 0x10a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10b1090 0x10a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47794/0 (socket says 172.21.2.116:47794) 2021-11-21T21:58:05.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 -- 172.21.2.116:0/3699497147 learned_addr learned my addr 172.21.2.116:0/3699497147 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 -- 172.21.2.116:0/3699497147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10e8c90 msgr2=0x10ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 --2- 172.21.2.116:0/3699497147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10e8c90 0x10ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 -- 172.21.2.116:0/3699497147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 msgr2=0x10e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 --2- 172.21.2.116:0/3699497147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x10e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 -- 172.21.2.116:0/3699497147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1056960 con 0x10b1090 2021-11-21T21:58:05.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35177fe700 1 --2- 172.21.2.116:0/3699497147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10b1090 0x10a9b30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x10b14f0 tx=0x7f3508007bc0).ready entity=mon.0 client_cookie=4c877bc5f3816e55 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f351e299700 1 -- 172.21.2.116:0/3699497147 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3508013450 con 0x10b1090 2021-11-21T21:58:05.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f351e299700 1 -- 172.21.2.116:0/3699497147 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35080135b0 con 0x10b1090 2021-11-21T21:58:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f3517fff700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34fc053600 0x7f34fc055ac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f3500006eb0 tx=0x7f3500015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f351e299700 1 -- 172.21.2.116:0/3699497147 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3508013910 con 0x10b1090 2021-11-21T21:58:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- 172.21.2.116:0/3699497147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10b1090 msgr2=0x10a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 --2- 172.21.2.116:0/3699497147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10b1090 0x10a9b30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x10b14f0 tx=0x7f3508007bc0).stop 2021-11-21T21:58:05.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- 172.21.2.116:0/3699497147 shutdown_connections 2021-11-21T21:58:05.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 --2- 172.21.2.116:0/3699497147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x10e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 --2- 172.21.2.116:0/3699497147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10b1090 0x10a9b30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 --2- 172.21.2.116:0/3699497147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10e8c90 0x10ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- 172.21.2.116:0/3699497147 >> 172.21.2.116:0/3699497147 conn(0x10aaf00 msgr2=0x10e8220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- 172.21.2.116:0/3699497147 shutdown_connections 2021-11-21T21:58:05.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.665+0000 7f35284dd740 1 -- 172.21.2.116:0/3699497147 wait complete. 2021-11-21T21:58:05.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- start start 2021-11-21T21:58:05.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 Processor -- start 2021-11-21T21:58:05.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e11d0 0x10b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e8c90 0x10f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x10f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x10e0b70 con 0x10e11d0 2021-11-21T21:58:05.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x10b1c10 con 0x10e8c90 2021-11-21T21:58:05.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x10b1eb0 con 0x10f34c0 2021-11-21T21:58:05.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 10 client.0 ll_register_callbacks cb 0x10132f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:58:05.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e11d0 0x10b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e11d0 0x10b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47800/0 (socket says 172.21.2.116:47800) 2021-11-21T21:58:05.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 -- 172.21.2.116:0/921923205 learned_addr learned my addr 172.21.2.116:0/921923205 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35177fe700 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e8c90 0x10f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f351c836700 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x10f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 -- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 msgr2=0x10f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x10f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 -- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e8c90 msgr2=0x10f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e8c90 0x10f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 -- 172.21.2.116:0/921923205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3518147e80 con 0x10e11d0 2021-11-21T21:58:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f3517fff700 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e11d0 0x10b4240 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f35000069c0 tx=0x7f3500020000).ready entity=mon.0 client_cookie=da0a80a866e2da5a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:58:05.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 1 -- 172.21.2.116:0/921923205 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3500027070 con 0x10e11d0 2021-11-21T21:58:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 1 -- 172.21.2.116:0/921923205 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3500021cb0 con 0x10e11d0 2021-11-21T21:58:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10158b0 con 0x10e11d0 2021-11-21T21:58:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 1 -- 172.21.2.116:0/921923205 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3500021e70 con 0x10e11d0 2021-11-21T21:58:05.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 10 client.5793 fetch_fsmap learned FSMap version 23 2021-11-21T21:58:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x100eea0 con 0x10e11d0 2021-11-21T21:58:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 1 -- 172.21.2.116:0/921923205 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500030070 con 0x10e11d0 2021-11-21T21:58:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 1 -- 172.21.2.116:0/921923205 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 23) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f3500022080 con 0x10e11d0 2021-11-21T21:58:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 10 client.5793 fetch_fsmap finished waiting for FSMap version 23 2021-11-21T21:58:05.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 10 client.5793 resolve_mds: resolved ID 'c' to GID 5586 2021-11-21T21:58:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 20 client.5793 populate_metadata read hostname 'gibba016' 2021-11-21T21:58:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x10f4060 0x10e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 4 client.5793 mds_command: new command op to 5586 tid=0[{"prefix": "get_command_descriptions"}] 2021-11-21T21:58:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 --> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x100d930 con 0x10f4060 2021-11-21T21:58:05.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35177fe700 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x10f4060 0x10e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35177fe700 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x10f4060 0x10e6db0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1bcdba684fdbb9e1 server_cookie=a07704bec9d78c5a in_seq=0 out_seq=0 2021-11-21T21:58:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 10 client.5793 ms_handle_connect on v2:172.21.2.116:6834/3159083787 2021-11-21T21:58:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 1 -- 172.21.2.116:0/921923205 <== mds.0 v2:172.21.2.116:6834/3159083787 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x100d930 con 0x10f4060 2021-11-21T21:58:05.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 10 client.5793 handle_command_reply: tid=0 2021-11-21T21:58:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 client.5793 shutdown 2021-11-21T21:58:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e11d0 msgr2=0x10b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e11d0 0x10b4240 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f35000069c0 tx=0x7f3500020000).stop 2021-11-21T21:58:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 shutdown_connections 2021-11-21T21:58:05.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x10f4060 0x10e6db0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x10f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e8c90 0x10f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 --2- 172.21.2.116:0/921923205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e11d0 0x10b4240 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 >> 172.21.2.116:0/921923205 conn(0x10b1090 msgr2=0x10e5400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f34f6ffd700 0 client.5793 ms_handle_reset on v2:172.21.2.116:6834/3159083787 2021-11-21T21:58:05.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 shutdown_connections 2021-11-21T21:58:05.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 1 -- 172.21.2.116:0/921923205 wait complete. 2021-11-21T21:58:05.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.669+0000 7f35284dd740 20 client.5793 trim_cache size 0 max 16384 2021-11-21T21:58:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 Processor -- start 2021-11-21T21:58:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- start start 2021-11-21T21:58:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x10ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x11420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x11425e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x10addb0 con 0x10e8c90 2021-11-21T21:58:05.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x10ae600 con 0x10e11d0 2021-11-21T21:58:05.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x10e0880 con 0x10f34c0 2021-11-21T21:58:05.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x11420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x11420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47808/0 (socket says 172.21.2.116:47808) 2021-11-21T21:58:05.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 -- 172.21.2.116:0/1283115793 learned_addr learned my addr 172.21.2.116:0/1283115793 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 -- 172.21.2.116:0/1283115793 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 msgr2=0x11425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 --2- 172.21.2.116:0/1283115793 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x11425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 -- 172.21.2.116:0/1283115793 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 msgr2=0x10ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 --2- 172.21.2.116:0/1283115793 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x10ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 -- 172.21.2.116:0/1283115793 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1017a90 con 0x10e8c90 2021-11-21T21:58:05.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35177fe700 1 --2- 172.21.2.116:0/1283115793 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x11420f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f3508017000 tx=0x7f3508013000).ready entity=mon.0 client_cookie=4a0389f734a098ea server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f350d7fa700 1 -- 172.21.2.116:0/1283115793 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3508013910 con 0x10e8c90 2021-11-21T21:58:05.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f350d7fa700 1 -- 172.21.2.116:0/1283115793 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508013a70 con 0x10e8c90 2021-11-21T21:58:05.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f350d7fa700 1 -- 172.21.2.116:0/1283115793 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3508013dd0 con 0x10e8c90 2021-11-21T21:58:05.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- 172.21.2.116:0/1283115793 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 msgr2=0x11420f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:05.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 --2- 172.21.2.116:0/1283115793 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x11420f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f3508017000 tx=0x7f3508013000).stop 2021-11-21T21:58:05.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- 172.21.2.116:0/1283115793 shutdown_connections 2021-11-21T21:58:05.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 --2- 172.21.2.116:0/1283115793 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x11425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 --2- 172.21.2.116:0/1283115793 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x11420f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:05.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 --2- 172.21.2.116:0/1283115793 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x10ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- 172.21.2.116:0/1283115793 >> 172.21.2.116:0/1283115793 conn(0x10b1090 msgr2=0x10f0990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:05.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- 172.21.2.116:0/1283115793 shutdown_connections 2021-11-21T21:58:05.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.677+0000 7f35284dd740 1 -- 172.21.2.116:0/1283115793 wait complete. 2021-11-21T21:58:05.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 -- start start 2021-11-21T21:58:05.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 Processor -- start 2021-11-21T21:58:05.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x113cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x1139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x113a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x10a8af0 con 0x10e8c90 2021-11-21T21:58:05.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x1139770 con 0x10e11d0 2021-11-21T21:58:05.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x1139a10 con 0x10f34c0 2021-11-21T21:58:05.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 10 client.0 ll_register_callbacks cb 0x10b2350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:58:05.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x1139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x1139eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47812/0 (socket says 172.21.2.116:47812) 2021-11-21T21:58:05.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 -- 172.21.2.116:0/1298854065 learned_addr learned my addr 172.21.2.116:0/1298854065 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:05.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 -- 172.21.2.116:0/1298854065 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 msgr2=0x113a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x113a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 -- 172.21.2.116:0/1298854065 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 msgr2=0x113cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:05.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x113cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:05.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 -- 172.21.2.116:0/1298854065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1056960 con 0x10e8c90 2021-11-21T21:58:05.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35177fe700 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x1139eb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f3508017000 tx=0x7f350801d000).ready entity=mon.0 client_cookie=5e4c915ff0878d13 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:05.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f34f7fff700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:58:05.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f34f7fff700 1 -- 172.21.2.116:0/1298854065 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3508021510 con 0x10e8c90 2021-11-21T21:58:05.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f34f7fff700 1 -- 172.21.2.116:0/1298854065 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3508003870 con 0x10e8c90 2021-11-21T21:58:05.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35080135b0 con 0x10e8c90 2021-11-21T21:58:05.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f34f7fff700 1 -- 172.21.2.116:0/1298854065 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3508003a30 con 0x10e8c90 2021-11-21T21:58:05.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 10 client.5799 fetch_fsmap learned FSMap version 23 2021-11-21T21:58:05.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x1014480 con 0x10e8c90 2021-11-21T21:58:05.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f34f7fff700 1 -- 172.21.2.116:0/1298854065 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350802f080 con 0x10e8c90 2021-11-21T21:58:05.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f34f7fff700 1 -- 172.21.2.116:0/1298854065 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 23) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f3508021c80 con 0x10e8c90 2021-11-21T21:58:05.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 10 client.5799 fetch_fsmap finished waiting for FSMap version 23 2021-11-21T21:58:05.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 10 client.5799 resolve_mds: resolved ID 'c' to GID 5586 2021-11-21T21:58:05.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 20 client.5799 populate_metadata read hostname 'gibba016' 2021-11-21T21:58:05.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x113a890 0x113e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:05.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 4 client.5799 mds_command: new command op to 5586 tid=0[{"prefix": "cache drop", "timeout": 5}] 2021-11-21T21:58:05.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 --> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0xf035e0 con 0x113a890 2021-11-21T21:58:05.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f3517fff700 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x113a890 0x113e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:05.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f34f7fff700 10 client.5799 ms_handle_connect on v2:172.21.2.116:6834/3159083787 2021-11-21T21:58:05.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:05.681+0000 7f3517fff700 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x113a890 0x113e630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d222e7021a50349d server_cookie=b98f82649337ea14 in_seq=0 out_seq=0 2021-11-21T21:58:10.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f34f7fff700 1 -- 172.21.2.116:0/1298854065 <== mds.0 v2:172.21.2.116:6834/3159083787 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0xf035e0 con 0x113a890 2021-11-21T21:58:10.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f34f7fff700 10 client.5799 handle_command_reply: tid=0 2021-11-21T21:58:10.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 client.5799 shutdown 2021-11-21T21:58:10.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 msgr2=0x1139eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:10.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x1139eb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f3508017000 tx=0x7f350801d000).stop 2021-11-21T21:58:10.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 shutdown_connections 2021-11-21T21:58:10.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:6834/3159083787,v1:172.21.2.116:6835/3159083787] conn(0x113a890 0x113e630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:10.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x10f34c0 0x113a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:10.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x10e8c90 0x1139eb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:10.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 --2- 172.21.2.116:0/1298854065 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x10e11d0 0x113cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:10.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f34f7fff700 0 client.5799 ms_handle_reset on v2:172.21.2.116:6834/3159083787 2021-11-21T21:58:10.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 >> 172.21.2.116:0/1298854065 conn(0x10b1090 msgr2=0x10f6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:10.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 shutdown_connections 2021-11-21T21:58:10.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 1 -- 172.21.2.116:0/1298854065 wait complete. 2021-11-21T21:58:10.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f35284dd740 20 client.5799 trim_cache size 0 max 16384 2021-11-21T21:58:10.774 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:58:10.774 INFO:teuthology.orchestra.run.gibba016.stdout: "client_recall": { 2021-11-21T21:58:10.775 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": -110, 2021-11-21T21:58:10.775 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "(110) Connection timed out", 2021-11-21T21:58:10.775 INFO:teuthology.orchestra.run.gibba016.stdout: "recalled": 120 2021-11-21T21:58:10.775 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:58:10.776 INFO:teuthology.orchestra.run.gibba016.stdout: "flush_journal": { 2021-11-21T21:58:10.776 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": 0, 2021-11-21T21:58:10.776 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "" 2021-11-21T21:58:10.776 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:58:10.776 INFO:teuthology.orchestra.run.gibba016.stdout: "trim_cache": { 2021-11-21T21:58:10.777 INFO:teuthology.orchestra.run.gibba016.stdout: "trimmed": 0 2021-11-21T21:58:10.777 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:58:10.777 INFO:teuthology.orchestra.run.gibba016.stdout: "cache": { 2021-11-21T21:58:10.777 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": { 2021-11-21T21:58:10.777 INFO:teuthology.orchestra.run.gibba016.stdout: "items": 19929, 2021-11-21T21:58:10.778 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes": 3095776 2021-11-21T21:58:10.778 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:58:10.778 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:58:10.778 INFO:teuthology.orchestra.run.gibba016.stdout: "duration": 5.0836289240000001 2021-11-21T21:58:10.780 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:58:10.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34fc053600 msgr2=0x7f34fc055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:10.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34fc053600 0x7f34fc055ac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f3500006eb0 tx=0x7f3500015000).stop 2021-11-21T21:58:10.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3518144810 msgr2=0x7f351815a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:10.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3518144810 0x7f351815a200 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f351000c000 tx=0x7f35100091d0).stop 2021-11-21T21:58:10.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 shutdown_connections 2021-11-21T21:58:10.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34fc053600 0x7f34fc055ac0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:10.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3518144810 0x7f351815a200 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:10.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35180ac600 0x7f3518159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:10.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 --2- 172.21.2.116:0/3010205940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35180abce0 0x7f351814a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:10.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 >> 172.21.2.116:0/3010205940 conn(0x7f351805bd20 msgr2=0x7f35180b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:10.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 shutdown_connections 2021-11-21T21:58:10.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.765+0000 7f351e299700 1 -- 172.21.2.116:0/3010205940 wait complete. 2021-11-21T21:58:10.784 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-11-21T21:58:10.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:10.785 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:58:10.786 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:10.796 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-11-21T21:58:10.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:58:10.797 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-11-21T21:58:10.798 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:58:10.798 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:58:10.799 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:10.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:10.857 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:58:10.857 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:10.861 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T21:58:10.862 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:58:10.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:58:10.863 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:10.867 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:58:10.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.973+0000 7f850fb74700 1 Processor -- start 2021-11-21T21:58:10.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- start start 2021-11-21T21:58:10.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85080a6aa0 0x7f850814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:10.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85080a6e80 con 0x7f8508062410 2021-11-21T21:58:10.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850814a7b0 con 0x7f85080a86b0 2021-11-21T21:58:10.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85080a8bc0 con 0x7f85080a6aa0 2021-11-21T21:58:10.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850e111700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8508062410 0x7f85081443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59786/0 (socket says 172.21.2.116:59786) 2021-11-21T21:58:10.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850e111700 1 -- 172.21.2.116:0/1573627440 learned_addr learned my addr 172.21.2.116:0/1573627440 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:58:10.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828164468 0 0) 0x7f85080a6e80 con 0x7f8508062410 2021-11-21T21:58:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f0003260 con 0x7f8508062410 2021-11-21T21:58:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85080a6aa0 0x7f850814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708024944 0 0) 0x7f84f0003260 con 0x7f8508062410 2021-11-21T21:58:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85080a6aa0 msgr2=0x7f850814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85080a6aa0 0x7f850814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 >> v1:172.21.2.117:6789/0 conn(0x7f85080a86b0 legacy=0x7f85081470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850814aa10 con 0x7f8508062410 2021-11-21T21:58:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8500001a00 con 0x7f8508062410 2021-11-21T21:58:10.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 >> v1:172.21.2.116:6789/0 conn(0x7f8508062410 legacy=0x7f85081443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:10.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84f0003800 0x7f84f0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85080a6aa0 0x7f84f0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84f0004b10 0x7f84f0004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85080a8bc0 con 0x7f84f0003800 2021-11-21T21:58:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f000b910 con 0x7f84f0004b10 2021-11-21T21:58:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f000bad0 con 0x7f85080a6aa0 2021-11-21T21:58:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84f0003800 0x7f84f0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 -- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85080a6aa0 msgr2=0x7f84f0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85080a6aa0 0x7f84f0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 -- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84f0004b10 msgr2=0x7f84f0004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84f0004b10 0x7f84f0004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 -- 172.21.2.116:0/1573627440 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0003260 con 0x7f84f0003800 2021-11-21T21:58:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850d910700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84f0003800 0x7f84f0003c00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f850400a000 tx=0x7f8504008040).ready entity=mon.0 client_cookie=d29aa86f84e9273f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8504013670 con 0x7f84f0003800 2021-11-21T21:58:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85040137d0 con 0x7f84f0003800 2021-11-21T21:58:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850c90e700 1 -- 172.21.2.116:0/1573627440 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8504013b30 con 0x7f84f0003800 2021-11-21T21:58:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- 172.21.2.116:0/1573627440 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84f0003800 msgr2=0x7f84f0003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 --2- 172.21.2.116:0/1573627440 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84f0003800 0x7f84f0003c00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f850400a000 tx=0x7f8504008040).stop 2021-11-21T21:58:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850e111700 1 -- 172.21.2.116:0/1573627440 reap_dead start 2021-11-21T21:58:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- 172.21.2.116:0/1573627440 shutdown_connections 2021-11-21T21:58:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- 172.21.2.116:0/1573627440 >> 172.21.2.116:0/1573627440 conn(0x7f850805bd20 msgr2=0x7f850805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- 172.21.2.116:0/1573627440 shutdown_connections 2021-11-21T21:58:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- 172.21.2.116:0/1573627440 wait complete. 2021-11-21T21:58:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 Processor -- start 2021-11-21T21:58:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 -- start start 2021-11-21T21:58:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.977+0000 7f850fb74700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84f0003800 0x7f8508143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f850813b5d0 0x7f85081391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85081396b0 0x7f85081565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850813eba0 con 0x7f85081396b0 2021-11-21T21:58:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8508139b60 con 0x7f850813b5d0 2021-11-21T21:58:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850813ee60 con 0x7f84f0003800 2021-11-21T21:58:11.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85081396b0 0x7f85081565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85081396b0 0x7f85081565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47842/0 (socket says 172.21.2.116:47842) 2021-11-21T21:58:11.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 -- 172.21.2.116:0/3327159449 learned_addr learned my addr 172.21.2.116:0/3327159449 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 -- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84f0003800 msgr2=0x7f8508143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84f0003800 0x7f8508143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 -- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f850813b5d0 msgr2=0x7f85081391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f850813b5d0 0x7f85081391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 -- 172.21.2.116:0/3327159449 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850813e660 con 0x7f85081396b0 2021-11-21T21:58:11.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850e111700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85081396b0 0x7f85081565a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f8508139cf0 tx=0x7f850000c8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f84faffd700 1 -- 172.21.2.116:0/3327159449 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8500004ab0 con 0x7f85081396b0 2021-11-21T21:58:11.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f84faffd700 1 -- 172.21.2.116:0/3327159449 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500004c10 con 0x7f85081396b0 2021-11-21T21:58:11.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850813e900 con 0x7f85081396b0 2021-11-21T21:58:11.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f84faffd700 1 -- 172.21.2.116:0/3327159449 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8500007170 con 0x7f85081396b0 2021-11-21T21:58:11.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850813f080 con 0x7f85081396b0 2021-11-21T21:58:11.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85080573d0 con 0x7f85081396b0 2021-11-21T21:58:11.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f84faffd700 1 -- 172.21.2.116:0/3327159449 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f850001c070 con 0x7f85081396b0 2021-11-21T21:58:11.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f84faffd700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84e8053610 0x7f84e8055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f84faffd700 1 -- 172.21.2.116:0/3327159449 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f850006af20 con 0x7f85081396b0 2021-11-21T21:58:11.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850d910700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84e8053610 0x7f84e8055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.981+0000 7f850d910700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84e8053610 0x7f84e8055ad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f850400a000 tx=0x7f8504008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:10.985+0000 7f84faffd700 1 -- 172.21.2.116:0/3327159449 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8500042080 con 0x7f85081396b0 2021-11-21T21:58:11.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.065+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f850814ae10 con 0x7f85081396b0 2021-11-21T21:58:11.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.065+0000 7f84faffd700 1 -- 172.21.2.116:0/3327159449 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16040 (secure 0 0 0) 0x7f85000456a0 con 0x7f85081396b0 2021-11-21T21:58:11.071 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:58:11.072 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":55,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:57:47.221698+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:57:44.702609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-11-21T21:57:45.184798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:58:11.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84e8053610 msgr2=0x7f84e8055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84e8053610 0x7f84e8055ad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f850400a000 tx=0x7f8504008040).stop 2021-11-21T21:58:11.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85081396b0 msgr2=0x7f85081565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85081396b0 0x7f85081565a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f8508139cf0 tx=0x7f850000c8f0).stop 2021-11-21T21:58:11.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 shutdown_connections 2021-11-21T21:58:11.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84e8053610 0x7f84e8055ad0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85081396b0 0x7f85081565a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f850813b5d0 0x7f85081391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 --2- 172.21.2.116:0/3327159449 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84f0003800 0x7f8508143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 >> 172.21.2.116:0/3327159449 conn(0x7f850805bd20 msgr2=0x7f8508148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 shutdown_connections 2021-11-21T21:58:11.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.069+0000 7f850fb74700 1 -- 172.21.2.116:0/3327159449 wait complete. 2021-11-21T21:58:11.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:57:44.702609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:57:45.184798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:58:11.079 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:58:11.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 Processor -- start 2021-11-21T21:58:11.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- start start 2021-11-21T21:58:11.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1d10144810 0x7f1d101478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d100ac600 0x7f1d100aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d10147e80 con 0x7f1d100abce0 2021-11-21T21:58:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d100b1490 con 0x7f1d100ac600 2021-11-21T21:58:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d100b1fd0 con 0x7f1d10144810 2021-11-21T21:58:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d100ac600 0x7f1d100aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d100ac600 0x7f1d100aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47848/0 (socket says 172.21.2.116:47848) 2021-11-21T21:58:11.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 -- 172.21.2.116:0/402334182 learned_addr learned my addr 172.21.2.116:0/402334182 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 -- 172.21.2.116:0/402334182 >> v2:172.21.2.117:3300/0 conn(0x7f1d10144810 msgr2=0x7f1d101478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 --2- 172.21.2.116:0/402334182 >> v2:172.21.2.117:3300/0 conn(0x7f1d10144810 0x7f1d101478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 -- 172.21.2.116:0/402334182 >> v1:172.21.2.117:6789/0 conn(0x7f1d100abce0 legacy=0x7f1d100b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 -- 172.21.2.116:0/402334182 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d100af5a0 con 0x7f1d100ac600 2021-11-21T21:58:11.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d15e8b700 1 --2- 172.21.2.116:0/402334182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d100aca50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f1d100b4e00 tx=0x7f1d08009290).ready entity=mon.0 client_cookie=b7c16ce518a572df server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1cfffff700 1 -- 172.21.2.116:0/402334182 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0800b830 con 0x7f1d100ac600 2021-11-21T21:58:11.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1cfffff700 1 -- 172.21.2.116:0/402334182 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0800b990 con 0x7f1d100ac600 2021-11-21T21:58:11.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1cfffff700 1 -- 172.21.2.116:0/402334182 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0800bb40 con 0x7f1d100ac600 2021-11-21T21:58:11.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- 172.21.2.116:0/402334182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 msgr2=0x7f1d100aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- 172.21.2.116:0/402334182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d100aca50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f1d100b4e00 tx=0x7f1d08009290).stop 2021-11-21T21:58:11.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- 172.21.2.116:0/402334182 shutdown_connections 2021-11-21T21:58:11.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- 172.21.2.116:0/402334182 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d100aca50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- 172.21.2.116:0/402334182 >> v2:172.21.2.117:3300/0 conn(0x7f1d10144810 0x7f1d101478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- 172.21.2.116:0/402334182 >> 172.21.2.116:0/402334182 conn(0x7f1d1005bd20 msgr2=0x7f1d1005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- 172.21.2.116:0/402334182 shutdown_connections 2021-11-21T21:58:11.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- 172.21.2.116:0/402334182 wait complete. 2021-11-21T21:58:11.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 Processor -- start 2021-11-21T21:58:11.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- start start 2021-11-21T21:58:11.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d100abce0 0x7f1d10145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d1013c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d10144810 0x7f1d1013cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1014acf0 con 0x7f1d100ac600 2021-11-21T21:58:11.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100b3be0 con 0x7f1d10144810 2021-11-21T21:58:11.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d178ee700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100b42b0 con 0x7f1d100abce0 2021-11-21T21:58:11.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d1013c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d1013c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47856/0 (socket says 172.21.2.116:47856) 2021-11-21T21:58:11.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 -- 172.21.2.116:0/133838033 learned_addr learned my addr 172.21.2.116:0/133838033 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 -- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d100abce0 msgr2=0x7f1d10145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d100abce0 0x7f1d10145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 -- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d10144810 msgr2=0x7f1d1013cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d10144810 0x7f1d1013cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 -- 172.21.2.116:0/133838033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08009530 con 0x7f1d100ac600 2021-11-21T21:58:11.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.129+0000 7f1d14e89700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d1013c610 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f1d10156660 tx=0x7f1d00006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1cfe7fc700 1 -- 172.21.2.116:0/133838033 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d00013460 con 0x7f1d100ac600 2021-11-21T21:58:11.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1cfe7fc700 1 -- 172.21.2.116:0/133838033 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000135c0 con 0x7f1d100ac600 2021-11-21T21:58:11.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1014a1f0 con 0x7f1d100ac600 2021-11-21T21:58:11.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d100674d0 con 0x7f1d100ac600 2021-11-21T21:58:11.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1cfe7fc700 1 -- 172.21.2.116:0/133838033 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d00013920 con 0x7f1d100ac600 2021-11-21T21:58:11.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d100af060 con 0x7f1d100ac600 2021-11-21T21:58:11.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1cfe7fc700 1 -- 172.21.2.116:0/133838033 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1d0001a060 con 0x7f1d100ac600 2021-11-21T21:58:11.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1cfe7fc700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf0053580 0x7f1cf0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1cfe7fc700 1 -- 172.21.2.116:0/133838033 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1d0006b1d0 con 0x7f1d100ac600 2021-11-21T21:58:11.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1d1568a700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf0053580 0x7f1cf0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1d1568a700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf0053580 0x7f1cf0055a40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f1d0c002f70 tx=0x7f1d0c0069f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.133+0000 7f1cfe7fc700 1 -- 172.21.2.116:0/133838033 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d00042330 con 0x7f1d100ac600 2021-11-21T21:58:11.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1d100573d0 con 0x7f1d100ac600 2021-11-21T21:58:11.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1cfe7fc700 1 -- 172.21.2.116:0/133838033 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1d00018020 con 0x7f1d100ac600 2021-11-21T21:58:11.222 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:58:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf0053580 msgr2=0x7f1cf0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf0053580 0x7f1cf0055a40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f1d0c002f70 tx=0x7f1d0c0069f0).stop 2021-11-21T21:58:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 msgr2=0x7f1d1013c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d1013c610 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f1d10156660 tx=0x7f1d00006df0).stop 2021-11-21T21:58:11.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 shutdown_connections 2021-11-21T21:58:11.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d10144810 0x7f1d1013cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf0053580 0x7f1cf0055a40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d100ac600 0x7f1d1013c610 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 --2- 172.21.2.116:0/133838033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d100abce0 0x7f1d10145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 >> 172.21.2.116:0/133838033 conn(0x7f1d1005bd20 msgr2=0x7f1d100b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 shutdown_connections 2021-11-21T21:58:11.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.217+0000 7f1d178ee700 1 -- 172.21.2.116:0/133838033 wait complete. 2021-11-21T21:58:11.229 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:58:11.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.277+0000 7f3240a8d700 1 Processor -- start 2021-11-21T21:58:11.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.277+0000 7f3240a8d700 1 -- start start 2021-11-21T21:58:11.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.277+0000 7f3240a8d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c062d00 0x7f323c064c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.277+0000 7f3240a8d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f323c0af850 0x7f323c13dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.277+0000 7f3240a8d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c144640 con 0x7f323c0b0120 2021-11-21T21:58:11.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.277+0000 7f3240a8d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c0630e0 con 0x7f323c0af850 2021-11-21T21:58:11.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.277+0000 7f3240a8d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c064550 con 0x7f323c062d00 2021-11-21T21:58:11.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f323c0af850 0x7f323c13dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f323c0af850 0x7f323c13dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47860/0 (socket says 172.21.2.116:47860) 2021-11-21T21:58:11.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 -- 172.21.2.116:0/335146997 learned_addr learned my addr 172.21.2.116:0/335146997 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 -- 172.21.2.116:0/335146997 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c062d00 msgr2=0x7f323c064c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 --2- 172.21.2.116:0/335146997 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c062d00 0x7f323c064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 -- 172.21.2.116:0/335146997 >> v1:172.21.2.116:6789/0 conn(0x7f323c0b0120 legacy=0x7f323c143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 -- 172.21.2.116:0/335146997 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0b23f0 con 0x7f323c0af850 2021-11-21T21:58:11.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323ad9d700 1 --2- 172.21.2.116:0/335146997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c13dd10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f32340032f0 tx=0x7f3234009290).ready entity=mon.0 client_cookie=6763231a9da06774 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323959a700 1 -- 172.21.2.116:0/335146997 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f323400b8a0 con 0x7f323c0af850 2021-11-21T21:58:11.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323959a700 1 -- 172.21.2.116:0/335146997 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323400ba00 con 0x7f323c0af850 2021-11-21T21:58:11.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f323959a700 1 -- 172.21.2.116:0/335146997 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f323400bb70 con 0x7f323c0af850 2021-11-21T21:58:11.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/335146997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 msgr2=0x7f323c13dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 --2- 172.21.2.116:0/335146997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c13dd10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f32340032f0 tx=0x7f3234009290).stop 2021-11-21T21:58:11.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/335146997 shutdown_connections 2021-11-21T21:58:11.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 --2- 172.21.2.116:0/335146997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c13dd10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 --2- 172.21.2.116:0/335146997 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c062d00 0x7f323c064c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/335146997 >> 172.21.2.116:0/335146997 conn(0x7f323c05bd20 msgr2=0x7f323c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/335146997 shutdown_connections 2021-11-21T21:58:11.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/335146997 wait complete. 2021-11-21T21:58:11.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 Processor -- start 2021-11-21T21:58:11.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- start start 2021-11-21T21:58:11.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f323c062d00 0x7f323c13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c0b0120 0x7f323c153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c13aea0 con 0x7f323c0af850 2021-11-21T21:58:11.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c1403f0 con 0x7f323c062d00 2021-11-21T21:58:11.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c159ad0 con 0x7f323c0b0120 2021-11-21T21:58:11.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47870/0 (socket says 172.21.2.116:47870) 2021-11-21T21:58:11.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 -- 172.21.2.116:0/824777320 learned_addr learned my addr 172.21.2.116:0/824777320 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 -- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c0b0120 msgr2=0x7f323c153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c0b0120 0x7f323c153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 -- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f323c062d00 msgr2=0x7f323c13fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f323c062d00 0x7f323c13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 -- 172.21.2.116:0/824777320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3234009530 con 0x7f323c0af850 2021-11-21T21:58:11.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3239d9b700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c153340 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f323c159c60 tx=0x7f3224007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f32337fe700 1 -- 172.21.2.116:0/824777320 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f322400ce20 con 0x7f323c0af850 2021-11-21T21:58:11.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f32337fe700 1 -- 172.21.2.116:0/824777320 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400f470 con 0x7f323c0af850 2021-11-21T21:58:11.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c142740 con 0x7f323c0af850 2021-11-21T21:58:11.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c159830 con 0x7f323c0af850 2021-11-21T21:58:11.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f32337fe700 1 -- 172.21.2.116:0/824777320 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f322400f7d0 con 0x7f323c0af850 2021-11-21T21:58:11.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.281+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c0573d0 con 0x7f323c0af850 2021-11-21T21:58:11.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.285+0000 7f32337fe700 1 -- 172.21.2.116:0/824777320 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3224019030 con 0x7f323c0af850 2021-11-21T21:58:11.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.285+0000 7f32337fe700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3220053610 0x7f3220055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.285+0000 7f32337fe700 1 -- 172.21.2.116:0/824777320 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f322406ad30 con 0x7f323c0af850 2021-11-21T21:58:11.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.285+0000 7f323a59c700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3220053610 0x7f3220055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.285+0000 7f323a59c700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3220053610 0x7f3220055ad0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f322c002f70 tx=0x7f322c0069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.285+0000 7f32337fe700 1 -- 172.21.2.116:0/824777320 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3224041e30 con 0x7f323c0af850 2021-11-21T21:58:11.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f323c068320 con 0x7f323c0af850 2021-11-21T21:58:11.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f32337fe700 1 -- 172.21.2.116:0/824777320 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3224045450 con 0x7f323c0af850 2021-11-21T21:58:11.375 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:58:11.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3220053610 msgr2=0x7f3220055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3220053610 0x7f3220055ad0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f322c002f70 tx=0x7f322c0069e0).stop 2021-11-21T21:58:11.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 msgr2=0x7f323c153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c153340 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f323c159c60 tx=0x7f3224007c50).stop 2021-11-21T21:58:11.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 shutdown_connections 2021-11-21T21:58:11.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f323c0b0120 0x7f323c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3220053610 0x7f3220055ad0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f323c0af850 0x7f323c153340 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 --2- 172.21.2.116:0/824777320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f323c062d00 0x7f323c13fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 >> 172.21.2.116:0/824777320 conn(0x7f323c05bd20 msgr2=0x7f323c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 shutdown_connections 2021-11-21T21:58:11.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.369+0000 7f3240a8d700 1 -- 172.21.2.116:0/824777320 wait complete. 2021-11-21T21:58:11.381 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:58:11.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.429+0000 7fecf8cd7700 1 Processor -- start 2021-11-21T21:58:11.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.429+0000 7fecf8cd7700 1 -- start start 2021-11-21T21:58:11.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.429+0000 7fecf8cd7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fecf4062e60 0x7fecf4142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40afae0 con 0x7fecf41426e0 2021-11-21T21:58:11.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40b0f60 con 0x7fecf40ad470 2021-11-21T21:58:11.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf4063470 con 0x7fecf4062e60 2021-11-21T21:58:11.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf259c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fecf41426e0 0x7fecf40af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59830/0 (socket says 172.21.2.116:59830) 2021-11-21T21:58:11.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf259c700 1 -- 172.21.2.116:0/3178951564 learned_addr learned my addr 172.21.2.116:0/3178951564 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:58:11.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf1d9b700 1 --2- 172.21.2.116:0/3178951564 >> v2:172.21.2.116:3300/0 conn(0x7fecf4062e60 0x7fecf4142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf1d9b700 1 -- 172.21.2.116:0/3178951564 >> v1:172.21.2.117:6789/0 conn(0x7fecf40ad470 legacy=0x7fecf40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf1d9b700 1 -- 172.21.2.116:0/3178951564 >> v1:172.21.2.116:6789/0 conn(0x7fecf41426e0 legacy=0x7fecf40af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf1d9b700 1 -- 172.21.2.116:0/3178951564 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4143050 con 0x7fecf4062e60 2021-11-21T21:58:11.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf1d9b700 1 --2- 172.21.2.116:0/3178951564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf4062e60 0x7fecf4142ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fece8001110 tx=0x7fece4006df0).ready entity=mon.0 client_cookie=a4486c000c35edb4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf159a700 1 -- 172.21.2.116:0/3178951564 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fece400d4a0 con 0x7fecf4062e60 2021-11-21T21:58:11.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf159a700 1 -- 172.21.2.116:0/3178951564 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4000e90 con 0x7fecf4062e60 2021-11-21T21:58:11.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf159a700 1 -- 172.21.2.116:0/3178951564 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fece400d7f0 con 0x7fecf4062e60 2021-11-21T21:58:11.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- 172.21.2.116:0/3178951564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf4062e60 msgr2=0x7fecf4142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/3178951564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf4062e60 0x7fecf4142ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fece8001110 tx=0x7fece4006df0).stop 2021-11-21T21:58:11.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- 172.21.2.116:0/3178951564 shutdown_connections 2021-11-21T21:58:11.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/3178951564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf4062e60 0x7fecf4142ac0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- 172.21.2.116:0/3178951564 >> 172.21.2.116:0/3178951564 conn(0x7fecf40b5730 msgr2=0x7fecf40b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- 172.21.2.116:0/3178951564 shutdown_connections 2021-11-21T21:58:11.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- 172.21.2.116:0/3178951564 wait complete. 2021-11-21T21:58:11.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 Processor -- start 2021-11-21T21:58:11.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- start start 2021-11-21T21:58:11.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fecf40ad470 0x7fecf413de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fecf41426e0 0x7fecf4066da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf41457e0 0x7fecf4067290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf413fef0 con 0x7fecf41457e0 2021-11-21T21:58:11.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4145fc0 con 0x7fecf41426e0 2021-11-21T21:58:11.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4146260 con 0x7fecf40ad470 2021-11-21T21:58:11.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf41457e0 0x7fecf4067290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf41457e0 0x7fecf4067290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47884/0 (socket says 172.21.2.116:47884) 2021-11-21T21:58:11.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 -- 172.21.2.116:0/1532835632 learned_addr learned my addr 172.21.2.116:0/1532835632 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 -- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fecf40ad470 msgr2=0x7fecf413de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fecf40ad470 0x7fecf413de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 -- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fecf41426e0 msgr2=0x7fecf4066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fecf41426e0 0x7fecf4066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 -- 172.21.2.116:0/1532835632 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece400b180 con 0x7fecf41457e0 2021-11-21T21:58:11.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf2d9d700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf41457e0 0x7fecf4067290 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fecec003310 tx=0x7fecec00b430).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fece37fe700 1 -- 172.21.2.116:0/1532835632 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecec00ed50 con 0x7fecf41457e0 2021-11-21T21:58:11.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fece37fe700 1 -- 172.21.2.116:0/1532835632 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec017420 con 0x7fecf41457e0 2021-11-21T21:58:11.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf413fa50 con 0x7fecf41457e0 2021-11-21T21:58:11.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fece37fe700 1 -- 172.21.2.116:0/1532835632 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecec017780 con 0x7fecf41457e0 2021-11-21T21:58:11.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.433+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf40637a0 con 0x7fecf41457e0 2021-11-21T21:58:11.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.437+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf4146420 con 0x7fecf41457e0 2021-11-21T21:58:11.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.437+0000 7fece37fe700 1 -- 172.21.2.116:0/1532835632 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fecec0178e0 con 0x7fecf41457e0 2021-11-21T21:58:11.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.437+0000 7fece37fe700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fecd0053580 0x7fecd0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.437+0000 7fece37fe700 1 -- 172.21.2.116:0/1532835632 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fecec06c050 con 0x7fecf41457e0 2021-11-21T21:58:11.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.437+0000 7fecf259c700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fecd0053580 0x7fecd0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.437+0000 7fecf259c700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fecd0053580 0x7fecd0055a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fece8000bc0 tx=0x7fece8007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.437+0000 7fece37fe700 1 -- 172.21.2.116:0/1532835632 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fecec0431b0 con 0x7fecf41457e0 2021-11-21T21:58:11.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecf413fbe0 con 0x7fecf41457e0 2021-11-21T21:58:11.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fece37fe700 1 -- 172.21.2.116:0/1532835632 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7fecec0467d0 con 0x7fecf41457e0 2021-11-21T21:58:11.526 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:58:11.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fecd0053580 msgr2=0x7fecd0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fecd0053580 0x7fecd0055a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fece8000bc0 tx=0x7fece8007810).stop 2021-11-21T21:58:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf41457e0 msgr2=0x7fecf4067290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf41457e0 0x7fecf4067290 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fecec003310 tx=0x7fecec00b430).stop 2021-11-21T21:58:11.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 shutdown_connections 2021-11-21T21:58:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fecd0053580 0x7fecd0055a40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fecf41457e0 0x7fecf4067290 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fecf41426e0 0x7fecf4066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 --2- 172.21.2.116:0/1532835632 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fecf40ad470 0x7fecf413de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 >> 172.21.2.116:0/1532835632 conn(0x7fecf40b5730 msgr2=0x7fecf41272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 shutdown_connections 2021-11-21T21:58:11.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.521+0000 7fecf8cd7700 1 -- 172.21.2.116:0/1532835632 wait complete. 2021-11-21T21:58:11.533 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:58:11.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.581+0000 7fc22f3d7700 1 Processor -- start 2021-11-21T21:58:11.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.581+0000 7fc22f3d7700 1 -- start start 2021-11-21T21:58:11.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.581+0000 7fc22f3d7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc228144810 0x7fc2281478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.581+0000 7fc22f3d7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc2280ac600 0x7fc2280aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.581+0000 7fc22f3d7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc228147e80 con 0x7fc2280abce0 2021-11-21T21:58:11.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.581+0000 7fc22f3d7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2280b1490 con 0x7fc2280ac600 2021-11-21T21:58:11.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.581+0000 7fc22f3d7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2280b1fd0 con 0x7fc228144810 2021-11-21T21:58:11.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc2280ac600 0x7fc2280aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc2280ac600 0x7fc2280aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47890/0 (socket says 172.21.2.116:47890) 2021-11-21T21:58:11.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/236453371 learned_addr learned my addr 172.21.2.116:0/236453371 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/236453371 >> v2:172.21.2.117:3300/0 conn(0x7fc228144810 msgr2=0x7fc2281478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- 172.21.2.116:0/236453371 >> v2:172.21.2.117:3300/0 conn(0x7fc228144810 0x7fc2281478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/236453371 >> v1:172.21.2.116:6789/0 conn(0x7fc2280abce0 legacy=0x7fc2280b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/236453371 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280af5a0 con 0x7fc2280ac600 2021-11-21T21:58:11.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- 172.21.2.116:0/236453371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2280ac600 0x7fc2280aca50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fc2200032f0 tx=0x7fc220009290).ready entity=mon.0 client_cookie=b1bc221aa5d82ea9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc217fff700 1 -- 172.21.2.116:0/236453371 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22000b870 con 0x7fc2280ac600 2021-11-21T21:58:11.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc217fff700 1 -- 172.21.2.116:0/236453371 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22000b9d0 con 0x7fc2280ac600 2021-11-21T21:58:11.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc217fff700 1 -- 172.21.2.116:0/236453371 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22000bb40 con 0x7fc2280ac600 2021-11-21T21:58:11.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- 172.21.2.116:0/236453371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2280ac600 msgr2=0x7fc2280aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/236453371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2280ac600 0x7fc2280aca50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fc2200032f0 tx=0x7fc220009290).stop 2021-11-21T21:58:11.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- 172.21.2.116:0/236453371 shutdown_connections 2021-11-21T21:58:11.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/236453371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc2280ac600 0x7fc2280aca50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/236453371 >> v2:172.21.2.117:3300/0 conn(0x7fc228144810 0x7fc2281478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- 172.21.2.116:0/236453371 >> 172.21.2.116:0/236453371 conn(0x7fc22805bd20 msgr2=0x7fc22805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- 172.21.2.116:0/236453371 shutdown_connections 2021-11-21T21:58:11.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- 172.21.2.116:0/236453371 wait complete. 2021-11-21T21:58:11.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 Processor -- start 2021-11-21T21:58:11.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- start start 2021-11-21T21:58:11.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280abce0 0x7fc2280afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2280ac600 0x7fc2280b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228144810 0x7fc228159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280b3e70 con 0x7fc228144810 2021-11-21T21:58:11.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280b40a0 con 0x7fc2280abce0 2021-11-21T21:58:11.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280b4340 con 0x7fc2280ac600 2021-11-21T21:58:11.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228144810 0x7fc228159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228144810 0x7fc228159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47898/0 (socket says 172.21.2.116:47898) 2021-11-21T21:58:11.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/2195809564 learned_addr learned my addr 172.21.2.116:0/2195809564 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2280ac600 msgr2=0x7fc2280b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2280ac600 0x7fc2280b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280abce0 msgr2=0x7fc2280afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280abce0 0x7fc2280afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 -- 172.21.2.116:0/2195809564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc220009530 con 0x7fc228144810 2021-11-21T21:58:11.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22d974700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228144810 0x7fc228159d40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fc2200032f0 tx=0x7fc220016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc2167fc700 1 -- 172.21.2.116:0/2195809564 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22001f3f0 con 0x7fc228144810 2021-11-21T21:58:11.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc2167fc700 1 -- 172.21.2.116:0/2195809564 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22001f550 con 0x7fc228144810 2021-11-21T21:58:11.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22813cf20 con 0x7fc228144810 2021-11-21T21:58:11.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc2167fc700 1 -- 172.21.2.116:0/2195809564 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22001f8b0 con 0x7fc228144810 2021-11-21T21:58:11.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.585+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22815a230 con 0x7fc228144810 2021-11-21T21:58:11.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.589+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2280b3cc0 con 0x7fc228144810 2021-11-21T21:58:11.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.589+0000 7fc2167fc700 1 -- 172.21.2.116:0/2195809564 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc220026030 con 0x7fc228144810 2021-11-21T21:58:11.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.589+0000 7fc2167fc700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc2080536a0 0x7fc208055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.589+0000 7fc2167fc700 1 -- 172.21.2.116:0/2195809564 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc22006c7d0 con 0x7fc228144810 2021-11-21T21:58:11.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.589+0000 7fc22d173700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc2080536a0 0x7fc208055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.589+0000 7fc22d173700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc2080536a0 0x7fc208055b60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fc224002f70 tx=0x7fc2240069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.589+0000 7fc2167fc700 1 -- 172.21.2.116:0/2195809564 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc220043930 con 0x7fc228144810 2021-11-21T21:58:11.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.673+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2280b4500 con 0x7fc228144810 2021-11-21T21:58:11.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.673+0000 7fc2167fc700 1 -- 172.21.2.116:0/2195809564 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc2200240a0 con 0x7fc228144810 2021-11-21T21:58:11.678 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:58:11.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.673+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc2080536a0 msgr2=0x7fc208055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.673+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc2080536a0 0x7fc208055b60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fc224002f70 tx=0x7fc2240069e0).stop 2021-11-21T21:58:11.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.673+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228144810 msgr2=0x7fc228159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.673+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228144810 0x7fc228159d40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fc2200032f0 tx=0x7fc220016000).stop 2021-11-21T21:58:11.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 shutdown_connections 2021-11-21T21:58:11.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc2080536a0 0x7fc208055b60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228144810 0x7fc228159d40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc2280ac600 0x7fc2280b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 --2- 172.21.2.116:0/2195809564 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280abce0 0x7fc2280afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 >> 172.21.2.116:0/2195809564 conn(0x7fc22805bd20 msgr2=0x7fc2280b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 shutdown_connections 2021-11-21T21:58:11.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.677+0000 7fc22f3d7700 1 -- 172.21.2.116:0/2195809564 wait complete. 2021-11-21T21:58:11.685 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:58:11.686 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:58:11.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f8390bc4700 1 Processor -- start 2021-11-21T21:58:11.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f8390bc4700 1 -- start start 2021-11-21T21:58:11.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f8390bc4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f838c0a86b0 0x7f838c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f8390bc4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c0a6e80 con 0x7f838c0a6aa0 2021-11-21T21:58:11.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f8390bc4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c14a7b0 con 0x7f838c062410 2021-11-21T21:58:11.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f8390bc4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0a8bc0 con 0x7f838c0a86b0 2021-11-21T21:58:11.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838a59c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f838c0a6aa0 0x7f838c1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59858/0 (socket says 172.21.2.116:59858) 2021-11-21T21:58:11.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838a59c700 1 -- 172.21.2.116:0/2083157569 learned_addr learned my addr 172.21.2.116:0/2083157569 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:58:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413599468 0 0) 0x7f838c0a6e80 con 0x7f838c0a6aa0 2021-11-21T21:58:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f836c003260 con 0x7f838c0a6aa0 2021-11-21T21:58:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f8389d9b700 1 --2- 172.21.2.116:0/2083157569 >> v2:172.21.2.117:3300/0 conn(0x7f838c0a86b0 0x7f838c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3439773806 0 0) 0x7f836c003260 con 0x7f838c0a6aa0 2021-11-21T21:58:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 >> v2:172.21.2.117:3300/0 conn(0x7f838c0a86b0 msgr2=0x7f838c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 --2- 172.21.2.116:0/2083157569 >> v2:172.21.2.117:3300/0 conn(0x7f838c0a86b0 0x7f838c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 >> v1:172.21.2.117:6789/0 conn(0x7f838c062410 legacy=0x7f838c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c14aa10 con 0x7f838c0a6aa0 2021-11-21T21:58:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8374001680 con 0x7f838c0a6aa0 2021-11-21T21:58:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 >> v1:172.21.2.116:6789/0 conn(0x7f838c0a6aa0 legacy=0x7f838c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c003820 0x7f836c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836c004500 0x7f836c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c005050 0x7f836c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0a8bc0 con 0x7f836c005050 2021-11-21T21:58:11.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c00bf40 con 0x7f836c004500 2021-11-21T21:58:11.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c00c100 con 0x7f836c003820 2021-11-21T21:58:11.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c005050 0x7f836c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 -- 172.21.2.116:0/2083157569 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c003820 msgr2=0x7f836c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c003820 0x7f836c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 -- 172.21.2.116:0/2083157569 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836c004500 msgr2=0x7f836c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:58:11.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836c004500 0x7f836c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 -- 172.21.2.116:0/2083157569 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c003260 con 0x7f836c005050 2021-11-21T21:58:11.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 -- 172.21.2.116:0/2083157569 reap_dead start 2021-11-21T21:58:11.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838ad9d700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c005050 0x7f836c00b750 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f83840012f0 tx=0x7f8384007d60).ready entity=mon.0 client_cookie=24b8ab301cfbb4d7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83840014c0 con 0x7f836c005050 2021-11-21T21:58:11.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384006870 con 0x7f836c005050 2021-11-21T21:58:11.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.733+0000 7f838959a700 1 -- 172.21.2.116:0/2083157569 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8384006bd0 con 0x7f836c005050 2021-11-21T21:58:11.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/2083157569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c005050 msgr2=0x7f836c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c005050 0x7f836c00b750 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f83840012f0 tx=0x7f8384007d60).stop 2021-11-21T21:58:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/2083157569 shutdown_connections 2021-11-21T21:58:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 --2- 172.21.2.116:0/2083157569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c005050 0x7f836c00b750 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/2083157569 >> 172.21.2.116:0/2083157569 conn(0x7f838c05bd20 msgr2=0x7f838c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/2083157569 shutdown_connections 2021-11-21T21:58:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/2083157569 wait complete. 2021-11-21T21:58:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 Processor -- start 2021-11-21T21:58:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- start start 2021-11-21T21:58:11.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c005050 0x7f838c156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f838c13e3b0 0x7f838c149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838c14b370 0x7f838c0ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c13ce90 con 0x7f838c14b370 2021-11-21T21:58:11.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c155ed0 con 0x7f838c13e3b0 2021-11-21T21:58:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c156130 con 0x7f836c005050 2021-11-21T21:58:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838c14b370 0x7f838c0ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838c14b370 0x7f838c0ab2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47918/0 (socket says 172.21.2.116:47918) 2021-11-21T21:58:11.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 -- 172.21.2.116:0/3283057225 learned_addr learned my addr 172.21.2.116:0/3283057225 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:58:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 -- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c005050 msgr2=0x7f838c156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c005050 0x7f838c156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 -- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f838c13e3b0 msgr2=0x7f838c149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f838c13e3b0 0x7f838c149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 -- 172.21.2.116:0/3283057225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c1563d0 con 0x7f838c14b370 2021-11-21T21:58:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f838ad9d700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838c14b370 0x7f838c0ab2c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f83840012f0 tx=0x7f8384016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f83837fe700 1 -- 172.21.2.116:0/3283057225 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8384014670 con 0x7f838c14b370 2021-11-21T21:58:11.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f83837fe700 1 -- 172.21.2.116:0/3283057225 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83840147d0 con 0x7f838c14b370 2021-11-21T21:58:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f83837fe700 1 -- 172.21.2.116:0/3283057225 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8384014b30 con 0x7f838c14b370 2021-11-21T21:58:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c156670 con 0x7f838c14b370 2021-11-21T21:58:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c13e8e0 con 0x7f838c14b370 2021-11-21T21:58:11.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c156800 con 0x7f838c14b370 2021-11-21T21:58:11.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f83837fe700 1 -- 172.21.2.116:0/3283057225 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8384028070 con 0x7f838c14b370 2021-11-21T21:58:11.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f83837fe700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8370053600 0x7f8370055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:58:11.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.737+0000 7f83837fe700 1 -- 172.21.2.116:0/3283057225 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f838406c990 con 0x7f838c14b370 2021-11-21T21:58:11.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.741+0000 7f838a59c700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8370053600 0x7f8370055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:58:11.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.741+0000 7f838a59c700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8370053600 0x7f8370055ac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f838c061b50 tx=0x7f838c0a8520).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:58:11.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.741+0000 7f83837fe700 1 -- 172.21.2.116:0/3283057225 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83840439e0 con 0x7f838c14b370 2021-11-21T21:58:11.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f838c13ce70 con 0x7f838c14b370 2021-11-21T21:58:11.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f83837fe700 1 -- 172.21.2.116:0/3283057225 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f8384047000 con 0x7f838c14b370 2021-11-21T21:58:11.853 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:58:11.853 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:58:11.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8370053600 msgr2=0x7f8370055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8370053600 0x7f8370055ac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f838c061b50 tx=0x7f838c0a8520).stop 2021-11-21T21:58:11.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838c14b370 msgr2=0x7f838c0ab2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:58:11.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838c14b370 0x7f838c0ab2c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f83840012f0 tx=0x7f8384016000).stop 2021-11-21T21:58:11.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 shutdown_connections 2021-11-21T21:58:11.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8370053600 0x7f8370055ac0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838c14b370 0x7f838c0ab2c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:58:11.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f838c13e3b0 0x7f838c149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 --2- 172.21.2.116:0/3283057225 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c005050 0x7f838c156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:58:11.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 >> 172.21.2.116:0/3283057225 conn(0x7f838c05bd20 msgr2=0x7f838c1421a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:58:11.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 shutdown_connections 2021-11-21T21:58:11.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:58:11.849+0000 7f8390bc4700 1 -- 172.21.2.116:0/3283057225 wait complete. 2021-11-21T21:58:11.857 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:58:12.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:57:46.218398+0000', 'modified': '2021-11-21T21:57:47.223289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5586}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5586': {'gid': 5586, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3159083787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3159083787}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3159083787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:58:12.162 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:58:12.162 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:58:12.242 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5586,"want_state":"up:active","state":"up:active","rank_uptime":26.017917034,"mdsmap_epoch":23,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":30.250712193999998} 2021-11-21T21:58:12.242 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5586, "mdsmap_epoch": 23, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 26.017917034, "state": "up:active", "uptime": 30.250712194, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:58:12.243 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:58:12.244 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T21:58:12.245 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T21:58:12.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:12.245 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:12.256 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:58:12.268 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:58:12.268 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:58:12.274 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:58:12.275 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:58:12.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:12.276 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:58:12.322 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:58:12.322 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:12.325 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:58:12.335 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:58:12.336 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:58:12.341 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:58:12.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:58:12.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:12.344 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:58:12.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T21:58:12.360+0000 7fd8fe302080 -1 init, newargv = 0x558ebd283bb0 newargc=9 2021-11-21T21:58:12.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14413]: starting ceph client 2021-11-21T21:58:12.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14413]: starting fuse 2021-11-21T21:58:13.471 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:58:13.484 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:58:13.484 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:58:13.532 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:58:13.532 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:58:13.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:13.534 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:58:13.539 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T21:58:13.540 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:58:13.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:13.542 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T21:58:13.542 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T21:58:13.543 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T21:58:13.543 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:58:13.543 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T21:58:13.544 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:58:13.544 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T21:58:13.545 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T21:58:13.545 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T21:58:13.545 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:58:13.546 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T21:58:13.546 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T21:58:13.547 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T21:58:13.547 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T21:58:13.547 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:58:13.548 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T21:58:13.548 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:58:13.549 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:58:13.549 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T21:58:13.549 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:58:13.550 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:58:13.550 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T21:58:13.551 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:58:13.579 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14413.asok 2021-11-21T21:58:13.581 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14413.asok 2021-11-21T21:58:13.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:13.582 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14413.asok status 2021-11-21T21:58:13.692 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T21:58:13.692 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T21:58:13.693 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:58:13.693 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:58:13.693 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T21:58:13.693 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T21:58:13.694 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T21:58:13.694 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14413", 2021-11-21T21:58:13.694 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T21:58:13.694 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:58:13.695 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T21:58:13.695 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T21:58:13.695 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5850, 2021-11-21T21:58:13.695 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T21:58:13.696 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T21:58:13.696 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T21:58:13.696 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5850 2021-11-21T21:58:13.696 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:58:13.696 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:58:13.697 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:58:13.697 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:58:13.697 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1813189506 2021-11-21T21:58:13.697 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T21:58:13.698 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:58:13.698 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:58:13.698 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:58:13.698 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:58:13.699 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1813189506 2021-11-21T21:58:13.699 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:58:13.699 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5850 172.21.2.119:0/1813189506", 2021-11-21T21:58:13.699 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/1813189506", 2021-11-21T21:58:13.700 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T21:58:13.700 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 23, 2021-11-21T21:58:13.700 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 55, 2021-11-21T21:58:13.700 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:58:13.701 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T21:58:13.701 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T21:58:13.701 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T21:58:13.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:13.703 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:58:13.703 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:13.708 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:58:13.709 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:58:13.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:58:13.710 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:58:34.066 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:58:34.071 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:58:34.073 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:58:34.078 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:04.118 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:04.122 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:04.126 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:04.129 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:06.997 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:59:06.998 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:07.026 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:07.040 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:07.053 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:07.067 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:59:07.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:07.068 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:07.068 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:07.075 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:59:07.075 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:59:07.075 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:59:07.076 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:59:07.076 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:07.128 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14413]: fuse finished with error 0 and tester_r 0 2021-11-21T21:59:07.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:07.130 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:07.130 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:07.134 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T21:59:07.135 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:59:07.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:59:07.247 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:07.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:07.252 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:07.252 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:59:07.261 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:59:07.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:07.263 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:59:07.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:59:07.264 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:59:07.273 INFO:teuthology.orchestra.run.gibba036.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:59:07.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:07.275 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:59:07.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.326+0000 7f6b3b19c700 1 Processor -- start 2021-11-21T21:59:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.326+0000 7f6b3b19c700 1 -- start start 2021-11-21T21:59:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.326+0000 7f6b3b19c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b340a86b0 0x7f6b3414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.326+0000 7f6b3b19c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340a6e80 con 0x7f6b34062410 2021-11-21T21:59:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.326+0000 7f6b3b19c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3414a7b0 con 0x7f6b340a6aa0 2021-11-21T21:59:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.326+0000 7f6b3b19c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340a8bc0 con 0x7f6b340a86b0 2021-11-21T21:59:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f6b34062410 0x7f6b341443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59882/0 (socket says 172.21.2.116:59882) 2021-11-21T21:59:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/1462046267 learned_addr learned my addr 172.21.2.116:0/1462046267 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789850502 0 0) 0x7f6b340a6e80 con 0x7f6b34062410 2021-11-21T21:59:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b20003260 con 0x7f6b34062410 2021-11-21T21:59:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3422011617 0 0) 0x7f6b20003260 con 0x7f6b34062410 2021-11-21T21:59:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b340a86b0 msgr2=0x7f6b3414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b340a86b0 0x7f6b3414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 >> v1:172.21.2.117:6789/0 conn(0x7f6b340a6aa0 legacy=0x7f6b341470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3414aa10 con 0x7f6b34062410 2021-11-21T21:59:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f6b2c001a00 con 0x7f6b34062410 2021-11-21T21:59:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 >> v1:172.21.2.116:6789/0 conn(0x7f6b34062410 legacy=0x7f6b341443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b20003800 0x7f6b20003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b340a86b0 0x7f6b20004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b20004bf0 0x7f6b2000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340a8bc0 con 0x7f6b20004bf0 2021-11-21T21:59:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200052a0 con 0x7f6b20003800 2021-11-21T21:59:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2000bc40 con 0x7f6b340a86b0 2021-11-21T21:59:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b20004bf0 0x7f6b2000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b340a86b0 msgr2=0x7f6b20004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b340a86b0 0x7f6b20004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b20003800 msgr2=0x7f6b20003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b20003800 0x7f6b20003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/1462046267 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b20003260 con 0x7f6b20004bf0 2021-11-21T21:59:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b20004bf0 0x7f6b2000b750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6b34061b50 tx=0x7f6b2c002100).ready entity=mon.0 client_cookie=ffe4cd7a0d4ced88 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2c00cb80 con 0x7f6b20004bf0 2021-11-21T21:59:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c00cce0 con 0x7f6b20004bf0 2021-11-21T21:59:07.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b337fe700 1 -- 172.21.2.116:0/1462046267 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2c004b50 con 0x7f6b20004bf0 2021-11-21T21:59:07.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- 172.21.2.116:0/1462046267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b20004bf0 msgr2=0x7f6b2000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 --2- 172.21.2.116:0/1462046267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b20004bf0 0x7f6b2000b750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6b34061b50 tx=0x7f6b2c002100).stop 2021-11-21T21:59:07.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/1462046267 reap_dead start 2021-11-21T21:59:07.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- 172.21.2.116:0/1462046267 shutdown_connections 2021-11-21T21:59:07.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- 172.21.2.116:0/1462046267 >> 172.21.2.116:0/1462046267 conn(0x7f6b3405bd20 msgr2=0x7f6b3405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:07.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- 172.21.2.116:0/1462046267 shutdown_connections 2021-11-21T21:59:07.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- 172.21.2.116:0/1462046267 wait complete. 2021-11-21T21:59:07.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 Processor -- start 2021-11-21T21:59:07.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- start start 2021-11-21T21:59:07.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b20004bf0 0x7f6b34143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b340a8f20 0x7f6b341562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b340652c0 0x7f6b340642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b341567e0 con 0x7f6b340652c0 2021-11-21T21:59:07.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340a94a0 con 0x7f6b340a8f20 2021-11-21T21:59:07.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b3b19c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340a9740 con 0x7f6b20004bf0 2021-11-21T21:59:07.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b340652c0 0x7f6b340642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b340652c0 0x7f6b340642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47938/0 (socket says 172.21.2.116:47938) 2021-11-21T21:59:07.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/3903505487 learned_addr learned my addr 172.21.2.116:0/3903505487 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:07.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b20004bf0 msgr2=0x7f6b34143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b20004bf0 0x7f6b34143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b340a8f20 msgr2=0x7f6b341562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b340a8f20 0x7f6b341562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.330+0000 7f6b39739700 1 -- 172.21.2.116:0/3903505487 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340a99e0 con 0x7f6b340652c0 2021-11-21T21:59:07.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b39739700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b340652c0 0x7f6b340642b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f6b34060090 tx=0x7f6b2c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b31ffb700 1 -- 172.21.2.116:0/3903505487 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2c001ee0 con 0x7f6b340652c0 2021-11-21T21:59:07.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b31ffb700 1 -- 172.21.2.116:0/3903505487 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c019030 con 0x7f6b340652c0 2021-11-21T21:59:07.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b31ffb700 1 -- 172.21.2.116:0/3903505487 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2c007570 con 0x7f6b340652c0 2021-11-21T21:59:07.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b34063e10 con 0x7f6b340652c0 2021-11-21T21:59:07.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b340647a0 con 0x7f6b340652c0 2021-11-21T21:59:07.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b34063fd0 con 0x7f6b340652c0 2021-11-21T21:59:07.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b31ffb700 1 -- 172.21.2.116:0/3903505487 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6b2c027070 con 0x7f6b340652c0 2021-11-21T21:59:07.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b31ffb700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b140535b0 0x7f6b14055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b31ffb700 1 -- 172.21.2.116:0/3903505487 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6b2c01c040 con 0x7f6b340652c0 2021-11-21T21:59:07.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b38f38700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b140535b0 0x7f6b14055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.334+0000 7f6b38f38700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b140535b0 0x7f6b14055a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6b24000f70 tx=0x7f6b240069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.338+0000 7f6b31ffb700 1 -- 172.21.2.116:0/3903505487 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b2c043130 con 0x7f6b340652c0 2021-11-21T21:59:07.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6b340573d0 con 0x7f6b340652c0 2021-11-21T21:59:07.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b31ffb700 1 -- 172.21.2.116:0/3903505487 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+33647 (secure 0 0 0) 0x7f6b2c046750 con 0x7f6b340652c0 2021-11-21T21:59:07.421 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:07.421 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:59:07.422 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 55, 2021-11-21T21:59:07.422 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:59:07.422 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:59:07.422 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:57:47.221698+0000", 2021-11-21T21:59:07.423 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:07.423 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:07.423 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:59:07.423 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:59:07.424 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:59:07.424 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:59:07.424 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:59:07.424 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:59:07.424 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:59:07.425 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:07.425 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:59:07.425 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:59:07.425 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:59:07.426 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:59:07.426 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:59:07.426 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 10, 2021-11-21T21:59:07.426 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:59:07.427 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:59:07.427 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:59:07.427 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:59:07.427 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:59:07.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.428 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:59:07.428 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:59:07.428 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:59:07.428 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:07.428 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:07.429 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:07.429 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:07.429 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:07.429 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:07.430 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:07.430 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:07.430 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:59:07.430 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:59:07.430 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:59:07.431 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:59:07.431 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:59:07.431 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:07.432 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:07.432 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:07.432 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:07.432 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:07.432 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:07.433 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:07.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.433 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:59:07.434 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:07.434 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:07.434 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:07.434 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:07.435 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:07.435 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:07.435 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:07.435 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:07.436 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:07.436 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:07.436 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:07.436 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:07.437 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:07.437 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:07.437 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:07.437 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:07.437 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:07.438 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:07.438 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:07.438 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:07.441 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:07.442 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:07.442 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:07.442 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:07.442 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:07.442 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:07.443 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.443 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:07.443 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:07.443 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:07.444 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:07.444 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:07.444 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:07.444 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:07.444 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:07.445 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:07.445 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:07.445 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:07.445 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:59:07.446 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:59:07.446 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.446 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:07.446 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:59:07.447 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.447 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.447 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.447 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:59:07.447 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:59:07.448 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:59:07.448 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:07.448 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:07.448 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:07.449 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:07.449 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:07.449 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:07.449 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:07.449 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:07.450 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:07.450 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:07.450 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:07.450 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:07.451 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:07.451 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:07.451 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:07.451 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:07.451 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:07.452 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:07.452 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:07.452 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:07.452 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.453 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:59:07.453 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:07.453 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:07.453 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:07.454 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:07.454 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:07.454 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:07.454 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:07.454 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:07.455 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:07.455 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:07.455 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:07.455 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:07.456 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:07.456 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:07.456 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:07.456 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:07.456 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:07.457 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:07.457 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:07.457 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:07.457 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:07.457 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:07.458 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:07.458 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:07.458 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:07.458 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:07.459 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.459 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:07.459 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:07.459 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:07.459 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:07.460 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:07.460 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:07.460 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:07.460 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:07.461 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:07.461 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:07.461 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:07.461 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:59:07.461 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:07.462 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:59:07.462 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.462 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.463 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 9, 2021-11-21T21:59:07.463 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T21:59:07.463 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:57:44.702609+0000", 2021-11-21T21:59:07.463 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:07.463 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:07.464 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:07.464 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:07.464 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:07.464 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:07.465 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:07.465 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:07.465 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:07.465 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:07.466 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:07.466 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:07.466 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:07.466 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:07.466 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:07.467 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:07.467 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:07.467 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:07.467 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:07.468 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:07.468 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.468 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "54", 2021-11-21T21:59:07.468 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:07.469 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:07.469 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:07.469 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:07.469 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:07.469 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:07.470 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:07.470 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:07.470 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:07.470 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:07.471 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:07.471 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:07.471 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:07.471 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:07.471 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:07.472 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:07.472 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:07.472 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:07.472 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:07.473 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:07.473 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:07.473 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:07.473 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:07.473 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:07.474 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:07.474 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:07.474 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.474 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:07.475 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:07.475 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:07.475 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:07.475 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:07.475 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:07.476 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:07.476 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:07.476 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:07.476 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:07.477 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:07.477 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:59:07.477 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T21:59:07.477 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T21:59:07.477 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T21:59:07.478 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.478 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:07.478 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:59:07.478 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T21:59:07.478 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.479 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.479 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.479 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 10, 2021-11-21T21:59:07.480 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T21:59:07.480 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:57:45.184798+0000", 2021-11-21T21:59:07.480 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:07.480 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:07.480 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:07.481 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:07.481 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:07.481 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:07.481 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:07.482 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:07.482 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:07.482 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:07.482 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:07.482 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:07.483 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:07.483 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:07.483 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:07.483 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:07.484 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:07.484 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:07.484 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:07.484 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:07.484 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.485 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "55", 2021-11-21T21:59:07.485 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:07.485 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:07.485 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:07.486 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:07.486 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:07.486 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:07.486 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:07.486 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:07.487 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:07.487 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:07.487 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:07.487 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:07.488 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:07.488 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:07.488 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:07.488 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:07.488 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:07.489 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:07.489 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:07.489 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:07.489 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:07.490 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:07.490 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:07.490 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:07.490 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:07.491 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:07.491 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.491 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:07.491 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:07.492 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:07.492 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:07.492 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:07.492 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:07.492 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:07.493 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:07.493 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:07.493 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:07.493 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:07.493 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:59:07.494 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:07.494 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:59:07.494 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T21:59:07.494 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.495 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.495 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.495 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:07.495 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:59:07.496 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.496 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:07.496 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:59:07.496 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.496 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.497 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.497 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.497 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.497 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.497 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.498 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:07.498 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.498 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.498 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.499 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.499 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:59:07.500 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.500 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.500 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.500 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.500 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:59:07.501 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.501 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.501 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.501 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.502 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.502 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.502 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.502 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.502 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:59:07.503 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.503 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.503 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.503 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.504 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:59:07.504 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.504 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.504 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.505 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.505 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.505 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.505 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.505 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.506 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:59:07.506 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.506 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.506 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.507 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.507 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:59:07.507 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.507 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.507 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.508 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.508 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.508 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.508 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.509 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:59:07.509 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.509 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.509 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.510 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:59:07.510 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:07.510 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.511 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.511 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.511 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:59:07.511 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:59:07.511 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:59:07.512 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:59:07.512 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.512 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.512 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.512 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.513 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.513 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.513 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:07.513 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:59:07.513 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.514 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.514 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.514 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.514 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.515 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.515 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.515 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:07.515 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.515 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.516 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.516 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.516 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.516 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.516 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:59:07.517 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.517 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.517 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.517 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.518 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:59:07.518 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.518 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.518 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.518 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.519 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.519 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.519 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.519 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.520 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:59:07.520 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.520 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.520 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.520 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.521 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:59:07.521 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.521 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.521 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.521 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.522 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.522 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.522 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.522 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.523 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:59:07.523 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.523 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.523 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.523 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.524 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:59:07.524 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.524 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.524 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.524 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.525 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.525 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.525 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.525 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.526 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:59:07.526 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.526 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.526 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.526 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:59:07.527 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:07.527 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.527 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.528 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.528 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:59:07.528 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:59:07.528 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:59:07.528 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:59:07.529 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.529 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.529 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.529 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.529 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.530 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.530 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:07.530 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:59:07.530 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.531 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.531 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.531 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.531 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.531 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.532 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.532 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:07.532 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.532 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.532 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.533 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.533 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.533 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.533 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:59:07.534 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.534 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.534 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.534 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:59:07.535 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.535 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.535 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.535 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.536 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.536 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.536 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.536 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.537 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:59:07.537 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.537 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.537 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.538 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.538 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:59:07.538 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.538 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.538 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.539 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.539 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.539 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.539 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.540 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.540 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:59:07.540 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.540 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.540 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.541 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.541 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:59:07.541 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.541 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.542 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.542 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.542 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.542 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.542 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.543 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.543 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:59:07.543 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.543 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.544 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.544 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:59:07.544 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:07.545 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.545 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.545 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.545 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:59:07.545 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:59:07.546 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:59:07.546 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:59:07.546 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.546 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.547 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.547 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.547 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.547 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.548 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:07.548 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:59:07.548 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.548 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.548 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.549 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.549 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.549 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.549 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.549 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 52, 2021-11-21T21:59:07.550 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.550 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.550 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.550 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.551 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.551 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.551 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:59:07.551 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.551 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.552 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.552 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.552 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:59:07.552 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.553 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.553 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.553 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.553 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.553 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.554 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.554 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.554 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:59:07.554 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.555 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.555 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.555 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.555 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:59:07.555 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.556 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.556 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.556 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.556 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.557 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.557 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.557 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.557 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:59:07.557 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.558 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.558 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.558 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.558 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:59:07.559 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.559 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.559 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.559 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.559 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.560 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.560 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.560 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.560 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:59:07.561 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.561 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.561 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.561 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.561 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:59:07.562 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:07.562 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.562 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.562 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.563 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:59:07.563 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:59:07.563 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:59:07.563 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:59:07.564 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.564 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.564 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.564 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.564 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.565 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.565 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:07.565 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:59:07.565 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.566 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.567 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.567 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:07.567 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.567 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.567 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.568 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.568 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.568 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.568 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:59:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.569 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:59:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.570 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.571 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:59:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.572 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:59:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.573 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.574 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.574 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.574 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.574 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.575 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:59:07.575 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.575 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.575 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.576 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.576 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:59:07.576 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.576 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.576 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.577 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.577 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.577 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.577 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.578 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.578 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:59:07.578 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.578 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.578 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.579 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.579 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:59:07.579 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:07.579 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.580 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.580 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.580 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:59:07.580 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:59:07.580 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:59:07.581 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:59:07.581 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.581 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.581 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.581 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.582 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.582 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.582 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:07.582 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:59:07.583 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.583 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.583 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.583 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.584 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.584 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.584 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.584 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:07.584 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.585 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.585 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.585 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.585 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.586 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.586 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:59:07.586 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.586 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.586 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.587 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.587 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:59:07.587 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.587 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.587 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.588 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.588 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.588 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.588 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.589 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.589 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:59:07.589 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.589 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.589 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.590 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.590 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:59:07.590 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.590 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.591 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.591 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.591 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.591 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.591 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.592 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.592 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:59:07.592 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.592 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.592 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.593 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.593 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:59:07.593 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.593 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.594 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.594 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.594 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.594 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.595 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.595 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.595 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:59:07.595 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.595 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.596 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.596 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.596 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:59:07.596 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:07.596 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.597 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.597 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.597 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:59:07.597 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:59:07.598 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:59:07.598 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:59:07.598 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.598 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.598 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.599 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.599 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.599 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.599 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:07.599 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:59:07.600 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.600 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.600 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.600 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.601 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.601 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.601 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.601 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:07.601 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.602 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.602 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.602 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.602 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.603 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.603 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:59:07.603 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.603 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.603 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.604 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.604 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:59:07.604 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.604 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.604 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.605 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.605 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.605 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.605 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.606 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.606 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:59:07.606 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.606 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.606 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.607 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.607 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:59:07.607 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.607 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.608 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.608 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.608 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.608 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.608 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.609 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.609 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:59:07.609 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.609 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.609 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.610 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.610 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:59:07.610 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.610 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.611 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.611 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.611 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.611 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.611 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.612 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.612 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:59:07.612 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.612 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.613 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.613 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.613 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:59:07.613 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:07.613 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.614 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.614 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.614 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:59:07.614 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:59:07.614 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:59:07.615 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:59:07.615 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.615 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.615 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:59:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:07.617 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 52, 2021-11-21T21:59:07.618 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.619 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.620 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.620 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:59:07.620 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.620 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.620 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:59:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.621 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.622 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.622 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:07.622 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.622 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.623 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.623 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:59:07.623 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.623 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.623 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.624 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.624 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:59:07.624 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.624 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.625 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.625 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.625 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:07.625 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.625 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.626 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.626 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:59:07.626 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.626 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.627 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.627 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.627 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:59:07.627 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.628 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.628 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.628 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.628 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:07.628 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:07.629 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.629 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:07.629 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:59:07.629 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.630 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.630 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.630 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:07.630 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:59:07.630 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:07.631 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.631 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.631 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.631 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:59:07.631 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:59:07.632 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:59:07.632 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:59:07.632 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:07.632 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:07.633 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:07.633 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:07.633 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.633 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:07.633 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:59:07.634 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.634 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:07.634 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.634 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.635 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.635 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.635 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.635 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:59:07.635 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.636 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.636 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.636 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:07.636 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.637 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.637 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.637 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.637 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.637 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:59:07.638 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.638 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.638 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.638 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:07.639 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.639 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.639 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.639 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.639 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.640 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:59:07.640 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.640 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.640 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.641 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:07.641 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.641 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.641 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.641 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.642 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.642 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:59:07.642 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.642 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.643 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.643 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:07.643 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.643 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.643 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.644 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.644 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.644 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:59:07.644 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.644 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.645 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.645 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:07.645 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.645 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.646 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.646 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.646 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.646 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:59:07.647 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.647 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.647 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.647 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:07.647 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.648 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.648 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.648 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.648 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.649 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:59:07.649 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.649 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.649 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:07.649 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:07.650 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:07.650 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:07.650 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:07.650 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:07.651 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:07.651 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:59:07.651 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:07.651 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.651 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:07.652 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:59:07.652 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:59:07.652 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:59:07.652 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:59:07.652 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T21:59:07.653 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:59:07.653 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:59:07.653 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:59:07.653 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:59:07.654 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:59:07.654 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:59:07.654 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:59:07.654 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:59:07.654 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:07.655 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:07.655 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:59:07.655 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:59:07.655 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:59:07.656 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:59:07.656 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:59:07.656 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:59:07.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b140535b0 msgr2=0x7f6b14055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b140535b0 0x7f6b14055a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6b24000f70 tx=0x7f6b240069b0).stop 2021-11-21T21:59:07.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b340652c0 msgr2=0x7f6b340642b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b340652c0 0x7f6b340642b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f6b34060090 tx=0x7f6b2c015000).stop 2021-11-21T21:59:07.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 shutdown_connections 2021-11-21T21:59:07.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b140535b0 0x7f6b14055a70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b340652c0 0x7f6b340642b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b340a8f20 0x7f6b341562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 --2- 172.21.2.116:0/3903505487 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b20004bf0 0x7f6b34143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 >> 172.21.2.116:0/3903505487 conn(0x7f6b3405bd20 msgr2=0x7f6b341441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:07.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 shutdown_connections 2021-11-21T21:59:07.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.418+0000 7f6b3b19c700 1 -- 172.21.2.116:0/3903505487 wait complete. 2021-11-21T21:59:07.661 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:07.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 Processor -- start 2021-11-21T21:59:07.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- start start 2021-11-21T21:59:07.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8de80a86b0 0x7f8de814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8de8062410 0x7f8de81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de80a6e80 con 0x7f8de80a6aa0 2021-11-21T21:59:07.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de80a8bc0 con 0x7f8de8062410 2021-11-21T21:59:07.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de814a7b0 con 0x7f8de80a86b0 2021-11-21T21:59:07.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8de8062410 0x7f8de81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8de8062410 0x7f8de81443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47942/0 (socket says 172.21.2.116:47942) 2021-11-21T21:59:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 -- 172.21.2.116:0/1443699655 learned_addr learned my addr 172.21.2.116:0/1443699655 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de659c700 1 --1- 172.21.2.116:0/1443699655 >> v1:172.21.2.116:6789/0 conn(0x7f8de80a6aa0 0x7f8de81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59898/0 (socket says 172.21.2.116:59898) 2021-11-21T21:59:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de5d9b700 1 --2- 172.21.2.116:0/1443699655 >> v2:172.21.2.117:3300/0 conn(0x7f8de80a86b0 0x7f8de814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 -- 172.21.2.116:0/1443699655 >> v2:172.21.2.117:3300/0 conn(0x7f8de80a86b0 msgr2=0x7f8de814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 --2- 172.21.2.116:0/1443699655 >> v2:172.21.2.117:3300/0 conn(0x7f8de80a86b0 0x7f8de814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 -- 172.21.2.116:0/1443699655 >> v1:172.21.2.116:6789/0 conn(0x7f8de80a6aa0 legacy=0x7f8de81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de559a700 1 -- 172.21.2.116:0/1443699655 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577833890 0 0) 0x7f8de80a6e80 con 0x7f8de80a6aa0 2021-11-21T21:59:07.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 -- 172.21.2.116:0/1443699655 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de814aa10 con 0x7f8de8062410 2021-11-21T21:59:07.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de5d9b700 1 --2- 172.21.2.116:0/1443699655 >> v2:172.21.2.117:3300/0 conn(0x7f8de80a86b0 0x7f8de814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:07.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de6d9d700 1 --2- 172.21.2.116:0/1443699655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de8062410 0x7f8de81443f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f8de00032f0 tx=0x7f8de0009290).ready entity=mon.0 client_cookie=5cdd30d807f3202 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de559a700 1 -- 172.21.2.116:0/1443699655 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de000b870 con 0x7f8de8062410 2021-11-21T21:59:07.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de559a700 1 -- 172.21.2.116:0/1443699655 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de000b9d0 con 0x7f8de8062410 2021-11-21T21:59:07.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8de559a700 1 -- 172.21.2.116:0/1443699655 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de000bb40 con 0x7f8de8062410 2021-11-21T21:59:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- 172.21.2.116:0/1443699655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de8062410 msgr2=0x7f8de81443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 --2- 172.21.2.116:0/1443699655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de8062410 0x7f8de81443f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f8de00032f0 tx=0x7f8de0009290).stop 2021-11-21T21:59:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- 172.21.2.116:0/1443699655 shutdown_connections 2021-11-21T21:59:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 --2- 172.21.2.116:0/1443699655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de8062410 0x7f8de81443f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 --2- 172.21.2.116:0/1443699655 >> v2:172.21.2.117:3300/0 conn(0x7f8de80a86b0 0x7f8de814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- 172.21.2.116:0/1443699655 >> 172.21.2.116:0/1443699655 conn(0x7f8de805bd20 msgr2=0x7f8de805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.730+0000 7f8decc54700 1 -- 172.21.2.116:0/1443699655 shutdown_connections 2021-11-21T21:59:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- 172.21.2.116:0/1443699655 wait complete. 2021-11-21T21:59:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 Processor -- start 2021-11-21T21:59:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- start start 2021-11-21T21:59:07.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8de8062410 0x7f8de80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de80a6aa0 0x7f8de813f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8de80a86b0 0x7f8de814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de813c580 con 0x7f8de80a6aa0 2021-11-21T21:59:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de814b830 con 0x7f8de80a86b0 2021-11-21T21:59:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de813ade0 con 0x7f8de8062410 2021-11-21T21:59:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de80a6aa0 0x7f8de813f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de80a6aa0 0x7f8de813f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47950/0 (socket says 172.21.2.116:47950) 2021-11-21T21:59:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 -- 172.21.2.116:0/3082715741 learned_addr learned my addr 172.21.2.116:0/3082715741 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:07.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 -- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8de8062410 msgr2=0x7f8de80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8de8062410 0x7f8de80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 -- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8de80a86b0 msgr2=0x7f8de814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8de80a86b0 0x7f8de814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 -- 172.21.2.116:0/3082715741 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de0009530 con 0x7f8de80a6aa0 2021-11-21T21:59:07.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de5d9b700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de80a6aa0 0x7f8de813f7e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f8dd8007000 tx=0x7f8dd8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8ddf7fe700 1 -- 172.21.2.116:0/3082715741 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dd8010580 con 0x7f8de80a6aa0 2021-11-21T21:59:07.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8ddf7fe700 1 -- 172.21.2.116:0/3082715741 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd80106e0 con 0x7f8de80a6aa0 2021-11-21T21:59:07.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8ddf7fe700 1 -- 172.21.2.116:0/3082715741 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dd8010a40 con 0x7f8de80a6aa0 2021-11-21T21:59:07.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de813b080 con 0x7f8de80a6aa0 2021-11-21T21:59:07.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de8139af0 con 0x7f8de80a6aa0 2021-11-21T21:59:07.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de814b9f0 con 0x7f8de80a6aa0 2021-11-21T21:59:07.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8ddf7fe700 1 -- 172.21.2.116:0/3082715741 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8dd8010ba0 con 0x7f8de80a6aa0 2021-11-21T21:59:07.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8ddf7fe700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8dcc053690 0x7f8dcc055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8ddf7fe700 1 -- 172.21.2.116:0/3082715741 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8dd806b880 con 0x7f8de80a6aa0 2021-11-21T21:59:07.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.734+0000 7f8de659c700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8dcc053690 0x7f8dcc055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.738+0000 7f8de659c700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8dcc053690 0x7f8dcc055b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f8dd0000bc0 tx=0x7f8dd0007680).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.738+0000 7f8ddf7fe700 1 -- 172.21.2.116:0/3082715741 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8dd8042870 con 0x7f8de80a6aa0 2021-11-21T21:59:07.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8de80573d0 con 0x7f8de80a6aa0 2021-11-21T21:59:07.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8ddf7fe700 1 -- 172.21.2.116:0/3082715741 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f8dd8045e90 con 0x7f8de80a6aa0 2021-11-21T21:59:07.848 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:07.849 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:57:47.223289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3159083787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3159083787},{"type":"v1","addr":"172.21.2.116:6835","nonce":3159083787}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:59:07.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8dcc053690 msgr2=0x7f8dcc055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8dcc053690 0x7f8dcc055b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f8dd0000bc0 tx=0x7f8dd0007680).stop 2021-11-21T21:59:07.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de80a6aa0 msgr2=0x7f8de813f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de80a6aa0 0x7f8de813f7e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f8dd8007000 tx=0x7f8dd8008040).stop 2021-11-21T21:59:07.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 shutdown_connections 2021-11-21T21:59:07.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8de80a86b0 0x7f8de814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8dcc053690 0x7f8dcc055b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8de80a6aa0 0x7f8de813f7e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 --2- 172.21.2.116:0/3082715741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8de8062410 0x7f8de80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 >> 172.21.2.116:0/3082715741 conn(0x7f8de805bd20 msgr2=0x7f8de8142270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:07.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 shutdown_connections 2021-11-21T21:59:07.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.846+0000 7f8decc54700 1 -- 172.21.2.116:0/3082715741 wait complete. 2021-11-21T21:59:07.852 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 23 2021-11-21T21:59:07.855 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T21:59:07.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f1175baa700 1 Processor -- start 2021-11-21T21:59:07.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f1175baa700 1 -- start start 2021-11-21T21:59:07.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f1175baa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f11700abce0 0x7f11701478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f1175baa700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1170147e80 con 0x7f11700ac600 2021-11-21T21:59:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f1175baa700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11700af5a0 con 0x7f1170144810 2021-11-21T21:59:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f1175baa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11700b1490 con 0x7f11700abce0 2021-11-21T21:59:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f116ffff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f11700ac600 0x7f11700acab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59914/0 (socket says 172.21.2.116:59914) 2021-11-21T21:59:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.902+0000 7f116ffff700 1 -- 172.21.2.116:0/446375918 learned_addr learned my addr 172.21.2.116:0/446375918 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178231606 0 0) 0x7f1170147e80 con 0x7f11700ac600 2021-11-21T21:59:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1150003260 con 0x7f11700ac600 2021-11-21T21:59:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1113703382 0 0) 0x7f1150003260 con 0x7f11700ac600 2021-11-21T21:59:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f11700abce0 msgr2=0x7f11701478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f11700abce0 0x7f11701478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:07.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 >> v1:172.21.2.117:6789/0 conn(0x7f1170144810 legacy=0x7f11700b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700b1fd0 con 0x7f11700ac600 2021-11-21T21:59:07.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f11680019e0 con 0x7f11700ac600 2021-11-21T21:59:07.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 >> v1:172.21.2.116:6789/0 conn(0x7f11700ac600 legacy=0x7f11700acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1150003800 0x7f1150003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f11700abce0 0x7f1150004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1150004bf0 0x7f115000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11700b1490 con 0x7f1150004bf0 2021-11-21T21:59:07.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500052d0 con 0x7f1150003800 2021-11-21T21:59:07.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115000bc40 con 0x7f11700abce0 2021-11-21T21:59:07.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1150004bf0 0x7f115000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f11700abce0 0x7f1150004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116f7fe700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1150003800 0x7f1150003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 -- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f11700abce0 msgr2=0x7f1150004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f11700abce0 0x7f1150004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 -- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1150003800 msgr2=0x7f1150003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:07.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1150003800 0x7f1150003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 -- 172.21.2.116:0/446375918 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150003260 con 0x7f1150004bf0 2021-11-21T21:59:07.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1150004bf0 0x7f115000b750 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f11700ac4e0 tx=0x7f1170146e80).ready entity=mon.0 client_cookie=a73647521c324c69 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f116800cc60 con 0x7f1150004bf0 2021-11-21T21:59:07.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800cdc0 con 0x7f1150004bf0 2021-11-21T21:59:07.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116e7fc700 1 -- 172.21.2.116:0/446375918 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1168005b80 con 0x7f1150004bf0 2021-11-21T21:59:07.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- 172.21.2.116:0/446375918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1150004bf0 msgr2=0x7f115000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 --2- 172.21.2.116:0/446375918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1150004bf0 0x7f115000b750 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f11700ac4e0 tx=0x7f1170146e80).stop 2021-11-21T21:59:07.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 -- 172.21.2.116:0/446375918 reap_dead start 2021-11-21T21:59:07.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- 172.21.2.116:0/446375918 shutdown_connections 2021-11-21T21:59:07.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- 172.21.2.116:0/446375918 >> 172.21.2.116:0/446375918 conn(0x7f117005bd20 msgr2=0x7f117005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:07.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- 172.21.2.116:0/446375918 shutdown_connections 2021-11-21T21:59:07.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- 172.21.2.116:0/446375918 wait complete. 2021-11-21T21:59:07.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 Processor -- start 2021-11-21T21:59:07.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- start start 2021-11-21T21:59:07.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1150004bf0 0x7f11700b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1170145ea0 0x7f1170144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1170145420 0x7f11701458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1170062b40 con 0x7f1170145ea0 2021-11-21T21:59:07.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1170062de0 con 0x7f1170145420 2021-11-21T21:59:07.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1170063080 con 0x7f1150004bf0 2021-11-21T21:59:07.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1170145ea0 0x7f1170144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1170145ea0 0x7f1170144f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:47972/0 (socket says 172.21.2.116:47972) 2021-11-21T21:59:07.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 -- 172.21.2.116:0/1256038305 learned_addr learned my addr 172.21.2.116:0/1256038305 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:07.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116ffff700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1170145420 0x7f11701458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116f7fe700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1150004bf0 0x7f11700b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 -- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1150004bf0 msgr2=0x7f11700b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1150004bf0 0x7f11700b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 -- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1170145420 msgr2=0x7f11701458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:07.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1170145420 0x7f11701458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:07.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 -- 172.21.2.116:0/1256038305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700b2c90 con 0x7f1170145ea0 2021-11-21T21:59:07.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116effd700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1170145ea0 0x7f1170144f30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f116000a000 tx=0x7f1160008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116cff9700 1 -- 172.21.2.116:0/1256038305 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1160014630 con 0x7f1170145ea0 2021-11-21T21:59:07.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116cff9700 1 -- 172.21.2.116:0/1256038305 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116000f040 con 0x7f1170145ea0 2021-11-21T21:59:07.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11700b3a60 con 0x7f1170145ea0 2021-11-21T21:59:07.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f116cff9700 1 -- 172.21.2.116:0/1256038305 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1160014980 con 0x7f1170145ea0 2021-11-21T21:59:07.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.906+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1170143200 con 0x7f1170145ea0 2021-11-21T21:59:07.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.910+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11700af060 con 0x7f1170145ea0 2021-11-21T21:59:07.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.910+0000 7f116cff9700 1 -- 172.21.2.116:0/1256038305 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f116001b060 con 0x7f1170145ea0 2021-11-21T21:59:07.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.910+0000 7f116cff9700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1154053640 0x7f1154055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:07.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.910+0000 7f116cff9700 1 -- 172.21.2.116:0/1256038305 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f116006b1c0 con 0x7f1170145ea0 2021-11-21T21:59:07.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.910+0000 7f116f7fe700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1154053640 0x7f1154055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:07.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.910+0000 7f116f7fe700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1154053640 0x7f1154055b00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f115800d000 tx=0x7f115800c040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:07.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:07.910+0000 7f116cff9700 1 -- 172.21.2.116:0/1256038305 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11600421b0 con 0x7f1170145ea0 2021-11-21T21:59:08.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.022+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f11700573d0 con 0x7f1170145ea0 2021-11-21T21:59:08.865 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T21:59:08.862+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T21:59:08.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.874+0000 7f116cff9700 1 -- 172.21.2.116:0/1256038305 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1160019090 con 0x7f1170145ea0 2021-11-21T21:59:08.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1154053640 msgr2=0x7f1154055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:08.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1154053640 0x7f1154055b00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f115800d000 tx=0x7f115800c040).stop 2021-11-21T21:59:08.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1170145ea0 msgr2=0x7f1170144f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:08.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1170145ea0 0x7f1170144f30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f116000a000 tx=0x7f1160008040).stop 2021-11-21T21:59:08.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 shutdown_connections 2021-11-21T21:59:08.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1170145420 0x7f11701458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:08.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1154053640 0x7f1154055b00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:08.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1170145ea0 0x7f1170144f30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:08.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 --2- 172.21.2.116:0/1256038305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1150004bf0 0x7f11700b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:08.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 >> 172.21.2.116:0/1256038305 conn(0x7f117005bd20 msgr2=0x7f117013fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:08.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 shutdown_connections 2021-11-21T21:59:08.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.878+0000 7f1175baa700 1 -- 172.21.2.116:0/1256038305 wait complete. 2021-11-21T21:59:08.883 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T21:59:08.886 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:08.897 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:59:08.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.934+0000 7fce162bd700 1 Processor -- start 2021-11-21T21:59:08.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.934+0000 7fce162bd700 1 -- start start 2021-11-21T21:59:08.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fce10060c50 0x7fce10140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1005c440 con 0x7fce1005c060 2021-11-21T21:59:08.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce10143630 con 0x7fce100a9500 2021-11-21T21:59:08.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce100a9c10 con 0x7fce10060c50 2021-11-21T21:59:08.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fce1005c060 0x7fce1013fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59944/0 (socket says 172.21.2.116:59944) 2021-11-21T21:59:08.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 -- 172.21.2.116:0/1139322088 learned_addr learned my addr 172.21.2.116:0/1139322088 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:08.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996366038 0 0) 0x7fce1005c440 con 0x7fce1005c060 2021-11-21T21:59:08.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf0003260 con 0x7fce1005c060 2021-11-21T21:59:08.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683549387 0 0) 0x7fcdf0003260 con 0x7fce1005c060 2021-11-21T21:59:08.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 >> v2:172.21.2.117:3300/0 conn(0x7fce10060c50 msgr2=0x7fce10140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:08.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 --2- 172.21.2.116:0/1139322088 >> v2:172.21.2.117:3300/0 conn(0x7fce10060c50 0x7fce10140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:08.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 >> v1:172.21.2.117:6789/0 conn(0x7fce100a9500 legacy=0x7fce1005fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:08.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce101438a0 con 0x7fce1005c060 2021-11-21T21:59:08.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fcdf8001680 con 0x7fce1005c060 2021-11-21T21:59:08.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 >> v1:172.21.2.116:6789/0 conn(0x7fce1005c060 legacy=0x7fce1013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:08.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdf00037d0 0x7fcdf0003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcdf0004500 0x7fcdf0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fcdf000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100a9c10 con 0x7fcdf0005070 2021-11-21T21:59:08.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf000bef0 con 0x7fcdf0004500 2021-11-21T21:59:08.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf000c0b0 con 0x7fcdf00037d0 2021-11-21T21:59:08.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fcdf000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:08.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 -- 172.21.2.116:0/1139322088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdf00037d0 msgr2=0x7fcdf0003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:08.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdf00037d0 0x7fcdf0003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:08.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 -- 172.21.2.116:0/1139322088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcdf0004500 msgr2=0x7fcdf0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:08.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcdf0004500 0x7fcdf0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:08.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 -- 172.21.2.116:0/1139322088 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce101438a0 con 0x7fcdf0005070 2021-11-21T21:59:08.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 -- 172.21.2.116:0/1139322088 reap_dead start 2021-11-21T21:59:08.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce1485a700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fcdf000b750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fce080012f0 tx=0x7fce08007d60).ready entity=mon.0 client_cookie=2f57992782b22fc7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:08.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce080014c0 con 0x7fcdf0005070 2021-11-21T21:59:08.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce08006870 con 0x7fcdf0005070 2021-11-21T21:59:08.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0effd700 1 -- 172.21.2.116:0/1139322088 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce08006bd0 con 0x7fcdf0005070 2021-11-21T21:59:08.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- 172.21.2.116:0/1139322088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 msgr2=0x7fcdf000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:08.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fcdf000b750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fce080012f0 tx=0x7fce08007d60).stop 2021-11-21T21:59:08.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- 172.21.2.116:0/1139322088 shutdown_connections 2021-11-21T21:59:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 --2- 172.21.2.116:0/1139322088 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fcdf000b750 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- 172.21.2.116:0/1139322088 >> 172.21.2.116:0/1139322088 conn(0x7fce100b6000 msgr2=0x7fce100b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- 172.21.2.116:0/1139322088 shutdown_connections 2021-11-21T21:59:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- 172.21.2.116:0/1139322088 wait complete. 2021-11-21T21:59:08.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 Processor -- start 2021-11-21T21:59:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- start start 2021-11-21T21:59:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fce100aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce100aa440 0x7fce100b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce1013fe70 0x7fce101402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce10066eb0 con 0x7fcdf0005070 2021-11-21T21:59:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce101407e0 con 0x7fce100aa440 2021-11-21T21:59:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce162bd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce100ab6b0 con 0x7fce1013fe70 2021-11-21T21:59:08.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fce100aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:08.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fce100aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48002/0 (socket says 172.21.2.116:48002) 2021-11-21T21:59:08.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 -- 172.21.2.116:0/1736634571 learned_addr learned my addr 172.21.2.116:0/1736634571 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:08.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 -- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce1013fe70 msgr2=0x7fce101402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:08.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce1013fe70 0x7fce101402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:08.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 -- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce100aa440 msgr2=0x7fce100b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:08.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce100aa440 0x7fce100b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:08.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.938+0000 7fce0ffff700 1 -- 172.21.2.116:0/1736634571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce100670a0 con 0x7fcdf0005070 2021-11-21T21:59:08.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0ffff700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fce100aba20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fce10140920 tx=0x7fcdf8001a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:08.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0d7fa700 1 -- 172.21.2.116:0/1736634571 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf8007170 con 0x7fcdf0005070 2021-11-21T21:59:08.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0d7fa700 1 -- 172.21.2.116:0/1736634571 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf80072d0 con 0x7fcdf0005070 2021-11-21T21:59:08.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0d7fa700 1 -- 172.21.2.116:0/1736634571 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf8004920 con 0x7fcdf0005070 2021-11-21T21:59:08.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce10067340 con 0x7fcdf0005070 2021-11-21T21:59:08.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce101551d0 con 0x7fcdf0005070 2021-11-21T21:59:08.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce100674d0 con 0x7fcdf0005070 2021-11-21T21:59:08.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0d7fa700 1 -- 172.21.2.116:0/1736634571 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcdf8004a80 con 0x7fcdf0005070 2021-11-21T21:59:08.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0d7fa700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcdf40536f0 0x7fcdf4055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:08.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0d7fa700 1 -- 172.21.2.116:0/1736634571 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fcdf806b760 con 0x7fcdf0005070 2021-11-21T21:59:08.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0f7fe700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcdf40536f0 0x7fcdf4055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:08.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0f7fe700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcdf40536f0 0x7fcdf4055bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fce00002f70 tx=0x7fce00006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:08.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:08.942+0000 7fce0d7fa700 1 -- 172.21.2.116:0/1736634571 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcdf8042760 con 0x7fcdf0005070 2021-11-21T21:59:09.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.054+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce10143b40 con 0x7fcdf0005070 2021-11-21T21:59:09.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.054+0000 7fce0d7fa700 1 -- 172.21.2.116:0/1736634571 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2131 (secure 0 0 0) 0x7fce10143b40 con 0x7fcdf0005070 2021-11-21T21:59:09.059 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:09.059 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:57:46.218398+0000","modified":"2021-11-21T21:59:08.866807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-11-21T21:59:09.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcdf40536f0 msgr2=0x7fcdf4055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcdf40536f0 0x7fcdf4055bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fce00002f70 tx=0x7fce00006a00).stop 2021-11-21T21:59:09.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 msgr2=0x7fce100aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fce100aba20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fce10140920 tx=0x7fcdf8001a00).stop 2021-11-21T21:59:09.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 shutdown_connections 2021-11-21T21:59:09.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fce1013fe70 0x7fce101402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fce100aa440 0x7fce100b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcdf40536f0 0x7fcdf4055bb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 --2- 172.21.2.116:0/1736634571 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdf0005070 0x7fce100aba20 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 >> 172.21.2.116:0/1736634571 conn(0x7fce100b6000 msgr2=0x7fce10141770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 shutdown_connections 2021-11-21T21:59:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.058+0000 7fce162bd700 1 -- 172.21.2.116:0/1736634571 wait complete. 2021-11-21T21:59:09.064 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 24 2021-11-21T21:59:09.067 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T21:59:09.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 Processor -- start 2021-11-21T21:59:09.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- start start 2021-11-21T21:59:09.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdf880adc10 0x7fdf880b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880b3690 0x7fdf8813a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf880b3c50 con 0x7fdf88144a10 2021-11-21T21:59:09.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf880b3ef0 con 0x7fdf880adc10 2021-11-21T21:59:09.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b4190 con 0x7fdf880b3690 2021-11-21T21:59:09.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdf880adc10 0x7fdf880b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdf880adc10 0x7fdf880b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48008/0 (socket says 172.21.2.116:48008) 2021-11-21T21:59:09.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 -- 172.21.2.116:0/1783055344 learned_addr learned my addr 172.21.2.116:0/1783055344 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:09.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 -- 172.21.2.116:0/1783055344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880b3690 msgr2=0x7fdf8813a540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:09.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 --2- 172.21.2.116:0/1783055344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880b3690 0x7fdf8813a540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 -- 172.21.2.116:0/1783055344 >> v1:172.21.2.117:6789/0 conn(0x7fdf88144a10 legacy=0x7fdf88141020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:09.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 -- 172.21.2.116:0/1783055344 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88140b80 con 0x7fdf880adc10 2021-11-21T21:59:09.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8659c700 1 --2- 172.21.2.116:0/1783055344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf880adc10 0x7fdf880b2030 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fdf8805f430 tx=0x7fdf78006eb0).ready entity=mon.0 client_cookie=aabae3fefcb42107 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:09.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8559a700 1 -- 172.21.2.116:0/1783055344 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf7800d570 con 0x7fdf880adc10 2021-11-21T21:59:09.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8559a700 1 -- 172.21.2.116:0/1783055344 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78000e90 con 0x7fdf880adc10 2021-11-21T21:59:09.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8559a700 1 -- 172.21.2.116:0/1783055344 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf7800d8c0 con 0x7fdf880adc10 2021-11-21T21:59:09.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/1783055344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf880adc10 msgr2=0x7fdf880b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/1783055344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf880adc10 0x7fdf880b2030 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fdf8805f430 tx=0x7fdf78006eb0).stop 2021-11-21T21:59:09.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/1783055344 shutdown_connections 2021-11-21T21:59:09.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/1783055344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880b3690 0x7fdf8813a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/1783055344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf880adc10 0x7fdf880b2030 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/1783055344 >> 172.21.2.116:0/1783055344 conn(0x7fdf8805bd20 msgr2=0x7fdf8805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:09.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/1783055344 shutdown_connections 2021-11-21T21:59:09.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/1783055344 wait complete. 2021-11-21T21:59:09.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 Processor -- start 2021-11-21T21:59:09.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- start start 2021-11-21T21:59:09.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880adc10 0x7fdf88062890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf880b3690 0x7fdf881102a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf88144a10 0x7fdf88156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b15a0 con 0x7fdf88144a10 2021-11-21T21:59:09.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880af9c0 con 0x7fdf880b3690 2021-11-21T21:59:09.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b0090 con 0x7fdf880adc10 2021-11-21T21:59:09.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf88144a10 0x7fdf88156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf88144a10 0x7fdf88156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48016/0 (socket says 172.21.2.116:48016) 2021-11-21T21:59:09.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 -- 172.21.2.116:0/2493365690 learned_addr learned my addr 172.21.2.116:0/2493365690 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:09.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 -- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880adc10 msgr2=0x7fdf88062890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880adc10 0x7fdf88062890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 -- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf880b3690 msgr2=0x7fdf881102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf880b3690 0x7fdf881102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 -- 172.21.2.116:0/2493365690 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7800b1e0 con 0x7fdf88144a10 2021-11-21T21:59:09.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf86d9d700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf88144a10 0x7fdf88156b00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fdf80003330 tx=0x7fdf800091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:09.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf7f7fe700 1 -- 172.21.2.116:0/2493365690 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf80001520 con 0x7fdf88144a10 2021-11-21T21:59:09.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf7f7fe700 1 -- 172.21.2.116:0/2493365690 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf80001680 con 0x7fdf88144a10 2021-11-21T21:59:09.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.118+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf88112880 con 0x7fdf88144a10 2021-11-21T21:59:09.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf7f7fe700 1 -- 172.21.2.116:0/2493365690 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf800019e0 con 0x7fdf88144a10 2021-11-21T21:59:09.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf880afbf0 con 0x7fdf88144a10 2021-11-21T21:59:09.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf880b19d0 con 0x7fdf88144a10 2021-11-21T21:59:09.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf7f7fe700 1 -- 172.21.2.116:0/2493365690 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdf80018030 con 0x7fdf88144a10 2021-11-21T21:59:09.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf7f7fe700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf6c0536a0 0x7fdf6c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf7f7fe700 1 -- 172.21.2.116:0/2493365690 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fdf8006b440 con 0x7fdf88144a10 2021-11-21T21:59:09.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf8659c700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf6c0536a0 0x7fdf6c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf8659c700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf6c0536a0 0x7fdf6c055b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fdf8805f430 tx=0x7fdf78015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:09.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.122+0000 7fdf7f7fe700 1 -- 172.21.2.116:0/2493365690 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf80042560 con 0x7fdf88144a10 2021-11-21T21:59:09.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.230+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf880afdb0 con 0x7fdf88144a10 2021-11-21T21:59:09.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf7f7fe700 1 -- 172.21.2.116:0/2493365690 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf80045b80 con 0x7fdf88144a10 2021-11-21T21:59:09.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf6c0536a0 msgr2=0x7fdf6c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf6c0536a0 0x7fdf6c055b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fdf8805f430 tx=0x7fdf78015000).stop 2021-11-21T21:59:09.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf88144a10 msgr2=0x7fdf88156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf88144a10 0x7fdf88156b00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fdf80003330 tx=0x7fdf800091a0).stop 2021-11-21T21:59:09.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 shutdown_connections 2021-11-21T21:59:09.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf6c0536a0 0x7fdf6c055b60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf88144a10 0x7fdf88156b00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf880b3690 0x7fdf881102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 --2- 172.21.2.116:0/2493365690 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf880adc10 0x7fdf88062890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 >> 172.21.2.116:0/2493365690 conn(0x7fdf8805bd20 msgr2=0x7fdf8813e910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:09.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 shutdown_connections 2021-11-21T21:59:09.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.882+0000 7fdf8c8bc700 1 -- 172.21.2.116:0/2493365690 wait complete. 2021-11-21T21:59:09.892 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T21:59:09.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 Processor -- start 2021-11-21T21:59:09.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- start start 2021-11-21T21:59:09.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcac40affc0 0x7fcac405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcac40af7e0 0x7fcac413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac4143f70 con 0x7fcac40ad780 2021-11-21T21:59:09.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac40adb60 con 0x7fcac40affc0 2021-11-21T21:59:09.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac4143160 con 0x7fcac40af7e0 2021-11-21T21:59:09.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcac40affc0 0x7fcac405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcac40affc0 0x7fcac405f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48026/0 (socket says 172.21.2.116:48026) 2021-11-21T21:59:09.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 -- 172.21.2.116:0/2982818011 learned_addr learned my addr 172.21.2.116:0/2982818011 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:09.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 -- 172.21.2.116:0/2982818011 >> v2:172.21.2.117:3300/0 conn(0x7fcac40af7e0 msgr2=0x7fcac413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:09.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 --2- 172.21.2.116:0/2982818011 >> v2:172.21.2.117:3300/0 conn(0x7fcac40af7e0 0x7fcac413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 -- 172.21.2.116:0/2982818011 >> v1:172.21.2.116:6789/0 conn(0x7fcac40ad780 legacy=0x7fcac4143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 -- 172.21.2.116:0/2982818011 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac40b0ce0 con 0x7fcac40affc0 2021-11-21T21:59:09.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac2ffd700 1 --2- 172.21.2.116:0/2982818011 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40affc0 0x7fcac405f730 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fcab4002f70 tx=0x7fcab4006eb0).ready entity=mon.0 client_cookie=e33a0baae2c359ff server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:09.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac27fc700 1 -- 172.21.2.116:0/2982818011 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab400d610 con 0x7fcac40affc0 2021-11-21T21:59:09.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac27fc700 1 -- 172.21.2.116:0/2982818011 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab4000e90 con 0x7fcac40affc0 2021-11-21T21:59:09.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcac27fc700 1 -- 172.21.2.116:0/2982818011 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab400d9a0 con 0x7fcac40affc0 2021-11-21T21:59:09.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- 172.21.2.116:0/2982818011 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40affc0 msgr2=0x7fcac405f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 --2- 172.21.2.116:0/2982818011 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40affc0 0x7fcac405f730 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fcab4002f70 tx=0x7fcab4006eb0).stop 2021-11-21T21:59:09.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- 172.21.2.116:0/2982818011 shutdown_connections 2021-11-21T21:59:09.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 --2- 172.21.2.116:0/2982818011 >> v2:172.21.2.117:3300/0 conn(0x7fcac40af7e0 0x7fcac413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:09.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 --2- 172.21.2.116:0/2982818011 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40affc0 0x7fcac405f730 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- 172.21.2.116:0/2982818011 >> 172.21.2.116:0/2982818011 conn(0x7fcac40b6000 msgr2=0x7fcac40b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:09.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- 172.21.2.116:0/2982818011 shutdown_connections 2021-11-21T21:59:09.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- 172.21.2.116:0/2982818011 wait complete. 2021-11-21T21:59:09.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 Processor -- start 2021-11-21T21:59:09.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.946+0000 7fcaca00d700 1 -- start start 2021-11-21T21:59:09.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcac40ad780 0x7fcac41356a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40af7e0 0x7fcac40ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcac40affc0 0x7fcac40ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40accc0 con 0x7fcac40af7e0 2021-11-21T21:59:09.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40aad10 con 0x7fcac40ad780 2021-11-21T21:59:09.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40abc50 con 0x7fcac40affc0 2021-11-21T21:59:09.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40af7e0 0x7fcac40ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40af7e0 0x7fcac40ab270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48030/0 (socket says 172.21.2.116:48030) 2021-11-21T21:59:09.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 -- 172.21.2.116:0/3735746505 learned_addr learned my addr 172.21.2.116:0/3735746505 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:09.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac3fff700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcac40affc0 0x7fcac40ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 -- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcac40affc0 msgr2=0x7fcac40ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcac40affc0 0x7fcac40ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 -- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcac40ad780 msgr2=0x7fcac41356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:09.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac37fe700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcac40ad780 0x7fcac41356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcac40ad780 0x7fcac41356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:09.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 -- 172.21.2.116:0/3735746505 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab400b1e0 con 0x7fcac40af7e0 2021-11-21T21:59:09.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac3fff700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcac40affc0 0x7fcac40ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac37fe700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcac40ad780 0x7fcac41356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac2ffd700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40af7e0 0x7fcac40ab270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fcab4002f70 tx=0x7fcab4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac0ff9700 1 -- 172.21.2.116:0/3735746505 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab4006b60 con 0x7fcac40af7e0 2021-11-21T21:59:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac0ff9700 1 -- 172.21.2.116:0/3735746505 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab400d290 con 0x7fcac40af7e0 2021-11-21T21:59:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac40a9e10 con 0x7fcac40af7e0 2021-11-21T21:59:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac0ff9700 1 -- 172.21.2.116:0/3735746505 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab401e460 con 0x7fcac40af7e0 2021-11-21T21:59:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac4064bd0 con 0x7fcac40af7e0 2021-11-21T21:59:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac4067090 con 0x7fcac40af7e0 2021-11-21T21:59:09.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac0ff9700 1 -- 172.21.2.116:0/3735746505 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcab4025030 con 0x7fcac40af7e0 2021-11-21T21:59:09.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac0ff9700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaa8053720 0x7fcaa8055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:09.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac0ff9700 1 -- 172.21.2.116:0/3735746505 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fcab406ccf0 con 0x7fcac40af7e0 2021-11-21T21:59:09.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac37fe700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaa8053720 0x7fcaa8055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:09.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.950+0000 7fcac37fe700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaa8053720 0x7fcaa8055be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fcaac008000 tx=0x7fcaac009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:09.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:09.954+0000 7fcac0ff9700 1 -- 172.21.2.116:0/3735746505 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcab4043e00 con 0x7fcac40af7e0 2021-11-21T21:59:10.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.038+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcac40573d0 con 0x7fcac40af7e0 2021-11-21T21:59:10.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcac0ff9700 1 -- 172.21.2.116:0/3735746505 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 185+0+0 (secure 0 0 0) 0x7fcab40230e0 con 0x7fcac40af7e0 2021-11-21T21:59:10.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaa8053720 msgr2=0x7fcaa8055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:10.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaa8053720 0x7fcaa8055be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fcaac008000 tx=0x7fcaac009040).stop 2021-11-21T21:59:10.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40af7e0 msgr2=0x7fcac40ab270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:10.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40af7e0 0x7fcac40ab270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fcab4002f70 tx=0x7fcab4015000).stop 2021-11-21T21:59:10.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 shutdown_connections 2021-11-21T21:59:10.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcac40affc0 0x7fcac40ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:10.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaa8053720 0x7fcaa8055be0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:10.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcac40af7e0 0x7fcac40ab270 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:10.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 --2- 172.21.2.116:0/3735746505 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcac40ad780 0x7fcac41356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:10.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 >> 172.21.2.116:0/3735746505 conn(0x7fcac40b6000 msgr2=0x7fcac40b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:10.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 shutdown_connections 2021-11-21T21:59:10.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.914+0000 7fcaca00d700 1 -- 172.21.2.116:0/3735746505 wait complete. 2021-11-21T21:59:10.920 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T21:59:10.923 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T21:59:10.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.974+0000 7fe97b96e700 1 Processor -- start 2021-11-21T21:59:10.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.974+0000 7fe97b96e700 1 -- start start 2021-11-21T21:59:10.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740a8670 0x7fe974147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9740b1940 con 0x7fe9740b11c0 2021-11-21T21:59:10.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9740b15a0 con 0x7fe974143e60 2021-11-21T21:59:10.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740b1b70 con 0x7fe9740a8670 2021-11-21T21:59:10.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe9740b11c0 0x7fe974147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:59996/0 (socket says 172.21.2.116:59996) 2021-11-21T21:59:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 -- 172.21.2.116:0/2448076626 learned_addr learned my addr 172.21.2.116:0/2448076626 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458152060 0 0) 0x7fe9740b1940 con 0x7fe9740b11c0 2021-11-21T21:59:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe958003260 con 0x7fe9740b11c0 2021-11-21T21:59:10.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3712383408 0 0) 0x7fe958003260 con 0x7fe9740b11c0 2021-11-21T21:59:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740a8670 msgr2=0x7fe974147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740a8670 0x7fe974147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 >> v1:172.21.2.117:6789/0 conn(0x7fe974143e60 legacy=0x7fe9741442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740b1e10 con 0x7fe9740b11c0 2021-11-21T21:59:10.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe970001660 con 0x7fe9740b11c0 2021-11-21T21:59:10.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 >> v1:172.21.2.116:6789/0 conn(0x7fe9740b11c0 legacy=0x7fe974147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:10.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe958003800 0x7fe958003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740a8670 0x7fe958004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958004bf0 0x7fe95800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740b1b70 con 0x7fe958003800 2021-11-21T21:59:10.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580052f0 con 0x7fe958004bf0 2021-11-21T21:59:10.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95800bc40 con 0x7fe9740a8670 2021-11-21T21:59:10.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe958003800 0x7fe958003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:10.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe978f09700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740a8670 0x7fe958004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:10.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 -- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740a8670 msgr2=0x7fe958004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:10.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740a8670 0x7fe958004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:10.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 -- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958004bf0 msgr2=0x7fe95800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:10.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958004bf0 0x7fe95800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:10.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 -- 172.21.2.116:0/2448076626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe958003260 con 0x7fe958003800 2021-11-21T21:59:10.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe958003800 0x7fe958003c00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fe9740b26d0 tx=0x7fe9740b1fa0).ready entity=mon.0 client_cookie=eac39f8a7733c07b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:10.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9700070d0 con 0x7fe958003800 2021-11-21T21:59:10.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe970007230 con 0x7fe958003800 2021-11-21T21:59:10.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe96bfff700 1 -- 172.21.2.116:0/2448076626 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe97000c890 con 0x7fe958003800 2021-11-21T21:59:10.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- 172.21.2.116:0/2448076626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe958003800 msgr2=0x7fe958003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:10.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 --2- 172.21.2.116:0/2448076626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe958003800 0x7fe958003c00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fe9740b26d0 tx=0x7fe9740b1fa0).stop 2021-11-21T21:59:10.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 -- 172.21.2.116:0/2448076626 reap_dead start 2021-11-21T21:59:10.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- 172.21.2.116:0/2448076626 shutdown_connections 2021-11-21T21:59:10.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- 172.21.2.116:0/2448076626 >> 172.21.2.116:0/2448076626 conn(0x7fe97405bd20 msgr2=0x7fe97405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- 172.21.2.116:0/2448076626 shutdown_connections 2021-11-21T21:59:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- 172.21.2.116:0/2448076626 wait complete. 2021-11-21T21:59:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 Processor -- start 2021-11-21T21:59:10.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- start start 2021-11-21T21:59:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958003800 0x7fe9740b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe97413e080 0x7fe974156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe97415a050 0x7fe97413d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974141430 con 0x7fe97415a050 2021-11-21T21:59:10.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974156750 con 0x7fe958003800 2021-11-21T21:59:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97b96e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe974159950 con 0x7fe97413e080 2021-11-21T21:59:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe97415a050 0x7fe97413d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe97415a050 0x7fe97413d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48052/0 (socket says 172.21.2.116:48052) 2021-11-21T21:59:10.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 -- 172.21.2.116:0/506052835 learned_addr learned my addr 172.21.2.116:0/506052835 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe978f09700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe97413e080 0x7fe974156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe97970a700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958003800 0x7fe9740b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 -- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe97413e080 msgr2=0x7fe974156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe97413e080 0x7fe974156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:10.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 -- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958003800 msgr2=0x7fe9740b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958003800 0x7fe9740b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe979f0b700 1 -- 172.21.2.116:0/506052835 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe974141190 con 0x7fe97415a050 2021-11-21T21:59:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.978+0000 7fe978f09700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe97413e080 0x7fe974156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:59:10.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe979f0b700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe97415a050 0x7fe97413d150 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fe970001370 tx=0x7fe96c007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe96a7fc700 1 -- 172.21.2.116:0/506052835 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe96c004850 con 0x7fe97415a050 2021-11-21T21:59:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe96a7fc700 1 -- 172.21.2.116:0/506052835 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c0049b0 con 0x7fe97415a050 2021-11-21T21:59:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe974159bb0 con 0x7fe97415a050 2021-11-21T21:59:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe96a7fc700 1 -- 172.21.2.116:0/506052835 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe96c004d50 con 0x7fe97415a050 2021-11-21T21:59:10.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97415a4d0 con 0x7fe97415a050 2021-11-21T21:59:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe974159760 con 0x7fe97415a050 2021-11-21T21:59:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe96a7fc700 1 -- 172.21.2.116:0/506052835 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe96c01c070 con 0x7fe97415a050 2021-11-21T21:59:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe96a7fc700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe95c0536d0 0x7fe95c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:10.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe96a7fc700 1 -- 172.21.2.116:0/506052835 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fe96c06a930 con 0x7fe97415a050 2021-11-21T21:59:11.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe97970a700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe95c0536d0 0x7fe95c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:11.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.982+0000 7fe97970a700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe95c0536d0 0x7fe95c055b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fe970019000 tx=0x7fe970015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:11.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:10.986+0000 7fe96a7fc700 1 -- 172.21.2.116:0/506052835 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe96c041bf0 con 0x7fe97415a050 2021-11-21T21:59:11.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.070+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe974159d40 con 0x7fe97415a050 2021-11-21T21:59:11.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe96a7fc700 1 -- 172.21.2.116:0/506052835 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe96c045210 con 0x7fe97415a050 2021-11-21T21:59:11.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe95c0536d0 msgr2=0x7fe95c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:11.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe95c0536d0 0x7fe95c055b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fe970019000 tx=0x7fe970015000).stop 2021-11-21T21:59:11.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe97415a050 msgr2=0x7fe97413d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:11.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe97415a050 0x7fe97413d150 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fe970001370 tx=0x7fe96c007dc0).stop 2021-11-21T21:59:11.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 shutdown_connections 2021-11-21T21:59:11.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe95c0536d0 0x7fe95c055b90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:11.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe97415a050 0x7fe97413d150 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:11.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe97413e080 0x7fe974156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:11.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 --2- 172.21.2.116:0/506052835 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe958003800 0x7fe9740b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:11.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 >> 172.21.2.116:0/506052835 conn(0x7fe97405bd20 msgr2=0x7fe974142cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:11.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 shutdown_connections 2021-11-21T21:59:11.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.922+0000 7fe97b96e700 1 -- 172.21.2.116:0/506052835 wait complete. 2021-11-21T21:59:11.930 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T21:59:11.933 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T21:59:11.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cef7b3700 1 Processor -- start 2021-11-21T21:59:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cef7b3700 1 -- start start 2021-11-21T21:59:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cef7b3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8ce80adc10 0x7f8ce80b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cef7b3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80b3690 0x7f8ce813a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:11.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cef7b3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce80b3c50 con 0x7f8ce8144a10 2021-11-21T21:59:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cef7b3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce80b3ef0 con 0x7f8ce80adc10 2021-11-21T21:59:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cef7b3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80b4190 con 0x7f8ce80b3690 2021-11-21T21:59:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8ce80adc10 0x7f8ce80b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8ce80adc10 0x7f8ce80b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48056/0 (socket says 172.21.2.116:48056) 2021-11-21T21:59:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 -- 172.21.2.116:0/305394355 learned_addr learned my addr 172.21.2.116:0/305394355 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 -- 172.21.2.116:0/305394355 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80b3690 msgr2=0x7f8ce813a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 --2- 172.21.2.116:0/305394355 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80b3690 0x7f8ce813a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 -- 172.21.2.116:0/305394355 >> v1:172.21.2.116:6789/0 conn(0x7f8ce8144a10 legacy=0x7f8ce8141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 -- 172.21.2.116:0/305394355 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8140b80 con 0x7f8ce80adc10 2021-11-21T21:59:11.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cd7fff700 1 -- 172.21.2.116:0/305394355 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945033125 0 0) 0x7f8ce80b3c50 con 0x7f8ce8144a10 2021-11-21T21:59:11.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8ced54f700 1 --2- 172.21.2.116:0/305394355 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80adc10 0x7f8ce80b2030 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8cd8002f70 tx=0x7f8cd8006eb0).ready entity=mon.0 client_cookie=a27c4fc514c391ce server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:11.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cd7fff700 1 -- 172.21.2.116:0/305394355 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cd800d5c0 con 0x7f8ce80adc10 2021-11-21T21:59:11.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cd7fff700 1 -- 172.21.2.116:0/305394355 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd8000e90 con 0x7f8ce80adc10 2021-11-21T21:59:11.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.982+0000 7f8cd7fff700 1 -- 172.21.2.116:0/305394355 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cd800d910 con 0x7f8ce80adc10 2021-11-21T21:59:11.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/305394355 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80adc10 msgr2=0x7f8ce80b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:11.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/305394355 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80adc10 0x7f8ce80b2030 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8cd8002f70 tx=0x7f8cd8006eb0).stop 2021-11-21T21:59:11.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/305394355 shutdown_connections 2021-11-21T21:59:11.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/305394355 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80b3690 0x7f8ce813a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:11.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/305394355 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80adc10 0x7f8ce80b2030 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:11.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/305394355 >> 172.21.2.116:0/305394355 conn(0x7f8ce805bd20 msgr2=0x7f8ce805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:11.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/305394355 shutdown_connections 2021-11-21T21:59:11.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/305394355 wait complete. 2021-11-21T21:59:11.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 Processor -- start 2021-11-21T21:59:11.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- start start 2021-11-21T21:59:11.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80adc10 0x7f8ce8110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:11.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80b3690 0x7f8ce8153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:11.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8ce8144a10 0x7f8ce8153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:11.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80b1600 con 0x7f8ce80b3690 2021-11-21T21:59:11.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80ae3a0 con 0x7f8ce8144a10 2021-11-21T21:59:11.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce8153a40 con 0x7f8ce80adc10 2021-11-21T21:59:11.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80b3690 0x7f8ce8153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:11.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80b3690 0x7f8ce8153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48064/0 (socket says 172.21.2.116:48064) 2021-11-21T21:59:11.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 -- 172.21.2.116:0/918789817 learned_addr learned my addr 172.21.2.116:0/918789817 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:11.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 -- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80adc10 msgr2=0x7f8ce8110ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:11.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80adc10 0x7f8ce8110ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:11.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 -- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8ce8144a10 msgr2=0x7f8ce8153550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:11.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8ce8144a10 0x7f8ce8153550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:11.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 -- 172.21.2.116:0/918789817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd800b1e0 con 0x7f8ce80b3690 2021-11-21T21:59:12.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cecd4e700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80b3690 0x7f8ce8153060 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f8ce80ae140 tx=0x7f8ce4007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:12.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cd67fc700 1 -- 172.21.2.116:0/918789817 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ce4007e60 con 0x7f8ce80b3690 2021-11-21T21:59:12.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cd67fc700 1 -- 172.21.2.116:0/918789817 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce4013450 con 0x7f8ce80b3690 2021-11-21T21:59:12.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cd67fc700 1 -- 172.21.2.116:0/918789817 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ce40137b0 con 0x7f8ce80b3690 2021-11-21T21:59:12.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce80ae5d0 con 0x7f8ce80b3690 2021-11-21T21:59:12.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce80b5e10 con 0x7f8ce80b3690 2021-11-21T21:59:12.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce8156af0 con 0x7f8ce80b3690 2021-11-21T21:59:12.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cd67fc700 1 -- 172.21.2.116:0/918789817 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8ce4010070 con 0x7f8ce80b3690 2021-11-21T21:59:12.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cd67fc700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cc80536d0 0x7f8cc8055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8cd67fc700 1 -- 172.21.2.116:0/918789817 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8ce406af50 con 0x7f8ce80b3690 2021-11-21T21:59:12.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.986+0000 7f8ced54f700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cc80536d0 0x7f8cc8055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:12.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.990+0000 7f8ced54f700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cc80536d0 0x7f8cc8055b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8cd8002f70 tx=0x7f8cd8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:12.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:11.990+0000 7f8cd67fc700 1 -- 172.21.2.116:0/918789817 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ce4068e90 con 0x7f8ce80b3690 2021-11-21T21:59:12.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.074+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8ce80573d0 con 0x7f8ce80b3690 2021-11-21T21:59:12.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cd67fc700 1 -- 172.21.2.116:0/918789817 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8ce4042280 con 0x7f8ce80b3690 2021-11-21T21:59:12.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cc80536d0 msgr2=0x7f8cc8055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cc80536d0 0x7f8cc8055b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8cd8002f70 tx=0x7f8cd8015000).stop 2021-11-21T21:59:12.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80b3690 msgr2=0x7f8ce8153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80b3690 0x7f8ce8153060 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f8ce80ae140 tx=0x7f8ce4007bf0).stop 2021-11-21T21:59:12.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 shutdown_connections 2021-11-21T21:59:12.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8ce8144a10 0x7f8ce8153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:12.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cc80536d0 0x7f8cc8055b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:12.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ce80b3690 0x7f8ce8153060 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:12.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 --2- 172.21.2.116:0/918789817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ce80adc10 0x7f8ce8110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:12.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 >> 172.21.2.116:0/918789817 conn(0x7f8ce805bd20 msgr2=0x7f8ce813dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:12.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 shutdown_connections 2021-11-21T21:59:12.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.078+0000 7f8cef7b3700 1 -- 172.21.2.116:0/918789817 wait complete. 2021-11-21T21:59:12.089 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2021-11-21T21:59:12.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f4101d2b700 1 Processor -- start 2021-11-21T21:59:12.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f4101d2b700 1 -- start start 2021-11-21T21:59:12.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f4101d2b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f40fc0a6aa0 0x7f40fc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f4101d2b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f4101d2b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc0a6e80 con 0x7f40fc1475d0 2021-11-21T21:59:12.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f4101d2b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc0a8bc0 con 0x7f40fc0a6aa0 2021-11-21T21:59:12.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f4101d2b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc14a7b0 con 0x7f40fc0a86b0 2021-11-21T21:59:12.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40faffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:12.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40faffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:51418/0 (socket says 172.21.2.116:51418) 2021-11-21T21:59:12.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40faffd700 1 -- 172.21.2.116:0/1654552994 learned_addr learned my addr 172.21.2.116:0/1654552994 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:12.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40fb7fe700 1 --2- 172.21.2.116:0/1654552994 >> v2:172.21.2.117:3300/0 conn(0x7f40fc0a6aa0 0x7f40fc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:12.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40fb7fe700 1 -- 172.21.2.116:0/1654552994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 msgr2=0x7f40fc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:12.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40fb7fe700 1 --2- 172.21.2.116:0/1654552994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc1470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:12.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40fb7fe700 1 -- 172.21.2.116:0/1654552994 >> v1:172.21.2.117:6789/0 conn(0x7f40fc1475d0 legacy=0x7f40fc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40fb7fe700 1 -- 172.21.2.116:0/1654552994 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc14aa10 con 0x7f40fc0a6aa0 2021-11-21T21:59:12.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.138+0000 7f40fb7fe700 1 --2- 172.21.2.116:0/1654552994 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 0x7f40fc14b150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f40ec002f70 tx=0x7f40ec006df0).ready entity=mon.1 client_cookie=76422cbc8c072589 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:12.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fa7fc700 1 -- 172.21.2.116:0/1654552994 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ec00d570 con 0x7f40fc0a6aa0 2021-11-21T21:59:12.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fa7fc700 1 -- 172.21.2.116:0/1654552994 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec000e90 con 0x7f40fc0a6aa0 2021-11-21T21:59:12.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fa7fc700 1 -- 172.21.2.116:0/1654552994 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ec00d8c0 con 0x7f40fc0a6aa0 2021-11-21T21:59:12.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/1654552994 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 msgr2=0x7f40fc14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 --2- 172.21.2.116:0/1654552994 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 0x7f40fc14b150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f40ec002f70 tx=0x7f40ec006df0).stop 2021-11-21T21:59:12.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/1654552994 shutdown_connections 2021-11-21T21:59:12.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 --2- 172.21.2.116:0/1654552994 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 --2- 172.21.2.116:0/1654552994 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 0x7f40fc14b150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/1654552994 >> 172.21.2.116:0/1654552994 conn(0x7f40fc05bd20 msgr2=0x7f40fc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/1654552994 shutdown_connections 2021-11-21T21:59:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/1654552994 wait complete. 2021-11-21T21:59:12.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 Processor -- start 2021-11-21T21:59:12.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- start start 2021-11-21T21:59:12.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 0x7f40fc066110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc0a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40fc1475d0 0x7f40fc063d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc156ca0 con 0x7f40fc1475d0 2021-11-21T21:59:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0b1230 con 0x7f40fc0a6aa0 2021-11-21T21:59:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc146b50 con 0x7f40fc0a86b0 2021-11-21T21:59:12.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40fc1475d0 0x7f40fc063d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:12.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40fc1475d0 0x7f40fc063d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48078/0 (socket says 172.21.2.116:48078) 2021-11-21T21:59:12.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 -- 172.21.2.116:0/3931805527 learned_addr learned my addr 172.21.2.116:0/3931805527 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:12.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 -- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 msgr2=0x7f40fc0a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc0a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 -- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 msgr2=0x7f40fc066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 0x7f40fc066110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 -- 172.21.2.116:0/3931805527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec00b180 con 0x7f40fc1475d0 2021-11-21T21:59:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40fbfff700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40fc1475d0 0x7f40fc063d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f40fc061ab0 tx=0x7f40f4007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40f8ff9700 1 -- 172.21.2.116:0/3931805527 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f400f470 con 0x7f40fc1475d0 2021-11-21T21:59:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40f8ff9700 1 -- 172.21.2.116:0/3931805527 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f400f5d0 con 0x7f40fc1475d0 2021-11-21T21:59:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f40f8ff9700 1 -- 172.21.2.116:0/3931805527 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f400f930 con 0x7f40fc1475d0 2021-11-21T21:59:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc0b0f90 con 0x7f40fc1475d0 2021-11-21T21:59:12.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc0a9670 con 0x7f40fc1475d0 2021-11-21T21:59:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.142+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc156fe0 con 0x7f40fc1475d0 2021-11-21T21:59:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.146+0000 7f40f8ff9700 1 -- 172.21.2.116:0/3931805527 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f40f4019030 con 0x7f40fc1475d0 2021-11-21T21:59:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.146+0000 7f40f8ff9700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40e00534f0 0x7f40e00559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.146+0000 7f40f8ff9700 1 -- 172.21.2.116:0/3931805527 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f40f406afb0 con 0x7f40fc1475d0 2021-11-21T21:59:12.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.146+0000 7f40fb7fe700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40e00534f0 0x7f40e00559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:12.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.146+0000 7f40fb7fe700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40e00534f0 0x7f40e00559b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f40ec002f70 tx=0x7f40ec014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:12.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.146+0000 7f40f8ff9700 1 -- 172.21.2.116:0/3931805527 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40f40422e0 con 0x7f40fc1475d0 2021-11-21T21:59:12.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.258+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f40fc0573d0 con 0x7f40fc1475d0 2021-11-21T21:59:12.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.930+0000 7f40f8ff9700 1 -- 172.21.2.116:0/3931805527 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v198) v1 ==== 152+0+0 (secure 0 0 0) 0x7f40f4045900 con 0x7f40fc1475d0 2021-11-21T21:59:12.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40e00534f0 msgr2=0x7f40e00559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40e00534f0 0x7f40e00559b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f40ec002f70 tx=0x7f40ec014000).stop 2021-11-21T21:59:12.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40fc1475d0 msgr2=0x7f40fc063d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:12.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40fc1475d0 0x7f40fc063d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f40fc061ab0 tx=0x7f40f4007f50).stop 2021-11-21T21:59:12.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 shutdown_connections 2021-11-21T21:59:12.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40e00534f0 0x7f40e00559b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:12.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40fc1475d0 0x7f40fc063d00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:12.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40fc0a86b0 0x7f40fc0a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:12.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 --2- 172.21.2.116:0/3931805527 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40fc0a6aa0 0x7f40fc066110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:12.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 >> 172.21.2.116:0/3931805527 conn(0x7f40fc05bd20 msgr2=0x7f40fc13f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:12.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 shutdown_connections 2021-11-21T21:59:12.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.934+0000 7f4101d2b700 1 -- 172.21.2.116:0/3931805527 wait complete. 2021-11-21T21:59:12.942 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-11-21T21:59:12.943 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2021-11-21T21:59:12.943 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2021-11-21T21:59:12.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.990+0000 7f5ff2aa6700 1 Processor -- start 2021-11-21T21:59:12.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.990+0000 7f5ff2aa6700 1 -- start start 2021-11-21T21:59:12.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.990+0000 7f5ff2aa6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5fec0a6aa0 0x7f5fec14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.990+0000 7f5ff2aa6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a86b0 0x7f5fec1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.990+0000 7f5ff2aa6700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec0a6e80 con 0x7f5fec1475d0 2021-11-21T21:59:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.990+0000 7f5ff2aa6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec0a8bc0 con 0x7f5fec0a6aa0 2021-11-21T21:59:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.990+0000 7f5ff2aa6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec14a7b0 con 0x7f5fec0a86b0 2021-11-21T21:59:13.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5fec0a6aa0 0x7f5fec14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:13.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5fec0a6aa0 0x7f5fec14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48084/0 (socket says 172.21.2.116:48084) 2021-11-21T21:59:13.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 -- 172.21.2.116:0/2987163435 learned_addr learned my addr 172.21.2.116:0/2987163435 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:13.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff1043700 1 --1- 172.21.2.116:0/2987163435 >> v1:172.21.2.116:6789/0 conn(0x7f5fec1475d0 0x7f5fec1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60040/0 (socket says 172.21.2.116:60040) 2021-11-21T21:59:13.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 -- 172.21.2.116:0/2987163435 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a86b0 msgr2=0x7f5fec1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:13.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 --2- 172.21.2.116:0/2987163435 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a86b0 0x7f5fec1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:13.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 -- 172.21.2.116:0/2987163435 >> v1:172.21.2.116:6789/0 conn(0x7f5fec1475d0 legacy=0x7f5fec1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:13.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 -- 172.21.2.116:0/2987163435 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec14aa10 con 0x7f5fec0a6aa0 2021-11-21T21:59:13.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff0842700 1 --2- 172.21.2.116:0/2987163435 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a6aa0 0x7f5fec14b150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5fdc002f70 tx=0x7f5fdc006eb0).ready entity=mon.0 client_cookie=d46e0f45b9a4a6a0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:13.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5feb7fe700 1 -- 172.21.2.116:0/2987163435 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fdc00d5c0 con 0x7f5fec0a6aa0 2021-11-21T21:59:13.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5feb7fe700 1 -- 172.21.2.116:0/2987163435 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc000e90 con 0x7f5fec0a6aa0 2021-11-21T21:59:13.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5feb7fe700 1 -- 172.21.2.116:0/2987163435 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fdc00d950 con 0x7f5fec0a6aa0 2021-11-21T21:59:13.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/2987163435 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a6aa0 msgr2=0x7f5fec14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:13.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/2987163435 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a6aa0 0x7f5fec14b150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5fdc002f70 tx=0x7f5fdc006eb0).stop 2021-11-21T21:59:13.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/2987163435 shutdown_connections 2021-11-21T21:59:13.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/2987163435 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a86b0 0x7f5fec1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:13.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/2987163435 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a6aa0 0x7f5fec14b150 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:13.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/2987163435 >> 172.21.2.116:0/2987163435 conn(0x7f5fec05bd20 msgr2=0x7f5fec05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:13.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/2987163435 shutdown_connections 2021-11-21T21:59:13.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/2987163435 wait complete. 2021-11-21T21:59:13.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 Processor -- start 2021-11-21T21:59:13.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- start start 2021-11-21T21:59:13.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a6aa0 0x7f5fec0a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:13.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a86b0 0x7f5fec13dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:13.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fec1475d0 0x7f5fec13e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:13.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec066e90 con 0x7f5fec0a86b0 2021-11-21T21:59:13.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec144ff0 con 0x7f5fec1475d0 2021-11-21T21:59:13.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec156320 con 0x7f5fec0a6aa0 2021-11-21T21:59:13.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a86b0 0x7f5fec13dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:13.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a86b0 0x7f5fec13dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48090/0 (socket says 172.21.2.116:48090) 2021-11-21T21:59:13.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 -- 172.21.2.116:0/946479048 learned_addr learned my addr 172.21.2.116:0/946479048 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:13.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 -- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a6aa0 msgr2=0x7f5fec0a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:13.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a6aa0 0x7f5fec0a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:13.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 -- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fec1475d0 msgr2=0x7f5fec13e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:13.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fec1475d0 0x7f5fec13e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:13.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 -- 172.21.2.116:0/946479048 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc00b1e0 con 0x7f5fec0a86b0 2021-11-21T21:59:13.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5febfff700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a86b0 0x7f5fec13dfa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f5fd4002f70 tx=0x7f5fd4006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:13.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5fe9ffb700 1 -- 172.21.2.116:0/946479048 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd4013460 con 0x7f5fec0a86b0 2021-11-21T21:59:13.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5fe9ffb700 1 -- 172.21.2.116:0/946479048 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd40135c0 con 0x7f5fec0a86b0 2021-11-21T21:59:13.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec1565c0 con 0x7f5fec0a86b0 2021-11-21T21:59:13.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5fe9ffb700 1 -- 172.21.2.116:0/946479048 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd4013920 con 0x7f5fec0a86b0 2021-11-21T21:59:13.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.994+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec153e90 con 0x7f5fec0a86b0 2021-11-21T21:59:13.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.998+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec0573d0 con 0x7f5fec0a86b0 2021-11-21T21:59:13.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.998+0000 7f5fe9ffb700 1 -- 172.21.2.116:0/946479048 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5fd401a060 con 0x7f5fec0a86b0 2021-11-21T21:59:13.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.998+0000 7f5fe9ffb700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fd00536d0 0x7f5fd0055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:13.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.998+0000 7f5fe9ffb700 1 -- 172.21.2.116:0/946479048 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5fd406b030 con 0x7f5fec0a86b0 2021-11-21T21:59:13.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.998+0000 7f5ff0842700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fd00536d0 0x7f5fd0055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:13.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.998+0000 7f5ff0842700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fd00536d0 0x7f5fd0055b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f5fdc002f70 tx=0x7f5fdc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:13.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:12.998+0000 7f5fe9ffb700 1 -- 172.21.2.116:0/946479048 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5fd4042470 con 0x7f5fec0a86b0 2021-11-21T21:59:13.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.114+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f5fec1453c0 con 0x7f5fec0a86b0 2021-11-21T21:59:13.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5fe9ffb700 1 -- 172.21.2.116:0/946479048 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v199) v1 ==== 150+0+0 (secure 0 0 0) 0x7f5fd4045a90 con 0x7f5fec0a86b0 2021-11-21T21:59:13.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fd00536d0 msgr2=0x7f5fd0055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:13.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fd00536d0 0x7f5fd0055b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f5fdc002f70 tx=0x7f5fdc015000).stop 2021-11-21T21:59:13.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a86b0 msgr2=0x7f5fec13dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:13.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a86b0 0x7f5fec13dfa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f5fd4002f70 tx=0x7f5fd4006e90).stop 2021-11-21T21:59:13.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 shutdown_connections 2021-11-21T21:59:13.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5fec1475d0 0x7f5fec13e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:13.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5fd00536d0 0x7f5fd0055b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:13.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5fec0a86b0 0x7f5fec13dfa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:13.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 --2- 172.21.2.116:0/946479048 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5fec0a6aa0 0x7f5fec0a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:13.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 >> 172.21.2.116:0/946479048 conn(0x7f5fec05bd20 msgr2=0x7f5fec149ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:13.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 shutdown_connections 2021-11-21T21:59:13.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:13.938+0000 7f5ff2aa6700 1 -- 172.21.2.116:0/946479048 wait complete. 2021-11-21T21:59:13.949 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T21:59:14.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 Processor -- start 2021-11-21T21:59:14.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 -- start start 2021-11-21T21:59:14.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a980b0120 0x7f0a98064c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 0x7f0a9813dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a98144640 con 0x7f0a98062d00 2021-11-21T21:59:14.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980630e0 con 0x7f0a980b0120 2021-11-21T21:59:14.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98064550 con 0x7f0a980af850 2021-11-21T21:59:14.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a980b0120 0x7f0a98064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a980b0120 0x7f0a98064c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48100/0 (socket says 172.21.2.116:48100) 2021-11-21T21:59:14.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 -- 172.21.2.116:0/623721784 learned_addr learned my addr 172.21.2.116:0/623721784 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:14.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 -- 172.21.2.116:0/623721784 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 msgr2=0x7f0a9813dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:14.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 --2- 172.21.2.116:0/623721784 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 0x7f0a9813dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 -- 172.21.2.116:0/623721784 >> v1:172.21.2.116:6789/0 conn(0x7f0a98062d00 legacy=0x7f0a98143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 -- 172.21.2.116:0/623721784 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980b23f0 con 0x7f0a980b0120 2021-11-21T21:59:14.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9cf3c700 1 --2- 172.21.2.116:0/623721784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a980b0120 0x7f0a98064c50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f0a94002f70 tx=0x7f0a94006eb0).ready entity=mon.0 client_cookie=97b195ee3910c996 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a8bfff700 1 -- 172.21.2.116:0/623721784 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9400d5c0 con 0x7f0a980b0120 2021-11-21T21:59:14.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a8bfff700 1 -- 172.21.2.116:0/623721784 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a94000e90 con 0x7f0a980b0120 2021-11-21T21:59:14.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a8bfff700 1 -- 172.21.2.116:0/623721784 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9400d950 con 0x7f0a980b0120 2021-11-21T21:59:14.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/623721784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a980b0120 msgr2=0x7f0a98064c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/623721784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a980b0120 0x7f0a98064c50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f0a94002f70 tx=0x7f0a94006eb0).stop 2021-11-21T21:59:14.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/623721784 shutdown_connections 2021-11-21T21:59:14.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/623721784 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 0x7f0a9813dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/623721784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a980b0120 0x7f0a98064c50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.010+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/623721784 >> 172.21.2.116:0/623721784 conn(0x7f0a9805bd20 msgr2=0x7f0a9805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/623721784 shutdown_connections 2021-11-21T21:59:14.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/623721784 wait complete. 2021-11-21T21:59:14.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 Processor -- start 2021-11-21T21:59:14.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- start start 2021-11-21T21:59:14.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a98062d00 0x7f0a980acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 0x7f0a980b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a9809df50 0x7f0a980adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9815ab40 con 0x7f0a9809df50 2021-11-21T21:59:14.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980a0260 con 0x7f0a98062d00 2021-11-21T21:59:14.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980ad5e0 con 0x7f0a980af850 2021-11-21T21:59:14.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a9809df50 0x7f0a980adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a9809df50 0x7f0a980adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48108/0 (socket says 172.21.2.116:48108) 2021-11-21T21:59:14.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 -- 172.21.2.116:0/2679463415 learned_addr learned my addr 172.21.2.116:0/2679463415 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:14.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 -- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 msgr2=0x7f0a980b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 0x7f0a980b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 -- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a98062d00 msgr2=0x7f0a980acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a98062d00 0x7f0a980acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 -- 172.21.2.116:0/2679463415 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9400b1e0 con 0x7f0a9809df50 2021-11-21T21:59:14.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9df3e700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a9809df50 0x7f0a980adf50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f0a980a03a0 tx=0x7f0a90009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a8a7fc700 1 -- 172.21.2.116:0/2679463415 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9000d580 con 0x7f0a9809df50 2021-11-21T21:59:14.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a8a7fc700 1 -- 172.21.2.116:0/2679463415 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9000d6e0 con 0x7f0a9809df50 2021-11-21T21:59:14.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a980ac5a0 con 0x7f0a9809df50 2021-11-21T21:59:14.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980acae0 con 0x7f0a9809df50 2021-11-21T21:59:14.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a8a7fc700 1 -- 172.21.2.116:0/2679463415 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9000da40 con 0x7f0a9809df50 2021-11-21T21:59:14.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a980adcf0 con 0x7f0a9809df50 2021-11-21T21:59:14.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a8a7fc700 1 -- 172.21.2.116:0/2679463415 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0a9000dba0 con 0x7f0a9809df50 2021-11-21T21:59:14.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a8a7fc700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a800536a0 0x7f0a80055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a8a7fc700 1 -- 172.21.2.116:0/2679463415 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0a9006b9e0 con 0x7f0a9809df50 2021-11-21T21:59:14.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.014+0000 7f0a9d73d700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a800536a0 0x7f0a80055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.018+0000 7f0a9d73d700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a800536a0 0x7f0a80055b60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0a980b4e00 tx=0x7f0a8c007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.018+0000 7f0a8a7fc700 1 -- 172.21.2.116:0/2679463415 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a90042e20 con 0x7f0a9809df50 2021-11-21T21:59:14.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.102+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0a980573d0 con 0x7f0a9809df50 2021-11-21T21:59:14.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.106+0000 7f0a8a7fc700 1 -- 172.21.2.116:0/2679463415 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0a90046440 con 0x7f0a9809df50 2021-11-21T21:59:14.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.106+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a800536a0 msgr2=0x7f0a80055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.106+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a800536a0 0x7f0a80055b60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0a980b4e00 tx=0x7f0a8c007810).stop 2021-11-21T21:59:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.106+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a9809df50 msgr2=0x7f0a980adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.106+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a9809df50 0x7f0a980adf50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f0a980a03a0 tx=0x7f0a90009220).stop 2021-11-21T21:59:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 shutdown_connections 2021-11-21T21:59:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a800536a0 0x7f0a80055b60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a9809df50 0x7f0a980adf50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a980af850 0x7f0a980b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 --2- 172.21.2.116:0/2679463415 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a98062d00 0x7f0a980acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 >> 172.21.2.116:0/2679463415 conn(0x7f0a9805bd20 msgr2=0x7f0a9813df20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 shutdown_connections 2021-11-21T21:59:14.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.110+0000 7f0a9f9a1700 1 -- 172.21.2.116:0/2679463415 wait complete. 2021-11-21T21:59:14.118 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:59:14.118 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:14.127 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:14.143 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:14.157 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:14.170 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:59:14.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:14.171 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:14.172 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:14.205 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T21:59:14.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:14.206 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T21:59:14.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:14.207 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:14.208 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:59:14.217 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:59:14.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:14.219 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:59:14.219 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:59:14.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc721069700 1 Processor -- start 2021-11-21T21:59:14.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc721069700 1 -- start start 2021-11-21T21:59:14.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc721069700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc71c062410 0x7fc71c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc721069700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0a6e80 con 0x7fc71c0a86b0 2021-11-21T21:59:14.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc721069700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c14a7b0 con 0x7fc71c0a6aa0 2021-11-21T21:59:14.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc721069700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71c0a8bc0 con 0x7fc71c062410 2021-11-21T21:59:14.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc71a59c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc71c0a86b0 0x7fc71c1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60070/0 (socket says 172.21.2.116:60070) 2021-11-21T21:59:14.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc71a59c700 1 -- 172.21.2.116:0/3312147983 learned_addr learned my addr 172.21.2.116:0/3312147983 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:14.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.270+0000 7fc71b59e700 1 --2- 172.21.2.116:0/3312147983 >> v2:172.21.2.116:3300/0 conn(0x7fc71c062410 0x7fc71c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc719d9b700 1 -- 172.21.2.116:0/3312147983 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292777137 0 0) 0x7fc71c0a6e80 con 0x7fc71c0a86b0 2021-11-21T21:59:14.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc719d9b700 1 -- 172.21.2.116:0/3312147983 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6fc003260 con 0x7fc71c0a86b0 2021-11-21T21:59:14.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 -- 172.21.2.116:0/3312147983 >> v1:172.21.2.117:6789/0 conn(0x7fc71c0a6aa0 legacy=0x7fc71c1470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:14.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 -- 172.21.2.116:0/3312147983 >> v1:172.21.2.116:6789/0 conn(0x7fc71c0a86b0 legacy=0x7fc71c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 -- 172.21.2.116:0/3312147983 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c14aa10 con 0x7fc71c062410 2021-11-21T21:59:14.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 --2- 172.21.2.116:0/3312147983 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c062410 0x7fc71c144b00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fc71c061b50 tx=0x7fc7140091d0).ready entity=mon.0 client_cookie=f6effee230d16468 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc719d9b700 1 -- 172.21.2.116:0/3312147983 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc71400b680 con 0x7fc71c062410 2021-11-21T21:59:14.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc719d9b700 1 -- 172.21.2.116:0/3312147983 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71400b7e0 con 0x7fc71c062410 2021-11-21T21:59:14.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc719d9b700 1 -- 172.21.2.116:0/3312147983 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc71400b950 con 0x7fc71c062410 2021-11-21T21:59:14.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- 172.21.2.116:0/3312147983 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c062410 msgr2=0x7fc71c144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 --2- 172.21.2.116:0/3312147983 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c062410 0x7fc71c144b00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fc71c061b50 tx=0x7fc7140091d0).stop 2021-11-21T21:59:14.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- 172.21.2.116:0/3312147983 shutdown_connections 2021-11-21T21:59:14.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 --2- 172.21.2.116:0/3312147983 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c062410 0x7fc71c144b00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- 172.21.2.116:0/3312147983 >> 172.21.2.116:0/3312147983 conn(0x7fc71c05bd20 msgr2=0x7fc71c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- 172.21.2.116:0/3312147983 shutdown_connections 2021-11-21T21:59:14.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- 172.21.2.116:0/3312147983 wait complete. 2021-11-21T21:59:14.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 Processor -- start 2021-11-21T21:59:14.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- start start 2021-11-21T21:59:14.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc71c062410 0x7fc71c064050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc71c0a6aa0 0x7fc71c064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c0a86b0 0x7fc71c066bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c064a30 con 0x7fc71c0a86b0 2021-11-21T21:59:14.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0a9540 con 0x7fc71c0a6aa0 2021-11-21T21:59:14.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c156e50 con 0x7fc71c062410 2021-11-21T21:59:14.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c0a86b0 0x7fc71c066bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c0a86b0 0x7fc71c066bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48122/0 (socket says 172.21.2.116:48122) 2021-11-21T21:59:14.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 -- 172.21.2.116:0/3356747116 learned_addr learned my addr 172.21.2.116:0/3356747116 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:14.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 -- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc71c062410 msgr2=0x7fc71c064050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc71c062410 0x7fc71c064050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 -- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc71c0a6aa0 msgr2=0x7fc71c064540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc71c0a6aa0 0x7fc71c064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 -- 172.21.2.116:0/3356747116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714009470 con 0x7fc71c0a86b0 2021-11-21T21:59:14.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc71b59e700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c0a86b0 0x7fc71c066bd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fc71c061b50 tx=0x7fc714016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc713fff700 1 -- 172.21.2.116:0/3356747116 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc714021440 con 0x7fc71c0a86b0 2021-11-21T21:59:14.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc713fff700 1 -- 172.21.2.116:0/3356747116 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7140215a0 con 0x7fc71c0a86b0 2021-11-21T21:59:14.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c1570f0 con 0x7fc71c0a86b0 2021-11-21T21:59:14.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c0a8d20 con 0x7fc71c0a86b0 2021-11-21T21:59:14.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.274+0000 7fc713fff700 1 -- 172.21.2.116:0/3356747116 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc714021900 con 0x7fc71c0a86b0 2021-11-21T21:59:14.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.278+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f4003410 con 0x7fc71c0a86b0 2021-11-21T21:59:14.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.278+0000 7fc713fff700 1 -- 172.21.2.116:0/3356747116 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc714027020 con 0x7fc71c0a86b0 2021-11-21T21:59:14.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.278+0000 7fc713fff700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7000535b0 0x7fc700055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.278+0000 7fc713fff700 1 -- 172.21.2.116:0/3356747116 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc71406ca00 con 0x7fc71c0a86b0 2021-11-21T21:59:14.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.278+0000 7fc71ad9d700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7000535b0 0x7fc700055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.278+0000 7fc71ad9d700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7000535b0 0x7fc700055a70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fc704002f90 tx=0x7fc7040069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.278+0000 7fc713fff700 1 -- 172.21.2.116:0/3356747116 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc714043e40 con 0x7fc71c0a86b0 2021-11-21T21:59:14.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc6f40036c0 con 0x7fc71c0a86b0 2021-11-21T21:59:14.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc713fff700 1 -- 172.21.2.116:0/3356747116 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v58) v1 ==== 81+0+28828 (secure 0 0 0) 0x7fc714015020 con 0x7fc71c0a86b0 2021-11-21T21:59:14.367 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:14.367 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:59:14.368 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 58, 2021-11-21T21:59:14.368 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:59:14.369 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:59:14.369 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:59:11.912428+0000", 2021-11-21T21:59:14.370 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:14.370 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:14.370 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:59:14.371 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:59:14.371 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:59:14.373 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:59:14.373 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:59:14.373 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:59:14.373 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:59:14.374 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:14.374 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:59:14.374 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:59:14.374 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:59:14.375 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:59:14.375 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:59:14.375 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 10, 2021-11-21T21:59:14.375 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:59:14.376 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:59:14.376 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:59:14.376 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:59:14.376 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:59:14.377 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.377 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:59:14.377 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:59:14.377 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:59:14.378 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:14.378 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:14.378 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:14.378 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:14.378 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:14.379 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:14.379 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:14.379 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:14.379 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:59:14.380 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:59:14.380 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:59:14.380 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:59:14.380 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:59:14.380 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:14.381 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:14.381 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:14.381 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:14.381 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:14.382 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:14.382 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:14.382 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.382 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:59:14.382 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:14.383 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:14.383 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:14.383 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:14.383 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:14.383 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:14.384 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:14.384 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:14.384 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:14.384 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:14.384 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:14.385 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:14.385 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:14.385 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:14.385 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:14.386 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:14.386 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:14.386 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:14.386 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:14.387 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:14.387 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:14.387 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:14.387 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:14.388 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:14.388 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:14.388 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:14.388 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.389 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:14.389 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:14.389 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:14.389 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:14.390 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:14.390 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:14.390 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:14.390 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:14.391 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:14.391 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:14.391 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:14.391 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:59:14.392 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:59:14.392 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.392 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:14.392 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:59:14.392 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.393 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.393 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.393 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:59:14.393 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:59:14.393 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:59:14.394 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:14.394 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:14.394 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:14.394 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:14.395 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:14.395 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:14.395 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:14.395 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:14.396 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:14.396 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:14.396 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:14.396 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:14.397 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:14.397 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:14.397 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:14.397 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:14.398 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:14.398 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:14.398 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:14.398 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:14.399 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.399 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:59:14.399 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:14.399 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:14.400 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:14.400 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:14.400 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:14.400 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:14.400 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:14.401 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:14.401 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:14.401 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:14.401 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:14.402 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:14.402 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:14.402 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:14.402 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:14.402 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:14.403 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:14.403 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:14.403 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:14.403 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:14.403 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:14.404 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:14.404 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:14.404 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:14.404 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:14.405 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:14.405 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.405 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:14.405 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:14.406 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:14.406 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:14.406 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:14.406 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:14.407 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:14.407 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:14.407 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:14.407 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:14.408 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:14.408 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:59:14.408 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:14.408 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:59:14.409 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.409 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.409 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:14.409 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:59:14.409 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.410 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:14.410 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:59:14.410 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.410 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.411 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.411 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.411 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.411 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.412 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.412 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:14.412 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.412 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.412 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.413 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.413 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.413 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.413 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:59:14.414 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.414 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.414 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.414 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.415 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:59:14.415 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.415 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.415 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.416 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.416 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.416 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.416 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.416 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.417 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:59:14.417 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.417 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.417 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.418 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.418 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:59:14.418 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.418 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.418 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.419 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.419 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.419 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.419 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.420 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:59:14.420 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.420 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.420 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.421 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.421 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:59:14.421 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.421 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.421 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.422 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.422 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.422 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.422 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.423 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.423 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:59:14.423 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.423 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.423 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.424 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.424 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:59:14.424 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:14.424 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.425 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.425 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.425 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:59:14.425 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:59:14.425 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:59:14.426 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:59:14.426 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.426 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.426 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.427 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.427 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.427 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:14.427 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:59:14.428 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.428 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.428 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.428 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.429 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.429 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.429 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.429 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:14.430 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.430 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.430 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.430 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.430 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.431 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.431 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:59:14.431 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.431 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.431 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.432 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:59:14.432 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.432 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.433 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.433 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.433 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.433 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.434 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.434 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:59:14.434 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.434 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.435 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.435 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.435 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:59:14.435 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.435 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.436 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.436 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.436 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.436 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.437 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.437 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.437 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:59:14.437 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.437 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.438 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.438 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.438 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:59:14.438 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.439 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.439 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.439 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.440 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.440 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.440 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.440 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.440 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:59:14.441 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.441 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.441 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.441 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.442 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:59:14.442 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:14.442 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.443 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.443 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.443 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:59:14.443 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:59:14.444 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:59:14.444 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:59:14.444 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.444 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.445 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.445 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.445 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.446 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.446 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:14.446 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:59:14.447 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.447 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.447 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.448 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.448 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.448 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.448 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.449 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:14.449 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.449 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.450 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.450 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.450 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.451 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.451 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:59:14.451 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.452 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.452 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.453 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:59:14.453 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.453 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.454 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.454 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.454 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.454 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.455 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.455 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.455 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:59:14.455 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.456 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.456 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.456 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.456 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:59:14.457 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.457 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.457 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.457 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.458 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.458 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.458 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.458 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.459 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:59:14.459 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.459 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.460 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.460 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.460 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:59:14.460 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.461 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.461 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.461 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.461 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.461 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.462 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.462 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:59:14.462 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.463 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.463 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.463 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.463 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:59:14.463 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:14.464 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.464 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.464 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.464 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:59:14.465 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:59:14.465 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:59:14.465 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:59:14.465 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.466 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.466 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.466 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.466 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.466 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.467 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:14.467 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:59:14.468 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.468 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.468 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.468 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.468 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.469 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.469 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.469 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 52, 2021-11-21T21:59:14.469 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.470 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.470 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.470 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.470 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.471 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.471 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:59:14.471 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.471 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.472 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.472 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.472 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:59:14.472 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.472 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.473 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.473 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.473 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.473 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.474 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.474 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:59:14.474 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.475 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.475 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.475 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.475 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:59:14.475 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.476 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.476 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.476 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.476 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.477 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.477 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.477 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:59:14.477 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.478 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.478 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.478 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:59:14.478 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.479 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.479 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.479 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.480 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.480 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.480 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.480 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:59:14.480 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.481 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.481 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.481 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.481 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:59:14.482 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:14.482 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.482 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.482 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.482 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:59:14.483 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:59:14.483 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:59:14.483 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:59:14.483 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.484 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.484 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.484 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.484 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.485 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.485 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:14.485 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:59:14.485 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.485 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.486 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.486 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.486 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.486 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.487 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.487 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:14.487 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.487 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.487 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.488 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.488 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.488 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.488 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:59:14.489 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.489 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.489 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.489 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.489 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:59:14.490 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.490 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.490 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.490 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.491 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.491 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.491 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.491 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.491 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:59:14.492 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.492 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.492 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.492 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.493 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:59:14.493 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.493 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.493 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.493 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.494 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.494 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.494 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.494 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.495 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:59:14.495 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.495 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.495 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.495 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.496 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:59:14.496 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.496 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.496 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.497 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.497 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.497 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.497 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.498 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.498 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:59:14.498 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.498 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.498 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.499 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:59:14.499 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:14.499 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.500 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.500 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.500 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:59:14.500 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:59:14.500 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:59:14.501 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:59:14.501 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.501 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.501 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.502 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.502 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.502 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.502 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:14.502 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:59:14.503 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.503 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.503 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.503 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.503 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.504 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.504 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.504 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:14.504 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.505 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.505 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.505 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.505 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.505 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.506 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:59:14.506 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.506 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.506 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.507 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.507 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:59:14.507 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.507 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.507 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.508 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.508 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.508 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.508 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.509 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:59:14.509 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.509 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.509 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.510 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:59:14.510 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.510 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.511 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.511 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.511 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.511 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.511 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.512 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.512 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:59:14.512 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.512 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.513 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.513 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.513 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:59:14.513 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.513 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.514 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.514 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.514 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.514 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.515 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.515 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.515 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:59:14.515 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.515 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.516 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.516 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.516 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:59:14.516 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:14.517 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.517 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.517 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.517 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:59:14.518 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:59:14.518 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:59:14.518 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:59:14.518 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.519 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.519 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.519 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.520 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.520 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.520 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:14.520 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:59:14.520 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.521 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.521 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.521 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.521 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.522 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.522 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.522 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 53, 2021-11-21T21:59:14.522 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.522 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.523 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.523 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.523 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.524 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.524 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:59:14.524 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.524 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.524 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.525 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.525 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:59:14.525 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.525 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.525 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.526 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.526 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.526 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.526 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.527 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:59:14.527 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.527 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.528 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:59:14.528 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.528 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.529 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.529 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.529 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.529 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.529 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.530 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.530 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:59:14.530 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.530 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.530 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.531 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.531 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:59:14.531 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.531 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.532 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.532 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.532 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.532 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.533 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.533 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.533 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:59:14.533 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.533 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.534 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.534 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.534 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:59:14.534 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:14.535 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.535 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.535 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.535 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:59:14.536 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:59:14.536 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:59:14.536 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:59:14.536 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.537 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.537 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.537 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.537 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.537 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.538 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:14.538 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:59:14.538 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:14.539 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:14.539 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:14.539 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:14.539 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:14.539 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:14.540 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:14.540 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 52, 2021-11-21T21:59:14.540 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:14.540 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:14.540 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:14.541 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.541 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.541 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.541 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:59:14.542 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.542 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.542 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.542 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.542 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:59:14.543 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.543 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.543 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.543 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.544 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:14.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.544 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.544 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:59:14.545 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.545 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.546 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.546 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:59:14.546 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.546 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.547 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.547 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.547 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:14.547 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.547 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.548 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.548 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:59:14.548 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.548 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.548 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.549 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.549 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:59:14.549 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.549 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.550 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.550 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.550 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:14.550 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:14.551 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.551 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:14.551 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:59:14.551 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.551 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.552 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.552 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:14.552 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:59:14.552 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:14.553 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.553 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.553 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.553 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:59:14.553 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:59:14.554 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:59:14.554 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:59:14.554 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:14.554 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:14.555 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:14.555 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:14.555 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.555 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:14.556 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:59:14.556 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.556 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:14.556 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.556 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.557 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.557 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.557 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.557 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:59:14.557 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.558 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.558 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.558 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:14.558 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.559 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.559 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.559 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.559 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.559 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:59:14.560 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.560 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.560 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.560 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:14.561 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.561 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.561 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.561 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.561 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.562 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:59:14.562 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.562 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.562 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.563 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:14.563 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.563 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.563 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.563 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.564 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.564 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:59:14.564 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.564 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.565 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.565 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:14.565 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.565 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.565 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.566 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.566 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.566 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:59:14.566 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.567 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.567 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:14.567 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.567 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.568 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.568 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.568 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.568 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:59:14.569 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.569 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.569 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.569 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:14.569 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.570 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.570 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.570 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.570 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.571 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:59:14.571 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.571 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.571 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:14.571 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:14.572 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:14.572 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:14.572 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:14.572 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:14.573 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:14.573 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:59:14.573 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:14.573 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.573 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:14.574 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:59:14.574 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:59:14.574 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:59:14.574 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:59:14.575 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T21:59:14.575 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/3159083787": "2021-11-22T21:59:08.027170+0000", 2021-11-21T21:59:14.575 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/3159083787": "2021-11-22T21:59:08.027170+0000" 2021-11-21T21:59:14.575 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.575 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:59:14.576 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:59:14.576 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:59:14.576 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:59:14.576 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:59:14.576 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:59:14.577 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:59:14.577 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:59:14.577 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:14.577 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:14.578 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:59:14.578 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:59:14.578 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:59:14.578 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:59:14.578 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:59:14.579 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:59:14.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7000535b0 msgr2=0x7fc700055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7000535b0 0x7fc700055a70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fc704002f90 tx=0x7fc7040069b0).stop 2021-11-21T21:59:14.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c0a86b0 msgr2=0x7fc71c066bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c0a86b0 0x7fc71c066bd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fc71c061b50 tx=0x7fc714016000).stop 2021-11-21T21:59:14.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 shutdown_connections 2021-11-21T21:59:14.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7000535b0 0x7fc700055a70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc71c0a86b0 0x7fc71c066bd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc71c0a6aa0 0x7fc71c064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 --2- 172.21.2.116:0/3356747116 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc71c062410 0x7fc71c064050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 >> 172.21.2.116:0/3356747116 conn(0x7fc71c05bd20 msgr2=0x7fc71c149020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 shutdown_connections 2021-11-21T21:59:14.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.362+0000 7fc721069700 1 -- 172.21.2.116:0/3356747116 wait complete. 2021-11-21T21:59:14.584 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:14.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f777824a700 1 Processor -- start 2021-11-21T21:59:14.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f777824a700 1 -- start start 2021-11-21T21:59:14.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f777824a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f77700a6aa0 0x7f777014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f777824a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7770062410 0x7f77701443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f777824a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77700a6e80 con 0x7f77700a86b0 2021-11-21T21:59:14.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f777824a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77700a8bc0 con 0x7f7770062410 2021-11-21T21:59:14.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f777824a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777014a7b0 con 0x7f77700a6aa0 2021-11-21T21:59:14.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f77767e7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7770062410 0x7f77701443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f77757e5700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f77700a86b0 0x7f77701470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60082/0 (socket says 172.21.2.116:60082) 2021-11-21T21:59:14.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f77757e5700 1 -- 172.21.2.116:0/1560418933 learned_addr learned my addr 172.21.2.116:0/1560418933 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:14.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f77767e7700 1 -- 172.21.2.116:0/1560418933 >> v2:172.21.2.117:3300/0 conn(0x7f77700a6aa0 msgr2=0x7f777014b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:14.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77767e7700 1 --2- 172.21.2.116:0/1560418933 >> v2:172.21.2.117:3300/0 conn(0x7f77700a6aa0 0x7f777014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.694+0000 7f7774fe4700 1 -- 172.21.2.116:0/1560418933 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402633828 0 0) 0x7f77700a6e80 con 0x7f77700a86b0 2021-11-21T21:59:14.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77767e7700 1 -- 172.21.2.116:0/1560418933 >> v1:172.21.2.116:6789/0 conn(0x7f77700a86b0 legacy=0x7f77701470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77767e7700 1 -- 172.21.2.116:0/1560418933 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777014aa10 con 0x7f7770062410 2021-11-21T21:59:14.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77767e7700 1 --2- 172.21.2.116:0/1560418933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7770062410 0x7f77701443f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f77680032f0 tx=0x7f77680091d0).ready entity=mon.0 client_cookie=f71409ae2cc2e785 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f7774fe4700 1 -- 172.21.2.116:0/1560418933 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7768009ea0 con 0x7f7770062410 2021-11-21T21:59:14.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f7774fe4700 1 -- 172.21.2.116:0/1560418933 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776800b870 con 0x7f7770062410 2021-11-21T21:59:14.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f7774fe4700 1 -- 172.21.2.116:0/1560418933 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f776800b9e0 con 0x7f7770062410 2021-11-21T21:59:14.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- 172.21.2.116:0/1560418933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7770062410 msgr2=0x7f77701443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 --2- 172.21.2.116:0/1560418933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7770062410 0x7f77701443f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f77680032f0 tx=0x7f77680091d0).stop 2021-11-21T21:59:14.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- 172.21.2.116:0/1560418933 shutdown_connections 2021-11-21T21:59:14.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 --2- 172.21.2.116:0/1560418933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7770062410 0x7f77701443f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 --2- 172.21.2.116:0/1560418933 >> v2:172.21.2.117:3300/0 conn(0x7f77700a6aa0 0x7f777014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- 172.21.2.116:0/1560418933 >> 172.21.2.116:0/1560418933 conn(0x7f777005bd20 msgr2=0x7f777005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- 172.21.2.116:0/1560418933 shutdown_connections 2021-11-21T21:59:14.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- 172.21.2.116:0/1560418933 wait complete. 2021-11-21T21:59:14.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 Processor -- start 2021-11-21T21:59:14.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- start start 2021-11-21T21:59:14.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770062410 0x7f7770141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77700a6aa0 0x7f777015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77700a86b0 0x7f7770156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777013b700 con 0x7f77700a6aa0 2021-11-21T21:59:14.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777013b960 con 0x7f77700a86b0 2021-11-21T21:59:14.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777013bc00 con 0x7f7770062410 2021-11-21T21:59:14.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77700a6aa0 0x7f777015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77700a6aa0 0x7f777015a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48134/0 (socket says 172.21.2.116:48134) 2021-11-21T21:59:14.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 -- 172.21.2.116:0/107847613 learned_addr learned my addr 172.21.2.116:0/107847613 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:14.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 -- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770062410 msgr2=0x7f7770141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770062410 0x7f7770141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 -- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77700a86b0 msgr2=0x7f7770156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77700a86b0 0x7f7770156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 -- 172.21.2.116:0/107847613 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7768009470 con 0x7f77700a6aa0 2021-11-21T21:59:14.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f77757e5700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77700a6aa0 0x7f777015a220 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f7764000bc0 tx=0x7f7764007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f7762ffd700 1 -- 172.21.2.116:0/107847613 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7764001d50 con 0x7f77700a6aa0 2021-11-21T21:59:14.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f7762ffd700 1 -- 172.21.2.116:0/107847613 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77640072e0 con 0x7f77700a6aa0 2021-11-21T21:59:14.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f7762ffd700 1 -- 172.21.2.116:0/107847613 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77640057e0 con 0x7f77700a6aa0 2021-11-21T21:59:14.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777013bea0 con 0x7f77700a6aa0 2021-11-21T21:59:14.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.698+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777013ad10 con 0x7f77700a6aa0 2021-11-21T21:59:14.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.702+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777013c030 con 0x7f77700a6aa0 2021-11-21T21:59:14.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.702+0000 7f7762ffd700 1 -- 172.21.2.116:0/107847613 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7764010070 con 0x7f77700a6aa0 2021-11-21T21:59:14.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.702+0000 7f7762ffd700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500536a0 0x7f7750055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.702+0000 7f7762ffd700 1 -- 172.21.2.116:0/107847613 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f776406ad90 con 0x7f77700a6aa0 2021-11-21T21:59:14.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.702+0000 7f7775fe6700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500536a0 0x7f7750055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.702+0000 7f7775fe6700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500536a0 0x7f7750055b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f776c002f70 tx=0x7f776c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.702+0000 7f7762ffd700 1 -- 172.21.2.116:0/107847613 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7764042130 con 0x7f77700a6aa0 2021-11-21T21:59:14.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77700ac030 con 0x7f77700a6aa0 2021-11-21T21:59:14.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f7762ffd700 1 -- 172.21.2.116:0/107847613 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f7764045750 con 0x7f77700a6aa0 2021-11-21T21:59:14.817 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:14.817 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/833609720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":833609720},{"type":"v1","addr":"172.21.2.116:6835","nonce":833609720}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25}],"filesystems":[]} 2021-11-21T21:59:14.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500536a0 msgr2=0x7f7750055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500536a0 0x7f7750055b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f776c002f70 tx=0x7f776c006a80).stop 2021-11-21T21:59:14.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77700a6aa0 msgr2=0x7f777015a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77700a6aa0 0x7f777015a220 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f7764000bc0 tx=0x7f7764007c60).stop 2021-11-21T21:59:14.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 shutdown_connections 2021-11-21T21:59:14.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f77700a86b0 0x7f7770156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f77500536a0 0x7f7750055b60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f77700a6aa0 0x7f777015a220 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 --2- 172.21.2.116:0/107847613 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7770062410 0x7f7770141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 >> 172.21.2.116:0/107847613 conn(0x7f777005bd20 msgr2=0x7f7770148d60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 shutdown_connections 2021-11-21T21:59:14.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.814+0000 7f777824a700 1 -- 172.21.2.116:0/107847613 wait complete. 2021-11-21T21:59:14.821 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 25 2021-11-21T21:59:14.824 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:14.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.870+0000 7fb441674700 1 Processor -- start 2021-11-21T21:59:14.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.870+0000 7fb441674700 1 -- start start 2021-11-21T21:59:14.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.870+0000 7fb441674700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb43c0a6aa0 0x7fb43c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.870+0000 7fb441674700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c0a6e80 con 0x7fb43c062410 2021-11-21T21:59:14.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.870+0000 7fb441674700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c14a7b0 con 0x7fb43c0a86b0 2021-11-21T21:59:14.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.870+0000 7fb441674700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43c0a8bc0 con 0x7fb43c0a6aa0 2021-11-21T21:59:14.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43b7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb43c062410 0x7fb43c1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60096/0 (socket says 172.21.2.116:60096) 2021-11-21T21:59:14.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43b7fe700 1 -- 172.21.2.116:0/1683549717 learned_addr learned my addr 172.21.2.116:0/1683549717 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:14.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883293442 0 0) 0x7fb43c0a6e80 con 0x7fb43c062410 2021-11-21T21:59:14.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb41c003260 con 0x7fb43c062410 2021-11-21T21:59:14.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046556808 0 0) 0x7fb41c003260 con 0x7fb43c062410 2021-11-21T21:59:14.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 >> v2:172.21.2.117:3300/0 conn(0x7fb43c0a6aa0 msgr2=0x7fb43c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:14.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 --2- 172.21.2.116:0/1683549717 >> v2:172.21.2.117:3300/0 conn(0x7fb43c0a6aa0 0x7fb43c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 >> v1:172.21.2.117:6789/0 conn(0x7fb43c0a86b0 legacy=0x7fb43c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:14.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c14aa10 con 0x7fb43c062410 2021-11-21T21:59:14.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb42c001a00 con 0x7fb43c062410 2021-11-21T21:59:14.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 >> v1:172.21.2.116:6789/0 conn(0x7fb43c062410 legacy=0x7fb43c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb41c003820 0x7fb41c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb41c004500 0x7fb41c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb41c005050 0x7fb41c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0a8bc0 con 0x7fb41c003820 2021-11-21T21:59:14.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c00bf40 con 0x7fb41c005050 2021-11-21T21:59:14.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c00c100 con 0x7fb41c004500 2021-11-21T21:59:14.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43affd700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb41c003820 0x7fb41c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43affd700 1 -- 172.21.2.116:0/1683549717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb41c004500 msgr2=0x7fb41c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:14.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43affd700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb41c004500 0x7fb41c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43affd700 1 -- 172.21.2.116:0/1683549717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb41c005050 msgr2=0x7fb41c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:14.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43affd700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb41c005050 0x7fb41c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43b7fe700 1 -- 172.21.2.116:0/1683549717 reap_dead start 2021-11-21T21:59:14.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43affd700 1 -- 172.21.2.116:0/1683549717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c003260 con 0x7fb41c003820 2021-11-21T21:59:14.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43affd700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb41c003820 0x7fb41c003c20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fb424002f70 tx=0x7fb424006ca0).ready entity=mon.0 client_cookie=6d55d14d84eae14c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb424012460 con 0x7fb41c003820 2021-11-21T21:59:14.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4240125c0 con 0x7fb41c003820 2021-11-21T21:59:14.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb439ffb700 1 -- 172.21.2.116:0/1683549717 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb424012920 con 0x7fb41c003820 2021-11-21T21:59:14.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- 172.21.2.116:0/1683549717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb41c003820 msgr2=0x7fb41c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb41c003820 0x7fb41c003c20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fb424002f70 tx=0x7fb424006ca0).stop 2021-11-21T21:59:14.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- 172.21.2.116:0/1683549717 shutdown_connections 2021-11-21T21:59:14.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 --2- 172.21.2.116:0/1683549717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb41c003820 0x7fb41c003c20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- 172.21.2.116:0/1683549717 >> 172.21.2.116:0/1683549717 conn(0x7fb43c05bd20 msgr2=0x7fb43c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- 172.21.2.116:0/1683549717 shutdown_connections 2021-11-21T21:59:14.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- 172.21.2.116:0/1683549717 wait complete. 2021-11-21T21:59:14.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 Processor -- start 2021-11-21T21:59:14.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- start start 2021-11-21T21:59:14.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb41c003820 0x7fb43c064770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb43c063850 0x7fb43c0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb43c1489f0 0x7fb43c067130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c146ed0 con 0x7fb43c063850 2021-11-21T21:59:14.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c147130 con 0x7fb43c1489f0 2021-11-21T21:59:14.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c15a3e0 con 0x7fb41c003820 2021-11-21T21:59:14.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb43c063850 0x7fb43c0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb43c063850 0x7fb43c0b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48152/0 (socket says 172.21.2.116:48152) 2021-11-21T21:59:14.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 -- 172.21.2.116:0/1517400841 learned_addr learned my addr 172.21.2.116:0/1517400841 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:14.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 -- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb41c003820 msgr2=0x7fb43c064770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:14.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb41c003820 0x7fb43c064770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 -- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb43c1489f0 msgr2=0x7fb43c067130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:14.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb43c1489f0 0x7fb43c067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 -- 172.21.2.116:0/1517400841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c15a680 con 0x7fb43c063850 2021-11-21T21:59:14.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb43a7fc700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb43c063850 0x7fb43c0b0840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fb434000f70 tx=0x7fb434006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb433fff700 1 -- 172.21.2.116:0/1517400841 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb434013460 con 0x7fb43c063850 2021-11-21T21:59:14.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb433fff700 1 -- 172.21.2.116:0/1517400841 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4340135c0 con 0x7fb43c063850 2021-11-21T21:59:14.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c1462e0 con 0x7fb43c063850 2021-11-21T21:59:14.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb433fff700 1 -- 172.21.2.116:0/1517400841 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb434013920 con 0x7fb43c063850 2021-11-21T21:59:14.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.874+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c148e70 con 0x7fb43c063850 2021-11-21T21:59:14.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.878+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c0573d0 con 0x7fb43c063850 2021-11-21T21:59:14.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.878+0000 7fb433fff700 1 -- 172.21.2.116:0/1517400841 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb43401a060 con 0x7fb43c063850 2021-11-21T21:59:14.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.878+0000 7fb433fff700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4200534f0 0x7fb4200559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:14.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.878+0000 7fb433fff700 1 -- 172.21.2.116:0/1517400841 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb43406b060 con 0x7fb43c063850 2021-11-21T21:59:14.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.878+0000 7fb43affd700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4200534f0 0x7fb4200559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:14.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.878+0000 7fb43affd700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4200534f0 0x7fb4200559b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fb424002f70 tx=0x7fb424014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:14.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.878+0000 7fb433fff700 1 -- 172.21.2.116:0/1517400841 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb434042390 con 0x7fb43c063850 2021-11-21T21:59:14.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb43c05db70 con 0x7fb43c063850 2021-11-21T21:59:14.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb433fff700 1 -- 172.21.2.116:0/1517400841 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fb434018070 con 0x7fb43c063850 2021-11-21T21:59:14.992 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:14.993 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4130375126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4130375126},{"type":"v1","addr":"172.21.2.117:6835","nonce":4130375126}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1880208335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1880208335},{"type":"v1","addr":"172.21.2.116:6833","nonce":1880208335}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/833609720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":833609720},{"type":"v1","addr":"172.21.2.116:6835","nonce":833609720}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25}],"filesystems":[]} 2021-11-21T21:59:14.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4200534f0 msgr2=0x7fb4200559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4200534f0 0x7fb4200559b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fb424002f70 tx=0x7fb424014000).stop 2021-11-21T21:59:14.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb43c063850 msgr2=0x7fb43c0b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:14.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb43c063850 0x7fb43c0b0840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fb434000f70 tx=0x7fb434006df0).stop 2021-11-21T21:59:14.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 shutdown_connections 2021-11-21T21:59:14.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb43c1489f0 0x7fb43c067130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4200534f0 0x7fb4200559b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb43c063850 0x7fb43c0b0840 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:14.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 --2- 172.21.2.116:0/1517400841 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb41c003820 0x7fb43c064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:14.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 >> 172.21.2.116:0/1517400841 conn(0x7fb43c05bd20 msgr2=0x7fb43c147440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:14.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 shutdown_connections 2021-11-21T21:59:14.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:14.990+0000 7fb441674700 1 -- 172.21.2.116:0/1517400841 wait complete. 2021-11-21T21:59:14.996 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 25 2021-11-21T21:59:14.999 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T21:59:15.000 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T21:59:15.000 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T21:59:15.000 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:59:15.001 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T21:59:15.001 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T21:59:15.001 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T21:59:15.002 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:59:15.002 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T21:59:15.002 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T21:59:15.003 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T21:59:15.003 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:59:15.006 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T21:59:15.007 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T21:59:15.009 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T21:59:15.009 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T21:59:15.010 INFO:tasks.ceph.mds.a:Started 2021-11-21T21:59:15.011 DEBUG:teuthology.parallel:result is None 2021-11-21T21:59:15.012 INFO:tasks.ceph.mds.c:Started 2021-11-21T21:59:15.013 DEBUG:teuthology.parallel:result is None 2021-11-21T21:59:15.013 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T21:59:15.014 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T21:59:15.043 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T21:59:15.044 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:59:15.060 INFO:tasks.ceph.mds.b:Started 2021-11-21T21:59:15.061 DEBUG:teuthology.parallel:result is None 2021-11-21T21:59:15.061 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T21:59:15.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc794ce7700 1 Processor -- start 2021-11-21T21:59:15.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc794ce7700 1 -- start start 2021-11-21T21:59:15.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc794ce7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790144810 0x7fc7901478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:15.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc794ce7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc7900abce0 0x7fc7900b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:15.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc794ce7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc790147e80 con 0x7fc7900b1db0 2021-11-21T21:59:15.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc794ce7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7900b1490 con 0x7fc7900abce0 2021-11-21T21:59:15.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc794ce7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900af060 con 0x7fc790144810 2021-11-21T21:59:15.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc7900abce0 0x7fc7900b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:15.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc7900abce0 0x7fc7900b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48192/0 (socket says 172.21.2.116:48192) 2021-11-21T21:59:15.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 -- 172.21.2.116:0/1216567978 learned_addr learned my addr 172.21.2.116:0/1216567978 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:15.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78e59c700 1 --2- 172.21.2.116:0/1216567978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790144810 0x7fc7901478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:15.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 -- 172.21.2.116:0/1216567978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790144810 msgr2=0x7fc7901478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:15.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 --2- 172.21.2.116:0/1216567978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790144810 0x7fc7901478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:15.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 -- 172.21.2.116:0/1216567978 >> v1:172.21.2.117:6789/0 conn(0x7fc7900b1db0 legacy=0x7fc7900ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:15.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 -- 172.21.2.116:0/1216567978 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900af5a0 con 0x7fc7900abce0 2021-11-21T21:59:15.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78e59c700 1 --2- 172.21.2.116:0/1216567978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790144810 0x7fc7901478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:59:15.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78dd9b700 1 --2- 172.21.2.116:0/1216567978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7900abce0 0x7fc7900b18c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fc780002f70 tx=0x7fc780006eb0).ready entity=mon.0 client_cookie=90b0bf734511d1d9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:15.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78d59a700 1 -- 172.21.2.116:0/1216567978 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc78000d5c0 con 0x7fc7900abce0 2021-11-21T21:59:15.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78d59a700 1 -- 172.21.2.116:0/1216567978 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780000e90 con 0x7fc7900abce0 2021-11-21T21:59:15.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.110+0000 7fc78d59a700 1 -- 172.21.2.116:0/1216567978 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc78000d950 con 0x7fc7900abce0 2021-11-21T21:59:15.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1216567978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7900abce0 msgr2=0x7fc7900b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:15.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1216567978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7900abce0 0x7fc7900b18c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fc780002f70 tx=0x7fc780006eb0).stop 2021-11-21T21:59:15.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1216567978 shutdown_connections 2021-11-21T21:59:15.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1216567978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7900abce0 0x7fc7900b18c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:15.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1216567978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790144810 0x7fc7901478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:15.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1216567978 >> 172.21.2.116:0/1216567978 conn(0x7fc79005bd20 msgr2=0x7fc79005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:15.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1216567978 shutdown_connections 2021-11-21T21:59:15.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1216567978 wait complete. 2021-11-21T21:59:15.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 Processor -- start 2021-11-21T21:59:15.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- start start 2021-11-21T21:59:15.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7900abce0 0x7fc790156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:15.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc790144810 0x7fc790143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:15.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790145a20 0x7fc79015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:15.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc790159b80 con 0x7fc790144810 2021-11-21T21:59:15.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900b4be0 con 0x7fc7900abce0 2021-11-21T21:59:15.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901461d0 con 0x7fc790145a20 2021-11-21T21:59:15.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc790144810 0x7fc790143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:15.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78ed9d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790145a20 0x7fc79015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:15.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78e59c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7900abce0 0x7fc790156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:15.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78ed9d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790145a20 0x7fc79015a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:51542/0 (socket says 172.21.2.116:51542) 2021-11-21T21:59:15.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78ed9d700 1 -- 172.21.2.116:0/1502407146 learned_addr learned my addr 172.21.2.116:0/1502407146 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:15.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc790144810 0x7fc790143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48196/0 (socket says 172.21.2.116:48196) 2021-11-21T21:59:15.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 -- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790145a20 msgr2=0x7fc79015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:15.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790145a20 0x7fc79015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:15.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 -- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7900abce0 msgr2=0x7fc790156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:15.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7900abce0 0x7fc790156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:15.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 -- 172.21.2.116:0/1502407146 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78000b1e0 con 0x7fc790144810 2021-11-21T21:59:15.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78dd9b700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc790144810 0x7fc790143510 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fc780002f70 tx=0x7fc780015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:15.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc7877fe700 1 -- 172.21.2.116:0/1502407146 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7800116e0 con 0x7fc790144810 2021-11-21T21:59:15.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc7877fe700 1 -- 172.21.2.116:0/1502407146 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780011840 con 0x7fc790144810 2021-11-21T21:59:15.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc790146470 con 0x7fc790144810 2021-11-21T21:59:15.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc7877fe700 1 -- 172.21.2.116:0/1502407146 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc780011ba0 con 0x7fc790144810 2021-11-21T21:59:15.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc790063670 con 0x7fc790144810 2021-11-21T21:59:15.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7900af300 con 0x7fc790144810 2021-11-21T21:59:15.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc7877fe700 1 -- 172.21.2.116:0/1502407146 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc780025030 con 0x7fc790144810 2021-11-21T21:59:15.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc7877fe700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc774053660 0x7fc774055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:15.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc7877fe700 1 -- 172.21.2.116:0/1502407146 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc78006c730 con 0x7fc790144810 2021-11-21T21:59:15.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.114+0000 7fc78e59c700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc774053660 0x7fc774055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:15.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.118+0000 7fc78e59c700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc774053660 0x7fc774055b20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fc77800c000 tx=0x7fc778008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:15.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.118+0000 7fc7877fe700 1 -- 172.21.2.116:0/1502407146 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc780043b70 con 0x7fc790144810 2021-11-21T21:59:15.132 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T21:59:15.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:15.210+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fc7900573d0 con 0x7fc790144810 2021-11-21T21:59:16.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc7877fe700 1 -- 172.21.2.116:0/1502407146 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc780047190 con 0x7fc790144810 2021-11-21T21:59:16.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc774053660 msgr2=0x7fc774055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc774053660 0x7fc774055b20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fc77800c000 tx=0x7fc778008040).stop 2021-11-21T21:59:16.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc790144810 msgr2=0x7fc790143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc790144810 0x7fc790143510 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fc780002f70 tx=0x7fc780015000).stop 2021-11-21T21:59:16.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 shutdown_connections 2021-11-21T21:59:16.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc790145a20 0x7fc79015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc774053660 0x7fc774055b20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc790144810 0x7fc790143510 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 --2- 172.21.2.116:0/1502407146 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7900abce0 0x7fc790156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 >> 172.21.2.116:0/1502407146 conn(0x7fc79005bd20 msgr2=0x7fc7900b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 shutdown_connections 2021-11-21T21:59:16.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.178+0000 7fc794ce7700 1 -- 172.21.2.116:0/1502407146 wait complete. 2021-11-21T21:59:16.184 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T21:59:16.188 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T21:59:16.188 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T21:59:16.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4fa00d0700 1 Processor -- start 2021-11-21T21:59:16.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4fa00d0700 1 -- start start 2021-11-21T21:59:16.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4fa00d0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f980a6aa0 0x7f4f9814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4fa00d0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f980a6e80 con 0x7f4f980a86b0 2021-11-21T21:59:16.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4fa00d0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9814a7b0 con 0x7f4f98062410 2021-11-21T21:59:16.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4fa00d0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980a8bc0 con 0x7f4f980a6aa0 2021-11-21T21:59:16.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9d66b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f4f980a86b0 0x7f4f981470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60164/0 (socket says 172.21.2.116:60164) 2021-11-21T21:59:16.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9d66b700 1 -- 172.21.2.116:0/2166606849 learned_addr learned my addr 172.21.2.116:0/2166606849 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:16.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069405342 0 0) 0x7f4f980a6e80 con 0x7f4f980a86b0 2021-11-21T21:59:16.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f80003260 con 0x7f4f980a86b0 2021-11-21T21:59:16.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2961989829 0 0) 0x7f4f80003260 con 0x7f4f980a86b0 2021-11-21T21:59:16.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f980a6aa0 msgr2=0x7f4f9814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:16.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f980a6aa0 0x7f4f9814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 >> v1:172.21.2.117:6789/0 conn(0x7f4f98062410 legacy=0x7f4f981443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:16.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9814aa10 con 0x7f4f980a86b0 2021-11-21T21:59:16.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f4f88001680 con 0x7f4f980a86b0 2021-11-21T21:59:16.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 >> v1:172.21.2.116:6789/0 conn(0x7f4f980a86b0 legacy=0x7f4f981470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f80003800 0x7f4f80003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f980a6aa0 0x7f4f80004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f80004bf0 0x7f4f8000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980a8bc0 con 0x7f4f80003800 2021-11-21T21:59:16.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800052a0 con 0x7f4f80004bf0 2021-11-21T21:59:16.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8000bc40 con 0x7f4f980a6aa0 2021-11-21T21:59:16.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.250+0000 7f4f9de6c700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f80003800 0x7f4f80003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9de6c700 1 -- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f980a6aa0 msgr2=0x7f4f80004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:16.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9de6c700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f980a6aa0 0x7f4f80004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9de6c700 1 -- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f80004bf0 msgr2=0x7f4f8000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:16.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9de6c700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f80004bf0 0x7f4f8000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9de6c700 1 -- 172.21.2.116:0/2166606849 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80003260 con 0x7f4f80003800 2021-11-21T21:59:16.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9de6c700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f80003800 0x7f4f80003c00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f4f94002f70 tx=0x7f4f94006ca0).ready entity=mon.0 client_cookie=3951f30a810f5ea1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f94012410 con 0x7f4f80003800 2021-11-21T21:59:16.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94012570 con 0x7f4f80003800 2021-11-21T21:59:16.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9ce6a700 1 -- 172.21.2.116:0/2166606849 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f940128d0 con 0x7f4f80003800 2021-11-21T21:59:16.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- 172.21.2.116:0/2166606849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f80003800 msgr2=0x7f4f80003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 --2- 172.21.2.116:0/2166606849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f80003800 0x7f4f80003c00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f4f94002f70 tx=0x7f4f94006ca0).stop 2021-11-21T21:59:16.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 -- 172.21.2.116:0/2166606849 reap_dead start 2021-11-21T21:59:16.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- 172.21.2.116:0/2166606849 shutdown_connections 2021-11-21T21:59:16.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- 172.21.2.116:0/2166606849 >> 172.21.2.116:0/2166606849 conn(0x7f4f9805bd20 msgr2=0x7f4f9805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- 172.21.2.116:0/2166606849 shutdown_connections 2021-11-21T21:59:16.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- 172.21.2.116:0/2166606849 wait complete. 2021-11-21T21:59:16.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 Processor -- start 2021-11-21T21:59:16.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- start start 2021-11-21T21:59:16.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f80003800 0x7f4f981474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f981561e0 0x7f4f98159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f9815a250 0x7f4f98156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980acb40 con 0x7f4f9815a250 2021-11-21T21:59:16.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980ac600 con 0x7f4f981561e0 2021-11-21T21:59:16.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980ac8a0 con 0x7f4f80003800 2021-11-21T21:59:16.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f9815a250 0x7f4f98156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f9815a250 0x7f4f98156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48222/0 (socket says 172.21.2.116:48222) 2021-11-21T21:59:16.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 -- 172.21.2.116:0/4121823105 learned_addr learned my addr 172.21.2.116:0/4121823105 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9de6c700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f80003800 0x7f4f981474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9d66b700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f981561e0 0x7f4f98159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 -- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f80003800 msgr2=0x7f4f981474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f80003800 0x7f4f981474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 -- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f981561e0 msgr2=0x7f4f98159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f981561e0 0x7f4f98159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 -- 172.21.2.116:0/4121823105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f981598c0 con 0x7f4f9815a250 2021-11-21T21:59:16.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f9e66d700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f9815a250 0x7f4f98156800 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f4f88001390 tx=0x7f4f90007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f86ffd700 1 -- 172.21.2.116:0/4121823105 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f90004850 con 0x7f4f9815a250 2021-11-21T21:59:16.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f86ffd700 1 -- 172.21.2.116:0/4121823105 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f900049b0 con 0x7f4f9815a250 2021-11-21T21:59:16.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f98064ed0 con 0x7f4f9815a250 2021-11-21T21:59:16.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4f86ffd700 1 -- 172.21.2.116:0/4121823105 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f90004d10 con 0x7f4f9815a250 2021-11-21T21:59:16.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.254+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f980646f0 con 0x7f4f9815a250 2021-11-21T21:59:16.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.258+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f98159a50 con 0x7f4f9815a250 2021-11-21T21:59:16.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.258+0000 7f4f86ffd700 1 -- 172.21.2.116:0/4121823105 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4f9001b070 con 0x7f4f9815a250 2021-11-21T21:59:16.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.258+0000 7f4f86ffd700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f78053720 0x7f4f78055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.258+0000 7f4f86ffd700 1 -- 172.21.2.116:0/4121823105 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4f9006a9b0 con 0x7f4f9815a250 2021-11-21T21:59:16.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.258+0000 7f4f9de6c700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f78053720 0x7f4f78055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.258+0000 7f4f9de6c700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f78053720 0x7f4f78055be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f4f94019000 tx=0x7f4f94015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.258+0000 7f4f86ffd700 1 -- 172.21.2.116:0/4121823105 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f90041de0 con 0x7f4f9815a250 2021-11-21T21:59:16.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4f980573d0 con 0x7f4f9815a250 2021-11-21T21:59:16.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4f86ffd700 1 -- 172.21.2.116:0/4121823105 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f4f90045400 con 0x7f4f9815a250 2021-11-21T21:59:16.374 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:16.374 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:59:16.375 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T21:59:16.375 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.375 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T21:59:16.376 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T21:59:16.376 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.376 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:16.376 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.377 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.377 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.377 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.377 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.377 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.378 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T21:59:16.378 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T21:59:16.378 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.378 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:16.379 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.379 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.379 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.379 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.380 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.380 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.380 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T21:59:16.380 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T21:59:16.380 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:16.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.381 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.381 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.382 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.382 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.382 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T21:59:16.382 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T21:59:16.382 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.383 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.383 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.383 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.383 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.383 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.384 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.384 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T21:59:16.384 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T21:59:16.384 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.385 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.385 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.385 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.386 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.386 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.386 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.386 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T21:59:16.387 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T21:59:16.387 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.387 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.387 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.388 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.388 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.388 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.388 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.389 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T21:59:16.389 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T21:59:16.389 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.389 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.390 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.390 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.390 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.390 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.390 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.391 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T21:59:16.391 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T21:59:16.391 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.391 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.392 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.392 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.392 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.392 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.392 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.393 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T21:59:16.393 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T21:59:16.393 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.393 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.393 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.394 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.394 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.394 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.394 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.395 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T21:59:16.395 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T21:59:16.395 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.395 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.395 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.396 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.396 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.396 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.396 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.397 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T21:59:16.397 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T21:59:16.397 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.397 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.397 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.398 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.398 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.398 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.398 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.398 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T21:59:16.399 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T21:59:16.399 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.399 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:16.399 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:59:16.400 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:59:16.400 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.400 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.400 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.400 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T21:59:16.401 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T21:59:16.401 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.401 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:16.401 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:59:16.402 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:59:16.402 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.402 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.402 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.402 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T21:59:16.403 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T21:59:16.403 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.403 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:59:16.403 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:16.403 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:16.404 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.404 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.404 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.405 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.405 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T21:59:16.405 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T21:59:16.405 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.406 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:59:16.406 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:59:16.406 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.407 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.407 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:16.407 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:16.407 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T21:59:16.408 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T21:59:16.408 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:16.408 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:59:16.408 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:59:16.408 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:16.409 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.409 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:16.409 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:16.409 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:59:16.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f78053720 msgr2=0x7f4f78055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f78053720 0x7f4f78055be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f4f94019000 tx=0x7f4f94015000).stop 2021-11-21T21:59:16.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f9815a250 msgr2=0x7f4f98156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f9815a250 0x7f4f98156800 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f4f88001390 tx=0x7f4f90007e50).stop 2021-11-21T21:59:16.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 shutdown_connections 2021-11-21T21:59:16.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f78053720 0x7f4f78055be0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f9815a250 0x7f4f98156800 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f981561e0 0x7f4f98159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 --2- 172.21.2.116:0/4121823105 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f80003800 0x7f4f981474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 >> 172.21.2.116:0/4121823105 conn(0x7f4f9805bd20 msgr2=0x7f4f98142640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 shutdown_connections 2021-11-21T21:59:16.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.370+0000 7f4fa00d0700 1 -- 172.21.2.116:0/4121823105 wait complete. 2021-11-21T21:59:16.414 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:59:16.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc330c700 1 Processor -- start 2021-11-21T21:59:16.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc330c700 1 -- start start 2021-11-21T21:59:16.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc330c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9fbc067850 0x7f9fbc0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc330c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc0b29b0 con 0x7f9fbc1443a0 2021-11-21T21:59:16.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc330c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc067c30 con 0x7f9fbc0b7070 2021-11-21T21:59:16.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc330c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc0ab720 con 0x7f9fbc067850 2021-11-21T21:59:16.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc10a8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9fbc067850 0x7f9fbc0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc18a9700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9fbc1443a0 0x7f9fbc144810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60186/0 (socket says 172.21.2.116:60186) 2021-11-21T21:59:16.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc18a9700 1 -- 172.21.2.116:0/1182083267 learned_addr learned my addr 172.21.2.116:0/1182083267 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:16.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9faffff700 1 -- 172.21.2.116:0/1182083267 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003314870 0 0) 0x7f9fbc0b29b0 con 0x7f9fbc1443a0 2021-11-21T21:59:16.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9faffff700 1 -- 172.21.2.116:0/1182083267 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa4003260 con 0x7f9fbc1443a0 2021-11-21T21:59:16.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc10a8700 1 -- 172.21.2.116:0/1182083267 >> v1:172.21.2.117:6789/0 conn(0x7f9fbc0b7070 legacy=0x7f9fbc0ab010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:16.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc10a8700 1 -- 172.21.2.116:0/1182083267 >> v1:172.21.2.116:6789/0 conn(0x7f9fbc1443a0 legacy=0x7f9fbc144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.462+0000 7f9fc10a8700 1 -- 172.21.2.116:0/1182083267 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc0b7910 con 0x7f9fbc067850 2021-11-21T21:59:16.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc10a8700 1 --2- 172.21.2.116:0/1182083267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc067850 0x7f9fbc0b3320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f9fbc064400 tx=0x7f9fb8006df0).ready entity=mon.0 client_cookie=481ba6d8fe60964a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9faffff700 1 -- 172.21.2.116:0/1182083267 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fb800d5c0 con 0x7f9fbc067850 2021-11-21T21:59:16.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9faffff700 1 -- 172.21.2.116:0/1182083267 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8000e90 con 0x7f9fbc067850 2021-11-21T21:59:16.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9faffff700 1 -- 172.21.2.116:0/1182083267 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fb800d910 con 0x7f9fbc067850 2021-11-21T21:59:16.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/1182083267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc067850 msgr2=0x7f9fbc0b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 --2- 172.21.2.116:0/1182083267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc067850 0x7f9fbc0b3320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f9fbc064400 tx=0x7f9fb8006df0).stop 2021-11-21T21:59:16.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/1182083267 shutdown_connections 2021-11-21T21:59:16.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 --2- 172.21.2.116:0/1182083267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc067850 0x7f9fbc0b3320 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/1182083267 >> 172.21.2.116:0/1182083267 conn(0x7f9fbc064d60 msgr2=0x7f9fbc065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/1182083267 shutdown_connections 2021-11-21T21:59:16.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/1182083267 wait complete. 2021-11-21T21:59:16.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 Processor -- start 2021-11-21T21:59:16.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- start start 2021-11-21T21:59:16.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9fbc067850 0x7f9fbc0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc0b7070 0x7f9fbc0b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9fbc1443a0 0x7f9fbc09df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc156610 con 0x7f9fbc0b7070 2021-11-21T21:59:16.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc156800 con 0x7f9fbc1443a0 2021-11-21T21:59:16.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc156aa0 con 0x7f9fbc067850 2021-11-21T21:59:16.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc0b7070 0x7f9fbc0b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc0b7070 0x7f9fbc0b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48234/0 (socket says 172.21.2.116:48234) 2021-11-21T21:59:16.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 -- 172.21.2.116:0/524883624 learned_addr learned my addr 172.21.2.116:0/524883624 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 -- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9fbc067850 msgr2=0x7f9fbc0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9fbc067850 0x7f9fbc0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 -- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9fbc1443a0 msgr2=0x7f9fbc09df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9fbc1443a0 0x7f9fbc09df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 -- 172.21.2.116:0/524883624 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb800b180 con 0x7f9fbc0b7070 2021-11-21T21:59:16.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc08a7700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc0b7070 0x7f9fbc0b0ae0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f9fb0002fb0 tx=0x7f9fb0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fae7fc700 1 -- 172.21.2.116:0/524883624 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fb0013460 con 0x7f9fbc0b7070 2021-11-21T21:59:16.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fae7fc700 1 -- 172.21.2.116:0/524883624 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb00135c0 con 0x7f9fbc0b7070 2021-11-21T21:59:16.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc156d40 con 0x7f9fbc0b7070 2021-11-21T21:59:16.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc157480 con 0x7f9fbc0b7070 2021-11-21T21:59:16.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fae7fc700 1 -- 172.21.2.116:0/524883624 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fb0013920 con 0x7f9fbc0b7070 2021-11-21T21:59:16.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.466+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc0573d0 con 0x7f9fbc0b7070 2021-11-21T21:59:16.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.470+0000 7f9fae7fc700 1 -- 172.21.2.116:0/524883624 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9fb000f070 con 0x7f9fbc0b7070 2021-11-21T21:59:16.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.470+0000 7f9fae7fc700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9f9c053660 0x7f9f9c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.470+0000 7f9fae7fc700 1 -- 172.21.2.116:0/524883624 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9fb006b140 con 0x7f9fbc0b7070 2021-11-21T21:59:16.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.470+0000 7f9fc10a8700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9f9c053660 0x7f9f9c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.470+0000 7f9fc10a8700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9f9c053660 0x7f9f9c055b20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9fbc064400 tx=0x7f9fb8014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.470+0000 7f9fae7fc700 1 -- 172.21.2.116:0/524883624 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9fb00424c0 con 0x7f9fbc0b7070 2021-11-21T21:59:16.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fbc0b35c0 con 0x7f9fbc0b7070 2021-11-21T21:59:16.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fae7fc700 1 -- 172.21.2.116:0/524883624 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13182 (secure 0 0 0) 0x7f9fb0045ae0 con 0x7f9fbc0b7070 2021-11-21T21:59:16.558 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:16.558 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":62,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:59:16.168222+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:59:16.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9f9c053660 msgr2=0x7f9f9c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9f9c053660 0x7f9f9c055b20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9fbc064400 tx=0x7f9fb8014000).stop 2021-11-21T21:59:16.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc0b7070 msgr2=0x7f9fbc0b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc0b7070 0x7f9fbc0b0ae0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f9fb0002fb0 tx=0x7f9fb0006df0).stop 2021-11-21T21:59:16.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 shutdown_connections 2021-11-21T21:59:16.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9fbc1443a0 0x7f9fbc09df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9f9c053660 0x7f9f9c055b20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9fbc0b7070 0x7f9fbc0b0ae0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 --2- 172.21.2.116:0/524883624 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9fbc067850 0x7f9fbc0b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 >> 172.21.2.116:0/524883624 conn(0x7f9fbc064d60 msgr2=0x7f9fbc144810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 shutdown_connections 2021-11-21T21:59:16.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.554+0000 7f9fc330c700 1 -- 172.21.2.116:0/524883624 wait complete. 2021-11-21T21:59:16.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T21:59:16.566 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:59:16.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44fac2b700 1 Processor -- start 2021-11-21T21:59:16.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44fac2b700 1 -- start start 2021-11-21T21:59:16.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44fac2b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f41426e0 0x7f44f4142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44fac2b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44f4062e60 0x7f44f40af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44fac2b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f40b0f60 con 0x7f44f40ad470 2021-11-21T21:59:16.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44fac2b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f40ae030 con 0x7f44f4062e60 2021-11-21T21:59:16.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44fac2b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4063470 con 0x7f44f41426e0 2021-11-21T21:59:16.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44f3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44f4062e60 0x7f44f40af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44f3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44f4062e60 0x7f44f40af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48242/0 (socket says 172.21.2.116:48242) 2021-11-21T21:59:16.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.614+0000 7f44f3fff700 1 -- 172.21.2.116:0/1881872174 learned_addr learned my addr 172.21.2.116:0/1881872174 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 -- 172.21.2.116:0/1881872174 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f41426e0 msgr2=0x7f44f4142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:16.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 --2- 172.21.2.116:0/1881872174 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f41426e0 0x7f44f4142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 -- 172.21.2.116:0/1881872174 >> v1:172.21.2.117:6789/0 conn(0x7f44f40ad470 legacy=0x7f44f40ad920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:16.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 -- 172.21.2.116:0/1881872174 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4143050 con 0x7f44f4062e60 2021-11-21T21:59:16.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 --2- 172.21.2.116:0/1881872174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f4062e60 0x7f44f40af3d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f44f41432f0 tx=0x7f44e4006f40).ready entity=mon.0 client_cookie=252455cb54685713 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f37fe700 1 -- 172.21.2.116:0/1881872174 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e400f940 con 0x7f44f4062e60 2021-11-21T21:59:16.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f37fe700 1 -- 172.21.2.116:0/1881872174 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4000e90 con 0x7f44f4062e60 2021-11-21T21:59:16.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f37fe700 1 -- 172.21.2.116:0/1881872174 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e400fc90 con 0x7f44f4062e60 2021-11-21T21:59:16.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1881872174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f4062e60 msgr2=0x7f44f40af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1881872174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f4062e60 0x7f44f40af3d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f44f41432f0 tx=0x7f44e4006f40).stop 2021-11-21T21:59:16.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1881872174 shutdown_connections 2021-11-21T21:59:16.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1881872174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f4062e60 0x7f44f40af3d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1881872174 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f41426e0 0x7f44f4142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1881872174 >> 172.21.2.116:0/1881872174 conn(0x7f44f40b5730 msgr2=0x7f44f40b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1881872174 shutdown_connections 2021-11-21T21:59:16.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1881872174 wait complete. 2021-11-21T21:59:16.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 Processor -- start 2021-11-21T21:59:16.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- start start 2021-11-21T21:59:16.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f40ad470 0x7f44f4066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f41426e0 0x7f44f4151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44f4143740 0x7f44f4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40642b0 con 0x7f44f41426e0 2021-11-21T21:59:16.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4064550 con 0x7f44f4143740 2021-11-21T21:59:16.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4155460 con 0x7f44f40ad470 2021-11-21T21:59:16.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f41426e0 0x7f44f4151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f41426e0 0x7f44f4151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48248/0 (socket says 172.21.2.116:48248) 2021-11-21T21:59:16.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 -- 172.21.2.116:0/1689346197 learned_addr learned my addr 172.21.2.116:0/1689346197 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 -- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f40ad470 msgr2=0x7f44f4066d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f40ad470 0x7f44f4066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 -- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44f4143740 msgr2=0x7f44f4155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44f4143740 0x7f44f4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 -- 172.21.2.116:0/1689346197 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e400a360 con 0x7f44f41426e0 2021-11-21T21:59:16.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f3fff700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f41426e0 0x7f44f4151ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f44f41432f0 tx=0x7f44e4018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f1ffb700 1 -- 172.21.2.116:0/1689346197 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e4017bc0 con 0x7f44f41426e0 2021-11-21T21:59:16.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f1ffb700 1 -- 172.21.2.116:0/1689346197 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4017d20 con 0x7f44f41426e0 2021-11-21T21:59:16.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f1ffb700 1 -- 172.21.2.116:0/1689346197 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e4020600 con 0x7f44f41426e0 2021-11-21T21:59:16.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f4155700 con 0x7f44f41426e0 2021-11-21T21:59:16.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f4064da0 con 0x7f44f41426e0 2021-11-21T21:59:16.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f4155890 con 0x7f44f41426e0 2021-11-21T21:59:16.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.618+0000 7f44f1ffb700 1 -- 172.21.2.116:0/1689346197 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f44e4027030 con 0x7f44f41426e0 2021-11-21T21:59:16.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.622+0000 7f44f1ffb700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d8053570 0x7f44d8055a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.622+0000 7f44f1ffb700 1 -- 172.21.2.116:0/1689346197 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f44e406c8a0 con 0x7f44f41426e0 2021-11-21T21:59:16.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.622+0000 7f44f89c7700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d8053570 0x7f44d8055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.622+0000 7f44f89c7700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d8053570 0x7f44d8055a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f44dc000ed0 tx=0x7f44dc006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.622+0000 7f44f1ffb700 1 -- 172.21.2.116:0/1689346197 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44e4043cd0 con 0x7f44f41426e0 2021-11-21T21:59:16.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f44f40b1e00 con 0x7f44f41426e0 2021-11-21T21:59:16.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44f1ffb700 1 -- 172.21.2.116:0/1689346197 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v62) v1 ==== 110+0+10 (secure 0 0 0) 0x7f44e40250b0 con 0x7f44f41426e0 2021-11-21T21:59:16.710 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:59:16.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d8053570 msgr2=0x7f44d8055a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d8053570 0x7f44d8055a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f44dc000ed0 tx=0x7f44dc006930).stop 2021-11-21T21:59:16.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f41426e0 msgr2=0x7f44f4151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f41426e0 0x7f44f4151ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f44f41432f0 tx=0x7f44e4018000).stop 2021-11-21T21:59:16.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 shutdown_connections 2021-11-21T21:59:16.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44f4143740 0x7f44f4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d8053570 0x7f44d8055a30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44f41426e0 0x7f44f4151ed0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 --2- 172.21.2.116:0/1689346197 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44f40ad470 0x7f44f4066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 >> 172.21.2.116:0/1689346197 conn(0x7f44f40b5730 msgr2=0x7f44f40ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 shutdown_connections 2021-11-21T21:59:16.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.706+0000 7f44fac2b700 1 -- 172.21.2.116:0/1689346197 wait complete. 2021-11-21T21:59:16.716 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:59:16.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 Processor -- start 2021-11-21T21:59:16.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- start start 2021-11-21T21:59:16.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fea581426e0 0x7fea58142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fea58062e60 0x7fea580af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea580b0f60 con 0x7fea580ad470 2021-11-21T21:59:16.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea58063470 con 0x7fea581426e0 2021-11-21T21:59:16.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea58143050 con 0x7fea58062e60 2021-11-21T21:59:16.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fea581426e0 0x7fea58142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fea581426e0 0x7fea58142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48252/0 (socket says 172.21.2.116:48252) 2021-11-21T21:59:16.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 -- 172.21.2.116:0/1070497397 learned_addr learned my addr 172.21.2.116:0/1070497397 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 -- 172.21.2.116:0/1070497397 >> v2:172.21.2.117:3300/0 conn(0x7fea58062e60 msgr2=0x7fea580af3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:16.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 --2- 172.21.2.116:0/1070497397 >> v2:172.21.2.117:3300/0 conn(0x7fea58062e60 0x7fea580af3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 -- 172.21.2.116:0/1070497397 >> v1:172.21.2.117:6789/0 conn(0x7fea580ad470 legacy=0x7fea580ad8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:16.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 -- 172.21.2.116:0/1070497397 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea580adfc0 con 0x7fea581426e0 2021-11-21T21:59:16.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea56ffd700 1 --2- 172.21.2.116:0/1070497397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea581426e0 0x7fea58142ac0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fea58062d10 tx=0x7fea40006e40).ready entity=mon.0 client_cookie=aaade97654bf9a5f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea55ffb700 1 -- 172.21.2.116:0/1070497397 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4000bcb0 con 0x7fea581426e0 2021-11-21T21:59:16.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea55ffb700 1 -- 172.21.2.116:0/1070497397 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea40000e90 con 0x7fea581426e0 2021-11-21T21:59:16.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea55ffb700 1 -- 172.21.2.116:0/1070497397 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea400155e0 con 0x7fea581426e0 2021-11-21T21:59:16.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- 172.21.2.116:0/1070497397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea581426e0 msgr2=0x7fea58142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 --2- 172.21.2.116:0/1070497397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea581426e0 0x7fea58142ac0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fea58062d10 tx=0x7fea40006e40).stop 2021-11-21T21:59:16.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- 172.21.2.116:0/1070497397 shutdown_connections 2021-11-21T21:59:16.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 --2- 172.21.2.116:0/1070497397 >> v2:172.21.2.117:3300/0 conn(0x7fea58062e60 0x7fea580af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 --2- 172.21.2.116:0/1070497397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea581426e0 0x7fea58142ac0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- 172.21.2.116:0/1070497397 >> 172.21.2.116:0/1070497397 conn(0x7fea580b5730 msgr2=0x7fea580b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- 172.21.2.116:0/1070497397 shutdown_connections 2021-11-21T21:59:16.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.766+0000 7fea5d306700 1 -- 172.21.2.116:0/1070497397 wait complete. 2021-11-21T21:59:16.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 Processor -- start 2021-11-21T21:59:16.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 -- start start 2021-11-21T21:59:16.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea580ad470 0x7fea580672e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea581426e0 0x7fea58141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea5813e270 0x7fea58145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5813ef10 con 0x7fea5813e270 2021-11-21T21:59:16.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea581457c0 con 0x7fea581426e0 2021-11-21T21:59:16.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5813f9c0 con 0x7fea580ad470 2021-11-21T21:59:16.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea5813e270 0x7fea58145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea5813e270 0x7fea58145dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48260/0 (socket says 172.21.2.116:48260) 2021-11-21T21:59:16.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 -- 172.21.2.116:0/1602581594 learned_addr learned my addr 172.21.2.116:0/1602581594 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 -- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea580ad470 msgr2=0x7fea580672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea580ad470 0x7fea580672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 -- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea581426e0 msgr2=0x7fea58141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea581426e0 0x7fea58141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 -- 172.21.2.116:0/1602581594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4000d4d0 con 0x7fea5813e270 2021-11-21T21:59:16.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea577fe700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea5813e270 0x7fea58145dd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fea50003330 tx=0x7fea500091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea4ffff700 1 -- 172.21.2.116:0/1602581594 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea50001520 con 0x7fea5813e270 2021-11-21T21:59:16.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea4ffff700 1 -- 172.21.2.116:0/1602581594 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea50001680 con 0x7fea5813e270 2021-11-21T21:59:16.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea4ffff700 1 -- 172.21.2.116:0/1602581594 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea500019e0 con 0x7fea5813e270 2021-11-21T21:59:16.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea581459f0 con 0x7fea5813e270 2021-11-21T21:59:16.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5813fec0 con 0x7fea5813e270 2021-11-21T21:59:16.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea580b1e00 con 0x7fea5813e270 2021-11-21T21:59:16.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea4ffff700 1 -- 172.21.2.116:0/1602581594 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fea50018030 con 0x7fea5813e270 2021-11-21T21:59:16.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea4ffff700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea3c053640 0x7fea3c055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea4ffff700 1 -- 172.21.2.116:0/1602581594 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fea5006b0f0 con 0x7fea5813e270 2021-11-21T21:59:16.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.770+0000 7fea56ffd700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea3c053640 0x7fea3c055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.774+0000 7fea56ffd700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea3c053640 0x7fea3c055b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fea58062d10 tx=0x7fea40018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.774+0000 7fea4ffff700 1 -- 172.21.2.116:0/1602581594 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea500424a0 con 0x7fea5813e270 2021-11-21T21:59:16.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea580573d0 con 0x7fea5813e270 2021-11-21T21:59:16.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea4ffff700 1 -- 172.21.2.116:0/1602581594 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea50045ac0 con 0x7fea5813e270 2021-11-21T21:59:16.861 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:59:16.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea3c053640 msgr2=0x7fea3c055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea3c053640 0x7fea3c055b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fea58062d10 tx=0x7fea40018000).stop 2021-11-21T21:59:16.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea5813e270 msgr2=0x7fea58145dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea5813e270 0x7fea58145dd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fea50003330 tx=0x7fea500091a0).stop 2021-11-21T21:59:16.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 shutdown_connections 2021-11-21T21:59:16.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea3c053640 0x7fea3c055b00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea5813e270 0x7fea58145dd0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea581426e0 0x7fea58141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 --2- 172.21.2.116:0/1602581594 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea580ad470 0x7fea580672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 >> 172.21.2.116:0/1602581594 conn(0x7fea580b5730 msgr2=0x7fea580ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 shutdown_connections 2021-11-21T21:59:16.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.858+0000 7fea5d306700 1 -- 172.21.2.116:0/1602581594 wait complete. 2021-11-21T21:59:16.868 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T21:59:16.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.914+0000 7fa87e35d700 1 Processor -- start 2021-11-21T21:59:16.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.914+0000 7fa87e35d700 1 -- start start 2021-11-21T21:59:16.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.914+0000 7fa87e35d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa8780a86b0 0x7fa87814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.914+0000 7fa87e35d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa878062410 0x7fa8781443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.914+0000 7fa87e35d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780a6e80 con 0x7fa8780a6aa0 2021-11-21T21:59:16.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780a8bc0 con 0x7fa8780a86b0 2021-11-21T21:59:16.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87814a7b0 con 0x7fa878062410 2021-11-21T21:59:16.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa8780a86b0 0x7fa87814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa8780a86b0 0x7fa87814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48268/0 (socket says 172.21.2.116:48268) 2021-11-21T21:59:16.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/2511922150 learned_addr learned my addr 172.21.2.116:0/2511922150 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/2511922150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa878062410 msgr2=0x7fa8781443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:16.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- 172.21.2.116:0/2511922150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa878062410 0x7fa8781443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/2511922150 >> v1:172.21.2.117:6789/0 conn(0x7fa8780a6aa0 legacy=0x7fa8781470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:16.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/2511922150 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87814aa10 con 0x7fa8780a86b0 2021-11-21T21:59:16.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- 172.21.2.116:0/2511922150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a86b0 0x7fa87814b150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fa878061ab0 tx=0x7fa860006eb0).ready entity=mon.0 client_cookie=6bc048f08ae9ddca server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa876ffd700 1 -- 172.21.2.116:0/2511922150 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa86000d570 con 0x7fa8780a86b0 2021-11-21T21:59:16.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa876ffd700 1 -- 172.21.2.116:0/2511922150 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860000e90 con 0x7fa8780a86b0 2021-11-21T21:59:16.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa876ffd700 1 -- 172.21.2.116:0/2511922150 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa86000d8c0 con 0x7fa8780a86b0 2021-11-21T21:59:16.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/2511922150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a86b0 msgr2=0x7fa87814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 --2- 172.21.2.116:0/2511922150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a86b0 0x7fa87814b150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fa878061ab0 tx=0x7fa860006eb0).stop 2021-11-21T21:59:16.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/2511922150 shutdown_connections 2021-11-21T21:59:16.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 --2- 172.21.2.116:0/2511922150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa878062410 0x7fa8781443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 --2- 172.21.2.116:0/2511922150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a86b0 0x7fa87814b150 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:16.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/2511922150 >> 172.21.2.116:0/2511922150 conn(0x7fa87805bd20 msgr2=0x7fa87805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:16.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/2511922150 shutdown_connections 2021-11-21T21:59:16.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/2511922150 wait complete. 2021-11-21T21:59:16.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 Processor -- start 2021-11-21T21:59:16.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- start start 2021-11-21T21:59:16.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa878062410 0x7fa8781438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a6aa0 0x7fa878156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8780a86b0 0x7fa8781567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781458e0 con 0x7fa8780a6aa0 2021-11-21T21:59:16.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878145b10 con 0x7fa878062410 2021-11-21T21:59:16.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa878145db0 con 0x7fa8780a86b0 2021-11-21T21:59:16.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a6aa0 0x7fa878156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a6aa0 0x7fa878156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48274/0 (socket says 172.21.2.116:48274) 2021-11-21T21:59:16.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/1935132033 learned_addr learned my addr 172.21.2.116:0/1935132033 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:16.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8780a86b0 msgr2=0x7fa8781567f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:16.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8780a86b0 0x7fa8781567f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa878062410 msgr2=0x7fa8781438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:16.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa878062410 0x7fa8781438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:16.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 -- 172.21.2.116:0/1935132033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86000b1e0 con 0x7fa8780a6aa0 2021-11-21T21:59:16.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8777fe700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a6aa0 0x7fa878156300 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fa878061ab0 tx=0x7fa860015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8757fa700 1 -- 172.21.2.116:0/1935132033 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa860006b80 con 0x7fa8780a6aa0 2021-11-21T21:59:16.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8757fa700 1 -- 172.21.2.116:0/1935132033 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86000de10 con 0x7fa8780a6aa0 2021-11-21T21:59:16.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa878146050 con 0x7fa8780a6aa0 2021-11-21T21:59:16.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa8757fa700 1 -- 172.21.2.116:0/1935132033 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa86001e460 con 0x7fa8780a6aa0 2021-11-21T21:59:16.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8780a95c0 con 0x7fa8780a6aa0 2021-11-21T21:59:16.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.918+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8781461e0 con 0x7fa8780a6aa0 2021-11-21T21:59:16.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.922+0000 7fa8757fa700 1 -- 172.21.2.116:0/1935132033 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa860025030 con 0x7fa8780a6aa0 2021-11-21T21:59:16.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.922+0000 7fa8757fa700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa85c0535d0 0x7fa85c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:16.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.922+0000 7fa8757fa700 1 -- 172.21.2.116:0/1935132033 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa86006c9a0 con 0x7fa8780a6aa0 2021-11-21T21:59:16.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.922+0000 7fa877fff700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa85c0535d0 0x7fa85c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:16.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.922+0000 7fa877fff700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa85c0535d0 0x7fa85c055a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa868002f90 tx=0x7fa8680069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:16.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:16.922+0000 7fa8757fa700 1 -- 172.21.2.116:0/1935132033 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa860043e30 con 0x7fa8780a6aa0 2021-11-21T21:59:17.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa87805db70 con 0x7fa8780a6aa0 2021-11-21T21:59:17.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa8757fa700 1 -- 172.21.2.116:0/1935132033 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa860023070 con 0x7fa8780a6aa0 2021-11-21T21:59:17.012 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:17.013 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T21:59:17.013 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:17.013 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T21:59:17.013 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T21:59:17.014 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:17.014 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:17.014 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T21:59:17.014 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T21:59:17.015 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:17.015 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T21:59:17.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa85c0535d0 msgr2=0x7fa85c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:17.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa85c0535d0 0x7fa85c055a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa868002f90 tx=0x7fa8680069b0).stop 2021-11-21T21:59:17.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a6aa0 msgr2=0x7fa878156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:17.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a6aa0 0x7fa878156300 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fa878061ab0 tx=0x7fa860015000).stop 2021-11-21T21:59:17.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 shutdown_connections 2021-11-21T21:59:17.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8780a86b0 0x7fa8781567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:17.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa85c0535d0 0x7fa85c055a90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:17.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8780a6aa0 0x7fa878156300 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:17.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 --2- 172.21.2.116:0/1935132033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa878062410 0x7fa8781438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:17.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 >> 172.21.2.116:0/1935132033 conn(0x7fa87805bd20 msgr2=0x7fa878148e90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:17.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 shutdown_connections 2021-11-21T21:59:17.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.010+0000 7fa87e35d700 1 -- 172.21.2.116:0/1935132033 wait complete. 2021-11-21T21:59:17.019 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T21:59:17.020 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T21:59:17.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 Processor -- start 2021-11-21T21:59:17.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- start start 2021-11-21T21:59:17.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff620062e60 0x7ff620142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:17.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 0x7ff6200ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:17.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6200afae0 con 0x7ff6201426e0 2021-11-21T21:59:17.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6200b0f60 con 0x7ff620062e60 2021-11-21T21:59:17.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff620063470 con 0x7ff6200ad470 2021-11-21T21:59:17.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff620062e60 0x7ff620142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:17.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff620062e60 0x7ff620142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48282/0 (socket says 172.21.2.116:48282) 2021-11-21T21:59:17.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 -- 172.21.2.116:0/1419120431 learned_addr learned my addr 172.21.2.116:0/1419120431 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:17.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 -- 172.21.2.116:0/1419120431 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 msgr2=0x7ff6200ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:17.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 --2- 172.21.2.116:0/1419120431 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 0x7ff6200ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:17.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 -- 172.21.2.116:0/1419120431 >> v1:172.21.2.117:6789/0 conn(0x7ff6201426e0 legacy=0x7ff6200af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:17.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 -- 172.21.2.116:0/1419120431 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620143050 con 0x7ff620062e60 2021-11-21T21:59:17.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61dd9b700 1 --2- 172.21.2.116:0/1419120431 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620062e60 0x7ff620142ac0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7ff620062d10 tx=0x7ff610006eb0).ready entity=mon.0 client_cookie=3435f3deb3645451 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:17.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61d59a700 1 -- 172.21.2.116:0/1419120431 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff610010810 con 0x7ff620062e60 2021-11-21T21:59:17.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61d59a700 1 -- 172.21.2.116:0/1419120431 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610000e90 con 0x7ff620062e60 2021-11-21T21:59:17.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff61d59a700 1 -- 172.21.2.116:0/1419120431 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff610010ba0 con 0x7ff620062e60 2021-11-21T21:59:17.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- 172.21.2.116:0/1419120431 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620062e60 msgr2=0x7ff620142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:17.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 --2- 172.21.2.116:0/1419120431 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620062e60 0x7ff620142ac0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7ff620062d10 tx=0x7ff610006eb0).stop 2021-11-21T21:59:17.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- 172.21.2.116:0/1419120431 shutdown_connections 2021-11-21T21:59:17.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 --2- 172.21.2.116:0/1419120431 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 0x7ff6200ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:17.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 --2- 172.21.2.116:0/1419120431 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620062e60 0x7ff620142ac0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:17.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- 172.21.2.116:0/1419120431 >> 172.21.2.116:0/1419120431 conn(0x7ff6200b5730 msgr2=0x7ff6200b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:17.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- 172.21.2.116:0/1419120431 shutdown_connections 2021-11-21T21:59:17.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.070+0000 7ff624da8700 1 -- 172.21.2.116:0/1419120431 wait complete. 2021-11-21T21:59:17.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 Processor -- start 2021-11-21T21:59:17.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 -- start start 2021-11-21T21:59:17.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 0x7ff62013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:17.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff6201426e0 0x7ff620141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:17.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620151100 0x7ff6201515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:17.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff620124de0 con 0x7ff620151100 2021-11-21T21:59:17.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62013eae0 con 0x7ff6201426e0 2021-11-21T21:59:17.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62013f1b0 con 0x7ff6200ad470 2021-11-21T21:59:17.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620151100 0x7ff6201515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:17.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620151100 0x7ff6201515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48288/0 (socket says 172.21.2.116:48288) 2021-11-21T21:59:17.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 -- 172.21.2.116:0/2073776621 learned_addr learned my addr 172.21.2.116:0/2073776621 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:17.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61dd9b700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff6201426e0 0x7ff620141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:17.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 -- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 msgr2=0x7ff62013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:17.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 0x7ff62013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:17.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 -- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff6201426e0 msgr2=0x7ff620141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:17.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff6201426e0 0x7ff620141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:17.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 -- 172.21.2.116:0/2073776621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61000c2f0 con 0x7ff620151100 2021-11-21T21:59:17.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61dd9b700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff6201426e0 0x7ff620141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:17.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61ed9d700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620151100 0x7ff6201515b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7ff6180032f0 tx=0x7ff618009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:17.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff6177fe700 1 -- 172.21.2.116:0/2073776621 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff61800d580 con 0x7ff620151100 2021-11-21T21:59:17.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff6177fe700 1 -- 172.21.2.116:0/2073776621 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61800d6e0 con 0x7ff620151100 2021-11-21T21:59:17.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff6177fe700 1 -- 172.21.2.116:0/2073776621 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff61800da40 con 0x7ff620151100 2021-11-21T21:59:17.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6200672a0 con 0x7ff620151100 2021-11-21T21:59:17.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff620066fd0 con 0x7ff620151100 2021-11-21T21:59:17.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff620066e00 con 0x7ff620151100 2021-11-21T21:59:17.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff6177fe700 1 -- 172.21.2.116:0/2073776621 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff61800dba0 con 0x7ff620151100 2021-11-21T21:59:17.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff6177fe700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff604053580 0x7ff604055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:17.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff6177fe700 1 -- 172.21.2.116:0/2073776621 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff61806b960 con 0x7ff620151100 2021-11-21T21:59:17.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61e59c700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff604053580 0x7ff604055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:17.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.074+0000 7ff61e59c700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff604053580 0x7ff604055a40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7ff608002ef0 tx=0x7ff608006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:17.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.078+0000 7ff6177fe700 1 -- 172.21.2.116:0/2073776621 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff618042df0 con 0x7ff620151100 2021-11-21T21:59:17.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.158+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7ff6200b1e00 con 0x7ff620151100 2021-11-21T21:59:17.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.966+0000 7ff6177fe700 1 -- 172.21.2.116:0/2073776621 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v63) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff618046410 con 0x7ff620151100 2021-11-21T21:59:17.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.966+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff604053580 msgr2=0x7ff604055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:17.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.966+0000 7ff624da8700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff604053580 0x7ff604055a40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7ff608002ef0 tx=0x7ff608006910).stop 2021-11-21T21:59:17.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.966+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620151100 msgr2=0x7ff6201515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:17.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.966+0000 7ff624da8700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620151100 0x7ff6201515b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7ff6180032f0 tx=0x7ff618009220).stop 2021-11-21T21:59:17.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 shutdown_connections 2021-11-21T21:59:17.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff604053580 0x7ff604055a40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:17.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff620151100 0x7ff6201515b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:17.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff6201426e0 0x7ff620141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:17.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 --2- 172.21.2.116:0/2073776621 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6200ad470 0x7ff62013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:17.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 >> 172.21.2.116:0/2073776621 conn(0x7ff6200b5730 msgr2=0x7ff6200b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:17.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 shutdown_connections 2021-11-21T21:59:17.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:17.970+0000 7ff624da8700 1 -- 172.21.2.116:0/2073776621 wait complete. 2021-11-21T21:59:17.974 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T21:59:17.977 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T21:59:18.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 Processor -- start 2021-11-21T21:59:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- start start 2021-11-21T21:59:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7eff880b02e0 0x7eff8805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880afab0 0x7eff88143680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff880b06c0 con 0x7eff880af620 2021-11-21T21:59:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880adf20 con 0x7eff880b02e0 2021-11-21T21:59:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880b2470 con 0x7eff880afab0 2021-11-21T21:59:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7eff880b02e0 0x7eff8805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7eff880b02e0 0x7eff8805f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48296/0 (socket says 172.21.2.116:48296) 2021-11-21T21:59:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 -- 172.21.2.116:0/657496226 learned_addr learned my addr 172.21.2.116:0/657496226 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 -- 172.21.2.116:0/657496226 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880afab0 msgr2=0x7eff88143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 --2- 172.21.2.116:0/657496226 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880afab0 0x7eff88143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 -- 172.21.2.116:0/657496226 >> v1:172.21.2.117:6789/0 conn(0x7eff880af620 legacy=0x7eff88142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 -- 172.21.2.116:0/657496226 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880b26d0 con 0x7eff880b02e0 2021-11-21T21:59:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8daee700 1 --2- 172.21.2.116:0/657496226 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880b02e0 0x7eff8805f730 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7eff88063350 tx=0x7eff84006eb0).ready entity=mon.0 client_cookie=eabebd26f5f16533 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8caec700 1 -- 172.21.2.116:0/657496226 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff8400d570 con 0x7eff880b02e0 2021-11-21T21:59:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8caec700 1 -- 172.21.2.116:0/657496226 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff84000e90 con 0x7eff880b02e0 2021-11-21T21:59:18.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8caec700 1 -- 172.21.2.116:0/657496226 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff8400d8c0 con 0x7eff880b02e0 2021-11-21T21:59:18.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- 172.21.2.116:0/657496226 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880b02e0 msgr2=0x7eff8805f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:18.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- 172.21.2.116:0/657496226 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880b02e0 0x7eff8805f730 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7eff88063350 tx=0x7eff84006eb0).stop 2021-11-21T21:59:18.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- 172.21.2.116:0/657496226 shutdown_connections 2021-11-21T21:59:18.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- 172.21.2.116:0/657496226 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880afab0 0x7eff88143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:18.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- 172.21.2.116:0/657496226 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880b02e0 0x7eff8805f730 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:18.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- 172.21.2.116:0/657496226 >> 172.21.2.116:0/657496226 conn(0x7eff880b5ff0 msgr2=0x7eff880b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:18.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- 172.21.2.116:0/657496226 shutdown_connections 2021-11-21T21:59:18.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- 172.21.2.116:0/657496226 wait complete. 2021-11-21T21:59:18.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 Processor -- start 2021-11-21T21:59:18.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- start start 2021-11-21T21:59:18.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff880af620 0x7eff880a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:18.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880afab0 0x7eff880a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:18.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880b02e0 0x7eff8813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:18.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8813dcf0 con 0x7eff880afab0 2021-11-21T21:59:18.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff880a2950 con 0x7eff880af620 2021-11-21T21:59:18.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880a0e00 con 0x7eff880b02e0 2021-11-21T21:59:18.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880afab0 0x7eff880a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:18.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880afab0 0x7eff880a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48298/0 (socket says 172.21.2.116:48298) 2021-11-21T21:59:18.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 -- 172.21.2.116:0/4055094424 learned_addr learned my addr 172.21.2.116:0/4055094424 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:18.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 -- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880b02e0 msgr2=0x7eff8813cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:18.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880b02e0 0x7eff8813cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:18.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 -- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff880af620 msgr2=0x7eff880a2000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:18.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff880af620 0x7eff880a2000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:18.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 -- 172.21.2.116:0/4055094424 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8400b1e0 con 0x7eff880afab0 2021-11-21T21:59:18.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8d2ed700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880afab0 0x7eff880a01d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7eff78002f90 tx=0x7eff78006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:18.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff76ffd700 1 -- 172.21.2.116:0/4055094424 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff78013460 con 0x7eff880afab0 2021-11-21T21:59:18.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff76ffd700 1 -- 172.21.2.116:0/4055094424 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff780135c0 con 0x7eff880afab0 2021-11-21T21:59:18.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff76ffd700 1 -- 172.21.2.116:0/4055094424 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff78013920 con 0x7eff880afab0 2021-11-21T21:59:18.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff880a26b0 con 0x7eff880afab0 2021-11-21T21:59:18.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.030+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff880a19c0 con 0x7eff880afab0 2021-11-21T21:59:18.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.034+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff880b0da0 con 0x7eff880afab0 2021-11-21T21:59:18.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.034+0000 7eff76ffd700 1 -- 172.21.2.116:0/4055094424 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7eff7800f070 con 0x7eff880afab0 2021-11-21T21:59:18.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.034+0000 7eff76ffd700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff680535b0 0x7eff68055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:18.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.034+0000 7eff76ffd700 1 -- 172.21.2.116:0/4055094424 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5603+0+0 (secure 0 0 0) 0x7eff7806b020 con 0x7eff880afab0 2021-11-21T21:59:18.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.034+0000 7eff8daee700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff680535b0 0x7eff68055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:18.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.034+0000 7eff8daee700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff680535b0 0x7eff68055a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7eff88063350 tx=0x7eff84015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:18.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.034+0000 7eff76ffd700 1 -- 172.21.2.116:0/4055094424 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7eff78042240 con 0x7eff880afab0 2021-11-21T21:59:18.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.122+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7eff880573d0 con 0x7eff880afab0 2021-11-21T21:59:18.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.978+0000 7eff76ffd700 1 -- 172.21.2.116:0/4055094424 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v64) v1 ==== 125+0+0 (secure 0 0 0) 0x7eff78045860 con 0x7eff880afab0 2021-11-21T21:59:18.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff680535b0 msgr2=0x7eff68055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:18.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff680535b0 0x7eff68055a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7eff88063350 tx=0x7eff84015000).stop 2021-11-21T21:59:18.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880afab0 msgr2=0x7eff880a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:18.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880afab0 0x7eff880a01d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7eff78002f90 tx=0x7eff78006df0).stop 2021-11-21T21:59:18.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 shutdown_connections 2021-11-21T21:59:18.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff880b02e0 0x7eff8813cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:18.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff680535b0 0x7eff68055a70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:18.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff880afab0 0x7eff880a01d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:18.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 --2- 172.21.2.116:0/4055094424 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff880af620 0x7eff880a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:18.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 >> 172.21.2.116:0/4055094424 conn(0x7eff880b5ff0 msgr2=0x7eff880b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:18.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 shutdown_connections 2021-11-21T21:59:18.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:18.982+0000 7eff8fd52700 1 -- 172.21.2.116:0/4055094424 wait complete. 2021-11-21T21:59:18.987 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T21:59:18.991 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T21:59:19.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 Processor -- start 2021-11-21T21:59:19.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- start start 2021-11-21T21:59:19.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 0x7fd3880b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd3880aed90 0x7fd38813c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880b00d0 con 0x7fd3880af360 2021-11-21T21:59:19.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3880adba0 con 0x7fd3880aed90 2021-11-21T21:59:19.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880af7a0 con 0x7fd3880b24a0 2021-11-21T21:59:19.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38d29e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd3880af360 0x7fd3880ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60262/0 (socket says 172.21.2.116:60262) 2021-11-21T21:59:19.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38d29e700 1 -- 172.21.2.116:0/2271765451 learned_addr learned my addr 172.21.2.116:0/2271765451 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:19.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38e2a0700 1 --2- 172.21.2.116:0/2271765451 >> v2:172.21.2.117:3300/0 conn(0x7fd3880aed90 0x7fd38813c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 0x7fd3880b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236485730 0 0) 0x7fd3880b00d0 con 0x7fd3880af360 2021-11-21T21:59:19.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd370003260 con 0x7fd3880af360 2021-11-21T21:59:19.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677085945 0 0) 0x7fd370003260 con 0x7fd3880af360 2021-11-21T21:59:19.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 msgr2=0x7fd3880b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 0x7fd3880b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 >> v2:172.21.2.117:3300/0 conn(0x7fd3880aed90 msgr2=0x7fd38813c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 --2- 172.21.2.116:0/2271765451 >> v2:172.21.2.117:3300/0 conn(0x7fd3880aed90 0x7fd38813c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880b2880 con 0x7fd3880af360 2021-11-21T21:59:19.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd378001660 con 0x7fd3880af360 2021-11-21T21:59:19.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 >> v1:172.21.2.116:6789/0 conn(0x7fd3880af360 legacy=0x7fd3880ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3700037d0 0x7fd370003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd370004500 0x7fd370004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 0x7fd370004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880adba0 con 0x7fd3700037d0 2021-11-21T21:59:19.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880af7a0 con 0x7fd370004500 2021-11-21T21:59:19.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37000b750 con 0x7fd3880b24a0 2021-11-21T21:59:19.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3700037d0 0x7fd370003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38e2a0700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 0x7fd370004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38d29e700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd370004500 0x7fd370004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 -- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 msgr2=0x7fd370004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd3880b24a0 0x7fd370004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 -- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd370004500 msgr2=0x7fd370004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd370004500 0x7fd370004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 -- 172.21.2.116:0/2271765451 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370003260 con 0x7fd3700037d0 2021-11-21T21:59:19.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38d29e700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd370004500 0x7fd370004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:19.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38da9f700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3700037d0 0x7fd370003c10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fd38400c000 tx=0x7fd384006df0).ready entity=mon.0 client_cookie=2fbc01f9815bf2e2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd38400e830 con 0x7fd3700037d0 2021-11-21T21:59:19.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38400e990 con 0x7fd3700037d0 2021-11-21T21:59:19.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38ca9d700 1 -- 172.21.2.116:0/2271765451 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd38400ecf0 con 0x7fd3700037d0 2021-11-21T21:59:19.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- 172.21.2.116:0/2271765451 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3700037d0 msgr2=0x7fd370003c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 --2- 172.21.2.116:0/2271765451 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3700037d0 0x7fd370003c10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fd38400c000 tx=0x7fd384006df0).stop 2021-11-21T21:59:19.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38e2a0700 1 -- 172.21.2.116:0/2271765451 reap_dead start 2021-11-21T21:59:19.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- 172.21.2.116:0/2271765451 shutdown_connections 2021-11-21T21:59:19.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- 172.21.2.116:0/2271765451 >> 172.21.2.116:0/2271765451 conn(0x7fd38805fd00 msgr2=0x7fd3880600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:19.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- 172.21.2.116:0/2271765451 shutdown_connections 2021-11-21T21:59:19.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- 172.21.2.116:0/2271765451 wait complete. 2021-11-21T21:59:19.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 Processor -- start 2021-11-21T21:59:19.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.042+0000 7fd38fd03700 1 -- start start 2021-11-21T21:59:19.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3700037d0 0x7fd3880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3880b3700 0x7fd388065c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd38814d650 0x7fd388066120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd388064930 con 0x7fd3880b3700 2021-11-21T21:59:19.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880b4980 con 0x7fd3700037d0 2021-11-21T21:59:19.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880b4010 con 0x7fd38814d650 2021-11-21T21:59:19.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3880b3700 0x7fd388065c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3880b3700 0x7fd388065c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48320/0 (socket says 172.21.2.116:48320) 2021-11-21T21:59:19.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 -- 172.21.2.116:0/44599023 learned_addr learned my addr 172.21.2.116:0/44599023 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:19.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38e2a0700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd38814d650 0x7fd388066120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 -- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd38814d650 msgr2=0x7fd388066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd38814d650 0x7fd388066120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 -- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3700037d0 msgr2=0x7fd3880b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3700037d0 0x7fd3880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 -- 172.21.2.116:0/44599023 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388066ea0 con 0x7fd3880b3700 2021-11-21T21:59:19.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38d29e700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3880b3700 0x7fd388065c30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fd3880657c0 tx=0x7fd378009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd376ffd700 1 -- 172.21.2.116:0/44599023 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3780071d0 con 0x7fd3880b3700 2021-11-21T21:59:19.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd376ffd700 1 -- 172.21.2.116:0/44599023 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd378007330 con 0x7fd3880b3700 2021-11-21T21:59:19.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3880b3c80 con 0x7fd3880b3700 2021-11-21T21:59:19.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd376ffd700 1 -- 172.21.2.116:0/44599023 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd378004920 con 0x7fd3880b3700 2021-11-21T21:59:19.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3880b44e0 con 0x7fd3880b3700 2021-11-21T21:59:19.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3880573d0 con 0x7fd3880b3700 2021-11-21T21:59:19.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd376ffd700 1 -- 172.21.2.116:0/44599023 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd37801a020 con 0x7fd3880b3700 2021-11-21T21:59:19.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd376ffd700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd3680536d0 0x7fd368055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd376ffd700 1 -- 172.21.2.116:0/44599023 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fd37806ba70 con 0x7fd3880b3700 2021-11-21T21:59:19.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38da9f700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd3680536d0 0x7fd368055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.046+0000 7fd38da9f700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd3680536d0 0x7fd368055b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fd38400c000 tx=0x7fd384019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.050+0000 7fd376ffd700 1 -- 172.21.2.116:0/44599023 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd378042c40 con 0x7fd3880b3700 2021-11-21T21:59:19.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.158+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd38805f2d0 con 0x7fd3880b3700 2021-11-21T21:59:19.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd376ffd700 1 -- 172.21.2.116:0/44599023 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v27) v1 ==== 189+0+0 (secure 0 0 0) 0x7fd37806be60 con 0x7fd3880b3700 2021-11-21T21:59:19.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd3680536d0 msgr2=0x7fd368055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd3680536d0 0x7fd368055b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fd38400c000 tx=0x7fd384019000).stop 2021-11-21T21:59:19.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3880b3700 msgr2=0x7fd388065c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3880b3700 0x7fd388065c30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fd3880657c0 tx=0x7fd378009040).stop 2021-11-21T21:59:19.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 shutdown_connections 2021-11-21T21:59:19.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd38814d650 0x7fd388066120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd3680536d0 0x7fd368055b90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd3880b3700 0x7fd388065c30 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 --2- 172.21.2.116:0/44599023 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd3700037d0 0x7fd3880b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 >> 172.21.2.116:0/44599023 conn(0x7fd38805fd00 msgr2=0x7fd3880699d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:19.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 shutdown_connections 2021-11-21T21:59:19.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.166+0000 7fd38fd03700 1 -- 172.21.2.116:0/44599023 wait complete. 2021-11-21T21:59:19.173 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 11 and data pool 12 2021-11-21T21:59:19.177 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:59:19.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 Processor -- start 2021-11-21T21:59:19.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- start start 2021-11-21T21:59:19.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb03c0aed90 0x7fb03c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0b00d0 con 0x7fb03c0b24a0 2021-11-21T21:59:19.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0b2880 con 0x7fb03c0af360 2021-11-21T21:59:19.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c0adba0 con 0x7fb03c0aed90 2021-11-21T21:59:19.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03a59c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb03c0b24a0 0x7fb03c0ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60294/0 (socket says 172.21.2.116:60294) 2021-11-21T21:59:19.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03a59c700 1 -- 172.21.2.116:0/3847373264 learned_addr learned my addr 172.21.2.116:0/3847373264 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:19.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882163499 0 0) 0x7fb03c0b00d0 con 0x7fb03c0b24a0 2021-11-21T21:59:19.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb01c003260 con 0x7fb03c0b24a0 2021-11-21T21:59:19.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 --2- 172.21.2.116:0/3847373264 >> v2:172.21.2.117:3300/0 conn(0x7fb03c0aed90 0x7fb03c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3865621210 0 0) 0x7fb01c003260 con 0x7fb03c0b24a0 2021-11-21T21:59:19.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 >> v2:172.21.2.117:3300/0 conn(0x7fb03c0aed90 msgr2=0x7fb03c0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 --2- 172.21.2.116:0/3847373264 >> v2:172.21.2.117:3300/0 conn(0x7fb03c0aed90 0x7fb03c0b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 >> v1:172.21.2.117:6789/0 conn(0x7fb03c0af360 legacy=0x7fb03c13c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0b14a0 con 0x7fb03c0b24a0 2021-11-21T21:59:19.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb024001660 con 0x7fb03c0b24a0 2021-11-21T21:59:19.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 >> v1:172.21.2.116:6789/0 conn(0x7fb03c0b24a0 legacy=0x7fb03c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb01c003750 0x7fb01c003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb01c004500 0x7fb01c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb01c004e70 0x7fb01c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0adba0 con 0x7fb01c004500 2021-11-21T21:59:19.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c00bd20 con 0x7fb01c003750 2021-11-21T21:59:19.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c00bee0 con 0x7fb01c004e70 2021-11-21T21:59:19.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb01c004500 0x7fb01c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03a59c700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb01c003750 0x7fb01c003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 -- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb01c004e70 msgr2=0x7fb01c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb01c004e70 0x7fb01c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 -- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb01c003750 msgr2=0x7fb01c003b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb01c003750 0x7fb01c003b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 -- 172.21.2.116:0/3847373264 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c003260 con 0x7fb01c004500 2021-11-21T21:59:19.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03a59c700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb01c003750 0x7fb01c003b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:19.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03ad9d700 1 -- 172.21.2.116:0/3847373264 reap_dead start 2021-11-21T21:59:19.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb039d9b700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb01c004500 0x7fb01c004980 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fb02c00d000 tx=0x7fb02c008040).ready entity=mon.0 client_cookie=eb8b13634123075c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb02c00e720 con 0x7fb01c004500 2021-11-21T21:59:19.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c00e880 con 0x7fb01c004500 2021-11-21T21:59:19.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb03959a700 1 -- 172.21.2.116:0/3847373264 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb02c00ebe0 con 0x7fb01c004500 2021-11-21T21:59:19.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- 172.21.2.116:0/3847373264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb01c004500 msgr2=0x7fb01c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb01c004500 0x7fb01c004980 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fb02c00d000 tx=0x7fb02c008040).stop 2021-11-21T21:59:19.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- 172.21.2.116:0/3847373264 shutdown_connections 2021-11-21T21:59:19.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 --2- 172.21.2.116:0/3847373264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb01c004500 0x7fb01c004980 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- 172.21.2.116:0/3847373264 >> 172.21.2.116:0/3847373264 conn(0x7fb03c05fd00 msgr2=0x7fb03c0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:19.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- 172.21.2.116:0/3847373264 shutdown_connections 2021-11-21T21:59:19.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- 172.21.2.116:0/3847373264 wait complete. 2021-11-21T21:59:19.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 Processor -- start 2021-11-21T21:59:19.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.226+0000 7fb040995700 1 -- start start 2021-11-21T21:59:19.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb01c004500 0x7fb03c137300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb03c065eb0 0x7fb03c141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb03c063ea0 0x7fb03c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c141580 con 0x7fb03c063ea0 2021-11-21T21:59:19.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c151ad0 con 0x7fb03c065eb0 2021-11-21T21:59:19.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c151d70 con 0x7fb01c004500 2021-11-21T21:59:19.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb03c063ea0 0x7fb03c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb03c063ea0 0x7fb03c14db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48350/0 (socket says 172.21.2.116:48350) 2021-11-21T21:59:19.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 -- 172.21.2.116:0/3183560038 learned_addr learned my addr 172.21.2.116:0/3183560038 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:19.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 -- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb01c004500 msgr2=0x7fb03c137300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb01c004500 0x7fb03c137300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 -- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb03c065eb0 msgr2=0x7fb03c141090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:19.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb03c065eb0 0x7fb03c141090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 -- 172.21.2.116:0/3183560038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c140bf0 con 0x7fb03c063ea0 2021-11-21T21:59:19.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03ad9d700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb03c063ea0 0x7fb03c14db20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fb024001430 tx=0x7fb034007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb0337fe700 1 -- 172.21.2.116:0/3183560038 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb034005880 con 0x7fb03c063ea0 2021-11-21T21:59:19.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb0337fe700 1 -- 172.21.2.116:0/3183560038 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0340059e0 con 0x7fb03c063ea0 2021-11-21T21:59:19.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c137bc0 con 0x7fb03c063ea0 2021-11-21T21:59:19.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb0337fe700 1 -- 172.21.2.116:0/3183560038 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb034005d80 con 0x7fb03c063ea0 2021-11-21T21:59:19.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c145ab0 con 0x7fb03c063ea0 2021-11-21T21:59:19.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c151f30 con 0x7fb03c063ea0 2021-11-21T21:59:19.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb0337fe700 1 -- 172.21.2.116:0/3183560038 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb034010070 con 0x7fb03c063ea0 2021-11-21T21:59:19.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb0337fe700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0200536d0 0x7fb020055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb0337fe700 1 -- 172.21.2.116:0/3183560038 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb03406add0 con 0x7fb03c063ea0 2021-11-21T21:59:19.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03a59c700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0200536d0 0x7fb020055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb03a59c700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0200536d0 0x7fb020055b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fb024008000 tx=0x7fb024009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.230+0000 7fb0337fe700 1 -- 172.21.2.116:0/3183560038 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb034041f30 con 0x7fb03c063ea0 2021-11-21T21:59:19.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb03c140d80 con 0x7fb03c063ea0 2021-11-21T21:59:19.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb0337fe700 1 -- 172.21.2.116:0/3183560038 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16054 (secure 0 0 0) 0x7fb034045550 con 0x7fb03c063ea0 2021-11-21T21:59:19.321 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:19.322 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:59:19.164503+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:17.175021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-11-21T21:59:18.135656+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:59:19.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0200536d0 msgr2=0x7fb020055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0200536d0 0x7fb020055b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fb024008000 tx=0x7fb024009040).stop 2021-11-21T21:59:19.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb03c063ea0 msgr2=0x7fb03c14db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb03c063ea0 0x7fb03c14db20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fb024001430 tx=0x7fb034007be0).stop 2021-11-21T21:59:19.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 shutdown_connections 2021-11-21T21:59:19.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb0200536d0 0x7fb020055b90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb03c063ea0 0x7fb03c14db20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb03c065eb0 0x7fb03c141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 --2- 172.21.2.116:0/3183560038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb01c004500 0x7fb03c137300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 >> 172.21.2.116:0/3183560038 conn(0x7fb03c05fd00 msgr2=0x7fb03c069680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:19.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 shutdown_connections 2021-11-21T21:59:19.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.318+0000 7fb040995700 1 -- 172.21.2.116:0/3183560038 wait complete. 2021-11-21T21:59:19.329 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:59:19.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.374+0000 7f707a179700 1 Processor -- start 2021-11-21T21:59:19.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.374+0000 7f707a179700 1 -- start start 2021-11-21T21:59:19.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f70740a6aa0 0x7f707414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7074062410 0x7f70741443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70740a6e80 con 0x7f70740a86b0 2021-11-21T21:59:19.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70740a8bc0 con 0x7f70740a6aa0 2021-11-21T21:59:19.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707414a7b0 con 0x7f7074062410 2021-11-21T21:59:19.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f70740a6aa0 0x7f707414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f70740a6aa0 0x7f707414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48358/0 (socket says 172.21.2.116:48358) 2021-11-21T21:59:19.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 -- 172.21.2.116:0/12132982 learned_addr learned my addr 172.21.2.116:0/12132982 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:19.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 -- 172.21.2.116:0/12132982 >> v2:172.21.2.117:3300/0 conn(0x7f7074062410 msgr2=0x7f70741443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:19.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 --2- 172.21.2.116:0/12132982 >> v2:172.21.2.117:3300/0 conn(0x7f7074062410 0x7f70741443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 -- 172.21.2.116:0/12132982 >> v1:172.21.2.117:6789/0 conn(0x7f70740a86b0 legacy=0x7f70741470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:19.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 -- 172.21.2.116:0/12132982 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707414aa10 con 0x7f70740a6aa0 2021-11-21T21:59:19.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70737fe700 1 --2- 172.21.2.116:0/12132982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a6aa0 0x7f707414b150 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7074061ab0 tx=0x7f7064006eb0).ready entity=mon.0 client_cookie=6c53bf052af67dce server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70727fc700 1 -- 172.21.2.116:0/12132982 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706400d5c0 con 0x7f70740a6aa0 2021-11-21T21:59:19.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70727fc700 1 -- 172.21.2.116:0/12132982 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7064000e90 con 0x7f70740a6aa0 2021-11-21T21:59:19.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f70727fc700 1 -- 172.21.2.116:0/12132982 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706400d910 con 0x7f70740a6aa0 2021-11-21T21:59:19.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- 172.21.2.116:0/12132982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a6aa0 msgr2=0x7f707414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- 172.21.2.116:0/12132982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a6aa0 0x7f707414b150 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7074061ab0 tx=0x7f7064006eb0).stop 2021-11-21T21:59:19.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- 172.21.2.116:0/12132982 shutdown_connections 2021-11-21T21:59:19.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- 172.21.2.116:0/12132982 >> v2:172.21.2.117:3300/0 conn(0x7f7074062410 0x7f70741443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- 172.21.2.116:0/12132982 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a6aa0 0x7f707414b150 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- 172.21.2.116:0/12132982 >> 172.21.2.116:0/12132982 conn(0x7f707405bd20 msgr2=0x7f707405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:19.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- 172.21.2.116:0/12132982 shutdown_connections 2021-11-21T21:59:19.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- 172.21.2.116:0/12132982 wait complete. 2021-11-21T21:59:19.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 Processor -- start 2021-11-21T21:59:19.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- start start 2021-11-21T21:59:19.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7074062410 0x7f70740ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70740a6aa0 0x7f707415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a86b0 0x7f7074156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70740638f0 con 0x7f70740a86b0 2021-11-21T21:59:19.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7074063b50 con 0x7f70740a6aa0 2021-11-21T21:59:19.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7074063df0 con 0x7f7074062410 2021-11-21T21:59:19.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a86b0 0x7f7074156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a86b0 0x7f7074156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48368/0 (socket says 172.21.2.116:48368) 2021-11-21T21:59:19.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 -- 172.21.2.116:0/2089176546 learned_addr learned my addr 172.21.2.116:0/2089176546 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:19.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 -- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7074062410 msgr2=0x7f70740ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7074062410 0x7f70740ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 -- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70740a6aa0 msgr2=0x7f707415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70740a6aa0 0x7f707415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 -- 172.21.2.116:0/2089176546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706400b1e0 con 0x7f70740a86b0 2021-11-21T21:59:19.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7073fff700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a86b0 0x7f7074156300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f706c003330 tx=0x7f706c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7070ff9700 1 -- 172.21.2.116:0/2089176546 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706c001520 con 0x7f70740a86b0 2021-11-21T21:59:19.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7070ff9700 1 -- 172.21.2.116:0/2089176546 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c001680 con 0x7f70740a86b0 2021-11-21T21:59:19.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7074148760 con 0x7f70740a86b0 2021-11-21T21:59:19.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f7070ff9700 1 -- 172.21.2.116:0/2089176546 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706c0019e0 con 0x7f70740a86b0 2021-11-21T21:59:19.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.378+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70740675c0 con 0x7f70740a86b0 2021-11-21T21:59:19.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.382+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70741488f0 con 0x7f70740a86b0 2021-11-21T21:59:19.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.382+0000 7f7070ff9700 1 -- 172.21.2.116:0/2089176546 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f706c018030 con 0x7f70740a86b0 2021-11-21T21:59:19.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.382+0000 7f7070ff9700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7058053660 0x7f7058055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.382+0000 7f7070ff9700 1 -- 172.21.2.116:0/2089176546 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f706c06b4b0 con 0x7f70740a86b0 2021-11-21T21:59:19.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.382+0000 7f70737fe700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7058053660 0x7f7058055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.382+0000 7f70737fe700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7058053660 0x7f7058055b20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f7074061ab0 tx=0x7f7064015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.382+0000 7f7070ff9700 1 -- 172.21.2.116:0/2089176546 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f706c0425b0 con 0x7f70740a86b0 2021-11-21T21:59:19.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f707405db70 con 0x7f70740a86b0 2021-11-21T21:59:19.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f7070ff9700 1 -- 172.21.2.116:0/2089176546 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f706c045bd0 con 0x7f70740a86b0 2021-11-21T21:59:19.470 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:19.471 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:59:19.164503+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:17.175021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-11-21T21:59:18.135656+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:59:19.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7058053660 msgr2=0x7f7058055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7058053660 0x7f7058055b20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f7074061ab0 tx=0x7f7064015000).stop 2021-11-21T21:59:19.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a86b0 msgr2=0x7f7074156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a86b0 0x7f7074156300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f706c003330 tx=0x7f706c0091a0).stop 2021-11-21T21:59:19.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 shutdown_connections 2021-11-21T21:59:19.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7058053660 0x7f7058055b20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70740a86b0 0x7f7074156300 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70740a6aa0 0x7f707415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 --2- 172.21.2.116:0/2089176546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7074062410 0x7f70740ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 >> 172.21.2.116:0/2089176546 conn(0x7f707405bd20 msgr2=0x7f70741426b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:19.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 shutdown_connections 2021-11-21T21:59:19.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.466+0000 7f707a179700 1 -- 172.21.2.116:0/2089176546 wait complete. 2021-11-21T21:59:19.477 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T21:59:19.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 Processor -- start 2021-11-21T21:59:19.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 -- start start 2021-11-21T21:59:19.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd79c142ac0 0x7fd79c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c0ae070 con 0x7fd79c1426e0 2021-11-21T21:59:19.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c0afae0 con 0x7fd79c062e60 2021-11-21T21:59:19.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c0b0f60 con 0x7fd79c142ac0 2021-11-21T21:59:19.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd79bfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd79c1426e0 0x7fd79c0af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60328/0 (socket says 172.21.2.116:60328) 2021-11-21T21:59:19.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd79bfff700 1 -- 172.21.2.116:0/2638839442 learned_addr learned my addr 172.21.2.116:0/2638839442 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:19.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a0929700 1 --2- 172.21.2.116:0/2638839442 >> v2:172.21.2.116:3300/0 conn(0x7fd79c142ac0 0x7fd79c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd79affd700 1 -- 172.21.2.116:0/2638839442 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503889307 0 0) 0x7fd784001110 con 0x7fd79c1426e0 2021-11-21T21:59:19.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd79affd700 1 -- 172.21.2.116:0/2638839442 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd77c003260 con 0x7fd79c1426e0 2021-11-21T21:59:19.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a0929700 1 -- 172.21.2.116:0/2638839442 >> v1:172.21.2.117:6789/0 conn(0x7fd79c062e60 legacy=0x7fd79c0ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:19.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a0929700 1 -- 172.21.2.116:0/2638839442 >> v1:172.21.2.116:6789/0 conn(0x7fd79c1426e0 legacy=0x7fd79c0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a0929700 1 -- 172.21.2.116:0/2638839442 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c063470 con 0x7fd79c142ac0 2021-11-21T21:59:19.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a0929700 1 --2- 172.21.2.116:0/2638839442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c0adb80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fd79c062db0 tx=0x7fd7940091d0).ready entity=mon.0 client_cookie=67f523657dfadeb8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd79affd700 1 -- 172.21.2.116:0/2638839442 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79400b680 con 0x7fd79c142ac0 2021-11-21T21:59:19.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd79affd700 1 -- 172.21.2.116:0/2638839442 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79400b7e0 con 0x7fd79c142ac0 2021-11-21T21:59:19.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd79affd700 1 -- 172.21.2.116:0/2638839442 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79400b950 con 0x7fd79c142ac0 2021-11-21T21:59:19.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 -- 172.21.2.116:0/2638839442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 msgr2=0x7fd79c0adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.526+0000 7fd7a238c700 1 --2- 172.21.2.116:0/2638839442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c0adb80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fd79c062db0 tx=0x7fd7940091d0).stop 2021-11-21T21:59:19.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- 172.21.2.116:0/2638839442 shutdown_connections 2021-11-21T21:59:19.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 --2- 172.21.2.116:0/2638839442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c0adb80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:19.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- 172.21.2.116:0/2638839442 >> 172.21.2.116:0/2638839442 conn(0x7fd79c0b5730 msgr2=0x7fd79c0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:19.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- 172.21.2.116:0/2638839442 shutdown_connections 2021-11-21T21:59:19.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- 172.21.2.116:0/2638839442 wait complete. 2021-11-21T21:59:19.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 Processor -- start 2021-11-21T21:59:19.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- start start 2021-11-21T21:59:19.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd79c1426e0 0x7fd79c1414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c13de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd79c13d170 0x7fd79c13e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c14d9d0 con 0x7fd79c142ac0 2021-11-21T21:59:19.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c1565d0 con 0x7fd79c13d170 2021-11-21T21:59:19.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c13ffb0 con 0x7fd79c1426e0 2021-11-21T21:59:19.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c13de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c13de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48380/0 (socket says 172.21.2.116:48380) 2021-11-21T21:59:19.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 -- 172.21.2.116:0/1034471131 learned_addr learned my addr 172.21.2.116:0/1034471131 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:19.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 -- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd79c1426e0 msgr2=0x7fd79c1414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd79c1426e0 0x7fd79c1414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 -- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd79c13d170 msgr2=0x7fd79c13e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:19.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd79c13d170 0x7fd79c13e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:19.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 -- 172.21.2.116:0/1034471131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd794009470 con 0x7fd79c142ac0 2021-11-21T21:59:19.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd79b7fe700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c13de20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd79c14d780 tx=0x7fd78c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7997fa700 1 -- 172.21.2.116:0/1034471131 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd78c013460 con 0x7fd79c142ac0 2021-11-21T21:59:19.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7997fa700 1 -- 172.21.2.116:0/1034471131 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c0135c0 con 0x7fd79c142ac0 2021-11-21T21:59:19.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7997fa700 1 -- 172.21.2.116:0/1034471131 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd78c013920 con 0x7fd79c142ac0 2021-11-21T21:59:19.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c13d880 con 0x7fd79c142ac0 2021-11-21T21:59:19.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c063920 con 0x7fd79c142ac0 2021-11-21T21:59:19.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c156720 con 0x7fd79c142ac0 2021-11-21T21:59:19.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7997fa700 1 -- 172.21.2.116:0/1034471131 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd78c01a060 con 0x7fd79c142ac0 2021-11-21T21:59:19.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7997fa700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd780053660 0x7fd780055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:19.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.530+0000 7fd7997fa700 1 -- 172.21.2.116:0/1034471131 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd78c06b350 con 0x7fd79c142ac0 2021-11-21T21:59:19.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.534+0000 7fd79bfff700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd780053660 0x7fd780055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:19.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.534+0000 7fd79bfff700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd780053660 0x7fd780055b20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fd7840012a0 tx=0x7fd784001180).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:19.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.534+0000 7fd7997fa700 1 -- 172.21.2.116:0/1034471131 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd79c156720 con 0x7fd79c142ac0 2021-11-21T21:59:19.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:19.642+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd79c156720 con 0x7fd79c142ac0 2021-11-21T21:59:20.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7997fa700 1 -- 172.21.2.116:0/1034471131 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd79c156720 con 0x7fd79c142ac0 2021-11-21T21:59:20.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd780053660 msgr2=0x7fd780055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd780053660 0x7fd780055b20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fd7840012a0 tx=0x7fd784001180).stop 2021-11-21T21:59:20.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 msgr2=0x7fd79c13de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c13de20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd79c14d780 tx=0x7fd78c006df0).stop 2021-11-21T21:59:20.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 shutdown_connections 2021-11-21T21:59:20.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd79c13d170 0x7fd79c13e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd780053660 0x7fd780055b20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd79c142ac0 0x7fd79c13de20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 --2- 172.21.2.116:0/1034471131 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd79c1426e0 0x7fd79c1414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 >> 172.21.2.116:0/1034471131 conn(0x7fd79c0b5730 msgr2=0x7fd79c13c310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 shutdown_connections 2021-11-21T21:59:20.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.174+0000 7fd7a238c700 1 -- 172.21.2.116:0/1034471131 wait complete. 2021-11-21T21:59:20.185 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:20.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 Processor -- start 2021-11-21T21:59:20.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- start start 2021-11-21T21:59:20.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7e980a6aa0 0x7f7e9814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a86b0 0x7f7e981470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7e981475d0 0x7f7e981443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980a8bc0 con 0x7f7e980a6aa0 2021-11-21T21:59:20.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980a6e80 con 0x7f7e981475d0 2021-11-21T21:59:20.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9814a7b0 con 0x7f7e980a86b0 2021-11-21T21:59:20.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7e980a6aa0 0x7f7e9814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7e980a6aa0 0x7f7e9814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48386/0 (socket says 172.21.2.116:48386) 2021-11-21T21:59:20.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 -- 172.21.2.116:0/598937374 learned_addr learned my addr 172.21.2.116:0/598937374 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 -- 172.21.2.116:0/598937374 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a86b0 msgr2=0x7f7e981470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 --2- 172.21.2.116:0/598937374 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a86b0 0x7f7e981470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 -- 172.21.2.116:0/598937374 >> v2:172.21.2.117:3300/0 conn(0x7f7e981475d0 msgr2=0x7f7e981443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 --2- 172.21.2.116:0/598937374 >> v2:172.21.2.117:3300/0 conn(0x7f7e981475d0 0x7f7e981443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 -- 172.21.2.116:0/598937374 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9814aa10 con 0x7f7e980a6aa0 2021-11-21T21:59:20.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9d790700 1 --2- 172.21.2.116:0/598937374 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a6aa0 0x7f7e9814b150 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f7e8c002f70 tx=0x7f7e8c006eb0).ready entity=mon.0 client_cookie=b65fd3a3c102e1ab server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e8bfff700 1 -- 172.21.2.116:0/598937374 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e8c00d570 con 0x7f7e980a6aa0 2021-11-21T21:59:20.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e8bfff700 1 -- 172.21.2.116:0/598937374 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c000e90 con 0x7f7e980a6aa0 2021-11-21T21:59:20.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e8bfff700 1 -- 172.21.2.116:0/598937374 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e8c00d900 con 0x7f7e980a6aa0 2021-11-21T21:59:20.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/598937374 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a6aa0 msgr2=0x7f7e9814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/598937374 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a6aa0 0x7f7e9814b150 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f7e8c002f70 tx=0x7f7e8c006eb0).stop 2021-11-21T21:59:20.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/598937374 shutdown_connections 2021-11-21T21:59:20.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/598937374 >> v2:172.21.2.117:3300/0 conn(0x7f7e981475d0 0x7f7e981443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/598937374 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a86b0 0x7f7e981470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/598937374 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a6aa0 0x7f7e9814b150 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/598937374 >> 172.21.2.116:0/598937374 conn(0x7f7e9805bd20 msgr2=0x7f7e9805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/598937374 shutdown_connections 2021-11-21T21:59:20.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/598937374 wait complete. 2021-11-21T21:59:20.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 Processor -- start 2021-11-21T21:59:20.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- start start 2021-11-21T21:59:20.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a6aa0 0x7f7e9814ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a86b0 0x7f7e981597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e981475d0 0x7f7e98159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980b3b30 con 0x7f7e980a86b0 2021-11-21T21:59:20.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98065640 con 0x7f7e981475d0 2021-11-21T21:59:20.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.234+0000 7f7e9f9f4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980b3890 con 0x7f7e980a6aa0 2021-11-21T21:59:20.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a86b0 0x7f7e981597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a86b0 0x7f7e981597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48392/0 (socket says 172.21.2.116:48392) 2021-11-21T21:59:20.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 -- 172.21.2.116:0/1884293651 learned_addr learned my addr 172.21.2.116:0/1884293651 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9d790700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a6aa0 0x7f7e9814ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 -- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a6aa0 msgr2=0x7f7e9814ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a6aa0 0x7f7e9814ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 -- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e981475d0 msgr2=0x7f7e98159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e981475d0 0x7f7e98159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 -- 172.21.2.116:0/1884293651 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c00b1e0 con 0x7f7e980a86b0 2021-11-21T21:59:20.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9d790700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a6aa0 0x7f7e9814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:20.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9cf8f700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a86b0 0x7f7e981597f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f7e94002f70 tx=0x7f7e94006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e8a7fc700 1 -- 172.21.2.116:0/1884293651 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e94013460 con 0x7f7e980a86b0 2021-11-21T21:59:20.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e8a7fc700 1 -- 172.21.2.116:0/1884293651 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e940135c0 con 0x7f7e980a86b0 2021-11-21T21:59:20.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e980658a0 con 0x7f7e980a86b0 2021-11-21T21:59:20.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e8a7fc700 1 -- 172.21.2.116:0/1884293651 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e94013920 con 0x7f7e980a86b0 2021-11-21T21:59:20.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e98153dd0 con 0x7f7e980a86b0 2021-11-21T21:59:20.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e980b3540 con 0x7f7e980a86b0 2021-11-21T21:59:20.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e8a7fc700 1 -- 172.21.2.116:0/1884293651 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7e9401a060 con 0x7f7e980a86b0 2021-11-21T21:59:20.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e8a7fc700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e780536a0 0x7f7e78055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e8a7fc700 1 -- 172.21.2.116:0/1884293651 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7e9406b320 con 0x7f7e980a86b0 2021-11-21T21:59:20.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9d790700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e780536a0 0x7f7e78055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.238+0000 7f7e9d790700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e780536a0 0x7f7e78055b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f7e8c017000 tx=0x7f7e8c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.242+0000 7f7e8a7fc700 1 -- 172.21.2.116:0/1884293651 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7e94042480 con 0x7f7e980a86b0 2021-11-21T21:59:20.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e98065a30 con 0x7f7e980a86b0 2021-11-21T21:59:20.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e8a7fc700 1 -- 172.21.2.116:0/1884293651 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f7e94045aa0 con 0x7f7e980a86b0 2021-11-21T21:59:20.353 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:20.353 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:20.167894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3617788104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3617788104},{"type":"v1","addr":"172.21.2.116:6835","nonce":3617788104}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:20.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e780536a0 msgr2=0x7f7e78055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e780536a0 0x7f7e78055b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f7e8c017000 tx=0x7f7e8c015000).stop 2021-11-21T21:59:20.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a86b0 msgr2=0x7f7e981597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a86b0 0x7f7e981597f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f7e94002f70 tx=0x7f7e94006e90).stop 2021-11-21T21:59:20.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 shutdown_connections 2021-11-21T21:59:20.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e981475d0 0x7f7e98159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e780536a0 0x7f7e78055b60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e980a86b0 0x7f7e981597f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 --2- 172.21.2.116:0/1884293651 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e980a6aa0 0x7f7e9814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 >> 172.21.2.116:0/1884293651 conn(0x7f7e9805bd20 msgr2=0x7f7e981490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 shutdown_connections 2021-11-21T21:59:20.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.350+0000 7f7e9f9f4700 1 -- 172.21.2.116:0/1884293651 wait complete. 2021-11-21T21:59:20.359 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 28 2021-11-21T21:59:20.360 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:59:20.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 Processor -- start 2021-11-21T21:59:20.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 -- start start 2021-11-21T21:59:20.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f17200a6aa0 0x7f172014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f17200a86b0 0x7f17201470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 0x7f17201443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200a8bc0 con 0x7f17200a86b0 2021-11-21T21:59:20.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200a6e80 con 0x7f17200a6aa0 2021-11-21T21:59:20.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.406+0000 7f172671b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172014a7b0 con 0x7f17201475d0 2021-11-21T21:59:20.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f17200a86b0 0x7f17201470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f17200a86b0 0x7f17201470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48400/0 (socket says 172.21.2.116:48400) 2021-11-21T21:59:20.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 -- 172.21.2.116:0/2717841788 learned_addr learned my addr 172.21.2.116:0/2717841788 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 -- 172.21.2.116:0/2717841788 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 msgr2=0x7f17201443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:20.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 --2- 172.21.2.116:0/2717841788 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 0x7f17201443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 -- 172.21.2.116:0/2717841788 >> v2:172.21.2.117:3300/0 conn(0x7f17200a6aa0 msgr2=0x7f172014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 --2- 172.21.2.116:0/2717841788 >> v2:172.21.2.117:3300/0 conn(0x7f17200a6aa0 0x7f172014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 -- 172.21.2.116:0/2717841788 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172014aa10 con 0x7f17200a86b0 2021-11-21T21:59:20.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171f7fe700 1 --2- 172.21.2.116:0/2717841788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a86b0 0x7f17201470e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f1710002f70 tx=0x7f1710006eb0).ready entity=mon.0 client_cookie=d548fe3e0136a330 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171effd700 1 -- 172.21.2.116:0/2717841788 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171000d570 con 0x7f17200a86b0 2021-11-21T21:59:20.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171effd700 1 -- 172.21.2.116:0/2717841788 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1710000e90 con 0x7f17200a86b0 2021-11-21T21:59:20.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171effd700 1 -- 172.21.2.116:0/2717841788 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171000d900 con 0x7f17200a86b0 2021-11-21T21:59:20.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- 172.21.2.116:0/2717841788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a86b0 msgr2=0x7f17201470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 --2- 172.21.2.116:0/2717841788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a86b0 0x7f17201470e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f1710002f70 tx=0x7f1710006eb0).stop 2021-11-21T21:59:20.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- 172.21.2.116:0/2717841788 shutdown_connections 2021-11-21T21:59:20.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 --2- 172.21.2.116:0/2717841788 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 0x7f17201443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 --2- 172.21.2.116:0/2717841788 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a86b0 0x7f17201470e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 --2- 172.21.2.116:0/2717841788 >> v2:172.21.2.117:3300/0 conn(0x7f17200a6aa0 0x7f172014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- 172.21.2.116:0/2717841788 >> 172.21.2.116:0/2717841788 conn(0x7f172005bd20 msgr2=0x7f172005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- 172.21.2.116:0/2717841788 shutdown_connections 2021-11-21T21:59:20.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- 172.21.2.116:0/2717841788 wait complete. 2021-11-21T21:59:20.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 Processor -- start 2021-11-21T21:59:20.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- start start 2021-11-21T21:59:20.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a6aa0 0x7f17200b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17200a86b0 0x7f17200a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 0x7f1720145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1720156610 con 0x7f17200a6aa0 2021-11-21T21:59:20.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201568b0 con 0x7f17200a86b0 2021-11-21T21:59:20.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1720156b50 con 0x7f17201475d0 2021-11-21T21:59:20.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a6aa0 0x7f17200b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a6aa0 0x7f17200b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48406/0 (socket says 172.21.2.116:48406) 2021-11-21T21:59:20.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 -- 172.21.2.116:0/3500504068 learned_addr learned my addr 172.21.2.116:0/3500504068 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 -- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 msgr2=0x7f1720145ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:20.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 0x7f1720145ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 -- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17200a86b0 msgr2=0x7f17200a8d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:20.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17200a86b0 0x7f17200a8d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 -- 172.21.2.116:0/3500504068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171000b1e0 con 0x7f17200a6aa0 2021-11-21T21:59:20.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171ffff700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a6aa0 0x7f17200b4040 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f1708002f70 tx=0x7f1708006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171d7fa700 1 -- 172.21.2.116:0/3500504068 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1708013460 con 0x7f17200a6aa0 2021-11-21T21:59:20.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171d7fa700 1 -- 172.21.2.116:0/3500504068 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17080135c0 con 0x7f17200a6aa0 2021-11-21T21:59:20.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f171d7fa700 1 -- 172.21.2.116:0/3500504068 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1708013920 con 0x7f17200a6aa0 2021-11-21T21:59:20.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1720156df0 con 0x7f17200a6aa0 2021-11-21T21:59:20.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.410+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1720064f80 con 0x7f17200a6aa0 2021-11-21T21:59:20.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.414+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17200573d0 con 0x7f17200a6aa0 2021-11-21T21:59:20.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.414+0000 7f171d7fa700 1 -- 172.21.2.116:0/3500504068 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f170801a060 con 0x7f17200a6aa0 2021-11-21T21:59:20.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.414+0000 7f171d7fa700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1704053660 0x7f1704055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.414+0000 7f171d7fa700 1 -- 172.21.2.116:0/3500504068 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f170806b440 con 0x7f17200a6aa0 2021-11-21T21:59:20.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.414+0000 7f171f7fe700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1704053660 0x7f1704055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.414+0000 7f171f7fe700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1704053660 0x7f1704055b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f1710002f70 tx=0x7f1710015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.414+0000 7f171d7fa700 1 -- 172.21.2.116:0/3500504068 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1708042490 con 0x7f17200a6aa0 2021-11-21T21:59:20.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.498+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f172005faa0 con 0x7f17200a6aa0 2021-11-21T21:59:20.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.498+0000 7f171d7fa700 1 -- 172.21.2.116:0/3500504068 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f1708045ab0 con 0x7f17200a6aa0 2021-11-21T21:59:20.503 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:20.503 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":66,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:59:20.169181+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:17.175021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-11-21T21:59:18.135656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:59:20.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1704053660 msgr2=0x7f1704055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1704053660 0x7f1704055b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f1710002f70 tx=0x7f1710015000).stop 2021-11-21T21:59:20.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a6aa0 msgr2=0x7f17200b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a6aa0 0x7f17200b4040 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f1708002f70 tx=0x7f1708006e20).stop 2021-11-21T21:59:20.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 shutdown_connections 2021-11-21T21:59:20.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17201475d0 0x7f1720145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17200a86b0 0x7f17200a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1704053660 0x7f1704055b20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 --2- 172.21.2.116:0/3500504068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17200a6aa0 0x7f17200b4040 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 >> 172.21.2.116:0/3500504068 conn(0x7f172005bd20 msgr2=0x7f172013f130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 shutdown_connections 2021-11-21T21:59:20.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.502+0000 7f172671b700 1 -- 172.21.2.116:0/3500504068 wait complete. 2021-11-21T21:59:20.510 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:59:20.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.562+0000 7f61523a5700 1 Processor -- start 2021-11-21T21:59:20.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.562+0000 7f61523a5700 1 -- start start 2021-11-21T21:59:20.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.562+0000 7f61523a5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f614c0b11c0 0x7f614c147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.562+0000 7f61523a5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.562+0000 7f61523a5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c0b1940 con 0x7f614c147c10 2021-11-21T21:59:20.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.562+0000 7f61523a5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c0b15a0 con 0x7f614c0b11c0 2021-11-21T21:59:20.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.562+0000 7f61523a5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0b1b70 con 0x7f614c0a8670 2021-11-21T21:59:20.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614bfff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f614c0b11c0 0x7f614c147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:51758/0 (socket says 172.21.2.116:51758) 2021-11-21T21:59:20.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 -- 172.21.2.116:0/1722782107 learned_addr learned my addr 172.21.2.116:0/1722782107 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 -- 172.21.2.116:0/1722782107 >> v2:172.21.2.117:3300/0 conn(0x7f614c0b11c0 msgr2=0x7f614c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 --2- 172.21.2.116:0/1722782107 >> v2:172.21.2.117:3300/0 conn(0x7f614c0b11c0 0x7f614c147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 -- 172.21.2.116:0/1722782107 >> v1:172.21.2.117:6789/0 conn(0x7f614c147c10 legacy=0x7f614c14a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 -- 172.21.2.116:0/1722782107 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c0b1e10 con 0x7f614c0a8670 2021-11-21T21:59:20.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614b7fe700 1 --2- 172.21.2.116:0/1722782107 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c147720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f6134002f70 tx=0x7f6134006eb0).ready entity=mon.2 client_cookie=1c4c9edb6372a9e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614affd700 1 -- 172.21.2.116:0/1722782107 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f613400d570 con 0x7f614c0a8670 2021-11-21T21:59:20.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614affd700 1 -- 172.21.2.116:0/1722782107 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6134000e90 con 0x7f614c0a8670 2021-11-21T21:59:20.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f614affd700 1 -- 172.21.2.116:0/1722782107 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f613400d900 con 0x7f614c0a8670 2021-11-21T21:59:20.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- 172.21.2.116:0/1722782107 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 msgr2=0x7f614c147720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 --2- 172.21.2.116:0/1722782107 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c147720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f6134002f70 tx=0x7f6134006eb0).stop 2021-11-21T21:59:20.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- 172.21.2.116:0/1722782107 shutdown_connections 2021-11-21T21:59:20.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 --2- 172.21.2.116:0/1722782107 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c147720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 --2- 172.21.2.116:0/1722782107 >> v2:172.21.2.117:3300/0 conn(0x7f614c0b11c0 0x7f614c147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- 172.21.2.116:0/1722782107 >> 172.21.2.116:0/1722782107 conn(0x7f614c05bd20 msgr2=0x7f614c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- 172.21.2.116:0/1722782107 shutdown_connections 2021-11-21T21:59:20.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- 172.21.2.116:0/1722782107 wait complete. 2021-11-21T21:59:20.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 Processor -- start 2021-11-21T21:59:20.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- start start 2021-11-21T21:59:20.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c063bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f614c0b11c0 0x7f614c14d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f614c147c10 0x7f614c064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c14ac20 con 0x7f614c147c10 2021-11-21T21:59:20.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c14b230 con 0x7f614c0b11c0 2021-11-21T21:59:20.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.566+0000 7f61523a5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c067340 con 0x7f614c0a8670 2021-11-21T21:59:20.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f614c147c10 0x7f614c064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f614c147c10 0x7f614c064400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48418/0 (socket says 172.21.2.116:48418) 2021-11-21T21:59:20.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 -- 172.21.2.116:0/1546004973 learned_addr learned my addr 172.21.2.116:0/1546004973 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f614bfff700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c063bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f614b7fe700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f614c0b11c0 0x7f614c14d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 -- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 msgr2=0x7f614c063bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c063bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 -- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f614c0b11c0 msgr2=0x7f614c14d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f614c0b11c0 0x7f614c14d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 -- 172.21.2.116:0/1546004973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613400b1e0 con 0x7f614c147c10 2021-11-21T21:59:20.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f6150942700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f614c147c10 0x7f614c064400 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f614c0a8390 tx=0x7f61440013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61497fa700 1 -- 172.21.2.116:0/1546004973 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6144006aa0 con 0x7f614c147c10 2021-11-21T21:59:20.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61497fa700 1 -- 172.21.2.116:0/1546004973 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6144006c00 con 0x7f614c147c10 2021-11-21T21:59:20.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61497fa700 1 -- 172.21.2.116:0/1546004973 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6144007030 con 0x7f614c147c10 2021-11-21T21:59:20.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c067050 con 0x7f614c147c10 2021-11-21T21:59:20.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c14a7a0 con 0x7f614c147c10 2021-11-21T21:59:20.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614c0638c0 con 0x7f614c147c10 2021-11-21T21:59:20.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61497fa700 1 -- 172.21.2.116:0/1546004973 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6144010030 con 0x7f614c147c10 2021-11-21T21:59:20.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61497fa700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6130053640 0x7f6130055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f61497fa700 1 -- 172.21.2.116:0/1546004973 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f614406b910 con 0x7f614c147c10 2021-11-21T21:59:20.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f614bfff700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6130053640 0x7f6130055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.570+0000 7f614bfff700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6130053640 0x7f6130055b00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f613c00b000 tx=0x7f613c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.574+0000 7f61497fa700 1 -- 172.21.2.116:0/1546004973 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6144042960 con 0x7f614c147c10 2021-11-21T21:59:20.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.682+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f614c0b20b0 con 0x7f614c147c10 2021-11-21T21:59:20.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61497fa700 1 -- 172.21.2.116:0/1546004973 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6144045f80 con 0x7f614c147c10 2021-11-21T21:59:20.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6130053640 msgr2=0x7f6130055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6130053640 0x7f6130055b00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f613c00b000 tx=0x7f613c008040).stop 2021-11-21T21:59:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f614c147c10 msgr2=0x7f614c064400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f614c147c10 0x7f614c064400 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f614c0a8390 tx=0x7f61440013a0).stop 2021-11-21T21:59:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 shutdown_connections 2021-11-21T21:59:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6130053640 0x7f6130055b00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f614c147c10 0x7f614c064400 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f614c0b11c0 0x7f614c14d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 --2- 172.21.2.116:0/1546004973 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f614c0a8670 0x7f614c063bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 >> 172.21.2.116:0/1546004973 conn(0x7f614c05bd20 msgr2=0x7f614c1484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 shutdown_connections 2021-11-21T21:59:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.690+0000 7f61523a5700 1 -- 172.21.2.116:0/1546004973 wait complete. 2021-11-21T21:59:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T21:59:20.700 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T21:59:20.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.750+0000 7f9e33efb700 1 Processor -- start 2021-11-21T21:59:20.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.750+0000 7f9e33efb700 1 -- start start 2021-11-21T21:59:20.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c05efb0 0x7f9e2c0b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9e2c0b1660 0x7f9e2c13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9e2c0aff60 0x7f9e2c13c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2c0aef20 con 0x7f9e2c0aff60 2021-11-21T21:59:20.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2c13c8f0 con 0x7f9e2c0b1660 2021-11-21T21:59:20.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c0ae540 con 0x7f9e2c05efb0 2021-11-21T21:59:20.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9e2c0aff60 0x7f9e2c13c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9e2c0aff60 0x7f9e2c13c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48428/0 (socket says 172.21.2.116:48428) 2021-11-21T21:59:20.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 -- 172.21.2.116:0/3237652376 learned_addr learned my addr 172.21.2.116:0/3237652376 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 -- 172.21.2.116:0/3237652376 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c05efb0 msgr2=0x7f9e2c0b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 --2- 172.21.2.116:0/3237652376 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c05efb0 0x7f9e2c0b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 -- 172.21.2.116:0/3237652376 >> v2:172.21.2.117:3300/0 conn(0x7f9e2c0b1660 msgr2=0x7f9e2c13bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 --2- 172.21.2.116:0/3237652376 >> v2:172.21.2.117:3300/0 conn(0x7f9e2c0b1660 0x7f9e2c13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 -- 172.21.2.116:0/3237652376 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c0ae7e0 con 0x7f9e2c0aff60 2021-11-21T21:59:20.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 --2- 172.21.2.116:0/3237652376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c13c400 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f9e240032f0 tx=0x7f9e24009290).ready entity=mon.0 client_cookie=a5b9e6f708efdf95 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e30c95700 1 -- 172.21.2.116:0/3237652376 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2400b870 con 0x7f9e2c0aff60 2021-11-21T21:59:20.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e30c95700 1 -- 172.21.2.116:0/3237652376 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2400b9d0 con 0x7f9e2c0aff60 2021-11-21T21:59:20.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e30c95700 1 -- 172.21.2.116:0/3237652376 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2400bb40 con 0x7f9e2c0aff60 2021-11-21T21:59:20.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- 172.21.2.116:0/3237652376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c0aff60 msgr2=0x7f9e2c13c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- 172.21.2.116:0/3237652376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c13c400 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f9e240032f0 tx=0x7f9e24009290).stop 2021-11-21T21:59:20.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- 172.21.2.116:0/3237652376 shutdown_connections 2021-11-21T21:59:20.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- 172.21.2.116:0/3237652376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c13c400 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- 172.21.2.116:0/3237652376 >> v2:172.21.2.117:3300/0 conn(0x7f9e2c0b1660 0x7f9e2c13bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- 172.21.2.116:0/3237652376 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c05efb0 0x7f9e2c0b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- 172.21.2.116:0/3237652376 >> 172.21.2.116:0/3237652376 conn(0x7f9e2c05f730 msgr2=0x7f9e2c063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- 172.21.2.116:0/3237652376 shutdown_connections 2021-11-21T21:59:20.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- 172.21.2.116:0/3237652376 wait complete. 2021-11-21T21:59:20.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 Processor -- start 2021-11-21T21:59:20.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- start start 2021-11-21T21:59:20.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c05efb0 0x7f9e2c149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c0a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c0b1660 0x7f9e2c155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c0abd90 con 0x7f9e2c05efb0 2021-11-21T21:59:20.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c0abfc0 con 0x7f9e2c0aff60 2021-11-21T21:59:20.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c0a9d80 con 0x7f9e2c0b1660 2021-11-21T21:59:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c05efb0 0x7f9e2c149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c05efb0 0x7f9e2c149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48432/0 (socket says 172.21.2.116:48432) 2021-11-21T21:59:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 -- 172.21.2.116:0/931487385 learned_addr learned my addr 172.21.2.116:0/931487385 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e32498700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c0b1660 0x7f9e2c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31496700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c0a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 -- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c0b1660 msgr2=0x7f9e2c155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c0b1660 0x7f9e2c155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 -- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e2c0aff60 msgr2=0x7f9e2c0a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c0a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 -- 172.21.2.116:0/931487385 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24009530 con 0x7f9e2c05efb0 2021-11-21T21:59:20.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31496700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c0a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:20.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e31c97700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c05efb0 0x7f9e2c149190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f9e20002f70 tx=0x7f9e20006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e1effd700 1 -- 172.21.2.116:0/931487385 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e20013460 con 0x7f9e2c05efb0 2021-11-21T21:59:20.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e1effd700 1 -- 172.21.2.116:0/931487385 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e200135c0 con 0x7f9e2c05efb0 2021-11-21T21:59:20.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c0a9a30 con 0x7f9e2c05efb0 2021-11-21T21:59:20.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c155960 con 0x7f9e2c05efb0 2021-11-21T21:59:20.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.754+0000 7f9e1effd700 1 -- 172.21.2.116:0/931487385 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e20013920 con 0x7f9e2c05efb0 2021-11-21T21:59:20.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.758+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2c0a9f40 con 0x7f9e2c05efb0 2021-11-21T21:59:20.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.758+0000 7f9e1effd700 1 -- 172.21.2.116:0/931487385 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9e2001a060 con 0x7f9e2c05efb0 2021-11-21T21:59:20.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.758+0000 7f9e1effd700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e0c053540 0x7f9e0c055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.758+0000 7f9e1effd700 1 -- 172.21.2.116:0/931487385 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9e2006b530 con 0x7f9e2c05efb0 2021-11-21T21:59:20.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.758+0000 7f9e31496700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e0c053540 0x7f9e0c055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.758+0000 7f9e31496700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e0c053540 0x7f9e0c055a00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f9e28007000 tx=0x7f9e28008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.758+0000 7f9e1effd700 1 -- 172.21.2.116:0/931487385 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e20042580 con 0x7f9e2c05efb0 2021-11-21T21:59:20.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.870+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9e2c0aea80 con 0x7f9e2c05efb0 2021-11-21T21:59:20.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e1effd700 1 -- 172.21.2.116:0/931487385 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9e20018030 con 0x7f9e2c05efb0 2021-11-21T21:59:20.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e0c053540 msgr2=0x7f9e0c055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e0c053540 0x7f9e0c055a00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f9e28007000 tx=0x7f9e28008040).stop 2021-11-21T21:59:20.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c05efb0 msgr2=0x7f9e2c149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c05efb0 0x7f9e2c149190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f9e20002f70 tx=0x7f9e20006ec0).stop 2021-11-21T21:59:20.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 shutdown_connections 2021-11-21T21:59:20.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e2c0b1660 0x7f9e2c155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e2c0aff60 0x7f9e2c0a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e0c053540 0x7f9e0c055a00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 --2- 172.21.2.116:0/931487385 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e2c05efb0 0x7f9e2c149190 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 >> 172.21.2.116:0/931487385 conn(0x7f9e2c05f730 msgr2=0x7f9e2c139040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 shutdown_connections 2021-11-21T21:59:20.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.874+0000 7f9e33efb700 1 -- 172.21.2.116:0/931487385 wait complete. 2021-11-21T21:59:20.880 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T21:59:20.884 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:20.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.930+0000 7f30d7991700 1 Processor -- start 2021-11-21T21:59:20.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.930+0000 7f30d7991700 1 -- start start 2021-11-21T21:59:20.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.930+0000 7f30d7991700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f30d00a6aa0 0x7f30d014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.930+0000 7f30d7991700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0062410 0x7f30d01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.930+0000 7f30d7991700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d00a6e80 con 0x7f30d00a86b0 2021-11-21T21:59:20.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.930+0000 7f30d7991700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d00a8bc0 con 0x7f30d00a6aa0 2021-11-21T21:59:20.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.930+0000 7f30d7991700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d014a7b0 con 0x7f30d0062410 2021-11-21T21:59:20.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d4f2c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f30d00a86b0 0x7f30d01470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60396/0 (socket says 172.21.2.116:60396) 2021-11-21T21:59:20.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d4f2c700 1 -- 172.21.2.116:0/1818497415 learned_addr learned my addr 172.21.2.116:0/1818497415 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:20.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179174608 0 0) 0x7f30d00a6e80 con 0x7f30d00a86b0 2021-11-21T21:59:20.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30b8003260 con 0x7f30d00a86b0 2021-11-21T21:59:20.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d5f2e700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0062410 0x7f30d01443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- 172.21.2.116:0/1818497415 >> v2:172.21.2.117:3300/0 conn(0x7f30d00a6aa0 0x7f30d014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2961307552 0 0) 0x7f30b8003260 con 0x7f30d00a86b0 2021-11-21T21:59:20.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0062410 msgr2=0x7f30d01443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0062410 0x7f30d01443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 >> v2:172.21.2.117:3300/0 conn(0x7f30d00a6aa0 msgr2=0x7f30d014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 --2- 172.21.2.116:0/1818497415 >> v2:172.21.2.117:3300/0 conn(0x7f30d00a6aa0 0x7f30d014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:20.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d014aa10 con 0x7f30d00a86b0 2021-11-21T21:59:20.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f30c4001680 con 0x7f30d00a86b0 2021-11-21T21:59:20.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 >> v1:172.21.2.116:6789/0 conn(0x7f30d00a86b0 legacy=0x7f30d01470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30b8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f30b8004500 0x7f30b8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0062410 0x7f30b8004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00a8bc0 con 0x7f30b80037d0 2021-11-21T21:59:20.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d014a7b0 con 0x7f30b8004500 2021-11-21T21:59:20.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b800b900 con 0x7f30d0062410 2021-11-21T21:59:20.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30b8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 -- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0062410 msgr2=0x7f30b8004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0062410 0x7f30b8004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 -- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f30b8004500 msgr2=0x7f30b8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:20.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f30b8004500 0x7f30b8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 -- 172.21.2.116:0/1818497415 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8003260 con 0x7f30b80037d0 2021-11-21T21:59:20.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30b8003bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f30cc00a000 tx=0x7f30cc008040).ready entity=mon.0 client_cookie=4c6e46c8861a6474 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc013670 con 0x7f30b80037d0 2021-11-21T21:59:20.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc0137d0 con 0x7f30b80037d0 2021-11-21T21:59:20.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30c3fff700 1 -- 172.21.2.116:0/1818497415 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc013b30 con 0x7f30b80037d0 2021-11-21T21:59:20.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- 172.21.2.116:0/1818497415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 msgr2=0x7f30b8003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 --2- 172.21.2.116:0/1818497415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30b8003bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f30cc00a000 tx=0x7f30cc008040).stop 2021-11-21T21:59:20.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d5f2e700 1 -- 172.21.2.116:0/1818497415 reap_dead start 2021-11-21T21:59:20.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- 172.21.2.116:0/1818497415 shutdown_connections 2021-11-21T21:59:20.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- 172.21.2.116:0/1818497415 >> 172.21.2.116:0/1818497415 conn(0x7f30d005bd20 msgr2=0x7f30d005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:20.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- 172.21.2.116:0/1818497415 shutdown_connections 2021-11-21T21:59:20.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- 172.21.2.116:0/1818497415 wait complete. 2021-11-21T21:59:20.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 Processor -- start 2021-11-21T21:59:20.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- start start 2021-11-21T21:59:20.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30d014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f30d0064b00 0x7f30d01561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0159e30 0x7f30d0142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00a9d80 con 0x7f30b80037d0 2021-11-21T21:59:20.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0159730 con 0x7f30d0064b00 2021-11-21T21:59:20.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d7991700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00642f0 con 0x7f30d0159e30 2021-11-21T21:59:20.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30d014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30d014b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48450/0 (socket says 172.21.2.116:48450) 2021-11-21T21:59:20.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 -- 172.21.2.116:0/1322542476 learned_addr learned my addr 172.21.2.116:0/1322542476 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:20.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 -- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0159e30 msgr2=0x7f30d0142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0159e30 0x7f30d0142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 -- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f30d0064b00 msgr2=0x7f30d01561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:20.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f30d0064b00 0x7f30d01561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:20.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.934+0000 7f30d572d700 1 -- 172.21.2.116:0/1322542476 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0064590 con 0x7f30b80037d0 2021-11-21T21:59:20.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30d572d700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30d014b9f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f30cc00a000 tx=0x7f30cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30c27fc700 1 -- 172.21.2.116:0/1322542476 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc012d30 con 0x7f30b80037d0 2021-11-21T21:59:20.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30c27fc700 1 -- 172.21.2.116:0/1322542476 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc000e90 con 0x7f30b80037d0 2021-11-21T21:59:20.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30c27fc700 1 -- 172.21.2.116:0/1322542476 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc01d600 con 0x7f30b80037d0 2021-11-21T21:59:20.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d0159990 con 0x7f30b80037d0 2021-11-21T21:59:20.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d0065050 con 0x7f30b80037d0 2021-11-21T21:59:20.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d00a9d80 con 0x7f30b80037d0 2021-11-21T21:59:20.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30c27fc700 1 -- 172.21.2.116:0/1322542476 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f30cc01d760 con 0x7f30b80037d0 2021-11-21T21:59:20.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30c27fc700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30b0053540 0x7f30b0055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:20.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30c27fc700 1 -- 172.21.2.116:0/1322542476 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f30cc06d210 con 0x7f30b80037d0 2021-11-21T21:59:20.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30d4f2c700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30b0053540 0x7f30b0055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:20.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30d4f2c700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30b0053540 0x7f30b0055a00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f30c4001390 tx=0x7f30c4001590).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:20.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:20.938+0000 7f30c27fc700 1 -- 172.21.2.116:0/1322542476 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30cc044310 con 0x7f30b80037d0 2021-11-21T21:59:21.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.046+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30d0159b20 con 0x7f30b80037d0 2021-11-21T21:59:21.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.046+0000 7f30c27fc700 1 -- 172.21.2.116:0/1322542476 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f30cc00b120 con 0x7f30b80037d0 2021-11-21T21:59:21.052 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:21.052 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:20.167894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3617788104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3617788104},{"type":"v1","addr":"172.21.2.116:6835","nonce":3617788104}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:21.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30b0053540 msgr2=0x7f30b0055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30b0053540 0x7f30b0055a00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f30c4001390 tx=0x7f30c4001590).stop 2021-11-21T21:59:21.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 msgr2=0x7f30d014b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30d014b9f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f30cc00a000 tx=0x7f30cc008040).stop 2021-11-21T21:59:21.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 shutdown_connections 2021-11-21T21:59:21.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f30d0159e30 0x7f30d0142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f30d0064b00 0x7f30d01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30b0053540 0x7f30b0055a00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 --2- 172.21.2.116:0/1322542476 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f30b80037d0 0x7f30d014b9f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 >> 172.21.2.116:0/1322542476 conn(0x7f30d005bd20 msgr2=0x7f30d0146d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 shutdown_connections 2021-11-21T21:59:21.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.050+0000 7f30d7991700 1 -- 172.21.2.116:0/1322542476 wait complete. 2021-11-21T21:59:21.056 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 28 2021-11-21T21:59:21.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:59:19.164615+0000', 'modified': '2021-11-21T21:59:20.167894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5973': {'gid': 5973, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3617788104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3617788104}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3617788104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:59:21.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:59:21.362 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:59:21.445 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5973,"want_state":"up:active","state":"up:active","rank_uptime":2.2741179570000001,"mdsmap_epoch":28,"osdmap_epoch":66,"osdmap_epoch_barrier":65,"uptime":6.3946673709999997} 2021-11-21T21:59:21.445 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5973, "mdsmap_epoch": 28, "osdmap_epoch": 66, "osdmap_epoch_barrier": 65, "rank_uptime": 2.274117957, "state": "up:active", "uptime": 6.394667371, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:59:21.446 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:59:21.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac880d8700 1 Processor -- start 2021-11-21T21:59:21.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac880d8700 1 -- start start 2021-11-21T21:59:21.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac880d8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac800a6aa0 0x7fac8014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac880d8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fac800a86b0 0x7fac801470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac880d8700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac800a6e80 con 0x7fac801475d0 2021-11-21T21:59:21.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac880d8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac800a8bc0 con 0x7fac800a86b0 2021-11-21T21:59:21.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac880d8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8014a7b0 con 0x7fac800a6aa0 2021-11-21T21:59:21.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85e74700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac800a6aa0 0x7fac8014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85673700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fac800a86b0 0x7fac801470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85e74700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac800a6aa0 0x7fac8014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:51802/0 (socket says 172.21.2.116:51802) 2021-11-21T21:59:21.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85e74700 1 -- 172.21.2.116:0/996558843 learned_addr learned my addr 172.21.2.116:0/996558843 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85e74700 1 -- 172.21.2.116:0/996558843 >> v2:172.21.2.117:3300/0 conn(0x7fac800a86b0 msgr2=0x7fac801470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:21.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85e74700 1 --2- 172.21.2.116:0/996558843 >> v2:172.21.2.117:3300/0 conn(0x7fac800a86b0 0x7fac801470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85e74700 1 -- 172.21.2.116:0/996558843 >> v1:172.21.2.117:6789/0 conn(0x7fac801475d0 legacy=0x7fac801443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.494+0000 7fac85e74700 1 -- 172.21.2.116:0/996558843 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8014aa10 con 0x7fac800a6aa0 2021-11-21T21:59:21.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 --2- 172.21.2.116:0/996558843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac800a6aa0 0x7fac8014b150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fac7c002f70 tx=0x7fac7c006eb0).ready entity=mon.2 client_cookie=e1db7bc819ccd99f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac84e72700 1 -- 172.21.2.116:0/996558843 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac7c00d570 con 0x7fac800a6aa0 2021-11-21T21:59:21.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac84e72700 1 -- 172.21.2.116:0/996558843 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c000e90 con 0x7fac800a6aa0 2021-11-21T21:59:21.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac84e72700 1 -- 172.21.2.116:0/996558843 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac7c00d900 con 0x7fac800a6aa0 2021-11-21T21:59:21.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- 172.21.2.116:0/996558843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac800a6aa0 msgr2=0x7fac8014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 --2- 172.21.2.116:0/996558843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac800a6aa0 0x7fac8014b150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fac7c002f70 tx=0x7fac7c006eb0).stop 2021-11-21T21:59:21.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- 172.21.2.116:0/996558843 shutdown_connections 2021-11-21T21:59:21.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 --2- 172.21.2.116:0/996558843 >> v2:172.21.2.117:3300/0 conn(0x7fac800a86b0 0x7fac801470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 --2- 172.21.2.116:0/996558843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac800a6aa0 0x7fac8014b150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- 172.21.2.116:0/996558843 >> 172.21.2.116:0/996558843 conn(0x7fac8005bd20 msgr2=0x7fac8005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- 172.21.2.116:0/996558843 shutdown_connections 2021-11-21T21:59:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- 172.21.2.116:0/996558843 wait complete. 2021-11-21T21:59:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 Processor -- start 2021-11-21T21:59:21.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- start start 2021-11-21T21:59:21.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac800a6aa0 0x7fac800b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac800a86b0 0x7fac800a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac801475d0 0x7fac80156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80063a40 con 0x7fac800a6aa0 2021-11-21T21:59:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80067400 con 0x7fac800a86b0 2021-11-21T21:59:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac800b35c0 con 0x7fac801475d0 2021-11-21T21:59:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac800a6aa0 0x7fac800b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac800a6aa0 0x7fac800b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48462/0 (socket says 172.21.2.116:48462) 2021-11-21T21:59:21.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 -- 172.21.2.116:0/300892290 learned_addr learned my addr 172.21.2.116:0/300892290 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 -- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac801475d0 msgr2=0x7fac80156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac801475d0 0x7fac80156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 -- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac800a86b0 msgr2=0x7fac800a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac800a86b0 0x7fac800a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 -- 172.21.2.116:0/300892290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c00b1e0 con 0x7fac800a6aa0 2021-11-21T21:59:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac85e74700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac800a6aa0 0x7fac800b30d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fac7c002f70 tx=0x7fac7c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac72ffd700 1 -- 172.21.2.116:0/300892290 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac7c006b40 con 0x7fac800a6aa0 2021-11-21T21:59:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac72ffd700 1 -- 172.21.2.116:0/300892290 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c0068b0 con 0x7fac800a6aa0 2021-11-21T21:59:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac800a91f0 con 0x7fac800a6aa0 2021-11-21T21:59:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac72ffd700 1 -- 172.21.2.116:0/300892290 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac7c00de30 con 0x7fac800a6aa0 2021-11-21T21:59:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.498+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac801472b0 con 0x7fac800a6aa0 2021-11-21T21:59:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.502+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac80063bb0 con 0x7fac800a6aa0 2021-11-21T21:59:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.502+0000 7fac72ffd700 1 -- 172.21.2.116:0/300892290 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fac7c025030 con 0x7fac800a6aa0 2021-11-21T21:59:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.502+0000 7fac72ffd700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac60053610 0x7fac60055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.502+0000 7fac72ffd700 1 -- 172.21.2.116:0/300892290 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fac7c06cc20 con 0x7fac800a6aa0 2021-11-21T21:59:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.502+0000 7fac85673700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac60053610 0x7fac60055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.502+0000 7fac85673700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac60053610 0x7fac60055ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fac7400c000 tx=0x7fac74006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.502+0000 7fac72ffd700 1 -- 172.21.2.116:0/300892290 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac7c043c70 con 0x7fac800a6aa0 2021-11-21T21:59:21.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac800573d0 con 0x7fac800a6aa0 2021-11-21T21:59:21.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac72ffd700 1 -- 172.21.2.116:0/300892290 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16041 (secure 0 0 0) 0x7fac7c023020 con 0x7fac800a6aa0 2021-11-21T21:59:21.593 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:21.593 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":66,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:59:20.169181+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:17.175021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-11-21T21:59:18.135656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:59:21.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac60053610 msgr2=0x7fac60055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac60053610 0x7fac60055ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fac7400c000 tx=0x7fac74006df0).stop 2021-11-21T21:59:21.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac800a6aa0 msgr2=0x7fac800b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac800a6aa0 0x7fac800b30d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fac7c002f70 tx=0x7fac7c015000).stop 2021-11-21T21:59:21.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 shutdown_connections 2021-11-21T21:59:21.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac801475d0 0x7fac80156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac800a86b0 0x7fac800a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac60053610 0x7fac60055ad0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 --2- 172.21.2.116:0/300892290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac800a6aa0 0x7fac800b30d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 >> 172.21.2.116:0/300892290 conn(0x7fac8005bd20 msgr2=0x7fac8013f150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 shutdown_connections 2021-11-21T21:59:21.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.590+0000 7fac880d8700 1 -- 172.21.2.116:0/300892290 wait complete. 2021-11-21T21:59:21.599 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:59:17.175021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T21:59:18.135656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T21:59:21.600 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:59:21.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 Processor -- start 2021-11-21T21:59:21.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- start start 2021-11-21T21:59:21.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5210144810 0x7f52101478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f52100ac600 0x7f52100aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5210147e80 con 0x7f52100abce0 2021-11-21T21:59:21.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52100b1490 con 0x7f52100ac600 2021-11-21T21:59:21.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52100b1fd0 con 0x7f5210144810 2021-11-21T21:59:21.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f52100ac600 0x7f52100aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f52100ac600 0x7f52100aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48470/0 (socket says 172.21.2.116:48470) 2021-11-21T21:59:21.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 -- 172.21.2.116:0/583274140 learned_addr learned my addr 172.21.2.116:0/583274140 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 -- 172.21.2.116:0/583274140 >> v2:172.21.2.117:3300/0 conn(0x7f5210144810 msgr2=0x7f52101478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:21.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 --2- 172.21.2.116:0/583274140 >> v2:172.21.2.117:3300/0 conn(0x7f5210144810 0x7f52101478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 -- 172.21.2.116:0/583274140 >> v1:172.21.2.117:6789/0 conn(0x7f52100abce0 legacy=0x7f52100b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:21.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 -- 172.21.2.116:0/583274140 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100af5a0 con 0x7f52100ac600 2021-11-21T21:59:21.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5215e3b700 1 --2- 172.21.2.116:0/583274140 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f52100aca50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f52100b4e00 tx=0x7f5208009290).ready entity=mon.0 client_cookie=dbe590718949475b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5203fff700 1 -- 172.21.2.116:0/583274140 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f520800b830 con 0x7f52100ac600 2021-11-21T21:59:21.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5203fff700 1 -- 172.21.2.116:0/583274140 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520800b990 con 0x7f52100ac600 2021-11-21T21:59:21.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f5203fff700 1 -- 172.21.2.116:0/583274140 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f520800bb40 con 0x7f52100ac600 2021-11-21T21:59:21.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- 172.21.2.116:0/583274140 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 msgr2=0x7f52100aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 --2- 172.21.2.116:0/583274140 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f52100aca50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f52100b4e00 tx=0x7f5208009290).stop 2021-11-21T21:59:21.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- 172.21.2.116:0/583274140 shutdown_connections 2021-11-21T21:59:21.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 --2- 172.21.2.116:0/583274140 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f52100aca50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 --2- 172.21.2.116:0/583274140 >> v2:172.21.2.117:3300/0 conn(0x7f5210144810 0x7f52101478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- 172.21.2.116:0/583274140 >> 172.21.2.116:0/583274140 conn(0x7f521005bd20 msgr2=0x7f521005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- 172.21.2.116:0/583274140 shutdown_connections 2021-11-21T21:59:21.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.650+0000 7f521789e700 1 -- 172.21.2.116:0/583274140 wait complete. 2021-11-21T21:59:21.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 Processor -- start 2021-11-21T21:59:21.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 -- start start 2021-11-21T21:59:21.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52100abce0 0x7f52100b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f5210143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5210144810 0x7f5210156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5210146850 con 0x7f52100ac600 2021-11-21T21:59:21.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5210144d20 con 0x7f52100abce0 2021-11-21T21:59:21.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5210146290 con 0x7f5210144810 2021-11-21T21:59:21.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f5210143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f5210143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48478/0 (socket says 172.21.2.116:48478) 2021-11-21T21:59:21.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 -- 172.21.2.116:0/335489252 learned_addr learned my addr 172.21.2.116:0/335489252 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 -- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5210144810 msgr2=0x7f5210156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5210144810 0x7f5210156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 -- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52100abce0 msgr2=0x7f52100b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52100abce0 0x7f52100b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 -- 172.21.2.116:0/335489252 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208009530 con 0x7f52100ac600 2021-11-21T21:59:21.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f5214e39700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f5210143320 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f5204002fb0 tx=0x7f5204006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f52027fc700 1 -- 172.21.2.116:0/335489252 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5204013460 con 0x7f52100ac600 2021-11-21T21:59:21.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f52027fc700 1 -- 172.21.2.116:0/335489252 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52040135c0 con 0x7f52100ac600 2021-11-21T21:59:21.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f52027fc700 1 -- 172.21.2.116:0/335489252 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5204013920 con 0x7f52100ac600 2021-11-21T21:59:21.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5210146a80 con 0x7f52100ac600 2021-11-21T21:59:21.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52101439a0 con 0x7f52100ac600 2021-11-21T21:59:21.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52100af060 con 0x7f52100ac600 2021-11-21T21:59:21.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f52027fc700 1 -- 172.21.2.116:0/335489252 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f520400f070 con 0x7f52100ac600 2021-11-21T21:59:21.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f52027fc700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535b0 0x7f51f0055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f52027fc700 1 -- 172.21.2.116:0/335489252 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f520406aff0 con 0x7f52100ac600 2021-11-21T21:59:21.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.654+0000 7f521563a700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535b0 0x7f51f0055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.658+0000 7f521563a700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535b0 0x7f51f0055a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f520c000ed0 tx=0x7f520c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.658+0000 7f52027fc700 1 -- 172.21.2.116:0/335489252 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5204042150 con 0x7f52100ac600 2021-11-21T21:59:21.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.738+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f52100573d0 con 0x7f52100ac600 2021-11-21T21:59:21.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.738+0000 7f52027fc700 1 -- 172.21.2.116:0/335489252 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v66) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5204018030 con 0x7f52100ac600 2021-11-21T21:59:21.743 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:59:21.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535b0 msgr2=0x7f51f0055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535b0 0x7f51f0055a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f520c000ed0 tx=0x7f520c006930).stop 2021-11-21T21:59:21.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 msgr2=0x7f5210143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f5210143320 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f5204002fb0 tx=0x7f5204006df0).stop 2021-11-21T21:59:21.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 shutdown_connections 2021-11-21T21:59:21.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5210144810 0x7f5210156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535b0 0x7f51f0055a70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52100ac600 0x7f5210143320 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 --2- 172.21.2.116:0/335489252 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52100abce0 0x7f52100b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 >> 172.21.2.116:0/335489252 conn(0x7f521005bd20 msgr2=0x7f52100af780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 shutdown_connections 2021-11-21T21:59:21.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.742+0000 7f521789e700 1 -- 172.21.2.116:0/335489252 wait complete. 2021-11-21T21:59:21.750 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:59:21.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.798+0000 7f793655b700 1 Processor -- start 2021-11-21T21:59:21.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.798+0000 7f793655b700 1 -- start start 2021-11-21T21:59:21.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7930067890 0x7f79300b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f79300ac5a0 0x7f79300634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7930140930 con 0x7f79300b0ca0 2021-11-21T21:59:21.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79300b1080 con 0x7f79300ac5a0 2021-11-21T21:59:21.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79300acd20 con 0x7f7930067890 2021-11-21T21:59:21.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f79300ac5a0 0x7f79300634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f79300ac5a0 0x7f79300634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48486/0 (socket says 172.21.2.116:48486) 2021-11-21T21:59:21.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 -- 172.21.2.116:0/1624188635 learned_addr learned my addr 172.21.2.116:0/1624188635 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 -- 172.21.2.116:0/1624188635 >> v2:172.21.2.117:3300/0 conn(0x7f7930067890 msgr2=0x7f79300b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:21.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 --2- 172.21.2.116:0/1624188635 >> v2:172.21.2.117:3300/0 conn(0x7f7930067890 0x7f79300b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 -- 172.21.2.116:0/1624188635 >> v1:172.21.2.117:6789/0 conn(0x7f79300b0ca0 legacy=0x7f7930140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:21.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 -- 172.21.2.116:0/1624188635 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79300acfc0 con 0x7f79300ac5a0 2021-11-21T21:59:21.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f7934af8700 1 --2- 172.21.2.116:0/1624188635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f79300634b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f7930062f30 tx=0x7f792800b530).ready entity=mon.0 client_cookie=3a85f23f75af442f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792effd700 1 -- 172.21.2.116:0/1624188635 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f792800be70 con 0x7f79300ac5a0 2021-11-21T21:59:21.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792effd700 1 -- 172.21.2.116:0/1624188635 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7928010470 con 0x7f79300ac5a0 2021-11-21T21:59:21.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792effd700 1 -- 172.21.2.116:0/1624188635 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79280107d0 con 0x7f79300ac5a0 2021-11-21T21:59:21.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- 172.21.2.116:0/1624188635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 msgr2=0x7f79300634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- 172.21.2.116:0/1624188635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f79300634b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f7930062f30 tx=0x7f792800b530).stop 2021-11-21T21:59:21.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- 172.21.2.116:0/1624188635 shutdown_connections 2021-11-21T21:59:21.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- 172.21.2.116:0/1624188635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f79300634b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- 172.21.2.116:0/1624188635 >> v2:172.21.2.117:3300/0 conn(0x7f7930067890 0x7f79300b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- 172.21.2.116:0/1624188635 >> 172.21.2.116:0/1624188635 conn(0x7f793005bd20 msgr2=0x7f793005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- 172.21.2.116:0/1624188635 shutdown_connections 2021-11-21T21:59:21.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- 172.21.2.116:0/1624188635 wait complete. 2021-11-21T21:59:21.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 Processor -- start 2021-11-21T21:59:21.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- start start 2021-11-21T21:59:21.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7930067890 0x7f79301521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f7930137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79300b0ca0 0x7f793014e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7930151830 con 0x7f79300ac5a0 2021-11-21T21:59:21.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793013a310 con 0x7f79300b0ca0 2021-11-21T21:59:21.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7930151a60 con 0x7f7930067890 2021-11-21T21:59:21.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f7930137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f7930137660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48492/0 (socket says 172.21.2.116:48492) 2021-11-21T21:59:21.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 -- 172.21.2.116:0/737608194 learned_addr learned my addr 172.21.2.116:0/737608194 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 -- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7930067890 msgr2=0x7f79301521a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:21.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7930067890 0x7f79301521a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 -- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79300b0ca0 msgr2=0x7f793014e240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:21.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79300b0ca0 0x7f793014e240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 -- 172.21.2.116:0/737608194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792800b7d0 con 0x7f79300ac5a0 2021-11-21T21:59:21.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792f7fe700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f7930137660 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f7920002f70 tx=0x7f7920006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792d7fa700 1 -- 172.21.2.116:0/737608194 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7920013460 con 0x7f79300ac5a0 2021-11-21T21:59:21.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792d7fa700 1 -- 172.21.2.116:0/737608194 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79200135c0 con 0x7f79300ac5a0 2021-11-21T21:59:21.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7930151d00 con 0x7f79300ac5a0 2021-11-21T21:59:21.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f792d7fa700 1 -- 172.21.2.116:0/737608194 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7920013920 con 0x7f79300ac5a0 2021-11-21T21:59:21.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.802+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793013a5b0 con 0x7f79300ac5a0 2021-11-21T21:59:21.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.806+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7930151e90 con 0x7f79300ac5a0 2021-11-21T21:59:21.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.806+0000 7f792d7fa700 1 -- 172.21.2.116:0/737608194 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f792001a060 con 0x7f79300ac5a0 2021-11-21T21:59:21.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.806+0000 7f792d7fa700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7914053610 0x7f7914055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.806+0000 7f792d7fa700 1 -- 172.21.2.116:0/737608194 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f792006b400 con 0x7f79300ac5a0 2021-11-21T21:59:21.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.806+0000 7f792ffff700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7914053610 0x7f7914055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.806+0000 7f792ffff700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7914053610 0x7f7914055ad0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f7918002ef0 tx=0x7f7918006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.806+0000 7f792d7fa700 1 -- 172.21.2.116:0/737608194 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7920042560 con 0x7f79300ac5a0 2021-11-21T21:59:21.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79300573d0 con 0x7f79300ac5a0 2021-11-21T21:59:21.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f792d7fa700 1 -- 172.21.2.116:0/737608194 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7920045b80 con 0x7f79300ac5a0 2021-11-21T21:59:21.895 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:59:21.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7914053610 msgr2=0x7f7914055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7914053610 0x7f7914055ad0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f7918002ef0 tx=0x7f7918006890).stop 2021-11-21T21:59:21.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 msgr2=0x7f7930137660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f7930137660 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f7920002f70 tx=0x7f7920006ec0).stop 2021-11-21T21:59:21.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 shutdown_connections 2021-11-21T21:59:21.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79300b0ca0 0x7f793014e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7914053610 0x7f7914055ad0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79300ac5a0 0x7f7930137660 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 --2- 172.21.2.116:0/737608194 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7930067890 0x7f79301521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.890+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 >> 172.21.2.116:0/737608194 conn(0x7f793005bd20 msgr2=0x7f7930138410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.894+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 shutdown_connections 2021-11-21T21:59:21.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.894+0000 7f793655b700 1 -- 172.21.2.116:0/737608194 wait complete. 2021-11-21T21:59:21.901 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T21:59:21.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.950+0000 7fde67691700 1 Processor -- start 2021-11-21T21:59:21.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.950+0000 7fde67691700 1 -- start start 2021-11-21T21:59:21.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.950+0000 7fde67691700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fde600adc10 0x7fde600b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.950+0000 7fde67691700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde600b3c50 con 0x7fde60144a10 2021-11-21T21:59:21.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.950+0000 7fde67691700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde600b3ef0 con 0x7fde600b3690 2021-11-21T21:59:21.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.950+0000 7fde67691700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde600b4190 con 0x7fde600adc10 2021-11-21T21:59:21.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde6542d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fde600adc10 0x7fde600b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde6542d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fde600adc10 0x7fde600b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48498/0 (socket says 172.21.2.116:48498) 2021-11-21T21:59:21.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde6542d700 1 -- 172.21.2.116:0/3665884588 learned_addr learned my addr 172.21.2.116:0/3665884588 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde6542d700 1 -- 172.21.2.116:0/3665884588 >> v1:172.21.2.117:6789/0 conn(0x7fde600b3690 legacy=0x7fde6013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:21.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde6542d700 1 -- 172.21.2.116:0/3665884588 >> v1:172.21.2.116:6789/0 conn(0x7fde60144a10 legacy=0x7fde60141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde6542d700 1 -- 172.21.2.116:0/3665884588 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60140b80 con 0x7fde600adc10 2021-11-21T21:59:21.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde6542d700 1 --2- 172.21.2.116:0/3665884588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde600adc10 0x7fde600b2030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fde600b7000 tx=0x7fde50006eb0).ready entity=mon.0 client_cookie=131f64f597bd24bb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde4ffff700 1 -- 172.21.2.116:0/3665884588 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde5000d610 con 0x7fde600adc10 2021-11-21T21:59:21.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde4ffff700 1 -- 172.21.2.116:0/3665884588 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde50000e90 con 0x7fde600adc10 2021-11-21T21:59:21.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde4ffff700 1 -- 172.21.2.116:0/3665884588 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde5000d960 con 0x7fde600adc10 2021-11-21T21:59:21.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- 172.21.2.116:0/3665884588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde600adc10 msgr2=0x7fde600b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 --2- 172.21.2.116:0/3665884588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde600adc10 0x7fde600b2030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fde600b7000 tx=0x7fde50006eb0).stop 2021-11-21T21:59:21.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- 172.21.2.116:0/3665884588 shutdown_connections 2021-11-21T21:59:21.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 --2- 172.21.2.116:0/3665884588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde600adc10 0x7fde600b2030 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:21.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- 172.21.2.116:0/3665884588 >> 172.21.2.116:0/3665884588 conn(0x7fde6005bd20 msgr2=0x7fde6005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:21.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- 172.21.2.116:0/3665884588 shutdown_connections 2021-11-21T21:59:21.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- 172.21.2.116:0/3665884588 wait complete. 2021-11-21T21:59:21.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 Processor -- start 2021-11-21T21:59:21.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- start start 2021-11-21T21:59:21.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde600adc10 0x7fde6013cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde600b3690 0x7fde6013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde60144a10 0x7fde60140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6014da60 con 0x7fde60144a10 2021-11-21T21:59:21.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde600afbd0 con 0x7fde600b3690 2021-11-21T21:59:21.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6014ea20 con 0x7fde600adc10 2021-11-21T21:59:21.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde60144a10 0x7fde60140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde60144a10 0x7fde60140280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48506/0 (socket says 172.21.2.116:48506) 2021-11-21T21:59:21.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 -- 172.21.2.116:0/3628159485 learned_addr learned my addr 172.21.2.116:0/3628159485 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:21.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 -- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde600adc10 msgr2=0x7fde6013cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde600adc10 0x7fde6013cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 -- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde600b3690 msgr2=0x7fde6013fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:21.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde600b3690 0x7fde6013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:21.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 -- 172.21.2.116:0/3628159485 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5000b1e0 con 0x7fde60144a10 2021-11-21T21:59:21.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde65c2e700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde60144a10 0x7fde60140280 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fde600b14f0 tx=0x7fde580091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde4e7fc700 1 -- 172.21.2.116:0/3628159485 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde58001500 con 0x7fde60144a10 2021-11-21T21:59:21.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde4e7fc700 1 -- 172.21.2.116:0/3628159485 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58001660 con 0x7fde60144a10 2021-11-21T21:59:21.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde600af690 con 0x7fde60144a10 2021-11-21T21:59:21.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde60111690 con 0x7fde60144a10 2021-11-21T21:59:21.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.954+0000 7fde4e7fc700 1 -- 172.21.2.116:0/3628159485 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde580019c0 con 0x7fde60144a10 2021-11-21T21:59:21.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.958+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde60151110 con 0x7fde60144a10 2021-11-21T21:59:21.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.958+0000 7fde4e7fc700 1 -- 172.21.2.116:0/3628159485 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fde58018030 con 0x7fde60144a10 2021-11-21T21:59:21.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.958+0000 7fde4e7fc700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde40053580 0x7fde40055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:21.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.958+0000 7fde4e7fc700 1 -- 172.21.2.116:0/3628159485 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fde5806b340 con 0x7fde60144a10 2021-11-21T21:59:21.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.958+0000 7fde6542d700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde40053580 0x7fde40055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:21.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.958+0000 7fde6542d700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde40053580 0x7fde40055a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fde600b7000 tx=0x7fde50015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:21.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:21.958+0000 7fde4e7fc700 1 -- 172.21.2.116:0/3628159485 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde58042440 con 0x7fde60144a10 2021-11-21T21:59:22.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.042+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde600573d0 con 0x7fde60144a10 2021-11-21T21:59:22.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.042+0000 7fde4e7fc700 1 -- 172.21.2.116:0/3628159485 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+10 (secure 0 0 0) 0x7fde58045a60 con 0x7fde60144a10 2021-11-21T21:59:22.047 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:59:22.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde40053580 msgr2=0x7fde40055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde40053580 0x7fde40055a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fde600b7000 tx=0x7fde50015000).stop 2021-11-21T21:59:22.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde60144a10 msgr2=0x7fde60140280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde60144a10 0x7fde60140280 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fde600b14f0 tx=0x7fde580091a0).stop 2021-11-21T21:59:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 shutdown_connections 2021-11-21T21:59:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde40053580 0x7fde40055a40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde60144a10 0x7fde60140280 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde600b3690 0x7fde6013fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 --2- 172.21.2.116:0/3628159485 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde600adc10 0x7fde6013cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 >> 172.21.2.116:0/3628159485 conn(0x7fde6005bd20 msgr2=0x7fde6013e440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:22.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 shutdown_connections 2021-11-21T21:59:22.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.046+0000 7fde67691700 1 -- 172.21.2.116:0/3628159485 wait complete. 2021-11-21T21:59:22.054 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T21:59:22.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 Processor -- start 2021-11-21T21:59:22.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- start start 2021-11-21T21:59:22.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc144810 0x7facfc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7facfc0ac600 0x7facfc0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facfc147e80 con 0x7facfc0abce0 2021-11-21T21:59:22.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facfc0b1490 con 0x7facfc0ac600 2021-11-21T21:59:22.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0b1fd0 con 0x7facfc144810 2021-11-21T21:59:22.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7facfc0ac600 0x7facfc0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:22.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7facfc0ac600 0x7facfc0aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48512/0 (socket says 172.21.2.116:48512) 2021-11-21T21:59:22.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 -- 172.21.2.116:0/1842256591 learned_addr learned my addr 172.21.2.116:0/1842256591 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:22.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 -- 172.21.2.116:0/1842256591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc144810 msgr2=0x7facfc1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:22.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 --2- 172.21.2.116:0/1842256591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc144810 0x7facfc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 -- 172.21.2.116:0/1842256591 >> v1:172.21.2.116:6789/0 conn(0x7facfc0abce0 legacy=0x7facfc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 -- 172.21.2.116:0/1842256591 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0af5a0 con 0x7facfc0ac600 2021-11-21T21:59:22.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facfad9d700 1 --2- 172.21.2.116:0/1842256591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc0aca90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7facf40032f0 tx=0x7facf4009290).ready entity=mon.0 client_cookie=13f30953a206c2c0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:22.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facf959a700 1 -- 172.21.2.116:0/1842256591 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf400b8a0 con 0x7facfc0ac600 2021-11-21T21:59:22.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facf959a700 1 -- 172.21.2.116:0/1842256591 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf400ba00 con 0x7facfc0ac600 2021-11-21T21:59:22.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7facf959a700 1 -- 172.21.2.116:0/1842256591 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf400bb70 con 0x7facfc0ac600 2021-11-21T21:59:22.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- 172.21.2.116:0/1842256591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 msgr2=0x7facfc0aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1842256591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc0aca90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7facf40032f0 tx=0x7facf4009290).stop 2021-11-21T21:59:22.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- 172.21.2.116:0/1842256591 shutdown_connections 2021-11-21T21:59:22.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1842256591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc0aca90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1842256591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc144810 0x7facfc1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- 172.21.2.116:0/1842256591 >> 172.21.2.116:0/1842256591 conn(0x7facfc05bd20 msgr2=0x7facfc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:22.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- 172.21.2.116:0/1842256591 shutdown_connections 2021-11-21T21:59:22.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- 172.21.2.116:0/1842256591 wait complete. 2021-11-21T21:59:22.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 Processor -- start 2021-11-21T21:59:22.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- start start 2021-11-21T21:59:22.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc0abce0 0x7facfc149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7facfc144810 0x7facfc15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0a74c0 con 0x7facfc0ac600 2021-11-21T21:59:22.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0a7760 con 0x7facfc144810 2021-11-21T21:59:22.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.106+0000 7fad00e3b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc159f00 con 0x7facfc0abce0 2021-11-21T21:59:22.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:22.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc15a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48520/0 (socket says 172.21.2.116:48520) 2021-11-21T21:59:22.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 -- 172.21.2.116:0/1448260231 learned_addr learned my addr 172.21.2.116:0/1448260231 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:22.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 -- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc0abce0 msgr2=0x7facfc149850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:22.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc0abce0 0x7facfc149850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 -- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7facfc144810 msgr2=0x7facfc15a890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:22.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7facfc144810 0x7facfc15a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 -- 172.21.2.116:0/1448260231 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4009530 con 0x7facfc0ac600 2021-11-21T21:59:22.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf9d9b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc15a3a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7facfc0b2a30 tx=0x7facec007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:22.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf37fe700 1 -- 172.21.2.116:0/1448260231 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facec00ce30 con 0x7facfc0ac600 2021-11-21T21:59:22.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf37fe700 1 -- 172.21.2.116:0/1448260231 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec00f470 con 0x7facfc0ac600 2021-11-21T21:59:22.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf37fe700 1 -- 172.21.2.116:0/1448260231 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facec00f7d0 con 0x7facfc0ac600 2021-11-21T21:59:22.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc142460 con 0x7facfc0ac600 2021-11-21T21:59:22.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc0a6fb0 con 0x7facfc0ac600 2021-11-21T21:59:22.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facfc0b3000 con 0x7facfc0ac600 2021-11-21T21:59:22.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf37fe700 1 -- 172.21.2.116:0/1448260231 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7facec019030 con 0x7facfc0ac600 2021-11-21T21:59:22.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf37fe700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7face0053580 0x7face0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facf37fe700 1 -- 172.21.2.116:0/1448260231 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7facec06ade0 con 0x7facfc0ac600 2021-11-21T21:59:22.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facfa59c700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7face0053580 0x7face0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:22.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.110+0000 7facfa59c700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7face0053580 0x7face0055a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7face4002f70 tx=0x7face4006960).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:22.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.114+0000 7facf37fe700 1 -- 172.21.2.116:0/1448260231 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7facfc0b3000 con 0x7facfc0ac600 2021-11-21T21:59:22.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facfc0b3000 con 0x7facfc0ac600 2021-11-21T21:59:22.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7facf37fe700 1 -- 172.21.2.116:0/1448260231 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+10 (secure 0 0 0) 0x7facfc0b3000 con 0x7facfc0ac600 2021-11-21T21:59:22.201 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:59:22.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7face0053580 msgr2=0x7face0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7face0053580 0x7face0055a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7face4002f70 tx=0x7face4006960).stop 2021-11-21T21:59:22.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 msgr2=0x7facfc15a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc15a3a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7facfc0b2a30 tx=0x7facec007c50).stop 2021-11-21T21:59:22.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 shutdown_connections 2021-11-21T21:59:22.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7facfc144810 0x7facfc15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7face0053580 0x7face0055a40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7facfc0ac600 0x7facfc15a3a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 --2- 172.21.2.116:0/1448260231 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7facfc0abce0 0x7facfc149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 >> 172.21.2.116:0/1448260231 conn(0x7facfc05bd20 msgr2=0x7facfc140720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:22.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 shutdown_connections 2021-11-21T21:59:22.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.198+0000 7fad00e3b700 1 -- 172.21.2.116:0/1448260231 wait complete. 2021-11-21T21:59:22.208 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T21:59:22.209 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:22.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.254+0000 7fc3f9dce700 1 Processor -- start 2021-11-21T21:59:22.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.254+0000 7fc3f9dce700 1 -- start start 2021-11-21T21:59:22.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc3f40a86b0 0x7fc3f414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc3f4062410 0x7fc3f41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f40a6e80 con 0x7fc3f40a6aa0 2021-11-21T21:59:22.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f40a8bc0 con 0x7fc3f4062410 2021-11-21T21:59:22.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f414a7b0 con 0x7fc3f40a86b0 2021-11-21T21:59:22.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc3f4062410 0x7fc3f41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:22.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc3f40a6aa0 0x7fc3f41470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60482/0 (socket says 172.21.2.116:60482) 2021-11-21T21:59:22.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 -- 172.21.2.116:0/2217643097 learned_addr learned my addr 172.21.2.116:0/2217643097 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:22.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f3fff700 1 -- 172.21.2.116:0/2217643097 >> v2:172.21.2.117:3300/0 conn(0x7fc3f40a86b0 msgr2=0x7fc3f414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:22.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f3fff700 1 --2- 172.21.2.116:0/2217643097 >> v2:172.21.2.117:3300/0 conn(0x7fc3f40a86b0 0x7fc3f414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f3fff700 1 -- 172.21.2.116:0/2217643097 >> v1:172.21.2.116:6789/0 conn(0x7fc3f40a6aa0 legacy=0x7fc3f41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f3fff700 1 -- 172.21.2.116:0/2217643097 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f414aa10 con 0x7fc3f4062410 2021-11-21T21:59:22.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f3fff700 1 --2- 172.21.2.116:0/2217643097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f41443f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fc3ec0032f0 tx=0x7fc3ec0091d0).ready entity=mon.0 client_cookie=4d4330ecb7767fb6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:22.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f27fc700 1 -- 172.21.2.116:0/2217643097 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3ec009ea0 con 0x7fc3f4062410 2021-11-21T21:59:22.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f27fc700 1 -- 172.21.2.116:0/2217643097 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec00b870 con 0x7fc3f4062410 2021-11-21T21:59:22.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f27fc700 1 -- 172.21.2.116:0/2217643097 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3ec00b9e0 con 0x7fc3f4062410 2021-11-21T21:59:22.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2217643097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 msgr2=0x7fc3f41443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2217643097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f41443f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fc3ec0032f0 tx=0x7fc3ec0091d0).stop 2021-11-21T21:59:22.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2217643097 shutdown_connections 2021-11-21T21:59:22.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2217643097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f41443f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2217643097 >> v2:172.21.2.117:3300/0 conn(0x7fc3f40a86b0 0x7fc3f414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2217643097 >> 172.21.2.116:0/2217643097 conn(0x7fc3f405bd20 msgr2=0x7fc3f405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:22.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2217643097 shutdown_connections 2021-11-21T21:59:22.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2217643097 wait complete. 2021-11-21T21:59:22.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 Processor -- start 2021-11-21T21:59:22.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- start start 2021-11-21T21:59:22.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f4063fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3f40a6aa0 0x7fc3f4066bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3f40a86b0 0x7fc3f40670c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40a95e0 con 0x7fc3f4062410 2021-11-21T21:59:22.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4064800 con 0x7fc3f40a6aa0 2021-11-21T21:59:22.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f9dce700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40aae00 con 0x7fc3f40a86b0 2021-11-21T21:59:22.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f4063fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:22.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f4063fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48534/0 (socket says 172.21.2.116:48534) 2021-11-21T21:59:22.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 -- 172.21.2.116:0/2719193411 learned_addr learned my addr 172.21.2.116:0/2719193411 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:22.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 -- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3f40a86b0 msgr2=0x7fc3f40670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3f40a86b0 0x7fc3f40670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 -- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3f40a6aa0 msgr2=0x7fc3f4066bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3f40a6aa0 0x7fc3f4066bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 -- 172.21.2.116:0/2719193411 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec009470 con 0x7fc3f4062410 2021-11-21T21:59:22.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f37fe700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f4063fc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fc3f4064940 tx=0x7fc3dc007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:22.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.258+0000 7fc3f0ff9700 1 -- 172.21.2.116:0/2719193411 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3dc00ce20 con 0x7fc3f4062410 2021-11-21T21:59:22.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f0ff9700 1 -- 172.21.2.116:0/2719193411 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0072e0 con 0x7fc3f4062410 2021-11-21T21:59:22.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f0ff9700 1 -- 172.21.2.116:0/2719193411 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3dc0057e0 con 0x7fc3f4062410 2021-11-21T21:59:22.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f41476e0 con 0x7fc3f4062410 2021-11-21T21:59:22.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f40aaaf0 con 0x7fc3f4062410 2021-11-21T21:59:22.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f40573d0 con 0x7fc3f4062410 2021-11-21T21:59:22.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f0ff9700 1 -- 172.21.2.116:0/2719193411 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc3dc010070 con 0x7fc3f4062410 2021-11-21T21:59:22.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f0ff9700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3d8053640 0x7fc3d8055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:22.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f0ff9700 1 -- 172.21.2.116:0/2719193411 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc3dc06aef0 con 0x7fc3f4062410 2021-11-21T21:59:22.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f2ffd700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3d8053640 0x7fc3d8055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:22.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f2ffd700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3d8053640 0x7fc3d8055b00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc3e4002f70 tx=0x7fc3e40069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:22.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.262+0000 7fc3f0ff9700 1 -- 172.21.2.116:0/2719193411 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc3dc042050 con 0x7fc3f4062410 2021-11-21T21:59:22.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f40aa850 con 0x7fc3f4062410 2021-11-21T21:59:22.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f0ff9700 1 -- 172.21.2.116:0/2719193411 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fc3dc018070 con 0x7fc3f4062410 2021-11-21T21:59:22.374 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:22.375 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:20.167894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3617788104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3617788104},{"type":"v1","addr":"172.21.2.116:6835","nonce":3617788104}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:22.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3d8053640 msgr2=0x7fc3d8055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3d8053640 0x7fc3d8055b00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc3e4002f70 tx=0x7fc3e40069e0).stop 2021-11-21T21:59:22.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 msgr2=0x7fc3f4063fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:22.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f4063fc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fc3f4064940 tx=0x7fc3dc007c00).stop 2021-11-21T21:59:22.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 shutdown_connections 2021-11-21T21:59:22.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3f40a86b0 0x7fc3f40670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3f40a6aa0 0x7fc3f4066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:22.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3d8053640 0x7fc3d8055b00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 --2- 172.21.2.116:0/2719193411 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3f4062410 0x7fc3f4063fc0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:22.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 >> 172.21.2.116:0/2719193411 conn(0x7fc3f405bd20 msgr2=0x7fc3f41441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:22.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 shutdown_connections 2021-11-21T21:59:22.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:22.370+0000 7fc3f9dce700 1 -- 172.21.2.116:0/2719193411 wait complete. 2021-11-21T21:59:22.379 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 28 2021-11-21T21:59:22.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T21:59:19.164615+0000', 'modified': '2021-11-21T21:59:20.167894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5973': {'gid': 5973, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3617788104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3617788104}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3617788104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T21:59:22.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T21:59:22.684 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T21:59:22.747 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":5973,"want_state":"up:active","state":"up:active","rank_uptime":3.5770403179999999,"mdsmap_epoch":28,"osdmap_epoch":66,"osdmap_epoch_barrier":65,"uptime":7.697590581} 2021-11-21T21:59:22.748 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 5973, "mdsmap_epoch": 28, "osdmap_epoch": 66, "osdmap_epoch_barrier": 65, "rank_uptime": 3.577040318, "state": "up:active", "uptime": 7.697590581, "want_state": "up:active", "whoami": 0 } 2021-11-21T21:59:22.749 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T21:59:22.749 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T21:59:22.749 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T21:59:22.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:22.750 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:22.761 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:59:22.773 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:59:22.773 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:59:22.823 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:59:22.824 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:59:22.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:22.825 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:59:22.830 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T21:59:22.831 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:22.833 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:59:22.843 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:59:22.844 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:59:22.849 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:59:22.850 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:59:22.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:22.851 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:59:22.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14514]: starting ceph client 2021-11-21T21:59:22.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T21:59:22.873+0000 7f2c349aa080 -1 init, newargv = 0x55f2ac8a0f20 newargc=9 2021-11-21T21:59:22.887 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14514]: starting fuse 2021-11-21T21:59:23.979 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T21:59:23.991 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T21:59:23.991 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T21:59:24.039 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T21:59:24.040 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T21:59:24.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:24.041 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T21:59:24.046 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T21:59:24.047 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T21:59:24.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:24.048 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T21:59:24.049 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T21:59:24.049 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T21:59:24.049 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:59:24.050 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T21:59:24.050 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:59:24.051 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T21:59:24.051 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T21:59:24.052 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T21:59:24.052 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:59:24.052 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T21:59:24.053 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T21:59:24.053 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T21:59:24.053 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T21:59:24.054 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:59:24.054 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T21:59:24.055 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T21:59:24.055 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T21:59:24.056 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T21:59:24.056 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T21:59:24.056 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:59:24.057 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T21:59:24.058 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:59:24.085 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14514.asok 2021-11-21T21:59:24.087 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14514.asok 2021-11-21T21:59:24.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:24.089 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14514.asok status 2021-11-21T21:59:24.179 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T21:59:24.179 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T21:59:24.179 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T21:59:24.180 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T21:59:24.180 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T21:59:24.180 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T21:59:24.180 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T21:59:24.181 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14514", 2021-11-21T21:59:24.181 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T21:59:24.181 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:59:24.181 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T21:59:24.181 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T21:59:24.182 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 6132, 2021-11-21T21:59:24.182 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T21:59:24.183 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T21:59:24.183 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T21:59:24.183 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 6132 2021-11-21T21:59:24.183 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:59:24.184 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:59:24.184 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:59:24.185 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:59:24.185 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2764025405 2021-11-21T21:59:24.186 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T21:59:24.186 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:59:24.186 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T21:59:24.186 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T21:59:24.187 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T21:59:24.187 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2764025405 2021-11-21T21:59:24.187 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T21:59:24.188 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.6132 172.21.2.119:0/2764025405", 2021-11-21T21:59:24.188 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/2764025405", 2021-11-21T21:59:24.188 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T21:59:24.188 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 28, 2021-11-21T21:59:24.189 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 66, 2021-11-21T21:59:24.189 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T21:59:24.189 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T21:59:24.189 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T21:59:24.189 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T21:59:24.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:24.191 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:24.192 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:24.197 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:59:24.197 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:59:24.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:24.199 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:24.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:24.209 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:24.210 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:24.258 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:59:24.259 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:59:24.260 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T21:59:24.261 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-11-21T21:59:24.261 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-11-21T21:59:24.261 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T21:59:24.262 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:59:24.262 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-11-21T21:59:24.263 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-11-21T21:59:24.263 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:59:24.263 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-11-21T21:59:24.264 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-11-21T21:59:24.264 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T21:59:24.265 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-11-21T21:59:24.265 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-11-21T21:59:24.266 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-11-21T21:59:24.266 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-11-21T21:59:24.267 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-11-21T21:59:24.267 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-11-21T21:59:24.268 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-11-21T21:59:24.269 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T21:59:34.185 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:34.188 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:34.191 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:34.194 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T21:59:38.773 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.830+0000 7fd5be11a700 1 Processor -- start 2021-11-21T21:59:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.830+0000 7fd5be11a700 1 -- start start 2021-11-21T21:59:38.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.830+0000 7fd5be11a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd5b80a6aa0 0x7fd5b814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:38.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.830+0000 7fd5be11a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 0x7fd5b81470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:38.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.830+0000 7fd5be11a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b80a6e80 con 0x7fd5b81475d0 2021-11-21T21:59:38.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.830+0000 7fd5be11a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b80a8bc0 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.830+0000 7fd5be11a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b814a7b0 con 0x7fd5b80a86b0 2021-11-21T21:59:38.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd5b80a6aa0 0x7fd5b814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:38.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd5b80a6aa0 0x7fd5b814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48540/0 (socket says 172.21.2.116:48540) 2021-11-21T21:59:38.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/3544811146 learned_addr learned my addr 172.21.2.116:0/3544811146 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/3544811146 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 msgr2=0x7fd5b81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- 172.21.2.116:0/3544811146 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 0x7fd5b81470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/3544811146 >> v1:172.21.2.117:6789/0 conn(0x7fd5b81475d0 legacy=0x7fd5b81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/3544811146 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b814aa10 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- 172.21.2.116:0/3544811146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b814b150 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd5b8061ab0 tx=0x7fd5a0006eb0).ready entity=mon.0 client_cookie=bd63a6a2bd166393 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b67fc700 1 -- 172.21.2.116:0/3544811146 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5a000d570 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b67fc700 1 -- 172.21.2.116:0/3544811146 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a0000e90 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b67fc700 1 -- 172.21.2.116:0/3544811146 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5a000d8c0 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- 172.21.2.116:0/3544811146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 msgr2=0x7fd5b814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 --2- 172.21.2.116:0/3544811146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b814b150 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd5b8061ab0 tx=0x7fd5a0006eb0).stop 2021-11-21T21:59:38.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- 172.21.2.116:0/3544811146 shutdown_connections 2021-11-21T21:59:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 --2- 172.21.2.116:0/3544811146 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 0x7fd5b81470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 --2- 172.21.2.116:0/3544811146 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b814b150 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- 172.21.2.116:0/3544811146 >> 172.21.2.116:0/3544811146 conn(0x7fd5b805bd20 msgr2=0x7fd5b805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:38.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- 172.21.2.116:0/3544811146 shutdown_connections 2021-11-21T21:59:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- 172.21.2.116:0/3544811146 wait complete. 2021-11-21T21:59:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 Processor -- start 2021-11-21T21:59:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- start start 2021-11-21T21:59:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b8142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:38.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 0x7fd5b8156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:38.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd5b81475d0 0x7fd5b8064690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:38.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81565a0 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8065240 con 0x7fd5b81475d0 2021-11-21T21:59:38.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8064fa0 con 0x7fd5b80a86b0 2021-11-21T21:59:38.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b8142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:38.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b8142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48546/0 (socket says 172.21.2.116:48546) 2021-11-21T21:59:38.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/105296265 learned_addr learned my addr 172.21.2.116:0/105296265 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:38.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 msgr2=0x7fd5b8156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:38.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 0x7fd5b8156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:38.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd5b81475d0 msgr2=0x7fd5b8064690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:38.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd5b81475d0 0x7fd5b8064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:38.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 -- 172.21.2.116:0/105296265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a000b1e0 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b77fe700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b8142050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd5b8061ab0 tx=0x7fd5a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:38.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b4ff9700 1 -- 172.21.2.116:0/105296265 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5a000dea0 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b4ff9700 1 -- 172.21.2.116:0/105296265 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a0002f70 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b8156790 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5b4ff9700 1 -- 172.21.2.116:0/105296265 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5a0006b80 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.834+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b8064b80 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.838+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b80573d0 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.838+0000 7fd5b4ff9700 1 -- 172.21.2.116:0/105296265 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd5a0025030 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.838+0000 7fd5b4ff9700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd59c0536a0 0x7fd59c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:38.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.838+0000 7fd5b4ff9700 1 -- 172.21.2.116:0/105296265 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd5a006cdb0 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.838+0000 7fd5b6ffd700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd59c0536a0 0x7fd59c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:38.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.838+0000 7fd5b6ffd700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd59c0536a0 0x7fd59c055b60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fd5a8002f70 tx=0x7fd5a8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:38.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.838+0000 7fd5b4ff9700 1 -- 172.21.2.116:0/105296265 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5a0043e00 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.946+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5b805db70 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5b4ff9700 1 -- 172.21.2.116:0/105296265 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fd5a0023020 con 0x7fd5b80a6aa0 2021-11-21T21:59:38.952 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:38.953 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:20.167894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3617788104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3617788104},{"type":"v1","addr":"172.21.2.116:6835","nonce":3617788104}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:38.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd59c0536a0 msgr2=0x7fd59c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:38.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd59c0536a0 0x7fd59c055b60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fd5a8002f70 tx=0x7fd5a8006a80).stop 2021-11-21T21:59:38.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 msgr2=0x7fd5b8142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:38.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b8142050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd5b8061ab0 tx=0x7fd5a0015000).stop 2021-11-21T21:59:38.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 shutdown_connections 2021-11-21T21:59:38.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd5b81475d0 0x7fd5b8064690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:38.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd5b80a86b0 0x7fd5b8156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:38.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd59c0536a0 0x7fd59c055b60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:38.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 --2- 172.21.2.116:0/105296265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd5b80a6aa0 0x7fd5b8142050 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:38.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 >> 172.21.2.116:0/105296265 conn(0x7fd5b805bd20 msgr2=0x7fd5b8148850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:38.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 shutdown_connections 2021-11-21T21:59:38.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:38.950+0000 7fd5be11a700 1 -- 172.21.2.116:0/105296265 wait complete. 2021-11-21T21:59:38.956 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 28 2021-11-21T21:59:38.959 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2021-11-21T21:59:39.017 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-11-21T21:59:39.018 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-11-21T21:59:39.018 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:39.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 Processor -- start 2021-11-21T21:59:39.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- start start 2021-11-21T21:59:39.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a040aed90 0x7f7a040b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040af360 0x7f7a04069e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a040b00d0 con 0x7f7a040b24a0 2021-11-21T21:59:39.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a040adba0 con 0x7f7a040aed90 2021-11-21T21:59:39.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040b2880 con 0x7f7a040af360 2021-11-21T21:59:39.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a037fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7a040b24a0 0x7f7a040ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60510/0 (socket says 172.21.2.116:60510) 2021-11-21T21:59:39.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a037fe700 1 -- 172.21.2.116:0/2669554540 learned_addr learned my addr 172.21.2.116:0/2669554540 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:39.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a02ffd700 1 --2- 172.21.2.116:0/2669554540 >> v2:172.21.2.116:3300/0 conn(0x7f7a040aed90 0x7f7a040b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a02ffd700 1 -- 172.21.2.116:0/2669554540 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040af360 msgr2=0x7f7a04069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a02ffd700 1 --2- 172.21.2.116:0/2669554540 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040af360 0x7f7a04069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a02ffd700 1 -- 172.21.2.116:0/2669554540 >> v1:172.21.2.116:6789/0 conn(0x7f7a040b24a0 legacy=0x7f7a040ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a02ffd700 1 -- 172.21.2.116:0/2669554540 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a040b14a0 con 0x7f7a040aed90 2021-11-21T21:59:39.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a02ffd700 1 --2- 172.21.2.116:0/2669554540 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040aed90 0x7f7a040b1a40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f79f4001110 tx=0x7f79f8006df0).ready entity=mon.0 client_cookie=3c6e75187dda23de server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a027fc700 1 -- 172.21.2.116:0/2669554540 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79f800d570 con 0x7f7a040aed90 2021-11-21T21:59:39.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a027fc700 1 -- 172.21.2.116:0/2669554540 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8000e90 con 0x7f7a040aed90 2021-11-21T21:59:39.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a027fc700 1 -- 172.21.2.116:0/2669554540 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79f800d8c0 con 0x7f7a040aed90 2021-11-21T21:59:39.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- 172.21.2.116:0/2669554540 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040aed90 msgr2=0x7f7a040b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/2669554540 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040aed90 0x7f7a040b1a40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f79f4001110 tx=0x7f79f8006df0).stop 2021-11-21T21:59:39.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- 172.21.2.116:0/2669554540 shutdown_connections 2021-11-21T21:59:39.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/2669554540 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040af360 0x7f7a04069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/2669554540 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040aed90 0x7f7a040b1a40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- 172.21.2.116:0/2669554540 >> 172.21.2.116:0/2669554540 conn(0x7f7a0405fd00 msgr2=0x7f7a040600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.066+0000 7f7a0a10a700 1 -- 172.21.2.116:0/2669554540 shutdown_connections 2021-11-21T21:59:39.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- 172.21.2.116:0/2669554540 wait complete. 2021-11-21T21:59:39.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 Processor -- start 2021-11-21T21:59:39.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- start start 2021-11-21T21:59:39.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040aed90 0x7f7a04064e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040af360 0x7f7a04069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a040b24a0 0x7f7a04137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a04140370 con 0x7f7a040af360 2021-11-21T21:59:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a04066320 con 0x7f7a040b24a0 2021-11-21T21:59:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a04066550 con 0x7f7a040aed90 2021-11-21T21:59:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040af360 0x7f7a04069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040af360 0x7f7a04069b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48560/0 (socket says 172.21.2.116:48560) 2021-11-21T21:59:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 -- 172.21.2.116:0/1627493022 learned_addr learned my addr 172.21.2.116:0/1627493022 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 -- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040aed90 msgr2=0x7f7a04064e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040aed90 0x7f7a04064e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 -- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a040b24a0 msgr2=0x7f7a04137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a040b24a0 0x7f7a04137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 -- 172.21.2.116:0/1627493022 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f800b180 con 0x7f7a040af360 2021-11-21T21:59:39.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a02ffd700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040af360 0x7f7a04069b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f79f4001110 tx=0x7f79f8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a00ff9700 1 -- 172.21.2.116:0/1627493022 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79f8016070 con 0x7f7a040af360 2021-11-21T21:59:39.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a00ff9700 1 -- 172.21.2.116:0/1627493022 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8002f50 con 0x7f7a040af360 2021-11-21T21:59:39.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a04142b20 con 0x7f7a040af360 2021-11-21T21:59:39.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a041426b0 con 0x7f7a040af360 2021-11-21T21:59:39.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a00ff9700 1 -- 172.21.2.116:0/1627493022 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79f8006b20 con 0x7f7a040af360 2021-11-21T21:59:39.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc003410 con 0x7f7a040af360 2021-11-21T21:59:39.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a00ff9700 1 -- 172.21.2.116:0/1627493022 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f79f8025030 con 0x7f7a040af360 2021-11-21T21:59:39.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a00ff9700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79e4053640 0x7f79e4055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a00ff9700 1 -- 172.21.2.116:0/1627493022 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f79f806c900 con 0x7f7a040af360 2021-11-21T21:59:39.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.070+0000 7f7a037fe700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79e4053640 0x7f79e4055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.074+0000 7f7a037fe700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79e4053640 0x7f79e4055b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f79f4000bc0 tx=0x7f79f40077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.074+0000 7f7a00ff9700 1 -- 172.21.2.116:0/1627493022 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79f8043950 con 0x7f7a040af360 2021-11-21T21:59:39.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.182+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79dc003c40 con 0x7f7a040af360 2021-11-21T21:59:39.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.182+0000 7f7a00ff9700 1 -- 172.21.2.116:0/1627493022 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f79f8023090 con 0x7f7a040af360 2021-11-21T21:59:39.187 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:39.188 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:20.167894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3617788104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3617788104},{"type":"v1","addr":"172.21.2.116:6835","nonce":3617788104}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:39.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79e4053640 msgr2=0x7f79e4055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79e4053640 0x7f79e4055b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f79f4000bc0 tx=0x7f79f40077f0).stop 2021-11-21T21:59:39.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040af360 msgr2=0x7f7a04069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040af360 0x7f7a04069b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f79f4001110 tx=0x7f79f8014000).stop 2021-11-21T21:59:39.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 shutdown_connections 2021-11-21T21:59:39.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a040b24a0 0x7f7a04137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79e4053640 0x7f79e4055b00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a040af360 0x7f7a04069b50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 --2- 172.21.2.116:0/1627493022 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a040aed90 0x7f7a04064e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 >> 172.21.2.116:0/1627493022 conn(0x7f7a0405fd00 msgr2=0x7f7a040b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 shutdown_connections 2021-11-21T21:59:39.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.186+0000 7f7a0a10a700 1 -- 172.21.2.116:0/1627493022 wait complete. 2021-11-21T21:59:39.191 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 28 2021-11-21T21:59:39.194 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2021-11-21T21:59:39.251 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-11-21T21:59:39.252 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-11-21T21:59:39.252 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:39.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 Processor -- start 2021-11-21T21:59:39.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 -- start start 2021-11-21T21:59:39.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 0x7ff8c414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff8c40a86b0 0x7ff8c41470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff8c41475d0 0x7ff8c41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c40a8bc0 con 0x7ff8c41475d0 2021-11-21T21:59:39.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c40a6e80 con 0x7ff8c40a86b0 2021-11-21T21:59:39.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8cbe2c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c414a7b0 con 0x7ff8c40a6aa0 2021-11-21T21:59:39.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff8c41475d0 0x7ff8c41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff8c41475d0 0x7ff8c41443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48572/0 (socket says 172.21.2.116:48572) 2021-11-21T21:59:39.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/2196585137 learned_addr learned my addr 172.21.2.116:0/2196585137 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/2196585137 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 msgr2=0x7ff8c414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 --2- 172.21.2.116:0/2196585137 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 0x7ff8c414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/2196585137 >> v2:172.21.2.117:3300/0 conn(0x7ff8c40a86b0 msgr2=0x7ff8c41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 --2- 172.21.2.116:0/2196585137 >> v2:172.21.2.117:3300/0 conn(0x7ff8c40a86b0 0x7ff8c41470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/2196585137 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c414aa10 con 0x7ff8c41475d0 2021-11-21T21:59:39.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8ca3c9700 1 --2- 172.21.2.116:0/2196585137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c41443f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7ff8bc0032f0 tx=0x7ff8bc009290).ready entity=mon.0 client_cookie=e1a14616bf7f7f94 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8c8bc6700 1 -- 172.21.2.116:0/2196585137 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8bc00b870 con 0x7ff8c41475d0 2021-11-21T21:59:39.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8c8bc6700 1 -- 172.21.2.116:0/2196585137 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc00b9d0 con 0x7ff8c41475d0 2021-11-21T21:59:39.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.298+0000 7ff8c8bc6700 1 -- 172.21.2.116:0/2196585137 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8bc00bb40 con 0x7ff8c41475d0 2021-11-21T21:59:39.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/2196585137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 msgr2=0x7ff8c41443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/2196585137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c41443f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7ff8bc0032f0 tx=0x7ff8bc009290).stop 2021-11-21T21:59:39.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/2196585137 shutdown_connections 2021-11-21T21:59:39.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/2196585137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c41443f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/2196585137 >> v2:172.21.2.117:3300/0 conn(0x7ff8c40a86b0 0x7ff8c41470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/2196585137 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 0x7ff8c414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/2196585137 >> 172.21.2.116:0/2196585137 conn(0x7ff8c405bd20 msgr2=0x7ff8c405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/2196585137 shutdown_connections 2021-11-21T21:59:39.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/2196585137 wait complete. 2021-11-21T21:59:39.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 Processor -- start 2021-11-21T21:59:39.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- start start 2021-11-21T21:59:39.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 0x7ff8c40b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8c40a86b0 0x7ff8c4146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c4066010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4063bd0 con 0x7ff8c41475d0 2021-11-21T21:59:39.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41469f0 con 0x7ff8c40a86b0 2021-11-21T21:59:39.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4146c90 con 0x7ff8c40a6aa0 2021-11-21T21:59:39.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c4066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c4066010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48578/0 (socket says 172.21.2.116:48578) 2021-11-21T21:59:39.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/3421439019 learned_addr learned my addr 172.21.2.116:0/3421439019 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8c9bc8700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 0x7ff8c40b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8c93c7700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8c40a86b0 0x7ff8c4146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 msgr2=0x7ff8c40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 0x7ff8c40b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8c40a86b0 msgr2=0x7ff8c4146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8c40a86b0 0x7ff8c4146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 -- 172.21.2.116:0/3421439019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc009530 con 0x7ff8c41475d0 2021-11-21T21:59:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8ca3c9700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c4066010 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7ff8bc0032f0 tx=0x7ff8bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8b6ffd700 1 -- 172.21.2.116:0/3421439019 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8bc00b870 con 0x7ff8c41475d0 2021-11-21T21:59:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8b6ffd700 1 -- 172.21.2.116:0/3421439019 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc020620 con 0x7ff8c41475d0 2021-11-21T21:59:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8b6ffd700 1 -- 172.21.2.116:0/3421439019 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8bc020980 con 0x7ff8c41475d0 2021-11-21T21:59:39.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c4146f30 con 0x7ff8c41475d0 2021-11-21T21:59:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c40b41d0 con 0x7ff8c41475d0 2021-11-21T21:59:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c40b11f0 con 0x7ff8c41475d0 2021-11-21T21:59:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8b6ffd700 1 -- 172.21.2.116:0/3421439019 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff8bc026020 con 0x7ff8c41475d0 2021-11-21T21:59:39.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8b6ffd700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8a4053610 0x7ff8a4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.302+0000 7ff8b6ffd700 1 -- 172.21.2.116:0/3421439019 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff8bc06c7b0 con 0x7ff8c41475d0 2021-11-21T21:59:39.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.306+0000 7ff8c9bc8700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8a4053610 0x7ff8a4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.306+0000 7ff8c9bc8700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8a4053610 0x7ff8a4055ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7ff8b800a000 tx=0x7ff8b8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.306+0000 7ff8b6ffd700 1 -- 172.21.2.116:0/3421439019 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8bc043910 con 0x7ff8c41475d0 2021-11-21T21:59:39.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.414+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8c40573d0 con 0x7ff8c41475d0 2021-11-21T21:59:39.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.414+0000 7ff8b6ffd700 1 -- 172.21.2.116:0/3421439019 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2493 (secure 0 0 0) 0x7ff8bc015020 con 0x7ff8c41475d0 2021-11-21T21:59:39.419 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:39.419 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:20.167894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3617788104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3617788104},{"type":"v1","addr":"172.21.2.116:6835","nonce":3617788104}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:39.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.414+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8a4053610 msgr2=0x7ff8a4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.414+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8a4053610 0x7ff8a4055ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7ff8b800a000 tx=0x7ff8b8008040).stop 2021-11-21T21:59:39.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 msgr2=0x7ff8c4066010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c4066010 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7ff8bc0032f0 tx=0x7ff8bc016000).stop 2021-11-21T21:59:39.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 shutdown_connections 2021-11-21T21:59:39.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8a4053610 0x7ff8a4055ad0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8c41475d0 0x7ff8c4066010 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff8c40a86b0 0x7ff8c4146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 --2- 172.21.2.116:0/3421439019 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff8c40a6aa0 0x7ff8c40b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 >> 172.21.2.116:0/3421439019 conn(0x7ff8c405bd20 msgr2=0x7ff8c4148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 shutdown_connections 2021-11-21T21:59:39.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.418+0000 7ff8cbe2c700 1 -- 172.21.2.116:0/3421439019 wait complete. 2021-11-21T21:59:39.423 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 28 2021-11-21T21:59:39.426 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 10 2021-11-21T21:59:39.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f988d5f0700 1 Processor -- start 2021-11-21T21:59:39.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f988d5f0700 1 -- start start 2021-11-21T21:59:39.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f988d5f0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98881478c0 0x7f98880acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f988d5f0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9888147e80 con 0x7f98880abce0 2021-11-21T21:59:39.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f988d5f0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98880af5a0 con 0x7f9888144810 2021-11-21T21:59:39.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f988d5f0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98880b1490 con 0x7f98881478c0 2021-11-21T21:59:39.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f98867fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f98880abce0 0x7f98880ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60540/0 (socket says 172.21.2.116:60540) 2021-11-21T21:59:39.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.522+0000 7f98867fc700 1 -- 172.21.2.116:0/3847126083 learned_addr learned my addr 172.21.2.116:0/3847126083 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:39.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225460412 0 0) 0x7f9888147e80 con 0x7f98880abce0 2021-11-21T21:59:39.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9868003260 con 0x7f98880abce0 2021-11-21T21:59:39.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1707782523 0 0) 0x7f9868003260 con 0x7f98880abce0 2021-11-21T21:59:39.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98881478c0 msgr2=0x7f98880acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98881478c0 0x7f98880acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 >> v1:172.21.2.117:6789/0 conn(0x7f9888144810 legacy=0x7f98880b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880b1fd0 con 0x7f98880abce0 2021-11-21T21:59:39.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9878001680 con 0x7f98880abce0 2021-11-21T21:59:39.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 >> v1:172.21.2.116:6789/0 conn(0x7f98880abce0 legacy=0x7f98880ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9868003800 0x7f9868003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9868004500 0x7f9868004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9868004f90 0x7f986800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98880b1490 con 0x7f9868004500 2021-11-21T21:59:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986800be50 con 0x7f9868004f90 2021-11-21T21:59:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986800c010 con 0x7f9868003800 2021-11-21T21:59:39.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9868004500 0x7f9868004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 -- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9868003800 msgr2=0x7f9868003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9868003800 0x7f9868003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 -- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9868004f90 msgr2=0x7f986800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9868004f90 0x7f986800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 -- 172.21.2.116:0/3847126083 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880b1fd0 con 0x7f9868004500 2021-11-21T21:59:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9868004500 0x7f9868004980 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f9878001040 tx=0x7f9878001e40).ready entity=mon.0 client_cookie=21cc00a2a94bfc70 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9878007170 con 0x7f9868004500 2021-11-21T21:59:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98780072d0 con 0x7f9868004500 2021-11-21T21:59:39.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f9885ffb700 1 -- 172.21.2.116:0/3847126083 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f987800c990 con 0x7f9868004500 2021-11-21T21:59:39.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- 172.21.2.116:0/3847126083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9868004500 msgr2=0x7f9868004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 --2- 172.21.2.116:0/3847126083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9868004500 0x7f9868004980 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f9878001040 tx=0x7f9878001e40).stop 2021-11-21T21:59:39.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98877fe700 1 -- 172.21.2.116:0/3847126083 reap_dead start 2021-11-21T21:59:39.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- 172.21.2.116:0/3847126083 shutdown_connections 2021-11-21T21:59:39.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- 172.21.2.116:0/3847126083 >> 172.21.2.116:0/3847126083 conn(0x7f988805bd20 msgr2=0x7f988805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- 172.21.2.116:0/3847126083 shutdown_connections 2021-11-21T21:59:39.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- 172.21.2.116:0/3847126083 wait complete. 2021-11-21T21:59:39.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 Processor -- start 2021-11-21T21:59:39.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- start start 2021-11-21T21:59:39.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9868004500 0x7f9888145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9888143bd0 0x7f98881424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98881429b0 0x7f988813cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98880632e0 con 0x7f9888143bd0 2021-11-21T21:59:39.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98880b2ab0 con 0x7f98881429b0 2021-11-21T21:59:39.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f988d5f0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98880b2d50 con 0x7f9868004500 2021-11-21T21:59:39.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9888143bd0 0x7f98881424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9888143bd0 0x7f98881424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48596/0 (socket says 172.21.2.116:48596) 2021-11-21T21:59:39.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 -- 172.21.2.116:0/3012634542 learned_addr learned my addr 172.21.2.116:0/3012634542 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 -- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9868004500 msgr2=0x7f9888145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9868004500 0x7f9888145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 -- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98881429b0 msgr2=0x7f988813cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98881429b0 0x7f988813cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 -- 172.21.2.116:0/3012634542 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888142020 con 0x7f9888143bd0 2021-11-21T21:59:39.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.526+0000 7f98867fc700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9888143bd0 0x7f98881424c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f9878001040 tx=0x7f9878015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f987ffff700 1 -- 172.21.2.116:0/3012634542 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9878004920 con 0x7f9888143bd0 2021-11-21T21:59:39.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98880b2ff0 con 0x7f9888143bd0 2021-11-21T21:59:39.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f987ffff700 1 -- 172.21.2.116:0/3012634542 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9878004a80 con 0x7f9888143bd0 2021-11-21T21:59:39.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f987ffff700 1 -- 172.21.2.116:0/3012634542 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9878004de0 con 0x7f9888143bd0 2021-11-21T21:59:39.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9888156690 con 0x7f9888143bd0 2021-11-21T21:59:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f987ffff700 1 -- 172.21.2.116:0/3012634542 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f987801d070 con 0x7f9888143bd0 2021-11-21T21:59:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f987ffff700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f986c0536a0 0x7f986c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f987ffff700 1 -- 172.21.2.116:0/3012634542 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f987806c780 con 0x7f9888143bd0 2021-11-21T21:59:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9886ffd700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f986c0536a0 0x7f986c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 Processor -- start 2021-11-21T21:59:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- start start 2021-11-21T21:59:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f96c90 0x1f5c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f5f090 0x1f57b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f8f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ecd7f0 con 0x1f8f1d0 2021-11-21T21:59:39.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eadc70 con 0x1f96c90 2021-11-21T21:59:39.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f97070 con 0x1f5f090 2021-11-21T21:59:39.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f8f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9886ffd700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f986c0536a0 0x7f986c055b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f9870000f70 tx=0x7f98700069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f8f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48606/0 (socket says 172.21.2.116:48606) 2021-11-21T21:59:39.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 -- 172.21.2.116:0/3174962827 learned_addr learned my addr 172.21.2.116:0/3174962827 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9886ffd700 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f96c90 0x1f5c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98867fc700 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f5f090 0x1f57b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 -- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f5f090 msgr2=0x1f57b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f5f090 0x1f57b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 -- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f96c90 msgr2=0x1f5c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f96c90 0x1f5c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 -- 172.21.2.116:0/3174962827 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f04960 con 0x1f8f1d0 2021-11-21T21:59:39.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f98877fe700 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f8f680 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x1f5f4f0 tx=0x7f9880007bd0).ready entity=mon.0 client_cookie=530c1c45b3f95a05 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f988d5f0700 1 -- 172.21.2.116:0/3174962827 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98800070d0 con 0x1f8f1d0 2021-11-21T21:59:39.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f988d5f0700 1 -- 172.21.2.116:0/3174962827 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880007230 con 0x1f8f1d0 2021-11-21T21:59:39.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f988d5f0700 1 -- 172.21.2.116:0/3174962827 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98800047e0 con 0x1f8f1d0 2021-11-21T21:59:39.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- 172.21.2.116:0/3174962827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 msgr2=0x1f8f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f8f680 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x1f5f4f0 tx=0x7f9880007bd0).stop 2021-11-21T21:59:39.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- 172.21.2.116:0/3174962827 shutdown_connections 2021-11-21T21:59:39.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f8f680 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f5f090 0x1f57b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 --2- 172.21.2.116:0/3174962827 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f96c90 0x1f5c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- 172.21.2.116:0/3174962827 >> 172.21.2.116:0/3174962827 conn(0x1f58f00 msgr2=0x1f96220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- 172.21.2.116:0/3174962827 shutdown_connections 2021-11-21T21:59:39.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.530+0000 7f9897834740 1 -- 172.21.2.116:0/3174962827 wait complete. 2021-11-21T21:59:39.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- start start 2021-11-21T21:59:39.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 Processor -- start 2021-11-21T21:59:39.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1fa5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1fa5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 0x1f61640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa4980 con 0x1f8f1d0 2021-11-21T21:59:39.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f60000 con 0x1f61190 2021-11-21T21:59:39.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f602a0 con 0x1f96c90 2021-11-21T21:59:39.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 10 client.0 ll_register_callbacks cb 0x1ec12f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:59:39.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1fa5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1fa5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48608/0 (socket says 172.21.2.116:48608) 2021-11-21T21:59:39.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 -- 172.21.2.116:0/3833955552 learned_addr learned my addr 172.21.2.116:0/3833955552 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 -- 172.21.2.116:0/3833955552 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 msgr2=0x1fa5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1fa5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 -- 172.21.2.116:0/3833955552 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 msgr2=0x1f61640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 0x1f61640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 -- 172.21.2.116:0/3833955552 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98800026d0 con 0x1f8f1d0 2021-11-21T21:59:39.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9886ffd700 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1fa5050 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f9870011000 tx=0x7f987000f000).ready entity=mon.0 client_cookie=ae8803658d10ac0e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:59:39.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 1 -- 172.21.2.116:0/3833955552 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f987001a070 con 0x1f8f1d0 2021-11-21T21:59:39.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 1 -- 172.21.2.116:0/3833955552 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9870015240 con 0x1f8f1d0 2021-11-21T21:59:39.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ec38b0 con 0x1f8f1d0 2021-11-21T21:59:39.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 1 -- 172.21.2.116:0/3833955552 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f987001f420 con 0x1f8f1d0 2021-11-21T21:59:39.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 10 client.6165 fetch_fsmap learned FSMap version 28 2021-11-21T21:59:39.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x1ebcea0 con 0x1f8f1d0 2021-11-21T21:59:39.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 1 -- 172.21.2.116:0/3833955552 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ec38b0 con 0x1f8f1d0 2021-11-21T21:59:39.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 1 -- 172.21.2.116:0/3833955552 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 28) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f987001d020 con 0x1f8f1d0 2021-11-21T21:59:39.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 10 client.6165 fetch_fsmap finished waiting for FSMap version 28 2021-11-21T21:59:39.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 10 client.6165 resolve_mds: resolved ID 'c' to GID 5973 2021-11-21T21:59:39.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 20 client.6165 populate_metadata read hostname 'gibba016' 2021-11-21T21:59:39.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f95250 0x1f95650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 4 client.6165 mds_command: new command op to 5973 tid=0[{"prefix": "get_command_descriptions"}] 2021-11-21T21:59:39.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 --> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ebb930 con 0x1f95250 2021-11-21T21:59:39.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f98867fc700 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f95250 0x1f95650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 10 client.6165 ms_handle_connect on v2:172.21.2.116:6834/3617788104 2021-11-21T21:59:39.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f98867fc700 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f95250 0x1f95650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8d272a2f57eadc90 server_cookie=7a9c6af17bf24452 in_seq=0 out_seq=0 2021-11-21T21:59:39.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 1 -- 172.21.2.116:0/3833955552 <== mds.0 v2:172.21.2.116:6834/3617788104 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1ebb930 con 0x1f95250 2021-11-21T21:59:39.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 10 client.6165 handle_command_reply: tid=0 2021-11-21T21:59:39.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 client.6165 shutdown 2021-11-21T21:59:39.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 msgr2=0x1fa5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1fa5050 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f9870011000 tx=0x7f987000f000).stop 2021-11-21T21:59:39.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 shutdown_connections 2021-11-21T21:59:39.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f95250 0x1f95650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 0x1f61640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1fa5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 --2- 172.21.2.116:0/3833955552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1fa5050 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9865ffb700 0 client.6165 ms_handle_reset on v2:172.21.2.116:6834/3617788104 2021-11-21T21:59:39.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 >> 172.21.2.116:0/3833955552 conn(0x1f5f090 msgr2=0x1f93400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 shutdown_connections 2021-11-21T21:59:39.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 1 -- 172.21.2.116:0/3833955552 wait complete. 2021-11-21T21:59:39.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.534+0000 7f9897834740 20 client.6165 trim_cache size 0 max 16384 2021-11-21T21:59:39.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 Processor -- start 2021-11-21T21:59:39.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- start start 2021-11-21T21:59:39.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 0x1f915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f9e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1f9eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e9fdc0 con 0x1f8f1d0 2021-11-21T21:59:39.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f64140 con 0x1f61190 2021-11-21T21:59:39.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f23720 con 0x1f96c90 2021-11-21T21:59:39.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f9e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f9e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48618/0 (socket says 172.21.2.116:48618) 2021-11-21T21:59:39.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 -- 172.21.2.116:0/121960209 learned_addr learned my addr 172.21.2.116:0/121960209 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 -- 172.21.2.116:0/121960209 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 msgr2=0x1f9eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:39.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 --2- 172.21.2.116:0/121960209 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1f9eb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 -- 172.21.2.116:0/121960209 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 msgr2=0x1f915c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:39.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 --2- 172.21.2.116:0/121960209 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 0x1f915c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 -- 172.21.2.116:0/121960209 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ec5a90 con 0x1f8f1d0 2021-11-21T21:59:39.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f98867fc700 1 --2- 172.21.2.116:0/121960209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f9e630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x1f94d40 tx=0x7f98780017f0).ready entity=mon.0 client_cookie=50a28629f9a39f73 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9867fff700 1 -- 172.21.2.116:0/121960209 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9878043840 con 0x1f8f1d0 2021-11-21T21:59:39.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9867fff700 1 -- 172.21.2.116:0/121960209 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98780474a0 con 0x1f8f1d0 2021-11-21T21:59:39.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9867fff700 1 -- 172.21.2.116:0/121960209 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98780458a0 con 0x1f8f1d0 2021-11-21T21:59:39.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- 172.21.2.116:0/121960209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 msgr2=0x1f9e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:39.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- 172.21.2.116:0/121960209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f9e630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x1f94d40 tx=0x7f98780017f0).stop 2021-11-21T21:59:39.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- 172.21.2.116:0/121960209 shutdown_connections 2021-11-21T21:59:39.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- 172.21.2.116:0/121960209 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1f9eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- 172.21.2.116:0/121960209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f8f1d0 0x1f9e630 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:39.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- 172.21.2.116:0/121960209 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f61190 0x1f915c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- 172.21.2.116:0/121960209 >> 172.21.2.116:0/121960209 conn(0x1f5f090 msgr2=0x1f5b480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:39.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- 172.21.2.116:0/121960209 shutdown_connections 2021-11-21T21:59:39.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- 172.21.2.116:0/121960209 wait complete. 2021-11-21T21:59:39.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- start start 2021-11-21T21:59:39.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 Processor -- start 2021-11-21T21:59:39.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f61190 0x1f9eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f8f1d0 0x1f9a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1fee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ff7ed0 con 0x1f61190 2021-11-21T21:59:39.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa5550 con 0x1f8f1d0 2021-11-21T21:59:39.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.542+0000 7f9897834740 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f9f0c0 con 0x1f96c90 2021-11-21T21:59:39.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 10 client.0 ll_register_callbacks cb 0x1f623b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-11-21T21:59:39.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f61190 0x1f9eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f61190 0x1f9eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48622/0 (socket says 172.21.2.116:48622) 2021-11-21T21:59:39.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 -- 172.21.2.116:0/988627108 learned_addr learned my addr 172.21.2.116:0/988627108 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:39.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 -- 172.21.2.116:0/988627108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 msgr2=0x1fee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1fee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 -- 172.21.2.116:0/988627108 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f8f1d0 msgr2=0x1f9a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:39.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f8f1d0 0x1f9a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:39.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 -- 172.21.2.116:0/988627108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9888156690 con 0x1f61190 2021-11-21T21:59:39.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9886ffd700 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f61190 0x1f9eb20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x1ff8010 tx=0x7f987001c000).ready entity=mon.0 client_cookie=b1da9ad6466f4921 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:39.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9866ffd700 10 client.0 ms_handle_connect on v2:172.21.2.116:3300/0 2021-11-21T21:59:39.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9866ffd700 1 -- 172.21.2.116:0/988627108 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f987001f820 con 0x1f61190 2021-11-21T21:59:39.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9866ffd700 1 -- 172.21.2.116:0/988627108 <== mon.0 v2:172.21.2.116:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9870015d20 con 0x1f61190 2021-11-21T21:59:39.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9880007230 con 0x1f61190 2021-11-21T21:59:39.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9866ffd700 1 -- 172.21.2.116:0/988627108 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f987001f980 con 0x1f61190 2021-11-21T21:59:39.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 10 client.6171 fetch_fsmap learned FSMap version 28 2021-11-21T21:59:39.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x1ec2480 con 0x1f61190 2021-11-21T21:59:39.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9866ffd700 1 -- 172.21.2.116:0/988627108 <== mon.0 v2:172.21.2.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880007230 con 0x1f61190 2021-11-21T21:59:39.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9866ffd700 1 -- 172.21.2.116:0/988627108 <== mon.0 v2:172.21.2.116:3300/0 5 ==== fsmap(e 28) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f9870032070 con 0x1f61190 2021-11-21T21:59:39.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 10 client.6171 fetch_fsmap finished waiting for FSMap version 28 2021-11-21T21:59:39.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 10 client.6171 resolve_mds: resolved ID 'c' to GID 5973 2021-11-21T21:59:39.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 20 client.6171 populate_metadata read hostname 'gibba016' 2021-11-21T21:59:39.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f5ae80 0x1feea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:39.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 4 client.6171 mds_command: new command op to 5973 tid=0[{"prefix": "cache drop", "timeout": 10}] 2021-11-21T21:59:39.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 --> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x1db15e0 con 0x1f5ae80 2021-11-21T21:59:39.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f98867fc700 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f5ae80 0x1feea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:39.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f98867fc700 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f5ae80 0x1feea70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f24ae524dcebf65d server_cookie=5783da3bc4cb29b6 in_seq=0 out_seq=0 2021-11-21T21:59:39.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:39.546+0000 7f9866ffd700 10 client.6171 ms_handle_connect on v2:172.21.2.116:6834/3617788104 2021-11-21T21:59:49.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9866ffd700 1 -- 172.21.2.116:0/988627108 <== mds.0 v2:172.21.2.116:6834/3617788104 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+400 (crc 0 0 0) 0x1db15e0 con 0x1f5ae80 2021-11-21T21:59:49.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9866ffd700 10 client.6171 handle_command_reply: tid=0 2021-11-21T21:59:49.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 client.6171 shutdown 2021-11-21T21:59:49.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f61190 msgr2=0x1f9eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:49.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f61190 0x1f9eb20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x1ff8010 tx=0x7f987001c000).stop 2021-11-21T21:59:49.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 shutdown_connections 2021-11-21T21:59:49.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:6834/3617788104,v1:172.21.2.116:6835/3617788104] conn(0x1f5ae80 0x1feea70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:49.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x1f96c90 0x1fee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:49.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x1f8f1d0 0x1f9a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:49.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 --2- 172.21.2.116:0/988627108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x1f61190 0x1f9eb20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:49.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 >> 172.21.2.116:0/988627108 conn(0x1f5f090 msgr2=0x1f917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:49.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9866ffd700 0 client.6171 ms_handle_reset on v2:172.21.2.116:6834/3617788104 2021-11-21T21:59:49.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 shutdown_connections 2021-11-21T21:59:49.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 1 -- 172.21.2.116:0/988627108 wait complete. 2021-11-21T21:59:49.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f9897834740 20 client.6171 trim_cache size 0 max 16384 2021-11-21T21:59:49.692 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:59:49.693 INFO:teuthology.orchestra.run.gibba016.stdout: "client_recall": { 2021-11-21T21:59:49.693 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": -110, 2021-11-21T21:59:49.693 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "(110) Connection timed out", 2021-11-21T21:59:49.694 INFO:teuthology.orchestra.run.gibba016.stdout: "recalled": 240 2021-11-21T21:59:49.694 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:49.694 INFO:teuthology.orchestra.run.gibba016.stdout: "flush_journal": { 2021-11-21T21:59:49.695 INFO:teuthology.orchestra.run.gibba016.stdout: "return_code": 0, 2021-11-21T21:59:49.695 INFO:teuthology.orchestra.run.gibba016.stdout: "message": "" 2021-11-21T21:59:49.695 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:49.695 INFO:teuthology.orchestra.run.gibba016.stdout: "trim_cache": { 2021-11-21T21:59:49.696 INFO:teuthology.orchestra.run.gibba016.stdout: "trimmed": 220 2021-11-21T21:59:49.696 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:49.697 INFO:teuthology.orchestra.run.gibba016.stdout: "cache": { 2021-11-21T21:59:49.697 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": { 2021-11-21T21:59:49.697 INFO:teuthology.orchestra.run.gibba016.stdout: "items": 15450, 2021-11-21T21:59:49.697 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes": 2428480 2021-11-21T21:59:49.698 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:49.698 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:49.699 INFO:teuthology.orchestra.run.gibba016.stdout: "duration": 10.135464355 2021-11-21T21:59:49.699 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:59:49.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f986c0536a0 msgr2=0x7f986c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:49.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f986c0536a0 0x7f986c055b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f9870000f70 tx=0x7f98700069b0).stop 2021-11-21T21:59:49.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9888143bd0 msgr2=0x7f98881424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:49.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9888143bd0 0x7f98881424c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f9878001040 tx=0x7f9878015000).stop 2021-11-21T21:59:49.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 shutdown_connections 2021-11-21T21:59:49.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98881429b0 0x7f988813cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:49.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f986c0536a0 0x7f986c055b60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:49.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9888143bd0 0x7f98881424c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:49.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 --2- 172.21.2.116:0/3012634542 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9868004500 0x7f9888145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:49.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 >> 172.21.2.116:0/3012634542 conn(0x7f988805bd20 msgr2=0x7f9888148520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:49.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 shutdown_connections 2021-11-21T21:59:49.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:49.682+0000 7f988d5f0700 1 -- 172.21.2.116:0/3012634542 wait complete. 2021-11-21T21:59:49.705 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:59:49.705 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:49.719 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:49.753 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:49.770 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:49.790 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:59:49.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:49.791 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:49.791 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:49.797 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T21:59:49.799 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T21:59:49.799 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T21:59:49.800 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T21:59:49.800 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:49.854 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14514]: fuse finished with error 0 and tester_r 0 2021-11-21T21:59:49.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:49.856 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:49.856 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:49.862 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T21:59:49.863 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T21:59:49.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:59:49.931 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:49.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:49.938 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:49.939 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:59:49.956 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:59:49.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:49.959 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:59:49.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T21:59:49.960 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:59:50.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:50.010 INFO:teuthology.orchestra.run.gibba036.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:59:50.010 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:59:50.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7f675700 1 Processor -- start 2021-11-21T21:59:50.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7f675700 1 -- start start 2021-11-21T21:59:50.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7f675700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf780a8790 0x7faf7813fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7f675700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780ad340 0x7faf780af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7f675700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7813f170 con 0x7faf780ac750 2021-11-21T21:59:50.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7f675700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf780a8b70 con 0x7faf780a8790 2021-11-21T21:59:50.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7f675700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf780af530 con 0x7faf780ad340 2021-11-21T21:59:50.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf780a8790 0x7faf7813fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf780a8790 0x7faf7813fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48632/0 (socket says 172.21.2.116:48632) 2021-11-21T21:59:50.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 -- 172.21.2.116:0/1108769527 learned_addr learned my addr 172.21.2.116:0/1108769527 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:50.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7dc12700 1 --1- 172.21.2.116:0/1108769527 >> v1:172.21.2.116:6789/0 conn(0x7faf780ac750 0x7faf78142630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60592/0 (socket says 172.21.2.116:60592) 2021-11-21T21:59:50.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 -- 172.21.2.116:0/1108769527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780ad340 msgr2=0x7faf780af8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:50.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 --2- 172.21.2.116:0/1108769527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780ad340 0x7faf780af8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 -- 172.21.2.116:0/1108769527 >> v1:172.21.2.116:6789/0 conn(0x7faf780ac750 legacy=0x7faf78142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 -- 172.21.2.116:0/1108769527 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78062410 con 0x7faf780a8790 2021-11-21T21:59:50.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.126+0000 7faf7d411700 1 --2- 172.21.2.116:0/1108769527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780a8790 0x7faf7813fae0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c006eb0).ready entity=mon.0 client_cookie=ac61374e9369e633 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf6bfff700 1 -- 172.21.2.116:0/1108769527 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c00d5c0 con 0x7faf780a8790 2021-11-21T21:59:50.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf6bfff700 1 -- 172.21.2.116:0/1108769527 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c000e90 con 0x7faf780a8790 2021-11-21T21:59:50.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf6bfff700 1 -- 172.21.2.116:0/1108769527 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c00d950 con 0x7faf780a8790 2021-11-21T21:59:50.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/1108769527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780a8790 msgr2=0x7faf7813fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 --2- 172.21.2.116:0/1108769527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780a8790 0x7faf7813fae0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c006eb0).stop 2021-11-21T21:59:50.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/1108769527 shutdown_connections 2021-11-21T21:59:50.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 --2- 172.21.2.116:0/1108769527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780ad340 0x7faf780af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 --2- 172.21.2.116:0/1108769527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780a8790 0x7faf7813fae0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:50.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/1108769527 >> 172.21.2.116:0/1108769527 conn(0x7faf7805bd20 msgr2=0x7faf7805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:50.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/1108769527 shutdown_connections 2021-11-21T21:59:50.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/1108769527 wait complete. 2021-11-21T21:59:50.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 Processor -- start 2021-11-21T21:59:50.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- start start 2021-11-21T21:59:50.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780a8790 0x7faf78144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780ac750 0x7faf78152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf780ad340 0x7faf78152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf780b56a0 con 0x7faf780ac750 2021-11-21T21:59:50.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf780ab650 con 0x7faf780ad340 2021-11-21T21:59:50.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf780ab880 con 0x7faf780a8790 2021-11-21T21:59:50.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780ac750 0x7faf78152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780ac750 0x7faf78152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48642/0 (socket says 172.21.2.116:48642) 2021-11-21T21:59:50.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 -- 172.21.2.116:0/3053836919 learned_addr learned my addr 172.21.2.116:0/3053836919 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:50.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 -- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780a8790 msgr2=0x7faf78144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780a8790 0x7faf78144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 -- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf780ad340 msgr2=0x7faf78152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf780ad340 0x7faf78152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 -- 172.21.2.116:0/3053836919 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c00b1e0 con 0x7faf780ac750 2021-11-21T21:59:50.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7cc10700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780ac750 0x7faf78152120 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7faf74002ef0 tx=0x7faf74006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf6a7fc700 1 -- 172.21.2.116:0/3053836919 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf74009e00 con 0x7faf780ac750 2021-11-21T21:59:50.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf780a9490 con 0x7faf780ac750 2021-11-21T21:59:50.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf780a9090 con 0x7faf780ac750 2021-11-21T21:59:50.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf6a7fc700 1 -- 172.21.2.116:0/3053836919 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7401c440 con 0x7faf780ac750 2021-11-21T21:59:50.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf6a7fc700 1 -- 172.21.2.116:0/3053836919 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf7401c7a0 con 0x7faf780ac750 2021-11-21T21:59:50.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.130+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf54003410 con 0x7faf780ac750 2021-11-21T21:59:50.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.134+0000 7faf6a7fc700 1 -- 172.21.2.116:0/3053836919 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7faf7400f030 con 0x7faf780ac750 2021-11-21T21:59:50.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.134+0000 7faf6a7fc700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf580536a0 0x7faf58055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.134+0000 7faf6a7fc700 1 -- 172.21.2.116:0/3053836919 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faf7406b730 con 0x7faf780ac750 2021-11-21T21:59:50.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.134+0000 7faf7d411700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf580536a0 0x7faf58055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.134+0000 7faf7d411700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf580536a0 0x7faf58055b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.138+0000 7faf6a7fc700 1 -- 172.21.2.116:0/3053836919 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faf740426a0 con 0x7faf780ac750 2021-11-21T21:59:50.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.218+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faf540036c0 con 0x7faf780ac750 2021-11-21T21:59:50.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.218+0000 7faf6a7fc700 1 -- 172.21.2.116:0/3053836919 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v66) v1 ==== 81+0+33648 (secure 0 0 0) 0x7faf74045cc0 con 0x7faf780ac750 2021-11-21T21:59:50.225 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:50.226 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:59:50.226 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 66, 2021-11-21T21:59:50.227 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:59:50.227 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:59:50.227 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:59:20.169181+0000", 2021-11-21T21:59:50.228 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:50.228 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:50.229 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:59:50.230 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:59:50.231 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:59:50.231 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:59:50.232 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:59:50.232 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:59:50.232 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:59:50.233 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:50.233 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:59:50.234 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:59:50.234 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:59:50.234 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:59:50.235 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:59:50.235 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 12, 2021-11-21T21:59:50.236 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:59:50.236 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:59:50.236 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:59:50.237 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:59:50.237 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:59:50.237 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.238 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:59:50.238 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:59:50.239 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:59:50.239 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:50.239 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:50.240 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:50.240 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:50.241 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:50.241 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:50.242 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:50.242 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:50.242 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:59:50.243 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:59:50.243 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:59:50.244 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:59:50.244 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:59:50.245 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:50.245 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:50.246 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:50.246 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:50.246 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:50.247 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:50.247 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:50.247 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.248 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:59:50.248 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:50.248 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:50.249 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:50.249 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:50.249 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:50.250 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:50.250 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:50.251 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:50.251 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:50.252 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:50.252 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:50.252 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:50.253 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:50.253 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:50.254 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:50.254 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:50.255 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:50.255 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:50.256 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:50.256 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:50.257 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:50.257 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:50.257 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:50.257 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:50.258 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:50.258 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:50.258 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.258 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:50.259 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:50.259 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:50.259 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:50.259 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:50.259 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:50.260 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:50.260 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:50.260 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:50.260 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:50.261 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:50.261 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:59:50.261 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:59:50.261 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.261 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:50.262 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:59:50.262 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.262 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.262 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.263 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:59:50.263 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:59:50.263 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:59:50.263 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:50.263 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:50.264 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:50.264 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:50.264 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:50.264 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:50.265 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:50.265 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:50.265 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:50.266 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:50.266 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:50.266 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:50.267 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:50.267 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:50.267 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:50.268 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:50.268 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:50.269 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:50.269 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:50.270 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:50.270 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.271 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:59:50.271 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:50.271 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:50.272 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:50.272 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:50.273 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:50.273 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:50.274 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:50.274 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:50.275 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:50.275 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:50.275 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:50.276 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:50.276 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:50.277 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:50.277 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:50.277 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:50.277 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:50.278 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:50.278 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:50.278 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:50.279 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:50.279 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:50.279 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:50.280 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:50.280 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:50.280 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:50.280 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.280 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:50.281 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:50.281 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:50.281 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:50.281 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:50.282 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:50.282 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:50.282 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:50.282 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:50.283 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:50.283 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:50.283 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:59:50.283 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:50.284 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:59:50.284 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.284 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.285 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.285 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 11, 2021-11-21T21:59:50.285 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T21:59:50.285 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:59:17.175021+0000", 2021-11-21T21:59:50.286 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:50.286 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:50.286 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:50.286 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:50.287 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:50.287 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:50.287 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:50.287 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:50.287 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:50.288 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:50.288 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:50.288 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:50.288 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:50.289 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:50.289 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:50.289 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:50.289 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:50.289 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:50.290 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:50.290 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:50.290 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.290 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "65", 2021-11-21T21:59:50.291 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:50.291 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:50.291 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:50.291 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:50.291 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:50.292 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:50.292 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:50.292 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:50.292 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:50.292 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:50.293 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:50.293 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:50.293 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:50.293 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:50.294 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:50.294 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:50.294 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:50.294 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:50.295 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:50.295 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:50.295 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:50.295 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:50.296 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:50.296 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:50.296 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:50.296 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:50.297 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.297 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:50.297 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:50.297 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:50.297 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:50.298 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:50.298 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:50.299 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:50.299 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:50.299 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:50.299 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:50.300 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:50.300 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:59:50.300 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T21:59:50.300 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T21:59:50.300 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T21:59:50.301 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.301 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:50.301 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:59:50.301 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T21:59:50.302 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.302 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.302 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.302 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.302 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 12, 2021-11-21T21:59:50.303 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T21:59:50.303 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:59:18.135656+0000", 2021-11-21T21:59:50.303 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:50.303 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:50.304 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:50.304 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:50.304 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:50.304 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:50.304 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:50.305 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:50.305 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:50.305 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:50.305 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:50.306 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:50.306 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:50.306 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:50.306 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:50.306 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:50.307 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:50.307 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:50.307 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:50.307 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:50.307 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.308 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "66", 2021-11-21T21:59:50.308 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:50.308 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:50.308 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:50.309 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:50.309 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:50.309 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:50.309 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:50.309 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:50.310 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:50.310 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:50.310 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:50.310 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:50.311 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:50.311 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:50.311 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:50.311 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:50.311 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:50.312 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:50.312 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:50.312 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:50.312 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:50.312 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:50.313 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:50.313 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:50.313 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:50.313 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:50.314 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.314 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:50.314 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:50.314 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:50.314 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:50.315 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:50.315 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:50.315 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:50.315 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:50.316 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:50.316 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:50.316 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:50.316 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:59:50.316 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:50.317 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T21:59:50.317 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T21:59:50.317 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.317 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.317 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.318 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:50.318 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:59:50.318 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.318 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:50.319 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:59:50.319 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.319 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.319 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.319 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.320 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.320 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.320 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.320 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:50.321 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.321 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.321 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.321 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.321 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.322 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.322 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:59:50.322 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.322 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.322 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.323 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.323 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:59:50.323 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.323 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.324 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.324 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.324 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.324 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.324 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.325 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.325 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:59:50.325 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.325 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.326 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.326 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.326 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:59:50.326 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.326 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.327 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.327 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.327 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.327 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.327 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.331 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.331 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:59:50.332 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.332 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.333 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.333 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.333 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:59:50.334 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.334 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.335 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.335 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.336 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.336 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.336 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.337 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.337 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:59:50.338 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.338 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.339 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.339 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.339 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:59:50.340 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:50.340 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.341 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.341 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.342 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:59:50.342 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:59:50.342 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:59:50.343 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:59:50.343 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.344 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.344 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.345 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.345 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.345 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.346 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:50.346 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:59:50.347 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.347 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.347 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.348 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.348 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.348 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.349 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.349 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:50.349 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.350 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.350 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.351 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.351 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.351 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.352 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:59:50.352 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.352 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.352 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.352 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.353 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:59:50.353 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.353 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.353 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.354 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.354 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.354 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.354 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.354 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.355 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:59:50.355 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.355 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.355 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.356 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.356 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:59:50.356 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.356 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.356 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.357 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.357 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.357 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.357 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.358 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.358 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:59:50.358 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.358 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.358 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.359 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.359 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:59:50.359 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.359 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.360 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.360 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.360 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.360 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.360 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.361 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.361 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:59:50.361 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.361 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.361 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.362 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.362 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:59:50.362 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:50.362 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.363 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.363 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.363 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:59:50.363 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:59:50.363 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:59:50.364 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:59:50.364 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.364 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.364 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.364 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.365 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.365 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.365 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:50.365 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:59:50.366 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.366 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.366 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.366 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.367 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.367 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.367 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.367 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 63, 2021-11-21T21:59:50.367 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.368 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.368 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.368 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.368 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.368 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.369 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:59:50.369 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.369 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.369 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.370 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.370 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:59:50.370 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.370 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.370 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.371 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.371 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.371 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.371 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.372 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.372 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:59:50.372 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.372 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.372 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.373 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.373 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:59:50.373 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.373 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.373 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.374 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.374 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.374 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.374 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.375 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.375 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:59:50.375 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.375 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.376 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.376 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.376 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:59:50.376 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.376 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.377 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.377 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.377 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.377 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.378 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.378 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.378 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:59:50.378 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.378 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.379 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.379 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.379 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:59:50.379 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:50.380 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.380 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.380 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.380 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:59:50.380 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:59:50.381 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:59:50.381 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:59:50.381 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.381 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.382 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.382 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.382 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.382 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.382 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:50.383 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:59:50.383 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.383 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.383 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.383 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.384 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.384 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.384 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.384 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 52, 2021-11-21T21:59:50.385 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.385 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.385 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.385 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.386 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.386 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.386 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:59:50.386 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.387 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.387 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.387 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.387 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:59:50.388 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.388 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.388 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.389 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.389 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.389 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.390 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.390 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.390 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:59:50.390 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.390 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.391 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.391 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.391 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:59:50.392 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.392 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.392 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.392 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.393 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.393 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.393 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.393 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.394 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:59:50.394 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.394 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.394 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.394 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.395 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:59:50.395 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.395 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.395 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.396 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.396 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.396 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.397 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.397 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.397 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:59:50.397 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.398 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.398 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.398 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.398 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:59:50.398 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:50.399 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.399 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.399 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.399 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:59:50.400 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:59:50.400 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:59:50.400 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:59:50.400 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.400 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.401 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.401 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.401 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.401 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.401 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:50.402 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:59:50.402 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.402 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.402 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.403 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.403 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.403 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.403 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.403 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:50.404 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.404 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.404 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.404 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.405 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.405 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.405 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:59:50.405 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.405 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.406 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.406 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.406 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:59:50.406 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.406 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.407 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.407 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.407 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.407 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.408 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.408 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.408 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:59:50.408 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.408 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.409 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.409 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.409 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:59:50.409 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.410 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.410 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.410 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.410 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.411 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.411 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.411 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.411 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:59:50.411 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.412 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.412 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.412 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.412 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:59:50.413 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.413 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.413 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.413 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.414 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.414 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.414 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.414 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.415 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:59:50.415 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.415 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.415 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.416 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.416 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:59:50.416 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:50.416 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.417 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.417 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.417 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:59:50.417 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:59:50.417 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:59:50.418 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:59:50.418 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.418 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.418 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.419 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.419 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.419 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:50.420 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:59:50.420 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.420 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.420 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.420 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.421 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.421 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.421 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.421 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 63, 2021-11-21T21:59:50.422 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.422 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.422 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.423 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.423 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.423 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.423 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:59:50.423 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.424 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.424 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.424 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.424 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:59:50.424 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.425 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.425 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.425 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.425 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.426 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.426 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.426 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.426 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:59:50.426 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.427 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.427 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.427 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:59:50.427 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.428 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.428 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.428 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.428 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.429 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.429 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.429 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.429 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:59:50.430 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.430 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.430 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.430 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.430 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:59:50.431 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.431 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.431 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.431 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.432 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.432 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.432 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:59:50.433 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.433 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.433 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.433 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:59:50.434 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:50.434 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.434 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.435 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.435 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:59:50.435 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:59:50.435 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:59:50.435 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:59:50.436 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.436 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.436 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.436 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.436 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.437 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.437 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:50.437 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:59:50.437 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.438 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.438 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.438 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.438 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.438 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.439 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.439 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:50.439 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.439 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.440 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.440 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.440 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.440 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.440 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:59:50.441 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.441 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.441 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.441 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.442 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:59:50.442 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.442 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.442 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.442 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.443 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.443 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.443 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.443 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.444 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:59:50.444 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.444 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.444 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.444 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.445 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:59:50.445 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.445 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.445 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.445 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.446 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.446 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.446 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.446 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.446 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:59:50.447 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.447 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.447 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.447 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.448 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:59:50.448 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.448 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.448 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.448 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.449 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.449 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.449 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.450 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:59:50.450 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.450 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.450 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.450 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.451 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:59:50.451 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:50.451 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.451 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.452 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.452 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:59:50.452 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:59:50.452 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:59:50.452 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:59:50.453 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.453 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.453 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.453 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.453 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.454 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.454 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:50.454 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:59:50.454 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:50.455 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:50.455 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:50.455 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:50.456 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:50.456 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:50.456 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:50.457 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 63, 2021-11-21T21:59:50.457 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:50.457 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:50.457 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:50.457 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.458 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.458 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.458 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:59:50.458 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.459 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.459 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.459 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.459 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:59:50.459 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.460 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.460 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.460 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.460 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:50.460 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.461 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.461 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.461 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:59:50.461 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.462 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.462 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.462 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:59:50.463 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.463 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.463 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.463 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.463 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:50.464 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.464 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.464 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.464 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:59:50.464 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.465 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.465 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.465 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.465 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:59:50.466 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.466 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.466 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.466 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.466 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:50.467 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:50.467 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.467 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:50.467 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:59:50.468 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.468 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.468 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.468 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:50.468 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:59:50.469 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:50.469 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.469 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.469 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.469 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:59:50.470 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:59:50.470 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:59:50.470 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:59:50.470 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:50.471 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:50.471 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:50.471 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:50.471 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.471 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:50.472 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:59:50.472 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.472 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:50.472 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.472 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.473 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.473 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.473 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.473 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:59:50.474 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.474 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.474 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:50.474 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.475 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.475 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.475 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.475 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.476 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:59:50.476 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.476 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.476 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:50.477 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.477 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.477 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.477 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.478 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.478 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:59:50.478 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.478 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.479 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:50.479 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.479 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.479 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.480 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.480 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.480 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:59:50.480 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.481 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.481 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.481 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:50.481 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.482 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.482 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.482 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.482 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.483 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:59:50.483 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.483 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.483 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.483 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:50.484 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.484 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.484 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.484 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.485 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.485 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:59:50.485 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.485 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.485 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.486 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:50.486 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.486 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.486 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.487 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.487 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.487 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:59:50.487 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.487 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.488 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:50.488 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:50.488 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:50.488 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:50.489 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:50.489 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:50.489 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:50.489 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:59:50.489 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:50.490 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.490 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:50.490 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:59:50.490 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:59:50.490 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:59:50.491 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:59:50.491 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T21:59:50.491 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:59:50.491 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:59:50.492 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:59:50.492 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:59:50.492 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:59:50.492 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:59:50.493 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:59:50.493 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:59:50.493 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:50.493 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:50.493 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:59:50.494 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:59:50.494 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:59:50.494 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:59:50.494 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:59:50.495 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:59:50.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf580536a0 msgr2=0x7faf58055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf580536a0 0x7faf58055b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c015000).stop 2021-11-21T21:59:50.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780ac750 msgr2=0x7faf78152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780ac750 0x7faf78152120 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7faf74002ef0 tx=0x7faf74006d40).stop 2021-11-21T21:59:50.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 shutdown_connections 2021-11-21T21:59:50.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf780ad340 0x7faf78152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf580536a0 0x7faf58055b60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:50.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780ac750 0x7faf78152120 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:50.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 --2- 172.21.2.116:0/3053836919 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780a8790 0x7faf78144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 >> 172.21.2.116:0/3053836919 conn(0x7faf7805bd20 msgr2=0x7faf7805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:50.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 shutdown_connections 2021-11-21T21:59:50.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.222+0000 7faf7f675700 1 -- 172.21.2.116:0/3053836919 wait complete. 2021-11-21T21:59:50.500 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:50.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b5576700 1 Processor -- start 2021-11-21T21:59:50.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b5576700 1 -- start start 2021-11-21T21:59:50.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b5576700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb1b0099d60 0x7fb1b014af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b5576700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b014a560 con 0x7fb1b00b5630 2021-11-21T21:59:50.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b5576700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b014a800 con 0x7fb1b0097ec0 2021-11-21T21:59:50.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b5576700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b0144b40 con 0x7fb1b0099d60 2021-11-21T21:59:50.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b4d75700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb1b0099d60 0x7fb1b014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b4d75700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb1b0099d60 0x7fb1b014af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48650/0 (socket says 172.21.2.116:48650) 2021-11-21T21:59:50.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b4d75700 1 -- 172.21.2.116:0/1020268519 learned_addr learned my addr 172.21.2.116:0/1020268519 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:50.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b4d75700 1 -- 172.21.2.116:0/1020268519 >> v1:172.21.2.117:6789/0 conn(0x7fb1b0097ec0 legacy=0x7fb1b0099650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:50.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b4d75700 1 -- 172.21.2.116:0/1020268519 >> v1:172.21.2.116:6789/0 conn(0x7fb1b00b5630 legacy=0x7fb1b0146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1b4d75700 1 -- 172.21.2.116:0/1020268519 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b014aaa0 con 0x7fb1b0099d60 2021-11-21T21:59:50.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.550+0000 7fb1aeffd700 1 -- 172.21.2.116:0/1020268519 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715771974 0 0) 0x7fb1b014a560 con 0x7fb1b00b5630 2021-11-21T21:59:50.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 --2- 172.21.2.116:0/1020268519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b0099d60 0x7fb1b014af40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fb1b0062330 tx=0x7fb1a8009290).ready entity=mon.0 client_cookie=eb5e6413d682ed59 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1aeffd700 1 -- 172.21.2.116:0/1020268519 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1a800b830 con 0x7fb1b0099d60 2021-11-21T21:59:50.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1aeffd700 1 -- 172.21.2.116:0/1020268519 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a800b990 con 0x7fb1b0099d60 2021-11-21T21:59:50.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1aeffd700 1 -- 172.21.2.116:0/1020268519 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1a800bb00 con 0x7fb1b0099d60 2021-11-21T21:59:50.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/1020268519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b0099d60 msgr2=0x7fb1b014af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 --2- 172.21.2.116:0/1020268519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b0099d60 0x7fb1b014af40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fb1b0062330 tx=0x7fb1a8009290).stop 2021-11-21T21:59:50.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/1020268519 shutdown_connections 2021-11-21T21:59:50.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 --2- 172.21.2.116:0/1020268519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b0099d60 0x7fb1b014af40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:50.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/1020268519 >> 172.21.2.116:0/1020268519 conn(0x7fb1b005c7f0 msgr2=0x7fb1b00b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:50.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/1020268519 shutdown_connections 2021-11-21T21:59:50.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/1020268519 wait complete. 2021-11-21T21:59:50.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 Processor -- start 2021-11-21T21:59:50.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- start start 2021-11-21T21:59:50.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1b0097ec0 0x7fb1b0156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1b0099d60 0x7fb1b0157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b00b5630 0x7fb1b0145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0065e60 con 0x7fb1b00b5630 2021-11-21T21:59:50.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0156290 con 0x7fb1b0099d60 2021-11-21T21:59:50.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b00adfd0 con 0x7fb1b0097ec0 2021-11-21T21:59:50.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b00b5630 0x7fb1b0145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b00b5630 0x7fb1b0145c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48656/0 (socket says 172.21.2.116:48656) 2021-11-21T21:59:50.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 -- 172.21.2.116:0/957713015 learned_addr learned my addr 172.21.2.116:0/957713015 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:50.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 -- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1b0097ec0 msgr2=0x7fb1b0156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1b0097ec0 0x7fb1b0156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 -- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1b0099d60 msgr2=0x7fb1b0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1b0099d60 0x7fb1b0157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 -- 172.21.2.116:0/957713015 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009530 con 0x7fb1b00b5630 2021-11-21T21:59:50.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b4d75700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b00b5630 0x7fb1b0145c40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fb1b0062330 tx=0x7fb1a8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1ad7fa700 1 -- 172.21.2.116:0/957713015 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1a801f3f0 con 0x7fb1b00b5630 2021-11-21T21:59:50.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1ad7fa700 1 -- 172.21.2.116:0/957713015 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a801f550 con 0x7fb1b00b5630 2021-11-21T21:59:50.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1ad7fa700 1 -- 172.21.2.116:0/957713015 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1a801f8b0 con 0x7fb1b00b5630 2021-11-21T21:59:50.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b01564c0 con 0x7fb1b00b5630 2021-11-21T21:59:50.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b0157170 con 0x7fb1b00b5630 2021-11-21T21:59:50.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b00573d0 con 0x7fb1b00b5630 2021-11-21T21:59:50.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.554+0000 7fb1ad7fa700 1 -- 172.21.2.116:0/957713015 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb1a8026030 con 0x7fb1b00b5630 2021-11-21T21:59:50.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.558+0000 7fb1ad7fa700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb194053690 0x7fb194055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.558+0000 7fb1ad7fa700 1 -- 172.21.2.116:0/957713015 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb1a806c740 con 0x7fb1b00b5630 2021-11-21T21:59:50.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.558+0000 7fb1affff700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb194053690 0x7fb194055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.558+0000 7fb1affff700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb194053690 0x7fb194055b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fb1a0000bc0 tx=0x7fb1a00077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.558+0000 7fb1ad7fa700 1 -- 172.21.2.116:0/957713015 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1a8043790 con 0x7fb1b00b5630 2021-11-21T21:59:50.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1b0147440 con 0x7fb1b00b5630 2021-11-21T21:59:50.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1ad7fa700 1 -- 172.21.2.116:0/957713015 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fb1a80249d0 con 0x7fb1b00b5630 2021-11-21T21:59:50.669 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:50.669 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:20.167894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3617788104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3617788104},{"type":"v1","addr":"172.21.2.116:6835","nonce":3617788104}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:50.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb194053690 msgr2=0x7fb194055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb194053690 0x7fb194055b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fb1a0000bc0 tx=0x7fb1a00077f0).stop 2021-11-21T21:59:50.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b00b5630 msgr2=0x7fb1b0145c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b00b5630 0x7fb1b0145c40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fb1b0062330 tx=0x7fb1a8016000).stop 2021-11-21T21:59:50.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 shutdown_connections 2021-11-21T21:59:50.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb194053690 0x7fb194055b50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:50.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1b00b5630 0x7fb1b0145c40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:50.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1b0099d60 0x7fb1b0157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 --2- 172.21.2.116:0/957713015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1b0097ec0 0x7fb1b0156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 >> 172.21.2.116:0/957713015 conn(0x7fb1b005c7f0 msgr2=0x7fb1b0145090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:50.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 shutdown_connections 2021-11-21T21:59:50.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.666+0000 7fb1b5576700 1 -- 172.21.2.116:0/957713015 wait complete. 2021-11-21T21:59:50.673 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 28 2021-11-21T21:59:50.675 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T21:59:50.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f03282d4700 1 Processor -- start 2021-11-21T21:59:50.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f03282d4700 1 -- start start 2021-11-21T21:59:50.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f03282d4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f03201426e0 0x7f0320142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f03282d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320062e60 0x7f03200af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f03282d4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03200b0f60 con 0x7f0320069e70 2021-11-21T21:59:50.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f03282d4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0320063470 con 0x7f03201426e0 2021-11-21T21:59:50.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f03282d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0320143050 con 0x7f0320062e60 2021-11-21T21:59:50.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f03201426e0 0x7f0320142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f03201426e0 0x7f0320142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48660/0 (socket says 172.21.2.116:48660) 2021-11-21T21:59:50.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 -- 172.21.2.116:0/1677559090 learned_addr learned my addr 172.21.2.116:0/1677559090 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:50.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 -- 172.21.2.116:0/1677559090 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320062e60 msgr2=0x7f03200af3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:50.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 --2- 172.21.2.116:0/1677559090 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320062e60 0x7f03200af3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 -- 172.21.2.116:0/1677559090 >> v1:172.21.2.116:6789/0 conn(0x7f0320069e70 legacy=0x7f03200ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 -- 172.21.2.116:0/1677559090 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200b1e00 con 0x7f03201426e0 2021-11-21T21:59:50.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f0326070700 1 --2- 172.21.2.116:0/1677559090 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320142ac0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f031c002f70 tx=0x7f031c006eb0).ready entity=mon.0 client_cookie=c9e7f18a0da8cf7c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f032506e700 1 -- 172.21.2.116:0/1677559090 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f031c00d5c0 con 0x7f03201426e0 2021-11-21T21:59:50.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f032506e700 1 -- 172.21.2.116:0/1677559090 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c000e90 con 0x7f03201426e0 2021-11-21T21:59:50.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.722+0000 7f032506e700 1 -- 172.21.2.116:0/1677559090 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f031c00d950 con 0x7f03201426e0 2021-11-21T21:59:50.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/1677559090 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 msgr2=0x7f0320142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:50.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 --2- 172.21.2.116:0/1677559090 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320142ac0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f031c002f70 tx=0x7f031c006eb0).stop 2021-11-21T21:59:50.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/1677559090 shutdown_connections 2021-11-21T21:59:50.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 --2- 172.21.2.116:0/1677559090 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320062e60 0x7f03200af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 --2- 172.21.2.116:0/1677559090 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320142ac0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:50.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/1677559090 >> 172.21.2.116:0/1677559090 conn(0x7f03200b5730 msgr2=0x7f03200b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:50.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/1677559090 shutdown_connections 2021-11-21T21:59:50.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/1677559090 wait complete. 2021-11-21T21:59:50.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 Processor -- start 2021-11-21T21:59:50.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- start start 2021-11-21T21:59:50.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320069e70 0x7f0320063960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320120280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f03201420c0 0x7f0320140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032013ef70 con 0x7f03201426e0 2021-11-21T21:59:50.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032013fab0 con 0x7f03201420c0 2021-11-21T21:59:50.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032013d610 con 0x7f0320069e70 2021-11-21T21:59:50.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320120280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320120280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48666/0 (socket says 172.21.2.116:48666) 2021-11-21T21:59:50.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 -- 172.21.2.116:0/3079149631 learned_addr learned my addr 172.21.2.116:0/3079149631 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:50.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 -- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320069e70 msgr2=0x7f0320063960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:50.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320069e70 0x7f0320063960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 -- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f03201420c0 msgr2=0x7f0320140b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:50.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f03201420c0 0x7f0320140b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:50.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 -- 172.21.2.116:0/3079149631 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c00b1e0 con 0x7f03201426e0 2021-11-21T21:59:50.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f032586f700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320120280 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f0314002fb0 tx=0x7f0314006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03137fe700 1 -- 172.21.2.116:0/3079149631 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0314013460 con 0x7f03201426e0 2021-11-21T21:59:50.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03137fe700 1 -- 172.21.2.116:0/3079149631 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140135c0 con 0x7f03201426e0 2021-11-21T21:59:50.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032013d250 con 0x7f03201426e0 2021-11-21T21:59:50.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03137fe700 1 -- 172.21.2.116:0/3079149631 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0314013920 con 0x7f03201426e0 2021-11-21T21:59:50.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201208b0 con 0x7f03201426e0 2021-11-21T21:59:50.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03200573d0 con 0x7f03201426e0 2021-11-21T21:59:50.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03137fe700 1 -- 172.21.2.116:0/3079149631 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f031400f070 con 0x7f03201426e0 2021-11-21T21:59:50.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03137fe700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03000536a0 0x7f0300055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:50.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.726+0000 7f03137fe700 1 -- 172.21.2.116:0/3079149631 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f031406b0b0 con 0x7f03201426e0 2021-11-21T21:59:50.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.730+0000 7f0326070700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03000536a0 0x7f0300055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:50.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.730+0000 7f0326070700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03000536a0 0x7f0300055b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f031c002f70 tx=0x7f031c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:50.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.730+0000 7f03137fe700 1 -- 172.21.2.116:0/3079149631 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0314042210 con 0x7f03201426e0 2021-11-21T21:59:50.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:50.838+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f032005f2d0 con 0x7f03201426e0 2021-11-21T21:59:51.136 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T21:59:51.134+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T21:59:51.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.142+0000 7f03137fe700 1 -- 172.21.2.116:0/3079149631 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0314018030 con 0x7f03201426e0 2021-11-21T21:59:51.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03000536a0 msgr2=0x7f0300055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03000536a0 0x7f0300055b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f031c002f70 tx=0x7f031c015000).stop 2021-11-21T21:59:51.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 msgr2=0x7f0320120280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320120280 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f0314002fb0 tx=0x7f0314006df0).stop 2021-11-21T21:59:51.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 shutdown_connections 2021-11-21T21:59:51.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f03201420c0 0x7f0320140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03000536a0 0x7f0300055b60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f03201426e0 0x7f0320120280 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 --2- 172.21.2.116:0/3079149631 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0320069e70 0x7f0320063960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 >> 172.21.2.116:0/3079149631 conn(0x7f03200b5730 msgr2=0x7f03200ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:51.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 shutdown_connections 2021-11-21T21:59:51.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.146+0000 7f03282d4700 1 -- 172.21.2.116:0/3079149631 wait complete. 2021-11-21T21:59:51.151 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T21:59:51.157 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:51.169 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:59:51.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 Processor -- start 2021-11-21T21:59:51.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- start start 2021-11-21T21:59:51.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f044c0a6aa0 0x7f044c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c0a86b0 0x7f044c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f044c1475d0 0x7f044c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0a8bc0 con 0x7f044c1475d0 2021-11-21T21:59:51.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0a6e80 con 0x7f044c0a6aa0 2021-11-21T21:59:51.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c14a7b0 con 0x7f044c0a86b0 2021-11-21T21:59:51.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f044c1475d0 0x7f044c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f044c1475d0 0x7f044c1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48686/0 (socket says 172.21.2.116:48686) 2021-11-21T21:59:51.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 -- 172.21.2.116:0/1919931264 learned_addr learned my addr 172.21.2.116:0/1919931264 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:51.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 -- 172.21.2.116:0/1919931264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c0a86b0 msgr2=0x7f044c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 --2- 172.21.2.116:0/1919931264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c0a86b0 0x7f044c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 -- 172.21.2.116:0/1919931264 >> v2:172.21.2.117:3300/0 conn(0x7f044c0a6aa0 msgr2=0x7f044c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 --2- 172.21.2.116:0/1919931264 >> v2:172.21.2.117:3300/0 conn(0x7f044c0a6aa0 0x7f044c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 -- 172.21.2.116:0/1919931264 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c14aa10 con 0x7f044c1475d0 2021-11-21T21:59:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f0451da8700 1 --2- 172.21.2.116:0/1919931264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c1475d0 0x7f044c1443f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f04440032f0 tx=0x7f0444009290).ready entity=mon.0 client_cookie=ef9586b8c52d645e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:51.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f043bfff700 1 -- 172.21.2.116:0/1919931264 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044400b870 con 0x7f044c1475d0 2021-11-21T21:59:51.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f043bfff700 1 -- 172.21.2.116:0/1919931264 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044400b9d0 con 0x7f044c1475d0 2021-11-21T21:59:51.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f043bfff700 1 -- 172.21.2.116:0/1919931264 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044400bb40 con 0x7f044c1475d0 2021-11-21T21:59:51.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- 172.21.2.116:0/1919931264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c1475d0 msgr2=0x7f044c1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 --2- 172.21.2.116:0/1919931264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c1475d0 0x7f044c1443f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f04440032f0 tx=0x7f0444009290).stop 2021-11-21T21:59:51.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- 172.21.2.116:0/1919931264 shutdown_connections 2021-11-21T21:59:51.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 --2- 172.21.2.116:0/1919931264 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c1475d0 0x7f044c1443f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 --2- 172.21.2.116:0/1919931264 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c0a86b0 0x7f044c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 --2- 172.21.2.116:0/1919931264 >> v2:172.21.2.117:3300/0 conn(0x7f044c0a6aa0 0x7f044c14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- 172.21.2.116:0/1919931264 >> 172.21.2.116:0/1919931264 conn(0x7f044c05bd20 msgr2=0x7f044c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:51.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- 172.21.2.116:0/1919931264 shutdown_connections 2021-11-21T21:59:51.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.206+0000 7f045380b700 1 -- 172.21.2.116:0/1919931264 wait complete. 2021-11-21T21:59:51.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 Processor -- start 2021-11-21T21:59:51.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 -- start start 2021-11-21T21:59:51.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c0a6aa0 0x7f044c142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f044c0a86b0 0x7f044c13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c1475d0 0x7f044c13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c146840 con 0x7f044c0a6aa0 2021-11-21T21:59:51.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c05adb0 con 0x7f044c0a86b0 2021-11-21T21:59:51.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c05d3b0 con 0x7f044c1475d0 2021-11-21T21:59:51.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c0a6aa0 0x7f044c142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c0a6aa0 0x7f044c142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48690/0 (socket says 172.21.2.116:48690) 2021-11-21T21:59:51.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 -- 172.21.2.116:0/129324545 learned_addr learned my addr 172.21.2.116:0/129324545 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:51.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 -- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c1475d0 msgr2=0x7f044c13f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c1475d0 0x7f044c13f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 -- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f044c0a86b0 msgr2=0x7f044c13ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f044c0a86b0 0x7f044c13ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 -- 172.21.2.116:0/129324545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444009530 con 0x7f044c0a6aa0 2021-11-21T21:59:51.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f04515a7700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c0a6aa0 0x7f044c142580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f044c146980 tx=0x7f0448006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:51.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f043a7fc700 1 -- 172.21.2.116:0/129324545 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0448013460 con 0x7f044c0a6aa0 2021-11-21T21:59:51.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f043a7fc700 1 -- 172.21.2.116:0/129324545 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04480135c0 con 0x7f044c0a6aa0 2021-11-21T21:59:51.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c1471a0 con 0x7f044c0a6aa0 2021-11-21T21:59:51.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c149780 con 0x7f044c0a6aa0 2021-11-21T21:59:51.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f043a7fc700 1 -- 172.21.2.116:0/129324545 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0448013920 con 0x7f044c0a6aa0 2021-11-21T21:59:51.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044c0573d0 con 0x7f044c0a6aa0 2021-11-21T21:59:51.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f043a7fc700 1 -- 172.21.2.116:0/129324545 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f044801a060 con 0x7f044c0a6aa0 2021-11-21T21:59:51.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f043a7fc700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f042c053660 0x7f042c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f043a7fc700 1 -- 172.21.2.116:0/129324545 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f044806b4f0 con 0x7f044c0a6aa0 2021-11-21T21:59:51.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f0450da6700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f042c053660 0x7f042c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.210+0000 7f0450da6700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f042c053660 0x7f042c055b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f043c002f70 tx=0x7f043c006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:51.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.214+0000 7f043a7fc700 1 -- 172.21.2.116:0/129324545 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04480424f0 con 0x7f044c0a6aa0 2021-11-21T21:59:51.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f044c147330 con 0x7f044c0a6aa0 2021-11-21T21:59:51.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f043a7fc700 1 -- 172.21.2.116:0/129324545 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f0448018020 con 0x7f044c0a6aa0 2021-11-21T21:59:51.331 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:51.331 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":29,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T21:59:19.164615+0000","modified":"2021-11-21T21:59:51.137673+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-11-21T21:59:51.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f042c053660 msgr2=0x7f042c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f042c053660 0x7f042c055b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f043c002f70 tx=0x7f043c006a00).stop 2021-11-21T21:59:51.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c0a6aa0 msgr2=0x7f044c142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c0a6aa0 0x7f044c142580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f044c146980 tx=0x7f0448006ec0).stop 2021-11-21T21:59:51.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 shutdown_connections 2021-11-21T21:59:51.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f044c1475d0 0x7f044c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f044c0a86b0 0x7f044c13ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f042c053660 0x7f042c055b20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 --2- 172.21.2.116:0/129324545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f044c0a6aa0 0x7f044c142580 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.326+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 >> 172.21.2.116:0/129324545 conn(0x7f044c05bd20 msgr2=0x7f044c0b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:51.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.330+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 shutdown_connections 2021-11-21T21:59:51.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.330+0000 7f045380b700 1 -- 172.21.2.116:0/129324545 wait complete. 2021-11-21T21:59:51.336 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 29 2021-11-21T21:59:51.338 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T21:59:51.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80cc8b700 1 Processor -- start 2021-11-21T21:59:51.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80cc8b700 1 -- start start 2021-11-21T21:59:51.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80cc8b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8080abce0 0x7fa8081478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80cc8b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa8080ac600 0x7fa8080acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80cc8b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa808147e80 con 0x7fa808144810 2021-11-21T21:59:51.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80cc8b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8080b1490 con 0x7fa8080ac600 2021-11-21T21:59:51.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80cc8b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080b1fd0 con 0x7fa8080abce0 2021-11-21T21:59:51.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80659c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa808144810 0x7fa8080b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60654/0 (socket says 172.21.2.116:60654) 2021-11-21T21:59:51.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80659c700 1 -- 172.21.2.116:0/4089249442 learned_addr learned my addr 172.21.2.116:0/4089249442 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:51.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828013122 0 0) 0x7fa808147e80 con 0x7fa808144810 2021-11-21T21:59:51.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec003260 con 0x7fa808144810 2021-11-21T21:59:51.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa805d9b700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8080abce0 0x7fa8081478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 --2- 172.21.2.116:0/4089249442 >> v2:172.21.2.117:3300/0 conn(0x7fa8080ac600 0x7fa8080acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1490024695 0 0) 0x7fa7ec003260 con 0x7fa808144810 2021-11-21T21:59:51.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8080abce0 msgr2=0x7fa8081478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:51.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8080abce0 0x7fa8081478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 >> v2:172.21.2.117:3300/0 conn(0x7fa8080ac600 msgr2=0x7fa8080acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:51.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 --2- 172.21.2.116:0/4089249442 >> v2:172.21.2.117:3300/0 conn(0x7fa8080ac600 0x7fa8080acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080af5a0 con 0x7fa808144810 2021-11-21T21:59:51.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fa7f8001660 con 0x7fa808144810 2021-11-21T21:59:51.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 >> v1:172.21.2.116:6789/0 conn(0x7fa808144810 legacy=0x7fa8080b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0037d0 0x7fa7ec003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8080abce0 0x7fa7ec004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7ec0049f0 0x7fa7ec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080b1490 con 0x7fa7ec0049f0 2021-11-21T21:59:51.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080b1fd0 con 0x7fa7ec0037d0 2021-11-21T21:59:51.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00b8b0 con 0x7fa8080abce0 2021-11-21T21:59:51.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7ec0049f0 0x7fa7ec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 -- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8080abce0 msgr2=0x7fa7ec004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:51.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8080abce0 0x7fa7ec004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 -- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0037d0 msgr2=0x7fa7ec003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:51.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0037d0 0x7fa7ec003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:51.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 -- 172.21.2.116:0/4089249442 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec003260 con 0x7fa7ec0049f0 2021-11-21T21:59:51.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa806d9d700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7ec0049f0 0x7fa7ec004e70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fa80000a000 tx=0x7fa800008040).ready entity=mon.0 client_cookie=ab87bccc8adbf5c0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:51.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80000c070 con 0x7fa7ec0049f0 2021-11-21T21:59:51.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80000e5f0 con 0x7fa7ec0049f0 2021-11-21T21:59:51.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.386+0000 7fa80559a700 1 -- 172.21.2.116:0/4089249442 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80000e950 con 0x7fa7ec0049f0 2021-11-21T21:59:51.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/4089249442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7ec0049f0 msgr2=0x7fa7ec004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 --2- 172.21.2.116:0/4089249442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7ec0049f0 0x7fa7ec004e70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fa80000a000 tx=0x7fa800008040).stop 2021-11-21T21:59:51.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa806d9d700 1 -- 172.21.2.116:0/4089249442 reap_dead start 2021-11-21T21:59:51.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/4089249442 shutdown_connections 2021-11-21T21:59:51.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/4089249442 >> 172.21.2.116:0/4089249442 conn(0x7fa80805bd20 msgr2=0x7fa80805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:51.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/4089249442 shutdown_connections 2021-11-21T21:59:51.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/4089249442 wait complete. 2021-11-21T21:59:51.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 Processor -- start 2021-11-21T21:59:51.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- start start 2021-11-21T21:59:51.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0049f0 0x7fa80814a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa808143420 0x7fa80813c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa80813b100 0x7fa808156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa808144fb0 con 0x7fa808143420 2021-11-21T21:59:51.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa808143930 con 0x7fa7ec0049f0 2021-11-21T21:59:51.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80813b870 con 0x7fa80813b100 2021-11-21T21:59:51.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa808143420 0x7fa80813c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa808143420 0x7fa80813c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48714/0 (socket says 172.21.2.116:48714) 2021-11-21T21:59:51.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 -- 172.21.2.116:0/291732547 learned_addr learned my addr 172.21.2.116:0/291732547 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:51.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa806d9d700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa80813b100 0x7fa808156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 -- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa80813b100 msgr2=0x7fa808156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa80813b100 0x7fa808156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 -- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0049f0 msgr2=0x7fa80814a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:51.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80659c700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0049f0 0x7fa80814a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0049f0 0x7fa80814a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:51.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 -- 172.21.2.116:0/291732547 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80813bb10 con 0x7fa808143420 2021-11-21T21:59:51.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa806d9d700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa80813b100 0x7fa808156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:51.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa805d9b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa808143420 0x7fa80813c250 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fa7fc00a000 tx=0x7fa7fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:51.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa7f77fe700 1 -- 172.21.2.116:0/291732547 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7fc016070 con 0x7fa808143420 2021-11-21T21:59:51.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa7f77fe700 1 -- 172.21.2.116:0/291732547 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc000e90 con 0x7fa808143420 2021-11-21T21:59:51.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa7f77fe700 1 -- 172.21.2.116:0/291732547 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7fc011a40 con 0x7fa808143420 2021-11-21T21:59:51.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80813bdb0 con 0x7fa808143420 2021-11-21T21:59:51.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa808156b40 con 0x7fa808143420 2021-11-21T21:59:51.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80813bf40 con 0x7fa808143420 2021-11-21T21:59:51.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa7f77fe700 1 -- 172.21.2.116:0/291732547 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa7fc011ba0 con 0x7fa808143420 2021-11-21T21:59:51.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.390+0000 7fa7f77fe700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7e4053600 0x7fa7e4055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:51.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.394+0000 7fa7f77fe700 1 -- 172.21.2.116:0/291732547 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fa7fc06b980 con 0x7fa808143420 2021-11-21T21:59:51.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.394+0000 7fa80659c700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7e4053600 0x7fa7e4055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:51.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.394+0000 7fa80659c700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7e4053600 0x7fa7e4055ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa7f8008000 tx=0x7fa7f8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:51.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.394+0000 7fa7f77fe700 1 -- 172.21.2.116:0/291732547 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7fc042a30 con 0x7fa808143420 2021-11-21T21:59:51.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:51.502+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa8080af060 con 0x7fa808143420 2021-11-21T21:59:52.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa7f77fe700 1 -- 172.21.2.116:0/291732547 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa7fc046050 con 0x7fa808143420 2021-11-21T21:59:52.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7e4053600 msgr2=0x7fa7e4055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:52.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7e4053600 0x7fa7e4055ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa7f8008000 tx=0x7fa7f8009040).stop 2021-11-21T21:59:52.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa808143420 msgr2=0x7fa80813c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:52.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa808143420 0x7fa80813c250 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fa7fc00a000 tx=0x7fa7fc008040).stop 2021-11-21T21:59:52.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 shutdown_connections 2021-11-21T21:59:52.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa80813b100 0x7fa808156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:52.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7e4053600 0x7fa7e4055ac0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:52.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa808143420 0x7fa80813c250 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:52.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 --2- 172.21.2.116:0/291732547 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7ec0049f0 0x7fa80814a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:52.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 >> 172.21.2.116:0/291732547 conn(0x7fa80805bd20 msgr2=0x7fa8080af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:52.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 shutdown_connections 2021-11-21T21:59:52.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.162+0000 7fa80cc8b700 1 -- 172.21.2.116:0/291732547 wait complete. 2021-11-21T21:59:52.172 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T21:59:52.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 Processor -- start 2021-11-21T21:59:52.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 -- start start 2021-11-21T21:59:52.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100a8670 0x7f0610147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:52.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0610143e60 0x7f06101442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:52.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06100b1940 con 0x7f06100b11c0 2021-11-21T21:59:52.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100b15a0 con 0x7f0610143e60 2021-11-21T21:59:52.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100b1b70 con 0x7f06100a8670 2021-11-21T21:59:52.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0610143e60 0x7f06101442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:52.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0610143e60 0x7f06101442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48720/0 (socket says 172.21.2.116:48720) 2021-11-21T21:59:52.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 -- 172.21.2.116:0/2798017180 learned_addr learned my addr 172.21.2.116:0/2798017180 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:52.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 -- 172.21.2.116:0/2798017180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100a8670 msgr2=0x7f0610147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:52.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 --2- 172.21.2.116:0/2798017180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100a8670 0x7f0610147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:52.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 -- 172.21.2.116:0/2798017180 >> v1:172.21.2.117:6789/0 conn(0x7f06100b11c0 legacy=0x7f0610147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:52.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 -- 172.21.2.116:0/2798017180 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100b1e10 con 0x7f0610143e60 2021-11-21T21:59:52.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060ed9d700 1 --2- 172.21.2.116:0/2798017180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06101442f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f06100a8390 tx=0x7f0608009290).ready entity=mon.0 client_cookie=b0cd4409a52949ca server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:52.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060d59a700 1 -- 172.21.2.116:0/2798017180 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f060800b860 con 0x7f0610143e60 2021-11-21T21:59:52.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060d59a700 1 -- 172.21.2.116:0/2798017180 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060800b9c0 con 0x7f0610143e60 2021-11-21T21:59:52.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f060d59a700 1 -- 172.21.2.116:0/2798017180 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f060800bb70 con 0x7f0610143e60 2021-11-21T21:59:52.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 -- 172.21.2.116:0/2798017180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 msgr2=0x7f06101442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:52.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.222+0000 7f061497f700 1 --2- 172.21.2.116:0/2798017180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06101442f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f06100a8390 tx=0x7f0608009290).stop 2021-11-21T21:59:52.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- 172.21.2.116:0/2798017180 shutdown_connections 2021-11-21T21:59:52.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 --2- 172.21.2.116:0/2798017180 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06101442f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:52.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 --2- 172.21.2.116:0/2798017180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100a8670 0x7f0610147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:52.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- 172.21.2.116:0/2798017180 >> 172.21.2.116:0/2798017180 conn(0x7f061005bd20 msgr2=0x7f061005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:52.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- 172.21.2.116:0/2798017180 shutdown_connections 2021-11-21T21:59:52.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- 172.21.2.116:0/2798017180 wait complete. 2021-11-21T21:59:52.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 Processor -- start 2021-11-21T21:59:52.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- start start 2021-11-21T21:59:52.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06100a8670 0x7f0610063ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:52.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100b11c0 0x7f06100aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:52.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06100a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:52.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061005adb0 con 0x7f0610143e60 2021-11-21T21:59:52.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0610067770 con 0x7f06100a8670 2021-11-21T21:59:52.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610064530 con 0x7f06100b11c0 2021-11-21T21:59:52.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06100a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:52.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06100a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48730/0 (socket says 172.21.2.116:48730) 2021-11-21T21:59:52.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 -- 172.21.2.116:0/3150484033 learned_addr learned my addr 172.21.2.116:0/3150484033 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:52.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 -- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100b11c0 msgr2=0x7f06100aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:52.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100b11c0 0x7f06100aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:52.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 -- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06100a8670 msgr2=0x7f0610063ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:52.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06100a8670 0x7f0610063ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:52.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 -- 172.21.2.116:0/3150484033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608009530 con 0x7f0610143e60 2021-11-21T21:59:52.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060ed9d700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06100a8d20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f06100a8390 tx=0x7f0608016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:52.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f05ff7fe700 1 -- 172.21.2.116:0/3150484033 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06080032d0 con 0x7f0610143e60 2021-11-21T21:59:52.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f05ff7fe700 1 -- 172.21.2.116:0/3150484033 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060800bb30 con 0x7f0610143e60 2021-11-21T21:59:52.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f05ff7fe700 1 -- 172.21.2.116:0/3150484033 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0608020620 con 0x7f0610143e60 2021-11-21T21:59:52.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100b5810 con 0x7f0610143e60 2021-11-21T21:59:52.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0610065d00 con 0x7f0610143e60 2021-11-21T21:59:52.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0610064220 con 0x7f0610143e60 2021-11-21T21:59:52.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f05ff7fe700 1 -- 172.21.2.116:0/3150484033 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0608026020 con 0x7f0610143e60 2021-11-21T21:59:52.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f05ff7fe700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05ec053580 0x7f05ec055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:52.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f05ff7fe700 1 -- 172.21.2.116:0/3150484033 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f060806c9f0 con 0x7f0610143e60 2021-11-21T21:59:52.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.226+0000 7f060e59c700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05ec053580 0x7f05ec055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:52.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.230+0000 7f060e59c700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05ec053580 0x7f05ec055a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f0600002f90 tx=0x7f06000069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:52.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.230+0000 7f05ff7fe700 1 -- 172.21.2.116:0/3150484033 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06080439f0 con 0x7f0610143e60 2021-11-21T21:59:52.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:52.310+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06100a9210 con 0x7f0610143e60 2021-11-21T21:59:53.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f05ff7fe700 1 -- 172.21.2.116:0/3150484033 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 185+0+0 (secure 0 0 0) 0x7f0608015070 con 0x7f0610143e60 2021-11-21T21:59:53.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05ec053580 msgr2=0x7f05ec055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:53.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05ec053580 0x7f05ec055a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f0600002f90 tx=0x7f06000069b0).stop 2021-11-21T21:59:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 msgr2=0x7f06100a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06100a8d20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f06100a8390 tx=0x7f0608016000).stop 2021-11-21T21:59:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 shutdown_connections 2021-11-21T21:59:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f05ec053580 0x7f05ec055a40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0610143e60 0x7f06100a8d20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06100b11c0 0x7f06100aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 --2- 172.21.2.116:0/3150484033 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06100a8670 0x7f0610063ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 >> 172.21.2.116:0/3150484033 conn(0x7f061005bd20 msgr2=0x7f061014c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:53.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 shutdown_connections 2021-11-21T21:59:53.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.178+0000 7f061497f700 1 -- 172.21.2.116:0/3150484033 wait complete. 2021-11-21T21:59:53.184 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T21:59:53.188 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T21:59:53.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27b5f2700 1 Processor -- start 2021-11-21T21:59:53.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27b5f2700 1 -- start start 2021-11-21T21:59:53.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27b5f2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 0x7fa27406a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27b5f2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27413fe00 con 0x7fa274067ae0 2021-11-21T21:59:53.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27b5f2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2741400a0 con 0x7fa27413f8c0 2021-11-21T21:59:53.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27b5f2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa274140340 con 0x7fa2740b6010 2021-11-21T21:59:53.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa279b8f700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa274067ae0 0x7fa2740b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60694/0 (socket says 172.21.2.116:60694) 2021-11-21T21:59:53.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa279b8f700 1 -- 172.21.2.116:0/2254407325 learned_addr learned my addr 172.21.2.116:0/2254407325 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:53.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27938e700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 0x7fa27406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:53.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733251973 0 0) 0x7fa27413fe00 con 0x7fa274067ae0 2021-11-21T21:59:53.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa25c003260 con 0x7fa274067ae0 2021-11-21T21:59:53.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3923738764 0 0) 0x7fa25c003260 con 0x7fa274067ae0 2021-11-21T21:59:53.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 msgr2=0x7fa27406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:53.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 0x7fa27406a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 >> v1:172.21.2.117:6789/0 conn(0x7fa27413f8c0 legacy=0x7fa2740673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2741405e0 con 0x7fa274067ae0 2021-11-21T21:59:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fa270001660 con 0x7fa274067ae0 2021-11-21T21:59:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 >> v1:172.21.2.116:6789/0 conn(0x7fa274067ae0 legacy=0x7fa2740b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:53.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa25c003780 0x7fa25c003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa25c004500 0x7fa25c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 0x7fa25c004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274140340 con 0x7fa25c004500 2021-11-21T21:59:53.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c00fc10 con 0x7fa25c003780 2021-11-21T21:59:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c00fdd0 con 0x7fa2740b6010 2021-11-21T21:59:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa278b8d700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa25c004500 0x7fa25c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa27938e700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa25c003780 0x7fa25c003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa279b8f700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 0x7fa25c004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:53.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa278b8d700 1 -- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 msgr2=0x7fa25c004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:53.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa278b8d700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2740b6010 0x7fa25c004eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:53.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa278b8d700 1 -- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa25c003780 msgr2=0x7fa25c003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:53.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa278b8d700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa25c003780 0x7fa25c003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:53.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa278b8d700 1 -- 172.21.2.116:0/2254407325 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c003260 con 0x7fa25c004500 2021-11-21T21:59:53.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa278b8d700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa25c004500 0x7fa25c004980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fa274140960 tx=0x7fa264007a00).ready entity=mon.0 client_cookie=3096f031c88bb998 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:53.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa264007d10 con 0x7fa25c004500 2021-11-21T21:59:53.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa264007030 con 0x7fa25c004500 2021-11-21T21:59:53.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.238+0000 7fa263fff700 1 -- 172.21.2.116:0/2254407325 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa264007390 con 0x7fa25c004500 2021-11-21T21:59:53.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/2254407325 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa25c004500 msgr2=0x7fa25c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:53.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 --2- 172.21.2.116:0/2254407325 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa25c004500 0x7fa25c004980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fa274140960 tx=0x7fa264007a00).stop 2021-11-21T21:59:53.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 -- 172.21.2.116:0/2254407325 reap_dead start 2021-11-21T21:59:53.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/2254407325 shutdown_connections 2021-11-21T21:59:53.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/2254407325 >> 172.21.2.116:0/2254407325 conn(0x7fa274060200 msgr2=0x7fa2740b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:53.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/2254407325 shutdown_connections 2021-11-21T21:59:53.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/2254407325 wait complete. 2021-11-21T21:59:53.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 Processor -- start 2021-11-21T21:59:53.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- start start 2021-11-21T21:59:53.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa25c004500 0x7fa2740ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2740ae8f0 0x7fa274150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa274150530 0x7fa2740a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274157530 con 0x7fa274150530 2021-11-21T21:59:53.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274156ed0 con 0x7fa2740ae8f0 2021-11-21T21:59:53.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741570d0 con 0x7fa25c004500 2021-11-21T21:59:53.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa274150530 0x7fa2740a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:53.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa274150530 0x7fa2740a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48750/0 (socket says 172.21.2.116:48750) 2021-11-21T21:59:53.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 -- 172.21.2.116:0/3456191894 learned_addr learned my addr 172.21.2.116:0/3456191894 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:53.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 -- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa25c004500 msgr2=0x7fa2740ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:53.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa25c004500 0x7fa2740ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:53.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 -- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2740ae8f0 msgr2=0x7fa274150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:53.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2740ae8f0 0x7fa274150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:53.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 -- 172.21.2.116:0/3456191894 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2740a7270 con 0x7fa274150530 2021-11-21T21:59:53.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa279b8f700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa274150530 0x7fa2740a7ae0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fa2740a91b0 tx=0x7fa2740ab5e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:53.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa2627fc700 1 -- 172.21.2.116:0/3456191894 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2700116b0 con 0x7fa274150530 2021-11-21T21:59:53.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa2627fc700 1 -- 172.21.2.116:0/3456191894 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa270011810 con 0x7fa274150530 2021-11-21T21:59:53.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2740aed70 con 0x7fa274150530 2021-11-21T21:59:53.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa2627fc700 1 -- 172.21.2.116:0/3456191894 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa270011b70 con 0x7fa274150530 2021-11-21T21:59:53.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2740a6fd0 con 0x7fa274150530 2021-11-21T21:59:53.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2740573d0 con 0x7fa274150530 2021-11-21T21:59:53.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa2627fc700 1 -- 172.21.2.116:0/3456191894 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa27001b030 con 0x7fa274150530 2021-11-21T21:59:53.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa2627fc700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2540534f0 0x7fa2540559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:53.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa2627fc700 1 -- 172.21.2.116:0/3456191894 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fa27006b040 con 0x7fa274150530 2021-11-21T21:59:53.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.242+0000 7fa27938e700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2540534f0 0x7fa2540559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:53.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.246+0000 7fa27938e700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2540534f0 0x7fa2540559b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fa26c00b000 tx=0x7fa26c0091d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:53.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.246+0000 7fa2627fc700 1 -- 172.21.2.116:0/3456191894 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa270042300 con 0x7fa274150530 2021-11-21T21:59:53.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:53.330+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa27405c5a0 con 0x7fa274150530 2021-11-21T21:59:54.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.194+0000 7fa2627fc700 1 -- 172.21.2.116:0/3456191894 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa270045920 con 0x7fa274150530 2021-11-21T21:59:54.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2540534f0 msgr2=0x7fa2540559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2540534f0 0x7fa2540559b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fa26c00b000 tx=0x7fa26c0091d0).stop 2021-11-21T21:59:54.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa274150530 msgr2=0x7fa2740a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa274150530 0x7fa2740a7ae0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fa2740a91b0 tx=0x7fa2740ab5e0).stop 2021-11-21T21:59:54.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 shutdown_connections 2021-11-21T21:59:54.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2540534f0 0x7fa2540559b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa274150530 0x7fa2740a7ae0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2740ae8f0 0x7fa274150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 --2- 172.21.2.116:0/3456191894 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa25c004500 0x7fa2740ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 >> 172.21.2.116:0/3456191894 conn(0x7fa274060200 msgr2=0x7fa2740b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:54.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 shutdown_connections 2021-11-21T21:59:54.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.198+0000 7fa27b5f2700 1 -- 172.21.2.116:0/3456191894 wait complete. 2021-11-21T21:59:54.204 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T21:59:54.207 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T21:59:54.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 Processor -- start 2021-11-21T21:59:54.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- start start 2021-11-21T21:59:54.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbf7c144810 0x7fbf7c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbf7c0abce0 0x7fbf7c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c147e80 con 0x7fbf7c0b1db0 2021-11-21T21:59:54.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b1490 con 0x7fbf7c144810 2021-11-21T21:59:54.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c0af060 con 0x7fbf7c0abce0 2021-11-21T21:59:54.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbf7c144810 0x7fbf7c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbf7c144810 0x7fbf7c1478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48754/0 (socket says 172.21.2.116:48754) 2021-11-21T21:59:54.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 -- 172.21.2.116:0/1522424415 learned_addr learned my addr 172.21.2.116:0/1522424415 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:54.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 -- 172.21.2.116:0/1522424415 >> v2:172.21.2.117:3300/0 conn(0x7fbf7c0abce0 msgr2=0x7fbf7c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:54.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 --2- 172.21.2.116:0/1522424415 >> v2:172.21.2.117:3300/0 conn(0x7fbf7c0abce0 0x7fbf7c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 -- 172.21.2.116:0/1522424415 >> v1:172.21.2.117:6789/0 conn(0x7fbf7c0b1db0 legacy=0x7fbf7c0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:54.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 -- 172.21.2.116:0/1522424415 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0af5a0 con 0x7fbf7c144810 2021-11-21T21:59:54.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf7affd700 1 --2- 172.21.2.116:0/1522424415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c144810 0x7fbf7c1478c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7fbf7c0b4e00 tx=0x7fbf6c006eb0).ready entity=mon.0 client_cookie=d78d347990aec536 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:54.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf79ffb700 1 -- 172.21.2.116:0/1522424415 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6c00d5c0 con 0x7fbf7c144810 2021-11-21T21:59:54.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf79ffb700 1 -- 172.21.2.116:0/1522424415 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c000e90 con 0x7fbf7c144810 2021-11-21T21:59:54.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf79ffb700 1 -- 172.21.2.116:0/1522424415 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6c00d910 con 0x7fbf7c144810 2021-11-21T21:59:54.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- 172.21.2.116:0/1522424415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c144810 msgr2=0x7fbf7c1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 --2- 172.21.2.116:0/1522424415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c144810 0x7fbf7c1478c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7fbf7c0b4e00 tx=0x7fbf6c006eb0).stop 2021-11-21T21:59:54.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- 172.21.2.116:0/1522424415 shutdown_connections 2021-11-21T21:59:54.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 --2- 172.21.2.116:0/1522424415 >> v2:172.21.2.117:3300/0 conn(0x7fbf7c0abce0 0x7fbf7c0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 --2- 172.21.2.116:0/1522424415 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c144810 0x7fbf7c1478c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- 172.21.2.116:0/1522424415 >> 172.21.2.116:0/1522424415 conn(0x7fbf7c05bd20 msgr2=0x7fbf7c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:54.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- 172.21.2.116:0/1522424415 shutdown_connections 2021-11-21T21:59:54.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- 172.21.2.116:0/1522424415 wait complete. 2021-11-21T21:59:54.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 Processor -- start 2021-11-21T21:59:54.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.258+0000 7fbf812b4700 1 -- start start 2021-11-21T21:59:54.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf7c0abce0 0x7fbf7c143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf7c144810 0x7fbf7c158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c1457d0 0x7fbf7c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c159d20 con 0x7fbf7c1457d0 2021-11-21T21:59:54.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c1451c0 con 0x7fbf7c0abce0 2021-11-21T21:59:54.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b3bc0 con 0x7fbf7c144810 2021-11-21T21:59:54.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c1457d0 0x7fbf7c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c1457d0 0x7fbf7c1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48760/0 (socket says 172.21.2.116:48760) 2021-11-21T21:59:54.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 -- 172.21.2.116:0/696692503 learned_addr learned my addr 172.21.2.116:0/696692503 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:54.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7affd700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf7c0abce0 0x7fbf7c143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7a7fc700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf7c144810 0x7fbf7c158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 -- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf7c144810 msgr2=0x7fbf7c158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf7c144810 0x7fbf7c158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 -- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf7c0abce0 msgr2=0x7fbf7c143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf7c0abce0 0x7fbf7c143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 -- 172.21.2.116:0/696692503 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c00b1e0 con 0x7fbf7c1457d0 2021-11-21T21:59:54.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7b7fe700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c1457d0 0x7fbf7c1594a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fbf74003310 tx=0x7fbf740091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:54.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf73fff700 1 -- 172.21.2.116:0/696692503 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf74001500 con 0x7fbf7c1457d0 2021-11-21T21:59:54.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf73fff700 1 -- 172.21.2.116:0/696692503 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf74001660 con 0x7fbf7c1457d0 2021-11-21T21:59:54.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c159990 con 0x7fbf7c1457d0 2021-11-21T21:59:54.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf73fff700 1 -- 172.21.2.116:0/696692503 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf740019c0 con 0x7fbf7c1457d0 2021-11-21T21:59:54.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c144ec0 con 0x7fbf7c1457d0 2021-11-21T21:59:54.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c0af300 con 0x7fbf7c1457d0 2021-11-21T21:59:54.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf73fff700 1 -- 172.21.2.116:0/696692503 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbf74018030 con 0x7fbf7c1457d0 2021-11-21T21:59:54.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf73fff700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf60053640 0x7fbf60055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf73fff700 1 -- 172.21.2.116:0/696692503 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbf7406afc0 con 0x7fbf7c1457d0 2021-11-21T21:59:54.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7affd700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf60053640 0x7fbf60055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.262+0000 7fbf7affd700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf60053640 0x7fbf60055b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fbf6c01a000 tx=0x7fbf6c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:54.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.266+0000 7fbf73fff700 1 -- 172.21.2.116:0/696692503 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf740424b0 con 0x7fbf7c1457d0 2021-11-21T21:59:54.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.346+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbf7c145380 con 0x7fbf7c1457d0 2021-11-21T21:59:54.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf73fff700 1 -- 172.21.2.116:0/696692503 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf74045ad0 con 0x7fbf7c1457d0 2021-11-21T21:59:54.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf60053640 msgr2=0x7fbf60055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf60053640 0x7fbf60055b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fbf6c01a000 tx=0x7fbf6c015000).stop 2021-11-21T21:59:54.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c1457d0 msgr2=0x7fbf7c1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c1457d0 0x7fbf7c1594a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fbf74003310 tx=0x7fbf740091a0).stop 2021-11-21T21:59:54.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 shutdown_connections 2021-11-21T21:59:54.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf60053640 0x7fbf60055b00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf7c1457d0 0x7fbf7c1594a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf7c144810 0x7fbf7c158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 --2- 172.21.2.116:0/696692503 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf7c0abce0 0x7fbf7c143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 >> 172.21.2.116:0/696692503 conn(0x7fbf7c05bd20 msgr2=0x7fbf7c1409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:54.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 shutdown_connections 2021-11-21T21:59:54.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.354+0000 7fbf812b4700 1 -- 172.21.2.116:0/696692503 wait complete. 2021-11-21T21:59:54.364 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2021-11-21T21:59:54.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fe001e700 1 Processor -- start 2021-11-21T21:59:54.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fe001e700 1 -- start start 2021-11-21T21:59:54.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fe001e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2fd80a6aa0 0x7f2fd814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fe001e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2fd8062410 0x7f2fd81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fe001e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd80a6e80 con 0x7f2fd80a86b0 2021-11-21T21:59:54.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fe001e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd80a8bc0 con 0x7f2fd8062410 2021-11-21T21:59:54.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fe001e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd814a7b0 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2fd8062410 0x7f2fd81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2fd8062410 0x7f2fd81443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48768/0 (socket says 172.21.2.116:48768) 2021-11-21T21:59:54.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 -- 172.21.2.116:0/907408395 learned_addr learned my addr 172.21.2.116:0/907408395 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:54.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 -- 172.21.2.116:0/907408395 >> v2:172.21.2.117:3300/0 conn(0x7f2fd80a6aa0 msgr2=0x7f2fd814b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:54.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 --2- 172.21.2.116:0/907408395 >> v2:172.21.2.117:3300/0 conn(0x7f2fd80a6aa0 0x7f2fd814b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 -- 172.21.2.116:0/907408395 >> v1:172.21.2.117:6789/0 conn(0x7f2fd80a86b0 legacy=0x7f2fd81470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:54.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 -- 172.21.2.116:0/907408395 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd814aa10 con 0x7f2fd8062410 2021-11-21T21:59:54.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fde5bb700 1 --2- 172.21.2.116:0/907408395 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd8062410 0x7f2fd81443f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f2fd8061ab0 tx=0x7f2fd0009290).ready entity=mon.0 client_cookie=a404c877dbf9ce79 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:54.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fdcdb8700 1 -- 172.21.2.116:0/907408395 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd000b830 con 0x7f2fd8062410 2021-11-21T21:59:54.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fdcdb8700 1 -- 172.21.2.116:0/907408395 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd000b990 con 0x7f2fd8062410 2021-11-21T21:59:54.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.410+0000 7f2fdcdb8700 1 -- 172.21.2.116:0/907408395 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd000bb40 con 0x7f2fd8062410 2021-11-21T21:59:54.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/907408395 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd8062410 msgr2=0x7f2fd81443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 --2- 172.21.2.116:0/907408395 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd8062410 0x7f2fd81443f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f2fd8061ab0 tx=0x7f2fd0009290).stop 2021-11-21T21:59:54.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/907408395 shutdown_connections 2021-11-21T21:59:54.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 --2- 172.21.2.116:0/907408395 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd8062410 0x7f2fd81443f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:54.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 --2- 172.21.2.116:0/907408395 >> v2:172.21.2.117:3300/0 conn(0x7f2fd80a6aa0 0x7f2fd814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/907408395 >> 172.21.2.116:0/907408395 conn(0x7f2fd805bd20 msgr2=0x7f2fd805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:54.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/907408395 shutdown_connections 2021-11-21T21:59:54.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/907408395 wait complete. 2021-11-21T21:59:54.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 Processor -- start 2021-11-21T21:59:54.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- start start 2021-11-21T21:59:54.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fd8062410 0x7f2fd80ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd80a6aa0 0x7f2fd8063a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fd80a86b0 0x7f2fd8064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8140f60 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8159ba0 con 0x7f2fd80a86b0 2021-11-21T21:59:54.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8064830 con 0x7f2fd8062410 2021-11-21T21:59:54.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd80a6aa0 0x7f2fd8063a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd80a6aa0 0x7f2fd8063a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48774/0 (socket says 172.21.2.116:48774) 2021-11-21T21:59:54.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 -- 172.21.2.116:0/1907881105 learned_addr learned my addr 172.21.2.116:0/1907881105 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:54.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 -- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fd8062410 msgr2=0x7f2fd80ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fd8062410 0x7f2fd80ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 -- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fd80a86b0 msgr2=0x7f2fd8064ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:54.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fd80a86b0 0x7f2fd8064ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:54.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 -- 172.21.2.116:0/1907881105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009530 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fdd5b9700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd80a6aa0 0x7f2fd8063a60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f2fd80ab9d0 tx=0x7f2fc8006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:54.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fc6ffd700 1 -- 172.21.2.116:0/1907881105 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc8009e90 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fc6ffd700 1 -- 172.21.2.116:0/1907881105 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc8002320 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd8147e80 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fc6ffd700 1 -- 172.21.2.116:0/1907881105 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc8002680 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd8147940 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd80573d0 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fc6ffd700 1 -- 172.21.2.116:0/1907881105 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2fc800f030 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fc6ffd700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fb8053570 0x7f2fb8055a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:54.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.414+0000 7f2fc6ffd700 1 -- 172.21.2.116:0/1907881105 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2fc806b480 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.418+0000 7f2fdddba700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fb8053570 0x7f2fb8055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:54.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.418+0000 7f2fdddba700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fb8053570 0x7f2fb8055a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f2fd4002fb0 tx=0x7f2fd40069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:54.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.418+0000 7f2fc6ffd700 1 -- 172.21.2.116:0/1907881105 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fc80427b0 con 0x7f2fd80a6aa0 2021-11-21T21:59:54.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:54.530+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f2fd8142ba0 con 0x7f2fd80a6aa0 2021-11-21T21:59:55.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.198+0000 7f2fc6ffd700 1 -- 172.21.2.116:0/1907881105 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v227) v1 ==== 147+0+0 (secure 0 0 0) 0x7f2fc8045dd0 con 0x7f2fd80a6aa0 2021-11-21T21:59:55.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fb8053570 msgr2=0x7f2fb8055a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:55.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fb8053570 0x7f2fb8055a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f2fd4002fb0 tx=0x7f2fd40069b0).stop 2021-11-21T21:59:55.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd80a6aa0 msgr2=0x7f2fd8063a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd80a6aa0 0x7f2fd8063a60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f2fd80ab9d0 tx=0x7f2fc8006d50).stop 2021-11-21T21:59:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 shutdown_connections 2021-11-21T21:59:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fd80a86b0 0x7f2fd8064ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2fb8053570 0x7f2fb8055a30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2fd80a6aa0 0x7f2fd8063a60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 --2- 172.21.2.116:0/1907881105 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fd8062410 0x7f2fd80ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 >> 172.21.2.116:0/1907881105 conn(0x7f2fd805bd20 msgr2=0x7f2fd80b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 shutdown_connections 2021-11-21T21:59:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.202+0000 7f2fe001e700 1 -- 172.21.2.116:0/1907881105 wait complete. 2021-11-21T21:59:55.210 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-11-21T21:59:55.211 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2021-11-21T21:59:55.212 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2021-11-21T21:59:55.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11295ec700 1 Processor -- start 2021-11-21T21:59:55.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11295ec700 1 -- start start 2021-11-21T21:59:55.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11295ec700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1124147ca0 0x7f11240ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:55.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11295ec700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f112405cb00 0x7f11240ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:55.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11295ec700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112405c440 con 0x7f112405c060 2021-11-21T21:59:55.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11295ec700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11240b17d0 con 0x7f1124147ca0 2021-11-21T21:59:55.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11295ec700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11240aeee0 con 0x7f112405cb00 2021-11-21T21:59:55.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11227fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1124147ca0 0x7f11240ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:55.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11227fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1124147ca0 0x7f11240ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48786/0 (socket says 172.21.2.116:48786) 2021-11-21T21:59:55.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.258+0000 7f11227fc700 1 -- 172.21.2.116:0/741915265 learned_addr learned my addr 172.21.2.116:0/741915265 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:55.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 -- 172.21.2.116:0/741915265 >> v2:172.21.2.117:3300/0 conn(0x7f112405cb00 msgr2=0x7f11240ac840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:55.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 --2- 172.21.2.116:0/741915265 >> v2:172.21.2.117:3300/0 conn(0x7f112405cb00 0x7f11240ac840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:55.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 -- 172.21.2.116:0/741915265 >> v1:172.21.2.117:6789/0 conn(0x7f112405c060 legacy=0x7f1124147590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:55.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 -- 172.21.2.116:0/741915265 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124067a50 con 0x7f1124147ca0 2021-11-21T21:59:55.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 --2- 172.21.2.116:0/741915265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1124147ca0 0x7f11240ac350 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f112405f020 tx=0x7f1118006eb0).ready entity=mon.0 client_cookie=34fc6ec57da7cf3d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:55.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f1121ffb700 1 -- 172.21.2.116:0/741915265 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111800d5c0 con 0x7f1124147ca0 2021-11-21T21:59:55.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f1121ffb700 1 -- 172.21.2.116:0/741915265 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118000e90 con 0x7f1124147ca0 2021-11-21T21:59:55.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f1121ffb700 1 -- 172.21.2.116:0/741915265 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111800d910 con 0x7f1124147ca0 2021-11-21T21:59:55.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- 172.21.2.116:0/741915265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1124147ca0 msgr2=0x7f11240ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:55.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 --2- 172.21.2.116:0/741915265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1124147ca0 0x7f11240ac350 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f112405f020 tx=0x7f1118006eb0).stop 2021-11-21T21:59:55.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- 172.21.2.116:0/741915265 shutdown_connections 2021-11-21T21:59:55.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 --2- 172.21.2.116:0/741915265 >> v2:172.21.2.117:3300/0 conn(0x7f112405cb00 0x7f11240ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:55.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 --2- 172.21.2.116:0/741915265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1124147ca0 0x7f11240ac350 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:55.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- 172.21.2.116:0/741915265 >> 172.21.2.116:0/741915265 conn(0x7f11240b4f10 msgr2=0x7f11240b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:55.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- 172.21.2.116:0/741915265 shutdown_connections 2021-11-21T21:59:55.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- 172.21.2.116:0/741915265 wait complete. 2021-11-21T21:59:55.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 Processor -- start 2021-11-21T21:59:55.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- start start 2021-11-21T21:59:55.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f112405c060 0x7f11240ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:55.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f112405cb00 0x7f1124156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:55.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1124147ca0 0x7f1124063ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:55.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11241573d0 con 0x7f112405cb00 2021-11-21T21:59:55.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11240a7ed0 con 0x7f1124147ca0 2021-11-21T21:59:55.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11240a8170 con 0x7f112405c060 2021-11-21T21:59:55.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f112405cb00 0x7f1124156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:55.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f112405cb00 0x7f1124156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48788/0 (socket says 172.21.2.116:48788) 2021-11-21T21:59:55.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 -- 172.21.2.116:0/158294993 learned_addr learned my addr 172.21.2.116:0/158294993 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:55.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 -- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f112405c060 msgr2=0x7f11240ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:55.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f112405c060 0x7f11240ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:55.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 -- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1124147ca0 msgr2=0x7f1124063ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:55.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1124147ca0 0x7f1124063ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:55.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 -- 172.21.2.116:0/158294993 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111800b1e0 con 0x7f112405cb00 2021-11-21T21:59:55.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11227fc700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f112405cb00 0x7f1124156ee0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f112405f020 tx=0x7f1118015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:55.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f1113fff700 1 -- 172.21.2.116:0/158294993 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1118002f70 con 0x7f112405cb00 2021-11-21T21:59:55.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f1113fff700 1 -- 172.21.2.116:0/158294993 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111800de70 con 0x7f112405cb00 2021-11-21T21:59:55.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11240a8410 con 0x7f112405cb00 2021-11-21T21:59:55.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11240a7b50 con 0x7f112405cb00 2021-11-21T21:59:55.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.262+0000 7f1113fff700 1 -- 172.21.2.116:0/158294993 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111801e460 con 0x7f112405cb00 2021-11-21T21:59:55.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.266+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11240b0df0 con 0x7f112405cb00 2021-11-21T21:59:55.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.266+0000 7f1113fff700 1 -- 172.21.2.116:0/158294993 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1118025030 con 0x7f112405cb00 2021-11-21T21:59:55.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.266+0000 7f1113fff700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1100053660 0x7f1100055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:55.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.266+0000 7f1113fff700 1 -- 172.21.2.116:0/158294993 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f111806cb00 con 0x7f112405cb00 2021-11-21T21:59:55.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.266+0000 7f1122ffd700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1100053660 0x7f1100055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:55.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.266+0000 7f1122ffd700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1100053660 0x7f1100055b20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f1114002fb0 tx=0x7f11140069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:55.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.266+0000 7f1113fff700 1 -- 172.21.2.116:0/158294993 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1118043ee0 con 0x7f112405cb00 2021-11-21T21:59:55.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:55.378+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f11240af0f0 con 0x7f112405cb00 2021-11-21T21:59:56.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f1113fff700 1 -- 172.21.2.116:0/158294993 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v228) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1118023020 con 0x7f112405cb00 2021-11-21T21:59:56.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1100053660 msgr2=0x7f1100055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1100053660 0x7f1100055b20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f1114002fb0 tx=0x7f11140069b0).stop 2021-11-21T21:59:56.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f112405cb00 msgr2=0x7f1124156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f112405cb00 0x7f1124156ee0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f112405f020 tx=0x7f1118015000).stop 2021-11-21T21:59:56.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 shutdown_connections 2021-11-21T21:59:56.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1124147ca0 0x7f1124063ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1100053660 0x7f1100055b20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f112405cb00 0x7f1124156ee0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 --2- 172.21.2.116:0/158294993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f112405c060 0x7f11240ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 >> 172.21.2.116:0/158294993 conn(0x7f11240b4f10 msgr2=0x7f11240b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:56.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 shutdown_connections 2021-11-21T21:59:56.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.206+0000 7f11295ec700 1 -- 172.21.2.116:0/158294993 wait complete. 2021-11-21T21:59:56.219 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T21:59:56.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 Processor -- start 2021-11-21T21:59:56.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- start start 2021-11-21T21:59:56.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fdef81478c0 0x7fdef80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef8147e80 con 0x7fdef80abce0 2021-11-21T21:59:56.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef80af5a0 con 0x7fdef8144810 2021-11-21T21:59:56.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80b1490 con 0x7fdef81478c0 2021-11-21T21:59:56.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefcb10700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fdef80abce0 0x7fdef80ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60756/0 (socket says 172.21.2.116:60756) 2021-11-21T21:59:56.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefcb10700 1 -- 172.21.2.116:0/3681665236 learned_addr learned my addr 172.21.2.116:0/3681665236 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:56.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079193078 0 0) 0x7fdef8147e80 con 0x7fdef80abce0 2021-11-21T21:59:56.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee0003260 con 0x7fdef80abce0 2021-11-21T21:59:56.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 --2- 172.21.2.116:0/3681665236 >> v2:172.21.2.117:3300/0 conn(0x7fdef81478c0 0x7fdef80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2627215890 0 0) 0x7fdee0003260 con 0x7fdef80abce0 2021-11-21T21:59:56.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 >> v2:172.21.2.117:3300/0 conn(0x7fdef81478c0 msgr2=0x7fdef80acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 --2- 172.21.2.116:0/3681665236 >> v2:172.21.2.117:3300/0 conn(0x7fdef81478c0 0x7fdef80acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 >> v1:172.21.2.117:6789/0 conn(0x7fdef8144810 legacy=0x7fdef80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80b1fd0 con 0x7fdef80abce0 2021-11-21T21:59:56.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fdef4001660 con 0x7fdef80abce0 2021-11-21T21:59:56.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 >> v1:172.21.2.116:6789/0 conn(0x7fdef80abce0 legacy=0x7fdef80ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdee0003820 0x7fdee0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdee0004500 0x7fdee0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdee0005050 0x7fdee000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80b1490 con 0x7fdee0005050 2021-11-21T21:59:56.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee000bf40 con 0x7fdee0004500 2021-11-21T21:59:56.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee000c100 con 0x7fdee0003820 2021-11-21T21:59:56.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdee0005050 0x7fdee000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 -- 172.21.2.116:0/3681665236 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdee0003820 msgr2=0x7fdee0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdee0003820 0x7fdee0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 -- 172.21.2.116:0/3681665236 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdee0004500 msgr2=0x7fdee0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdee0004500 0x7fdee0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 -- 172.21.2.116:0/3681665236 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0003260 con 0x7fdee0005050 2021-11-21T21:59:56.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 -- 172.21.2.116:0/3681665236 reap_dead start 2021-11-21T21:59:56.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdefdb12700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdee0005050 0x7fdee000b750 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fdef0007000 tx=0x7fdef0008040).ready entity=mon.0 client_cookie=22180863a73dfd88 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef0015070 con 0x7fdee0005050 2021-11-21T21:59:56.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef000c640 con 0x7fdee0005050 2021-11-21T21:59:56.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdee7fff700 1 -- 172.21.2.116:0/3681665236 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef000a4a0 con 0x7fdee0005050 2021-11-21T21:59:56.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- 172.21.2.116:0/3681665236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdee0005050 msgr2=0x7fdee000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdee0005050 0x7fdee000b750 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fdef0007000 tx=0x7fdef0008040).stop 2021-11-21T21:59:56.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- 172.21.2.116:0/3681665236 shutdown_connections 2021-11-21T21:59:56.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 --2- 172.21.2.116:0/3681665236 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdee0005050 0x7fdee000b750 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- 172.21.2.116:0/3681665236 >> 172.21.2.116:0/3681665236 conn(0x7fdef805bd20 msgr2=0x7fdef805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:56.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- 172.21.2.116:0/3681665236 shutdown_connections 2021-11-21T21:59:56.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.270+0000 7fdeff575700 1 -- 172.21.2.116:0/3681665236 wait complete. 2021-11-21T21:59:56.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 Processor -- start 2021-11-21T21:59:56.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 -- start start 2021-11-21T21:59:56.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdee0005050 0x7fdef8146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdef80b3370 0x7fdef8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdef80629c0 0x7fdef8062e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8153f40 con 0x7fdef80b3370 2021-11-21T21:59:56.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81458c0 con 0x7fdee0005050 2021-11-21T21:59:56.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80b2c30 con 0x7fdef80629c0 2021-11-21T21:59:56.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdef80b3370 0x7fdef8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdef80b3370 0x7fdef8156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48810/0 (socket says 172.21.2.116:48810) 2021-11-21T21:59:56.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 -- 172.21.2.116:0/3950536178 learned_addr learned my addr 172.21.2.116:0/3950536178 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:56.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 -- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdef80629c0 msgr2=0x7fdef8062e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdef80629c0 0x7fdef8062e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 -- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdee0005050 msgr2=0x7fdef8146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdee0005050 0x7fdef8146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 -- 172.21.2.116:0/3950536178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80b2ed0 con 0x7fdef80b3370 2021-11-21T21:59:56.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefcb10700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdef80b3370 0x7fdef8156290 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7fdef80b4e00 tx=0x7fdef80aee70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdee67fc700 1 -- 172.21.2.116:0/3950536178 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef4007170 con 0x7fdef80b3370 2021-11-21T21:59:56.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdee67fc700 1 -- 172.21.2.116:0/3950536178 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef40072d0 con 0x7fdef80b3370 2021-11-21T21:59:56.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdee67fc700 1 -- 172.21.2.116:0/3950536178 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef4004920 con 0x7fdef80b3370 2021-11-21T21:59:56.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef8143b30 con 0x7fdef80b3370 2021-11-21T21:59:56.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef80b37f0 con 0x7fdef80b3370 2021-11-21T21:59:56.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef80b3060 con 0x7fdef80b3370 2021-11-21T21:59:56.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdee67fc700 1 -- 172.21.2.116:0/3950536178 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdef401b070 con 0x7fdef80b3370 2021-11-21T21:59:56.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdee67fc700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fded8053580 0x7fded8055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdee67fc700 1 -- 172.21.2.116:0/3950536178 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdef406ade0 con 0x7fdef80b3370 2021-11-21T21:59:56.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.274+0000 7fdefd311700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fded8053580 0x7fded8055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.278+0000 7fdefd311700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fded8053580 0x7fded8055a40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fdee8000f70 tx=0x7fdee80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.278+0000 7fdee67fc700 1 -- 172.21.2.116:0/3950536178 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdef40421c0 con 0x7fdef80b3370 2021-11-21T21:59:56.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.362+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdef80573d0 con 0x7fdef80b3370 2021-11-21T21:59:56.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdee67fc700 1 -- 172.21.2.116:0/3950536178 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdef4019020 con 0x7fdef80b3370 2021-11-21T21:59:56.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fded8053580 msgr2=0x7fded8055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fded8053580 0x7fded8055a40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fdee8000f70 tx=0x7fdee80069b0).stop 2021-11-21T21:59:56.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdef80b3370 msgr2=0x7fdef8156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdef80b3370 0x7fdef8156290 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7fdef80b4e00 tx=0x7fdef80aee70).stop 2021-11-21T21:59:56.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 shutdown_connections 2021-11-21T21:59:56.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdef80629c0 0x7fdef8062e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fded8053580 0x7fded8055a40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdef80b3370 0x7fdef8156290 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 --2- 172.21.2.116:0/3950536178 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdee0005050 0x7fdef8146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 >> 172.21.2.116:0/3950536178 conn(0x7fdef805bd20 msgr2=0x7fdef8148a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:56.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 shutdown_connections 2021-11-21T21:59:56.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.366+0000 7fdeff575700 1 -- 172.21.2.116:0/3950536178 wait complete. 2021-11-21T21:59:56.376 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T21:59:56.377 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:56.385 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:56.393 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:56.406 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T21:59:56.420 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T21:59:56.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:56.421 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:56.421 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T21:59:56.430 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T21:59:56.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:56.431 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T21:59:56.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T21:59:56.432 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T21:59:56.433 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T21:59:56.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T21:59:56.468 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T21:59:56.469 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T21:59:56.469 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T21:59:56.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 Processor -- start 2021-11-21T21:59:56.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- start start 2021-11-21T21:59:56.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6a640a6aa0 0x7f6a6414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a640a6e80 con 0x7f6a64062410 2021-11-21T21:59:56.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6414a7b0 con 0x7f6a640a86b0 2021-11-21T21:59:56.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a640a8bc0 con 0x7f6a640a6aa0 2021-11-21T21:59:56.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69a47700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f6a64062410 0x7f6a641443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60772/0 (socket says 172.21.2.116:60772) 2021-11-21T21:59:56.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69a47700 1 -- 172.21.2.116:0/1657767893 learned_addr learned my addr 172.21.2.116:0/1657767893 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:56.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357034245 0 0) 0x7f6a640a6e80 con 0x7f6a64062410 2021-11-21T21:59:56.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a4c003260 con 0x7f6a64062410 2021-11-21T21:59:56.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 --2- 172.21.2.116:0/1657767893 >> v2:172.21.2.117:3300/0 conn(0x7f6a640a6aa0 0x7f6a6414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1876775054 0 0) 0x7f6a4c003260 con 0x7f6a64062410 2021-11-21T21:59:56.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 >> v2:172.21.2.117:3300/0 conn(0x7f6a640a6aa0 msgr2=0x7f6a6414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 --2- 172.21.2.116:0/1657767893 >> v2:172.21.2.117:3300/0 conn(0x7f6a640a6aa0 0x7f6a6414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 >> v1:172.21.2.117:6789/0 conn(0x7f6a640a86b0 legacy=0x7f6a641470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6414aa10 con 0x7f6a64062410 2021-11-21T21:59:56.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f6a5c001a00 con 0x7f6a64062410 2021-11-21T21:59:56.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 >> v1:172.21.2.116:6789/0 conn(0x7f6a64062410 legacy=0x7f6a641443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a4c0037a0 0x7f6a4c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c004500 0x7f6a4c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a4c004e70 0x7f6a4c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a8bc0 con 0x7f6a4c0037a0 2021-11-21T21:59:56.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c00bda0 con 0x7f6a4c004500 2021-11-21T21:59:56.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c00bf60 con 0x7f6a4c004e70 2021-11-21T21:59:56.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a4c0037a0 0x7f6a4c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69a47700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a4c004e70 0x7f6a4c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a68a45700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c004500 0x7f6a4c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 -- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a4c004e70 msgr2=0x7f6a4c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a4c004e70 0x7f6a4c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 -- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c004500 msgr2=0x7f6a4c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c004500 0x7f6a4c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 -- 172.21.2.116:0/1657767893 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c003260 con 0x7f6a4c0037a0 2021-11-21T21:59:56.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a68a45700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c004500 0x7f6a4c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:59:56.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69a47700 1 -- 172.21.2.116:0/1657767893 reap_dead start 2021-11-21T21:59:56.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a69246700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a4c0037a0 0x7f6a4c003bc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f6a6000c000 tx=0x7f6a60006df0).ready entity=mon.0 client_cookie=2609a5a56e8e3ca8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a6000e830 con 0x7f6a4c0037a0 2021-11-21T21:59:56.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6000e990 con 0x7f6a4c0037a0 2021-11-21T21:59:56.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a57fff700 1 -- 172.21.2.116:0/1657767893 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a6000ecf0 con 0x7f6a4c0037a0 2021-11-21T21:59:56.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/1657767893 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a4c0037a0 msgr2=0x7f6a4c003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a4c0037a0 0x7f6a4c003bc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f6a6000c000 tx=0x7f6a60006df0).stop 2021-11-21T21:59:56.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/1657767893 shutdown_connections 2021-11-21T21:59:56.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/1657767893 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a4c0037a0 0x7f6a4c003bc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/1657767893 >> 172.21.2.116:0/1657767893 conn(0x7f6a6405bd20 msgr2=0x7f6a6405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:56.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/1657767893 shutdown_connections 2021-11-21T21:59:56.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.518+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/1657767893 wait complete. 2021-11-21T21:59:56.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 Processor -- start 2021-11-21T21:59:56.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 -- start start 2021-11-21T21:59:56.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c0037a0 0x7f6a6414cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a640651e0 0x7f6a64159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a64156190 0x7f6a6415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6414bc40 con 0x7f6a640651e0 2021-11-21T21:59:56.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6414be70 con 0x7f6a4c0037a0 2021-11-21T21:59:56.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6414c110 con 0x7f6a64156190 2021-11-21T21:59:56.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a640651e0 0x7f6a64159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a640651e0 0x7f6a64159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48830/0 (socket says 172.21.2.116:48830) 2021-11-21T21:59:56.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 -- 172.21.2.116:0/3522426544 learned_addr learned my addr 172.21.2.116:0/3522426544 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:56.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a69246700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c0037a0 0x7f6a6414cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a69a47700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a64156190 0x7f6a6415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 -- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a64156190 msgr2=0x7f6a6415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a64156190 0x7f6a6415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 -- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c0037a0 msgr2=0x7f6a6414cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c0037a0 0x7f6a6414cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 -- 172.21.2.116:0/3522426544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64159880 con 0x7f6a640651e0 2021-11-21T21:59:56.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a69a47700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a64156190 0x7f6a6415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T21:59:56.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a68a45700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a640651e0 0x7f6a64159d20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f6a580076e0 tx=0x7f6a58009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a567fc700 1 -- 172.21.2.116:0/3522426544 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a58014620 con 0x7f6a640651e0 2021-11-21T21:59:56.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a567fc700 1 -- 172.21.2.116:0/3522426544 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a58014780 con 0x7f6a640651e0 2021-11-21T21:59:56.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a640a9ab0 con 0x7f6a640651e0 2021-11-21T21:59:56.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a64142fd0 con 0x7f6a640651e0 2021-11-21T21:59:56.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a567fc700 1 -- 172.21.2.116:0/3522426544 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a58014ae0 con 0x7f6a640651e0 2021-11-21T21:59:56.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a64159690 con 0x7f6a640651e0 2021-11-21T21:59:56.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a567fc700 1 -- 172.21.2.116:0/3522426544 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6a58014c40 con 0x7f6a640651e0 2021-11-21T21:59:56.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a567fc700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a44053690 0x7f6a44055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a567fc700 1 -- 172.21.2.116:0/3522426544 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6a5806bbb0 con 0x7f6a640651e0 2021-11-21T21:59:56.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a69246700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a44053690 0x7f6a44055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.522+0000 7f6a69246700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a44053690 0x7f6a44055b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f6a6001b000 tx=0x7f6a60019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.526+0000 7f6a567fc700 1 -- 172.21.2.116:0/3522426544 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a58042ee0 con 0x7f6a640651e0 2021-11-21T21:59:56.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.610+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6a64159a10 con 0x7f6a640651e0 2021-11-21T21:59:56.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.610+0000 7f6a567fc700 1 -- 172.21.2.116:0/3522426544 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v69) v1 ==== 81+0+28828 (secure 0 0 0) 0x7f6a58007db0 con 0x7f6a640651e0 2021-11-21T21:59:56.616 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:56.616 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:59:56.616 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 69, 2021-11-21T21:59:56.616 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T21:59:56.617 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T21:59:56.617 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T21:59:54.181361+0000", 2021-11-21T21:59:56.617 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:56.617 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T21:59:56.618 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T21:59:56.618 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T21:59:56.618 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T21:59:56.618 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T21:59:56.619 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T21:59:56.619 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T21:59:56.619 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T21:59:56.619 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:56.620 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T21:59:56.620 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T21:59:56.620 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T21:59:56.620 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T21:59:56.620 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T21:59:56.621 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 12, 2021-11-21T21:59:56.622 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T21:59:56.622 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T21:59:56.622 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T21:59:56.622 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T21:59:56.623 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T21:59:56.623 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.623 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T21:59:56.623 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T21:59:56.624 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T21:59:56.624 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:56.624 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:56.624 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:56.624 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:56.625 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:56.625 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:56.625 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:56.625 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:56.626 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T21:59:56.626 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T21:59:56.626 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T21:59:56.626 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T21:59:56.626 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T21:59:56.627 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:56.627 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:56.627 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:56.627 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:56.628 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:56.628 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:56.628 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:56.628 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.628 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T21:59:56.629 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:56.629 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:56.629 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:56.629 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:56.629 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:56.630 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:56.630 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:56.630 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:56.630 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:56.631 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:56.631 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:56.631 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:56.631 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:56.631 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:56.632 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:56.632 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:56.632 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:56.632 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:56.633 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:56.633 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:56.633 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:56.633 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:56.634 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:56.634 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:56.634 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:56.634 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:56.634 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.635 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:56.635 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:56.635 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:56.635 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:56.636 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:56.636 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:56.636 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:56.636 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:56.636 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:56.637 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:56.637 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:56.637 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T21:59:56.637 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T21:59:56.638 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.638 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:56.638 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T21:59:56.638 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.638 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.639 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.639 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T21:59:56.639 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T21:59:56.639 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T21:59:56.639 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T21:59:56.640 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T21:59:56.640 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T21:59:56.640 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T21:59:56.640 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T21:59:56.641 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T21:59:56.641 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T21:59:56.641 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T21:59:56.641 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T21:59:56.641 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T21:59:56.642 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T21:59:56.642 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T21:59:56.642 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T21:59:56.642 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T21:59:56.642 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T21:59:56.643 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T21:59:56.643 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T21:59:56.643 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T21:59:56.643 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T21:59:56.644 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T21:59:56.644 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.644 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T21:59:56.644 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T21:59:56.644 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T21:59:56.644 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T21:59:56.645 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T21:59:56.645 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T21:59:56.645 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T21:59:56.645 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T21:59:56.645 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T21:59:56.646 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T21:59:56.646 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T21:59:56.646 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T21:59:56.646 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T21:59:56.647 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T21:59:56.647 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T21:59:56.647 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T21:59:56.647 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T21:59:56.647 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T21:59:56.648 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T21:59:56.648 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T21:59:56.648 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T21:59:56.648 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T21:59:56.648 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T21:59:56.649 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T21:59:56.649 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T21:59:56.649 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T21:59:56.649 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T21:59:56.650 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.650 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T21:59:56.650 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T21:59:56.650 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T21:59:56.650 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T21:59:56.651 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T21:59:56.651 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T21:59:56.651 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T21:59:56.651 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T21:59:56.652 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T21:59:56.652 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T21:59:56.652 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T21:59:56.652 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T21:59:56.652 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T21:59:56.653 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T21:59:56.653 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.653 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.653 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:56.653 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T21:59:56.654 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.654 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:56.654 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T21:59:56.654 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.654 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.655 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.655 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.655 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.655 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.656 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.656 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:56.656 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.656 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.656 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.657 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.657 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.657 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.657 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T21:59:56.657 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.658 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.658 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.658 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.658 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T21:59:56.658 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.659 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.659 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.659 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.659 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.660 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.660 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.660 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.660 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T21:59:56.660 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.661 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.661 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.661 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.661 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T21:59:56.661 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.662 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.662 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.662 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.662 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.663 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.663 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.663 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.663 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T21:59:56.663 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.664 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.664 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.664 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.664 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T21:59:56.664 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.665 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.665 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.665 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.665 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.665 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.666 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.666 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.666 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T21:59:56.666 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.666 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.667 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.667 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.667 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T21:59:56.667 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T21:59:56.667 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.668 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.668 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.668 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T21:59:56.668 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T21:59:56.668 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T21:59:56.669 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T21:59:56.669 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.669 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.669 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.669 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.670 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.670 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.670 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:56.670 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T21:59:56.671 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.671 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.671 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.671 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.671 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.672 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.672 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.672 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:56.672 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.673 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.673 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.673 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.673 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.673 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.674 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T21:59:56.674 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.674 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.674 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.675 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.675 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T21:59:56.675 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.675 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.675 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.676 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.676 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.676 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.676 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.676 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.677 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T21:59:56.677 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.677 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.677 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.678 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.678 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T21:59:56.678 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.678 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.678 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.679 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.679 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.679 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.679 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.680 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.680 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T21:59:56.680 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.680 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.680 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.681 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.681 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T21:59:56.681 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.681 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.681 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.682 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.682 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.682 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.682 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.683 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.683 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T21:59:56.683 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.683 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.683 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.684 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.684 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T21:59:56.684 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T21:59:56.684 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.684 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.685 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.685 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T21:59:56.685 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T21:59:56.685 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T21:59:56.686 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T21:59:56.686 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.686 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.686 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.686 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.687 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.687 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.687 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:56.687 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T21:59:56.687 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.688 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.688 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.688 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.688 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.689 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.689 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.689 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 63, 2021-11-21T21:59:56.689 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.689 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.690 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.690 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.691 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.691 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.692 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T21:59:56.692 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.692 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.693 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.693 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.693 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T21:59:56.693 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.694 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.694 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.694 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.694 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.695 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.695 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.695 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.695 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T21:59:56.696 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.696 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.696 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.696 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.697 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T21:59:56.697 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.697 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.697 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.698 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.698 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.698 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.698 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.699 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.699 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T21:59:56.699 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.700 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.700 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.700 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.700 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T21:59:56.701 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.701 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.701 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.701 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.702 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.702 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.702 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.702 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.703 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T21:59:56.703 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.703 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.703 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.703 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.704 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T21:59:56.704 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T21:59:56.704 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.704 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.705 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.705 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T21:59:56.705 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T21:59:56.706 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T21:59:56.706 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T21:59:56.706 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.706 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.707 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.707 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.707 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.707 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.708 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:56.708 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T21:59:56.708 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.708 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.709 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.709 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.709 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.709 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.709 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.710 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 52, 2021-11-21T21:59:56.710 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.710 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.710 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.714 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.715 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.715 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.715 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T21:59:56.716 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.716 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.716 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.716 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.717 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T21:59:56.717 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.717 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.717 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.717 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.718 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.718 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.718 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.718 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.719 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T21:59:56.719 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.719 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.719 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.719 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.720 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T21:59:56.720 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.720 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.720 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.721 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.721 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.721 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.721 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.721 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.722 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T21:59:56.722 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.722 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.722 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.723 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.723 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T21:59:56.723 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.723 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.723 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.724 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.724 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.724 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.724 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.725 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.725 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T21:59:56.725 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.725 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.725 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.726 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.726 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T21:59:56.726 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T21:59:56.726 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.726 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.727 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.727 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T21:59:56.727 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T21:59:56.727 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T21:59:56.728 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T21:59:56.728 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.728 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.728 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.728 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.729 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.729 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.729 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:56.729 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T21:59:56.730 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.730 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.730 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.730 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.730 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.731 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.731 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.731 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:56.731 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.732 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.732 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.732 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.732 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.732 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.733 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T21:59:56.733 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.733 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.733 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.733 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.734 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T21:59:56.734 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.734 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.734 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.735 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.735 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.735 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.735 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.735 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.736 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T21:59:56.736 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.736 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.737 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.737 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.738 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T21:59:56.738 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.738 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.738 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.739 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.739 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.739 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.739 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.739 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.740 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T21:59:56.740 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.740 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.740 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.740 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.741 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T21:59:56.741 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.741 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.741 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.742 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.742 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.742 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.742 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.742 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.743 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T21:59:56.743 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.743 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.743 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.743 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.744 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T21:59:56.744 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T21:59:56.744 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.744 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.745 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.745 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T21:59:56.745 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T21:59:56.745 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T21:59:56.745 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T21:59:56.746 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.746 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.746 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.746 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.746 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.747 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.747 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:56.747 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T21:59:56.747 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.748 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.748 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.748 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.748 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.748 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.749 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.749 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 63, 2021-11-21T21:59:56.749 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.749 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.749 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.750 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.750 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.750 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.750 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T21:59:56.751 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.751 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.751 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.751 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.751 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T21:59:56.752 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.752 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.752 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.752 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.752 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.753 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.753 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.753 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.753 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T21:59:56.754 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.754 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.754 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.754 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.754 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T21:59:56.755 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.755 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.755 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.755 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.755 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.756 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.756 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.756 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.756 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T21:59:56.757 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.757 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.757 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.757 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.757 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T21:59:56.758 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.758 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.758 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.758 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.758 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.759 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.759 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.759 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.759 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T21:59:56.760 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.760 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.760 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.760 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.760 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T21:59:56.761 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T21:59:56.761 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.761 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.761 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.761 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T21:59:56.762 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T21:59:56.762 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T21:59:56.762 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T21:59:56.762 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.762 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.763 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.763 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.763 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.763 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.764 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:56.764 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T21:59:56.764 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.764 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.764 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.765 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.765 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.765 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.765 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.766 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T21:59:56.766 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.766 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.766 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.766 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.767 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.767 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.767 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T21:59:56.767 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.768 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.768 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.768 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.768 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T21:59:56.768 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.769 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.769 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.769 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.769 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.769 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.770 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.770 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.770 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T21:59:56.770 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.770 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.771 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.771 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.771 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T21:59:56.771 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.772 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.772 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.772 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.772 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.772 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.773 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.773 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.773 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T21:59:56.773 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.773 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.774 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.774 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.774 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T21:59:56.774 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.774 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.775 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.775 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.775 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.775 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.776 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.776 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.776 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T21:59:56.776 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.776 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.777 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.777 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.777 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T21:59:56.777 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T21:59:56.777 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.778 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.778 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.778 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T21:59:56.778 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T21:59:56.778 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T21:59:56.779 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T21:59:56.779 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.779 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.779 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.780 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.780 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.780 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.780 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:56.780 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T21:59:56.781 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T21:59:56.781 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T21:59:56.781 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T21:59:56.781 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T21:59:56.781 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T21:59:56.782 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T21:59:56.782 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T21:59:56.782 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 63, 2021-11-21T21:59:56.782 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T21:59:56.783 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T21:59:56.783 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T21:59:56.783 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.783 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.783 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.784 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T21:59:56.784 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.784 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.784 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.785 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.785 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T21:59:56.785 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.785 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.785 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.786 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.786 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T21:59:56.786 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.786 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.786 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.787 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T21:59:56.787 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.787 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.787 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.788 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.788 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T21:59:56.788 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.788 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.788 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.789 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.789 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T21:59:56.789 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.789 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.789 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.790 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T21:59:56.790 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.790 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.790 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.790 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.791 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T21:59:56.791 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.791 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.791 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.792 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.792 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T21:59:56.792 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T21:59:56.792 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.792 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T21:59:56.793 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T21:59:56.793 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.793 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.793 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.793 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T21:59:56.794 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T21:59:56.794 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T21:59:56.794 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.794 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.795 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.795 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T21:59:56.795 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T21:59:56.795 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T21:59:56.795 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T21:59:56.796 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T21:59:56.796 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T21:59:56.796 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T21:59:56.796 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:56.796 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.797 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:56.797 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T21:59:56.797 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.797 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T21:59:56.797 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.798 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.798 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.798 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.798 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.799 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T21:59:56.799 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.799 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.799 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.799 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T21:59:56.800 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.800 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.800 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.800 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.801 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.801 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T21:59:56.801 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.801 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.801 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.802 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T21:59:56.802 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.802 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.802 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.802 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.803 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.803 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T21:59:56.803 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.803 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.804 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.804 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T21:59:56.804 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.804 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.804 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.805 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.805 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.805 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T21:59:56.805 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.806 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.806 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.806 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T21:59:56.806 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.806 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.807 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.807 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.807 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.807 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T21:59:56.807 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.808 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.808 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.808 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T21:59:56.808 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.809 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.809 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.809 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.809 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.809 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T21:59:56.810 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.810 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.810 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.810 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T21:59:56.811 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.811 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.811 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.811 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.811 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.812 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T21:59:56.812 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.812 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.812 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:56.812 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T21:59:56.813 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T21:59:56.813 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T21:59:56.813 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T21:59:56.813 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T21:59:56.813 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T21:59:56.814 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T21:59:56.814 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T21:59:56.814 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.814 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T21:59:56.815 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T21:59:56.815 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T21:59:56.815 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T21:59:56.815 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T21:59:56.815 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T21:59:56.816 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/3617788104": "2021-11-22T21:59:50.842249+0000", 2021-11-21T21:59:56.816 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/3617788104": "2021-11-22T21:59:50.842249+0000" 2021-11-21T21:59:56.816 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.816 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T21:59:56.817 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T21:59:56.817 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T21:59:56.817 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T21:59:56.817 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T21:59:56.817 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T21:59:56.818 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T21:59:56.818 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T21:59:56.818 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:56.818 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:56.818 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T21:59:56.819 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T21:59:56.819 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T21:59:56.819 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T21:59:56.819 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T21:59:56.819 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:59:56.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a44053690 msgr2=0x7f6a44055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a44053690 0x7f6a44055b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f6a6001b000 tx=0x7f6a60019000).stop 2021-11-21T21:59:56.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a640651e0 msgr2=0x7f6a64159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a640651e0 0x7f6a64159d20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f6a580076e0 tx=0x7f6a58009040).stop 2021-11-21T21:59:56.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 shutdown_connections 2021-11-21T21:59:56.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a64156190 0x7f6a6415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a44053690 0x7f6a44055b50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a640651e0 0x7f6a64159d20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 --2- 172.21.2.116:0/3522426544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a4c0037a0 0x7f6a6414cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 >> 172.21.2.116:0/3522426544 conn(0x7f6a6405bd20 msgr2=0x7f6a64148fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:56.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 shutdown_connections 2021-11-21T21:59:56.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.614+0000 7f6a6b4aa700 1 -- 172.21.2.116:0/3522426544 wait complete. 2021-11-21T21:59:56.824 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:56.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.874+0000 7f35e3543700 1 Processor -- start 2021-11-21T21:59:56.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.874+0000 7f35e3543700 1 -- start start 2021-11-21T21:59:56.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.874+0000 7f35e3543700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f35dc0a6aa0 0x7f35dc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.874+0000 7f35e3543700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc0a6e80 con 0x7f35dc062410 2021-11-21T21:59:56.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.874+0000 7f35e3543700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc14a7b0 con 0x7f35dc0a86b0 2021-11-21T21:59:56.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.874+0000 7f35e3543700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc0a8bc0 con 0x7f35dc0a6aa0 2021-11-21T21:59:56.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.874+0000 7f35e1ae0700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f35dc062410 0x7f35dc1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60796/0 (socket says 172.21.2.116:60796) 2021-11-21T21:59:56.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e1ae0700 1 -- 172.21.2.116:0/1894462398 learned_addr learned my addr 172.21.2.116:0/1894462398 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:56.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240990870 0 0) 0x7f35dc0a6e80 con 0x7f35dc062410 2021-11-21T21:59:56.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c4003260 con 0x7f35dc062410 2021-11-21T21:59:56.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652032097 0 0) 0x7f35c4003260 con 0x7f35dc062410 2021-11-21T21:59:56.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 >> v2:172.21.2.117:3300/0 conn(0x7f35dc0a6aa0 msgr2=0x7f35dc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 --2- 172.21.2.116:0/1894462398 >> v2:172.21.2.117:3300/0 conn(0x7f35dc0a6aa0 0x7f35dc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 >> v1:172.21.2.117:6789/0 conn(0x7f35dc0a86b0 legacy=0x7f35dc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc14aa10 con 0x7f35dc062410 2021-11-21T21:59:56.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f35d4001a00 con 0x7f35dc062410 2021-11-21T21:59:56.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 >> v1:172.21.2.116:6789/0 conn(0x7f35dc062410 legacy=0x7f35dc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35c4003750 0x7f35c4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35c4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35c4004f90 0x7f35c400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0a8bc0 con 0x7f35c4004500 2021-11-21T21:59:56.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c400be80 con 0x7f35c4004f90 2021-11-21T21:59:56.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c400c040 con 0x7f35c4003750 2021-11-21T21:59:56.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e0ade700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35c4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e0ade700 1 -- 172.21.2.116:0/1894462398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35c4003750 msgr2=0x7f35c4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e0ade700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35c4003750 0x7f35c4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e0ade700 1 -- 172.21.2.116:0/1894462398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35c4004f90 msgr2=0x7f35c400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:56.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e0ade700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35c4004f90 0x7f35c400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e1ae0700 1 -- 172.21.2.116:0/1894462398 reap_dead start 2021-11-21T21:59:56.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e0ade700 1 -- 172.21.2.116:0/1894462398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c4003260 con 0x7f35c4004500 2021-11-21T21:59:56.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e0ade700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35c4004980 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f35d0000f70 tx=0x7f35d00079e0).ready entity=mon.0 client_cookie=a24037dcc328aedb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0001d80 con 0x7f35c4004500 2021-11-21T21:59:56.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d0014410 con 0x7f35c4004500 2021-11-21T21:59:56.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35cffff700 1 -- 172.21.2.116:0/1894462398 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0007cf0 con 0x7f35c4004500 2021-11-21T21:59:56.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- 172.21.2.116:0/1894462398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 msgr2=0x7f35c4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35c4004980 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f35d0000f70 tx=0x7f35d00079e0).stop 2021-11-21T21:59:56.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- 172.21.2.116:0/1894462398 shutdown_connections 2021-11-21T21:59:56.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 --2- 172.21.2.116:0/1894462398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35c4004980 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:56.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- 172.21.2.116:0/1894462398 >> 172.21.2.116:0/1894462398 conn(0x7f35dc05bd20 msgr2=0x7f35dc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:56.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- 172.21.2.116:0/1894462398 shutdown_connections 2021-11-21T21:59:56.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- 172.21.2.116:0/1894462398 wait complete. 2021-11-21T21:59:56.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 Processor -- start 2021-11-21T21:59:56.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- start start 2021-11-21T21:59:56.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35dc14caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc1422e0 0x7f35dc159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc159c40 0x7f35dc14dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc13b0c0 con 0x7f35c4004500 2021-11-21T21:59:56.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc142030 con 0x7f35dc1422e0 2021-11-21T21:59:56.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e3543700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc13b2f0 con 0x7f35dc159c40 2021-11-21T21:59:56.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e12df700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35dc14caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e12df700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35dc14caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48848/0 (socket says 172.21.2.116:48848) 2021-11-21T21:59:56.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.878+0000 7f35e12df700 1 -- 172.21.2.116:0/104562626 learned_addr learned my addr 172.21.2.116:0/104562626 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:56.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e12df700 1 -- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc159c40 msgr2=0x7f35dc14dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e12df700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc159c40 0x7f35dc14dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e12df700 1 -- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc1422e0 msgr2=0x7f35dc159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e12df700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc1422e0 0x7f35dc159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e12df700 1 -- 172.21.2.116:0/104562626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc13b590 con 0x7f35c4004500 2021-11-21T21:59:56.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e12df700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35dc14caf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f35d8002f70 tx=0x7f35d8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35ce7fc700 1 -- 172.21.2.116:0/104562626 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d8013460 con 0x7f35c4004500 2021-11-21T21:59:56.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35ce7fc700 1 -- 172.21.2.116:0/104562626 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d80135c0 con 0x7f35c4004500 2021-11-21T21:59:56.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc13b830 con 0x7f35c4004500 2021-11-21T21:59:56.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc15a0f0 con 0x7f35c4004500 2021-11-21T21:59:56.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35ce7fc700 1 -- 172.21.2.116:0/104562626 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d8013920 con 0x7f35c4004500 2021-11-21T21:59:56.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc14c060 con 0x7f35c4004500 2021-11-21T21:59:56.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35ce7fc700 1 -- 172.21.2.116:0/104562626 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f35d801a060 con 0x7f35c4004500 2021-11-21T21:59:56.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35ce7fc700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053690 0x7f35bc055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:56.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35ce7fc700 1 -- 172.21.2.116:0/104562626 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f35d806b180 con 0x7f35c4004500 2021-11-21T21:59:56.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e0ade700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053690 0x7f35bc055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:56.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.882+0000 7f35e0ade700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053690 0x7f35bc055b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f35d0000f70 tx=0x7f35d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:56.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.886+0000 7f35ce7fc700 1 -- 172.21.2.116:0/104562626 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35d80424b0 con 0x7f35c4004500 2021-11-21T21:59:56.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.990+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35dc14d180 con 0x7f35c4004500 2021-11-21T21:59:56.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35ce7fc700 1 -- 172.21.2.116:0/104562626 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f35d8018030 con 0x7f35c4004500 2021-11-21T21:59:56.996 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:56.997 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/1685183092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1685183092},{"type":"v1","addr":"172.21.2.116:6835","nonce":1685183092}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[]} 2021-11-21T21:59:56.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053690 msgr2=0x7f35bc055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053690 0x7f35bc055b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f35d0000f70 tx=0x7f35d0015000).stop 2021-11-21T21:59:56.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 msgr2=0x7f35dc14caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:56.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35dc14caf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f35d8002f70 tx=0x7f35d8006ec0).stop 2021-11-21T21:59:56.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 shutdown_connections 2021-11-21T21:59:56.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc159c40 0x7f35dc14dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc1422e0 0x7f35dc159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:56.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053690 0x7f35bc055b50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 --2- 172.21.2.116:0/104562626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35c4004500 0x7f35dc14caf0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 >> 172.21.2.116:0/104562626 conn(0x7f35dc05bd20 msgr2=0x7f35dc1441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:57.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 shutdown_connections 2021-11-21T21:59:57.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:56.994+0000 7f35e3543700 1 -- 172.21.2.116:0/104562626 wait complete. 2021-11-21T21:59:57.000 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 30 2021-11-21T21:59:57.003 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T21:59:57.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026fda8700 1 Processor -- start 2021-11-21T21:59:57.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026fda8700 1 -- start start 2021-11-21T21:59:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026fda8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02680a6aa0 0x7f026814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026fda8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f02680a86b0 0x7f02681470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026fda8700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680a6e80 con 0x7f02681475d0 2021-11-21T21:59:57.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026fda8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680a8bc0 con 0x7f02680a86b0 2021-11-21T21:59:57.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026fda8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026814a7b0 con 0x7f02680a6aa0 2021-11-21T21:59:57.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026e345700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f02681475d0 0x7f02681443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60814/0 (socket says 172.21.2.116:60814) 2021-11-21T21:59:57.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026e345700 1 -- 172.21.2.116:0/1766290717 learned_addr learned my addr 172.21.2.116:0/1766290717 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:57.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905474388 0 0) 0x7f02680a6e80 con 0x7f02681475d0 2021-11-21T21:59:57.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0250003260 con 0x7f02681475d0 2021-11-21T21:59:57.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026db44700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02680a6aa0 0x7f026814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026d343700 1 --2- 172.21.2.116:0/1766290717 >> v2:172.21.2.117:3300/0 conn(0x7f02680a86b0 0x7f02681470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1386443347 0 0) 0x7f0250003260 con 0x7f02681475d0 2021-11-21T21:59:57.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02680a6aa0 msgr2=0x7f026814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:57.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02680a6aa0 0x7f026814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 >> v2:172.21.2.117:3300/0 conn(0x7f02680a86b0 msgr2=0x7f02681470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:57.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 --2- 172.21.2.116:0/1766290717 >> v2:172.21.2.117:3300/0 conn(0x7f02680a86b0 0x7f02681470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026814aa10 con 0x7f02681475d0 2021-11-21T21:59:57.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f02600019e0 con 0x7f02681475d0 2021-11-21T21:59:57.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 >> v1:172.21.2.116:6789/0 conn(0x7f02681475d0 legacy=0x7f02681443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0250003750 0x7f0250003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f0250004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02680a6aa0 0x7f0250004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680a8bc0 con 0x7f0250004500 2021-11-21T21:59:57.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026814a7b0 con 0x7f0250003750 2021-11-21T21:59:57.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025000b7a0 con 0x7f02680a6aa0 2021-11-21T21:59:57.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026d343700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f0250004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026d343700 1 -- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02680a6aa0 msgr2=0x7f0250004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:57.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026d343700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02680a6aa0 0x7f0250004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026d343700 1 -- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0250003750 msgr2=0x7f0250003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:57.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026d343700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0250003750 0x7f0250003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.050+0000 7f026d343700 1 -- 172.21.2.116:0/1766290717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250003260 con 0x7f0250004500 2021-11-21T21:59:57.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026d343700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f0250004980 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f026400a000 tx=0x7f0264008040).ready entity=mon.0 client_cookie=7c4b6489023ef257 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:57.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0264013670 con 0x7f0250004500 2021-11-21T21:59:57.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640137d0 con 0x7f0250004500 2021-11-21T21:59:57.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026cb42700 1 -- 172.21.2.116:0/1766290717 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0264013b30 con 0x7f0250004500 2021-11-21T21:59:57.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/1766290717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 msgr2=0x7f0250004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 --2- 172.21.2.116:0/1766290717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f0250004980 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f026400a000 tx=0x7f0264008040).stop 2021-11-21T21:59:57.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026e345700 1 -- 172.21.2.116:0/1766290717 reap_dead start 2021-11-21T21:59:57.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/1766290717 shutdown_connections 2021-11-21T21:59:57.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/1766290717 >> 172.21.2.116:0/1766290717 conn(0x7f026805bd20 msgr2=0x7f026805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:57.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/1766290717 shutdown_connections 2021-11-21T21:59:57.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/1766290717 wait complete. 2021-11-21T21:59:57.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 Processor -- start 2021-11-21T21:59:57.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- start start 2021-11-21T21:59:57.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f02680ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0268144d60 0x7f02680aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02680aabe0 0x7f0268159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026815a610 con 0x7f0250004500 2021-11-21T21:59:57.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680a8dc0 con 0x7f02680aabe0 2021-11-21T21:59:57.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680a9060 con 0x7f0268144d60 2021-11-21T21:59:57.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f02680ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f02680ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48868/0 (socket says 172.21.2.116:48868) 2021-11-21T21:59:57.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 -- 172.21.2.116:0/2237818167 learned_addr learned my addr 172.21.2.116:0/2237818167 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:57.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 -- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0268144d60 msgr2=0x7f02680aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0268144d60 0x7f02680aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 -- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02680aabe0 msgr2=0x7f0268159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02680aabe0 0x7f0268159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 -- 172.21.2.116:0/2237818167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680a9300 con 0x7f0250004500 2021-11-21T21:59:57.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026db44700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f02680ab2e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f025800a000 tx=0x7f0258008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:57.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f0256ffd700 1 -- 172.21.2.116:0/2237818167 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0258016070 con 0x7f0250004500 2021-11-21T21:59:57.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f0256ffd700 1 -- 172.21.2.116:0/2237818167 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0258000e90 con 0x7f0250004500 2021-11-21T21:59:57.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02680aa210 con 0x7f0250004500 2021-11-21T21:59:57.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026815a1b0 con 0x7f0250004500 2021-11-21T21:59:57.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f0256ffd700 1 -- 172.21.2.116:0/2237818167 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0258011a40 con 0x7f0250004500 2021-11-21T21:59:57.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02680573d0 con 0x7f0250004500 2021-11-21T21:59:57.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.054+0000 7f0256ffd700 1 -- 172.21.2.116:0/2237818167 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0258011ba0 con 0x7f0250004500 2021-11-21T21:59:57.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.058+0000 7f0256ffd700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02480534f0 0x7f02480559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.058+0000 7f0256ffd700 1 -- 172.21.2.116:0/2237818167 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f025806b400 con 0x7f0250004500 2021-11-21T21:59:57.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.058+0000 7f026d343700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02480534f0 0x7f02480559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.058+0000 7f026d343700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02480534f0 0x7f02480559b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f026400a000 tx=0x7f0264017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:57.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.058+0000 7f0256ffd700 1 -- 172.21.2.116:0/2237818167 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0258042840 con 0x7f0250004500 2021-11-21T21:59:57.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.166+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f026805db70 con 0x7f0250004500 2021-11-21T21:59:57.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.166+0000 7f0256ffd700 1 -- 172.21.2.116:0/2237818167 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f0258045e60 con 0x7f0250004500 2021-11-21T21:59:57.172 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:57.172 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4681,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/4261224988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":4261224988},{"type":"v1","addr":"172.21.2.117:6835","nonce":4261224988}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1357197064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1357197064},{"type":"v1","addr":"172.21.2.116:6833","nonce":1357197064}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/1685183092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1685183092},{"type":"v1","addr":"172.21.2.116:6835","nonce":1685183092}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[]} 2021-11-21T21:59:57.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02480534f0 msgr2=0x7f02480559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02480534f0 0x7f02480559b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f026400a000 tx=0x7f0264017000).stop 2021-11-21T21:59:57.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 msgr2=0x7f02680ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f02680ab2e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f025800a000 tx=0x7f0258008040).stop 2021-11-21T21:59:57.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 shutdown_connections 2021-11-21T21:59:57.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02680aabe0 0x7f0268159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0268144d60 0x7f02680aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f02480534f0 0x7f02480559b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 --2- 172.21.2.116:0/2237818167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0250004500 0x7f02680ab2e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 >> 172.21.2.116:0/2237818167 conn(0x7f026805bd20 msgr2=0x7f026813f150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:57.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 shutdown_connections 2021-11-21T21:59:57.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.170+0000 7f026fda8700 1 -- 172.21.2.116:0/2237818167 wait complete. 2021-11-21T21:59:57.176 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 30 2021-11-21T21:59:57.179 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T21:59:57.179 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T21:59:57.180 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T21:59:57.180 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:59:57.180 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T21:59:57.181 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T21:59:57.181 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T21:59:57.181 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:59:57.181 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T21:59:57.182 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T21:59:57.182 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T21:59:57.182 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T21:59:57.185 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T21:59:57.186 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T21:59:57.187 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T21:59:57.188 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T21:59:57.189 INFO:tasks.ceph.mds.a:Started 2021-11-21T21:59:57.189 DEBUG:teuthology.parallel:result is None 2021-11-21T21:59:57.190 INFO:tasks.ceph.mds.c:Started 2021-11-21T21:59:57.191 DEBUG:teuthology.parallel:result is None 2021-11-21T21:59:57.193 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T21:59:57.194 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T21:59:57.197 INFO:tasks.ceph.mds.b:Started 2021-11-21T21:59:57.197 DEBUG:teuthology.parallel:result is None 2021-11-21T21:59:57.198 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T21:59:57.221 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T21:59:57.223 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T21:59:57.236 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T21:59:57.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 Processor -- start 2021-11-21T21:59:57.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- start start 2021-11-21T21:59:57.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa4140b0320 0x7fa4140b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa4140aed90 0x7fa4140b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa41405bd20 0x7fa41405c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140ad360 con 0x7fa4140aed90 2021-11-21T21:59:57.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41405d080 con 0x7fa4140b0320 2021-11-21T21:59:57.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140aeb70 con 0x7fa41405bd20 2021-11-21T21:59:57.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa4140aed90 0x7fa4140b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa4140aed90 0x7fa4140b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48900/0 (socket says 172.21.2.116:48900) 2021-11-21T21:59:57.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 -- 172.21.2.116:0/180323330 learned_addr learned my addr 172.21.2.116:0/180323330 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:57.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 -- 172.21.2.116:0/180323330 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa41405bd20 msgr2=0x7fa41405c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:57.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 --2- 172.21.2.116:0/180323330 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa41405bd20 0x7fa41405c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 -- 172.21.2.116:0/180323330 >> v2:172.21.2.117:3300/0 conn(0x7fa4140b0320 msgr2=0x7fa4140b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:57.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 --2- 172.21.2.116:0/180323330 >> v2:172.21.2.117:3300/0 conn(0x7fa4140b0320 0x7fa4140b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 -- 172.21.2.116:0/180323330 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41405cde0 con 0x7fa4140aed90 2021-11-21T21:59:57.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 --2- 172.21.2.116:0/180323330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140aed90 0x7fa4140b15a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fa404002f70 tx=0x7fa404006eb0).ready entity=mon.0 client_cookie=e07d4a1b0126f536 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:57.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa41159a700 1 -- 172.21.2.116:0/180323330 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa40400d570 con 0x7fa4140aed90 2021-11-21T21:59:57.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa41159a700 1 -- 172.21.2.116:0/180323330 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404000e90 con 0x7fa4140aed90 2021-11-21T21:59:57.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa41159a700 1 -- 172.21.2.116:0/180323330 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa40400d900 con 0x7fa4140aed90 2021-11-21T21:59:57.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- 172.21.2.116:0/180323330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140aed90 msgr2=0x7fa4140b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- 172.21.2.116:0/180323330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140aed90 0x7fa4140b15a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fa404002f70 tx=0x7fa404006eb0).stop 2021-11-21T21:59:57.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- 172.21.2.116:0/180323330 shutdown_connections 2021-11-21T21:59:57.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- 172.21.2.116:0/180323330 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa41405bd20 0x7fa41405c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- 172.21.2.116:0/180323330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140aed90 0x7fa4140b15a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- 172.21.2.116:0/180323330 >> v2:172.21.2.117:3300/0 conn(0x7fa4140b0320 0x7fa4140b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- 172.21.2.116:0/180323330 >> 172.21.2.116:0/180323330 conn(0x7fa414063470 msgr2=0x7fa414063850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:57.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- 172.21.2.116:0/180323330 shutdown_connections 2021-11-21T21:59:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- 172.21.2.116:0/180323330 wait complete. 2021-11-21T21:59:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 Processor -- start 2021-11-21T21:59:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- start start 2021-11-21T21:59:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa41405bd20 0x7fa41414bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4140aed90 0x7fa41415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140b0320 0x7fa41415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa414157c60 con 0x7fa4140b0320 2021-11-21T21:59:57.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa414148e60 con 0x7fa41405bd20 2021-11-21T21:59:57.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa418ad8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140a9de0 con 0x7fa4140aed90 2021-11-21T21:59:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140b0320 0x7fa41415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140b0320 0x7fa41415ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48908/0 (socket says 172.21.2.116:48908) 2021-11-21T21:59:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 -- 172.21.2.116:0/2856796141 learned_addr learned my addr 172.21.2.116:0/2856796141 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4140aed90 0x7fa41415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 -- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4140aed90 msgr2=0x7fa41415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4140aed90 0x7fa41415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 -- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa41405bd20 msgr2=0x7fa41414bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa41405bd20 0x7fa41414bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 -- 172.21.2.116:0/2856796141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40400b1e0 con 0x7fa4140b0320 2021-11-21T21:59:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa411d9b700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4140aed90 0x7fa41415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:57.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa412d9d700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140b0320 0x7fa41415ac60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fa414157da0 tx=0x7fa40c009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:57.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa40b7fe700 1 -- 172.21.2.116:0/2856796141 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa40c00d580 con 0x7fa4140b0320 2021-11-21T21:59:57.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.250+0000 7fa40b7fe700 1 -- 172.21.2.116:0/2856796141 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c00d6e0 con 0x7fa4140b0320 2021-11-21T21:59:57.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa414148bc0 con 0x7fa4140b0320 2021-11-21T21:59:57.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41415a2d0 con 0x7fa4140b0320 2021-11-21T21:59:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa40b7fe700 1 -- 172.21.2.116:0/2856796141 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa40c00da40 con 0x7fa4140b0320 2021-11-21T21:59:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4140b2390 con 0x7fa4140b0320 2021-11-21T21:59:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa40b7fe700 1 -- 172.21.2.116:0/2856796141 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa40c00dba0 con 0x7fa4140b0320 2021-11-21T21:59:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa40b7fe700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa3f8053580 0x7fa3f8055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa40b7fe700 1 -- 172.21.2.116:0/2856796141 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa40c06ba00 con 0x7fa4140b0320 2021-11-21T21:59:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa41259c700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa3f8053580 0x7fa3f8055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa41259c700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa3f8053580 0x7fa3f8055a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fa3fc002f70 tx=0x7fa3fc006970).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.254+0000 7fa40b7fe700 1 -- 172.21.2.116:0/2856796141 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa40c042e40 con 0x7fa4140b0320 2021-11-21T21:59:57.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:57.350+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa4140573d0 con 0x7fa4140b0320 2021-11-21T21:59:58.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa40b7fe700 1 -- 172.21.2.116:0/2856796141 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v71) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa40c046460 con 0x7fa4140b0320 2021-11-21T21:59:58.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa3f8053580 msgr2=0x7fa3f8055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa3f8053580 0x7fa3f8055a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fa3fc002f70 tx=0x7fa3fc006970).stop 2021-11-21T21:59:58.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140b0320 msgr2=0x7fa41415ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140b0320 0x7fa41415ac60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fa414157da0 tx=0x7fa40c009220).stop 2021-11-21T21:59:58.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 shutdown_connections 2021-11-21T21:59:58.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa3f8053580 0x7fa3f8055a40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4140b0320 0x7fa41415ac60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4140aed90 0x7fa41415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 --2- 172.21.2.116:0/2856796141 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa41405bd20 0x7fa41414bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 >> 172.21.2.116:0/2856796141 conn(0x7fa414063470 msgr2=0x7fa41414cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 shutdown_connections 2021-11-21T21:59:58.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.230+0000 7fa418ad8700 1 -- 172.21.2.116:0/2856796141 wait complete. 2021-11-21T21:59:58.237 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T21:59:58.240 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T21:59:58.241 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T21:59:58.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 Processor -- start 2021-11-21T21:59:58.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- start start 2021-11-21T21:59:58.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6ce80a6aa0 0x7f6ce814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6ce80a86b0 0x7f6ce81470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce80a6e80 con 0x7f6ce81475d0 2021-11-21T21:59:58.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce80a8bc0 con 0x7f6ce80a86b0 2021-11-21T21:59:58.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce814a7b0 con 0x7f6ce80a6aa0 2021-11-21T21:59:58.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6ce80a86b0 0x7f6ce81470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cecd14700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f6ce81475d0 0x7f6ce81443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60872/0 (socket says 172.21.2.116:60872) 2021-11-21T21:59:58.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cecd14700 1 -- 172.21.2.116:0/3430934212 learned_addr learned my addr 172.21.2.116:0/3430934212 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:58.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce77fe700 1 -- 172.21.2.116:0/3430934212 >> v2:172.21.2.117:3300/0 conn(0x7f6ce80a6aa0 msgr2=0x7f6ce814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce77fe700 1 --2- 172.21.2.116:0/3430934212 >> v2:172.21.2.117:3300/0 conn(0x7f6ce80a6aa0 0x7f6ce814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce77fe700 1 -- 172.21.2.116:0/3430934212 >> v1:172.21.2.116:6789/0 conn(0x7f6ce81475d0 legacy=0x7f6ce81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce77fe700 1 -- 172.21.2.116:0/3430934212 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce814aa10 con 0x7f6ce80a86b0 2021-11-21T21:59:58.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce77fe700 1 --2- 172.21.2.116:0/3430934212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce81470e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f6cd0002f70 tx=0x7f6cd0006df0).ready entity=mon.0 client_cookie=4ce531c382b8c27c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce6ffd700 1 -- 172.21.2.116:0/3430934212 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd000d5c0 con 0x7f6ce80a86b0 2021-11-21T21:59:58.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce6ffd700 1 -- 172.21.2.116:0/3430934212 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd0000e90 con 0x7f6ce80a86b0 2021-11-21T21:59:58.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6ce6ffd700 1 -- 172.21.2.116:0/3430934212 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd000d910 con 0x7f6ce80a86b0 2021-11-21T21:59:58.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- 172.21.2.116:0/3430934212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 msgr2=0x7f6ce81470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 --2- 172.21.2.116:0/3430934212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce81470e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f6cd0002f70 tx=0x7f6cd0006df0).stop 2021-11-21T21:59:58.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- 172.21.2.116:0/3430934212 shutdown_connections 2021-11-21T21:59:58.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 --2- 172.21.2.116:0/3430934212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce81470e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 --2- 172.21.2.116:0/3430934212 >> v2:172.21.2.117:3300/0 conn(0x7f6ce80a6aa0 0x7f6ce814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- 172.21.2.116:0/3430934212 >> 172.21.2.116:0/3430934212 conn(0x7f6ce805bd20 msgr2=0x7f6ce805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- 172.21.2.116:0/3430934212 shutdown_connections 2021-11-21T21:59:58.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- 172.21.2.116:0/3430934212 wait complete. 2021-11-21T21:59:58.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 Processor -- start 2021-11-21T21:59:58.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.334+0000 7f6cee777700 1 -- start start 2021-11-21T21:59:58.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ce80a6aa0 0x7f6ce81452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce813e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ce81475d0 0x7f6ce813e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80a8f20 con 0x7f6ce80a86b0 2021-11-21T21:59:58.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80a9150 con 0x7f6ce80a6aa0 2021-11-21T21:59:58.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80a93f0 con 0x7f6ce81475d0 2021-11-21T21:59:58.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce813e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce813e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48920/0 (socket says 172.21.2.116:48920) 2021-11-21T21:59:58.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 -- 172.21.2.116:0/3734621757 learned_addr learned my addr 172.21.2.116:0/3734621757 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:58.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cecd14700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ce81475d0 0x7f6ce813e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 -- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ce81475d0 msgr2=0x7f6ce813e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ce81475d0 0x7f6ce813e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 -- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ce80a6aa0 msgr2=0x7f6ce81452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ce80a6aa0 0x7f6ce81452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 -- 172.21.2.116:0/3734621757 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd000b180 con 0x7f6ce80a86b0 2021-11-21T21:59:58.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce77fe700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce813e230 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f6cd0002f70 tx=0x7f6cd0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce57fa700 1 -- 172.21.2.116:0/3734621757 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd0006b20 con 0x7f6ce80a86b0 2021-11-21T21:59:58.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce57fa700 1 -- 172.21.2.116:0/3734621757 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd001e400 con 0x7f6ce80a86b0 2021-11-21T21:59:58.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce57fa700 1 -- 172.21.2.116:0/3734621757 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd001e760 con 0x7f6ce80a86b0 2021-11-21T21:59:58.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce80a9690 con 0x7f6ce80a86b0 2021-11-21T21:59:58.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce8145ab0 con 0x7f6ce80a86b0 2021-11-21T21:59:58.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce80573d0 con 0x7f6ce80a86b0 2021-11-21T21:59:58.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce57fa700 1 -- 172.21.2.116:0/3734621757 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6cd001c070 con 0x7f6ce80a86b0 2021-11-21T21:59:58.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce57fa700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ccc0536d0 0x7f6ccc055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce57fa700 1 -- 172.21.2.116:0/3734621757 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6cd006c120 con 0x7f6ce80a86b0 2021-11-21T21:59:58.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.338+0000 7f6ce7fff700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ccc0536d0 0x7f6ccc055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.342+0000 7f6ce7fff700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ccc0536d0 0x7f6ccc055b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f6cd8002f70 tx=0x7f6cd8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.342+0000 7f6ce57fa700 1 -- 172.21.2.116:0/3734621757 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6cd00435b0 con 0x7f6ce80a86b0 2021-11-21T21:59:58.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.450+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6ce805db70 con 0x7f6ce80a86b0 2021-11-21T21:59:58.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.450+0000 7f6ce57fa700 1 -- 172.21.2.116:0/3734621757 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f6cd0023020 con 0x7f6ce80a86b0 2021-11-21T21:59:58.456 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:58.456 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T21:59:58.456 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T21:59:58.456 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.457 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T21:59:58.457 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T21:59:58.457 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.457 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:58.458 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.458 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.458 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.458 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.458 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.459 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.459 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T21:59:58.459 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T21:59:58.459 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.460 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:58.460 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.460 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.460 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.462 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.462 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.463 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T21:59:58.463 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T21:59:58.463 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.463 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:58.464 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.464 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.464 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.464 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.464 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.465 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.465 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T21:59:58.465 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T21:59:58.465 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.466 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.466 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.466 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.466 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.466 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.467 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.467 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T21:59:58.467 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T21:59:58.467 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.468 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.468 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.468 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.468 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.468 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.469 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.469 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T21:59:58.469 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T21:59:58.469 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.470 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.470 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.470 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.470 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.470 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.471 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.471 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T21:59:58.471 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T21:59:58.471 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.471 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.472 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.472 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.472 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.472 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.473 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.473 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T21:59:58.473 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T21:59:58.473 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.473 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.474 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.474 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.474 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.474 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.475 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.475 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T21:59:58.475 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T21:59:58.475 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.475 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.476 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.476 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.476 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.476 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.477 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T21:59:58.477 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T21:59:58.477 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.477 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.478 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.478 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.478 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.478 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.479 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T21:59:58.479 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T21:59:58.479 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.479 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.479 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.480 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.480 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.480 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.480 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.481 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T21:59:58.481 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T21:59:58.481 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.481 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:58.481 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:59:58.482 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:59:58.482 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.482 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.482 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.483 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T21:59:58.483 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T21:59:58.483 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.483 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T21:59:58.483 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T21:59:58.484 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T21:59:58.484 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.484 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.484 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.484 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T21:59:58.485 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T21:59:58.485 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.485 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:59:58.485 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T21:59:58.486 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T21:59:58.486 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.486 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.486 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.486 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.487 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T21:59:58.487 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T21:59:58.487 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.487 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:59:58.488 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:59:58.488 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.488 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.488 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:58.488 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:58.489 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T21:59:58.489 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T21:59:58.489 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T21:59:58.489 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T21:59:58.489 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T21:59:58.490 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T21:59:58.490 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.490 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:58.490 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T21:59:58.491 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T21:59:58.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ccc0536d0 msgr2=0x7f6ccc055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ccc0536d0 0x7f6ccc055b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f6cd8002f70 tx=0x7f6cd8006a80).stop 2021-11-21T21:59:58.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 msgr2=0x7f6ce813e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce813e230 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f6cd0002f70 tx=0x7f6cd0014000).stop 2021-11-21T21:59:58.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 shutdown_connections 2021-11-21T21:59:58.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ce81475d0 0x7f6ce813e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ccc0536d0 0x7f6ccc055b90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ce80a86b0 0x7f6ce813e230 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 --2- 172.21.2.116:0/3734621757 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ce80a6aa0 0x7f6ce81452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 >> 172.21.2.116:0/3734621757 conn(0x7f6ce805bd20 msgr2=0x7f6ce8148610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 shutdown_connections 2021-11-21T21:59:58.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.454+0000 7f6cee777700 1 -- 172.21.2.116:0/3734621757 wait complete. 2021-11-21T21:59:58.495 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T21:59:58.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.542+0000 7f838b6c0700 1 Processor -- start 2021-11-21T21:59:58.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.542+0000 7f838b6c0700 1 -- start start 2021-11-21T21:59:58.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.542+0000 7f838b6c0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f83840aed90 0x7f83840b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 0x7f838413c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83840b00d0 con 0x7f83840b24a0 2021-11-21T21:59:58.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83840adba0 con 0x7f83840aed90 2021-11-21T21:59:58.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b2880 con 0x7f83840af360 2021-11-21T21:59:58.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838945c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f83840b24a0 0x7f83840ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60884/0 (socket says 172.21.2.116:60884) 2021-11-21T21:59:58.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838945c700 1 -- 172.21.2.116:0/895498519 learned_addr learned my addr 172.21.2.116:0/895498519 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:58.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492602367 0 0) 0x7f83840b00d0 con 0x7f83840b24a0 2021-11-21T21:59:58.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f836c003260 con 0x7f83840b24a0 2021-11-21T21:59:58.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8389c5d700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 0x7f838413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 --2- 172.21.2.116:0/895498519 >> v2:172.21.2.117:3300/0 conn(0x7f83840aed90 0x7f83840b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 888312060 0 0) 0x7f836c003260 con 0x7f83840b24a0 2021-11-21T21:59:58.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 msgr2=0x7f838413c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 0x7f838413c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 >> v2:172.21.2.117:3300/0 conn(0x7f83840aed90 msgr2=0x7f83840b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 --2- 172.21.2.116:0/895498519 >> v2:172.21.2.117:3300/0 conn(0x7f83840aed90 0x7f83840b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840b14a0 con 0x7f83840b24a0 2021-11-21T21:59:58.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8380001660 con 0x7f83840b24a0 2021-11-21T21:59:58.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 >> v1:172.21.2.116:6789/0 conn(0x7f83840b24a0 legacy=0x7f83840ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836c003750 0x7f836c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c004500 0x7f836c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 0x7f836c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83840adba0 con 0x7f836c004500 2021-11-21T21:59:58.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b2880 con 0x7f836c003750 2021-11-21T21:59:58.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c00c8b0 con 0x7f83840af360 2021-11-21T21:59:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c004500 0x7f836c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8389c5d700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 0x7f836c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 -- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 msgr2=0x7f836c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 0x7f836c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 -- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836c003750 msgr2=0x7f836c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838945c700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836c003750 0x7f836c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836c003750 0x7f836c003b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 -- 172.21.2.116:0/895498519 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c003260 con 0x7f836c004500 2021-11-21T21:59:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8389c5d700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83840af360 0x7f836c004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8388c5b700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c004500 0x7f836c004980 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f837400a000 tx=0x7f8374008040).ready entity=mon.0 client_cookie=5dd41c4da9cd7f64 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8374013670 con 0x7f836c004500 2021-11-21T21:59:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740137d0 con 0x7f836c004500 2021-11-21T21:59:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8373fff700 1 -- 172.21.2.116:0/895498519 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8374013b30 con 0x7f836c004500 2021-11-21T21:59:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- 172.21.2.116:0/895498519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c004500 msgr2=0x7f836c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 --2- 172.21.2.116:0/895498519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f836c004500 0x7f836c004980 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f837400a000 tx=0x7f8374008040).stop 2021-11-21T21:59:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8389c5d700 1 -- 172.21.2.116:0/895498519 reap_dead start 2021-11-21T21:59:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- 172.21.2.116:0/895498519 shutdown_connections 2021-11-21T21:59:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- 172.21.2.116:0/895498519 >> 172.21.2.116:0/895498519 conn(0x7f838405fd00 msgr2=0x7f83840600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- 172.21.2.116:0/895498519 shutdown_connections 2021-11-21T21:59:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- 172.21.2.116:0/895498519 wait complete. 2021-11-21T21:59:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 Processor -- start 2021-11-21T21:59:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- start start 2021-11-21T21:59:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c004500 0x7f83840662f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83841426a0 0x7f838413b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838413b700 0x7f838410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8384110260 con 0x7f838413b700 2021-11-21T21:59:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8384141c20 con 0x7f83841426a0 2021-11-21T21:59:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f838b6c0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83841431f0 con 0x7f836c004500 2021-11-21T21:59:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8389c5d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838413b700 0x7f838410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8389c5d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838413b700 0x7f838410e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48944/0 (socket says 172.21.2.116:48944) 2021-11-21T21:59:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.546+0000 7f8389c5d700 1 -- 172.21.2.116:0/849749783 learned_addr learned my addr 172.21.2.116:0/849749783 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f8389c5d700 1 -- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c004500 msgr2=0x7f83840662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f8389c5d700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c004500 0x7f83840662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f8389c5d700 1 -- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83841426a0 msgr2=0x7f838413b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f8389c5d700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83841426a0 0x7f838413b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f8389c5d700 1 -- 172.21.2.116:0/849749783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384156cb0 con 0x7f838413b700 2021-11-21T21:59:58.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f8389c5d700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838413b700 0x7f838410e920 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f83841421d0 tx=0x7f837c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f83727fc700 1 -- 172.21.2.116:0/849749783 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c0135f0 con 0x7f838413b700 2021-11-21T21:59:58.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f83727fc700 1 -- 172.21.2.116:0/849749783 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c00c030 con 0x7f838413b700 2021-11-21T21:59:58.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f83727fc700 1 -- 172.21.2.116:0/849749783 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c013940 con 0x7f838413b700 2021-11-21T21:59:58.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8384141e50 con 0x7f838413b700 2021-11-21T21:59:58.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8384153c60 con 0x7f838413b700 2021-11-21T21:59:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8384141fe0 con 0x7f838413b700 2021-11-21T21:59:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f83727fc700 1 -- 172.21.2.116:0/849749783 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f837c013aa0 con 0x7f838413b700 2021-11-21T21:59:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f83727fc700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8364053720 0x7f8364055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f83727fc700 1 -- 172.21.2.116:0/849749783 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f837c06b370 con 0x7f838413b700 2021-11-21T21:59:58.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f838945c700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8364053720 0x7f8364055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.550+0000 7f838945c700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8364053720 0x7f8364055be0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8380008000 tx=0x7f8380009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.554+0000 7f83727fc700 1 -- 172.21.2.116:0/849749783 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f837c0426f0 con 0x7f838413b700 2021-11-21T21:59:58.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.634+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83840573d0 con 0x7f838413b700 2021-11-21T21:59:58.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.634+0000 7f83727fc700 1 -- 172.21.2.116:0/849749783 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+13182 (secure 0 0 0) 0x7f837c017090 con 0x7f838413b700 2021-11-21T21:59:58.639 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:58.640 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":71,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T21:59:58.230259+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T21:59:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.634+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8364053720 msgr2=0x7f8364055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.634+0000 7f838b6c0700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8364053720 0x7f8364055be0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8380008000 tx=0x7f8380009040).stop 2021-11-21T21:59:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.634+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838413b700 msgr2=0x7f838410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.634+0000 7f838b6c0700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838413b700 0x7f838410e920 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f83841421d0 tx=0x7f837c008040).stop 2021-11-21T21:59:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 shutdown_connections 2021-11-21T21:59:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8364053720 0x7f8364055be0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f838413b700 0x7f838410e920 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83841426a0 0x7f838413b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 --2- 172.21.2.116:0/849749783 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836c004500 0x7f83840662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 >> 172.21.2.116:0/849749783 conn(0x7f838405fd00 msgr2=0x7f83840b6090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 shutdown_connections 2021-11-21T21:59:58.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.638+0000 7f838b6c0700 1 -- 172.21.2.116:0/849749783 wait complete. 2021-11-21T21:59:58.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T21:59:58.647 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T21:59:58.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 Processor -- start 2021-11-21T21:59:58.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- start start 2021-11-21T21:59:58.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd148144810 0x7fd1481478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd1480abce0 0x7fd1480b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd148147e80 con 0x7fd1480b1db0 2021-11-21T21:59:58.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1480b1490 con 0x7fd1480abce0 2021-11-21T21:59:58.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1480af060 con 0x7fd148144810 2021-11-21T21:59:58.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd1480abce0 0x7fd1480b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd1480abce0 0x7fd1480b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48950/0 (socket says 172.21.2.116:48950) 2021-11-21T21:59:58.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 -- 172.21.2.116:0/2059141137 learned_addr learned my addr 172.21.2.116:0/2059141137 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:58.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 -- 172.21.2.116:0/2059141137 >> v2:172.21.2.117:3300/0 conn(0x7fd148144810 msgr2=0x7fd1481478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:58.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 --2- 172.21.2.116:0/2059141137 >> v2:172.21.2.117:3300/0 conn(0x7fd148144810 0x7fd1481478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 -- 172.21.2.116:0/2059141137 >> v1:172.21.2.117:6789/0 conn(0x7fd1480b1db0 legacy=0x7fd1480ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T21:59:58.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 -- 172.21.2.116:0/2059141137 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480af5a0 con 0x7fd1480abce0 2021-11-21T21:59:58.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd1467fc700 1 --2- 172.21.2.116:0/2059141137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480b18c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fd1480b4e00 tx=0x7fd138006eb0).ready entity=mon.0 client_cookie=72c4d706e6f07afe server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd145ffb700 1 -- 172.21.2.116:0/2059141137 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd13800d5c0 con 0x7fd1480abce0 2021-11-21T21:59:58.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd145ffb700 1 -- 172.21.2.116:0/2059141137 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd138000e90 con 0x7fd1480abce0 2021-11-21T21:59:58.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd145ffb700 1 -- 172.21.2.116:0/2059141137 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd13800d910 con 0x7fd1480abce0 2021-11-21T21:59:58.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- 172.21.2.116:0/2059141137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 msgr2=0x7fd1480b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 --2- 172.21.2.116:0/2059141137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480b18c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fd1480b4e00 tx=0x7fd138006eb0).stop 2021-11-21T21:59:58.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- 172.21.2.116:0/2059141137 shutdown_connections 2021-11-21T21:59:58.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 --2- 172.21.2.116:0/2059141137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480b18c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 --2- 172.21.2.116:0/2059141137 >> v2:172.21.2.117:3300/0 conn(0x7fd148144810 0x7fd1481478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- 172.21.2.116:0/2059141137 >> 172.21.2.116:0/2059141137 conn(0x7fd14805bd20 msgr2=0x7fd14805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- 172.21.2.116:0/2059141137 shutdown_connections 2021-11-21T21:59:58.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- 172.21.2.116:0/2059141137 wait complete. 2021-11-21T21:59:58.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 Processor -- start 2021-11-21T21:59:58.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.698+0000 7fd14d279700 1 -- start start 2021-11-21T21:59:58.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd148144810 0x7fd1480b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1481456f0 0x7fd14815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480a7ce0 con 0x7fd1480abce0 2021-11-21T21:59:58.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480a7f80 con 0x7fd148144810 2021-11-21T21:59:58.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd148159da0 con 0x7fd1481456f0 2021-11-21T21:59:58.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480a76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48954/0 (socket says 172.21.2.116:48954) 2021-11-21T21:59:58.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 -- 172.21.2.116:0/1480598812 learned_addr learned my addr 172.21.2.116:0/1480598812 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:58.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 -- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1481456f0 msgr2=0x7fd14815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1481456f0 0x7fd14815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 -- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd148144810 msgr2=0x7fd1480b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd148144810 0x7fd1480b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 -- 172.21.2.116:0/1480598812 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13800b1e0 con 0x7fd1480abce0 2021-11-21T21:59:58.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd146ffd700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480a76b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fd130000ed0 tx=0x7fd130006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd13ffff700 1 -- 172.21.2.116:0/1480598812 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd130012410 con 0x7fd1480abce0 2021-11-21T21:59:58.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd13ffff700 1 -- 172.21.2.116:0/1480598812 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd130012570 con 0x7fd1480abce0 2021-11-21T21:59:58.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd13ffff700 1 -- 172.21.2.116:0/1480598812 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1300128d0 con 0x7fd1480abce0 2021-11-21T21:59:58.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1481433e0 con 0x7fd1480abce0 2021-11-21T21:59:58.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1481568d0 con 0x7fd1480abce0 2021-11-21T21:59:58.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1480a8180 con 0x7fd1480abce0 2021-11-21T21:59:58.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd13ffff700 1 -- 172.21.2.116:0/1480598812 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd13000f030 con 0x7fd1480abce0 2021-11-21T21:59:58.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd13ffff700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053660 0x7fd12c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd13ffff700 1 -- 172.21.2.116:0/1480598812 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd13006b310 con 0x7fd1480abce0 2021-11-21T21:59:58.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.702+0000 7fd1467fc700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053660 0x7fd12c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.706+0000 7fd1467fc700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053660 0x7fd12c055b20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fd1480b4e00 tx=0x7fd138015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.706+0000 7fd13ffff700 1 -- 172.21.2.116:0/1480598812 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd130042690 con 0x7fd1480abce0 2021-11-21T21:59:58.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.790+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd148159f60 con 0x7fd1480abce0 2021-11-21T21:59:58.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.790+0000 7fd13ffff700 1 -- 172.21.2.116:0/1480598812 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v71) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd130045cb0 con 0x7fd1480abce0 2021-11-21T21:59:58.796 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T21:59:58.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053660 msgr2=0x7fd12c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053660 0x7fd12c055b20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fd1480b4e00 tx=0x7fd138015000).stop 2021-11-21T21:59:58.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 msgr2=0x7fd1480a76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480a76b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fd130000ed0 tx=0x7fd130006d70).stop 2021-11-21T21:59:58.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 shutdown_connections 2021-11-21T21:59:58.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1481456f0 0x7fd14815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd148144810 0x7fd1480b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053660 0x7fd12c055b20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 --2- 172.21.2.116:0/1480598812 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1480abce0 0x7fd1480a76b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 >> 172.21.2.116:0/1480598812 conn(0x7fd14805bd20 msgr2=0x7fd148148780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 shutdown_connections 2021-11-21T21:59:58.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.794+0000 7fd14d279700 1 -- 172.21.2.116:0/1480598812 wait complete. 2021-11-21T21:59:58.803 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T21:59:58.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 Processor -- start 2021-11-21T21:59:58.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- start start 2021-11-21T21:59:58.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f122805fc40 0x7f12280b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12280b18d0 con 0x7f122805f730 2021-11-21T21:59:58.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1228064260 con 0x7f1228063e80 2021-11-21T21:59:58.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1228146fb0 con 0x7f122805fc40 2021-11-21T21:59:58.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d04b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f122805f730 0x7f12280b0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60918/0 (socket says 172.21.2.116:60918) 2021-11-21T21:59:58.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d04b700 1 -- 172.21.2.116:0/109831626 learned_addr learned my addr 172.21.2.116:0/109831626 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:58.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147470141 0 0) 0x7f12280b18d0 con 0x7f122805f730 2021-11-21T21:59:58.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1210003260 con 0x7f122805f730 2021-11-21T21:59:58.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122e04d700 1 --2- 172.21.2.116:0/109831626 >> v2:172.21.2.117:3300/0 conn(0x7f122805fc40 0x7f12280b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1223228378 0 0) 0x7f1210003260 con 0x7f122805f730 2021-11-21T21:59:58.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 >> v2:172.21.2.117:3300/0 conn(0x7f122805fc40 msgr2=0x7f12280b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 --2- 172.21.2.116:0/109831626 >> v2:172.21.2.117:3300/0 conn(0x7f122805fc40 0x7f12280b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 >> v1:172.21.2.117:6789/0 conn(0x7f1228063e80 legacy=0x7f1228144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228060360 con 0x7f122805f730 2021-11-21T21:59:58.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f1218001660 con 0x7f122805f730 2021-11-21T21:59:58.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 >> v1:172.21.2.116:6789/0 conn(0x7f122805f730 legacy=0x7f12280b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12100037a0 0x7f1210003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1210004500 0x7f1210004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1210004f90 0x7f121000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121000bea0 con 0x7f12100037a0 2021-11-21T21:59:58.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121000c010 con 0x7f1210004500 2021-11-21T21:59:58.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121000c1d0 con 0x7f1210004f90 2021-11-21T21:59:58.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d84c700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12100037a0 0x7f1210003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d84c700 1 -- 172.21.2.116:0/109831626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1210004f90 msgr2=0x7f121000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d84c700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1210004f90 0x7f121000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d84c700 1 -- 172.21.2.116:0/109831626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1210004500 msgr2=0x7f1210004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:58.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d84c700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1210004500 0x7f1210004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d84c700 1 -- 172.21.2.116:0/109831626 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210003260 con 0x7f12100037a0 2021-11-21T21:59:58.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122e04d700 1 -- 172.21.2.116:0/109831626 reap_dead start 2021-11-21T21:59:58.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122d84c700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12100037a0 0x7f1210003be0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f12280b06b0 tx=0x7f1224007a00).ready entity=mon.0 client_cookie=8aea18a5f24a6c05 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1224007d10 con 0x7f12100037a0 2021-11-21T21:59:58.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1224007030 con 0x7f12100037a0 2021-11-21T21:59:58.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122c84a700 1 -- 172.21.2.116:0/109831626 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1224007390 con 0x7f12100037a0 2021-11-21T21:59:58.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- 172.21.2.116:0/109831626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12100037a0 msgr2=0x7f1210003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12100037a0 0x7f1210003be0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f12280b06b0 tx=0x7f1224007a00).stop 2021-11-21T21:59:58.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- 172.21.2.116:0/109831626 shutdown_connections 2021-11-21T21:59:58.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 --2- 172.21.2.116:0/109831626 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12100037a0 0x7f1210003be0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- 172.21.2.116:0/109831626 >> 172.21.2.116:0/109831626 conn(0x7f1228061d40 msgr2=0x7f1228062120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- 172.21.2.116:0/109831626 shutdown_connections 2021-11-21T21:59:58.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- 172.21.2.116:0/109831626 wait complete. 2021-11-21T21:59:58.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 Processor -- start 2021-11-21T21:59:58.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.858+0000 7f122fab0700 1 -- start start 2021-11-21T21:59:58.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12100037a0 0x7f12281486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f122814d2b0 0x7f122814c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f122814b760 0x7f122813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228148df0 con 0x7f122814d2b0 2021-11-21T21:59:58.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122814cba0 con 0x7f12100037a0 2021-11-21T21:59:58.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122814cde0 con 0x7f122814b760 2021-11-21T21:59:58.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f122814d2b0 0x7f122814c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f122814d2b0 0x7f122814c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48978/0 (socket says 172.21.2.116:48978) 2021-11-21T21:59:58.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 -- 172.21.2.116:0/2793168397 learned_addr learned my addr 172.21.2.116:0/2793168397 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:58.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 -- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f122814b760 msgr2=0x7f122813f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f122814b760 0x7f122813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 -- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12100037a0 msgr2=0x7f12281486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12100037a0 0x7f12281486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 -- 172.21.2.116:0/2793168397 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122815a1f0 con 0x7f122814d2b0 2021-11-21T21:59:58.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d04b700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f122814d2b0 0x7f122814c270 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f1218001370 tx=0x7f1228147440).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f1216ffd700 1 -- 172.21.2.116:0/2793168397 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1218007170 con 0x7f122814d2b0 2021-11-21T21:59:58.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f1216ffd700 1 -- 172.21.2.116:0/2793168397 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12180072d0 con 0x7f122814d2b0 2021-11-21T21:59:58.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122815a490 con 0x7f122814d2b0 2021-11-21T21:59:58.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1228067a80 con 0x7f122814d2b0 2021-11-21T21:59:58.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f1216ffd700 1 -- 172.21.2.116:0/2793168397 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1218004920 con 0x7f122814d2b0 2021-11-21T21:59:58.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12281471a0 con 0x7f122814d2b0 2021-11-21T21:59:58.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f1216ffd700 1 -- 172.21.2.116:0/2793168397 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f121801b070 con 0x7f122814d2b0 2021-11-21T21:59:58.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f1216ffd700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12080535d0 0x7f1208055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:58.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f1216ffd700 1 -- 172.21.2.116:0/2793168397 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f121806aea0 con 0x7f122814d2b0 2021-11-21T21:59:58.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d84c700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12080535d0 0x7f1208055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:58.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.862+0000 7f122d84c700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12080535d0 0x7f1208055a90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f12280b06b0 tx=0x7f1224013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:58.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.866+0000 7f1216ffd700 1 -- 172.21.2.116:0/2793168397 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1218042220 con 0x7f122814d2b0 2021-11-21T21:59:58.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.946+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f12280573d0 con 0x7f122814d2b0 2021-11-21T21:59:58.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f1216ffd700 1 -- 172.21.2.116:0/2793168397 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1218045840 con 0x7f122814d2b0 2021-11-21T21:59:58.952 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T21:59:58.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12080535d0 msgr2=0x7f1208055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12080535d0 0x7f1208055a90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f12280b06b0 tx=0x7f1224013000).stop 2021-11-21T21:59:58.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f122814d2b0 msgr2=0x7f122814c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:58.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f122814d2b0 0x7f122814c270 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f1218001370 tx=0x7f1228147440).stop 2021-11-21T21:59:58.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 shutdown_connections 2021-11-21T21:59:58.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f122814b760 0x7f122813f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12080535d0 0x7f1208055a90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f122814d2b0 0x7f122814c270 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:58.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 --2- 172.21.2.116:0/2793168397 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12100037a0 0x7f12281486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:58.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 >> 172.21.2.116:0/2793168397 conn(0x7f1228061d40 msgr2=0x7f12280af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:58.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 shutdown_connections 2021-11-21T21:59:58.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:58.950+0000 7f122fab0700 1 -- 172.21.2.116:0/2793168397 wait complete. 2021-11-21T21:59:58.959 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T21:59:59.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79df048700 1 Processor -- start 2021-11-21T21:59:59.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79df048700 1 -- start start 2021-11-21T21:59:59.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79df048700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f79d80ad780 0x7f79d805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79df048700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d8143f70 con 0x7f79d80affc0 2021-11-21T21:59:59.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79df048700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80adb60 con 0x7f79d80af7e0 2021-11-21T21:59:59.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79df048700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d8143160 con 0x7f79d80ad780 2021-11-21T21:59:59.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79d7fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f79d80affc0 0x7f79d8143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60942/0 (socket says 172.21.2.116:60942) 2021-11-21T21:59:59.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79d7fff700 1 -- 172.21.2.116:0/2956261591 learned_addr learned my addr 172.21.2.116:0/2956261591 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T21:59:59.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219178440 0 0) 0x7f79d8143f70 con 0x7f79d80affc0 2021-11-21T21:59:59.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.006+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c4003260 con 0x7f79d80affc0 2021-11-21T21:59:59.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63418359 0 0) 0x7f79c4003260 con 0x7f79d80affc0 2021-11-21T21:59:59.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 >> v2:172.21.2.117:3300/0 conn(0x7f79d80ad780 msgr2=0x7f79d805f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:59.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 --2- 172.21.2.116:0/2956261591 >> v2:172.21.2.117:3300/0 conn(0x7f79d80ad780 0x7f79d805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:59.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 >> v1:172.21.2.117:6789/0 conn(0x7f79d80af7e0 legacy=0x7f79d813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:59.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80b0ce0 con 0x7f79d80affc0 2021-11-21T21:59:59.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f79cc001660 con 0x7f79d80affc0 2021-11-21T21:59:59.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 >> v1:172.21.2.116:6789/0 conn(0x7f79d80affc0 legacy=0x7f79d8143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79c40037d0 0x7f79c4003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79c4004500 0x7f79c4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79c4005070 0x7f79c400d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8143160 con 0x7f79c4005070 2021-11-21T21:59:59.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4005750 con 0x7f79c40037d0 2021-11-21T21:59:59.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4005910 con 0x7f79c4004500 2021-11-21T21:59:59.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79c4005070 0x7f79c400d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d7fff700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79c4004500 0x7f79c4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/2956261591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79c4004500 msgr2=0x7f79c4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:59.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79c4004500 0x7f79c4004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/2956261591 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79c40037d0 msgr2=0x7f79c4003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:59.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79c40037d0 0x7f79c4003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:59.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/2956261591 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80b0ce0 con 0x7f79c4005070 2021-11-21T21:59:59.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/2956261591 reap_dead start 2021-11-21T21:59:59.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79c4005070 0x7f79c400d960 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f79d00012f0 tx=0x7f79d0007d60).ready entity=mon.0 client_cookie=896bb3b7561af066 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:59.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d00014c0 con 0x7f79c4005070 2021-11-21T21:59:59.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0006870 con 0x7f79c4005070 2021-11-21T21:59:59.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d77fe700 1 -- 172.21.2.116:0/2956261591 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d0006bd0 con 0x7f79c4005070 2021-11-21T21:59:59.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/2956261591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79c4005070 msgr2=0x7f79c400d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79c4005070 0x7f79c400d960 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f79d00012f0 tx=0x7f79d0007d60).stop 2021-11-21T21:59:59.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/2956261591 shutdown_connections 2021-11-21T21:59:59.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 --2- 172.21.2.116:0/2956261591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79c4005070 0x7f79c400d960 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/2956261591 >> 172.21.2.116:0/2956261591 conn(0x7f79d80b6000 msgr2=0x7f79d80b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:59.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/2956261591 shutdown_connections 2021-11-21T21:59:59.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/2956261591 wait complete. 2021-11-21T21:59:59.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 Processor -- start 2021-11-21T21:59:59.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- start start 2021-11-21T21:59:59.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79c4005070 0x7f79d805c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79d81511f0 0x7f79d8151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79d814dcd0 0x7f79d814e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8150dc0 con 0x7f79d814dcd0 2021-11-21T21:59:59.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8142de0 con 0x7f79c4005070 2021-11-21T21:59:59.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8142000 con 0x7f79d81511f0 2021-11-21T21:59:59.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79d814dcd0 0x7f79d814e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79d814dcd0 0x7f79d814e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:48998/0 (socket says 172.21.2.116:48998) 2021-11-21T21:59:59.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/3063867773 learned_addr learned my addr 172.21.2.116:0/3063867773 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:59.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79d81511f0 msgr2=0x7f79d8151670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:59.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79d81511f0 0x7f79d8151670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:59.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79c4005070 msgr2=0x7f79d805c310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T21:59:59.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79c4005070 0x7f79d805c310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:59.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 -- 172.21.2.116:0/3063867773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d81422a0 con 0x7f79d814dcd0 2021-11-21T21:59:59.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79dd5e5700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79d814dcd0 0x7f79d814e150 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f79d00012f0 tx=0x7f79d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:59.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d5ffb700 1 -- 172.21.2.116:0/3063867773 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d0014670 con 0x7f79d814dcd0 2021-11-21T21:59:59.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d5ffb700 1 -- 172.21.2.116:0/3063867773 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d00147d0 con 0x7f79d814dcd0 2021-11-21T21:59:59.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8142540 con 0x7f79d814dcd0 2021-11-21T21:59:59.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d8151b60 con 0x7f79d814dcd0 2021-11-21T21:59:59.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79d5ffb700 1 -- 172.21.2.116:0/3063867773 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d0014b30 con 0x7f79d814dcd0 2021-11-21T21:59:59.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.010+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d80acdf0 con 0x7f79d814dcd0 2021-11-21T21:59:59.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.014+0000 7f79d5ffb700 1 -- 172.21.2.116:0/3063867773 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f79d0010070 con 0x7f79d814dcd0 2021-11-21T21:59:59.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.014+0000 7f79d5ffb700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79b8053660 0x7f79b8055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.014+0000 7f79d5ffb700 1 -- 172.21.2.116:0/3063867773 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f79d006cf80 con 0x7f79d814dcd0 2021-11-21T21:59:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.014+0000 7f79dcde4700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79b8053660 0x7f79b8055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.014+0000 7f79dcde4700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79b8053660 0x7f79b8055b20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f79c8002f70 tx=0x7f79c8006980).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.014+0000 7f79d5ffb700 1 -- 172.21.2.116:0/3063867773 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79d00443b0 con 0x7f79d814dcd0 2021-11-21T21:59:59.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.098+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f79d80acdf0 con 0x7f79d814dcd0 2021-11-21T21:59:59.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.098+0000 7f79d5ffb700 1 -- 172.21.2.116:0/3063867773 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) v1 ==== 84+0+143 (secure 0 0 0) 0x7f79d00479d0 con 0x7f79d814dcd0 2021-11-21T21:59:59.103 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T21:59:59.104 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T21:59:59.104 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:59.104 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T21:59:59.104 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T21:59:59.105 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T21:59:59.105 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T21:59:59.105 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T21:59:59.105 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T21:59:59.106 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T21:59:59.106 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T21:59:59.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79b8053660 msgr2=0x7f79b8055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79b8053660 0x7f79b8055b20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f79c8002f70 tx=0x7f79c8006980).stop 2021-11-21T21:59:59.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79d814dcd0 msgr2=0x7f79d814e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79d814dcd0 0x7f79d814e150 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f79d00012f0 tx=0x7f79d0016000).stop 2021-11-21T21:59:59.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 shutdown_connections 2021-11-21T21:59:59.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f79b8053660 0x7f79b8055b20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79d814dcd0 0x7f79d814e150 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79d81511f0 0x7f79d8151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:59.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 --2- 172.21.2.116:0/3063867773 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79c4005070 0x7f79d805c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T21:59:59.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 >> 172.21.2.116:0/3063867773 conn(0x7f79d80b6000 msgr2=0x7f79d80b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:59.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 shutdown_connections 2021-11-21T21:59:59.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.102+0000 7f79df048700 1 -- 172.21.2.116:0/3063867773 wait complete. 2021-11-21T21:59:59.111 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T21:59:59.111 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T21:59:59.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f283b6a9700 1 Processor -- start 2021-11-21T21:59:59.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f283b6a9700 1 -- start start 2021-11-21T21:59:59.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f283b6a9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2834144810 0x7f28341478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f283b6a9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f28340aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f283b6a9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2834147e80 con 0x7f28340abce0 2021-11-21T21:59:59.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f283b6a9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28340b1490 con 0x7f2834144810 2021-11-21T21:59:59.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f283b6a9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28340b1fd0 con 0x7f28340ac600 2021-11-21T21:59:59.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839c46700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f28340aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839c46700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f28340aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:52348/0 (socket says 172.21.2.116:52348) 2021-11-21T21:59:59.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839c46700 1 -- 172.21.2.116:0/1604496025 learned_addr learned my addr 172.21.2.116:0/1604496025 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:59.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839445700 1 --2- 172.21.2.116:0/1604496025 >> v2:172.21.2.117:3300/0 conn(0x7f2834144810 0x7f28341478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839445700 1 -- 172.21.2.116:0/1604496025 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 msgr2=0x7f28340aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T21:59:59.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839445700 1 --2- 172.21.2.116:0/1604496025 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f28340aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839445700 1 -- 172.21.2.116:0/1604496025 >> v1:172.21.2.117:6789/0 conn(0x7f28340abce0 legacy=0x7f28340b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839445700 1 -- 172.21.2.116:0/1604496025 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340af5a0 con 0x7f2834144810 2021-11-21T21:59:59.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.162+0000 7f2839445700 1 --2- 172.21.2.116:0/1604496025 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2834144810 0x7f28341478c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f2830002f70 tx=0x7f2830006df0).ready entity=mon.1 client_cookie=8c8313ec3117e1e4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:59.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2823fff700 1 -- 172.21.2.116:0/1604496025 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f283000d570 con 0x7f2834144810 2021-11-21T21:59:59.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2823fff700 1 -- 172.21.2.116:0/1604496025 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2830000e90 con 0x7f2834144810 2021-11-21T21:59:59.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2823fff700 1 -- 172.21.2.116:0/1604496025 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f283000d8c0 con 0x7f2834144810 2021-11-21T21:59:59.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/1604496025 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2834144810 msgr2=0x7f28341478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 --2- 172.21.2.116:0/1604496025 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2834144810 0x7f28341478c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f2830002f70 tx=0x7f2830006df0).stop 2021-11-21T21:59:59.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/1604496025 shutdown_connections 2021-11-21T21:59:59.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 --2- 172.21.2.116:0/1604496025 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f28340aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 --2- 172.21.2.116:0/1604496025 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2834144810 0x7f28341478c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/1604496025 >> 172.21.2.116:0/1604496025 conn(0x7f283405bd20 msgr2=0x7f283405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T21:59:59.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/1604496025 shutdown_connections 2021-11-21T21:59:59.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/1604496025 wait complete. 2021-11-21T21:59:59.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 Processor -- start 2021-11-21T21:59:59.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- start start 2021-11-21T21:59:59.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28340abce0 0x7f2834146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f2834066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2834144810 0x7f28340671d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28340b4c20 con 0x7f2834144810 2021-11-21T21:59:59.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28340676c0 con 0x7f28340abce0 2021-11-21T21:59:59.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28340b46e0 con 0x7f28340ac600 2021-11-21T21:59:59.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2834144810 0x7f28340671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2834144810 0x7f28340671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49012/0 (socket says 172.21.2.116:49012) 2021-11-21T21:59:59.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 -- 172.21.2.116:0/756559738 learned_addr learned my addr 172.21.2.116:0/756559738 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T21:59:59.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839445700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28340abce0 0x7f2834146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2838c44700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f2834066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 -- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 msgr2=0x7f2834066ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f2834066ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 -- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28340abce0 msgr2=0x7f2834146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T21:59:59.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28340abce0 0x7f2834146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T21:59:59.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 -- 172.21.2.116:0/756559738 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283000b180 con 0x7f2834144810 2021-11-21T21:59:59.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839445700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28340abce0 0x7f2834146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:59.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2838c44700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f2834066ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T21:59:59.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f2839c46700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2834144810 0x7f28340671d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f282c00c000 tx=0x7f282c009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:59.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f28227fc700 1 -- 172.21.2.116:0/756559738 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f282c00c7f0 con 0x7f2834144810 2021-11-21T21:59:59.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f28227fc700 1 -- 172.21.2.116:0/756559738 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c00c950 con 0x7f2834144810 2021-11-21T21:59:59.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28340b4980 con 0x7f2834144810 2021-11-21T21:59:59.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28340b2e80 con 0x7f2834144810 2021-11-21T21:59:59.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f28227fc700 1 -- 172.21.2.116:0/756559738 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f282c00ccb0 con 0x7f2834144810 2021-11-21T21:59:59.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.166+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2834148ed0 con 0x7f2834144810 2021-11-21T21:59:59.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.170+0000 7f28227fc700 1 -- 172.21.2.116:0/756559738 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f282c022070 con 0x7f2834144810 2021-11-21T21:59:59.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.170+0000 7f28227fc700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28140536a0 0x7f2814055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T21:59:59.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.170+0000 7f28227fc700 1 -- 172.21.2.116:0/756559738 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f282c06be40 con 0x7f2834144810 2021-11-21T21:59:59.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.170+0000 7f2839445700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28140536a0 0x7f2814055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T21:59:59.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.170+0000 7f2839445700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28140536a0 0x7f2814055b60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f2830015000 tx=0x7f2830014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T21:59:59.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.170+0000 7f28227fc700 1 -- 172.21.2.116:0/756559738 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f282c0432d0 con 0x7f2834144810 2021-11-21T21:59:59.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T21:59:59.254+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f28340af060 con 0x7f2834144810 2021-11-21T22:00:00.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f28227fc700 1 -- 172.21.2.116:0/756559738 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f282c016070 con 0x7f2834144810 2021-11-21T22:00:00.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28140536a0 msgr2=0x7f2814055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:00.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28140536a0 0x7f2814055b60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f2830015000 tx=0x7f2830014000).stop 2021-11-21T22:00:00.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2834144810 msgr2=0x7f28340671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:00.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2834144810 0x7f28340671d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f282c00c000 tx=0x7f282c009290).stop 2021-11-21T22:00:00.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 shutdown_connections 2021-11-21T22:00:00.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f28140536a0 0x7f2814055b60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:00.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2834144810 0x7f28340671d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:00.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28340ac600 0x7f2834066ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:00.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 --2- 172.21.2.116:0/756559738 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28340abce0 0x7f2834146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:00.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 >> 172.21.2.116:0/756559738 conn(0x7f283405bd20 msgr2=0x7f28340af760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:00.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 shutdown_connections 2021-11-21T22:00:00.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.246+0000 7f283b6a9700 1 -- 172.21.2.116:0/756559738 wait complete. 2021-11-21T22:00:00.252 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:00:00.256 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:00:00.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 Processor -- start 2021-11-21T22:00:00.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- start start 2021-11-21T22:00:00.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7cfc0b5490 0x7f7cfc0b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:00.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc097ec0 0x7f7cfc14a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:00.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0995c0 con 0x7f7cfc0af720 2021-11-21T22:00:00.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc144b40 con 0x7f7cfc0b5490 2021-11-21T22:00:00.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc14b140 con 0x7f7cfc097ec0 2021-11-21T22:00:00.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7cfc0b5490 0x7f7cfc0b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:00.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7cfc0b5490 0x7f7cfc0b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49016/0 (socket says 172.21.2.116:49016) 2021-11-21T22:00:00.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/264922402 learned_addr learned my addr 172.21.2.116:0/264922402 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:00.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/264922402 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc097ec0 msgr2=0x7f7cfc14a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:00.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 --2- 172.21.2.116:0/264922402 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc097ec0 0x7f7cfc14a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:00.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/264922402 >> v1:172.21.2.117:6789/0 conn(0x7f7cfc0af720 legacy=0x7f7cfc146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:00.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/264922402 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0997f0 con 0x7f7cfc0b5490 2021-11-21T22:00:00.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfb7fe700 1 --2- 172.21.2.116:0/264922402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc0b5490 0x7f7cfc0b5870 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f7cfc062380 tx=0x7f7cec006eb0).ready entity=mon.0 client_cookie=759126e9c2e0be7b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:00.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfa7fc700 1 -- 172.21.2.116:0/264922402 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cec00d570 con 0x7f7cfc0b5490 2021-11-21T22:00:00.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfa7fc700 1 -- 172.21.2.116:0/264922402 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec000e90 con 0x7f7cfc0b5490 2021-11-21T22:00:00.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7cfa7fc700 1 -- 172.21.2.116:0/264922402 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cec00d8c0 con 0x7f7cfc0b5490 2021-11-21T22:00:00.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- 172.21.2.116:0/264922402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc0b5490 msgr2=0x7f7cfc0b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:00.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 --2- 172.21.2.116:0/264922402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc0b5490 0x7f7cfc0b5870 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f7cfc062380 tx=0x7f7cec006eb0).stop 2021-11-21T22:00:00.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- 172.21.2.116:0/264922402 shutdown_connections 2021-11-21T22:00:00.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 --2- 172.21.2.116:0/264922402 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc097ec0 0x7f7cfc14a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:00.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 --2- 172.21.2.116:0/264922402 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc0b5490 0x7f7cfc0b5870 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:00.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- 172.21.2.116:0/264922402 >> 172.21.2.116:0/264922402 conn(0x7f7cfc05c7a0 msgr2=0x7f7cfc0b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:00.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- 172.21.2.116:0/264922402 shutdown_connections 2021-11-21T22:00:00.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.306+0000 7f7d00b37700 1 -- 172.21.2.116:0/264922402 wait complete. 2021-11-21T22:00:00.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 Processor -- start 2021-11-21T22:00:00.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 -- start start 2021-11-21T22:00:00.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc097ec0 0x7f7cfc065940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:00.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc0af720 0x7f7cfc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:00.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7cfc0b5490 0x7f7cfc0ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:00.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0aee60 con 0x7f7cfc097ec0 2021-11-21T22:00:00.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc066300 con 0x7f7cfc0b5490 2021-11-21T22:00:00.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc157040 con 0x7f7cfc0af720 2021-11-21T22:00:00.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc097ec0 0x7f7cfc065940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:00.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc097ec0 0x7f7cfc065940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49022/0 (socket says 172.21.2.116:49022) 2021-11-21T22:00:00.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/304653861 learned_addr learned my addr 172.21.2.116:0/304653861 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:00.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfaffd700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc0af720 0x7f7cfc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:00.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfbfff700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7cfc0b5490 0x7f7cfc0ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:00.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc0af720 msgr2=0x7f7cfc0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:00.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc0af720 0x7f7cfc0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:00.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7cfc0b5490 msgr2=0x7f7cfc0ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:00.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7cfc0b5490 0x7f7cfc0ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:00.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 -- 172.21.2.116:0/304653861 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec00b1e0 con 0x7f7cfc097ec0 2021-11-21T22:00:00.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfb7fe700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc097ec0 0x7f7cfc065940 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f7cfc062380 tx=0x7f7cec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:00.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cf8ff9700 1 -- 172.21.2.116:0/304653861 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cec006be0 con 0x7f7cfc097ec0 2021-11-21T22:00:00.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cf8ff9700 1 -- 172.21.2.116:0/304653861 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec002f70 con 0x7f7cfc097ec0 2021-11-21T22:00:00.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc0b0fc0 con 0x7f7cfc097ec0 2021-11-21T22:00:00.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cf8ff9700 1 -- 172.21.2.116:0/304653861 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cec00de20 con 0x7f7cfc097ec0 2021-11-21T22:00:00.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc062a20 con 0x7f7cfc097ec0 2021-11-21T22:00:00.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc099a90 con 0x7f7cfc097ec0 2021-11-21T22:00:00.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cf8ff9700 1 -- 172.21.2.116:0/304653861 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7cec025030 con 0x7f7cfc097ec0 2021-11-21T22:00:00.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cf8ff9700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7cdc0535b0 0x7f7cdc055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:00.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cf8ff9700 1 -- 172.21.2.116:0/304653861 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f7cec06cb00 con 0x7f7cfc097ec0 2021-11-21T22:00:00.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.310+0000 7f7cfaffd700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7cdc0535b0 0x7f7cdc055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:00.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.314+0000 7f7cfaffd700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7cdc0535b0 0x7f7cdc055a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f7cf000b000 tx=0x7f7cf0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:00.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.314+0000 7f7cf8ff9700 1 -- 172.21.2.116:0/304653861 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7cec043e30 con 0x7f7cfc097ec0 2021-11-21T22:00:00.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:00.398+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f7cfc0573d0 con 0x7f7cfc097ec0 2021-11-21T22:00:01.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.238+0000 7f7cf8ff9700 1 -- 172.21.2.116:0/304653861 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v73) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7cec023020 con 0x7f7cfc097ec0 2021-11-21T22:00:01.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7cdc0535b0 msgr2=0x7f7cdc055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7cdc0535b0 0x7f7cdc055a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f7cf000b000 tx=0x7f7cf0009040).stop 2021-11-21T22:00:01.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc097ec0 msgr2=0x7f7cfc065940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc097ec0 0x7f7cfc065940 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f7cfc062380 tx=0x7f7cec015000).stop 2021-11-21T22:00:01.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 shutdown_connections 2021-11-21T22:00:01.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7cfc0b5490 0x7f7cfc0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7cfc0af720 0x7f7cfc0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7cdc0535b0 0x7f7cdc055a70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 --2- 172.21.2.116:0/304653861 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7cfc097ec0 0x7f7cfc065940 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 >> 172.21.2.116:0/304653861 conn(0x7f7cfc05c7a0 msgr2=0x7f7cfc14a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 shutdown_connections 2021-11-21T22:00:01.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.242+0000 7f7d00b37700 1 -- 172.21.2.116:0/304653861 wait complete. 2021-11-21T22:00:01.247 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:00:01.251 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:00:01.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.298+0000 7f78b11f4700 1 Processor -- start 2021-11-21T22:00:01.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.298+0000 7f78b11f4700 1 -- start start 2021-11-21T22:00:01.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f78ac0a8670 0x7f78ac147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0b1940 con 0x7f78ac0b11c0 2021-11-21T22:00:01.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0b15a0 con 0x7f78ac143e60 2021-11-21T22:00:01.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac0b1b70 con 0x7f78ac0a8670 2021-11-21T22:00:01.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aad9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f78ac0b11c0 0x7f78ac147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:60986/0 (socket says 172.21.2.116:60986) 2021-11-21T22:00:01.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aad9d700 1 -- 172.21.2.116:0/2941355333 learned_addr learned my addr 172.21.2.116:0/2941355333 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:01.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297541448 0 0) 0x7f78ac0b1940 con 0x7f78ac0b11c0 2021-11-21T22:00:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78a259c700 1 --2- 172.21.2.116:0/2941355333 >> v2:172.21.2.117:3300/0 conn(0x7f78ac0a8670 0x7f78ac147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f788c003260 con 0x7f78ac0b11c0 2021-11-21T22:00:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3215318254 0 0) 0x7f788c003260 con 0x7f78ac0b11c0 2021-11-21T22:00:01.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 >> v2:172.21.2.117:3300/0 conn(0x7f78ac0a8670 msgr2=0x7f78ac147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:01.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 --2- 172.21.2.116:0/2941355333 >> v2:172.21.2.117:3300/0 conn(0x7f78ac0a8670 0x7f78ac147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 >> v1:172.21.2.117:6789/0 conn(0x7f78ac143e60 legacy=0x7f78ac1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0b1e10 con 0x7f78ac0b11c0 2021-11-21T22:00:01.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f789c001660 con 0x7f78ac0b11c0 2021-11-21T22:00:01.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 >> v1:172.21.2.116:6789/0 conn(0x7f78ac0b11c0 legacy=0x7f78ac147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f788c0037a0 0x7f788c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004500 0x7f788c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f788c004e70 0x7f788c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0b1b70 con 0x7f788c004e70 2021-11-21T22:00:01.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c00be50 con 0x7f788c004500 2021-11-21T22:00:01.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c00c010 con 0x7f788c0037a0 2021-11-21T22:00:01.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f788c004e70 0x7f788c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aad9d700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f788c0037a0 0x7f788c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78a259c700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004500 0x7f788c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 -- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f788c0037a0 msgr2=0x7f788c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:01.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f788c0037a0 0x7f788c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 -- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004500 msgr2=0x7f788c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:01.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004500 0x7f788c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 -- 172.21.2.116:0/2941355333 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c003260 con 0x7f788c004e70 2021-11-21T22:00:01.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 -- 172.21.2.116:0/2941355333 reap_dead start 2021-11-21T22:00:01.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78ab59e700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f788c004e70 0x7f788c00b750 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f78ac068600 tx=0x7f78a4007d80).ready entity=mon.0 client_cookie=289a2d20a1a959d7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a4001620 con 0x7f788c004e70 2021-11-21T22:00:01.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a4006890 con 0x7f788c004e70 2021-11-21T22:00:01.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78aa59c700 1 -- 172.21.2.116:0/2941355333 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a4006bf0 con 0x7f788c004e70 2021-11-21T22:00:01.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- 172.21.2.116:0/2941355333 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f788c004e70 msgr2=0x7f788c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f788c004e70 0x7f788c00b750 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f78ac068600 tx=0x7f78a4007d80).stop 2021-11-21T22:00:01.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- 172.21.2.116:0/2941355333 shutdown_connections 2021-11-21T22:00:01.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 --2- 172.21.2.116:0/2941355333 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f788c004e70 0x7f788c00b750 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- 172.21.2.116:0/2941355333 >> 172.21.2.116:0/2941355333 conn(0x7f78ac05bd20 msgr2=0x7f78ac05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- 172.21.2.116:0/2941355333 shutdown_connections 2021-11-21T22:00:01.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- 172.21.2.116:0/2941355333 wait complete. 2021-11-21T22:00:01.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 Processor -- start 2021-11-21T22:00:01.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.302+0000 7f78b11f4700 1 -- start start 2021-11-21T22:00:01.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004e70 0x7f78ac15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f78ac140860 0x7f78ac066f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f78ac140cc0 0x7f78ac14a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac13f490 con 0x7f78ac140cc0 2021-11-21T22:00:01.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac15a5c0 con 0x7f788c004e70 2021-11-21T22:00:01.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0b2e50 con 0x7f78ac140860 2021-11-21T22:00:01.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f78ac140cc0 0x7f78ac14a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f78ac140cc0 0x7f78ac14a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49044/0 (socket says 172.21.2.116:49044) 2021-11-21T22:00:01.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 -- 172.21.2.116:0/1653316968 learned_addr learned my addr 172.21.2.116:0/1653316968 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:01.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78aad9d700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004e70 0x7f78ac15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a259c700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f78ac140860 0x7f78ac066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 -- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f78ac140860 msgr2=0x7f78ac066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f78ac140860 0x7f78ac066f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 -- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004e70 msgr2=0x7f78ac15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004e70 0x7f78ac15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 -- 172.21.2.116:0/1653316968 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0b30f0 con 0x7f78ac140cc0 2021-11-21T22:00:01.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78aad9d700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004e70 0x7f78ac15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:00:01.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a259c700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f78ac140860 0x7f78ac066f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:00:01.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78ab59e700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f78ac140cc0 0x7f78ac14a3e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f78ac068600 tx=0x7f78a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a8d99700 1 -- 172.21.2.116:0/1653316968 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a400c860 con 0x7f78ac140cc0 2021-11-21T22:00:01.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a8d99700 1 -- 172.21.2.116:0/1653316968 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a400c9c0 con 0x7f78ac140cc0 2021-11-21T22:00:01.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a8d99700 1 -- 172.21.2.116:0/1653316968 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a400cd20 con 0x7f78ac140cc0 2021-11-21T22:00:01.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac0b3390 con 0x7f78ac140cc0 2021-11-21T22:00:01.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac14b2b0 con 0x7f78ac140cc0 2021-11-21T22:00:01.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac0b20b0 con 0x7f78ac140cc0 2021-11-21T22:00:01.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a8d99700 1 -- 172.21.2.116:0/1653316968 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f78a401b030 con 0x7f78ac140cc0 2021-11-21T22:00:01.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a8d99700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f78900536a0 0x7f7890055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78a8d99700 1 -- 172.21.2.116:0/1653316968 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f78a406c8e0 con 0x7f78ac140cc0 2021-11-21T22:00:01.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78aad9d700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f78900536a0 0x7f7890055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.306+0000 7f78aad9d700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f78900536a0 0x7f7890055b60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f789c00d000 tx=0x7f789c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.310+0000 7f78a8d99700 1 -- 172.21.2.116:0/1653316968 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f78a40439a0 con 0x7f78ac140cc0 2021-11-21T22:00:01.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.422+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f78ac0573d0 con 0x7f78ac140cc0 2021-11-21T22:00:01.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78a8d99700 1 -- 172.21.2.116:0/1653316968 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v32) v1 ==== 189+0+0 (secure 0 0 0) 0x7f78a4024030 con 0x7f78ac140cc0 2021-11-21T22:00:01.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f78900536a0 msgr2=0x7f7890055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f78900536a0 0x7f7890055b60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f789c00d000 tx=0x7f789c009040).stop 2021-11-21T22:00:01.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f78ac140cc0 msgr2=0x7f78ac14a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f78ac140cc0 0x7f78ac14a3e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f78ac068600 tx=0x7f78a4016000).stop 2021-11-21T22:00:01.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 shutdown_connections 2021-11-21T22:00:01.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f78900536a0 0x7f7890055b60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f78ac140cc0 0x7f78ac14a3e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f78ac140860 0x7f78ac066f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 --2- 172.21.2.116:0/1653316968 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f788c004e70 0x7f78ac15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 >> 172.21.2.116:0/1653316968 conn(0x7f78ac05bd20 msgr2=0x7f78ac1421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 shutdown_connections 2021-11-21T22:00:01.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.426+0000 7f78b11f4700 1 -- 172.21.2.116:0/1653316968 wait complete. 2021-11-21T22:00:01.434 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 13 and data pool 14 2021-11-21T22:00:01.439 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:00:01.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 Processor -- start 2021-11-21T22:00:01.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- start start 2021-11-21T22:00:01.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f509c0ad780 0x7f509c05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f509c0af7e0 0x7f509c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c143f70 con 0x7f509c0affc0 2021-11-21T22:00:01.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c0adb60 con 0x7f509c0af7e0 2021-11-21T22:00:01.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c143160 con 0x7f509c0ad780 2021-11-21T22:00:01.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509affd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f509c0affc0 0x7f509c143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:32786/0 (socket says 172.21.2.116:32786) 2021-11-21T22:00:01.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509affd700 1 -- 172.21.2.116:0/1522235323 learned_addr learned my addr 172.21.2.116:0/1522235323 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:01.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509bfff700 1 --2- 172.21.2.116:0/1522235323 >> v2:172.21.2.116:3300/0 conn(0x7f509c0af7e0 0x7f509c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509bfff700 1 -- 172.21.2.116:0/1522235323 >> v2:172.21.2.117:3300/0 conn(0x7f509c0ad780 msgr2=0x7f509c05f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:01.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509bfff700 1 --2- 172.21.2.116:0/1522235323 >> v2:172.21.2.117:3300/0 conn(0x7f509c0ad780 0x7f509c05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509bfff700 1 -- 172.21.2.116:0/1522235323 >> v1:172.21.2.116:6789/0 conn(0x7f509c0affc0 legacy=0x7f509c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509bfff700 1 -- 172.21.2.116:0/1522235323 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0b0ce0 con 0x7f509c0af7e0 2021-11-21T22:00:01.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509bfff700 1 --2- 172.21.2.116:0/1522235323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c13fe70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f50940032f0 tx=0x7f50940091d0).ready entity=mon.0 client_cookie=19e9c89e1f3d758d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509a7fc700 1 -- 172.21.2.116:0/1522235323 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5094009ea0 con 0x7f509c0af7e0 2021-11-21T22:00:01.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509a7fc700 1 -- 172.21.2.116:0/1522235323 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509400b870 con 0x7f509c0af7e0 2021-11-21T22:00:01.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f509a7fc700 1 -- 172.21.2.116:0/1522235323 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509400b9e0 con 0x7f509c0af7e0 2021-11-21T22:00:01.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- 172.21.2.116:0/1522235323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 msgr2=0x7f509c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/1522235323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c13fe70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f50940032f0 tx=0x7f50940091d0).stop 2021-11-21T22:00:01.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- 172.21.2.116:0/1522235323 shutdown_connections 2021-11-21T22:00:01.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/1522235323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c13fe70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/1522235323 >> v2:172.21.2.117:3300/0 conn(0x7f509c0ad780 0x7f509c05f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- 172.21.2.116:0/1522235323 >> 172.21.2.116:0/1522235323 conn(0x7f509c0b6000 msgr2=0x7f509c0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- 172.21.2.116:0/1522235323 shutdown_connections 2021-11-21T22:00:01.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.486+0000 7f50a1ddc700 1 -- 172.21.2.116:0/1522235323 wait complete. 2021-11-21T22:00:01.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 Processor -- start 2021-11-21T22:00:01.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 -- start start 2021-11-21T22:00:01.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f509c0ad780 0x7f509c0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f509c0affc0 0x7f509c140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c066010 con 0x7f509c0af7e0 2021-11-21T22:00:01.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a3b70 con 0x7f509c0affc0 2021-11-21T22:00:01.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a9210 con 0x7f509c0ad780 2021-11-21T22:00:01.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49072/0 (socket says 172.21.2.116:49072) 2021-11-21T22:00:01.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 -- 172.21.2.116:0/4006112467 learned_addr learned my addr 172.21.2.116:0/4006112467 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:01.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 -- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f509c0ad780 msgr2=0x7f509c0ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f509c0ad780 0x7f509c0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 -- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f509c0affc0 msgr2=0x7f509c140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f509c0affc0 0x7f509c140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 -- 172.21.2.116:0/4006112467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094009470 con 0x7f509c0af7e0 2021-11-21T22:00:01.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509affd700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c140330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f509c068820 tx=0x7f508c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f5098ff9700 1 -- 172.21.2.116:0/4006112467 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f508c001d70 con 0x7f509c0af7e0 2021-11-21T22:00:01.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f5098ff9700 1 -- 172.21.2.116:0/4006112467 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00ce30 con 0x7f509c0af7e0 2021-11-21T22:00:01.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c065b70 con 0x7f509c0af7e0 2021-11-21T22:00:01.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f5098ff9700 1 -- 172.21.2.116:0/4006112467 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f508c005030 con 0x7f509c0af7e0 2021-11-21T22:00:01.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c0ac830 con 0x7f509c0af7e0 2021-11-21T22:00:01.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c0a9410 con 0x7f509c0af7e0 2021-11-21T22:00:01.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f5098ff9700 1 -- 172.21.2.116:0/4006112467 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f508c01b070 con 0x7f509c0af7e0 2021-11-21T22:00:01.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f5098ff9700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f507c053610 0x7f507c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f5098ff9700 1 -- 172.21.2.116:0/4006112467 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f508c06ae40 con 0x7f509c0af7e0 2021-11-21T22:00:01.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509b7fe700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f507c053610 0x7f507c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.490+0000 7f509b7fe700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f507c053610 0x7f507c055ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f5090002f70 tx=0x7f5090006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.494+0000 7f5098ff9700 1 -- 172.21.2.116:0/4006112467 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f508c041fa0 con 0x7f509c0af7e0 2021-11-21T22:00:01.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f509c065d00 con 0x7f509c0af7e0 2021-11-21T22:00:01.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f5098ff9700 1 -- 172.21.2.116:0/4006112467 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f508c0455c0 con 0x7f509c0af7e0 2021-11-21T22:00:01.583 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:01.583 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":75,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:00:01.426048+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:59.269756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-11-21T22:00:00.412748+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:00:01.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f507c053610 msgr2=0x7f507c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f507c053610 0x7f507c055ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f5090002f70 tx=0x7f5090006a80).stop 2021-11-21T22:00:01.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 msgr2=0x7f509c140330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c140330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f509c068820 tx=0x7f508c007c30).stop 2021-11-21T22:00:01.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 shutdown_connections 2021-11-21T22:00:01.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f509c0affc0 0x7f509c140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f507c053610 0x7f507c055ad0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f509c0af7e0 0x7f509c140330 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 --2- 172.21.2.116:0/4006112467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f509c0ad780 0x7f509c0ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 >> 172.21.2.116:0/4006112467 conn(0x7f509c0b6000 msgr2=0x7f509c144150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 shutdown_connections 2021-11-21T22:00:01.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.578+0000 7f50a1ddc700 1 -- 172.21.2.116:0/4006112467 wait complete. 2021-11-21T22:00:01.590 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:00:01.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 Processor -- start 2021-11-21T22:00:01.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- start start 2021-11-21T22:00:01.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcd780a6aa0 0x7fcd7814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 0x7fcd781470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd780a6e80 con 0x7fcd781475d0 2021-11-21T22:00:01.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780a8bc0 con 0x7fcd780a6aa0 2021-11-21T22:00:01.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7814a7b0 con 0x7fcd780a86b0 2021-11-21T22:00:01.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcd780a6aa0 0x7fcd7814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcd780a6aa0 0x7fcd7814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49076/0 (socket says 172.21.2.116:49076) 2021-11-21T22:00:01.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 -- 172.21.2.116:0/1809920741 learned_addr learned my addr 172.21.2.116:0/1809920741 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:01.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 -- 172.21.2.116:0/1809920741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 msgr2=0x7fcd781470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:01.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 --2- 172.21.2.116:0/1809920741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 0x7fcd781470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 -- 172.21.2.116:0/1809920741 >> v1:172.21.2.116:6789/0 conn(0x7fcd781475d0 legacy=0x7fcd781443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 -- 172.21.2.116:0/1809920741 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7814aa10 con 0x7fcd780a6aa0 2021-11-21T22:00:01.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7dce4700 1 --2- 172.21.2.116:0/1809920741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd780a6aa0 0x7fcd7814b150 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fcd70001490 tx=0x7fcd74006eb0).ready entity=mon.0 client_cookie=94dc7f57dcfbf5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7cce2700 1 -- 172.21.2.116:0/1809920741 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd7400d570 con 0x7fcd780a6aa0 2021-11-21T22:00:01.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7cce2700 1 -- 172.21.2.116:0/1809920741 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd74000e90 con 0x7fcd780a6aa0 2021-11-21T22:00:01.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7cce2700 1 -- 172.21.2.116:0/1809920741 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd7400d900 con 0x7fcd780a6aa0 2021-11-21T22:00:01.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- 172.21.2.116:0/1809920741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd780a6aa0 msgr2=0x7fcd7814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/1809920741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd780a6aa0 0x7fcd7814b150 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fcd70001490 tx=0x7fcd74006eb0).stop 2021-11-21T22:00:01.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- 172.21.2.116:0/1809920741 shutdown_connections 2021-11-21T22:00:01.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/1809920741 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 0x7fcd781470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/1809920741 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd780a6aa0 0x7fcd7814b150 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- 172.21.2.116:0/1809920741 >> 172.21.2.116:0/1809920741 conn(0x7fcd7805bd20 msgr2=0x7fcd7805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- 172.21.2.116:0/1809920741 shutdown_connections 2021-11-21T22:00:01.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.638+0000 7fcd7ff48700 1 -- 172.21.2.116:0/1809920741 wait complete. 2021-11-21T22:00:01.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 Processor -- start 2021-11-21T22:00:01.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 -- start start 2021-11-21T22:00:01.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd780a6aa0 0x7fcd78146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 0x7fcd78156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd781475d0 0x7fcd78156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78142820 con 0x7fcd781475d0 2021-11-21T22:00:01.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78066620 con 0x7fcd780a6aa0 2021-11-21T22:00:01.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815a170 con 0x7fcd780a86b0 2021-11-21T22:00:01.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd781475d0 0x7fcd78156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd781475d0 0x7fcd78156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49086/0 (socket says 172.21.2.116:49086) 2021-11-21T22:00:01.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 -- 172.21.2.116:0/3562277910 learned_addr learned my addr 172.21.2.116:0/3562277910 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:01.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 -- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 msgr2=0x7fcd78156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 0x7fcd78156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 -- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd780a6aa0 msgr2=0x7fcd78146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd780a6aa0 0x7fcd78146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 -- 172.21.2.116:0/3562277910 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7400b1e0 con 0x7fcd781475d0 2021-11-21T22:00:01.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7e4e5700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd781475d0 0x7fcd78156770 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fcd70001380 tx=0x7fcd70007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd66ffd700 1 -- 172.21.2.116:0/3562277910 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd70006890 con 0x7fcd781475d0 2021-11-21T22:00:01.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd66ffd700 1 -- 172.21.2.116:0/3562277910 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd700069f0 con 0x7fcd781475d0 2021-11-21T22:00:01.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd66ffd700 1 -- 172.21.2.116:0/3562277910 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd70006b60 con 0x7fcd781475d0 2021-11-21T22:00:01.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd78066380 con 0x7fcd781475d0 2021-11-21T22:00:01.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd781472d0 con 0x7fcd781475d0 2021-11-21T22:00:01.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd66ffd700 1 -- 172.21.2.116:0/3562277910 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcd70018020 con 0x7fcd781475d0 2021-11-21T22:00:01.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd780573d0 con 0x7fcd781475d0 2021-11-21T22:00:01.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd66ffd700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd580536d0 0x7fcd58055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd66ffd700 1 -- 172.21.2.116:0/3562277910 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcd7006b3e0 con 0x7fcd781475d0 2021-11-21T22:00:01.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.642+0000 7fcd7dce4700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd580536d0 0x7fcd58055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.646+0000 7fcd7dce4700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd580536d0 0x7fcd58055b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fcd70001490 tx=0x7fcd74015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.646+0000 7fcd66ffd700 1 -- 172.21.2.116:0/3562277910 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd70042540 con 0x7fcd781475d0 2021-11-21T22:00:01.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd7805db70 con 0x7fcd781475d0 2021-11-21T22:00:01.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd66ffd700 1 -- 172.21.2.116:0/3562277910 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16054 (secure 0 0 0) 0x7fcd70016020 con 0x7fcd781475d0 2021-11-21T22:00:01.733 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:01.733 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":75,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:00:01.426048+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:59.269756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-11-21T22:00:00.412748+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:00:01.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd580536d0 msgr2=0x7fcd58055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd580536d0 0x7fcd58055b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fcd70001490 tx=0x7fcd74015000).stop 2021-11-21T22:00:01.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd781475d0 msgr2=0x7fcd78156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd781475d0 0x7fcd78156770 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fcd70001380 tx=0x7fcd70007f70).stop 2021-11-21T22:00:01.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 shutdown_connections 2021-11-21T22:00:01.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd580536d0 0x7fcd58055b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd781475d0 0x7fcd78156770 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd780a86b0 0x7fcd78156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 --2- 172.21.2.116:0/3562277910 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd780a6aa0 0x7fcd78146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 >> 172.21.2.116:0/3562277910 conn(0x7fcd7805bd20 msgr2=0x7fcd78148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 shutdown_connections 2021-11-21T22:00:01.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.730+0000 7fcd7ff48700 1 -- 172.21.2.116:0/3562277910 wait complete. 2021-11-21T22:00:01.740 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:00:01.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 Processor -- start 2021-11-21T22:00:01.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- start start 2021-11-21T22:00:01.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 0x7fcaec1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcaec0ac600 0x7fcaec0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaec147e80 con 0x7fcaec0abce0 2021-11-21T22:00:01.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec0b1490 con 0x7fcaec0ac600 2021-11-21T22:00:01.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec0b1fd0 con 0x7fcaec144810 2021-11-21T22:00:01.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcaec0ac600 0x7fcaec0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcaec0ac600 0x7fcaec0aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49092/0 (socket says 172.21.2.116:49092) 2021-11-21T22:00:01.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 -- 172.21.2.116:0/2278604239 learned_addr learned my addr 172.21.2.116:0/2278604239 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:01.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 -- 172.21.2.116:0/2278604239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 msgr2=0x7fcaec1478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:01.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 --2- 172.21.2.116:0/2278604239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 0x7fcaec1478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 -- 172.21.2.116:0/2278604239 >> v1:172.21.2.116:6789/0 conn(0x7fcaec0abce0 legacy=0x7fcaec0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:01.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 -- 172.21.2.116:0/2278604239 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0af5a0 con 0x7fcaec0ac600 2021-11-21T22:00:01.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf09f7700 1 --2- 172.21.2.116:0/2278604239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0ac600 0x7fcaec0aca90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fcaec0b2a30 tx=0x7fcae4009290).ready entity=mon.0 client_cookie=3c61b55e740e8a0d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaeaffd700 1 -- 172.21.2.116:0/2278604239 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcae400b860 con 0x7fcaec0ac600 2021-11-21T22:00:01.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaeaffd700 1 -- 172.21.2.116:0/2278604239 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae400b9c0 con 0x7fcaec0ac600 2021-11-21T22:00:01.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaeaffd700 1 -- 172.21.2.116:0/2278604239 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcae400bb70 con 0x7fcaec0ac600 2021-11-21T22:00:01.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- 172.21.2.116:0/2278604239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0ac600 msgr2=0x7fcaec0aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 --2- 172.21.2.116:0/2278604239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0ac600 0x7fcaec0aca90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fcaec0b2a30 tx=0x7fcae4009290).stop 2021-11-21T22:00:01.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- 172.21.2.116:0/2278604239 shutdown_connections 2021-11-21T22:00:01.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 --2- 172.21.2.116:0/2278604239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0ac600 0x7fcaec0aca90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:01.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 --2- 172.21.2.116:0/2278604239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 0x7fcaec1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- 172.21.2.116:0/2278604239 >> 172.21.2.116:0/2278604239 conn(0x7fcaec05bd20 msgr2=0x7fcaec05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:01.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- 172.21.2.116:0/2278604239 shutdown_connections 2021-11-21T22:00:01.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- 172.21.2.116:0/2278604239 wait complete. 2021-11-21T22:00:01.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 Processor -- start 2021-11-21T22:00:01.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.790+0000 7fcaf245a700 1 -- start start 2021-11-21T22:00:01.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0abce0 0x7fcaec143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaec0ac600 0x7fcaec156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 0x7fcaec1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1457e0 con 0x7fcaec0abce0 2021-11-21T22:00:01.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec145a50 con 0x7fcaec0ac600 2021-11-21T22:00:01.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1452a0 con 0x7fcaec144810 2021-11-21T22:00:01.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0abce0 0x7fcaec143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0abce0 0x7fcaec143970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49096/0 (socket says 172.21.2.116:49096) 2021-11-21T22:00:01.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 -- 172.21.2.116:0/1354915220 learned_addr learned my addr 172.21.2.116:0/1354915220 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:01.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 -- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 msgr2=0x7fcaec1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 0x7fcaec1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 -- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaec0ac600 msgr2=0x7fcaec156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:01.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaec0ac600 0x7fcaec156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:01.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 -- 172.21.2.116:0/1354915220 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4009530 con 0x7fcaec0abce0 2021-11-21T22:00:01.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaebfff700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0abce0 0x7fcaec143970 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fcadc002ef0 tx=0x7fcadc006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcae97fa700 1 -- 172.21.2.116:0/1354915220 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcadc012410 con 0x7fcaec0abce0 2021-11-21T22:00:01.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcae97fa700 1 -- 172.21.2.116:0/1354915220 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc012570 con 0x7fcaec0abce0 2021-11-21T22:00:01.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec145540 con 0x7fcaec0abce0 2021-11-21T22:00:01.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcae97fa700 1 -- 172.21.2.116:0/1354915220 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcadc0128d0 con 0x7fcaec0abce0 2021-11-21T22:00:01.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec064520 con 0x7fcaec0abce0 2021-11-21T22:00:01.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaec159e00 con 0x7fcaec0abce0 2021-11-21T22:00:01.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcae97fa700 1 -- 172.21.2.116:0/1354915220 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcadc00f030 con 0x7fcaec0abce0 2021-11-21T22:00:01.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcae97fa700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcad00535b0 0x7fcad0055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:01.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcae97fa700 1 -- 172.21.2.116:0/1354915220 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcadc06b600 con 0x7fcaec0abce0 2021-11-21T22:00:01.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.794+0000 7fcaeb7fe700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcad00535b0 0x7fcad0055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:01.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.798+0000 7fcaeb7fe700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcad00535b0 0x7fcad0055a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fcad4001fa0 tx=0x7fcad40077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:01.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.798+0000 7fcae97fa700 1 -- 172.21.2.116:0/1354915220 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcadc042760 con 0x7fcaec0abce0 2021-11-21T22:00:01.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:01.906+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcaec0af060 con 0x7fcaec0abce0 2021-11-21T22:00:02.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcae97fa700 1 -- 172.21.2.116:0/1354915220 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcadc045d80 con 0x7fcaec0abce0 2021-11-21T22:00:02.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcad00535b0 msgr2=0x7fcad0055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcad00535b0 0x7fcad0055a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fcad4001fa0 tx=0x7fcad40077f0).stop 2021-11-21T22:00:02.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0abce0 msgr2=0x7fcaec143970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0abce0 0x7fcaec143970 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fcadc002ef0 tx=0x7fcadc006d50).stop 2021-11-21T22:00:02.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 shutdown_connections 2021-11-21T22:00:02.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaec144810 0x7fcaec1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaec0ac600 0x7fcaec156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcad00535b0 0x7fcad0055a70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 --2- 172.21.2.116:0/1354915220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaec0abce0 0x7fcaec143970 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 >> 172.21.2.116:0/1354915220 conn(0x7fcaec05bd20 msgr2=0x7fcaec05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:02.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 shutdown_connections 2021-11-21T22:00:02.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.434+0000 7fcaf245a700 1 -- 172.21.2.116:0/1354915220 wait complete. 2021-11-21T22:00:02.446 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:02.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 Processor -- start 2021-11-21T22:00:02.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- start start 2021-11-21T22:00:02.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3d74099d60 0x7f3d7414af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7414a560 con 0x7f3d74097ec0 2021-11-21T22:00:02.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7414a800 con 0x7f3d740b5630 2021-11-21T22:00:02.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d74144b40 con 0x7f3d74099d60 2021-11-21T22:00:02.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d79cfe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f3d74097ec0 0x7f3d74099650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:32830/0 (socket says 172.21.2.116:32830) 2021-11-21T22:00:02.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d79cfe700 1 -- 172.21.2.116:0/2364036955 learned_addr learned my addr 172.21.2.116:0/2364036955 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:02.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70941316 0 0) 0x7f3d7414a560 con 0x7f3d74097ec0 2021-11-21T22:00:02.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d5c003260 con 0x7f3d74097ec0 2021-11-21T22:00:02.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/2364036955 >> v2:172.21.2.117:3300/0 conn(0x7f3d74099d60 0x7f3d7414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255011028 0 0) 0x7f3d5c003260 con 0x7f3d74097ec0 2021-11-21T22:00:02.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 >> v2:172.21.2.117:3300/0 conn(0x7f3d74099d60 msgr2=0x7f3d7414af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:02.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 --2- 172.21.2.116:0/2364036955 >> v2:172.21.2.117:3300/0 conn(0x7f3d74099d60 0x7f3d7414af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 >> v1:172.21.2.117:6789/0 conn(0x7f3d740b5630 legacy=0x7f3d74146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7414aaa0 con 0x7f3d74097ec0 2021-11-21T22:00:02.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f3d64001660 con 0x7f3d74097ec0 2021-11-21T22:00:02.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 >> v1:172.21.2.116:6789/0 conn(0x7f3d74097ec0 legacy=0x7f3d74099650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c0037a0 0x7f3d5c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004500 0x7f3d5c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c004f90 0x7f3d5c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74144b40 con 0x7f3d5c004f90 2021-11-21T22:00:02.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c00be30 con 0x7f3d5c0037a0 2021-11-21T22:00:02.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c00bff0 con 0x7f3d5c004500 2021-11-21T22:00:02.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c004f90 0x7f3d5c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d79cfe700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004500 0x7f3d5c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7a4ff700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c0037a0 0x7f3d5c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 -- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004500 msgr2=0x7f3d5c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:02.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004500 0x7f3d5c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 -- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c0037a0 msgr2=0x7f3d5c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:02.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c0037a0 0x7f3d5c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 -- 172.21.2.116:0/2364036955 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c003260 con 0x7f3d5c004f90 2021-11-21T22:00:02.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7a4ff700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c0037a0 0x7f3d5c003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:00:02.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 -- 172.21.2.116:0/2364036955 reap_dead start 2021-11-21T22:00:02.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c004f90 0x7f3d5c00b750 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f3d6c00a000 tx=0x7f3d6c008040).ready entity=mon.0 client_cookie=2894ab294e9021c6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d6c00c070 con 0x7f3d5c004f90 2021-11-21T22:00:02.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c00e7e0 con 0x7f3d5c004f90 2021-11-21T22:00:02.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d794fd700 1 -- 172.21.2.116:0/2364036955 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d6c00eb40 con 0x7f3d5c004f90 2021-11-21T22:00:02.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- 172.21.2.116:0/2364036955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c004f90 msgr2=0x7f3d5c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c004f90 0x7f3d5c00b750 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f3d6c00a000 tx=0x7f3d6c008040).stop 2021-11-21T22:00:02.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- 172.21.2.116:0/2364036955 shutdown_connections 2021-11-21T22:00:02.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 --2- 172.21.2.116:0/2364036955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c004f90 0x7f3d5c00b750 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- 172.21.2.116:0/2364036955 >> 172.21.2.116:0/2364036955 conn(0x7f3d7405c7f0 msgr2=0x7f3d740b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:02.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- 172.21.2.116:0/2364036955 shutdown_connections 2021-11-21T22:00:02.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.506+0000 7f3d7b501700 1 -- 172.21.2.116:0/2364036955 wait complete. 2021-11-21T22:00:02.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 Processor -- start 2021-11-21T22:00:02.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 -- start start 2021-11-21T22:00:02.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004f90 0x7f3d740ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d740ae2f0 0x7f3d740b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d74156aa0 0x7f3d74065680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74062cd0 con 0x7f3d74156aa0 2021-11-21T22:00:02.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74062f70 con 0x7f3d740ae2f0 2021-11-21T22:00:02.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74156600 con 0x7f3d5c004f90 2021-11-21T22:00:02.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d74156aa0 0x7f3d74065680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d74156aa0 0x7f3d74065680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49118/0 (socket says 172.21.2.116:49118) 2021-11-21T22:00:02.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 -- 172.21.2.116:0/3810716214 learned_addr learned my addr 172.21.2.116:0/3810716214 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:02.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d79cfe700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d740ae2f0 0x7f3d740b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7a4ff700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004f90 0x7f3d740ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 -- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004f90 msgr2=0x7f3d740ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004f90 0x7f3d740ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 -- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d740ae2f0 msgr2=0x7f3d740b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d740ae2f0 0x7f3d740b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 -- 172.21.2.116:0/3810716214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d741562f0 con 0x7f3d74156aa0 2021-11-21T22:00:02.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7ad00700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d74156aa0 0x7f3d74065680 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f3d6c00a000 tx=0x7f3d6c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d637fe700 1 -- 172.21.2.116:0/3810716214 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d6c022070 con 0x7f3d74156aa0 2021-11-21T22:00:02.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d637fe700 1 -- 172.21.2.116:0/3810716214 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c01e690 con 0x7f3d74156aa0 2021-11-21T22:00:02.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d740aebb0 con 0x7f3d74156aa0 2021-11-21T22:00:02.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d637fe700 1 -- 172.21.2.116:0/3810716214 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d6c01e9f0 con 0x7f3d74156aa0 2021-11-21T22:00:02.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d74062a40 con 0x7f3d74156aa0 2021-11-21T22:00:02.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d740573d0 con 0x7f3d74156aa0 2021-11-21T22:00:02.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d637fe700 1 -- 172.21.2.116:0/3810716214 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3d6c01eb50 con 0x7f3d74156aa0 2021-11-21T22:00:02.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d637fe700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d54053660 0x7f3d54055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d637fe700 1 -- 172.21.2.116:0/3810716214 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3d6c06c9f0 con 0x7f3d74156aa0 2021-11-21T22:00:02.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.510+0000 7f3d7a4ff700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d54053660 0x7f3d54055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.514+0000 7f3d7a4ff700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d54053660 0x7f3d54055b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3d7000d000 tx=0x7f3d70009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.514+0000 7f3d637fe700 1 -- 172.21.2.116:0/3810716214 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3d6c043a40 con 0x7f3d74156aa0 2021-11-21T22:00:02.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d74061070 con 0x7f3d74156aa0 2021-11-21T22:00:02.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d637fe700 1 -- 172.21.2.116:0/3810716214 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f3d6c047060 con 0x7f3d74156aa0 2021-11-21T22:00:02.629 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:02.630 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:02.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d54053660 msgr2=0x7f3d54055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d54053660 0x7f3d54055b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3d7000d000 tx=0x7f3d70009040).stop 2021-11-21T22:00:02.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d74156aa0 msgr2=0x7f3d74065680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d74156aa0 0x7f3d74065680 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f3d6c00a000 tx=0x7f3d6c008040).stop 2021-11-21T22:00:02.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 shutdown_connections 2021-11-21T22:00:02.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d54053660 0x7f3d54055b20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d74156aa0 0x7f3d74065680 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d740ae2f0 0x7f3d740b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 --2- 172.21.2.116:0/3810716214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c004f90 0x7f3d740ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 >> 172.21.2.116:0/3810716214 conn(0x7f3d7405c7f0 msgr2=0x7f3d74149ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:02.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 shutdown_connections 2021-11-21T22:00:02.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.626+0000 7f3d7b501700 1 -- 172.21.2.116:0/3810716214 wait complete. 2021-11-21T22:00:02.634 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:02.637 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:00:02.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.682+0000 7f947f444700 1 Processor -- start 2021-11-21T22:00:02.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.682+0000 7f947f444700 1 -- start start 2021-11-21T22:00:02.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.682+0000 7f947f444700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 0x7f947813c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.682+0000 7f947f444700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780b00d0 con 0x7f94780aed90 2021-11-21T22:00:02.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.682+0000 7f947f444700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780b2880 con 0x7f94780b24a0 2021-11-21T22:00:02.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.682+0000 7f947f444700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94780adba0 con 0x7f94780b1a40 2021-11-21T22:00:02.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947c9df700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f94780aed90 0x7f947813c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:32850/0 (socket says 172.21.2.116:32850) 2021-11-21T22:00:02.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947c9df700 1 -- 172.21.2.116:0/652340726 learned_addr learned my addr 172.21.2.116:0/652340726 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:02.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163422892 0 0) 0x7f94780b00d0 con 0x7f94780aed90 2021-11-21T22:00:02.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9460003260 con 0x7f94780aed90 2021-11-21T22:00:02.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d9e1700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 0x7f947813c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762638387 0 0) 0x7f9460003260 con 0x7f94780aed90 2021-11-21T22:00:02.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 msgr2=0x7f947813c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:02.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 0x7f947813c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 >> v1:172.21.2.117:6789/0 conn(0x7f94780b24a0 legacy=0x7f94780ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:02.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780b14a0 con 0x7f94780aed90 2021-11-21T22:00:02.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9474001680 con 0x7f94780aed90 2021-11-21T22:00:02.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 >> v1:172.21.2.116:6789/0 conn(0x7f94780aed90 legacy=0x7f947813c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f9460003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 0x7f9460004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9460004ac0 0x7f9460004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780adba0 con 0x7f9460003800 2021-11-21T22:00:02.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460005930 con 0x7f9460004ac0 2021-11-21T22:00:02.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9460005af0 con 0x7f94780b1a40 2021-11-21T22:00:02.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d1e0700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f9460003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947c9df700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 0x7f9460004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d9e1700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9460004ac0 0x7f9460004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d1e0700 1 -- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 msgr2=0x7f9460004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:02.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d1e0700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94780b1a40 0x7f9460004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d1e0700 1 -- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9460004ac0 msgr2=0x7f9460004f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:02.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d1e0700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9460004ac0 0x7f9460004f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d1e0700 1 -- 172.21.2.116:0/652340726 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460003260 con 0x7f9460003800 2021-11-21T22:00:02.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d9e1700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9460004ac0 0x7f9460004f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:02.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d1e0700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f9460003c20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f946c001fa0 tx=0x7f946c007a00).ready entity=mon.0 client_cookie=bdd3d719465109c7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c007070 con 0x7f9460003800 2021-11-21T22:00:02.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c0071d0 con 0x7f9460003800 2021-11-21T22:00:02.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f946bfff700 1 -- 172.21.2.116:0/652340726 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c006510 con 0x7f9460003800 2021-11-21T22:00:02.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 -- 172.21.2.116:0/652340726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 msgr2=0x7f9460003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 --2- 172.21.2.116:0/652340726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f9460003c20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f946c001fa0 tx=0x7f946c007a00).stop 2021-11-21T22:00:02.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947d9e1700 1 -- 172.21.2.116:0/652340726 reap_dead start 2021-11-21T22:00:02.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 -- 172.21.2.116:0/652340726 shutdown_connections 2021-11-21T22:00:02.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 -- 172.21.2.116:0/652340726 >> 172.21.2.116:0/652340726 conn(0x7f947805fd00 msgr2=0x7f94780600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:02.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 -- 172.21.2.116:0/652340726 shutdown_connections 2021-11-21T22:00:02.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 -- 172.21.2.116:0/652340726 wait complete. 2021-11-21T22:00:02.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 Processor -- start 2021-11-21T22:00:02.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.686+0000 7f947f444700 1 -- start start 2021-11-21T22:00:02.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f94781427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f947814daa0 0x7f9478064530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9478066b50 0x7f9478064a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9478063ea0 con 0x7f9460003800 2021-11-21T22:00:02.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781117b0 con 0x7f9478066b50 2021-11-21T22:00:02.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9478065720 con 0x7f947814daa0 2021-11-21T22:00:02.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f94781427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f94781427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49140/0 (socket says 172.21.2.116:49140) 2021-11-21T22:00:02.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 -- 172.21.2.116:0/1920105230 learned_addr learned my addr 172.21.2.116:0/1920105230 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:02.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 -- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f947814daa0 msgr2=0x7f9478064530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f947814daa0 0x7f9478064530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 -- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9478066b50 msgr2=0x7f9478064a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9478066b50 0x7f9478064a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 -- 172.21.2.116:0/1920105230 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780659c0 con 0x7f9460003800 2021-11-21T22:00:02.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947d1e0700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f94781427c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f946c001fa0 tx=0x7f946c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f946a7fc700 1 -- 172.21.2.116:0/1920105230 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c014620 con 0x7f9460003800 2021-11-21T22:00:02.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f946a7fc700 1 -- 172.21.2.116:0/1920105230 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c000e90 con 0x7f9460003800 2021-11-21T22:00:02.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9478065c60 con 0x7f9460003800 2021-11-21T22:00:02.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f946a7fc700 1 -- 172.21.2.116:0/1920105230 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c014970 con 0x7f9460003800 2021-11-21T22:00:02.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9478066030 con 0x7f9460003800 2021-11-21T22:00:02.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9478065df0 con 0x7f9460003800 2021-11-21T22:00:02.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f946a7fc700 1 -- 172.21.2.116:0/1920105230 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f946c029030 con 0x7f9460003800 2021-11-21T22:00:02.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f946a7fc700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94580535b0 0x7f9458055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f946a7fc700 1 -- 172.21.2.116:0/1920105230 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f946c06cd00 con 0x7f9460003800 2021-11-21T22:00:02.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.690+0000 7f947c9df700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94580535b0 0x7f9458055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.694+0000 7f947c9df700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94580535b0 0x7f9458055a70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f947400b000 tx=0x7f9474000fe0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.694+0000 7f946a7fc700 1 -- 172.21.2.116:0/1920105230 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f946c070030 con 0x7f9460003800 2021-11-21T22:00:02.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94780b0300 con 0x7f9460003800 2021-11-21T22:00:02.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f946a7fc700 1 -- 172.21.2.116:0/1920105230 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f946c043e60 con 0x7f9460003800 2021-11-21T22:00:02.781 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:02.781 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":76,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:00:02.430725+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:59.269756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-11-21T22:00:00.412748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:00:02.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94580535b0 msgr2=0x7f9458055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94580535b0 0x7f9458055a70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f947400b000 tx=0x7f9474000fe0).stop 2021-11-21T22:00:02.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 msgr2=0x7f94781427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f94781427c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f946c001fa0 tx=0x7f946c015000).stop 2021-11-21T22:00:02.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 shutdown_connections 2021-11-21T22:00:02.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9478066b50 0x7f9478064a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f947814daa0 0x7f9478064530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94580535b0 0x7f9458055a70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 --2- 172.21.2.116:0/1920105230 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9460003800 0x7f94781427c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 >> 172.21.2.116:0/1920105230 conn(0x7f947805fd00 msgr2=0x7f94780acac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:02.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 shutdown_connections 2021-11-21T22:00:02.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.778+0000 7f947f444700 1 -- 172.21.2.116:0/1920105230 wait complete. 2021-11-21T22:00:02.789 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:00:02.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 Processor -- start 2021-11-21T22:00:02.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- start start 2021-11-21T22:00:02.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3ed00b11c0 0x7f3ed0147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ed0143e60 0x7f3ed01442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed00b1940 con 0x7f3ed00a8670 2021-11-21T22:00:02.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00b15a0 con 0x7f3ed0143e60 2021-11-21T22:00:02.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00b1b70 con 0x7f3ed00b11c0 2021-11-21T22:00:02.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ed0143e60 0x7f3ed01442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ed0143e60 0x7f3ed01442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49146/0 (socket says 172.21.2.116:49146) 2021-11-21T22:00:02.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 -- 172.21.2.116:0/2023883227 learned_addr learned my addr 172.21.2.116:0/2023883227 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:02.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 -- 172.21.2.116:0/2023883227 >> v2:172.21.2.117:3300/0 conn(0x7f3ed00b11c0 msgr2=0x7f3ed0147230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:02.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 --2- 172.21.2.116:0/2023883227 >> v2:172.21.2.117:3300/0 conn(0x7f3ed00b11c0 0x7f3ed0147230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 -- 172.21.2.116:0/2023883227 >> v1:172.21.2.116:6789/0 conn(0x7f3ed00a8670 legacy=0x7f3ed0147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 -- 172.21.2.116:0/2023883227 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00b1e10 con 0x7f3ed0143e60 2021-11-21T22:00:02.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6871700 1 --2- 172.21.2.116:0/2023883227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed0143e60 0x7f3ed01442d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f3ec80032f0 tx=0x7f3ec8009290).ready entity=mon.0 client_cookie=92094443f49ee8ff server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed506e700 1 -- 172.21.2.116:0/2023883227 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec800b870 con 0x7f3ed0143e60 2021-11-21T22:00:02.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed506e700 1 -- 172.21.2.116:0/2023883227 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec800b9d0 con 0x7f3ed0143e60 2021-11-21T22:00:02.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed506e700 1 -- 172.21.2.116:0/2023883227 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec800bb80 con 0x7f3ed0143e60 2021-11-21T22:00:02.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- 172.21.2.116:0/2023883227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed0143e60 msgr2=0x7f3ed01442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/2023883227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed0143e60 0x7f3ed01442d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f3ec80032f0 tx=0x7f3ec8009290).stop 2021-11-21T22:00:02.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- 172.21.2.116:0/2023883227 shutdown_connections 2021-11-21T22:00:02.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/2023883227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed0143e60 0x7f3ed01442d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/2023883227 >> v2:172.21.2.117:3300/0 conn(0x7f3ed00b11c0 0x7f3ed0147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- 172.21.2.116:0/2023883227 >> 172.21.2.116:0/2023883227 conn(0x7f3ed005bd20 msgr2=0x7f3ed005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:02.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- 172.21.2.116:0/2023883227 shutdown_connections 2021-11-21T22:00:02.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- 172.21.2.116:0/2023883227 wait complete. 2021-11-21T22:00:02.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 Processor -- start 2021-11-21T22:00:02.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- start start 2021-11-21T22:00:02.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed00a8670 0x7f3ed00b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ed00b11c0 0x7f3ed00643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ed0143e60 0x7f3ed00648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed005fe00 con 0x7f3ed00a8670 2021-11-21T22:00:02.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00b3ee0 con 0x7f3ed0143e60 2021-11-21T22:00:02.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed82d4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00b4610 con 0x7f3ed00b11c0 2021-11-21T22:00:02.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed00a8670 0x7f3ed00b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed00a8670 0x7f3ed00b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49150/0 (socket says 172.21.2.116:49150) 2021-11-21T22:00:02.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 -- 172.21.2.116:0/1096934715 learned_addr learned my addr 172.21.2.116:0/1096934715 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:02.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 -- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ed00b11c0 msgr2=0x7f3ed00643b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:02.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ed00b11c0 0x7f3ed00643b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 -- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ed0143e60 msgr2=0x7f3ed00648a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:02.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ed0143e60 0x7f3ed00648a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.842+0000 7f3ed6070700 1 -- 172.21.2.116:0/1096934715 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8009530 con 0x7f3ed00a8670 2021-11-21T22:00:02.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ed6070700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed00a8670 0x7f3ed00b2e50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f3ecc002fb0 tx=0x7f3ecc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ec37fe700 1 -- 172.21.2.116:0/1096934715 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ecc013460 con 0x7f3ed00a8670 2021-11-21T22:00:02.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ec37fe700 1 -- 172.21.2.116:0/1096934715 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0135c0 con 0x7f3ed00a8670 2021-11-21T22:00:02.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed014ae30 con 0x7f3ed00a8670 2021-11-21T22:00:02.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ec37fe700 1 -- 172.21.2.116:0/1096934715 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ecc013920 con 0x7f3ed00a8670 2021-11-21T22:00:02.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed00b4370 con 0x7f3ed00a8670 2021-11-21T22:00:02.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed00640f0 con 0x7f3ed00a8670 2021-11-21T22:00:02.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ec37fe700 1 -- 172.21.2.116:0/1096934715 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3ecc00f070 con 0x7f3ed00a8670 2021-11-21T22:00:02.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ec37fe700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3eb0053720 0x7f3eb0055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:02.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ec37fe700 1 -- 172.21.2.116:0/1096934715 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3ecc06b3c0 con 0x7f3ed00a8670 2021-11-21T22:00:02.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ed586f700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3eb0053720 0x7f3eb0055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:02.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ed586f700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3eb0053720 0x7f3eb0055be0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f3ed0147180 tx=0x7f3ec40077b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:02.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.846+0000 7f3ec37fe700 1 -- 172.21.2.116:0/1096934715 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ecc042520 con 0x7f3ed00a8670 2021-11-21T22:00:02.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.958+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3ed00b20b0 con 0x7f3ed00a8670 2021-11-21T22:00:02.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.962+0000 7f3ec37fe700 1 -- 172.21.2.116:0/1096934715 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3ecc045b40 con 0x7f3ed00a8670 2021-11-21T22:00:02.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3eb0053720 msgr2=0x7f3eb0055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3eb0053720 0x7f3eb0055be0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f3ed0147180 tx=0x7f3ec40077b0).stop 2021-11-21T22:00:02.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed00a8670 msgr2=0x7f3ed00b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:02.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed00a8670 0x7f3ed00b2e50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f3ecc002fb0 tx=0x7f3ecc006df0).stop 2021-11-21T22:00:02.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 shutdown_connections 2021-11-21T22:00:02.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ed0143e60 0x7f3ed00648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ed00b11c0 0x7f3ed00643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:02.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3eb0053720 0x7f3eb0055be0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 --2- 172.21.2.116:0/1096934715 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ed00a8670 0x7f3ed00b2e50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:02.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 >> 172.21.2.116:0/1096934715 conn(0x7f3ed005bd20 msgr2=0x7f3ed0142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:02.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 shutdown_connections 2021-11-21T22:00:02.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:02.966+0000 7f3ed82d4700 1 -- 172.21.2.116:0/1096934715 wait complete. 2021-11-21T22:00:02.971 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:00:02.975 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:00:03.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 Processor -- start 2021-11-21T22:00:03.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- start start 2021-11-21T22:00:03.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0a480ad780 0x7f0a4805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a480af7e0 0x7f0a4813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a48143f70 con 0x7f0a480affc0 2021-11-21T22:00:03.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a480adb60 con 0x7f0a480af7e0 2021-11-21T22:00:03.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a48143160 con 0x7f0a480ad780 2021-11-21T22:00:03.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a480af7e0 0x7f0a4813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0a480af7e0 0x7f0a4813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49160/0 (socket says 172.21.2.116:49160) 2021-11-21T22:00:03.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 -- 172.21.2.116:0/724970856 learned_addr learned my addr 172.21.2.116:0/724970856 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:03.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 -- 172.21.2.116:0/724970856 >> v2:172.21.2.117:3300/0 conn(0x7f0a480ad780 msgr2=0x7f0a4805f730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:03.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 --2- 172.21.2.116:0/724970856 >> v2:172.21.2.117:3300/0 conn(0x7f0a480ad780 0x7f0a4805f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 -- 172.21.2.116:0/724970856 >> v1:172.21.2.116:6789/0 conn(0x7f0a480affc0 legacy=0x7f0a48143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 -- 172.21.2.116:0/724970856 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a480b0ce0 con 0x7f0a480af7e0 2021-11-21T22:00:03.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4e8f5700 1 --2- 172.21.2.116:0/724970856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480af7e0 0x7f0a4813fe70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f0a400032f0 tx=0x7f0a40009290).ready entity=mon.0 client_cookie=50217e4c4d510068 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4d0f2700 1 -- 172.21.2.116:0/724970856 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4000b870 con 0x7f0a480af7e0 2021-11-21T22:00:03.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4d0f2700 1 -- 172.21.2.116:0/724970856 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000b9d0 con 0x7f0a480af7e0 2021-11-21T22:00:03.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a4d0f2700 1 -- 172.21.2.116:0/724970856 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4000bb80 con 0x7f0a480af7e0 2021-11-21T22:00:03.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- 172.21.2.116:0/724970856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480af7e0 msgr2=0x7f0a4813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 --2- 172.21.2.116:0/724970856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480af7e0 0x7f0a4813fe70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f0a400032f0 tx=0x7f0a40009290).stop 2021-11-21T22:00:03.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- 172.21.2.116:0/724970856 shutdown_connections 2021-11-21T22:00:03.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 --2- 172.21.2.116:0/724970856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480af7e0 0x7f0a4813fe70 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 --2- 172.21.2.116:0/724970856 >> v2:172.21.2.117:3300/0 conn(0x7f0a480ad780 0x7f0a4805f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- 172.21.2.116:0/724970856 >> 172.21.2.116:0/724970856 conn(0x7f0a480b6000 msgr2=0x7f0a480b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:03.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- 172.21.2.116:0/724970856 shutdown_connections 2021-11-21T22:00:03.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.026+0000 7f0a50358700 1 -- 172.21.2.116:0/724970856 wait complete. 2021-11-21T22:00:03.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 Processor -- start 2021-11-21T22:00:03.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 -- start start 2021-11-21T22:00:03.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480ad780 0x7f0a48066f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a480af7e0 0x7f0a480abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a480affc0 0x7f0a480ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480650e0 con 0x7f0a480ad780 2021-11-21T22:00:03.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480b5aa0 con 0x7f0a480af7e0 2021-11-21T22:00:03.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48064be0 con 0x7f0a480affc0 2021-11-21T22:00:03.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480ad780 0x7f0a48066f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480ad780 0x7f0a48066f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49166/0 (socket says 172.21.2.116:49166) 2021-11-21T22:00:03.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 -- 172.21.2.116:0/1447051408 learned_addr learned my addr 172.21.2.116:0/1447051408 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:03.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 -- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a480affc0 msgr2=0x7f0a480ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a480affc0 0x7f0a480ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 -- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a480af7e0 msgr2=0x7f0a480abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a480af7e0 0x7f0a480abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 -- 172.21.2.116:0/1447051408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a40009530 con 0x7f0a480ad780 2021-11-21T22:00:03.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4e0f4700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480ad780 0x7f0a48066f80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f0a3c002fb0 tx=0x7f0a3c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a3b7fe700 1 -- 172.21.2.116:0/1447051408 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a3c013460 con 0x7f0a480ad780 2021-11-21T22:00:03.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a3b7fe700 1 -- 172.21.2.116:0/1447051408 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c0135c0 con 0x7f0a480ad780 2021-11-21T22:00:03.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a48064e40 con 0x7f0a480ad780 2021-11-21T22:00:03.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a3b7fe700 1 -- 172.21.2.116:0/1447051408 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a3c013920 con 0x7f0a480ad780 2021-11-21T22:00:03.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4813f9e0 con 0x7f0a480ad780 2021-11-21T22:00:03.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a480b0a40 con 0x7f0a480ad780 2021-11-21T22:00:03.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a3b7fe700 1 -- 172.21.2.116:0/1447051408 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0a3c01b030 con 0x7f0a480ad780 2021-11-21T22:00:03.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a3b7fe700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a280536a0 0x7f0a28055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a3b7fe700 1 -- 172.21.2.116:0/1447051408 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0a3c06b690 con 0x7f0a480ad780 2021-11-21T22:00:03.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4d8f3700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a280536a0 0x7f0a28055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.030+0000 7f0a4d8f3700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a280536a0 0x7f0a28055b60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f0a48068820 tx=0x7f0a44007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.034+0000 7f0a3b7fe700 1 -- 172.21.2.116:0/1447051408 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a3c0427f0 con 0x7f0a480ad780 2021-11-21T22:00:03.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.146+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0a480573d0 con 0x7f0a480ad780 2021-11-21T22:00:03.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.150+0000 7f0a3b7fe700 1 -- 172.21.2.116:0/1447051408 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0a3c045e10 con 0x7f0a480ad780 2021-11-21T22:00:03.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a280536a0 msgr2=0x7f0a28055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a280536a0 0x7f0a28055b60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f0a48068820 tx=0x7f0a44007810).stop 2021-11-21T22:00:03.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480ad780 msgr2=0x7f0a48066f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480ad780 0x7f0a48066f80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f0a3c002fb0 tx=0x7f0a3c006df0).stop 2021-11-21T22:00:03.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 shutdown_connections 2021-11-21T22:00:03.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0a480affc0 0x7f0a480ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0a480af7e0 0x7f0a480abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0a280536a0 0x7f0a28055b60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 --2- 172.21.2.116:0/1447051408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0a480ad780 0x7f0a48066f80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 >> 172.21.2.116:0/1447051408 conn(0x7f0a480b6000 msgr2=0x7f0a4813d980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:03.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 shutdown_connections 2021-11-21T22:00:03.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.154+0000 7f0a50358700 1 -- 172.21.2.116:0/1447051408 wait complete. 2021-11-21T22:00:03.160 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:00:03.163 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:03.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff678397700 1 Processor -- start 2021-11-21T22:00:03.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff678397700 1 -- start start 2021-11-21T22:00:03.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff678397700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6701443a0 0x7ff670144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff678397700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6700b29b0 con 0x7ff6700b3320 2021-11-21T22:00:03.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff678397700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff670067c30 con 0x7ff670067850 2021-11-21T22:00:03.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff678397700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700ab720 con 0x7ff6701443a0 2021-11-21T22:00:03.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675932700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff6700b3320 0x7ff6700ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:32898/0 (socket says 172.21.2.116:32898) 2021-11-21T22:00:03.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675932700 1 -- 172.21.2.116:0/2643181289 learned_addr learned my addr 172.21.2.116:0/2643181289 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:03.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356355228 0 0) 0x7ff6700b29b0 con 0x7ff6700b3320 2021-11-21T22:00:03.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff658003260 con 0x7ff6700b3320 2021-11-21T22:00:03.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff676934700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6701443a0 0x7ff670144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519521002 0 0) 0x7ff658003260 con 0x7ff6700b3320 2021-11-21T22:00:03.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6701443a0 msgr2=0x7ff670144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6701443a0 0x7ff670144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 >> v1:172.21.2.117:6789/0 conn(0x7ff670067850 legacy=0x7ff6700b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700b3780 con 0x7ff6700b3320 2021-11-21T22:00:03.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7ff660001660 con 0x7ff6700b3320 2021-11-21T22:00:03.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 >> v1:172.21.2.116:6789/0 conn(0x7ff6700b3320 legacy=0x7ff6700ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.210+0000 7ff675131700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff658003780 0x7ff658003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6701443a0 0x7ff658004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff658004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700ab720 con 0x7ff6580049f0 2021-11-21T22:00:03.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65800b830 con 0x7ff658003780 2021-11-21T22:00:03.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65800b9f0 con 0x7ff6701443a0 2021-11-21T22:00:03.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff658004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 -- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6701443a0 msgr2=0x7ff658004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6701443a0 0x7ff658004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 -- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff658003780 msgr2=0x7ff658003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff658003780 0x7ff658003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 -- 172.21.2.116:0/2643181289 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658003260 con 0x7ff6580049f0 2021-11-21T22:00:03.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff658004e70 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7ff66800a000 tx=0x7ff668008040).ready entity=mon.0 client_cookie=2997a66e200113c2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66800c070 con 0x7ff6580049f0 2021-11-21T22:00:03.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66800e5f0 con 0x7ff6580049f0 2021-11-21T22:00:03.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff675131700 1 -- 172.21.2.116:0/2643181289 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66800e950 con 0x7ff6580049f0 2021-11-21T22:00:03.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2643181289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 msgr2=0x7ff658004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 --2- 172.21.2.116:0/2643181289 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff658004e70 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7ff66800a000 tx=0x7ff668008040).stop 2021-11-21T22:00:03.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676934700 1 -- 172.21.2.116:0/2643181289 reap_dead start 2021-11-21T22:00:03.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2643181289 shutdown_connections 2021-11-21T22:00:03.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2643181289 >> 172.21.2.116:0/2643181289 conn(0x7ff670064d60 msgr2=0x7ff670065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:03.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2643181289 shutdown_connections 2021-11-21T22:00:03.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2643181289 wait complete. 2021-11-21T22:00:03.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 Processor -- start 2021-11-21T22:00:03.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- start start 2021-11-21T22:00:03.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff67014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6700add40 0x7ff670140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff670153ba0 0x7ff67009e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700aeed0 con 0x7ff6580049f0 2021-11-21T22:00:03.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700ae990 con 0x7ff670153ba0 2021-11-21T22:00:03.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700aec30 con 0x7ff6700add40 2021-11-21T22:00:03.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff67014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff67014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49184/0 (socket says 172.21.2.116:49184) 2021-11-21T22:00:03.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 -- 172.21.2.116:0/2636921518 learned_addr learned my addr 172.21.2.116:0/2636921518 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:03.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 -- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6700add40 msgr2=0x7ff670140f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:03.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6700add40 0x7ff670140f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 -- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff670153ba0 msgr2=0x7ff67009e2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:03.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff670153ba0 0x7ff67009e2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 -- 172.21.2.116:0/2636921518 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670153460 con 0x7ff6580049f0 2021-11-21T22:00:03.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff676133700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff67014fe10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7ff66c001fa0 tx=0x7ff66c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff65f7fe700 1 -- 172.21.2.116:0/2636921518 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66c00ce00 con 0x7ff6580049f0 2021-11-21T22:00:03.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff65f7fe700 1 -- 172.21.2.116:0/2636921518 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c000e90 con 0x7ff6580049f0 2021-11-21T22:00:03.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff670153700 con 0x7ff6580049f0 2021-11-21T22:00:03.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff65f7fe700 1 -- 172.21.2.116:0/2636921518 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66c007430 con 0x7ff6580049f0 2021-11-21T22:00:03.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6700ae530 con 0x7ff6580049f0 2021-11-21T22:00:03.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff670153890 con 0x7ff6580049f0 2021-11-21T22:00:03.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff65f7fe700 1 -- 172.21.2.116:0/2636921518 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff66c01a060 con 0x7ff6580049f0 2021-11-21T22:00:03.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff65f7fe700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff6500536a0 0x7ff650055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.214+0000 7ff65f7fe700 1 -- 172.21.2.116:0/2636921518 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff66c06b120 con 0x7ff6580049f0 2021-11-21T22:00:03.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.218+0000 7ff675932700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff6500536a0 0x7ff650055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.218+0000 7ff675932700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff6500536a0 0x7ff650055b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ff6700644a0 tx=0x7ff6600014e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.218+0000 7ff65f7fe700 1 -- 172.21.2.116:0/2636921518 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff66c068e90 con 0x7ff6580049f0 2021-11-21T22:00:03.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.326+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6700b7910 con 0x7ff6580049f0 2021-11-21T22:00:03.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff65f7fe700 1 -- 172.21.2.116:0/2636921518 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7ff66c0479d0 con 0x7ff6580049f0 2021-11-21T22:00:03.332 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:03.333 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:03.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff6500536a0 msgr2=0x7ff650055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff6500536a0 0x7ff650055b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ff6700644a0 tx=0x7ff6600014e0).stop 2021-11-21T22:00:03.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 msgr2=0x7ff67014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff67014fe10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7ff66c001fa0 tx=0x7ff66c007c30).stop 2021-11-21T22:00:03.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 shutdown_connections 2021-11-21T22:00:03.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff670153ba0 0x7ff67009e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff6700add40 0x7ff670140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff6500536a0 0x7ff650055b60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 --2- 172.21.2.116:0/2636921518 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff6580049f0 0x7ff67014fe10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 >> 172.21.2.116:0/2636921518 conn(0x7ff670064d60 msgr2=0x7ff6700b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:03.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 shutdown_connections 2021-11-21T22:00:03.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.330+0000 7ff678397700 1 -- 172.21.2.116:0/2636921518 wait complete. 2021-11-21T22:00:03.337 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:03.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:00:01.426149+0000', 'modified': '2021-11-21T22:00:02.429552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6282': {'gid': 6282, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3502880248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3502880248}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3502880248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:00:03.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:00:03.643 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:00:03.708 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":6282,"want_state":"up:active","state":"up:active","rank_uptime":2.2752855520000002,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":6.4788498260000003} 2021-11-21T22:00:03.708 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 6282, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 2.275285552, "state": "up:active", "uptime": 6.478849826, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:00:03.709 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:00:03.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 Processor -- start 2021-11-21T22:00:03.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 -- start start 2021-11-21T22:00:03.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f40ac0a6aa0 0x7f40ac14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 0x7f40ac1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40ac1475d0 0x7f40ac1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0a8bc0 con 0x7f40ac1475d0 2021-11-21T22:00:03.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0a6e80 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.754+0000 7f40b3c04700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac14a7b0 con 0x7f40ac0a86b0 2021-11-21T22:00:03.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40ac1475d0 0x7f40ac1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40ac1475d0 0x7f40ac1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49192/0 (socket says 172.21.2.116:49192) 2021-11-21T22:00:03.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 -- 172.21.2.116:0/409358248 learned_addr learned my addr 172.21.2.116:0/409358248 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:03.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 -- 172.21.2.116:0/409358248 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 msgr2=0x7f40ac1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 --2- 172.21.2.116:0/409358248 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 0x7f40ac1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 -- 172.21.2.116:0/409358248 >> v2:172.21.2.117:3300/0 conn(0x7f40ac0a6aa0 msgr2=0x7f40ac14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 --2- 172.21.2.116:0/409358248 >> v2:172.21.2.117:3300/0 conn(0x7f40ac0a6aa0 0x7f40ac14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 -- 172.21.2.116:0/409358248 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac14aa10 con 0x7f40ac1475d0 2021-11-21T22:00:03.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b21a1700 1 --2- 172.21.2.116:0/409358248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac1475d0 0x7f40ac1443f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f40a40032f0 tx=0x7f40a4009290).ready entity=mon.0 client_cookie=18b43cbfdf302a7e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b099e700 1 -- 172.21.2.116:0/409358248 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a400b870 con 0x7f40ac1475d0 2021-11-21T22:00:03.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b099e700 1 -- 172.21.2.116:0/409358248 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a400b9d0 con 0x7f40ac1475d0 2021-11-21T22:00:03.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b099e700 1 -- 172.21.2.116:0/409358248 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a400bb40 con 0x7f40ac1475d0 2021-11-21T22:00:03.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- 172.21.2.116:0/409358248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac1475d0 msgr2=0x7f40ac1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 --2- 172.21.2.116:0/409358248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac1475d0 0x7f40ac1443f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f40a40032f0 tx=0x7f40a4009290).stop 2021-11-21T22:00:03.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- 172.21.2.116:0/409358248 shutdown_connections 2021-11-21T22:00:03.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 --2- 172.21.2.116:0/409358248 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac1475d0 0x7f40ac1443f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 --2- 172.21.2.116:0/409358248 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 0x7f40ac1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 --2- 172.21.2.116:0/409358248 >> v2:172.21.2.117:3300/0 conn(0x7f40ac0a6aa0 0x7f40ac14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- 172.21.2.116:0/409358248 >> 172.21.2.116:0/409358248 conn(0x7f40ac05bd20 msgr2=0x7f40ac05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:03.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- 172.21.2.116:0/409358248 shutdown_connections 2021-11-21T22:00:03.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- 172.21.2.116:0/409358248 wait complete. 2021-11-21T22:00:03.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 Processor -- start 2021-11-21T22:00:03.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- start start 2021-11-21T22:00:03.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac0a6aa0 0x7f40ac142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 0x7f40ac13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40ac1475d0 0x7f40ac13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac146840 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac05adb0 con 0x7f40ac1475d0 2021-11-21T22:00:03.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac05d3b0 con 0x7f40ac0a86b0 2021-11-21T22:00:03.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac0a6aa0 0x7f40ac142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac0a6aa0 0x7f40ac142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49198/0 (socket says 172.21.2.116:49198) 2021-11-21T22:00:03.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 -- 172.21.2.116:0/2924923577 learned_addr learned my addr 172.21.2.116:0/2924923577 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:03.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 -- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 msgr2=0x7f40ac13ed00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:03.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 0x7f40ac13ed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 -- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40ac1475d0 msgr2=0x7f40ac13f1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:03.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40ac1475d0 0x7f40ac13f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 -- 172.21.2.116:0/2924923577 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4009530 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b19a0700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac0a6aa0 0x7f40ac142580 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f40ac146980 tx=0x7f40a0006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f409effd700 1 -- 172.21.2.116:0/2924923577 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a0013460 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f409effd700 1 -- 172.21.2.116:0/2924923577 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a00135c0 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac1471a0 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f409effd700 1 -- 172.21.2.116:0/2924923577 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a0013920 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.758+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac149780 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.762+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac0573d0 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.762+0000 7f409effd700 1 -- 172.21.2.116:0/2924923577 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f40a001a060 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.762+0000 7f409effd700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f408c053660 0x7f408c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.762+0000 7f409effd700 1 -- 172.21.2.116:0/2924923577 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f40a006b4a0 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.762+0000 7f40b119f700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f408c053660 0x7f408c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.762+0000 7f40b119f700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f408c053660 0x7f408c055b20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f40a8002f70 tx=0x7f40a8006980).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.762+0000 7f409effd700 1 -- 172.21.2.116:0/2924923577 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40a00424f0 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40ac147330 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f409effd700 1 -- 172.21.2.116:0/2924923577 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f40a0018030 con 0x7f40ac0a6aa0 2021-11-21T22:00:03.851 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:03.851 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":76,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:00:02.430725+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:59.269756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-11-21T22:00:00.412748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:00:03.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f408c053660 msgr2=0x7f408c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f408c053660 0x7f408c055b20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f40a8002f70 tx=0x7f40a8006980).stop 2021-11-21T22:00:03.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac0a6aa0 msgr2=0x7f40ac142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac0a6aa0 0x7f40ac142580 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f40ac146980 tx=0x7f40a0006ec0).stop 2021-11-21T22:00:03.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 shutdown_connections 2021-11-21T22:00:03.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40ac1475d0 0x7f40ac13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40ac0a86b0 0x7f40ac13ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f408c053660 0x7f408c055b20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 --2- 172.21.2.116:0/2924923577 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40ac0a6aa0 0x7f40ac142580 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 >> 172.21.2.116:0/2924923577 conn(0x7f40ac05bd20 msgr2=0x7f40ac0b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:03.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 shutdown_connections 2021-11-21T22:00:03.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.846+0000 7f40b3c04700 1 -- 172.21.2.116:0/2924923577 wait complete. 2021-11-21T22:00:03.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:59:59.269756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:00:00.412748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:00:03.859 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:00:03.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee49d6e700 1 Processor -- start 2021-11-21T22:00:03.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee49d6e700 1 -- start start 2021-11-21T22:00:03.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee49d6e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4405f730 0x7fee44141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee49d6e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee440b06c0 con 0x7fee440b02e0 2021-11-21T22:00:03.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee49d6e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee440adf20 con 0x7fee44142370 2021-11-21T22:00:03.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee49d6e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee440b2470 con 0x7fee4405f730 2021-11-21T22:00:03.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee437fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fee440b02e0 0x7fee440af620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:32932/0 (socket says 172.21.2.116:32932) 2021-11-21T22:00:03.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.906+0000 7fee437fe700 1 -- 172.21.2.116:0/1379835115 learned_addr learned my addr 172.21.2.116:0/1379835115 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:03.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531292329 0 0) 0x7fee440b06c0 con 0x7fee440b02e0 2021-11-21T22:00:03.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee24003260 con 0x7fee440b02e0 2021-11-21T22:00:03.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3724307793 0 0) 0x7fee24003260 con 0x7fee440b02e0 2021-11-21T22:00:03.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4405f730 0x7fee44141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4405f730 msgr2=0x7fee44141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4405f730 0x7fee44141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:03.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 >> v1:172.21.2.117:6789/0 conn(0x7fee44142370 legacy=0x7fee4414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee440b26d0 con 0x7fee440b02e0 2021-11-21T22:00:03.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fee34001660 con 0x7fee440b02e0 2021-11-21T22:00:03.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 >> v1:172.21.2.116:6789/0 conn(0x7fee440b02e0 legacy=0x7fee440af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee240037d0 0x7fee24003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee24004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4405f730 0x7fee2400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee440b2470 con 0x7fee24004500 2021-11-21T22:00:03.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee240052b0 con 0x7fee240037d0 2021-11-21T22:00:03.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2400bc40 con 0x7fee4405f730 2021-11-21T22:00:03.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee24004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 -- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4405f730 msgr2=0x7fee2400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4405f730 0x7fee2400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 -- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee240037d0 msgr2=0x7fee24003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:03.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee240037d0 0x7fee24003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 -- 172.21.2.116:0/1379835115 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24003260 con 0x7fee24004500 2021-11-21T22:00:03.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee42ffd700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee24004980 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7fee2c007000 tx=0x7fee2c008040).ready entity=mon.0 client_cookie=e8d9751995cba6da server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee2c0138b0 con 0x7fee24004500 2021-11-21T22:00:03.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00c030 con 0x7fee24004500 2021-11-21T22:00:03.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee427fc700 1 -- 172.21.2.116:0/1379835115 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee2c013a10 con 0x7fee24004500 2021-11-21T22:00:03.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- 172.21.2.116:0/1379835115 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 msgr2=0x7fee24004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:03.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 --2- 172.21.2.116:0/1379835115 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee24004980 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7fee2c007000 tx=0x7fee2c008040).stop 2021-11-21T22:00:03.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee43fff700 1 -- 172.21.2.116:0/1379835115 reap_dead start 2021-11-21T22:00:03.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- 172.21.2.116:0/1379835115 shutdown_connections 2021-11-21T22:00:03.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- 172.21.2.116:0/1379835115 >> 172.21.2.116:0/1379835115 conn(0x7fee440b5ff0 msgr2=0x7fee440b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:03.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- 172.21.2.116:0/1379835115 shutdown_connections 2021-11-21T22:00:03.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- 172.21.2.116:0/1379835115 wait complete. 2021-11-21T22:00:03.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 Processor -- start 2021-11-21T22:00:03.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- start start 2021-11-21T22:00:03.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee44151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4409f340 0x7fee44143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee440abbf0 0x7fee440ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee44145ed0 con 0x7fee24004500 2021-11-21T22:00:03.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4409f8c0 con 0x7fee440abbf0 2021-11-21T22:00:03.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4409fb60 con 0x7fee4409f340 2021-11-21T22:00:03.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee44151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee44151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49220/0 (socket says 172.21.2.116:49220) 2021-11-21T22:00:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 -- 172.21.2.116:0/3379027209 learned_addr learned my addr 172.21.2.116:0/3379027209 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 -- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4409f340 msgr2=0x7fee44143650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4409f340 0x7fee44143650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 -- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee440abbf0 msgr2=0x7fee440ac830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee440abbf0 0x7fee440ac830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 -- 172.21.2.116:0/3379027209 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4409fe00 con 0x7fee24004500 2021-11-21T22:00:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee437fe700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee44151190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fee440af500 tx=0x7fee440b2860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee40ff9700 1 -- 172.21.2.116:0/3379027209 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee34007170 con 0x7fee24004500 2021-11-21T22:00:03.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee40ff9700 1 -- 172.21.2.116:0/3379027209 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee340072d0 con 0x7fee24004500 2021-11-21T22:00:03.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee440ac390 con 0x7fee24004500 2021-11-21T22:00:03.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee40ff9700 1 -- 172.21.2.116:0/3379027209 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee34004920 con 0x7fee24004500 2021-11-21T22:00:03.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.910+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee440acec0 con 0x7fee24004500 2021-11-21T22:00:03.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.914+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee440ac520 con 0x7fee24004500 2021-11-21T22:00:03.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.914+0000 7fee40ff9700 1 -- 172.21.2.116:0/3379027209 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fee34004a80 con 0x7fee24004500 2021-11-21T22:00:03.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.914+0000 7fee40ff9700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee28053600 0x7fee28055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:03.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.914+0000 7fee40ff9700 1 -- 172.21.2.116:0/3379027209 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fee3406b610 con 0x7fee24004500 2021-11-21T22:00:03.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.914+0000 7fee42ffd700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee28053600 0x7fee28055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:03.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.914+0000 7fee42ffd700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee28053600 0x7fee28055ac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fee2c007000 tx=0x7fee2c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:03.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.914+0000 7fee40ff9700 1 -- 172.21.2.116:0/3379027209 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee34042660 con 0x7fee24004500 2021-11-21T22:00:04.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.998+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fee440afd30 con 0x7fee24004500 2021-11-21T22:00:04.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:03.998+0000 7fee40ff9700 1 -- 172.21.2.116:0/3379027209 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7fee34045c80 con 0x7fee24004500 2021-11-21T22:00:04.003 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:00:04.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee28053600 msgr2=0x7fee28055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee28053600 0x7fee28055ac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fee2c007000 tx=0x7fee2c008040).stop 2021-11-21T22:00:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 msgr2=0x7fee44151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee44151190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fee440af500 tx=0x7fee440b2860).stop 2021-11-21T22:00:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 shutdown_connections 2021-11-21T22:00:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fee440abbf0 0x7fee440ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fee4409f340 0x7fee44143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fee28053600 0x7fee28055ac0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 --2- 172.21.2.116:0/3379027209 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fee24004500 0x7fee44151190 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 >> 172.21.2.116:0/3379027209 conn(0x7fee440b5ff0 msgr2=0x7fee440b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 shutdown_connections 2021-11-21T22:00:04.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.002+0000 7fee49d6e700 1 -- 172.21.2.116:0/3379027209 wait complete. 2021-11-21T22:00:04.010 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:00:04.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.058+0000 7f198ff26700 1 Processor -- start 2021-11-21T22:00:04.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.058+0000 7f198ff26700 1 -- start start 2021-11-21T22:00:04.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.058+0000 7f198ff26700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1988144810 0x7f19881478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.058+0000 7f198ff26700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 0x7f19880b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.058+0000 7f198ff26700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1988147e80 con 0x7f19880b1db0 2021-11-21T22:00:04.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.058+0000 7f198ff26700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880b1490 con 0x7f1988144810 2021-11-21T22:00:04.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.058+0000 7f198ff26700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880af060 con 0x7f19880abce0 2021-11-21T22:00:04.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1988144810 0x7f19881478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1988144810 0x7f19881478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49230/0 (socket says 172.21.2.116:49230) 2021-11-21T22:00:04.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 -- 172.21.2.116:0/760342291 learned_addr learned my addr 172.21.2.116:0/760342291 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:04.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 -- 172.21.2.116:0/760342291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 msgr2=0x7f19880b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:04.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 --2- 172.21.2.116:0/760342291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 0x7f19880b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 -- 172.21.2.116:0/760342291 >> v1:172.21.2.117:6789/0 conn(0x7f19880b1db0 legacy=0x7f19880ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:04.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 -- 172.21.2.116:0/760342291 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880af5a0 con 0x7f1988144810 2021-11-21T22:00:04.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198dcc2700 1 --2- 172.21.2.116:0/760342291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1988144810 0x7f19881478c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f19880b2a30 tx=0x7f1984006eb0).ready entity=mon.0 client_cookie=c649013cacbfacde server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ccc0700 1 -- 172.21.2.116:0/760342291 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f198400d570 con 0x7f1988144810 2021-11-21T22:00:04.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ccc0700 1 -- 172.21.2.116:0/760342291 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984000e90 con 0x7f1988144810 2021-11-21T22:00:04.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ccc0700 1 -- 172.21.2.116:0/760342291 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f198400d8c0 con 0x7f1988144810 2021-11-21T22:00:04.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- 172.21.2.116:0/760342291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1988144810 msgr2=0x7f19881478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 --2- 172.21.2.116:0/760342291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1988144810 0x7f19881478c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f19880b2a30 tx=0x7f1984006eb0).stop 2021-11-21T22:00:04.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- 172.21.2.116:0/760342291 shutdown_connections 2021-11-21T22:00:04.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 --2- 172.21.2.116:0/760342291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 0x7f19880b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 --2- 172.21.2.116:0/760342291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1988144810 0x7f19881478c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- 172.21.2.116:0/760342291 >> 172.21.2.116:0/760342291 conn(0x7f198805bd20 msgr2=0x7f198805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- 172.21.2.116:0/760342291 shutdown_connections 2021-11-21T22:00:04.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- 172.21.2.116:0/760342291 wait complete. 2021-11-21T22:00:04.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 Processor -- start 2021-11-21T22:00:04.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- start start 2021-11-21T22:00:04.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 0x7f198813f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1988144810 0x7f19880b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19880b36a0 0x7f1988159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988063de0 con 0x7f19880b36a0 2021-11-21T22:00:04.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880b3b50 con 0x7f1988144810 2021-11-21T22:00:04.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1988067400 con 0x7f19880abce0 2021-11-21T22:00:04.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19880b36a0 0x7f1988159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19880b36a0 0x7f1988159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49234/0 (socket says 172.21.2.116:49234) 2021-11-21T22:00:04.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 -- 172.21.2.116:0/458512436 learned_addr learned my addr 172.21.2.116:0/458512436 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:04.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 -- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 msgr2=0x7f198813f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 0x7f198813f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 -- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1988144810 msgr2=0x7f19880b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1988144810 0x7f19880b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 -- 172.21.2.116:0/458512436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198400b1e0 con 0x7f19880b36a0 2021-11-21T22:00:04.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198e4c3700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19880b36a0 0x7f1988159cb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f198813f8a0 tx=0x7f1980009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f197affd700 1 -- 172.21.2.116:0/458512436 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1980001500 con 0x7f19880b36a0 2021-11-21T22:00:04.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f197affd700 1 -- 172.21.2.116:0/458512436 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980001660 con 0x7f19880b36a0 2021-11-21T22:00:04.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f197affd700 1 -- 172.21.2.116:0/458512436 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19800019c0 con 0x7f19880b36a0 2021-11-21T22:00:04.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19880b2ab0 con 0x7f19880b36a0 2021-11-21T22:00:04.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.062+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19880b2d10 con 0x7f19880b36a0 2021-11-21T22:00:04.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.066+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19880b2ed0 con 0x7f19880b36a0 2021-11-21T22:00:04.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.066+0000 7f197affd700 1 -- 172.21.2.116:0/458512436 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1980018030 con 0x7f19880b36a0 2021-11-21T22:00:04.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.066+0000 7f197affd700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1968053660 0x7f1968055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.066+0000 7f197affd700 1 -- 172.21.2.116:0/458512436 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f198006b410 con 0x7f19880b36a0 2021-11-21T22:00:04.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.066+0000 7f198dcc2700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1968053660 0x7f1968055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.066+0000 7f198dcc2700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1968053660 0x7f1968055b20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f19880b2a30 tx=0x7f1984015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.066+0000 7f197affd700 1 -- 172.21.2.116:0/458512436 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1980068e90 con 0x7f19880b36a0 2021-11-21T22:00:04.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1988144ec0 con 0x7f19880b36a0 2021-11-21T22:00:04.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f197affd700 1 -- 172.21.2.116:0/458512436 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1980042510 con 0x7f19880b36a0 2021-11-21T22:00:04.154 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:04.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1968053660 msgr2=0x7f1968055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1968053660 0x7f1968055b20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f19880b2a30 tx=0x7f1984015000).stop 2021-11-21T22:00:04.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19880b36a0 msgr2=0x7f1988159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19880b36a0 0x7f1988159cb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f198813f8a0 tx=0x7f1980009120).stop 2021-11-21T22:00:04.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 shutdown_connections 2021-11-21T22:00:04.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1968053660 0x7f1968055b20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19880b36a0 0x7f1988159cb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1988144810 0x7f19880b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 --2- 172.21.2.116:0/458512436 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19880abce0 0x7f198813f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 >> 172.21.2.116:0/458512436 conn(0x7f198805bd20 msgr2=0x7f1988149de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 shutdown_connections 2021-11-21T22:00:04.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.150+0000 7f198ff26700 1 -- 172.21.2.116:0/458512436 wait complete. 2021-11-21T22:00:04.162 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:00:04.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.210+0000 7f0e9e4e1700 1 Processor -- start 2021-11-21T22:00:04.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.210+0000 7f0e9e4e1700 1 -- start start 2021-11-21T22:00:04.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.210+0000 7f0e9e4e1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980b02e0 0x7f0e9805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.210+0000 7f0e9e4e1700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0e980af620 0x7f0e98142f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.210+0000 7f0e9e4e1700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e980b06c0 con 0x7f0e980afa80 2021-11-21T22:00:04.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.210+0000 7f0e9e4e1700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e980adf20 con 0x7f0e980af620 2021-11-21T22:00:04.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.210+0000 7f0e9e4e1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980b2470 con 0x7f0e980b02e0 2021-11-21T22:00:04.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e977fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0e980af620 0x7f0e98142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980b02e0 0x7f0e9805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e977fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0e980af620 0x7f0e98142f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:50410/0 (socket says 172.21.2.116:50410) 2021-11-21T22:00:04.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e977fe700 1 -- 172.21.2.116:0/1803341944 learned_addr learned my addr 172.21.2.116:0/1803341944 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:04.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 -- 172.21.2.116:0/1803341944 >> v2:172.21.2.117:3300/0 conn(0x7f0e980af620 msgr2=0x7f0e98142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:04.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- 172.21.2.116:0/1803341944 >> v2:172.21.2.117:3300/0 conn(0x7f0e980af620 0x7f0e98142f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 -- 172.21.2.116:0/1803341944 >> v1:172.21.2.117:6789/0 conn(0x7f0e980afa80 legacy=0x7f0e98143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 -- 172.21.2.116:0/1803341944 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980b26d0 con 0x7f0e980b02e0 2021-11-21T22:00:04.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- 172.21.2.116:0/1803341944 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980b02e0 0x7f0e9805f730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0e88002f70 tx=0x7f0e88006df0).ready entity=mon.2 client_cookie=9907e3ccc5dc28c6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e96ffd700 1 -- 172.21.2.116:0/1803341944 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e8800d570 con 0x7f0e980b02e0 2021-11-21T22:00:04.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e96ffd700 1 -- 172.21.2.116:0/1803341944 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e88000e90 con 0x7f0e980b02e0 2021-11-21T22:00:04.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e96ffd700 1 -- 172.21.2.116:0/1803341944 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e8800d8c0 con 0x7f0e980b02e0 2021-11-21T22:00:04.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/1803341944 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980b02e0 msgr2=0x7f0e9805f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/1803341944 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980b02e0 0x7f0e9805f730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0e88002f70 tx=0x7f0e88006df0).stop 2021-11-21T22:00:04.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/1803341944 shutdown_connections 2021-11-21T22:00:04.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/1803341944 >> v2:172.21.2.117:3300/0 conn(0x7f0e980af620 0x7f0e98142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/1803341944 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980b02e0 0x7f0e9805f730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/1803341944 >> 172.21.2.116:0/1803341944 conn(0x7f0e980b5ff0 msgr2=0x7f0e980b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/1803341944 shutdown_connections 2021-11-21T22:00:04.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/1803341944 wait complete. 2021-11-21T22:00:04.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 Processor -- start 2021-11-21T22:00:04.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- start start 2021-11-21T22:00:04.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e980af620 0x7f0e98155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980afa80 0x7f0e9814a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e980b02e0 0x7f0e981527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98066400 con 0x7f0e980af620 2021-11-21T22:00:04.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98065a50 con 0x7f0e980b02e0 2021-11-21T22:00:04.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98066630 con 0x7f0e980afa80 2021-11-21T22:00:04.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e980af620 0x7f0e98155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e980af620 0x7f0e98155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49246/0 (socket says 172.21.2.116:49246) 2021-11-21T22:00:04.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 -- 172.21.2.116:0/670614097 learned_addr learned my addr 172.21.2.116:0/670614097 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:04.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9ca7e700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e980b02e0 0x7f0e981527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e977fe700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980afa80 0x7f0e9814a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 -- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980afa80 msgr2=0x7f0e9814a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980afa80 0x7f0e9814a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 -- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e980b02e0 msgr2=0x7f0e981527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e980b02e0 0x7f0e981527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 -- 172.21.2.116:0/670614097 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8800b180 con 0x7f0e980af620 2021-11-21T22:00:04.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e977fe700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980afa80 0x7f0e9814a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:00:04.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e97fff700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e980af620 0x7f0e98155600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f0e88002f70 tx=0x7f0e88014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e957fa700 1 -- 172.21.2.116:0/670614097 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e88016070 con 0x7f0e980af620 2021-11-21T22:00:04.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e957fa700 1 -- 172.21.2.116:0/670614097 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8800de40 con 0x7f0e980af620 2021-11-21T22:00:04.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e980668d0 con 0x7f0e980af620 2021-11-21T22:00:04.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e957fa700 1 -- 172.21.2.116:0/670614097 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e88013e90 con 0x7f0e980af620 2021-11-21T22:00:04.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.214+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e98152310 con 0x7f0e980af620 2021-11-21T22:00:04.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.218+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e98152510 con 0x7f0e980af620 2021-11-21T22:00:04.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.218+0000 7f0e957fa700 1 -- 172.21.2.116:0/670614097 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0e88025030 con 0x7f0e980af620 2021-11-21T22:00:04.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.218+0000 7f0e957fa700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e7c0534f0 0x7f0e7c0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.218+0000 7f0e957fa700 1 -- 172.21.2.116:0/670614097 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0e8806c850 con 0x7f0e980af620 2021-11-21T22:00:04.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.218+0000 7f0e977fe700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e7c0534f0 0x7f0e7c0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.218+0000 7f0e977fe700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e7c0534f0 0x7f0e7c0559b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f0e8000d000 tx=0x7f0e80006f40).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.218+0000 7f0e957fa700 1 -- 172.21.2.116:0/670614097 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e98152510 con 0x7f0e980af620 2021-11-21T22:00:04.236 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:04.239 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:04.241 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:04.244 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:04.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.302+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e98152510 con 0x7f0e980af620 2021-11-21T22:00:04.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.302+0000 7f0e957fa700 1 -- 172.21.2.116:0/670614097 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0e88023020 con 0x7f0e980af620 2021-11-21T22:00:04.308 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:04.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.302+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e7c0534f0 msgr2=0x7f0e7c0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.302+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e7c0534f0 0x7f0e7c0559b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f0e8000d000 tx=0x7f0e80006f40).stop 2021-11-21T22:00:04.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.302+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e980af620 msgr2=0x7f0e98155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.302+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e980af620 0x7f0e98155600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f0e88002f70 tx=0x7f0e88014000).stop 2021-11-21T22:00:04.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 shutdown_connections 2021-11-21T22:00:04.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e980b02e0 0x7f0e981527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e980afa80 0x7f0e9814a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e7c0534f0 0x7f0e7c0559b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 --2- 172.21.2.116:0/670614097 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e980af620 0x7f0e98155600 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 >> 172.21.2.116:0/670614097 conn(0x7f0e980b5ff0 msgr2=0x7f0e9805fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 shutdown_connections 2021-11-21T22:00:04.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.306+0000 7f0e9e4e1700 1 -- 172.21.2.116:0/670614097 wait complete. 2021-11-21T22:00:04.314 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:00:04.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefefeb700 1 Processor -- start 2021-11-21T22:00:04.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefefeb700 1 -- start start 2021-11-21T22:00:04.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefefeb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faef8144810 0x7faef81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefefeb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80ac600 0x7faef80aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefefeb700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef8147e80 con 0x7faef80abce0 2021-11-21T22:00:04.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefefeb700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef80b1490 con 0x7faef8144810 2021-11-21T22:00:04.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefefeb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef80b1fd0 con 0x7faef80ac600 2021-11-21T22:00:04.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefcd87700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faef8144810 0x7faef81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faef7fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7faef80abce0 0x7faef80b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:32978/0 (socket says 172.21.2.116:32978) 2021-11-21T22:00:04.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faef7fff700 1 -- 172.21.2.116:0/2461025312 learned_addr learned my addr 172.21.2.116:0/2461025312 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:04.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefcd87700 1 -- 172.21.2.116:0/2461025312 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80ac600 msgr2=0x7faef80aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:04.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefcd87700 1 --2- 172.21.2.116:0/2461025312 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80ac600 0x7faef80aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefcd87700 1 -- 172.21.2.116:0/2461025312 >> v1:172.21.2.116:6789/0 conn(0x7faef80abce0 legacy=0x7faef80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.362+0000 7faefcd87700 1 -- 172.21.2.116:0/2461025312 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80af5a0 con 0x7faef8144810 2021-11-21T22:00:04.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefcd87700 1 --2- 172.21.2.116:0/2461025312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef8144810 0x7faef81478c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7faeec002f70 tx=0x7faeec006df0).ready entity=mon.0 client_cookie=b6b6c47fd765159c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef77fe700 1 -- 172.21.2.116:0/2461025312 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeec00d570 con 0x7faef8144810 2021-11-21T22:00:04.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef77fe700 1 -- 172.21.2.116:0/2461025312 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec000e90 con 0x7faef8144810 2021-11-21T22:00:04.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef77fe700 1 -- 172.21.2.116:0/2461025312 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeec00d8c0 con 0x7faef8144810 2021-11-21T22:00:04.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/2461025312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef8144810 msgr2=0x7faef81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 --2- 172.21.2.116:0/2461025312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef8144810 0x7faef81478c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7faeec002f70 tx=0x7faeec006df0).stop 2021-11-21T22:00:04.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/2461025312 shutdown_connections 2021-11-21T22:00:04.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 --2- 172.21.2.116:0/2461025312 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80ac600 0x7faef80aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 --2- 172.21.2.116:0/2461025312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef8144810 0x7faef81478c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/2461025312 >> 172.21.2.116:0/2461025312 conn(0x7faef805bd20 msgr2=0x7faef805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/2461025312 shutdown_connections 2021-11-21T22:00:04.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/2461025312 wait complete. 2021-11-21T22:00:04.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 Processor -- start 2021-11-21T22:00:04.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- start start 2021-11-21T22:00:04.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80abce0 0x7faef8143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef80ac600 0x7faef8159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faef8144810 0x7faef815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8145360 con 0x7faef80ac600 2021-11-21T22:00:04.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8156570 con 0x7faef8144810 2021-11-21T22:00:04.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef8149820 con 0x7faef80abce0 2021-11-21T22:00:04.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef80ac600 0x7faef8159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef80ac600 0x7faef8159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49264/0 (socket says 172.21.2.116:49264) 2021-11-21T22:00:04.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 -- 172.21.2.116:0/484080752 learned_addr learned my addr 172.21.2.116:0/484080752 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:04.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 -- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80abce0 msgr2=0x7faef8143670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80abce0 0x7faef8143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 -- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faef8144810 msgr2=0x7faef815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faef8144810 0x7faef815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 -- 172.21.2.116:0/484080752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec00b180 con 0x7faef80ac600 2021-11-21T22:00:04.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef7fff700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef80ac600 0x7faef8159ec0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7faef80b4e00 tx=0x7faee8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef5ffb700 1 -- 172.21.2.116:0/484080752 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee800ce30 con 0x7faef80ac600 2021-11-21T22:00:04.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef5ffb700 1 -- 172.21.2.116:0/484080752 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee80072e0 con 0x7faef80ac600 2021-11-21T22:00:04.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef5ffb700 1 -- 172.21.2.116:0/484080752 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee8005800 con 0x7faef80ac600 2021-11-21T22:00:04.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8149ac0 con 0x7faef80ac600 2021-11-21T22:00:04.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef81494a0 con 0x7faef80ac600 2021-11-21T22:00:04.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef8149f60 con 0x7faef80ac600 2021-11-21T22:00:04.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef5ffb700 1 -- 172.21.2.116:0/484080752 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7faee8010070 con 0x7faef80ac600 2021-11-21T22:00:04.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef5ffb700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faed8053610 0x7faed8055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.366+0000 7faef5ffb700 1 -- 172.21.2.116:0/484080752 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faee806b090 con 0x7faef80ac600 2021-11-21T22:00:04.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.370+0000 7faefcd87700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faed8053610 0x7faed8055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.370+0000 7faefcd87700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faed8053610 0x7faed8055ad0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7faeec002f70 tx=0x7faeec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.370+0000 7faef5ffb700 1 -- 172.21.2.116:0/484080752 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faee80420e0 con 0x7faef80ac600 2021-11-21T22:00:04.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faef80af060 con 0x7faef80ac600 2021-11-21T22:00:04.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faef5ffb700 1 -- 172.21.2.116:0/484080752 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7faee8018030 con 0x7faef80ac600 2021-11-21T22:00:04.457 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:04.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faed8053610 msgr2=0x7faed8055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faed8053610 0x7faed8055ad0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7faeec002f70 tx=0x7faeec015000).stop 2021-11-21T22:00:04.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef80ac600 msgr2=0x7faef8159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef80ac600 0x7faef8159ec0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7faef80b4e00 tx=0x7faee8007c40).stop 2021-11-21T22:00:04.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 shutdown_connections 2021-11-21T22:00:04.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faef8144810 0x7faef815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faed8053610 0x7faed8055ad0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faef80ac600 0x7faef8159ec0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 --2- 172.21.2.116:0/484080752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faef80abce0 0x7faef8143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 >> 172.21.2.116:0/484080752 conn(0x7faef805bd20 msgr2=0x7faef80ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 shutdown_connections 2021-11-21T22:00:04.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.454+0000 7faefefeb700 1 -- 172.21.2.116:0/484080752 wait complete. 2021-11-21T22:00:04.464 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:00:04.464 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:04.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 Processor -- start 2021-11-21T22:00:04.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- start start 2021-11-21T22:00:04.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f461c0a6aa0 0x7f461c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a86b0 0x7f461c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0a6e80 con 0x7f461c1475d0 2021-11-21T22:00:04.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c0a8bc0 con 0x7f461c0a6aa0 2021-11-21T22:00:04.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c14a7b0 con 0x7f461c0a86b0 2021-11-21T22:00:04.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f461c0a6aa0 0x7f461c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f461c0a6aa0 0x7f461c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49268/0 (socket says 172.21.2.116:49268) 2021-11-21T22:00:04.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 -- 172.21.2.116:0/3351531552 learned_addr learned my addr 172.21.2.116:0/3351531552 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:04.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 -- 172.21.2.116:0/3351531552 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a86b0 msgr2=0x7f461c1470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:04.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 --2- 172.21.2.116:0/3351531552 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a86b0 0x7f461c1470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 -- 172.21.2.116:0/3351531552 >> v1:172.21.2.117:6789/0 conn(0x7f461c1475d0 legacy=0x7f461c1443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:04.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 -- 172.21.2.116:0/3351531552 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c14aa10 con 0x7f461c0a6aa0 2021-11-21T22:00:04.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f46217bd700 1 --2- 172.21.2.116:0/3351531552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a6aa0 0x7f461c14b150 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f461c061ab0 tx=0x7f4618006eb0).ready entity=mon.0 client_cookie=db6ce023242e6654 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f460ffff700 1 -- 172.21.2.116:0/3351531552 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f461800d570 con 0x7f461c0a6aa0 2021-11-21T22:00:04.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f460ffff700 1 -- 172.21.2.116:0/3351531552 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618000e90 con 0x7f461c0a6aa0 2021-11-21T22:00:04.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f460ffff700 1 -- 172.21.2.116:0/3351531552 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f461800d8c0 con 0x7f461c0a6aa0 2021-11-21T22:00:04.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- 172.21.2.116:0/3351531552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a6aa0 msgr2=0x7f461c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- 172.21.2.116:0/3351531552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a6aa0 0x7f461c14b150 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f461c061ab0 tx=0x7f4618006eb0).stop 2021-11-21T22:00:04.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- 172.21.2.116:0/3351531552 shutdown_connections 2021-11-21T22:00:04.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- 172.21.2.116:0/3351531552 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a86b0 0x7f461c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- 172.21.2.116:0/3351531552 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a6aa0 0x7f461c14b150 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- 172.21.2.116:0/3351531552 >> 172.21.2.116:0/3351531552 conn(0x7f461c05bd20 msgr2=0x7f461c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- 172.21.2.116:0/3351531552 shutdown_connections 2021-11-21T22:00:04.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- 172.21.2.116:0/3351531552 wait complete. 2021-11-21T22:00:04.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 Processor -- start 2021-11-21T22:00:04.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- start start 2021-11-21T22:00:04.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a6aa0 0x7f461c065da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a86b0 0x7f461c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f461c1475d0 0x7f461c0a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c065be0 con 0x7f461c0a86b0 2021-11-21T22:00:04.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c156510 con 0x7f461c1475d0 2021-11-21T22:00:04.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c1567b0 con 0x7f461c0a6aa0 2021-11-21T22:00:04.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a86b0 0x7f461c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a86b0 0x7f461c156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49276/0 (socket says 172.21.2.116:49276) 2021-11-21T22:00:04.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 -- 172.21.2.116:0/425889737 learned_addr learned my addr 172.21.2.116:0/425889737 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:04.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 -- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a6aa0 msgr2=0x7f461c065da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:04.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a6aa0 0x7f461c065da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 -- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f461c1475d0 msgr2=0x7f461c0a8ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:04.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f461c1475d0 0x7f461c0a8ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 -- 172.21.2.116:0/425889737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461800b1e0 con 0x7f461c0a86b0 2021-11-21T22:00:04.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4620fbc700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a86b0 0x7f461c156ef0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f4610002fb0 tx=0x7f4610006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f460e7fc700 1 -- 172.21.2.116:0/425889737 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4610013460 con 0x7f461c0a86b0 2021-11-21T22:00:04.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f460e7fc700 1 -- 172.21.2.116:0/425889737 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100135c0 con 0x7f461c0a86b0 2021-11-21T22:00:04.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c156a50 con 0x7f461c0a86b0 2021-11-21T22:00:04.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f460e7fc700 1 -- 172.21.2.116:0/425889737 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4610013920 con 0x7f461c0a86b0 2021-11-21T22:00:04.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.514+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c146a10 con 0x7f461c0a86b0 2021-11-21T22:00:04.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.518+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c0573d0 con 0x7f461c0a86b0 2021-11-21T22:00:04.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.518+0000 7f460e7fc700 1 -- 172.21.2.116:0/425889737 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f461000f070 con 0x7f461c0a86b0 2021-11-21T22:00:04.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.518+0000 7f460e7fc700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45fc053610 0x7f45fc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:04.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.518+0000 7f460e7fc700 1 -- 172.21.2.116:0/425889737 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f461006b4c0 con 0x7f461c0a86b0 2021-11-21T22:00:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.518+0000 7f46217bd700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45fc053610 0x7f45fc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.518+0000 7f46217bd700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45fc053610 0x7f45fc055ad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f461c061ab0 tx=0x7f4618015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.518+0000 7f460e7fc700 1 -- 172.21.2.116:0/425889737 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4610042510 con 0x7f461c0a86b0 2021-11-21T22:00:04.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.626+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f461c156be0 con 0x7f461c0a86b0 2021-11-21T22:00:04.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f460e7fc700 1 -- 172.21.2.116:0/425889737 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f4610045b30 con 0x7f461c0a86b0 2021-11-21T22:00:04.632 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:04.633 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:04.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45fc053610 msgr2=0x7f45fc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45fc053610 0x7f45fc055ad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f461c061ab0 tx=0x7f4618015000).stop 2021-11-21T22:00:04.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a86b0 msgr2=0x7f461c156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:04.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a86b0 0x7f461c156ef0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f4610002fb0 tx=0x7f4610006de0).stop 2021-11-21T22:00:04.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 shutdown_connections 2021-11-21T22:00:04.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f461c1475d0 0x7f461c0a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45fc053610 0x7f45fc055ad0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f461c0a86b0 0x7f461c156ef0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:04.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 --2- 172.21.2.116:0/425889737 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f461c0a6aa0 0x7f461c065da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:04.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 >> 172.21.2.116:0/425889737 conn(0x7f461c05bd20 msgr2=0x7f461c149b00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:04.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 shutdown_connections 2021-11-21T22:00:04.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:04.630+0000 7f4623a21700 1 -- 172.21.2.116:0/425889737 wait complete. 2021-11-21T22:00:04.637 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:04.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:00:01.426149+0000', 'modified': '2021-11-21T22:00:02.429552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6282': {'gid': 6282, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3502880248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3502880248}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3502880248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:00:04.944 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:00:04.944 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:00:05.009 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":6282,"want_state":"up:active","state":"up:active","rank_uptime":3.576477041,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":7.7800407849999997} 2021-11-21T22:00:05.010 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 6282, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 3.576477041, "state": "up:active", "uptime": 7.780040785, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:00:05.010 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:00:05.011 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:00:05.011 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:00:05.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:05.012 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:05.023 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:00:05.077 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:05.078 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:05.085 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:05.087 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:05.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:05.088 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:00:05.133 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:00:05.134 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:05.137 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:00:05.146 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:05.147 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:05.152 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:05.154 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:05.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:05.155 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:00:05.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14614]: starting ceph client 2021-11-21T22:00:05.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:00:05.173+0000 7fcc90e8b080 -1 init, newargv = 0x55a265856080 newargc=9 2021-11-21T22:00:05.187 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14614]: starting fuse 2021-11-21T22:00:06.260 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:00:06.273 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:06.274 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:06.324 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:06.325 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:06.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:06.327 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:00:06.332 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:00:06.332 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:00:06.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:06.334 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:00:06.334 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:00:06.335 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:00:06.335 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:00:06.336 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:00:06.336 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:06.337 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:00:06.337 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:00:06.338 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:00:06.338 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:06.338 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:00:06.339 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:00:06.339 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:00:06.340 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:00:06.340 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:06.341 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:00:06.341 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:00:06.342 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:00:06.342 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:00:06.343 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:00:06.344 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:06.344 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:00:06.345 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:00:06.374 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14614.asok 2021-11-21T22:00:06.377 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14614.asok 2021-11-21T22:00:06.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:06.378 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14614.asok status 2021-11-21T22:00:06.469 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:00:06.470 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:00:06.470 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:00:06.470 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:00:06.470 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:00:06.471 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:00:06.471 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:00:06.471 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14614", 2021-11-21T22:00:06.471 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:00:06.471 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:06.472 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:00:06.472 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:00:06.472 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4607, 2021-11-21T22:00:06.472 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:00:06.473 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:00:06.473 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:00:06.473 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4607 2021-11-21T22:00:06.473 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:06.474 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:00:06.474 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:00:06.474 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:00:06.474 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1262068513 2021-11-21T22:00:06.474 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:00:06.475 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:06.475 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:00:06.475 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:00:06.475 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:00:06.476 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1262068513 2021-11-21T22:00:06.476 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:06.476 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4607 172.21.2.119:0/1262068513", 2021-11-21T22:00:06.476 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/1262068513", 2021-11-21T22:00:06.477 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:00:06.477 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 33, 2021-11-21T22:00:06.477 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 76, 2021-11-21T22:00:06.477 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:00:06.477 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:00:06.478 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:00:06.478 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:00:06.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:06.480 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:06.480 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:06.489 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:00:06.490 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:00:06.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:06.491 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:06.511 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:00:06.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 Processor -- start 2021-11-21T22:00:06.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 -- start start 2021-11-21T22:00:06.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f02340a6aa0 0x7f023414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f02340a86b0 0x7f02341470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02341475d0 0x7f02341443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02340a8bc0 con 0x7f02340a6aa0 2021-11-21T22:00:06.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02340a6e80 con 0x7f02340a86b0 2021-11-21T22:00:06.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023c5ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023414a7b0 con 0x7f02341475d0 2021-11-21T22:00:06.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f02340a6aa0 0x7f023414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f02340a6aa0 0x7f023414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49282/0 (socket says 172.21.2.116:49282) 2021-11-21T22:00:06.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 -- 172.21.2.116:0/3235098126 learned_addr learned my addr 172.21.2.116:0/3235098126 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:06.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 -- 172.21.2.116:0/3235098126 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02341475d0 msgr2=0x7f02341443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:06.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 --2- 172.21.2.116:0/3235098126 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02341475d0 0x7f02341443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 -- 172.21.2.116:0/3235098126 >> v2:172.21.2.117:3300/0 conn(0x7f02340a86b0 msgr2=0x7f02341470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:06.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 --2- 172.21.2.116:0/3235098126 >> v2:172.21.2.117:3300/0 conn(0x7f02340a86b0 0x7f02341470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.562+0000 7f023a366700 1 -- 172.21.2.116:0/3235098126 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023414aa10 con 0x7f02340a6aa0 2021-11-21T22:00:06.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 --2- 172.21.2.116:0/3235098126 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f023414b150 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f0230002f70 tx=0x7f0230006eb0).ready entity=mon.0 client_cookie=b68dc8c2d2ba99d0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f0239364700 1 -- 172.21.2.116:0/3235098126 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f023000d570 con 0x7f02340a6aa0 2021-11-21T22:00:06.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f0239364700 1 -- 172.21.2.116:0/3235098126 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0230000e90 con 0x7f02340a6aa0 2021-11-21T22:00:06.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f0239364700 1 -- 172.21.2.116:0/3235098126 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f023000d900 con 0x7f02340a6aa0 2021-11-21T22:00:06.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/3235098126 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 msgr2=0x7f023414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 --2- 172.21.2.116:0/3235098126 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f023414b150 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f0230002f70 tx=0x7f0230006eb0).stop 2021-11-21T22:00:06.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/3235098126 shutdown_connections 2021-11-21T22:00:06.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 --2- 172.21.2.116:0/3235098126 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02341475d0 0x7f02341443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 --2- 172.21.2.116:0/3235098126 >> v2:172.21.2.117:3300/0 conn(0x7f02340a86b0 0x7f02341470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 --2- 172.21.2.116:0/3235098126 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f023414b150 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/3235098126 >> 172.21.2.116:0/3235098126 conn(0x7f023405bd20 msgr2=0x7f023405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:06.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/3235098126 shutdown_connections 2021-11-21T22:00:06.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/3235098126 wait complete. 2021-11-21T22:00:06.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 Processor -- start 2021-11-21T22:00:06.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- start start 2021-11-21T22:00:06.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f0234064730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02340a86b0 0x7f02340a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02341475d0 0x7f02340a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234156610 con 0x7f02340a6aa0 2021-11-21T22:00:06.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02341568b0 con 0x7f02341475d0 2021-11-21T22:00:06.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0234156b50 con 0x7f02340a86b0 2021-11-21T22:00:06.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f0234064730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f0234064730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49288/0 (socket says 172.21.2.116:49288) 2021-11-21T22:00:06.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 -- 172.21.2.116:0/1416875464 learned_addr learned my addr 172.21.2.116:0/1416875464 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:06.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 -- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02340a86b0 msgr2=0x7f02340a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02340a86b0 0x7f02340a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 -- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02341475d0 msgr2=0x7f02340a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02341475d0 0x7f02340a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 -- 172.21.2.116:0/1416875464 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023000b1e0 con 0x7f02340a6aa0 2021-11-21T22:00:06.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023a366700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f0234064730 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f0230002f70 tx=0x7f0230015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f02277fe700 1 -- 172.21.2.116:0/1416875464 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0230011790 con 0x7f02340a6aa0 2021-11-21T22:00:06.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f02277fe700 1 -- 172.21.2.116:0/1416875464 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02300118f0 con 0x7f02340a6aa0 2021-11-21T22:00:06.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0234156300 con 0x7f02340a6aa0 2021-11-21T22:00:06.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f02277fe700 1 -- 172.21.2.116:0/1416875464 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0230011c90 con 0x7f02340a6aa0 2021-11-21T22:00:06.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0234156ff0 con 0x7f02340a6aa0 2021-11-21T22:00:06.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.566+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0234156d10 con 0x7f02340a6aa0 2021-11-21T22:00:06.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.570+0000 7f02277fe700 1 -- 172.21.2.116:0/1416875464 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0230025030 con 0x7f02340a6aa0 2021-11-21T22:00:06.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.570+0000 7f02277fe700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0214053690 0x7f0214055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.570+0000 7f02277fe700 1 -- 172.21.2.116:0/1416875464 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f023006cbc0 con 0x7f02340a6aa0 2021-11-21T22:00:06.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.570+0000 7f0239b65700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0214053690 0x7f0214055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.570+0000 7f0239b65700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0214053690 0x7f0214055b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f0228002fb0 tx=0x7f02280069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.570+0000 7f02277fe700 1 -- 172.21.2.116:0/1416875464 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0230043d20 con 0x7f02340a6aa0 2021-11-21T22:00:06.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02340573d0 con 0x7f02340a6aa0 2021-11-21T22:00:06.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f02277fe700 1 -- 172.21.2.116:0/1416875464 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f0230023090 con 0x7f02340a6aa0 2021-11-21T22:00:06.657 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:06.657 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":76,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:00:02.430725+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:59.269756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-11-21T22:00:00.412748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:00:06.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0214053690 msgr2=0x7f0214055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0214053690 0x7f0214055b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f0228002fb0 tx=0x7f02280069b0).stop 2021-11-21T22:00:06.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 msgr2=0x7f0234064730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f0234064730 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f0230002f70 tx=0x7f0230015000).stop 2021-11-21T22:00:06.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 shutdown_connections 2021-11-21T22:00:06.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02341475d0 0x7f02340a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02340a86b0 0x7f02340a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0214053690 0x7f0214055b50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 --2- 172.21.2.116:0/1416875464 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02340a6aa0 0x7f0234064730 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 >> 172.21.2.116:0/1416875464 conn(0x7f023405bd20 msgr2=0x7f023413f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:06.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 shutdown_connections 2021-11-21T22:00:06.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.654+0000 7f023c5ca700 1 -- 172.21.2.116:0/1416875464 wait complete. 2021-11-21T22:00:06.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:59:59.269756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:00:00.412748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:00:06.665 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:00:06.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 Processor -- start 2021-11-21T22:00:06.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- start start 2021-11-21T22:00:06.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8e280abce0 0x7f8e281478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8e280ac600 0x7f8e280aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e28147e80 con 0x7f8e28144810 2021-11-21T22:00:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e280b1490 con 0x7f8e280ac600 2021-11-21T22:00:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e280b1fd0 con 0x7f8e280abce0 2021-11-21T22:00:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2659c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8e28144810 0x7f8e280b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33020/0 (socket says 172.21.2.116:33020) 2021-11-21T22:00:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2659c700 1 -- 172.21.2.116:0/725977961 learned_addr learned my addr 172.21.2.116:0/725977961 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e26d9d700 1 --2- 172.21.2.116:0/725977961 >> v2:172.21.2.116:3300/0 conn(0x7f8e280ac600 0x7f8e280aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2559a700 1 -- 172.21.2.116:0/725977961 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519604999 0 0) 0x7f8e28147e80 con 0x7f8e28144810 2021-11-21T22:00:06.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e26d9d700 1 -- 172.21.2.116:0/725977961 >> v2:172.21.2.117:3300/0 conn(0x7f8e280abce0 msgr2=0x7f8e281478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:06.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e26d9d700 1 --2- 172.21.2.116:0/725977961 >> v2:172.21.2.117:3300/0 conn(0x7f8e280abce0 0x7f8e281478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e26d9d700 1 -- 172.21.2.116:0/725977961 >> v1:172.21.2.116:6789/0 conn(0x7f8e28144810 legacy=0x7f8e280b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e26d9d700 1 -- 172.21.2.116:0/725977961 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280af5a0 con 0x7f8e280ac600 2021-11-21T22:00:06.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e26d9d700 1 --2- 172.21.2.116:0/725977961 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e280ac600 0x7f8e280aca50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f8e200032f0 tx=0x7f8e200091d0).ready entity=mon.0 client_cookie=76db61429e30e6e2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2559a700 1 -- 172.21.2.116:0/725977961 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e20009ea0 con 0x7f8e280ac600 2021-11-21T22:00:06.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2559a700 1 -- 172.21.2.116:0/725977961 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2000b870 con 0x7f8e280ac600 2021-11-21T22:00:06.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2559a700 1 -- 172.21.2.116:0/725977961 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2000ba20 con 0x7f8e280ac600 2021-11-21T22:00:06.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- 172.21.2.116:0/725977961 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e280ac600 msgr2=0x7f8e280aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/725977961 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e280ac600 0x7f8e280aca50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f8e200032f0 tx=0x7f8e200091d0).stop 2021-11-21T22:00:06.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- 172.21.2.116:0/725977961 shutdown_connections 2021-11-21T22:00:06.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/725977961 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e280ac600 0x7f8e280aca50 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/725977961 >> v2:172.21.2.117:3300/0 conn(0x7f8e280abce0 0x7f8e281478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- 172.21.2.116:0/725977961 >> 172.21.2.116:0/725977961 conn(0x7f8e2805bd20 msgr2=0x7f8e2805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:06.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- 172.21.2.116:0/725977961 shutdown_connections 2021-11-21T22:00:06.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.714+0000 7f8e2cebb700 1 -- 172.21.2.116:0/725977961 wait complete. 2021-11-21T22:00:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 Processor -- start 2021-11-21T22:00:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 -- start start 2021-11-21T22:00:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8e280abce0 0x7f8e280b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8e280ac600 0x7f8e2813c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e28144810 0x7f8e28146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28146c10 con 0x7f8e28144810 2021-11-21T22:00:06.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28143780 con 0x7f8e280ac600 2021-11-21T22:00:06.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280b34a0 con 0x7f8e280abce0 2021-11-21T22:00:06.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e28144810 0x7f8e28146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e28144810 0x7f8e28146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49304/0 (socket says 172.21.2.116:49304) 2021-11-21T22:00:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 -- 172.21.2.116:0/3375511536 learned_addr learned my addr 172.21.2.116:0/3375511536 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 -- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8e280abce0 msgr2=0x7f8e280b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8e280abce0 0x7f8e280b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 -- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8e280ac600 msgr2=0x7f8e2813c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8e280ac600 0x7f8e2813c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 -- 172.21.2.116:0/3375511536 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e20009470 con 0x7f8e28144810 2021-11-21T22:00:06.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e26d9d700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e28144810 0x7f8e28146720 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f8e200032f0 tx=0x7f8e20016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e1f7fe700 1 -- 172.21.2.116:0/3375511536 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e20009ea0 con 0x7f8e28144810 2021-11-21T22:00:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e1f7fe700 1 -- 172.21.2.116:0/3375511536 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2001f5e0 con 0x7f8e28144810 2021-11-21T22:00:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e281439b0 con 0x7f8e28144810 2021-11-21T22:00:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e28146450 con 0x7f8e28144810 2021-11-21T22:00:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e1f7fe700 1 -- 172.21.2.116:0/3375511536 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2001f940 con 0x7f8e28144810 2021-11-21T22:00:06.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e280b2c50 con 0x7f8e28144810 2021-11-21T22:00:06.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e1f7fe700 1 -- 172.21.2.116:0/3375511536 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8e20026030 con 0x7f8e28144810 2021-11-21T22:00:06.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e1f7fe700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8e0c053660 0x7f8e0c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e1f7fe700 1 -- 172.21.2.116:0/3375511536 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8e2006c910 con 0x7f8e28144810 2021-11-21T22:00:06.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.718+0000 7f8e2659c700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8e0c053660 0x7f8e0c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.722+0000 7f8e2659c700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8e0c053660 0x7f8e0c055b20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f8e18000bc0 tx=0x7f8e18007880).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.722+0000 7f8e1f7fe700 1 -- 172.21.2.116:0/3375511536 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e20043960 con 0x7f8e28144810 2021-11-21T22:00:06.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.802+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8e280b35e0 con 0x7f8e28144810 2021-11-21T22:00:06.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.802+0000 7f8e1f7fe700 1 -- 172.21.2.116:0/3375511536 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8e20046f80 con 0x7f8e28144810 2021-11-21T22:00:06.807 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:00:06.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8e0c053660 msgr2=0x7f8e0c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8e0c053660 0x7f8e0c055b20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f8e18000bc0 tx=0x7f8e18007880).stop 2021-11-21T22:00:06.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e28144810 msgr2=0x7f8e28146720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e28144810 0x7f8e28146720 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f8e200032f0 tx=0x7f8e20016000).stop 2021-11-21T22:00:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 shutdown_connections 2021-11-21T22:00:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8e0c053660 0x7f8e0c055b20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8e28144810 0x7f8e28146720 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8e280ac600 0x7f8e2813c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 --2- 172.21.2.116:0/3375511536 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8e280abce0 0x7f8e280b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 >> 172.21.2.116:0/3375511536 conn(0x7f8e2805bd20 msgr2=0x7f8e281493f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 shutdown_connections 2021-11-21T22:00:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.806+0000 7f8e2cebb700 1 -- 172.21.2.116:0/3375511536 wait complete. 2021-11-21T22:00:06.816 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:00:06.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 Processor -- start 2021-11-21T22:00:06.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- start start 2021-11-21T22:00:06.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa8cc144810 0x7fa8cc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc147e80 con 0x7fa8cc0b1db0 2021-11-21T22:00:06.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b1490 con 0x7fa8cc144810 2021-11-21T22:00:06.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0af060 con 0x7fa8cc0abce0 2021-11-21T22:00:06.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d0d68700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fa8cc0b1db0 0x7fa8cc0ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:60178/0 (socket says 172.21.2.116:60178) 2021-11-21T22:00:06.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d0d68700 1 -- 172.21.2.116:0/4248019425 learned_addr learned my addr 172.21.2.116:0/4248019425 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:06.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8cb7fe700 1 --2- 172.21.2.116:0/4248019425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8cbfff700 1 --2- 172.21.2.116:0/4248019425 >> v2:172.21.2.117:3300/0 conn(0x7fa8cc144810 0x7fa8cc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8caffd700 1 -- 172.21.2.116:0/4248019425 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430261432 0 0) 0x7fa8cc147e80 con 0x7fa8cc0b1db0 2021-11-21T22:00:06.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8cb7fe700 1 -- 172.21.2.116:0/4248019425 >> v2:172.21.2.117:3300/0 conn(0x7fa8cc144810 msgr2=0x7fa8cc1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:06.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8cb7fe700 1 --2- 172.21.2.116:0/4248019425 >> v2:172.21.2.117:3300/0 conn(0x7fa8cc144810 0x7fa8cc1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8cb7fe700 1 -- 172.21.2.116:0/4248019425 >> v1:172.21.2.117:6789/0 conn(0x7fa8cc0b1db0 legacy=0x7fa8cc0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8cb7fe700 1 -- 172.21.2.116:0/4248019425 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0af5a0 con 0x7fa8cc0abce0 2021-11-21T22:00:06.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8cb7fe700 1 --2- 172.21.2.116:0/4248019425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b18c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa8b4002f70 tx=0x7fa8b4006df0).ready entity=mon.2 client_cookie=49c9068e3277e154 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8caffd700 1 -- 172.21.2.116:0/4248019425 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8b400d570 con 0x7fa8cc0abce0 2021-11-21T22:00:06.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8caffd700 1 -- 172.21.2.116:0/4248019425 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4000e90 con 0x7fa8cc0abce0 2021-11-21T22:00:06.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8caffd700 1 -- 172.21.2.116:0/4248019425 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8b400d8c0 con 0x7fa8cc0abce0 2021-11-21T22:00:06.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- 172.21.2.116:0/4248019425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 msgr2=0x7fa8cc0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/4248019425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b18c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa8b4002f70 tx=0x7fa8b4006df0).stop 2021-11-21T22:00:06.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- 172.21.2.116:0/4248019425 shutdown_connections 2021-11-21T22:00:06.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/4248019425 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b18c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/4248019425 >> v2:172.21.2.117:3300/0 conn(0x7fa8cc144810 0x7fa8cc1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- 172.21.2.116:0/4248019425 >> 172.21.2.116:0/4248019425 conn(0x7fa8cc05bd20 msgr2=0x7fa8cc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:06.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- 172.21.2.116:0/4248019425 shutdown_connections 2021-11-21T22:00:06.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.866+0000 7fa8d27cb700 1 -- 172.21.2.116:0/4248019425 wait complete. 2021-11-21T22:00:06.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 Processor -- start 2021-11-21T22:00:06.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 -- start start 2021-11-21T22:00:06.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8cc144810 0x7fa8cc0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa8cc1569f0 0x7fa8cc156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc063520 con 0x7fa8cc144810 2021-11-21T22:00:06.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc159930 con 0x7fa8cc1569f0 2021-11-21T22:00:06.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc13ce30 con 0x7fa8cc0abce0 2021-11-21T22:00:06.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8cc144810 0x7fa8cc0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8cc144810 0x7fa8cc0b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49318/0 (socket says 172.21.2.116:49318) 2021-11-21T22:00:06.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 -- 172.21.2.116:0/538204335 learned_addr learned my addr 172.21.2.116:0/538204335 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:06.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cbfff700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d0d68700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa8cc1569f0 0x7fa8cc156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 -- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 msgr2=0x7fa8cc0b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 -- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa8cc1569f0 msgr2=0x7fa8cc156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa8cc1569f0 0x7fa8cc156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 -- 172.21.2.116:0/538204335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b400b180 con 0x7fa8cc144810 2021-11-21T22:00:06.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cb7fe700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8cc144810 0x7fa8cc0b3a40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fa8b4002f70 tx=0x7fa8b4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8c97fa700 1 -- 172.21.2.116:0/538204335 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8b4016070 con 0x7fa8cc144810 2021-11-21T22:00:06.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8c97fa700 1 -- 172.21.2.116:0/538204335 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4013d50 con 0x7fa8cc144810 2021-11-21T22:00:06.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8c97fa700 1 -- 172.21.2.116:0/538204335 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8b401e640 con 0x7fa8cc144810 2021-11-21T22:00:06.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc0b4350 con 0x7fa8cc144810 2021-11-21T22:00:06.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc0b4890 con 0x7fa8cc144810 2021-11-21T22:00:06.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc0af300 con 0x7fa8cc144810 2021-11-21T22:00:06.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8c97fa700 1 -- 172.21.2.116:0/538204335 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa8b400f030 con 0x7fa8cc144810 2021-11-21T22:00:06.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8c97fa700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa8b0053580 0x7fa8b0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:06.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8c97fa700 1 -- 172.21.2.116:0/538204335 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa8b406c7d0 con 0x7fa8cc144810 2021-11-21T22:00:06.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.870+0000 7fa8cbfff700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa8b0053580 0x7fa8b0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:06.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.874+0000 7fa8cbfff700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa8b0053580 0x7fa8b0055a40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fa8bc00b000 tx=0x7fa8bc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:06.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.874+0000 7fa8c97fa700 1 -- 172.21.2.116:0/538204335 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa8b4043820 con 0x7fa8cc144810 2021-11-21T22:00:06.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8cc0573d0 con 0x7fa8cc144810 2021-11-21T22:00:06.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8c97fa700 1 -- 172.21.2.116:0/538204335 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8b4023020 con 0x7fa8cc144810 2021-11-21T22:00:06.962 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:06.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa8b0053580 msgr2=0x7fa8b0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa8b0053580 0x7fa8b0055a40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fa8bc00b000 tx=0x7fa8bc008040).stop 2021-11-21T22:00:06.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8cc144810 msgr2=0x7fa8cc0b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:06.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8cc144810 0x7fa8cc0b3a40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fa8b4002f70 tx=0x7fa8b4014000).stop 2021-11-21T22:00:06.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 shutdown_connections 2021-11-21T22:00:06.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa8cc1569f0 0x7fa8cc156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa8b0053580 0x7fa8b0055a40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa8cc144810 0x7fa8cc0b3a40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 --2- 172.21.2.116:0/538204335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa8cc0abce0 0x7fa8cc0b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:06.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 >> 172.21.2.116:0/538204335 conn(0x7fa8cc05bd20 msgr2=0x7fa8cc13fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:06.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 shutdown_connections 2021-11-21T22:00:06.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:06.958+0000 7fa8d27cb700 1 -- 172.21.2.116:0/538204335 wait complete. 2021-11-21T22:00:06.969 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:00:07.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f3450453700 1 Processor -- start 2021-11-21T22:00:07.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f3450453700 1 -- start start 2021-11-21T22:00:07.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f3450453700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3448144810 0x7f34481478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f3450453700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f34480ac600 0x7f34480aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f3450453700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3448147e80 con 0x7f34480abce0 2021-11-21T22:00:07.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f3450453700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34480b1490 con 0x7f3448144810 2021-11-21T22:00:07.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f3450453700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34480b1fd0 con 0x7f34480ac600 2021-11-21T22:00:07.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f344e1ef700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3448144810 0x7f34481478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f344e1ef700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3448144810 0x7f34481478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49324/0 (socket says 172.21.2.116:49324) 2021-11-21T22:00:07.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f344e1ef700 1 -- 172.21.2.116:0/2629001566 learned_addr learned my addr 172.21.2.116:0/2629001566 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:07.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f344e1ef700 1 -- 172.21.2.116:0/2629001566 >> v2:172.21.2.117:3300/0 conn(0x7f34480ac600 msgr2=0x7f34480aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:07.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f344e1ef700 1 --2- 172.21.2.116:0/2629001566 >> v2:172.21.2.117:3300/0 conn(0x7f34480ac600 0x7f34480aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f344e1ef700 1 -- 172.21.2.116:0/2629001566 >> v1:172.21.2.117:6789/0 conn(0x7f34480abce0 legacy=0x7f34480b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:07.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.018+0000 7f344e1ef700 1 -- 172.21.2.116:0/2629001566 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34480af5a0 con 0x7f3448144810 2021-11-21T22:00:07.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344e1ef700 1 --2- 172.21.2.116:0/2629001566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3448144810 0x7f34481478c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f34480b4e00 tx=0x7f3438006eb0).ready entity=mon.0 client_cookie=fd3f5e5e219f78b9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d1ed700 1 -- 172.21.2.116:0/2629001566 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f343800d5c0 con 0x7f3448144810 2021-11-21T22:00:07.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d1ed700 1 -- 172.21.2.116:0/2629001566 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3438000e90 con 0x7f3448144810 2021-11-21T22:00:07.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d1ed700 1 -- 172.21.2.116:0/2629001566 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f343800d910 con 0x7f3448144810 2021-11-21T22:00:07.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2629001566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3448144810 msgr2=0x7f34481478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 --2- 172.21.2.116:0/2629001566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3448144810 0x7f34481478c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f34480b4e00 tx=0x7f3438006eb0).stop 2021-11-21T22:00:07.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2629001566 shutdown_connections 2021-11-21T22:00:07.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 --2- 172.21.2.116:0/2629001566 >> v2:172.21.2.117:3300/0 conn(0x7f34480ac600 0x7f34480aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 --2- 172.21.2.116:0/2629001566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3448144810 0x7f34481478c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2629001566 >> 172.21.2.116:0/2629001566 conn(0x7f344805bd20 msgr2=0x7f344805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:07.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2629001566 shutdown_connections 2021-11-21T22:00:07.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2629001566 wait complete. 2021-11-21T22:00:07.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 Processor -- start 2021-11-21T22:00:07.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- start start 2021-11-21T22:00:07.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34480abce0 0x7f3448143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34480ac600 0x7f344813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3448144810 0x7f344813a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3448148e80 con 0x7f34480ac600 2021-11-21T22:00:07.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3448159e30 con 0x7f3448144810 2021-11-21T22:00:07.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34480b30b0 con 0x7f34480abce0 2021-11-21T22:00:07.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34480ac600 0x7f344813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34480ac600 0x7f344813a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49334/0 (socket says 172.21.2.116:49334) 2021-11-21T22:00:07.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 -- 172.21.2.116:0/2232179678 learned_addr learned my addr 172.21.2.116:0/2232179678 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:07.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 -- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34480abce0 msgr2=0x7f3448143320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:07.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34480abce0 0x7f3448143320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 -- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3448144810 msgr2=0x7f344813a9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:07.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3448144810 0x7f344813a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 -- 172.21.2.116:0/2232179678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343800b1e0 con 0x7f34480ac600 2021-11-21T22:00:07.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f344d9ee700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34480ac600 0x7f344813a4d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f3444002fb0 tx=0x7f3444006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f34377fe700 1 -- 172.21.2.116:0/2232179678 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3444013460 con 0x7f34480ac600 2021-11-21T22:00:07.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f34377fe700 1 -- 172.21.2.116:0/2232179678 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34440135c0 con 0x7f34480ac600 2021-11-21T22:00:07.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3448146bf0 con 0x7f34480ac600 2021-11-21T22:00:07.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34481463e0 con 0x7f34480ac600 2021-11-21T22:00:07.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f34377fe700 1 -- 172.21.2.116:0/2232179678 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3444013920 con 0x7f34480ac600 2021-11-21T22:00:07.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3424003410 con 0x7f34480ac600 2021-11-21T22:00:07.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.022+0000 7f34377fe700 1 -- 172.21.2.116:0/2232179678 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f344400f070 con 0x7f34480ac600 2021-11-21T22:00:07.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.026+0000 7f34377fe700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3428053660 0x7f3428055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.026+0000 7f34377fe700 1 -- 172.21.2.116:0/2232179678 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f344406b4a0 con 0x7f34480ac600 2021-11-21T22:00:07.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.026+0000 7f344e1ef700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3428053660 0x7f3428055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.026+0000 7f344e1ef700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3428053660 0x7f3428055b20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f34480b4e00 tx=0x7f3438015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.026+0000 7f34377fe700 1 -- 172.21.2.116:0/2232179678 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f34440424f0 con 0x7f34480ac600 2021-11-21T22:00:07.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3424003c40 con 0x7f34480ac600 2021-11-21T22:00:07.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f34377fe700 1 -- 172.21.2.116:0/2232179678 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3444045b10 con 0x7f34480ac600 2021-11-21T22:00:07.114 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:07.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3428053660 msgr2=0x7f3428055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3428053660 0x7f3428055b20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f34480b4e00 tx=0x7f3438015000).stop 2021-11-21T22:00:07.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34480ac600 msgr2=0x7f344813a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34480ac600 0x7f344813a4d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f3444002fb0 tx=0x7f3444006df0).stop 2021-11-21T22:00:07.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 shutdown_connections 2021-11-21T22:00:07.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3448144810 0x7f344813a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3428053660 0x7f3428055b20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34480ac600 0x7f344813a4d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 --2- 172.21.2.116:0/2232179678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34480abce0 0x7f3448143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 >> 172.21.2.116:0/2232179678 conn(0x7f344805bd20 msgr2=0x7f34481400c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:07.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 shutdown_connections 2021-11-21T22:00:07.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.110+0000 7f3450453700 1 -- 172.21.2.116:0/2232179678 wait complete. 2021-11-21T22:00:07.121 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:00:07.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 Processor -- start 2021-11-21T22:00:07.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- start start 2021-11-21T22:00:07.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7faa1c144810 0x7faa1c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 0x7faa1c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faa1c0b1db0 0x7faa1c0ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c147e80 con 0x7faa1c0b1db0 2021-11-21T22:00:07.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c0af5a0 con 0x7faa1c144810 2021-11-21T22:00:07.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0b1490 con 0x7faa1c0abce0 2021-11-21T22:00:07.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faa1c0b1db0 0x7faa1c0ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faa1c0b1db0 0x7faa1c0ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49342/0 (socket says 172.21.2.116:49342) 2021-11-21T22:00:07.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 -- 172.21.2.116:0/2783417323 learned_addr learned my addr 172.21.2.116:0/2783417323 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:07.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 -- 172.21.2.116:0/2783417323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 msgr2=0x7faa1c0b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:07.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 --2- 172.21.2.116:0/2783417323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 0x7faa1c0b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 -- 172.21.2.116:0/2783417323 >> v2:172.21.2.117:3300/0 conn(0x7faa1c144810 msgr2=0x7faa1c1478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:07.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 --2- 172.21.2.116:0/2783417323 >> v2:172.21.2.117:3300/0 conn(0x7faa1c144810 0x7faa1c1478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 -- 172.21.2.116:0/2783417323 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c0af060 con 0x7faa1c0b1db0 2021-11-21T22:00:07.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa2194b700 1 --2- 172.21.2.116:0/2783417323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b1db0 0x7faa1c0ac600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7faa140032f0 tx=0x7faa14009290).ready entity=mon.0 client_cookie=eba78e5b5c4bbccf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa0bfff700 1 -- 172.21.2.116:0/2783417323 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1400b870 con 0x7faa1c0b1db0 2021-11-21T22:00:07.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa0bfff700 1 -- 172.21.2.116:0/2783417323 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1400b9d0 con 0x7faa1c0b1db0 2021-11-21T22:00:07.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa0bfff700 1 -- 172.21.2.116:0/2783417323 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1400bb80 con 0x7faa1c0b1db0 2021-11-21T22:00:07.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- 172.21.2.116:0/2783417323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b1db0 msgr2=0x7faa1c0ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 --2- 172.21.2.116:0/2783417323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b1db0 0x7faa1c0ac600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7faa140032f0 tx=0x7faa14009290).stop 2021-11-21T22:00:07.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- 172.21.2.116:0/2783417323 shutdown_connections 2021-11-21T22:00:07.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 --2- 172.21.2.116:0/2783417323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b1db0 0x7faa1c0ac600 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 --2- 172.21.2.116:0/2783417323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 0x7faa1c0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 --2- 172.21.2.116:0/2783417323 >> v2:172.21.2.117:3300/0 conn(0x7faa1c144810 0x7faa1c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- 172.21.2.116:0/2783417323 >> 172.21.2.116:0/2783417323 conn(0x7faa1c05bd20 msgr2=0x7faa1c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:07.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- 172.21.2.116:0/2783417323 shutdown_connections 2021-11-21T22:00:07.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.170+0000 7faa233ae700 1 -- 172.21.2.116:0/2783417323 wait complete. 2021-11-21T22:00:07.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 Processor -- start 2021-11-21T22:00:07.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 -- start start 2021-11-21T22:00:07.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 0x7faa1c143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faa1c144810 0x7faa1c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b3170 0x7faa1c145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c146960 con 0x7faa1c0b3170 2021-11-21T22:00:07.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c146c00 con 0x7faa1c144810 2021-11-21T22:00:07.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c144dc0 con 0x7faa1c0abce0 2021-11-21T22:00:07.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b3170 0x7faa1c145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b3170 0x7faa1c145500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49348/0 (socket says 172.21.2.116:49348) 2021-11-21T22:00:07.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 -- 172.21.2.116:0/1272595917 learned_addr learned my addr 172.21.2.116:0/1272595917 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:07.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 -- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 msgr2=0x7faa1c143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 0x7faa1c143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 -- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faa1c144810 msgr2=0x7faa1c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faa1c144810 0x7faa1c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 -- 172.21.2.116:0/1272595917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14009530 con 0x7faa1c0b3170 2021-11-21T22:00:07.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2194b700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b3170 0x7faa1c145500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7faa140032f0 tx=0x7faa14016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa0a7fc700 1 -- 172.21.2.116:0/1272595917 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1400b870 con 0x7faa1c0b3170 2021-11-21T22:00:07.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa0a7fc700 1 -- 172.21.2.116:0/1272595917 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14021620 con 0x7faa1c0b3170 2021-11-21T22:00:07.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c145060 con 0x7faa1c0b3170 2021-11-21T22:00:07.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa0a7fc700 1 -- 172.21.2.116:0/1272595917 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa14021980 con 0x7faa1c0b3170 2021-11-21T22:00:07.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c0b36d0 con 0x7faa1c0b3170 2021-11-21T22:00:07.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c1451f0 con 0x7faa1c0b3170 2021-11-21T22:00:07.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa0a7fc700 1 -- 172.21.2.116:0/1272595917 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7faa1400f070 con 0x7faa1c0b3170 2021-11-21T22:00:07.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa0a7fc700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa9fc0536a0 0x7fa9fc055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa0a7fc700 1 -- 172.21.2.116:0/1272595917 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faa1406cbd0 con 0x7faa1c0b3170 2021-11-21T22:00:07.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.174+0000 7faa2114a700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa9fc0536a0 0x7fa9fc055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.178+0000 7faa2114a700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa9fc0536a0 0x7fa9fc055b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7faa0c000ed0 tx=0x7faa0c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.178+0000 7faa0a7fc700 1 -- 172.21.2.116:0/1272595917 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa14043d30 con 0x7faa1c0b3170 2021-11-21T22:00:07.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.258+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa1c0af300 con 0x7faa1c0b3170 2021-11-21T22:00:07.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa0a7fc700 1 -- 172.21.2.116:0/1272595917 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7faa14015030 con 0x7faa1c0b3170 2021-11-21T22:00:07.264 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:07.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa9fc0536a0 msgr2=0x7fa9fc055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa9fc0536a0 0x7fa9fc055b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7faa0c000ed0 tx=0x7faa0c006930).stop 2021-11-21T22:00:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b3170 msgr2=0x7faa1c145500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b3170 0x7faa1c145500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7faa140032f0 tx=0x7faa14016000).stop 2021-11-21T22:00:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 shutdown_connections 2021-11-21T22:00:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa9fc0536a0 0x7fa9fc055b60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faa1c0b3170 0x7faa1c145500 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faa1c144810 0x7faa1c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 --2- 172.21.2.116:0/1272595917 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faa1c0abce0 0x7faa1c143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 >> 172.21.2.116:0/1272595917 conn(0x7faa1c05bd20 msgr2=0x7faa1c148660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 shutdown_connections 2021-11-21T22:00:07.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.262+0000 7faa233ae700 1 -- 172.21.2.116:0/1272595917 wait complete. 2021-11-21T22:00:07.271 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:00:07.272 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:07.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd288ab0700 1 Processor -- start 2021-11-21T22:00:07.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd288ab0700 1 -- start start 2021-11-21T22:00:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd288ab0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840a6aa0 0x7fd28414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd288ab0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd284062410 0x7fd2841443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd288ab0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2840a6e80 con 0x7fd2840a86b0 2021-11-21T22:00:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd288ab0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840a8bc0 con 0x7fd284062410 2021-11-21T22:00:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd288ab0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28414a7b0 con 0x7fd2840a6aa0 2021-11-21T22:00:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd281d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd2840a86b0 0x7fd2841470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33078/0 (socket says 172.21.2.116:33078) 2021-11-21T22:00:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd281d9b700 1 -- 172.21.2.116:0/2767095915 learned_addr learned my addr 172.21.2.116:0/2767095915 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131369524 0 0) 0x7fd2840a6e80 con 0x7fd2840a86b0 2021-11-21T22:00:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd264003260 con 0x7fd2840a86b0 2021-11-21T22:00:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 517241686 0 0) 0x7fd264003260 con 0x7fd2840a86b0 2021-11-21T22:00:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840a6aa0 msgr2=0x7fd28414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840a6aa0 0x7fd28414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 >> v2:172.21.2.117:3300/0 conn(0x7fd284062410 msgr2=0x7fd2841443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd282d9d700 1 --2- 172.21.2.116:0/2767095915 >> v2:172.21.2.117:3300/0 conn(0x7fd284062410 0x7fd2841443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 --2- 172.21.2.116:0/2767095915 >> v2:172.21.2.117:3300/0 conn(0x7fd284062410 0x7fd2841443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.318+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28414aa10 con 0x7fd2840a86b0 2021-11-21T22:00:07.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd274001680 con 0x7fd2840a86b0 2021-11-21T22:00:07.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 >> v1:172.21.2.116:6789/0 conn(0x7fd2840a86b0 legacy=0x7fd2841470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840a6aa0 0x7fd2640037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd264003ee0 0x7fd264004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd264004bf0 0x7fd26400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840a8bc0 con 0x7fd264003ee0 2021-11-21T22:00:07.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28414a7b0 con 0x7fd264004bf0 2021-11-21T22:00:07.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640053a0 con 0x7fd2840a6aa0 2021-11-21T22:00:07.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd264003ee0 0x7fd264004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 -- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840a6aa0 msgr2=0x7fd2640037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:07.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840a6aa0 0x7fd2640037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 -- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd264004bf0 msgr2=0x7fd26400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd264004bf0 0x7fd26400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 -- 172.21.2.116:0/2767095915 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd264003260 con 0x7fd264003ee0 2021-11-21T22:00:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd264003ee0 0x7fd264004500 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fd274001390 tx=0x7fd274001da0).ready entity=mon.0 client_cookie=c47ddeb5355f6d3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd274007170 con 0x7fd264003ee0 2021-11-21T22:00:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2740072d0 con 0x7fd264003ee0 2021-11-21T22:00:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd28159a700 1 -- 172.21.2.116:0/2767095915 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd27400c8c0 con 0x7fd264003ee0 2021-11-21T22:00:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- 172.21.2.116:0/2767095915 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd264003ee0 msgr2=0x7fd264004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 --2- 172.21.2.116:0/2767095915 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd264003ee0 0x7fd264004500 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fd274001390 tx=0x7fd274001da0).stop 2021-11-21T22:00:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd282d9d700 1 -- 172.21.2.116:0/2767095915 reap_dead start 2021-11-21T22:00:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- 172.21.2.116:0/2767095915 shutdown_connections 2021-11-21T22:00:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- 172.21.2.116:0/2767095915 >> 172.21.2.116:0/2767095915 conn(0x7fd28405bd20 msgr2=0x7fd28405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- 172.21.2.116:0/2767095915 shutdown_connections 2021-11-21T22:00:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- 172.21.2.116:0/2767095915 wait complete. 2021-11-21T22:00:07.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 Processor -- start 2021-11-21T22:00:07.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- start start 2021-11-21T22:00:07.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd264003ee0 0x7fd2840a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd284156c40 0x7fd2840a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840aae20 0x7fd2840ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840ab7a0 con 0x7fd284156c40 2021-11-21T22:00:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2841570c0 con 0x7fd264003ee0 2021-11-21T22:00:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840aa440 con 0x7fd2840aae20 2021-11-21T22:00:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd284156c40 0x7fd2840a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd284156c40 0x7fd2840a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49368/0 (socket says 172.21.2.116:49368) 2021-11-21T22:00:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 -- 172.21.2.116:0/1856506807 learned_addr learned my addr 172.21.2.116:0/1856506807 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:07.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 -- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840aae20 msgr2=0x7fd2840ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840aae20 0x7fd2840ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 -- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd264003ee0 msgr2=0x7fd2840a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd264003ee0 0x7fd2840a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 -- 172.21.2.116:0/1856506807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840aa6e0 con 0x7fd284156c40 2021-11-21T22:00:07.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd281d9b700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd284156c40 0x7fd2840a9a80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7fd274001390 tx=0x7fd274015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd27b7fe700 1 -- 172.21.2.116:0/1856506807 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd274007170 con 0x7fd284156c40 2021-11-21T22:00:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd27b7fe700 1 -- 172.21.2.116:0/1856506807 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd274004730 con 0x7fd284156c40 2021-11-21T22:00:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2840aa980 con 0x7fd284156c40 2021-11-21T22:00:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd27b7fe700 1 -- 172.21.2.116:0/1856506807 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd274004a90 con 0x7fd284156c40 2021-11-21T22:00:07.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.322+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd284146130 con 0x7fd284156c40 2021-11-21T22:00:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.326+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2840aab10 con 0x7fd284156c40 2021-11-21T22:00:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.326+0000 7fd27b7fe700 1 -- 172.21.2.116:0/1856506807 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd27401d070 con 0x7fd284156c40 2021-11-21T22:00:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.326+0000 7fd27b7fe700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd268053610 0x7fd268055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:07.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.326+0000 7fd27b7fe700 1 -- 172.21.2.116:0/1856506807 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd27406c590 con 0x7fd284156c40 2021-11-21T22:00:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.326+0000 7fd28259c700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd268053610 0x7fd268055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.326+0000 7fd28259c700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd268053610 0x7fd268055ad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fd26c002f70 tx=0x7fd26c006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:07.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.326+0000 7fd27b7fe700 1 -- 172.21.2.116:0/1856506807 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2740436f0 con 0x7fd284156c40 2021-11-21T22:00:07.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2840573d0 con 0x7fd284156c40 2021-11-21T22:00:07.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd27b7fe700 1 -- 172.21.2.116:0/1856506807 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fd274046d10 con 0x7fd284156c40 2021-11-21T22:00:07.441 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:07.441 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:07.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd268053610 msgr2=0x7fd268055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd268053610 0x7fd268055ad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fd26c002f70 tx=0x7fd26c006a10).stop 2021-11-21T22:00:07.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd284156c40 msgr2=0x7fd2840a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:07.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd284156c40 0x7fd2840a9a80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7fd274001390 tx=0x7fd274015000).stop 2021-11-21T22:00:07.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 shutdown_connections 2021-11-21T22:00:07.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd2840aae20 0x7fd2840ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd268053610 0x7fd268055ad0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd284156c40 0x7fd2840a9a80 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:07.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 --2- 172.21.2.116:0/1856506807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd264003ee0 0x7fd2840a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:07.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 >> 172.21.2.116:0/1856506807 conn(0x7fd28405bd20 msgr2=0x7fd2840b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:07.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 shutdown_connections 2021-11-21T22:00:07.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:07.438+0000 7fd288ab0700 1 -- 172.21.2.116:0/1856506807 wait complete. 2021-11-21T22:00:07.446 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:07.750 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:00:01.426149+0000', 'modified': '2021-11-21T22:00:02.429552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6282': {'gid': 6282, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3502880248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3502880248}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3502880248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:00:07.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:00:07.752 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:00:07.816 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":6282,"want_state":"up:active","state":"up:active","rank_uptime":6.3842662350000001,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":10.587830577} 2021-11-21T22:00:07.817 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 6282, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 6.384266235, "state": "up:active", "uptime": 10.587830577, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:00:07.818 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:00:07.818 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:00:07.819 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:00:07.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:07.820 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:07.831 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:00:07.887 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:07.888 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:07.894 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:07.895 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:07.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:07.897 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:00:07.944 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:00:07.945 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:07.948 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:00:07.959 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:07.960 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:07.966 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:07.967 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:07.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:07.968 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:00:07.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:00:07.989+0000 7f02edd80080 -1 init, newargv = 0x557fdd4bc8c0 newargc=9 2021-11-21T22:00:07.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14172]: starting ceph client 2021-11-21T22:00:08.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14172]: starting fuse 2021-11-21T22:00:09.092 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:00:09.104 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:09.105 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:09.155 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:09.156 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:09.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:09.157 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:00:09.163 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:00:09.163 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:00:09.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:09.164 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:00:09.165 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:00:09.165 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:00:09.165 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:00:09.166 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:00:09.166 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:00:09.167 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:00:09.167 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:00:09.168 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:00:09.168 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:00:09.169 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:00:09.169 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:00:09.169 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:00:09.170 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:00:09.170 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:00:09.171 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:00:09.171 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:00:09.172 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:00:09.172 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:00:09.172 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:00:09.173 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:00:09.173 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:00:09.174 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:00:09.206 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14172.asok 2021-11-21T22:00:09.208 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14172.asok 2021-11-21T22:00:09.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:09.209 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14172.asok status 2021-11-21T22:00:09.315 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:00:09.316 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:00:09.316 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:00:09.316 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:00:09.316 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:00:09.317 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:00:09.317 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:00:09.317 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14172", 2021-11-21T22:00:09.317 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:00:09.317 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:00:09.318 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:00:09.318 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:00:09.318 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 4616, 2021-11-21T22:00:09.318 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:00:09.319 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:00:09.319 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:00:09.319 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 4616 2021-11-21T22:00:09.319 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:00:09.320 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:00:09.320 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:00:09.320 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:00:09.320 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 4251078451 2021-11-21T22:00:09.321 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:00:09.321 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:00:09.321 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:00:09.321 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:00:09.321 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:00:09.322 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 4251078451 2021-11-21T22:00:09.322 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:00:09.322 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.4616 172.21.2.136:0/4251078451", 2021-11-21T22:00:09.322 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/4251078451", 2021-11-21T22:00:09.322 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:00:09.323 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 33, 2021-11-21T22:00:09.323 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 76, 2021-11-21T22:00:09.323 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:00:09.323 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:00:09.324 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:00:09.324 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:00:09.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:09.326 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:09.326 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:09.332 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:00:09.333 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:00:09.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:09.334 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:09.343 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:09.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.394+0000 7f9f13866700 1 Processor -- start 2021-11-21T22:00:09.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.394+0000 7f9f13866700 1 -- start start 2021-11-21T22:00:09.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9f0c0a6aa0 0x7f9f0c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9f0c0a86b0 0x7f9f0c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c0a6e80 con 0x7f9f0c1475d0 2021-11-21T22:00:09.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c0a8bc0 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c14a7b0 con 0x7f9f0c0a6aa0 2021-11-21T22:00:09.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9f0c0a86b0 0x7f9f0c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:09.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9f0c0a86b0 0x7f9f0c1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49376/0 (socket says 172.21.2.116:49376) 2021-11-21T22:00:09.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/2575551544 learned_addr learned my addr 172.21.2.116:0/2575551544 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:09.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/2575551544 >> v2:172.21.2.117:3300/0 conn(0x7f9f0c0a6aa0 msgr2=0x7f9f0c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:09.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- 172.21.2.116:0/2575551544 >> v2:172.21.2.117:3300/0 conn(0x7f9f0c0a6aa0 0x7f9f0c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/2575551544 >> v1:172.21.2.117:6789/0 conn(0x7f9f0c1475d0 legacy=0x7f9f0c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:09.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/2575551544 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c14aa10 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- 172.21.2.116:0/2575551544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c1470e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f9f0c061ab0 tx=0x7f9f00006eb0).ready entity=mon.0 client_cookie=b774477a88332247 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:09.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9efffff700 1 -- 172.21.2.116:0/2575551544 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f0000d5c0 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9efffff700 1 -- 172.21.2.116:0/2575551544 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f00000e90 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9efffff700 1 -- 172.21.2.116:0/2575551544 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f0000d910 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- 172.21.2.116:0/2575551544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 msgr2=0x7f9f0c1470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- 172.21.2.116:0/2575551544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c1470e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f9f0c061ab0 tx=0x7f9f00006eb0).stop 2021-11-21T22:00:09.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- 172.21.2.116:0/2575551544 shutdown_connections 2021-11-21T22:00:09.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- 172.21.2.116:0/2575551544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c1470e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:09.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- 172.21.2.116:0/2575551544 >> v2:172.21.2.117:3300/0 conn(0x7f9f0c0a6aa0 0x7f9f0c14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- 172.21.2.116:0/2575551544 >> 172.21.2.116:0/2575551544 conn(0x7f9f0c05bd20 msgr2=0x7f9f0c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:09.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- 172.21.2.116:0/2575551544 shutdown_connections 2021-11-21T22:00:09.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- 172.21.2.116:0/2575551544 wait complete. 2021-11-21T22:00:09.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 Processor -- start 2021-11-21T22:00:09.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- start start 2021-11-21T22:00:09.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9f0c0a6aa0 0x7f9f0c0aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9f0c1475d0 0x7f9f0c064770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0a94f0 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0a9240 con 0x7f9f0c0a6aa0 2021-11-21T22:00:09.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c157420 con 0x7f9f0c1475d0 2021-11-21T22:00:09.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:09.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c0aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49380/0 (socket says 172.21.2.116:49380) 2021-11-21T22:00:09.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/3699737772 learned_addr learned my addr 172.21.2.116:0/3699737772 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:09.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9f0c1475d0 msgr2=0x7f9f0c064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9f0c1475d0 0x7f9f0c064770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9f0c0a6aa0 msgr2=0x7f9f0c0aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9f0c0a6aa0 0x7f9f0c0aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 -- 172.21.2.116:0/3699737772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0000b1e0 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f10e01700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c0aacc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f9f0c061ab0 tx=0x7f9f00015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:09.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9efe7fc700 1 -- 172.21.2.116:0/3699737772 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f00002f50 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9efe7fc700 1 -- 172.21.2.116:0/3699737772 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0000d940 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9efe7fc700 1 -- 172.21.2.116:0/3699737772 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f00011e90 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c0a9720 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.398+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c146d60 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.402+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0c0573d0 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.402+0000 7f9efe7fc700 1 -- 172.21.2.116:0/3699737772 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9f00025030 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.402+0000 7f9efe7fc700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9eec053690 0x7f9eec055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.402+0000 7f9efe7fc700 1 -- 172.21.2.116:0/3699737772 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9f0006cc40 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.402+0000 7f9f11602700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9eec053690 0x7f9eec055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:09.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.402+0000 7f9f11602700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9eec053690 0x7f9eec055b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f9f08002f70 tx=0x7f9f08006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:09.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.402+0000 7f9efe7fc700 1 -- 172.21.2.116:0/3699737772 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f00043c90 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.510+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f0c0b2de0 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.510+0000 7f9efe7fc700 1 -- 172.21.2.116:0/3699737772 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f9f000472b0 con 0x7f9f0c0a86b0 2021-11-21T22:00:09.515 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:09.516 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:09.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9eec053690 msgr2=0x7f9eec055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9eec053690 0x7f9eec055b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f9f08002f70 tx=0x7f9f08006a80).stop 2021-11-21T22:00:09.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 msgr2=0x7f9f0c0aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c0aacc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f9f0c061ab0 tx=0x7f9f00015000).stop 2021-11-21T22:00:09.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 shutdown_connections 2021-11-21T22:00:09.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9f0c1475d0 0x7f9f0c064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9eec053690 0x7f9eec055b50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:09.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9f0c0a86b0 0x7f9f0c0aacc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:09.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 --2- 172.21.2.116:0/3699737772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9f0c0a6aa0 0x7f9f0c0aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 >> 172.21.2.116:0/3699737772 conn(0x7f9f0c05bd20 msgr2=0x7f9f0c13f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:09.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 shutdown_connections 2021-11-21T22:00:09.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.514+0000 7f9f13866700 1 -- 172.21.2.116:0/3699737772 wait complete. 2021-11-21T22:00:09.519 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:09.523 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:09.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 Processor -- start 2021-11-21T22:00:09.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- start start 2021-11-21T22:00:09.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4d7c0a6aa0 0x7f4d7c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c0a86b0 0x7f4d7c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f4d7c1475d0 0x7f4d7c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0a8bc0 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0a6e80 con 0x7f4d7c1475d0 2021-11-21T22:00:09.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c14a7b0 con 0x7f4d7c0a86b0 2021-11-21T22:00:09.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4d7c0a6aa0 0x7f4d7c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:09.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4d7c0a6aa0 0x7f4d7c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49386/0 (socket says 172.21.2.116:49386) 2021-11-21T22:00:09.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 -- 172.21.2.116:0/3061787311 learned_addr learned my addr 172.21.2.116:0/3061787311 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:09.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 -- 172.21.2.116:0/3061787311 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c0a86b0 msgr2=0x7f4d7c1470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:09.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 --2- 172.21.2.116:0/3061787311 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c0a86b0 0x7f4d7c1470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 -- 172.21.2.116:0/3061787311 >> v2:172.21.2.117:3300/0 conn(0x7f4d7c1475d0 msgr2=0x7f4d7c1443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:09.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 --2- 172.21.2.116:0/3061787311 >> v2:172.21.2.117:3300/0 conn(0x7f4d7c1475d0 0x7f4d7c1443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 -- 172.21.2.116:0/3061787311 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c14aa10 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d82385700 1 --2- 172.21.2.116:0/3061787311 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c14b150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f4d74002f70 tx=0x7f4d74006eb0).ready entity=mon.0 client_cookie=6dd2df8d275b7ea7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:09.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d81383700 1 -- 172.21.2.116:0/3061787311 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d7400d570 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d81383700 1 -- 172.21.2.116:0/3061787311 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74000e90 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d81383700 1 -- 172.21.2.116:0/3061787311 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d7400d900 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- 172.21.2.116:0/3061787311 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 msgr2=0x7f4d7c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 --2- 172.21.2.116:0/3061787311 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c14b150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f4d74002f70 tx=0x7f4d74006eb0).stop 2021-11-21T22:00:09.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- 172.21.2.116:0/3061787311 shutdown_connections 2021-11-21T22:00:09.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 --2- 172.21.2.116:0/3061787311 >> v2:172.21.2.117:3300/0 conn(0x7f4d7c1475d0 0x7f4d7c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 --2- 172.21.2.116:0/3061787311 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c0a86b0 0x7f4d7c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 --2- 172.21.2.116:0/3061787311 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c14b150 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:09.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- 172.21.2.116:0/3061787311 >> 172.21.2.116:0/3061787311 conn(0x7f4d7c05bd20 msgr2=0x7f4d7c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:09.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- 172.21.2.116:0/3061787311 shutdown_connections 2021-11-21T22:00:09.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.570+0000 7f4d845e9700 1 -- 172.21.2.116:0/3061787311 wait complete. 2021-11-21T22:00:09.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 Processor -- start 2021-11-21T22:00:09.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 -- start start 2021-11-21T22:00:09.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d7c0a86b0 0x7f4d7c1493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c1475d0 0x7f4d7c159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0673d0 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c067670 con 0x7f4d7c0a86b0 2021-11-21T22:00:09.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0661f0 con 0x7f4d7c1475d0 2021-11-21T22:00:09.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:09.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49394/0 (socket says 172.21.2.116:49394) 2021-11-21T22:00:09.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 -- 172.21.2.116:0/663917901 learned_addr learned my addr 172.21.2.116:0/663917901 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:09.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 -- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c1475d0 msgr2=0x7f4d7c159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c1475d0 0x7f4d7c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 -- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d7c0a86b0 msgr2=0x7f4d7c1493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d7c0a86b0 0x7f4d7c1493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 -- 172.21.2.116:0/663917901 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7400b1e0 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d82385700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c146fe0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f4d74002f70 tx=0x7f4d74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:09.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d7b7fe700 1 -- 172.21.2.116:0/663917901 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d7400de40 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c066490 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d7b7fe700 1 -- 172.21.2.116:0/663917901 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74011ce0 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d7b7fe700 1 -- 172.21.2.116:0/663917901 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d7401e650 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c149e10 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c066620 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d7b7fe700 1 -- 172.21.2.116:0/663917901 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4d74025030 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d7b7fe700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d5c0536a0 0x7f4d5c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:09.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d7b7fe700 1 -- 172.21.2.116:0/663917901 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4d7406cd50 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.574+0000 7f4d81b84700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d5c0536a0 0x7f4d5c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:09.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.578+0000 7f4d81b84700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d5c0536a0 0x7f4d5c055b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f4d70002fb0 tx=0x7f4d700069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:09.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.578+0000 7f4d7b7fe700 1 -- 172.21.2.116:0/663917901 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d74043eb0 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d7c0573d0 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d7b7fe700 1 -- 172.21.2.116:0/663917901 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f4d74023070 con 0x7f4d7c0a6aa0 2021-11-21T22:00:09.691 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:09.691 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:09.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d5c0536a0 msgr2=0x7f4d5c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d5c0536a0 0x7f4d5c055b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f4d70002fb0 tx=0x7f4d700069b0).stop 2021-11-21T22:00:09.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 msgr2=0x7f4d7c146fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:09.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c146fe0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f4d74002f70 tx=0x7f4d74015000).stop 2021-11-21T22:00:09.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 shutdown_connections 2021-11-21T22:00:09.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d7c1475d0 0x7f4d7c159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d7c0a86b0 0x7f4d7c1493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:09.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d5c0536a0 0x7f4d5c055b60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:09.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 --2- 172.21.2.116:0/663917901 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d7c0a6aa0 0x7f4d7c146fe0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:09.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 >> 172.21.2.116:0/663917901 conn(0x7f4d7c05bd20 msgr2=0x7f4d7c148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:09.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 shutdown_connections 2021-11-21T22:00:09.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:09.686+0000 7f4d845e9700 1 -- 172.21.2.116:0/663917901 wait complete. 2021-11-21T22:00:09.699 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:09.702 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2021-11-21T22:00:09.759 INFO:teuthology.orchestra.run.gibba016.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2021-11-21T22:00:09.760 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2021-11-21T22:00:09.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:09.761 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:09.761 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:09.773 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:00:09.774 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:00:09.775 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T22:00:09.776 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-11-21T22:00:09.776 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:09.776 DEBUG:teuthology.orchestra.run.gibba019:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2021-11-21T22:00:09.777 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-11-21T22:00:09.777 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-11-21T22:00:09.778 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content2'"'"') 2021-11-21T22:00:09.778 DEBUG:teuthology.orchestra.run.gibba019:> while True: 2021-11-21T22:00:09.779 DEBUG:teuthology.orchestra.run.gibba019:> time.sleep(1) 2021-11-21T22:00:09.779 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:00:09.820 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2021-11-21T22:00:09.833 INFO:teuthology.orchestra.run.gibba019.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2021-11-21T22:00:09.833 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-11-21T22:00:10.836 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2021-11-21T22:00:10.849 INFO:teuthology.orchestra.run.gibba019.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2021-11-21T22:00:10.849 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2021-11-21T22:00:10.850 DEBUG:teuthology.orchestra.run.gibba036:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2021-11-21T22:00:10.873 INFO:teuthology.orchestra.run.gibba036.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2021-11-21T22:00:10.874 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2021-11-21T22:00:10.874 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:00:10.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:10.902 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:10.902 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:10.921 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:00:10.922 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:00:10.923 DEBUG:teuthology.orchestra.run.gibba036:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T22:00:10.924 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:00:10.924 DEBUG:teuthology.orchestra.run.gibba036:> import time 2021-11-21T22:00:10.925 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:00:10.925 DEBUG:teuthology.orchestra.run.gibba036:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2021-11-21T22:00:10.926 DEBUG:teuthology.orchestra.run.gibba036:> try: 2021-11-21T22:00:10.926 DEBUG:teuthology.orchestra.run.gibba036:> while True: 2021-11-21T22:00:10.927 DEBUG:teuthology.orchestra.run.gibba036:> os.write(fd, b'"'"'content'"'"') 2021-11-21T22:00:10.927 DEBUG:teuthology.orchestra.run.gibba036:> time.sleep(1) 2021-11-21T22:00:10.928 DEBUG:teuthology.orchestra.run.gibba036:> if not False: 2021-11-21T22:00:10.928 DEBUG:teuthology.orchestra.run.gibba036:> break 2021-11-21T22:00:10.929 DEBUG:teuthology.orchestra.run.gibba036:> except IOError as e: 2021-11-21T22:00:10.929 DEBUG:teuthology.orchestra.run.gibba036:> pass 2021-11-21T22:00:10.930 DEBUG:teuthology.orchestra.run.gibba036:> os.close(fd) 2021-11-21T22:00:10.931 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:00:34.309 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:34.313 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:34.317 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:34.321 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:00:48.293 INFO:tasks.cephfs.test_misc:cap_waiter waited 37.3588752746582s 2021-11-21T22:00:48.294 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls --format=json 2021-11-21T22:00:48.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298749e700 1 Processor -- start 2021-11-21T22:00:48.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298749e700 1 -- start start 2021-11-21T22:00:48.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298749e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2980144810 0x7f29801478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298749e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f29800ac600 0x7f29800aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298749e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2980147e80 con 0x7f29800abce0 2021-11-21T22:00:48.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298749e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29800b1490 con 0x7f2980144810 2021-11-21T22:00:48.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298749e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29800b1fd0 con 0x7f29800ac600 2021-11-21T22:00:48.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2980144810 0x7f29801478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2980144810 0x7f29801478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49402/0 (socket says 172.21.2.116:49402) 2021-11-21T22:00:48.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 -- 172.21.2.116:0/3285125544 learned_addr learned my addr 172.21.2.116:0/3285125544 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:48.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f2984a39700 1 --1- 172.21.2.116:0/3285125544 >> v1:172.21.2.116:6789/0 conn(0x7f29800abce0 0x7f29800b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33128/0 (socket says 172.21.2.116:33128) 2021-11-21T22:00:48.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 -- 172.21.2.116:0/3285125544 >> v2:172.21.2.117:3300/0 conn(0x7f29800ac600 msgr2=0x7f29800aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:48.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 --2- 172.21.2.116:0/3285125544 >> v2:172.21.2.117:3300/0 conn(0x7f29800ac600 0x7f29800aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 -- 172.21.2.116:0/3285125544 >> v1:172.21.2.116:6789/0 conn(0x7f29800abce0 legacy=0x7f29800b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f297ffff700 1 -- 172.21.2.116:0/3285125544 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241516354 0 0) 0x7f2980147e80 con 0x7f29800abce0 2021-11-21T22:00:48.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 -- 172.21.2.116:0/3285125544 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800af5a0 con 0x7f2980144810 2021-11-21T22:00:48.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.346+0000 7f298523a700 1 --2- 172.21.2.116:0/3285125544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2980144810 0x7f29801478c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f2974002f70 tx=0x7f2974006eb0).ready entity=mon.0 client_cookie=482d30d27d0d24fb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297ffff700 1 -- 172.21.2.116:0/3285125544 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f297400d610 con 0x7f2980144810 2021-11-21T22:00:48.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297ffff700 1 -- 172.21.2.116:0/3285125544 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2974000e90 con 0x7f2980144810 2021-11-21T22:00:48.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297ffff700 1 -- 172.21.2.116:0/3285125544 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f297400d960 con 0x7f2980144810 2021-11-21T22:00:48.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3285125544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2980144810 msgr2=0x7f29801478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 --2- 172.21.2.116:0/3285125544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2980144810 0x7f29801478c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f2974002f70 tx=0x7f2974006eb0).stop 2021-11-21T22:00:48.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3285125544 shutdown_connections 2021-11-21T22:00:48.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 --2- 172.21.2.116:0/3285125544 >> v2:172.21.2.117:3300/0 conn(0x7f29800ac600 0x7f29800aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 --2- 172.21.2.116:0/3285125544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2980144810 0x7f29801478c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3285125544 >> 172.21.2.116:0/3285125544 conn(0x7f298005bd20 msgr2=0x7f298005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3285125544 shutdown_connections 2021-11-21T22:00:48.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3285125544 wait complete. 2021-11-21T22:00:48.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 Processor -- start 2021-11-21T22:00:48.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- start start 2021-11-21T22:00:48.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f29800abce0 0x7f2980145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f29800ac600 0x7f2980156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2980144810 0x7f2980156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298015a930 con 0x7f29800ac600 2021-11-21T22:00:48.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980144ee0 con 0x7f29800abce0 2021-11-21T22:00:48.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298015a420 con 0x7f2980144810 2021-11-21T22:00:48.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f29800ac600 0x7f2980156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f29800ac600 0x7f2980156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49410/0 (socket says 172.21.2.116:49410) 2021-11-21T22:00:48.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 -- 172.21.2.116:0/3846516817 learned_addr learned my addr 172.21.2.116:0/3846516817 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:48.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 -- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2980144810 msgr2=0x7f2980156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2980144810 0x7f2980156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 -- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f29800abce0 msgr2=0x7f2980145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f29800abce0 0x7f2980145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 -- 172.21.2.116:0/3846516817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297400b1e0 con 0x7f29800ac600 2021-11-21T22:00:48.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f2984a39700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f29800ac600 0x7f2980156480 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f298015a5b0 tx=0x7f2970007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297e7fc700 1 -- 172.21.2.116:0/3846516817 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29700072e0 con 0x7f29800ac600 2021-11-21T22:00:48.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297e7fc700 1 -- 172.21.2.116:0/3846516817 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970005550 con 0x7f29800ac600 2021-11-21T22:00:48.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298015a690 con 0x7f29800ac600 2021-11-21T22:00:48.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297e7fc700 1 -- 172.21.2.116:0/3846516817 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29700058b0 con 0x7f29800ac600 2021-11-21T22:00:48.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2980062d70 con 0x7f29800ac600 2021-11-21T22:00:48.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2980062f30 con 0x7f29800ac600 2021-11-21T22:00:48.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297e7fc700 1 -- 172.21.2.116:0/3846516817 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f297001a060 con 0x7f29800ac600 2021-11-21T22:00:48.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297e7fc700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2960053690 0x7f2960055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.350+0000 7f297e7fc700 1 -- 172.21.2.116:0/3846516817 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f297006b220 con 0x7f29800ac600 2021-11-21T22:00:48.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.354+0000 7f298523a700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2960053690 0x7f2960055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.354+0000 7f298523a700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2960053690 0x7f2960055b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f2974002f70 tx=0x7f2974015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.354+0000 7f297e7fc700 1 -- 172.21.2.116:0/3846516817 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2970042360 con 0x7f29800ac600 2021-11-21T22:00:48.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.442+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7f29800573d0 con 0x7f29800ac600 2021-11-21T22:00:48.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.442+0000 7f297e7fc700 1 -- 172.21.2.116:0/3846516817 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v77) v1 ==== 98+0+80 (secure 0 0 0) 0x7f2970018020 con 0x7f29800ac600 2021-11-21T22:00:48.449 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:48.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2960053690 msgr2=0x7f2960055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2960053690 0x7f2960055b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f2974002f70 tx=0x7f2974015000).stop 2021-11-21T22:00:48.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f29800ac600 msgr2=0x7f2980156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f29800ac600 0x7f2980156480 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f298015a5b0 tx=0x7f2970007ac0).stop 2021-11-21T22:00:48.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 shutdown_connections 2021-11-21T22:00:48.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2980144810 0x7f2980156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2960053690 0x7f2960055b50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f29800ac600 0x7f2980156480 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 --2- 172.21.2.116:0/3846516817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f29800abce0 0x7f2980145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 >> 172.21.2.116:0/3846516817 conn(0x7f298005bd20 msgr2=0x7f29800b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 shutdown_connections 2021-11-21T22:00:48.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.446+0000 7f298749e700 1 -- 172.21.2.116:0/3846516817 wait complete. 2021-11-21T22:00:48.456 INFO:teuthology.orchestra.run.gibba016.stderr:listed 1 entries 2021-11-21T22:00:48.457 INFO:teuthology.orchestra.run.gibba016.stdout:[{"addr":"172.21.2.119:0/1262068513","until":"2021-11-21T23:00:42.232370+0000"}] 2021-11-21T22:00:48.468 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-11-21T22:00:48.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:48.469 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:48.469 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:48.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:00:48.474 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-11-21T22:00:48.475 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-11-21T22:00:48.475 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:00:48.476 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:00:48.476 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:48.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:48.483 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:48.484 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:48.529 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:00:48.530 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:00:48.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:00:48.531 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:48.536 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:00:48.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c215ea700 1 Processor -- start 2021-11-21T22:00:48.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c215ea700 1 -- start start 2021-11-21T22:00:48.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c215ea700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7c1c0a6aa0 0x7f7c1c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c215ea700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7c1c062410 0x7f7c1c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c215ea700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1c0a6e80 con 0x7f7c1c0a86b0 2021-11-21T22:00:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c215ea700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c0a8bc0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c215ea700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c14a7b0 con 0x7f7c1c062410 2021-11-21T22:00:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7c1c0a6aa0 0x7f7c1c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7c1c0a6aa0 0x7f7c1c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49416/0 (socket says 172.21.2.116:49416) 2021-11-21T22:00:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 -- 172.21.2.116:0/579134099 learned_addr learned my addr 172.21.2.116:0/579134099 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:48.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 -- 172.21.2.116:0/579134099 >> v2:172.21.2.117:3300/0 conn(0x7f7c1c062410 msgr2=0x7f7c1c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:48.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1b7fe700 1 --2- 172.21.2.116:0/579134099 >> v2:172.21.2.117:3300/0 conn(0x7f7c1c062410 0x7f7c1c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 --2- 172.21.2.116:0/579134099 >> v2:172.21.2.117:3300/0 conn(0x7f7c1c062410 0x7f7c1c1443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 -- 172.21.2.116:0/579134099 >> v1:172.21.2.117:6789/0 conn(0x7f7c1c0a86b0 legacy=0x7f7c1c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 -- 172.21.2.116:0/579134099 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c14aa10 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1b7fe700 1 --2- 172.21.2.116:0/579134099 >> v2:172.21.2.117:3300/0 conn(0x7f7c1c062410 0x7f7c1c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c1affd700 1 --2- 172.21.2.116:0/579134099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14b150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f7c1c061ab0 tx=0x7f7c04006eb0).ready entity=mon.0 client_cookie=2f62f0ca53d1edf1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c19ffb700 1 -- 172.21.2.116:0/579134099 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c0400d5c0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c19ffb700 1 -- 172.21.2.116:0/579134099 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c04000e90 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.586+0000 7f7c19ffb700 1 -- 172.21.2.116:0/579134099 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c0400d910 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/579134099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 msgr2=0x7f7c1c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 --2- 172.21.2.116:0/579134099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14b150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f7c1c061ab0 tx=0x7f7c04006eb0).stop 2021-11-21T22:00:48.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/579134099 shutdown_connections 2021-11-21T22:00:48.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 --2- 172.21.2.116:0/579134099 >> v2:172.21.2.117:3300/0 conn(0x7f7c1c062410 0x7f7c1c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 --2- 172.21.2.116:0/579134099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14b150 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/579134099 >> 172.21.2.116:0/579134099 conn(0x7f7c1c05bd20 msgr2=0x7f7c1c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/579134099 shutdown_connections 2021-11-21T22:00:48.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/579134099 wait complete. 2021-11-21T22:00:48.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 Processor -- start 2021-11-21T22:00:48.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- start start 2021-11-21T22:00:48.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c1c062410 0x7f7c1c1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c1c0a86b0 0x7f7c1c0a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0aa020 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0aa740 con 0x7f7c1c0a86b0 2021-11-21T22:00:48.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0b40a0 con 0x7f7c1c062410 2021-11-21T22:00:48.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49424/0 (socket says 172.21.2.116:49424) 2021-11-21T22:00:48.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 -- 172.21.2.116:0/4108076206 learned_addr learned my addr 172.21.2.116:0/4108076206 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:48.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1b7fe700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c1c0a86b0 0x7f7c1c0a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1affd700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c1c062410 0x7f7c1c1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 -- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c1c062410 msgr2=0x7f7c1c1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c1c062410 0x7f7c1c1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 -- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c1c0a86b0 msgr2=0x7f7c1c0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c1c0a86b0 0x7f7c1c0a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 -- 172.21.2.116:0/4108076206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0400b1e0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1affd700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c1c062410 0x7f7c1c1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:48.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c1a7fc700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14dab0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f7c0c002f30 tx=0x7f7c0c006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c13fff700 1 -- 172.21.2.116:0/4108076206 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c0c002320 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c13fff700 1 -- 172.21.2.116:0/4108076206 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c002480 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c0aac20 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c13fff700 1 -- 172.21.2.116:0/4108076206 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c0c0027e0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c0a93b0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c0573d0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.590+0000 7f7c13fff700 1 -- 172.21.2.116:0/4108076206 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7c0c00f030 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.594+0000 7f7c13fff700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c00053580 0x7f7c00055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.594+0000 7f7c13fff700 1 -- 172.21.2.116:0/4108076206 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f7c0c06b640 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.594+0000 7f7c1affd700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c00053580 0x7f7c00055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.594+0000 7f7c1affd700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c00053580 0x7f7c00055a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f7c04017000 tx=0x7f7c04015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.594+0000 7f7c13fff700 1 -- 172.21.2.116:0/4108076206 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c0c042780 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c1c0aadb0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c13fff700 1 -- 172.21.2.116:0/4108076206 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16102 (secure 0 0 0) 0x7f7c0c045da0 con 0x7f7c1c0a6aa0 2021-11-21T22:00:48.683 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:48.684 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":77,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:00:43.063146+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-11-21T21:59:59.269756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-11-21T22:00:00.412748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.119:0/1262068513":"2021-11-21T23:00:42.232370+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:00:48.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c00053580 msgr2=0x7f7c00055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c00053580 0x7f7c00055a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f7c04017000 tx=0x7f7c04015000).stop 2021-11-21T22:00:48.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 msgr2=0x7f7c1c14dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14dab0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f7c0c002f30 tx=0x7f7c0c006d70).stop 2021-11-21T22:00:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 shutdown_connections 2021-11-21T22:00:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c1c0a86b0 0x7f7c1c0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c00053580 0x7f7c00055a40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c1c0a6aa0 0x7f7c1c14dab0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 --2- 172.21.2.116:0/4108076206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c1c062410 0x7f7c1c1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 >> 172.21.2.116:0/4108076206 conn(0x7f7c1c05bd20 msgr2=0x7f7c1c148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 shutdown_connections 2021-11-21T22:00:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.678+0000 7f7c215ea700 1 -- 172.21.2.116:0/4108076206 wait complete. 2021-11-21T22:00:48.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T21:59:59.269756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:00:00.412748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:00:48.691 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:00:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68fa59e700 1 Processor -- start 2021-11-21T22:00:48.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68fa59e700 1 -- start start 2021-11-21T22:00:48.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68fa59e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f4144810 0x7f68f41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68fa59e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f68f40ac600 0x7f68f40aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68fa59e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f4147e80 con 0x7f68f40abce0 2021-11-21T22:00:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68fa59e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40b1490 con 0x7f68f40ac600 2021-11-21T22:00:48.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68fa59e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40b1fd0 con 0x7f68f4144810 2021-11-21T22:00:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f8b3b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f68f40ac600 0x7f68f40aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f3fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f4144810 0x7f68f41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f37fe700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f68f40abce0 0x7f68f40b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:60298/0 (socket says 172.21.2.116:60298) 2021-11-21T22:00:48.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f37fe700 1 -- 172.21.2.116:0/4250306983 learned_addr learned my addr 172.21.2.116:0/4250306983 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:48.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f8b3b700 1 -- 172.21.2.116:0/4250306983 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f4144810 msgr2=0x7f68f41478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:48.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f8b3b700 1 --2- 172.21.2.116:0/4250306983 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f4144810 0x7f68f41478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f8b3b700 1 -- 172.21.2.116:0/4250306983 >> v1:172.21.2.117:6789/0 conn(0x7f68f40abce0 legacy=0x7f68f40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f2ffd700 1 -- 172.21.2.116:0/4250306983 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015174527 0 0) 0x7f68f4147e80 con 0x7f68f40abce0 2021-11-21T22:00:48.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f8b3b700 1 -- 172.21.2.116:0/4250306983 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40af5a0 con 0x7f68f40ac600 2021-11-21T22:00:48.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f8b3b700 1 --2- 172.21.2.116:0/4250306983 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 0x7f68f40aca90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f68ec0032f0 tx=0x7f68ec0091d0).ready entity=mon.1 client_cookie=bac081c17e0e532f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f2ffd700 1 -- 172.21.2.116:0/4250306983 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68ec00b6c0 con 0x7f68f40ac600 2021-11-21T22:00:48.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f2ffd700 1 -- 172.21.2.116:0/4250306983 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00b820 con 0x7f68f40ac600 2021-11-21T22:00:48.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.742+0000 7f68f2ffd700 1 -- 172.21.2.116:0/4250306983 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68ec00b990 con 0x7f68f40ac600 2021-11-21T22:00:48.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/4250306983 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 msgr2=0x7f68f40aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 --2- 172.21.2.116:0/4250306983 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 0x7f68f40aca90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f68ec0032f0 tx=0x7f68ec0091d0).stop 2021-11-21T22:00:48.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/4250306983 shutdown_connections 2021-11-21T22:00:48.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 --2- 172.21.2.116:0/4250306983 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 0x7f68f40aca90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 --2- 172.21.2.116:0/4250306983 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f4144810 0x7f68f41478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/4250306983 >> 172.21.2.116:0/4250306983 conn(0x7f68f405bd20 msgr2=0x7f68f405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/4250306983 shutdown_connections 2021-11-21T22:00:48.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/4250306983 wait complete. 2021-11-21T22:00:48.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 Processor -- start 2021-11-21T22:00:48.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- start start 2021-11-21T22:00:48.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f40abce0 0x7f68f4156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 0x7f68f4143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f68f4144810 0x7f68f40b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4146b10 con 0x7f68f4144810 2021-11-21T22:00:48.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4145050 con 0x7f68f40ac600 2021-11-21T22:00:48.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4153fb0 con 0x7f68f40abce0 2021-11-21T22:00:48.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f68f4144810 0x7f68f40b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f68f4144810 0x7f68f40b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49438/0 (socket says 172.21.2.116:49438) 2021-11-21T22:00:48.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 -- 172.21.2.116:0/2347961650 learned_addr learned my addr 172.21.2.116:0/2347961650 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:48.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 -- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f40abce0 msgr2=0x7f68f4156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f40abce0 0x7f68f4156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 -- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 msgr2=0x7f68f4143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 0x7f68f4143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 -- 172.21.2.116:0/2347961650 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec009470 con 0x7f68f4144810 2021-11-21T22:00:48.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f8b3b700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f68f4144810 0x7f68f40b2ab0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f68f41463d0 tx=0x7f68ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f17fa700 1 -- 172.21.2.116:0/2347961650 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68ec00b6c0 con 0x7f68f4144810 2021-11-21T22:00:48.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f17fa700 1 -- 172.21.2.116:0/2347961650 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec011e40 con 0x7f68f4144810 2021-11-21T22:00:48.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f4145280 con 0x7f68f4144810 2021-11-21T22:00:48.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f41457c0 con 0x7f68f4144810 2021-11-21T22:00:48.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68f17fa700 1 -- 172.21.2.116:0/2347961650 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68ec0107a0 con 0x7f68f4144810 2021-11-21T22:00:48.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.746+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f40af060 con 0x7f68f4144810 2021-11-21T22:00:48.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.750+0000 7f68f17fa700 1 -- 172.21.2.116:0/2347961650 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f68ec027020 con 0x7f68f4144810 2021-11-21T22:00:48.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.750+0000 7f68f17fa700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f68cc053580 0x7f68cc055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.750+0000 7f68f17fa700 1 -- 172.21.2.116:0/2347961650 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f68ec06cfc0 con 0x7f68f4144810 2021-11-21T22:00:48.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.750+0000 7f68f3fff700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f68cc053580 0x7f68cc055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.750+0000 7f68f3fff700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f68cc053580 0x7f68cc055a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f68dc00c000 tx=0x7f68dc006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.750+0000 7f68f17fa700 1 -- 172.21.2.116:0/2347961650 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68ec044100 con 0x7f68f4144810 2021-11-21T22:00:48.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f68f40573d0 con 0x7f68f4144810 2021-11-21T22:00:48.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68f17fa700 1 -- 172.21.2.116:0/2347961650 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v77) v1 ==== 110+0+10 (secure 0 0 0) 0x7f68ec015070 con 0x7f68f4144810 2021-11-21T22:00:48.838 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:00:48.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f68cc053580 msgr2=0x7f68cc055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f68cc053580 0x7f68cc055a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f68dc00c000 tx=0x7f68dc006df0).stop 2021-11-21T22:00:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f68f4144810 msgr2=0x7f68f40b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f68f4144810 0x7f68f40b2ab0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f68f41463d0 tx=0x7f68ec016000).stop 2021-11-21T22:00:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 shutdown_connections 2021-11-21T22:00:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f68cc053580 0x7f68cc055a40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f68f4144810 0x7f68f40b2ab0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f68f40ac600 0x7f68f4143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 --2- 172.21.2.116:0/2347961650 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f68f40abce0 0x7f68f4156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 >> 172.21.2.116:0/2347961650 conn(0x7f68f405bd20 msgr2=0x7f68f40b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 shutdown_connections 2021-11-21T22:00:48.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.834+0000 7f68fa59e700 1 -- 172.21.2.116:0/2347961650 wait complete. 2021-11-21T22:00:48.844 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:00:48.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aec93700 1 Processor -- start 2021-11-21T22:00:48.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aec93700 1 -- start start 2021-11-21T22:00:48.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aec93700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe6a81426e0 0x7fe6a8142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aec93700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a8062e60 0x7fe6a80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aec93700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a80b0f60 con 0x7fe6a80ad470 2021-11-21T22:00:48.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aec93700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a80ae030 con 0x7fe6a81426e0 2021-11-21T22:00:48.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aec93700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8063470 con 0x7fe6a8062e60 2021-11-21T22:00:48.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6a7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a8062e60 0x7fe6a80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aca2f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe6a81426e0 0x7fe6a8142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6a7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a8062e60 0x7fe6a80af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:52792/0 (socket says 172.21.2.116:52792) 2021-11-21T22:00:48.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6aca2f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe6a81426e0 0x7fe6a8142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:50612/0 (socket says 172.21.2.116:50612) 2021-11-21T22:00:48.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6a7fff700 1 -- 172.21.2.116:0/1147236837 learned_addr learned my addr 172.21.2.116:0/1147236837 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.894+0000 7fe6ad230700 1 --1- 172.21.2.116:0/1147236837 >> v1:172.21.2.117:6789/0 conn(0x7fe6a80ad470 0x7fe6a80ad920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:60310/0 (socket says 172.21.2.116:60310) 2021-11-21T22:00:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aca2f700 1 -- 172.21.2.116:0/1147236837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a8062e60 msgr2=0x7fe6a80af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aca2f700 1 --2- 172.21.2.116:0/1147236837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a8062e60 0x7fe6a80af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aca2f700 1 -- 172.21.2.116:0/1147236837 >> v1:172.21.2.117:6789/0 conn(0x7fe6a80ad470 legacy=0x7fe6a80ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aca2f700 1 -- 172.21.2.116:0/1147236837 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a8143050 con 0x7fe6a81426e0 2021-11-21T22:00:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aca2f700 1 --2- 172.21.2.116:0/1147236837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 0x7fe6a8142ac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fe698002f70 tx=0x7fe698006e80).ready entity=mon.1 client_cookie=a887499071c2ef54 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6a77fe700 1 -- 172.21.2.116:0/1147236837 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe698010810 con 0x7fe6a81426e0 2021-11-21T22:00:48.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6a77fe700 1 -- 172.21.2.116:0/1147236837 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe698000e90 con 0x7fe6a81426e0 2021-11-21T22:00:48.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6a77fe700 1 -- 172.21.2.116:0/1147236837 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe698010ba0 con 0x7fe6a81426e0 2021-11-21T22:00:48.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- 172.21.2.116:0/1147236837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 msgr2=0x7fe6a8142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 --2- 172.21.2.116:0/1147236837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 0x7fe6a8142ac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fe698002f70 tx=0x7fe698006e80).stop 2021-11-21T22:00:48.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- 172.21.2.116:0/1147236837 shutdown_connections 2021-11-21T22:00:48.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 --2- 172.21.2.116:0/1147236837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a8062e60 0x7fe6a80af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 --2- 172.21.2.116:0/1147236837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 0x7fe6a8142ac0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- 172.21.2.116:0/1147236837 >> 172.21.2.116:0/1147236837 conn(0x7fe6a80b5730 msgr2=0x7fe6a80b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- 172.21.2.116:0/1147236837 shutdown_connections 2021-11-21T22:00:48.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- 172.21.2.116:0/1147236837 wait complete. 2021-11-21T22:00:48.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 Processor -- start 2021-11-21T22:00:48.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- start start 2021-11-21T22:00:48.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a80ad470 0x7fe6a80646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 0x7fe6a814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6a8141a20 0x7fe6a814fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80655a0 con 0x7fe6a8141a20 2021-11-21T22:00:48.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8065250 con 0x7fe6a81426e0 2021-11-21T22:00:48.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8153bf0 con 0x7fe6a80ad470 2021-11-21T22:00:48.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6a8141a20 0x7fe6a814fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6a8141a20 0x7fe6a814fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49454/0 (socket says 172.21.2.116:49454) 2021-11-21T22:00:48.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 -- 172.21.2.116:0/234852384 learned_addr learned my addr 172.21.2.116:0/234852384 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:48.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 -- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a80ad470 msgr2=0x7fe6a80646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a80ad470 0x7fe6a80646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 -- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 msgr2=0x7fe6a814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 0x7fe6a814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 -- 172.21.2.116:0/234852384 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69800c310 con 0x7fe6a8141a20 2021-11-21T22:00:48.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6ad230700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6a8141a20 0x7fe6a814fff0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe6a8129040 tx=0x7fe6a0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6a5ffb700 1 -- 172.21.2.116:0/234852384 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6a0005ae0 con 0x7fe6a8141a20 2021-11-21T22:00:48.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6a5ffb700 1 -- 172.21.2.116:0/234852384 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0005c40 con 0x7fe6a8141a20 2021-11-21T22:00:48.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6a5ffb700 1 -- 172.21.2.116:0/234852384 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6a00070d0 con 0x7fe6a8141a20 2021-11-21T22:00:48.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a8153e90 con 0x7fe6a8141a20 2021-11-21T22:00:48.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.898+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a8064ed0 con 0x7fe6a8141a20 2021-11-21T22:00:48.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.902+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a80b1e00 con 0x7fe6a8141a20 2021-11-21T22:00:48.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.902+0000 7fe6a5ffb700 1 -- 172.21.2.116:0/234852384 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe6a0010070 con 0x7fe6a8141a20 2021-11-21T22:00:48.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.902+0000 7fe6a5ffb700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe68c053610 0x7fe68c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:48.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.902+0000 7fe6a5ffb700 1 -- 172.21.2.116:0/234852384 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fe6a006b2c0 con 0x7fe6a8141a20 2021-11-21T22:00:48.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.902+0000 7fe6aca2f700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe68c053610 0x7fe68c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:48.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.902+0000 7fe6aca2f700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe68c053610 0x7fe68c055ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fe698002f70 tx=0x7fe6a80af2f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:48.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.902+0000 7fe6a5ffb700 1 -- 172.21.2.116:0/234852384 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe6a0042400 con 0x7fe6a8141a20 2021-11-21T22:00:48.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe6a80573d0 con 0x7fe6a8141a20 2021-11-21T22:00:48.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6a5ffb700 1 -- 172.21.2.116:0/234852384 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v77) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6a0018070 con 0x7fe6a8141a20 2021-11-21T22:00:48.991 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:48.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe68c053610 msgr2=0x7fe68c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe68c053610 0x7fe68c055ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fe698002f70 tx=0x7fe6a80af2f0).stop 2021-11-21T22:00:48.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6a8141a20 msgr2=0x7fe6a814fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:48.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6a8141a20 0x7fe6a814fff0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe6a8129040 tx=0x7fe6a0007f40).stop 2021-11-21T22:00:48.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 shutdown_connections 2021-11-21T22:00:48.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe68c053610 0x7fe68c055ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6a8141a20 0x7fe6a814fff0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:48.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6a81426e0 0x7fe6a814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 --2- 172.21.2.116:0/234852384 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6a80ad470 0x7fe6a80646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:48.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 >> 172.21.2.116:0/234852384 conn(0x7fe6a80b5730 msgr2=0x7fe6a80b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:48.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 shutdown_connections 2021-11-21T22:00:48.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:48.986+0000 7fe6aec93700 1 -- 172.21.2.116:0/234852384 wait complete. 2021-11-21T22:00:48.998 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:00:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 Processor -- start 2021-11-21T22:00:49.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- start start 2021-11-21T22:00:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5edc144810 0x7f5edc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc147e80 con 0x7f5edc0abce0 2021-11-21T22:00:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc0af5a0 con 0x7f5edc0ac600 2021-11-21T22:00:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5edc0b1490 con 0x7f5edc144810 2021-11-21T22:00:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5edaffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5edc144810 0x7f5edc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5edaffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5edc144810 0x7f5edc1478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49458/0 (socket says 172.21.2.116:49458) 2021-11-21T22:00:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5edaffd700 1 -- 172.21.2.116:0/442231709 learned_addr learned my addr 172.21.2.116:0/442231709 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5eda7fc700 1 --1- 172.21.2.116:0/442231709 >> v1:172.21.2.116:6789/0 conn(0x7f5edc0abce0 0x7f5edc0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33184/0 (socket says 172.21.2.116:33184) 2021-11-21T22:00:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5edaffd700 1 -- 172.21.2.116:0/442231709 >> v1:172.21.2.117:6789/0 conn(0x7f5edc0ac600 legacy=0x7f5edc0aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5edaffd700 1 -- 172.21.2.116:0/442231709 >> v1:172.21.2.116:6789/0 conn(0x7f5edc0abce0 legacy=0x7f5edc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5edaffd700 1 -- 172.21.2.116:0/442231709 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc0b1fd0 con 0x7f5edc144810 2021-11-21T22:00:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5edaffd700 1 --2- 172.21.2.116:0/442231709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc144810 0x7f5edc1478c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f5edc05d170 tx=0x7f5ec4006eb0).ready entity=mon.0 client_cookie=5c7842fbc50bb64b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ed9ffb700 1 -- 172.21.2.116:0/442231709 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ec400d610 con 0x7f5edc144810 2021-11-21T22:00:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ed9ffb700 1 -- 172.21.2.116:0/442231709 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec4000e90 con 0x7f5edc144810 2021-11-21T22:00:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ed9ffb700 1 -- 172.21.2.116:0/442231709 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ec400d960 con 0x7f5edc144810 2021-11-21T22:00:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- 172.21.2.116:0/442231709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc144810 msgr2=0x7f5edc1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/442231709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc144810 0x7f5edc1478c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f5edc05d170 tx=0x7f5ec4006eb0).stop 2021-11-21T22:00:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- 172.21.2.116:0/442231709 shutdown_connections 2021-11-21T22:00:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/442231709 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc144810 0x7f5edc1478c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- 172.21.2.116:0/442231709 >> 172.21.2.116:0/442231709 conn(0x7f5edc05bd20 msgr2=0x7f5edc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:49.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- 172.21.2.116:0/442231709 shutdown_connections 2021-11-21T22:00:49.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.050+0000 7f5ee1a15700 1 -- 172.21.2.116:0/442231709 wait complete. 2021-11-21T22:00:49.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 Processor -- start 2021-11-21T22:00:49.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 -- start start 2021-11-21T22:00:49.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc0abce0 0x7f5edc146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5edc0ac600 0x7f5edc13d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5edc144810 0x7f5edc13d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc0b2ab0 con 0x7f5edc0abce0 2021-11-21T22:00:49.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc147010 con 0x7f5edc0ac600 2021-11-21T22:00:49.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc156910 con 0x7f5edc144810 2021-11-21T22:00:49.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc0abce0 0x7f5edc146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc0abce0 0x7f5edc146990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49466/0 (socket says 172.21.2.116:49466) 2021-11-21T22:00:49.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 -- 172.21.2.116:0/3071273660 learned_addr learned my addr 172.21.2.116:0/3071273660 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:49.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edb7fe700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5edc144810 0x7f5edc13d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5eda7fc700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5edc0ac600 0x7f5edc13d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 -- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5edc144810 msgr2=0x7f5edc13d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5edc144810 0x7f5edc13d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 -- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5edc0ac600 msgr2=0x7f5edc13d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5edc0ac600 0x7f5edc13d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 -- 172.21.2.116:0/3071273660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec400b1e0 con 0x7f5edc0abce0 2021-11-21T22:00:49.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5edaffd700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc0abce0 0x7f5edc146990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f5edc05d170 tx=0x7f5ec4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ed3fff700 1 -- 172.21.2.116:0/3071273660 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ec400d290 con 0x7f5edc0abce0 2021-11-21T22:00:49.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ed3fff700 1 -- 172.21.2.116:0/3071273660 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec4002f70 con 0x7f5edc0abce0 2021-11-21T22:00:49.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5edc144dd0 con 0x7f5edc0abce0 2021-11-21T22:00:49.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5edc145310 con 0x7f5edc0abce0 2021-11-21T22:00:49.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ed3fff700 1 -- 172.21.2.116:0/3071273660 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ec401e460 con 0x7f5edc0abce0 2021-11-21T22:00:49.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5edc0af060 con 0x7f5edc0abce0 2021-11-21T22:00:49.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ed3fff700 1 -- 172.21.2.116:0/3071273660 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5ec4025030 con 0x7f5edc0abce0 2021-11-21T22:00:49.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ed3fff700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ec0053660 0x7f5ec0055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.054+0000 7f5ed3fff700 1 -- 172.21.2.116:0/3071273660 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f5ec406ce20 con 0x7f5edc0abce0 2021-11-21T22:00:49.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.058+0000 7f5eda7fc700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ec0053660 0x7f5ec0055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.058+0000 7f5eda7fc700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ec0053660 0x7f5ec0055b20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f5ecc00b000 tx=0x7f5ecc007160).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.058+0000 7f5ed3fff700 1 -- 172.21.2.116:0/3071273660 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ec4043e50 con 0x7f5edc0abce0 2021-11-21T22:00:49.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5edc0573d0 con 0x7f5edc0abce0 2021-11-21T22:00:49.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ed3fff700 1 -- 172.21.2.116:0/3071273660 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5ec4023020 con 0x7f5edc0abce0 2021-11-21T22:00:49.146 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ec0053660 msgr2=0x7f5ec0055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ec0053660 0x7f5ec0055b20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f5ecc00b000 tx=0x7f5ecc007160).stop 2021-11-21T22:00:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc0abce0 msgr2=0x7f5edc146990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc0abce0 0x7f5edc146990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f5edc05d170 tx=0x7f5ec4015000).stop 2021-11-21T22:00:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 shutdown_connections 2021-11-21T22:00:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5edc144810 0x7f5edc13d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5edc0ac600 0x7f5edc13d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ec0053660 0x7f5ec0055b20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 --2- 172.21.2.116:0/3071273660 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5edc0abce0 0x7f5edc146990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 >> 172.21.2.116:0/3071273660 conn(0x7f5edc05bd20 msgr2=0x7f5edc149540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 shutdown_connections 2021-11-21T22:00:49.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.142+0000 7f5ee1a15700 1 -- 172.21.2.116:0/3071273660 wait complete. 2021-11-21T22:00:49.153 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:00:49.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffefed700 1 Processor -- start 2021-11-21T22:00:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffefed700 1 -- start start 2021-11-21T22:00:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffefed700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcff81426e0 0x7fcff8142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffefed700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff8062e60 0x7fcff80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffefed700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff80b0f60 con 0x7fcff80ad470 2021-11-21T22:00:49.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffefed700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff80ae030 con 0x7fcff81426e0 2021-11-21T22:00:49.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffefed700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8063470 con 0x7fcff8062e60 2021-11-21T22:00:49.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcff7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff8062e60 0x7fcff80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcff7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff8062e60 0x7fcff80af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:52820/0 (socket says 172.21.2.116:52820) 2021-11-21T22:00:49.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcff7fff700 1 -- 172.21.2.116:0/4246032705 learned_addr learned my addr 172.21.2.116:0/4246032705 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:49.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.202+0000 7fcffcd89700 1 --2- 172.21.2.116:0/4246032705 >> v2:172.21.2.117:3300/0 conn(0x7fcff81426e0 0x7fcff8142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff7fff700 1 -- 172.21.2.116:0/4246032705 >> v2:172.21.2.117:3300/0 conn(0x7fcff81426e0 msgr2=0x7fcff8142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:49.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff7fff700 1 --2- 172.21.2.116:0/4246032705 >> v2:172.21.2.117:3300/0 conn(0x7fcff81426e0 0x7fcff8142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff7fff700 1 -- 172.21.2.116:0/4246032705 >> v1:172.21.2.117:6789/0 conn(0x7fcff80ad470 legacy=0x7fcff80ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff7fff700 1 -- 172.21.2.116:0/4246032705 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff8143050 con 0x7fcff8062e60 2021-11-21T22:00:49.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff7fff700 1 --2- 172.21.2.116:0/4246032705 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff8062e60 0x7fcff80af3d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcfe8002f70 tx=0x7fcfe8006eb0).ready entity=mon.2 client_cookie=2165ff1959062dc8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff77fe700 1 -- 172.21.2.116:0/4246032705 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe800d570 con 0x7fcff8062e60 2021-11-21T22:00:49.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff77fe700 1 -- 172.21.2.116:0/4246032705 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8000e90 con 0x7fcff8062e60 2021-11-21T22:00:49.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff77fe700 1 -- 172.21.2.116:0/4246032705 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe800d900 con 0x7fcff8062e60 2021-11-21T22:00:49.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/4246032705 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff8062e60 msgr2=0x7fcff80af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 --2- 172.21.2.116:0/4246032705 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff8062e60 0x7fcff80af3d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcfe8002f70 tx=0x7fcfe8006eb0).stop 2021-11-21T22:00:49.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/4246032705 shutdown_connections 2021-11-21T22:00:49.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 --2- 172.21.2.116:0/4246032705 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff8062e60 0x7fcff80af3d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 --2- 172.21.2.116:0/4246032705 >> v2:172.21.2.117:3300/0 conn(0x7fcff81426e0 0x7fcff8142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/4246032705 >> 172.21.2.116:0/4246032705 conn(0x7fcff80b5730 msgr2=0x7fcff80b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:49.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/4246032705 shutdown_connections 2021-11-21T22:00:49.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/4246032705 wait complete. 2021-11-21T22:00:49.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 Processor -- start 2021-11-21T22:00:49.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- start start 2021-11-21T22:00:49.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff80ad470 0x7fcff81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff81426e0 0x7fcff814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff814dc60 0x7fcff814e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff814ec60 con 0x7fcff80ad470 2021-11-21T22:00:49.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8141f30 con 0x7fcff814dc60 2021-11-21T22:00:49.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81417b0 con 0x7fcff81426e0 2021-11-21T22:00:49.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff80ad470 0x7fcff81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff80ad470 0x7fcff81402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49482/0 (socket says 172.21.2.116:49482) 2021-11-21T22:00:49.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 -- 172.21.2.116:0/144591995 learned_addr learned my addr 172.21.2.116:0/144591995 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:49.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff7fff700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff81426e0 0x7fcff814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffd58a700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff814dc60 0x7fcff814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 -- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff81426e0 msgr2=0x7fcff814e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff81426e0 0x7fcff814e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 -- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff814dc60 msgr2=0x7fcff814e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff814dc60 0x7fcff814e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 -- 172.21.2.116:0/144591995 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe800b1e0 con 0x7fcff80ad470 2021-11-21T22:00:49.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff7fff700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff81426e0 0x7fcff814e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:00:49.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffcd89700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff80ad470 0x7fcff81402e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7fcfec00c000 tx=0x7fcfec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff5ffb700 1 -- 172.21.2.116:0/144591995 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfec010e30 con 0x7fcff80ad470 2021-11-21T22:00:49.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff5ffb700 1 -- 172.21.2.116:0/144591995 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec00e870 con 0x7fcff80ad470 2021-11-21T22:00:49.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff8141a50 con 0x7fcff80ad470 2021-11-21T22:00:49.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcff5ffb700 1 -- 172.21.2.116:0/144591995 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfec00ebd0 con 0x7fcff80ad470 2021-11-21T22:00:49.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff8123210 con 0x7fcff80ad470 2021-11-21T22:00:49.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.206+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff8141cd0 con 0x7fcff80ad470 2021-11-21T22:00:49.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.210+0000 7fcff5ffb700 1 -- 172.21.2.116:0/144591995 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcfec024070 con 0x7fcff80ad470 2021-11-21T22:00:49.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.210+0000 7fcff5ffb700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd8053580 0x7fcfd8055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.210+0000 7fcff5ffb700 1 -- 172.21.2.116:0/144591995 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fcfec06e040 con 0x7fcff80ad470 2021-11-21T22:00:49.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.210+0000 7fcff7fff700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd8053580 0x7fcfd8055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.210+0000 7fcff7fff700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd8053580 0x7fcfd8055a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fcfe801a000 tx=0x7fcfe8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.210+0000 7fcff5ffb700 1 -- 172.21.2.116:0/144591995 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcfec045010 con 0x7fcff80ad470 2021-11-21T22:00:49.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.294+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcff80b1e00 con 0x7fcff80ad470 2021-11-21T22:00:49.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.294+0000 7fcff5ffb700 1 -- 172.21.2.116:0/144591995 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcfec048630 con 0x7fcff80ad470 2021-11-21T22:00:49.300 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:00:49.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.294+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd8053580 msgr2=0x7fcfd8055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.294+0000 7fcffefed700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd8053580 0x7fcfd8055a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fcfe801a000 tx=0x7fcfe8015000).stop 2021-11-21T22:00:49.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.294+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff80ad470 msgr2=0x7fcff81402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.294+0000 7fcffefed700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff80ad470 0x7fcff81402e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7fcfec00c000 tx=0x7fcfec006df0).stop 2021-11-21T22:00:49.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 shutdown_connections 2021-11-21T22:00:49.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff814dc60 0x7fcff814e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff81426e0 0x7fcff814e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd8053580 0x7fcfd8055a40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 --2- 172.21.2.116:0/144591995 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff80ad470 0x7fcff81402e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 >> 172.21.2.116:0/144591995 conn(0x7fcff80b5730 msgr2=0x7fcff80b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:49.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 shutdown_connections 2021-11-21T22:00:49.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.298+0000 7fcffefed700 1 -- 172.21.2.116:0/144591995 wait complete. 2021-11-21T22:00:49.307 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:00:49.308 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:49.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd1285ec700 1 Processor -- start 2021-11-21T22:00:49.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd1285ec700 1 -- start start 2021-11-21T22:00:49.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd1285ec700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd1200b3320 0x7fd1200ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd1285ec700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1200b29b0 con 0x7fd1201443a0 2021-11-21T22:00:49.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd1285ec700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd120067c30 con 0x7fd120067850 2021-11-21T22:00:49.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd1285ec700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200b3780 con 0x7fd1200b3320 2021-11-21T22:00:49.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd126b89700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd1201443a0 0x7fd120144810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33214/0 (socket says 172.21.2.116:33214) 2021-11-21T22:00:49.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd126b89700 1 -- 172.21.2.116:0/1446636693 learned_addr learned my addr 172.21.2.116:0/1446636693 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:49.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682861780 0 0) 0x7fd1200b29b0 con 0x7fd1201443a0 2021-11-21T22:00:49.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd108003260 con 0x7fd1201443a0 2021-11-21T22:00:49.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991004924 0 0) 0x7fd108003260 con 0x7fd1201443a0 2021-11-21T22:00:49.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 >> v2:172.21.2.117:3300/0 conn(0x7fd1200b3320 msgr2=0x7fd1200ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:49.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 --2- 172.21.2.116:0/1446636693 >> v2:172.21.2.117:3300/0 conn(0x7fd1200b3320 0x7fd1200ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:49.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 >> v1:172.21.2.117:6789/0 conn(0x7fd120067850 legacy=0x7fd1200b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:49.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1200b7910 con 0x7fd1201443a0 2021-11-21T22:00:49.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd118001a00 con 0x7fd1201443a0 2021-11-21T22:00:49.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 >> v1:172.21.2.116:6789/0 conn(0x7fd1201443a0 legacy=0x7fd120144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd108003820 0x7fd108003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd108004500 0x7fd108004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd108005050 0x7fd10800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200b3780 con 0x7fd108004500 2021-11-21T22:00:49.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10800bf40 con 0x7fd108003820 2021-11-21T22:00:49.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10800c100 con 0x7fd108005050 2021-11-21T22:00:49.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.354+0000 7fd125b87700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd108004500 0x7fd108004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 -- 172.21.2.116:0/1446636693 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd108005050 msgr2=0x7fd10800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:49.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd108005050 0x7fd10800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:49.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 -- 172.21.2.116:0/1446636693 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd108003820 msgr2=0x7fd108003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:49.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd108003820 0x7fd108003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:49.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd126b89700 1 -- 172.21.2.116:0/1446636693 reap_dead start 2021-11-21T22:00:49.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 -- 172.21.2.116:0/1446636693 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108003260 con 0x7fd108004500 2021-11-21T22:00:49.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd108004500 0x7fd108004980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fd11c002f70 tx=0x7fd11c006ca0).ready entity=mon.0 client_cookie=39ad3cb952273895 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd11c013410 con 0x7fd108004500 2021-11-21T22:00:49.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c013570 con 0x7fd108004500 2021-11-21T22:00:49.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125386700 1 -- 172.21.2.116:0/1446636693 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd11c0138d0 con 0x7fd108004500 2021-11-21T22:00:49.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/1446636693 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd108004500 msgr2=0x7fd108004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd108004500 0x7fd108004980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fd11c002f70 tx=0x7fd11c006ca0).stop 2021-11-21T22:00:49.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/1446636693 shutdown_connections 2021-11-21T22:00:49.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 --2- 172.21.2.116:0/1446636693 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd108004500 0x7fd108004980 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/1446636693 >> 172.21.2.116:0/1446636693 conn(0x7fd120064d60 msgr2=0x7fd120065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:49.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/1446636693 shutdown_connections 2021-11-21T22:00:49.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/1446636693 wait complete. 2021-11-21T22:00:49.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 Processor -- start 2021-11-21T22:00:49.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- start start 2021-11-21T22:00:49.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd108004500 0x7fd1201497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd120149cd0 0x7fd1200af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1200afbd0 0x7fd1200b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200aa4a0 con 0x7fd120149cd0 2021-11-21T22:00:49.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200aa6d0 con 0x7fd108004500 2021-11-21T22:00:49.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd120147db0 con 0x7fd1200afbd0 2021-11-21T22:00:49.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd120149cd0 0x7fd1200af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd120149cd0 0x7fd1200af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49500/0 (socket says 172.21.2.116:49500) 2021-11-21T22:00:49.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 -- 172.21.2.116:0/3678584672 learned_addr learned my addr 172.21.2.116:0/3678584672 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:49.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 -- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1200afbd0 msgr2=0x7fd1200b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1200afbd0 0x7fd1200b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:49.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 -- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd108004500 msgr2=0x7fd1201497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd108004500 0x7fd1201497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:49.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 -- 172.21.2.116:0/3678584672 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120147b10 con 0x7fd120149cd0 2021-11-21T22:00:49.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd125b87700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd120149cd0 0x7fd1200af6e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fd1201486d0 tx=0x7fd11c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1137fe700 1 -- 172.21.2.116:0/3678584672 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd11c013410 con 0x7fd120149cd0 2021-11-21T22:00:49.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1137fe700 1 -- 172.21.2.116:0/3678584672 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c012da0 con 0x7fd120149cd0 2021-11-21T22:00:49.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd120147ff0 con 0x7fd120149cd0 2021-11-21T22:00:49.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1137fe700 1 -- 172.21.2.116:0/3678584672 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd11c021660 con 0x7fd120149cd0 2021-11-21T22:00:49.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1200b0f00 con 0x7fd120149cd0 2021-11-21T22:00:49.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.358+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1200b2be0 con 0x7fd120149cd0 2021-11-21T22:00:49.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.362+0000 7fd1137fe700 1 -- 172.21.2.116:0/3678584672 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd11c029030 con 0x7fd120149cd0 2021-11-21T22:00:49.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.362+0000 7fd1137fe700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd100053520 0x7fd1000559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:49.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.362+0000 7fd1137fe700 1 -- 172.21.2.116:0/3678584672 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fd11c01b040 con 0x7fd120149cd0 2021-11-21T22:00:49.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.362+0000 7fd126388700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd100053520 0x7fd1000559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:49.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.362+0000 7fd126388700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd100053520 0x7fd1000559e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fd114000f70 tx=0x7fd114006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:49.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.362+0000 7fd1137fe700 1 -- 172.21.2.116:0/3678584672 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd11c044100 con 0x7fd120149cd0 2021-11-21T22:00:49.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.470+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1200573d0 con 0x7fd120149cd0 2021-11-21T22:00:49.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.470+0000 7fd1137fe700 1 -- 172.21.2.116:0/3678584672 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fd11c0476a0 con 0x7fd120149cd0 2021-11-21T22:00:49.476 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:49.476 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:49.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd100053520 msgr2=0x7fd1000559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd100053520 0x7fd1000559e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fd114000f70 tx=0x7fd114006930).stop 2021-11-21T22:00:49.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd120149cd0 msgr2=0x7fd1200af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:49.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd120149cd0 0x7fd1200af6e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fd1201486d0 tx=0x7fd11c014000).stop 2021-11-21T22:00:49.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 shutdown_connections 2021-11-21T22:00:49.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd1200afbd0 0x7fd1200b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:49.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd100053520 0x7fd1000559e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd120149cd0 0x7fd1200af6e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:49.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 --2- 172.21.2.116:0/3678584672 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd108004500 0x7fd1201497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:49.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 >> 172.21.2.116:0/3678584672 conn(0x7fd120064d60 msgr2=0x7fd12014d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:49.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 shutdown_connections 2021-11-21T22:00:49.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:49.474+0000 7fd1285ec700 1 -- 172.21.2.116:0/3678584672 wait complete. 2021-11-21T22:00:49.480 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:49.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:00:01.426149+0000', 'modified': '2021-11-21T22:00:02.429552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6282': {'gid': 6282, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3502880248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3502880248}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3502880248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:00:49.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:00:49.787 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:00:49.852 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":6282,"want_state":"up:active","state":"up:active","rank_uptime":48.419596249999998,"mdsmap_epoch":33,"osdmap_epoch":77,"osdmap_epoch_barrier":77,"uptime":52.623160913} 2021-11-21T22:00:49.853 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 6282, "mdsmap_epoch": 33, "osdmap_epoch": 77, "osdmap_epoch_barrier": 77, "rank_uptime": 48.41959625, "state": "up:active", "uptime": 52.623160913, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:00:49.853 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:00:49.854 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:00:49.854 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:00:49.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:49.855 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:49.865 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:00:49.919 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:49.920 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:49.926 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:49.926 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:49.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:49.928 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:00:49.932 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:00:49.932 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:49.975 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:00:49.984 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:49.985 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:49.990 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:49.990 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:49.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:49.993 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:00:50.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:00:50.013+0000 7f6c6af6f080 -1 init, newargv = 0x558c146e1820 newargc=9 2021-11-21T22:00:50.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14699]: starting ceph client 2021-11-21T22:00:50.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14699]: starting fuse 2021-11-21T22:00:51.098 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:00:51.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:00:51.111 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:00:51.160 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:00:51.160 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:00:51.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.162 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:00:51.167 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:00:51.167 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:00:51.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.168 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:00:51.169 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:00:51.169 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:00:51.170 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:00:51.170 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:00:51.170 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:51.171 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:00:51.171 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:00:51.172 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:00:51.172 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:51.172 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:00:51.173 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:00:51.173 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:00:51.174 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:00:51.174 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:51.174 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:00:51.175 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:00:51.175 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:00:51.176 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:00:51.176 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:00:51.176 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:00:51.177 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:00:51.177 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:00:51.232 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14699.asok 2021-11-21T22:00:51.235 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14699.asok 2021-11-21T22:00:51.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.236 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14699.asok status 2021-11-21T22:00:51.284 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:00:51.285 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:00:51.285 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:00:51.285 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:00:51.285 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:00:51.286 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:00:51.286 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:00:51.286 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14699", 2021-11-21T22:00:51.286 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:00:51.287 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:51.287 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:00:51.287 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:00:51.287 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4867, 2021-11-21T22:00:51.288 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:00:51.288 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:00:51.288 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:00:51.288 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4867 2021-11-21T22:00:51.289 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:51.289 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:00:51.289 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:00:51.289 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:00:51.289 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2175174782 2021-11-21T22:00:51.290 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:00:51.290 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:51.290 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:00:51.290 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:00:51.291 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:00:51.291 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2175174782 2021-11-21T22:00:51.291 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:00:51.291 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4867 172.21.2.119:0/2175174782", 2021-11-21T22:00:51.292 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/2175174782", 2021-11-21T22:00:51.292 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:00:51.292 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 33, 2021-11-21T22:00:51.292 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 77, 2021-11-21T22:00:51.292 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:00:51.293 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:00:51.293 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:00:51.293 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:00:51.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.295 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:51.295 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:51.301 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:00:51.301 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:00:51.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.302 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:51.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:00:51.357 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:51.386 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:51.399 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:51.427 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:51.441 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:00:51.441 INFO:tasks.cephfs.mount:Terminating background process 2021-11-21T22:00:51.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.442 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:51.443 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:51.479 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:00:51.481 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:00:51.481 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:00:51.482 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:00:51.482 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:51.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14699]: fuse finished with error 0 and tester_r 0 2021-11-21T22:00:51.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.505 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:51.506 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:51.515 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:00:51.516 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:00:51.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:00:51.626 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:51.636 INFO:tasks.cephfs.mount:Terminating background process 2021-11-21T22:00:51.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.637 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:51.637 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:51.649 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:00:51.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:00:51.651 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:00:51.651 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:00:51.652 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:51.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14172]: fuse finished with error 0 and tester_r 0 2021-11-21T22:00:51.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:51.707 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:51.707 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:51.711 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:00:51.712 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:00:51.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:00:51.823 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:51.868 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:00:51.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.918+0000 7f9d73d75700 1 Processor -- start 2021-11-21T22:00:51.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.918+0000 7f9d73d75700 1 -- start start 2021-11-21T22:00:51.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 0x7f9d6c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9d6c0a86b0 0x7f9d6c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c0a6e80 con 0x7f9d6c1475d0 2021-11-21T22:00:51.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c0a8bc0 con 0x7f9d6c0a86b0 2021-11-21T22:00:51.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c14a7b0 con 0x7f9d6c0a6aa0 2021-11-21T22:00:51.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9d6c1475d0 0x7f9d6c1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33232/0 (socket says 172.21.2.116:33232) 2021-11-21T22:00:51.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 -- 172.21.2.116:0/174045344 learned_addr learned my addr 172.21.2.116:0/174045344 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:51.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302376659 0 0) 0x7f9d6c0a6e80 con 0x7f9d6c1475d0 2021-11-21T22:00:51.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d54003260 con 0x7f9d6c1475d0 2021-11-21T22:00:51.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d71b11700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 0x7f9d6c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:51.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d71310700 1 --2- 172.21.2.116:0/174045344 >> v2:172.21.2.117:3300/0 conn(0x7f9d6c0a86b0 0x7f9d6c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:51.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138130795 0 0) 0x7f9d54003260 con 0x7f9d6c1475d0 2021-11-21T22:00:51.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 msgr2=0x7f9d6c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:51.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 0x7f9d6c14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:51.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 >> v2:172.21.2.117:3300/0 conn(0x7f9d6c0a86b0 msgr2=0x7f9d6c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:51.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 --2- 172.21.2.116:0/174045344 >> v2:172.21.2.117:3300/0 conn(0x7f9d6c0a86b0 0x7f9d6c1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:51.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c14aa10 con 0x7f9d6c1475d0 2021-11-21T22:00:51.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9d640019e0 con 0x7f9d6c1475d0 2021-11-21T22:00:51.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 >> v1:172.21.2.116:6789/0 conn(0x7f9d6c1475d0 legacy=0x7f9d6c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:51.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 0x7f9d54003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d54003d30 0x7f9d54004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d540049f0 0x7f9d54004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0a8bc0 con 0x7f9d540049f0 2021-11-21T22:00:51.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c14a7b0 con 0x7f9d54003d30 2021-11-21T22:00:51.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5400b8e0 con 0x7f9d6c0a6aa0 2021-11-21T22:00:51.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d540049f0 0x7f9d54004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:51.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 -- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 msgr2=0x7f9d54003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:51.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d71b11700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 0x7f9d54003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:51.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d71310700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d54003d30 0x7f9d54004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:51.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 0x7f9d54003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:51.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 -- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d54003d30 msgr2=0x7f9d54004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:51.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d54003d30 0x7f9d54004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:51.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 -- 172.21.2.116:0/174045344 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d54003260 con 0x7f9d540049f0 2021-11-21T22:00:51.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d71b11700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0a6aa0 0x7f9d54003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:51.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d540049f0 0x7f9d54004e70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f9d640016f0 tx=0x7f9d640021d0).ready entity=mon.0 client_cookie=a911eda88bcea6d2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:51.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6400cc80 con 0x7f9d540049f0 2021-11-21T22:00:51.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6400cde0 con 0x7f9d540049f0 2021-11-21T22:00:51.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d70b0f700 1 -- 172.21.2.116:0/174045344 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d64005b80 con 0x7f9d540049f0 2021-11-21T22:00:51.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- 172.21.2.116:0/174045344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d540049f0 msgr2=0x7f9d54004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:51.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 --2- 172.21.2.116:0/174045344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d540049f0 0x7f9d54004e70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f9d640016f0 tx=0x7f9d640021d0).stop 2021-11-21T22:00:51.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 -- 172.21.2.116:0/174045344 reap_dead start 2021-11-21T22:00:51.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- 172.21.2.116:0/174045344 shutdown_connections 2021-11-21T22:00:51.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- 172.21.2.116:0/174045344 >> 172.21.2.116:0/174045344 conn(0x7f9d6c05bd20 msgr2=0x7f9d6c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:51.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- 172.21.2.116:0/174045344 shutdown_connections 2021-11-21T22:00:51.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- 172.21.2.116:0/174045344 wait complete. 2021-11-21T22:00:51.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 Processor -- start 2021-11-21T22:00:51.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- start start 2021-11-21T22:00:51.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d540049f0 0x7f9d6c13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0b0fb0 0x7f9d6c146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d6c144fd0 0x7f9d6c145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c063f50 con 0x7f9d6c144fd0 2021-11-21T22:00:51.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c063af0 con 0x7f9d540049f0 2021-11-21T22:00:51.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d73d75700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c146f50 con 0x7f9d6c0b0fb0 2021-11-21T22:00:51.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d6c144fd0 0x7f9d6c145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:51.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d6c144fd0 0x7f9d6c145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49520/0 (socket says 172.21.2.116:49520) 2021-11-21T22:00:51.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.922+0000 7f9d72312700 1 -- 172.21.2.116:0/921737562 learned_addr learned my addr 172.21.2.116:0/921737562 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:51.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d72312700 1 -- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0b0fb0 msgr2=0x7f9d6c146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:51.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d72312700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0b0fb0 0x7f9d6c146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:51.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d72312700 1 -- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d540049f0 msgr2=0x7f9d6c13f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:51.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d72312700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d540049f0 0x7f9d6c13f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:51.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d72312700 1 -- 172.21.2.116:0/921737562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c14d1d0 con 0x7f9d6c144fd0 2021-11-21T22:00:51.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d72312700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d6c144fd0 0x7f9d6c145480 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f9d6c14b030 tx=0x7f9d64015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:51.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d5affd700 1 -- 172.21.2.116:0/921737562 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d640074d0 con 0x7f9d6c144fd0 2021-11-21T22:00:51.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d5affd700 1 -- 172.21.2.116:0/921737562 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d640019d0 con 0x7f9d6c144fd0 2021-11-21T22:00:51.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c14cb30 con 0x7f9d6c144fd0 2021-11-21T22:00:51.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d5affd700 1 -- 172.21.2.116:0/921737562 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d64007820 con 0x7f9d6c144fd0 2021-11-21T22:00:51.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c064970 con 0x7f9d6c144fd0 2021-11-21T22:00:51.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c146750 con 0x7f9d6c144fd0 2021-11-21T22:00:51.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d5affd700 1 -- 172.21.2.116:0/921737562 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9d6401d070 con 0x7f9d6c144fd0 2021-11-21T22:00:51.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d5affd700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0534f0 0x7f9d4c0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:51.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d5affd700 1 -- 172.21.2.116:0/921737562 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f9d6406c8f0 con 0x7f9d6c144fd0 2021-11-21T22:00:51.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d71b11700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0534f0 0x7f9d4c0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:51.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d71b11700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0534f0 0x7f9d4c0559b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f9d68008000 tx=0x7f9d68008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:51.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:51.926+0000 7f9d5affd700 1 -- 172.21.2.116:0/921737562 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d64043a30 con 0x7f9d6c144fd0 2021-11-21T22:00:52.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.010+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9d6c0573d0 con 0x7f9d6c144fd0 2021-11-21T22:00:52.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.010+0000 7f9d5affd700 1 -- 172.21.2.116:0/921737562 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v77) v1 ==== 81+0+33724 (secure 0 0 0) 0x7f9d6401a020 con 0x7f9d6c144fd0 2021-11-21T22:00:52.017 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:52.018 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:00:52.018 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 77, 2021-11-21T22:00:52.018 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:00:52.018 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:00:52.019 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:00:43.063146+0000", 2021-11-21T22:00:52.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:00:52.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:00:52.020 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:00:52.020 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:00:52.020 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:00:52.021 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:00:52.021 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:00:52.021 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:00:52.021 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:00:52.022 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:52.022 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:00:52.022 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:00:52.022 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:00:52.023 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:00:52.023 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:00:52.023 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 14, 2021-11-21T22:00:52.024 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:00:52.024 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:00:52.024 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:00:52.025 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:00:52.025 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:00:52.025 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.025 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:00:52.026 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:00:52.026 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:00:52.026 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:00:52.026 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:00:52.027 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:00:52.027 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:00:52.027 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:00:52.027 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:00:52.028 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:00:52.028 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:00:52.028 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:00:52.028 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:00:52.029 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:00:52.029 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:00:52.029 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:00:52.030 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:00:52.030 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:00:52.030 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:00:52.030 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:00:52.031 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:00:52.031 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:00:52.031 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:00:52.031 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.032 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:00:52.032 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:00:52.032 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:00:52.032 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:00:52.033 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:00:52.033 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:00:52.033 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:00:52.033 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:00:52.034 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:00:52.034 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:00:52.034 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:00:52.034 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:00:52.035 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:00:52.035 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:00:52.035 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:00:52.035 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:00:52.036 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:00:52.036 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:00:52.036 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:00:52.036 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:00:52.037 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:00:52.037 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:00:52.037 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:00:52.037 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:00:52.037 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:00:52.038 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:00:52.038 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:00:52.038 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.038 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:00:52.039 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:00:52.039 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:00:52.039 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:00:52.040 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:00:52.040 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:00:52.040 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:00:52.040 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:00:52.041 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:00:52.041 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:00:52.041 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:00:52.042 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:00:52.042 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:00:52.042 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.043 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:00:52.043 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:00:52.043 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.043 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.044 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.044 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:00:52.044 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:00:52.045 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:00:52.045 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:00:52.045 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:00:52.046 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:00:52.046 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:00:52.046 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:00:52.046 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:00:52.047 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:00:52.047 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:00:52.047 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:00:52.047 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:00:52.048 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:00:52.048 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:00:52.048 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:00:52.048 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:00:52.048 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:00:52.049 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:00:52.049 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:00:52.049 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:00:52.049 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:00:52.050 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:00:52.050 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.050 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:00:52.050 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:00:52.051 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:00:52.051 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:00:52.051 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:00:52.051 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:00:52.052 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:00:52.052 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:00:52.052 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:00:52.053 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:00:52.053 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:00:52.053 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:00:52.053 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:00:52.054 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:00:52.054 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:00:52.054 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:00:52.054 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:00:52.055 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:00:52.055 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:00:52.055 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:00:52.055 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:00:52.056 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:00:52.056 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:00:52.056 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:00:52.056 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:00:52.057 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:00:52.057 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:00:52.057 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.057 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:00:52.058 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:00:52.058 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:00:52.058 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:00:52.058 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:00:52.059 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:00:52.059 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:00:52.059 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:00:52.059 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:00:52.059 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:00:52.060 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:00:52.060 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:00:52.060 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:00:52.060 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:00:52.061 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.061 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.061 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.061 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 13, 2021-11-21T22:00:52.061 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:00:52.062 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:59:59.269756+0000", 2021-11-21T22:00:52.062 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:00:52.062 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:00:52.062 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:00:52.063 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:00:52.063 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:00:52.063 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:00:52.063 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:00:52.064 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:00:52.064 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:00:52.064 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:00:52.064 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:00:52.064 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:00:52.065 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:00:52.065 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:00:52.065 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:00:52.065 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:00:52.066 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:00:52.066 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:00:52.066 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:00:52.066 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:00:52.066 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.067 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "75", 2021-11-21T22:00:52.067 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:00:52.067 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:00:52.067 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:00:52.067 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:00:52.068 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:00:52.068 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:00:52.068 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:00:52.068 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:00:52.069 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:00:52.069 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:00:52.069 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:00:52.069 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:00:52.069 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:00:52.070 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:00:52.070 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:00:52.070 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:00:52.070 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:00:52.071 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:00:52.071 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:00:52.071 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:00:52.071 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:00:52.072 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:00:52.072 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:00:52.072 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:00:52.072 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:00:52.072 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:00:52.073 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.073 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:00:52.073 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:00:52.073 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:00:52.074 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:00:52.074 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:00:52.074 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:00:52.074 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:00:52.074 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:00:52.075 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:00:52.075 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:00:52.075 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:00:52.075 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:00:52.076 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:00:52.076 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:00:52.076 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:00:52.076 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.076 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:00:52.077 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:00:52.077 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:00:52.077 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.077 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.078 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.078 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.078 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 14, 2021-11-21T22:00:52.079 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:00:52.079 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:00:00.412748+0000", 2021-11-21T22:00:52.079 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:00:52.080 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:00:52.080 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:00:52.080 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:00:52.080 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:00:52.080 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:00:52.081 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:00:52.081 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:00:52.081 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:00:52.081 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:00:52.082 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:00:52.082 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:00:52.082 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:00:52.083 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:00:52.083 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:00:52.083 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:00:52.083 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:00:52.083 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:00:52.084 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:00:52.084 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:00:52.084 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.084 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "76", 2021-11-21T22:00:52.085 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:00:52.085 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:00:52.085 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:00:52.086 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:00:52.086 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:00:52.087 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:00:52.087 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:00:52.088 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:00:52.088 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:00:52.089 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:00:52.089 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:00:52.089 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:00:52.090 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:00:52.090 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:00:52.091 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:00:52.091 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:00:52.092 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:00:52.092 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:00:52.092 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:00:52.093 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:00:52.093 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:00:52.094 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:00:52.094 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:00:52.094 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:00:52.095 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:00:52.095 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:00:52.096 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.096 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:00:52.097 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:00:52.097 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:00:52.098 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:00:52.098 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:00:52.098 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:00:52.099 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:00:52.099 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:00:52.100 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:00:52.100 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:00:52.101 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:00:52.101 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:00:52.102 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:00:52.102 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:00:52.102 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:00:52.103 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.103 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.103 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.103 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:52.104 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:00:52.104 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.104 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:00:52.104 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:00:52.105 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.105 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.105 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.105 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.106 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.106 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.106 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.106 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:52.107 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.107 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.107 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.107 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.108 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.108 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.108 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:00:52.109 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.109 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.110 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.110 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.110 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:00:52.111 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.111 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.112 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.112 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.113 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.113 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.114 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.114 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.114 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:00:52.114 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.115 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.115 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.115 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.115 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:00:52.116 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.116 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.116 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.116 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.117 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.117 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.117 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.117 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.118 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:00:52.118 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.118 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.118 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.119 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.119 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:00:52.119 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.120 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.120 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.120 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.120 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.120 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.121 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.121 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.122 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:00:52.122 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.122 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.122 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.123 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.123 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:00:52.123 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:52.123 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.124 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.124 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.124 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:00:52.124 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:00:52.124 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:00:52.125 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:00:52.125 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.125 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.125 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.126 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.126 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.126 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.126 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:00:52.127 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:00:52.127 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.127 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.127 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.127 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.128 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.128 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.128 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.129 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T22:00:52.129 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.129 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.129 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.130 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.130 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.130 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.130 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:00:52.131 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.131 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.131 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.131 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.132 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:00:52.132 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.132 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.133 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.133 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.133 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.133 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.134 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.134 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.134 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:00:52.134 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.134 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.135 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.135 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.135 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:00:52.135 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.136 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.136 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.137 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.137 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.137 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.138 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.138 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.139 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:00:52.139 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.140 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.140 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.140 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.141 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:00:52.141 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.142 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.142 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.143 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.143 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.143 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.144 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.144 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.145 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:00:52.145 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.145 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.146 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.146 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.147 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:00:52.147 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:52.148 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.148 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.149 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.149 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:00:52.149 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:00:52.150 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:00:52.150 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:00:52.151 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.151 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.152 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.152 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.152 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.153 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.153 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:00:52.154 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:00:52.154 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.155 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.155 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.155 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.156 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.156 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.157 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.157 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:52.158 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.158 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.159 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.159 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.159 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.160 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.160 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:00:52.160 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.161 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.161 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.161 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.162 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:00:52.162 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.162 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.163 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.163 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.163 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.164 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.164 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.164 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.164 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:00:52.165 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.165 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.165 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.165 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.165 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:00:52.166 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.166 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.166 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.166 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.167 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.167 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.167 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.167 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.168 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:00:52.168 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.168 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.168 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.169 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.169 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:00:52.169 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.169 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.170 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.170 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.170 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.170 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.171 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.171 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.171 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:00:52.171 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.171 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.172 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.172 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.172 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:00:52.173 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:52.173 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.173 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.173 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.174 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:00:52.174 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:00:52.174 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:00:52.174 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:00:52.175 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.175 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.175 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.175 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.175 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.176 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.176 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:00:52.176 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:00:52.176 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.176 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.177 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.177 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.177 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.178 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.178 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.178 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:52.179 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.179 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.180 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.180 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.180 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.180 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.181 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:00:52.181 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.181 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.181 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.182 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.182 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:00:52.182 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.182 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.183 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.183 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.183 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.184 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.184 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.184 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.185 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:00:52.185 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.186 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.186 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.186 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.187 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:00:52.187 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.188 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.188 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.189 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.189 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.189 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.190 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.190 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.191 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:00:52.191 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.192 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.192 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.192 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.193 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:00:52.193 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.194 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.194 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.195 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.195 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.195 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.196 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.196 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.196 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:00:52.196 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.197 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.197 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.197 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.197 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:00:52.198 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:52.198 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.198 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.198 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.199 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:00:52.199 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:00:52.199 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:00:52.200 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:00:52.200 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.200 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.201 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.201 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.201 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.202 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.202 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:00:52.202 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:00:52.203 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.203 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.203 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.203 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.204 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.204 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.204 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.204 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 72, 2021-11-21T22:00:52.204 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.205 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.205 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.205 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.205 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.206 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.206 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:00:52.206 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.206 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.207 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.207 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.207 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:00:52.207 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.207 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.208 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.208 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.208 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.208 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.209 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.209 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.209 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:00:52.209 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.210 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.210 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.210 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.210 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:00:52.210 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.211 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.211 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.211 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.211 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.212 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.212 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.212 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.212 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:00:52.213 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.213 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.213 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.213 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.213 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:00:52.214 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.214 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.214 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.214 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.215 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.215 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.215 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.216 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.216 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:00:52.216 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.217 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.217 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.217 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.217 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:00:52.217 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:52.218 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.218 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.218 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.218 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:00:52.219 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:00:52.219 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:00:52.219 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:00:52.219 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.220 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.220 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.220 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.220 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.221 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.221 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:00:52.221 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:00:52.221 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.221 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.222 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.222 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.222 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.222 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.223 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.223 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:52.223 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.223 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.224 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.224 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.224 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.224 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.224 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:00:52.225 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.225 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.225 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.225 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.226 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:00:52.226 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.226 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.226 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.226 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.227 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.227 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.227 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.227 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.228 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:00:52.228 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.228 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.228 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.229 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.229 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:00:52.229 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.229 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.229 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.230 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.230 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.230 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.230 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.231 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.231 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:00:52.231 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.231 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.232 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.232 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.232 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:00:52.232 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.233 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.233 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.233 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.233 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.234 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.234 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.234 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.234 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:00:52.235 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.235 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.236 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.236 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.236 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:00:52.237 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:52.237 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.238 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.238 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.239 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:00:52.239 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:00:52.239 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:00:52.240 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:00:52.240 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.241 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.241 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.242 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.242 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.243 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.243 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:00:52.243 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:00:52.244 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.244 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.245 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.245 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.245 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.246 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.246 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.247 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:52.247 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.248 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.248 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.248 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.249 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.249 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.250 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:00:52.250 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.251 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.251 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.252 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.252 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:00:52.252 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.253 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.253 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.254 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.254 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.255 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.255 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.256 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.256 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:00:52.257 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.257 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.257 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.258 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.258 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:00:52.259 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.259 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.260 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.260 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.260 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.261 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.261 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.262 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.262 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:00:52.263 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.263 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.264 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.264 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.264 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:00:52.265 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.265 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.266 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.266 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.267 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.267 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.267 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.268 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.268 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:00:52.268 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.269 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.269 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.269 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.270 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:00:52.270 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:52.270 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.271 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.271 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.271 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:00:52.272 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:00:52.272 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:00:52.272 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:00:52.272 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.273 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.273 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.273 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.273 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.274 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.274 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:00:52.274 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:00:52.274 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:52.275 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:52.275 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:52.275 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:52.275 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:52.276 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:52.276 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:52.276 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 72, 2021-11-21T22:00:52.276 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:52.276 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:52.277 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:52.277 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.277 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.277 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.278 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:00:52.278 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.278 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.278 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.278 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.279 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:00:52.279 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.280 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.280 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.280 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.281 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:52.281 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.281 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.281 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.282 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:00:52.282 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.282 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.282 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.283 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.283 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:00:52.283 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.284 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.284 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.284 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.285 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:52.285 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.285 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.285 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.286 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:00:52.286 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.286 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.286 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.287 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.287 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:00:52.287 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.287 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.288 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.288 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.288 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:52.288 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:52.288 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.289 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:52.289 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:00:52.289 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.290 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.290 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.291 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:52.291 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:00:52.291 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:52.292 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.292 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.292 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.292 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:00:52.293 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:00:52.293 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:00:52.293 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:00:52.293 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:52.293 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:52.294 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:52.294 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:52.294 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.294 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:52.295 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:00:52.295 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.295 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:00:52.295 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.296 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.296 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.296 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.296 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.297 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:00:52.297 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.297 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.297 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.297 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:00:52.298 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.298 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.298 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.298 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.306 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.307 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:00:52.307 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.307 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.307 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.308 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:00:52.308 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.308 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.308 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.309 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.309 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.309 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:00:52.309 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.309 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.310 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.310 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:00:52.310 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.310 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.311 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.311 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.311 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.311 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:00:52.311 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.312 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.312 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.312 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:00:52.312 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.313 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.313 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.313 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.313 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.314 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:00:52.314 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.314 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.314 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.314 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:00:52.315 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.315 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.315 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.315 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.316 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.316 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:00:52.316 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.316 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.317 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.317 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:00:52.317 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.317 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.317 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.318 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.318 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.318 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:00:52.318 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.319 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.319 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:52.319 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:00:52.319 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:52.320 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:52.320 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:52.320 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:52.320 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:52.320 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:00:52.321 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:52.321 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.321 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:52.321 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:00:52.322 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:00:52.322 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:00:52.322 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:00:52.322 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:00:52.323 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.119:0/1262068513": "2021-11-21T23:00:42.232370+0000" 2021-11-21T22:00:52.323 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.323 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:00:52.323 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:00:52.324 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:00:52.324 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:00:52.324 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:00:52.324 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:00:52.324 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:00:52.325 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:00:52.325 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:52.325 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:52.325 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:00:52.326 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:00:52.326 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:00:52.326 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:00:52.326 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:00:52.327 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:00:52.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0534f0 msgr2=0x7f9d4c0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0534f0 0x7f9d4c0559b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f9d68008000 tx=0x7f9d68008040).stop 2021-11-21T22:00:52.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d6c144fd0 msgr2=0x7f9d6c145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d6c144fd0 0x7f9d6c145480 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f9d6c14b030 tx=0x7f9d64015000).stop 2021-11-21T22:00:52.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 shutdown_connections 2021-11-21T22:00:52.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0534f0 0x7f9d4c0559b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:52.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d6c144fd0 0x7f9d6c145480 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:52.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d6c0b0fb0 0x7f9d6c146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 --2- 172.21.2.116:0/921737562 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d540049f0 0x7f9d6c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 >> 172.21.2.116:0/921737562 conn(0x7f9d6c05bd20 msgr2=0x7f9d6c148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:52.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 shutdown_connections 2021-11-21T22:00:52.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.014+0000 7f9d73d75700 1 -- 172.21.2.116:0/921737562 wait complete. 2021-11-21T22:00:52.332 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:52.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 Processor -- start 2021-11-21T22:00:52.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- start start 2021-11-21T22:00:52.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faad40a6aa0 0x7faad414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 0x7faad41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad40a6e80 con 0x7faad40a86b0 2021-11-21T22:00:52.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad40a8bc0 con 0x7faad40a6aa0 2021-11-21T22:00:52.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad414a7b0 con 0x7faad4062410 2021-11-21T22:00:52.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faad40a6aa0 0x7faad414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:52.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faad40a6aa0 0x7faad414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49526/0 (socket says 172.21.2.116:49526) 2021-11-21T22:00:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 -- 172.21.2.116:0/4161566298 learned_addr learned my addr 172.21.2.116:0/4161566298 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 -- 172.21.2.116:0/4161566298 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 msgr2=0x7faad41443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 --2- 172.21.2.116:0/4161566298 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 0x7faad41443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 -- 172.21.2.116:0/4161566298 >> v1:172.21.2.117:6789/0 conn(0x7faad40a86b0 legacy=0x7faad41470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:52.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 -- 172.21.2.116:0/4161566298 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad414aa10 con 0x7faad40a6aa0 2021-11-21T22:00:52.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad9edf700 1 --2- 172.21.2.116:0/4161566298 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a6aa0 0x7faad414b150 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7faad4061ab0 tx=0x7faac8006eb0).ready entity=mon.0 client_cookie=3291d7efbfa8f28c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:52.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad8edd700 1 -- 172.21.2.116:0/4161566298 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faac800d570 con 0x7faad40a6aa0 2021-11-21T22:00:52.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad8edd700 1 -- 172.21.2.116:0/4161566298 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac8000e90 con 0x7faad40a6aa0 2021-11-21T22:00:52.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faad8edd700 1 -- 172.21.2.116:0/4161566298 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faac800d8c0 con 0x7faad40a6aa0 2021-11-21T22:00:52.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- 172.21.2.116:0/4161566298 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a6aa0 msgr2=0x7faad414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 --2- 172.21.2.116:0/4161566298 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a6aa0 0x7faad414b150 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7faad4061ab0 tx=0x7faac8006eb0).stop 2021-11-21T22:00:52.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- 172.21.2.116:0/4161566298 shutdown_connections 2021-11-21T22:00:52.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 --2- 172.21.2.116:0/4161566298 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 0x7faad41443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 --2- 172.21.2.116:0/4161566298 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a6aa0 0x7faad414b150 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:52.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- 172.21.2.116:0/4161566298 >> 172.21.2.116:0/4161566298 conn(0x7faad405bd20 msgr2=0x7faad405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:52.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- 172.21.2.116:0/4161566298 shutdown_connections 2021-11-21T22:00:52.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.442+0000 7faadc143700 1 -- 172.21.2.116:0/4161566298 wait complete. 2021-11-21T22:00:52.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 Processor -- start 2021-11-21T22:00:52.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 -- start start 2021-11-21T22:00:52.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 0x7faad414cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faad40a6aa0 0x7faad413cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a86b0 0x7faad413d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4145d30 con 0x7faad40a86b0 2021-11-21T22:00:52.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4140390 con 0x7faad40a6aa0 2021-11-21T22:00:52.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad413fd20 con 0x7faad4062410 2021-11-21T22:00:52.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a86b0 0x7faad413d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:52.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a86b0 0x7faad413d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49534/0 (socket says 172.21.2.116:49534) 2021-11-21T22:00:52.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 -- 172.21.2.116:0/744310335 learned_addr learned my addr 172.21.2.116:0/744310335 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:52.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 -- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 msgr2=0x7faad414cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 0x7faad414cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 -- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faad40a6aa0 msgr2=0x7faad413cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faad40a6aa0 0x7faad413cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 -- 172.21.2.116:0/744310335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac800b1e0 con 0x7faad40a86b0 2021-11-21T22:00:52.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faada6e0700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a86b0 0x7faad413d250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7faacc003330 tx=0x7faacc0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:52.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faac6ffd700 1 -- 172.21.2.116:0/744310335 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faacc001520 con 0x7faad40a86b0 2021-11-21T22:00:52.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faac6ffd700 1 -- 172.21.2.116:0/744310335 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc001680 con 0x7faad40a86b0 2021-11-21T22:00:52.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad4145f60 con 0x7faad40a86b0 2021-11-21T22:00:52.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faac6ffd700 1 -- 172.21.2.116:0/744310335 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faacc0019e0 con 0x7faad40a86b0 2021-11-21T22:00:52.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad4146d30 con 0x7faad40a86b0 2021-11-21T22:00:52.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad41456a0 con 0x7faad40a86b0 2021-11-21T22:00:52.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faac6ffd700 1 -- 172.21.2.116:0/744310335 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7faacc018030 con 0x7faad40a86b0 2021-11-21T22:00:52.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faac6ffd700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faab40536a0 0x7faab4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.446+0000 7faac6ffd700 1 -- 172.21.2.116:0/744310335 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7faacc06b420 con 0x7faad40a86b0 2021-11-21T22:00:52.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.450+0000 7faad9edf700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faab40536a0 0x7faab4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:52.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.450+0000 7faad9edf700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faab40536a0 0x7faab4055b60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7faad4061ab0 tx=0x7faac8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:52.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.450+0000 7faac6ffd700 1 -- 172.21.2.116:0/744310335 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faacc042500 con 0x7faad40a86b0 2021-11-21T22:00:52.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faad40573d0 con 0x7faad40a86b0 2021-11-21T22:00:52.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faac6ffd700 1 -- 172.21.2.116:0/744310335 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7faacc045b20 con 0x7faad40a86b0 2021-11-21T22:00:52.561 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:52.562 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:02.429552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6282":{"gid":6282,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3502880248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3502880248},{"type":"v1","addr":"172.21.2.116:6835","nonce":3502880248}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:52.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faab40536a0 msgr2=0x7faab4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faab40536a0 0x7faab4055b60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7faad4061ab0 tx=0x7faac8015000).stop 2021-11-21T22:00:52.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a86b0 msgr2=0x7faad413d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a86b0 0x7faad413d250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7faacc003330 tx=0x7faacc0091a0).stop 2021-11-21T22:00:52.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 shutdown_connections 2021-11-21T22:00:52.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faab40536a0 0x7faab4055b60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:52.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faad40a86b0 0x7faad413d250 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:52.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faad40a6aa0 0x7faad413cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 --2- 172.21.2.116:0/744310335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faad4062410 0x7faad414cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 >> 172.21.2.116:0/744310335 conn(0x7faad405bd20 msgr2=0x7faad41441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:52.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 shutdown_connections 2021-11-21T22:00:52.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.558+0000 7faadc143700 1 -- 172.21.2.116:0/744310335 wait complete. 2021-11-21T22:00:52.566 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 33 2021-11-21T22:00:52.568 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:00:52.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.614+0000 7f07ab43d700 1 Processor -- start 2021-11-21T22:00:52.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- start start 2021-11-21T22:00:52.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a41426c0 0x7f07a40ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f07a40ae1c0 0x7f07a4142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a41433b0 con 0x7f07a40ac2f0 2021-11-21T22:00:52.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a4143110 con 0x7f07a40ae1c0 2021-11-21T22:00:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40ada80 con 0x7f07a41426c0 2021-11-21T22:00:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f07a40ae1c0 0x7f07a4142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f07a40ae1c0 0x7f07a4142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49542/0 (socket says 172.21.2.116:49542) 2021-11-21T22:00:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 -- 172.21.2.116:0/145112449 learned_addr learned my addr 172.21.2.116:0/145112449 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:52.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 -- 172.21.2.116:0/145112449 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a41426c0 msgr2=0x7f07a40ac800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:52.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 --2- 172.21.2.116:0/145112449 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a41426c0 0x7f07a40ac800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 -- 172.21.2.116:0/145112449 >> v1:172.21.2.117:6789/0 conn(0x7f07a40ac2f0 legacy=0x7f07a40af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:52.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 -- 172.21.2.116:0/145112449 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a40add20 con 0x7f07a40ae1c0 2021-11-21T22:00:52.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a99da700 1 --2- 172.21.2.116:0/145112449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a4142aa0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f07a40ae0a0 tx=0x7f079400b530).ready entity=mon.0 client_cookie=27ed5e6b01eeb82f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a3fff700 1 -- 172.21.2.116:0/145112449 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07940155e0 con 0x7f07a40ae1c0 2021-11-21T22:00:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a3fff700 1 -- 172.21.2.116:0/145112449 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0794015740 con 0x7f07a40ae1c0 2021-11-21T22:00:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a3fff700 1 -- 172.21.2.116:0/145112449 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0794015ae0 con 0x7f07a40ae1c0 2021-11-21T22:00:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- 172.21.2.116:0/145112449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 msgr2=0x7f07a4142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- 172.21.2.116:0/145112449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a4142aa0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f07a40ae0a0 tx=0x7f079400b530).stop 2021-11-21T22:00:52.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- 172.21.2.116:0/145112449 shutdown_connections 2021-11-21T22:00:52.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- 172.21.2.116:0/145112449 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a4142aa0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:52.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- 172.21.2.116:0/145112449 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a41426c0 0x7f07a40ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- 172.21.2.116:0/145112449 >> 172.21.2.116:0/145112449 conn(0x7f07a405bd20 msgr2=0x7f07a405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:52.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- 172.21.2.116:0/145112449 shutdown_connections 2021-11-21T22:00:52.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- 172.21.2.116:0/145112449 wait complete. 2021-11-21T22:00:52.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 Processor -- start 2021-11-21T22:00:52.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- start start 2021-11-21T22:00:52.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a40ac2f0 0x7f07a40668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a413be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07a41426c0 0x7f07a4159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4067290 con 0x7f07a40ae1c0 2021-11-21T22:00:52.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4067530 con 0x7f07a41426c0 2021-11-21T22:00:52.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4066190 con 0x7f07a40ac2f0 2021-11-21T22:00:52.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a413be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:52.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a413be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49548/0 (socket says 172.21.2.116:49548) 2021-11-21T22:00:52.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 -- 172.21.2.116:0/3252019028 learned_addr learned my addr 172.21.2.116:0/3252019028 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:52.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 -- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a40ac2f0 msgr2=0x7f07a40668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a40ac2f0 0x7f07a40668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 -- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07a41426c0 msgr2=0x7f07a4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:52.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07a41426c0 0x7f07a4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:52.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 -- 172.21.2.116:0/3252019028 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079400b7d0 con 0x7f07a40ae1c0 2021-11-21T22:00:52.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a89d8700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a413be40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f079c002f90 tx=0x7f079c006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:52.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a27fc700 1 -- 172.21.2.116:0/3252019028 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f079c013460 con 0x7f07a40ae1c0 2021-11-21T22:00:52.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a27fc700 1 -- 172.21.2.116:0/3252019028 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c0135c0 con 0x7f07a40ae1c0 2021-11-21T22:00:52.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a4066430 con 0x7f07a40ae1c0 2021-11-21T22:00:52.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07a27fc700 1 -- 172.21.2.116:0/3252019028 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f079c013920 con 0x7f07a40ae1c0 2021-11-21T22:00:52.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.618+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a4066df0 con 0x7f07a40ae1c0 2021-11-21T22:00:52.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.622+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a40665c0 con 0x7f07a40ae1c0 2021-11-21T22:00:52.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.622+0000 7f07a27fc700 1 -- 172.21.2.116:0/3252019028 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f079c01a060 con 0x7f07a40ae1c0 2021-11-21T22:00:52.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.622+0000 7f07a27fc700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0784053610 0x7f0784055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:52.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.622+0000 7f07a27fc700 1 -- 172.21.2.116:0/3252019028 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f079c06b220 con 0x7f07a40ae1c0 2021-11-21T22:00:52.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.622+0000 7f07a91d9700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0784053610 0x7f0784055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:52.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.622+0000 7f07a91d9700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0784053610 0x7f0784055ad0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0798000ed0 tx=0x7f0798006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:52.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.622+0000 7f07a27fc700 1 -- 172.21.2.116:0/3252019028 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f079c042360 con 0x7f07a40ae1c0 2021-11-21T22:00:52.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:52.730+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f07a40573d0 con 0x7f07a40ae1c0 2021-11-21T22:00:53.149 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:00:53.146+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:00:53.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.154+0000 7f07a27fc700 1 -- 172.21.2.116:0/3252019028 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f079c045980 con 0x7f07a40ae1c0 2021-11-21T22:00:53.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0784053610 msgr2=0x7f0784055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0784053610 0x7f0784055ad0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0798000ed0 tx=0x7f0798006930).stop 2021-11-21T22:00:53.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 msgr2=0x7f07a413be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a413be40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f079c002f90 tx=0x7f079c006de0).stop 2021-11-21T22:00:53.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 shutdown_connections 2021-11-21T22:00:53.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f07a41426c0 0x7f07a4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0784053610 0x7f0784055ad0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f07a40ae1c0 0x7f07a413be40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 --2- 172.21.2.116:0/3252019028 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f07a40ac2f0 0x7f07a40668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 >> 172.21.2.116:0/3252019028 conn(0x7f07a405bd20 msgr2=0x7f07a413a430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:53.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 shutdown_connections 2021-11-21T22:00:53.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.158+0000 7f07ab43d700 1 -- 172.21.2.116:0/3252019028 wait complete. 2021-11-21T22:00:53.164 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:00:53.168 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:53.176 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:00:53.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 Processor -- start 2021-11-21T22:00:53.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- start start 2021-11-21T22:00:53.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40b805bd20 0x7f40b805c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b8144de0 con 0x7f40b80607e0 2021-11-21T22:00:53.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b805fb60 con 0x7f40b81470d0 2021-11-21T22:00:53.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b805fe00 con 0x7f40b805bd20 2021-11-21T22:00:53.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40b805bd20 0x7f40b805c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:53.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40b805bd20 0x7f40b805c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49566/0 (socket says 172.21.2.116:49566) 2021-11-21T22:00:53.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b659c700 1 -- 172.21.2.116:0/2321157918 learned_addr learned my addr 172.21.2.116:0/2321157918 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:53.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b659c700 1 -- 172.21.2.116:0/2321157918 >> v1:172.21.2.117:6789/0 conn(0x7f40b81470d0 legacy=0x7f40b80b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:53.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b659c700 1 -- 172.21.2.116:0/2321157918 >> v1:172.21.2.116:6789/0 conn(0x7f40b80607e0 legacy=0x7f40b80b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b659c700 1 -- 172.21.2.116:0/2321157918 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80600a0 con 0x7f40b805bd20 2021-11-21T22:00:53.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b659c700 1 --2- 172.21.2.116:0/2321157918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b805bd20 0x7f40b805c100 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f40b80633e0 tx=0x7f40a8006eb0).ready entity=mon.0 client_cookie=ac7cbecdb1b50e05 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:53.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b559a700 1 -- 172.21.2.116:0/2321157918 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a800d610 con 0x7f40b805bd20 2021-11-21T22:00:53.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b559a700 1 -- 172.21.2.116:0/2321157918 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8000e90 con 0x7f40b805bd20 2021-11-21T22:00:53.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40b559a700 1 -- 172.21.2.116:0/2321157918 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a800d960 con 0x7f40b805bd20 2021-11-21T22:00:53.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2321157918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b805bd20 msgr2=0x7f40b805c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2321157918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b805bd20 0x7f40b805c100 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f40b80633e0 tx=0x7f40a8006eb0).stop 2021-11-21T22:00:53.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2321157918 shutdown_connections 2021-11-21T22:00:53.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2321157918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b805bd20 0x7f40b805c100 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2321157918 >> 172.21.2.116:0/2321157918 conn(0x7f40b80b6000 msgr2=0x7f40b80b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:53.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2321157918 shutdown_connections 2021-11-21T22:00:53.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2321157918 wait complete. 2021-11-21T22:00:53.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 Processor -- start 2021-11-21T22:00:53.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.218+0000 7f40bc8dd700 1 -- start start 2021-11-21T22:00:53.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40b805bd20 0x7f40b8159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b81470d0 0x7f40b81496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40b80a9de0 0x7f40b8066b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b814a2a0 con 0x7f40b81470d0 2021-11-21T22:00:53.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b815a580 con 0x7f40b805bd20 2021-11-21T22:00:53.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b815a820 con 0x7f40b80a9de0 2021-11-21T22:00:53.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b81470d0 0x7f40b81496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:53.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b81470d0 0x7f40b81496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49570/0 (socket says 172.21.2.116:49570) 2021-11-21T22:00:53.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 -- 172.21.2.116:0/2114695592 learned_addr learned my addr 172.21.2.116:0/2114695592 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:53.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b6d9d700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40b80a9de0 0x7f40b8066b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:53.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 -- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40b80a9de0 msgr2=0x7f40b8066b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40b80a9de0 0x7f40b8066b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 -- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40b805bd20 msgr2=0x7f40b8159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40b805bd20 0x7f40b8159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 -- 172.21.2.116:0/2114695592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a800b1e0 con 0x7f40b81470d0 2021-11-21T22:00:53.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b6d9d700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40b80a9de0 0x7f40b8066b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:53.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b5d9b700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b81470d0 0x7f40b81496b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f40b80606c0 tx=0x7f40ac007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:53.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40a77fe700 1 -- 172.21.2.116:0/2114695592 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ac00ce30 con 0x7f40b81470d0 2021-11-21T22:00:53.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40a77fe700 1 -- 172.21.2.116:0/2114695592 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac00f470 con 0x7f40b81470d0 2021-11-21T22:00:53.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b815aac0 con 0x7f40b81470d0 2021-11-21T22:00:53.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40a77fe700 1 -- 172.21.2.116:0/2114695592 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ac00f7d0 con 0x7f40b81470d0 2021-11-21T22:00:53.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b8067040 con 0x7f40b81470d0 2021-11-21T22:00:53.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b8156980 con 0x7f40b81470d0 2021-11-21T22:00:53.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40a77fe700 1 -- 172.21.2.116:0/2114695592 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f40ac019030 con 0x7f40b81470d0 2021-11-21T22:00:53.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40a77fe700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40940536d0 0x7f4094055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40a77fe700 1 -- 172.21.2.116:0/2114695592 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f40ac06aee0 con 0x7f40b81470d0 2021-11-21T22:00:53.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b659c700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40940536d0 0x7f4094055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:53.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.222+0000 7f40b659c700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40940536d0 0x7f4094055b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f40b80633e0 tx=0x7f40a8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:53.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.226+0000 7f40a77fe700 1 -- 172.21.2.116:0/2114695592 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40ac041f20 con 0x7f40b81470d0 2021-11-21T22:00:53.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40b8060340 con 0x7f40b81470d0 2021-11-21T22:00:53.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40a77fe700 1 -- 172.21.2.116:0/2114695592 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f40ac045540 con 0x7f40b81470d0 2021-11-21T22:00:53.343 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:53.344 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":34,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:00:01.426149+0000","modified":"2021-11-21T22:00:53.149789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-11-21T22:00:53.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40940536d0 msgr2=0x7f4094055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40940536d0 0x7f4094055b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f40b80633e0 tx=0x7f40a8015000).stop 2021-11-21T22:00:53.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b81470d0 msgr2=0x7f40b81496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b81470d0 0x7f40b81496b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f40b80606c0 tx=0x7f40ac007c20).stop 2021-11-21T22:00:53.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 shutdown_connections 2021-11-21T22:00:53.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40b80a9de0 0x7f40b8066b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40940536d0 0x7f4094055b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40b81470d0 0x7f40b81496b0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 --2- 172.21.2.116:0/2114695592 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40b805bd20 0x7f40b8159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 >> 172.21.2.116:0/2114695592 conn(0x7f40b80b6000 msgr2=0x7f40b8144fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:53.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 shutdown_connections 2021-11-21T22:00:53.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.338+0000 7f40bc8dd700 1 -- 172.21.2.116:0/2114695592 wait complete. 2021-11-21T22:00:53.348 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 34 2021-11-21T22:00:53.350 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:00:53.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.442+0000 7f80cfaaf700 1 Processor -- start 2021-11-21T22:00:53.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.442+0000 7f80cfaaf700 1 -- start start 2021-11-21T22:00:53.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80c8144810 0x7f80c81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c80ac600 0x7f80c80aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c8147e80 con 0x7f80c80abce0 2021-11-21T22:00:53.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c80b1490 con 0x7f80c8144810 2021-11-21T22:00:53.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c80b1fd0 con 0x7f80c80ac600 2021-11-21T22:00:53.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80c8144810 0x7f80c81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:53.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80c8144810 0x7f80c81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49578/0 (socket says 172.21.2.116:49578) 2021-11-21T22:00:53.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 -- 172.21.2.116:0/4056014936 learned_addr learned my addr 172.21.2.116:0/4056014936 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:53.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 -- 172.21.2.116:0/4056014936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c80ac600 msgr2=0x7f80c80aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:53.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 --2- 172.21.2.116:0/4056014936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c80ac600 0x7f80c80aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 -- 172.21.2.116:0/4056014936 >> v1:172.21.2.116:6789/0 conn(0x7f80c80abce0 legacy=0x7f80c80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 -- 172.21.2.116:0/4056014936 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c80af5a0 con 0x7f80c8144810 2021-11-21T22:00:53.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd84b700 1 --2- 172.21.2.116:0/4056014936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c8144810 0x7f80c81478c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f80b8002f70 tx=0x7f80b8006eb0).ready entity=mon.0 client_cookie=695c032a8a395e1a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:53.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cc849700 1 -- 172.21.2.116:0/4056014936 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80b800d5c0 con 0x7f80c8144810 2021-11-21T22:00:53.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cc849700 1 -- 172.21.2.116:0/4056014936 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8000e90 con 0x7f80c8144810 2021-11-21T22:00:53.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cc849700 1 -- 172.21.2.116:0/4056014936 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80b800d950 con 0x7f80c8144810 2021-11-21T22:00:53.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- 172.21.2.116:0/4056014936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c8144810 msgr2=0x7f80c81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/4056014936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c8144810 0x7f80c81478c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f80b8002f70 tx=0x7f80b8006eb0).stop 2021-11-21T22:00:53.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- 172.21.2.116:0/4056014936 shutdown_connections 2021-11-21T22:00:53.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/4056014936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c80ac600 0x7f80c80aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/4056014936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c8144810 0x7f80c81478c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:53.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- 172.21.2.116:0/4056014936 >> 172.21.2.116:0/4056014936 conn(0x7f80c805bd20 msgr2=0x7f80c805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:53.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- 172.21.2.116:0/4056014936 shutdown_connections 2021-11-21T22:00:53.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- 172.21.2.116:0/4056014936 wait complete. 2021-11-21T22:00:53.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 Processor -- start 2021-11-21T22:00:53.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- start start 2021-11-21T22:00:53.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80c80abce0 0x7f80c8146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c80ac600 0x7f80c814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c8144810 0x7f80c814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c814b2a0 con 0x7f80c80ac600 2021-11-21T22:00:53.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c813bd00 con 0x7f80c80abce0 2021-11-21T22:00:53.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c813bfa0 con 0x7f80c8144810 2021-11-21T22:00:53.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c80ac600 0x7f80c814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:53.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c80ac600 0x7f80c814a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49586/0 (socket says 172.21.2.116:49586) 2021-11-21T22:00:53.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 -- 172.21.2.116:0/1024098904 learned_addr learned my addr 172.21.2.116:0/1024098904 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:53.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 -- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c8144810 msgr2=0x7f80c814adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c8144810 0x7f80c814adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 -- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80c80abce0 msgr2=0x7f80c8146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:53.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80c80abce0 0x7f80c8146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:53.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 -- 172.21.2.116:0/1024098904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b800b1e0 con 0x7f80c80ac600 2021-11-21T22:00:53.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cd04a700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c80ac600 0x7f80c814a8c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f80c80b4e00 tx=0x7f80c4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:53.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80b6ffd700 1 -- 172.21.2.116:0/1024098904 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80c400ce30 con 0x7f80c80ac600 2021-11-21T22:00:53.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80b6ffd700 1 -- 172.21.2.116:0/1024098904 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c400f470 con 0x7f80c80ac600 2021-11-21T22:00:53.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c813c240 con 0x7f80c80ac600 2021-11-21T22:00:53.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c813b110 con 0x7f80c80ac600 2021-11-21T22:00:53.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.446+0000 7f80b6ffd700 1 -- 172.21.2.116:0/1024098904 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80c400f7d0 con 0x7f80c80ac600 2021-11-21T22:00:53.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.450+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c80af060 con 0x7f80c80ac600 2021-11-21T22:00:53.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.450+0000 7f80b6ffd700 1 -- 172.21.2.116:0/1024098904 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f80c4019030 con 0x7f80c80ac600 2021-11-21T22:00:53.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.450+0000 7f80b6ffd700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f80a8053660 0x7f80a8055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:53.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.450+0000 7f80b6ffd700 1 -- 172.21.2.116:0/1024098904 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f80c406af70 con 0x7f80c80ac600 2021-11-21T22:00:53.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.450+0000 7f80cd84b700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f80a8053660 0x7f80a8055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:53.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.450+0000 7f80cd84b700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f80a8053660 0x7f80a8055b20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f80b8002f70 tx=0x7f80b8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:53.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.450+0000 7f80b6ffd700 1 -- 172.21.2.116:0/1024098904 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f80c4041f40 con 0x7f80c80ac600 2021-11-21T22:00:53.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:53.558+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f80c80573d0 con 0x7f80c80ac600 2021-11-21T22:00:54.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80b6ffd700 1 -- 172.21.2.116:0/1024098904 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f80c4045560 con 0x7f80c80ac600 2021-11-21T22:00:54.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f80a8053660 msgr2=0x7f80a8055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:54.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f80a8053660 0x7f80a8055b20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f80b8002f70 tx=0x7f80b8015000).stop 2021-11-21T22:00:54.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c80ac600 msgr2=0x7f80c814a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:54.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c80ac600 0x7f80c814a8c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f80c80b4e00 tx=0x7f80c4007c50).stop 2021-11-21T22:00:54.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 shutdown_connections 2021-11-21T22:00:54.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80c8144810 0x7f80c814adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f80a8053660 0x7f80a8055b20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:54.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80c80ac600 0x7f80c814a8c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:54.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 --2- 172.21.2.116:0/1024098904 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80c80abce0 0x7f80c8146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 >> 172.21.2.116:0/1024098904 conn(0x7f80c805bd20 msgr2=0x7f80c80aca90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:54.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 shutdown_connections 2021-11-21T22:00:54.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.174+0000 7f80cfaaf700 1 -- 172.21.2.116:0/1024098904 wait complete. 2021-11-21T22:00:54.186 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:00:54.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 Processor -- start 2021-11-21T22:00:54.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 -- start start 2021-11-21T22:00:54.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 0x7f713c0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:54.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f713c0b7070 0x7f713c0ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:54.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f713c1443a0 0x7f713c144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:54.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c0b29b0 con 0x7f713c1443a0 2021-11-21T22:00:54.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c067c30 con 0x7f713c0b7070 2021-11-21T22:00:54.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7142384700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b7910 con 0x7f713c067850 2021-11-21T22:00:54.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7140921700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f713c1443a0 0x7f713c144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:54.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7140921700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f713c1443a0 0x7f713c144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49598/0 (socket says 172.21.2.116:49598) 2021-11-21T22:00:54.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7140921700 1 -- 172.21.2.116:0/158516750 learned_addr learned my addr 172.21.2.116:0/158516750 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:54.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7140921700 1 -- 172.21.2.116:0/158516750 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 msgr2=0x7f713c0b3320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:54.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.234+0000 7f7140921700 1 --2- 172.21.2.116:0/158516750 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 0x7f713c0b3320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7140921700 1 -- 172.21.2.116:0/158516750 >> v2:172.21.2.117:3300/0 conn(0x7f713c0b7070 msgr2=0x7f713c0ab010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:54.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7140921700 1 --2- 172.21.2.116:0/158516750 >> v2:172.21.2.117:3300/0 conn(0x7f713c0b7070 0x7f713c0ab010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7140921700 1 -- 172.21.2.116:0/158516750 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0b2be0 con 0x7f713c1443a0 2021-11-21T22:00:54.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7140921700 1 --2- 172.21.2.116:0/158516750 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c1443a0 0x7f713c144830 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f71340032f0 tx=0x7f7134009290).ready entity=mon.0 client_cookie=35ef4c888b56b394 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:54.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713affd700 1 -- 172.21.2.116:0/158516750 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713400b870 con 0x7f713c1443a0 2021-11-21T22:00:54.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713affd700 1 -- 172.21.2.116:0/158516750 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713400b9d0 con 0x7f713c1443a0 2021-11-21T22:00:54.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713affd700 1 -- 172.21.2.116:0/158516750 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713400bb80 con 0x7f713c1443a0 2021-11-21T22:00:54.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/158516750 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c1443a0 msgr2=0x7f713c144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:54.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 --2- 172.21.2.116:0/158516750 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c1443a0 0x7f713c144830 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f71340032f0 tx=0x7f7134009290).stop 2021-11-21T22:00:54.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/158516750 shutdown_connections 2021-11-21T22:00:54.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 --2- 172.21.2.116:0/158516750 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c1443a0 0x7f713c144830 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:54.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 --2- 172.21.2.116:0/158516750 >> v2:172.21.2.117:3300/0 conn(0x7f713c0b7070 0x7f713c0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 --2- 172.21.2.116:0/158516750 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 0x7f713c0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/158516750 >> 172.21.2.116:0/158516750 conn(0x7f713c064d60 msgr2=0x7f713c065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:54.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/158516750 shutdown_connections 2021-11-21T22:00:54.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/158516750 wait complete. 2021-11-21T22:00:54.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 Processor -- start 2021-11-21T22:00:54.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- start start 2021-11-21T22:00:54.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 0x7f713c150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:54.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c0b7070 0x7f713c0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:54.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f713c1443a0 0x7f713c0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:54.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0a1ae0 con 0x7f713c0b7070 2021-11-21T22:00:54.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1510b0 con 0x7f713c1443a0 2021-11-21T22:00:54.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0a1300 con 0x7f713c067850 2021-11-21T22:00:54.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c0b7070 0x7f713c0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:54.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c0b7070 0x7f713c0b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49602/0 (socket says 172.21.2.116:49602) 2021-11-21T22:00:54.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 -- 172.21.2.116:0/957453200 learned_addr learned my addr 172.21.2.116:0/957453200 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:54.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 -- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 msgr2=0x7f713c150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:54.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 0x7f713c150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 -- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f713c1443a0 msgr2=0x7f713c0a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:54.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f713c1443a0 0x7f713c0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:54.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 -- 172.21.2.116:0/957453200 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7134009530 con 0x7f713c0b7070 2021-11-21T22:00:54.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f713b7fe700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c0b7070 0x7f713c0b0a60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f7124002fb0 tx=0x7f7124006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:54.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f71397fa700 1 -- 172.21.2.116:0/957453200 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7124013460 con 0x7f713c0b7070 2021-11-21T22:00:54.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f71397fa700 1 -- 172.21.2.116:0/957453200 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71240135c0 con 0x7f713c0b7070 2021-11-21T22:00:54.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c0a15a0 con 0x7f713c0b7070 2021-11-21T22:00:54.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c0b0f50 con 0x7f713c0b7070 2021-11-21T22:00:54.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f71397fa700 1 -- 172.21.2.116:0/957453200 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7124013920 con 0x7f713c0b7070 2021-11-21T22:00:54.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c09e810 con 0x7f713c0b7070 2021-11-21T22:00:54.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.238+0000 7f71397fa700 1 -- 172.21.2.116:0/957453200 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f712400f070 con 0x7f713c0b7070 2021-11-21T22:00:54.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.242+0000 7f71397fa700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7120053640 0x7f7120055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:54.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.242+0000 7f71397fa700 1 -- 172.21.2.116:0/957453200 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f712406b510 con 0x7f713c0b7070 2021-11-21T22:00:54.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.242+0000 7f713bfff700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7120053640 0x7f7120055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:54.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.242+0000 7f713bfff700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7120053640 0x7f7120055b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f712c002ef0 tx=0x7f712c006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:54.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.242+0000 7f71397fa700 1 -- 172.21.2.116:0/957453200 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71240424f0 con 0x7f713c0b7070 2021-11-21T22:00:54.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:54.326+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f713c0b2fe0 con 0x7f713c0b7070 2021-11-21T22:00:55.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f71397fa700 1 -- 172.21.2.116:0/957453200 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 185+0+0 (secure 0 0 0) 0x7f7124045b10 con 0x7f713c0b7070 2021-11-21T22:00:55.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7120053640 msgr2=0x7f7120055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:55.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7120053640 0x7f7120055b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f712c002ef0 tx=0x7f712c006910).stop 2021-11-21T22:00:55.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c0b7070 msgr2=0x7f713c0b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:55.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c0b7070 0x7f713c0b0a60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f7124002fb0 tx=0x7f7124006df0).stop 2021-11-21T22:00:55.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 shutdown_connections 2021-11-21T22:00:55.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f713c1443a0 0x7f713c0a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7120053640 0x7f7120055b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:55.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f713c0b7070 0x7f713c0b0a60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:55.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 --2- 172.21.2.116:0/957453200 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f713c067850 0x7f713c150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 >> 172.21.2.116:0/957453200 conn(0x7f713c064d60 msgr2=0x7f713c065140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:55.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 shutdown_connections 2021-11-21T22:00:55.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.182+0000 7f7142384700 1 -- 172.21.2.116:0/957453200 wait complete. 2021-11-21T22:00:55.191 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:00:55.194 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:00:55.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.242+0000 7f282bf28700 1 Processor -- start 2021-11-21T22:00:55.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.242+0000 7f282bf28700 1 -- start start 2021-11-21T22:00:55.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28240b11c0 0x7f2824147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:55.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f28240a8670 0x7f2824147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:55.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2824147c10 0x7f282414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:55.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28240b1940 con 0x7f2824147c10 2021-11-21T22:00:55.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28240b15a0 con 0x7f28240a8670 2021-11-21T22:00:55.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28240b1b70 con 0x7f28240b11c0 2021-11-21T22:00:55.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2824147c10 0x7f282414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:55.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2824147c10 0x7f282414a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49608/0 (socket says 172.21.2.116:49608) 2021-11-21T22:00:55.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 -- 172.21.2.116:0/946644867 learned_addr learned my addr 172.21.2.116:0/946644867 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:55.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 -- 172.21.2.116:0/946644867 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28240b11c0 msgr2=0x7f2824147230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:55.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 --2- 172.21.2.116:0/946644867 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28240b11c0 0x7f2824147230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 -- 172.21.2.116:0/946644867 >> v2:172.21.2.117:3300/0 conn(0x7f28240a8670 msgr2=0x7f2824147720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:55.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 --2- 172.21.2.116:0/946644867 >> v2:172.21.2.117:3300/0 conn(0x7f28240a8670 0x7f2824147720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 -- 172.21.2.116:0/946644867 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28240b1e10 con 0x7f2824147c10 2021-11-21T22:00:55.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282a4c5700 1 --2- 172.21.2.116:0/946644867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2824147c10 0x7f282414a6a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f281c0032f0 tx=0x7f281c009290).ready entity=mon.0 client_cookie=e7d74189aac60e31 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:55.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f2828cc2700 1 -- 172.21.2.116:0/946644867 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f281c00b870 con 0x7f2824147c10 2021-11-21T22:00:55.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f2828cc2700 1 -- 172.21.2.116:0/946644867 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c00b9d0 con 0x7f2824147c10 2021-11-21T22:00:55.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f2828cc2700 1 -- 172.21.2.116:0/946644867 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f281c00bb80 con 0x7f2824147c10 2021-11-21T22:00:55.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- 172.21.2.116:0/946644867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2824147c10 msgr2=0x7f282414a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:55.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- 172.21.2.116:0/946644867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2824147c10 0x7f282414a6a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f281c0032f0 tx=0x7f281c009290).stop 2021-11-21T22:00:55.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- 172.21.2.116:0/946644867 shutdown_connections 2021-11-21T22:00:55.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- 172.21.2.116:0/946644867 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2824147c10 0x7f282414a6a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:55.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- 172.21.2.116:0/946644867 >> v2:172.21.2.117:3300/0 conn(0x7f28240a8670 0x7f2824147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- 172.21.2.116:0/946644867 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f28240b11c0 0x7f2824147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- 172.21.2.116:0/946644867 >> 172.21.2.116:0/946644867 conn(0x7f282405bd20 msgr2=0x7f282405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:55.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- 172.21.2.116:0/946644867 shutdown_connections 2021-11-21T22:00:55.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- 172.21.2.116:0/946644867 wait complete. 2021-11-21T22:00:55.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 Processor -- start 2021-11-21T22:00:55.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- start start 2021-11-21T22:00:55.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28240a8670 0x7f2824146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:55.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28240b11c0 0x7f28240654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:55.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2824147c10 0x7f28240ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:55.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28240a9140 con 0x7f28240b11c0 2021-11-21T22:00:55.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28240b4ed0 con 0x7f28240a8670 2021-11-21T22:00:55.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2824067750 con 0x7f2824147c10 2021-11-21T22:00:55.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28240b11c0 0x7f28240654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:55.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28240b11c0 0x7f28240654e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49618/0 (socket says 172.21.2.116:49618) 2021-11-21T22:00:55.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 -- 172.21.2.116:0/4200011524 learned_addr learned my addr 172.21.2.116:0/4200011524 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:55.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 -- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2824147c10 msgr2=0x7f28240ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:55.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2824147c10 0x7f28240ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 -- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28240a8670 msgr2=0x7f2824146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:55.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28240a8670 0x7f2824146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:55.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 -- 172.21.2.116:0/4200011524 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c009530 con 0x7f28240b11c0 2021-11-21T22:00:55.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f28294c3700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28240b11c0 0x7f28240654e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f2824067080 tx=0x7f2820006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:55.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f2816ffd700 1 -- 172.21.2.116:0/4200011524 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2820009e80 con 0x7f28240b11c0 2021-11-21T22:00:55.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f2816ffd700 1 -- 172.21.2.116:0/4200011524 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2820002320 con 0x7f28240b11c0 2021-11-21T22:00:55.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28240a8ea0 con 0x7f28240b11c0 2021-11-21T22:00:55.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2824063850 con 0x7f28240b11c0 2021-11-21T22:00:55.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.246+0000 7f2816ffd700 1 -- 172.21.2.116:0/4200011524 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2820002680 con 0x7f28240b11c0 2021-11-21T22:00:55.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.250+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28240b5aa0 con 0x7f28240b11c0 2021-11-21T22:00:55.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.250+0000 7f2816ffd700 1 -- 172.21.2.116:0/4200011524 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f282000f030 con 0x7f28240b11c0 2021-11-21T22:00:55.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.250+0000 7f2816ffd700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2804053660 0x7f2804055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:55.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.250+0000 7f2816ffd700 1 -- 172.21.2.116:0/4200011524 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5760+0+0 (secure 0 0 0) 0x7f282006b5a0 con 0x7f28240b11c0 2021-11-21T22:00:55.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.250+0000 7f2829cc4700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2804053660 0x7f2804055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:55.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.250+0000 7f2829cc4700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2804053660 0x7f2804055b20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f2818002fb0 tx=0x7f28180069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:55.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.250+0000 7f2816ffd700 1 -- 172.21.2.116:0/4200011524 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2820042730 con 0x7f28240b11c0 2021-11-21T22:00:55.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:55.334+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28240b20b0 con 0x7f28240b11c0 2021-11-21T22:00:56.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f2816ffd700 1 -- 172.21.2.116:0/4200011524 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2820045d50 con 0x7f28240b11c0 2021-11-21T22:00:56.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2804053660 msgr2=0x7f2804055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2804053660 0x7f2804055b20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f2818002fb0 tx=0x7f28180069b0).stop 2021-11-21T22:00:56.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28240b11c0 msgr2=0x7f28240654e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28240b11c0 0x7f28240654e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f2824067080 tx=0x7f2820006d50).stop 2021-11-21T22:00:56.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 shutdown_connections 2021-11-21T22:00:56.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2824147c10 0x7f28240ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:56.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2804053660 0x7f2804055b20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f28240b11c0 0x7f28240654e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 --2- 172.21.2.116:0/4200011524 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f28240a8670 0x7f2824146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:56.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 >> 172.21.2.116:0/4200011524 conn(0x7f282405bd20 msgr2=0x7f2824149230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:56.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 shutdown_connections 2021-11-21T22:00:56.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.194+0000 7f282bf28700 1 -- 172.21.2.116:0/4200011524 wait complete. 2021-11-21T22:00:56.203 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:00:56.206 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T22:00:56.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44aee44700 1 Processor -- start 2021-11-21T22:00:56.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44aee44700 1 -- start start 2021-11-21T22:00:56.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44aee44700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44a80b6770 0x7f44a80b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44aee44700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 0x7f44a80ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44aee44700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80b0070 con 0x7f44a80b1270 2021-11-21T22:00:56.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44aee44700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a80b05c0 con 0x7f44a80b6770 2021-11-21T22:00:56.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44aee44700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80b2230 con 0x7f44a80ad580 2021-11-21T22:00:56.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44a80b6770 0x7f44a80b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44a80b6770 0x7f44a80b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49622/0 (socket says 172.21.2.116:49622) 2021-11-21T22:00:56.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 -- 172.21.2.116:0/1505447368 learned_addr learned my addr 172.21.2.116:0/1505447368 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:56.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 -- 172.21.2.116:0/1505447368 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 msgr2=0x7f44a80ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:56.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 --2- 172.21.2.116:0/1505447368 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 0x7f44a80ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:56.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 -- 172.21.2.116:0/1505447368 >> v1:172.21.2.117:6789/0 conn(0x7f44a80b1270 legacy=0x7f44a81394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:56.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 -- 172.21.2.116:0/1505447368 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80b24d0 con 0x7f44a80b6770 2021-11-21T22:00:56.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44acbe0700 1 --2- 172.21.2.116:0/1505447368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b6770 0x7f44a80b6b50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f44a8060900 tx=0x7f4490006eb0).ready entity=mon.0 client_cookie=424cc0cd70d9ecdc server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:56.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44a77fe700 1 -- 172.21.2.116:0/1505447368 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f449000d570 con 0x7f44a80b6770 2021-11-21T22:00:56.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44a77fe700 1 -- 172.21.2.116:0/1505447368 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4490000e90 con 0x7f44a80b6770 2021-11-21T22:00:56.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.254+0000 7f44a77fe700 1 -- 172.21.2.116:0/1505447368 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f449000d8c0 con 0x7f44a80b6770 2021-11-21T22:00:56.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/1505447368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b6770 msgr2=0x7f44a80b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 --2- 172.21.2.116:0/1505447368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b6770 0x7f44a80b6b50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f44a8060900 tx=0x7f4490006eb0).stop 2021-11-21T22:00:56.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/1505447368 shutdown_connections 2021-11-21T22:00:56.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 --2- 172.21.2.116:0/1505447368 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 0x7f44a80ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:56.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 --2- 172.21.2.116:0/1505447368 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b6770 0x7f44a80b6b50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/1505447368 >> 172.21.2.116:0/1505447368 conn(0x7f44a8062040 msgr2=0x7f44a80b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:56.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/1505447368 shutdown_connections 2021-11-21T22:00:56.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/1505447368 wait complete. 2021-11-21T22:00:56.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 Processor -- start 2021-11-21T22:00:56.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- start start 2021-11-21T22:00:56.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 0x7f44a815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b1270 0x7f44a8135320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44a80b6770 0x7f44a8135810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a813d660 con 0x7f44a80b1270 2021-11-21T22:00:56.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a815a9c0 con 0x7f44a80b6770 2021-11-21T22:00:56.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a813cc80 con 0x7f44a80ad580 2021-11-21T22:00:56.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b1270 0x7f44a8135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b1270 0x7f44a8135320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49632/0 (socket says 172.21.2.116:49632) 2021-11-21T22:00:56.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 -- 172.21.2.116:0/2745533768 learned_addr learned my addr 172.21.2.116:0/2745533768 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:56.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44ad3e1700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44a80b6770 0x7f44a8135810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44acbe0700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 0x7f44a815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 -- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 msgr2=0x7f44a815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 0x7f44a815a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 -- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44a80b6770 msgr2=0x7f44a8135810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44a80b6770 0x7f44a8135810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 -- 172.21.2.116:0/2745533768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449000b1e0 con 0x7f44a80b1270 2021-11-21T22:00:56.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44ad3e1700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44a80b6770 0x7f44a8135810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:00:56.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a7fff700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b1270 0x7f44a8135320 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f4498002f90 tx=0x7f4498006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:56.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a5ffb700 1 -- 172.21.2.116:0/2745533768 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4498013460 con 0x7f44a80b1270 2021-11-21T22:00:56.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a5ffb700 1 -- 172.21.2.116:0/2745533768 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44980135c0 con 0x7f44a80b1270 2021-11-21T22:00:56.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a5ffb700 1 -- 172.21.2.116:0/2745533768 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4498013920 con 0x7f44a80b1270 2021-11-21T22:00:56.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a813cf20 con 0x7f44a80b1270 2021-11-21T22:00:56.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a815a590 con 0x7f44a80b1270 2021-11-21T22:00:56.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a815abc0 con 0x7f44a80b1270 2021-11-21T22:00:56.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a5ffb700 1 -- 172.21.2.116:0/2745533768 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f449801a060 con 0x7f44a80b1270 2021-11-21T22:00:56.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a5ffb700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f448c0535d0 0x7f448c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.258+0000 7f44a5ffb700 1 -- 172.21.2.116:0/2745533768 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f449806b150 con 0x7f44a80b1270 2021-11-21T22:00:56.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.262+0000 7f44acbe0700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f448c0535d0 0x7f448c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.262+0000 7f44acbe0700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f448c0535d0 0x7f448c055a90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f449001a000 tx=0x7f4490015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:56.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.262+0000 7f44a5ffb700 1 -- 172.21.2.116:0/2745533768 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4498042450 con 0x7f44a80b1270 2021-11-21T22:00:56.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.346+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f44a80573d0 con 0x7f44a80b1270 2021-11-21T22:00:56.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44a5ffb700 1 -- 172.21.2.116:0/2745533768 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4498045a70 con 0x7f44a80b1270 2021-11-21T22:00:56.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f448c0535d0 msgr2=0x7f448c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f448c0535d0 0x7f448c055a90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f449001a000 tx=0x7f4490015000).stop 2021-11-21T22:00:56.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b1270 msgr2=0x7f44a8135320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b1270 0x7f44a8135320 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f4498002f90 tx=0x7f4498006df0).stop 2021-11-21T22:00:56.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 shutdown_connections 2021-11-21T22:00:56.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44a80b6770 0x7f44a8135810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f448c0535d0 0x7f448c055a90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44a80b1270 0x7f44a8135320 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 --2- 172.21.2.116:0/2745533768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44a80ad580 0x7f44a815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 >> 172.21.2.116:0/2745533768 conn(0x7f44a8062040 msgr2=0x7f44a80af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:56.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 shutdown_connections 2021-11-21T22:00:56.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.350+0000 7f44aee44700 1 -- 172.21.2.116:0/2745533768 wait complete. 2021-11-21T22:00:56.361 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2021-11-21T22:00:56.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.406+0000 7f62df941700 1 Processor -- start 2021-11-21T22:00:56.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.406+0000 7f62df941700 1 -- start start 2021-11-21T22:00:56.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f62d80b2540 0x7f62d8143590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d8143a80 con 0x7f62d80b2160 2021-11-21T22:00:56.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80b0170 con 0x7f62d80b1870 2021-11-21T22:00:56.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d8142a90 con 0x7f62d80b2540 2021-11-21T22:00:56.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62dd6dd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f62d80b2160 0x7f62d8140000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33360/0 (socket says 172.21.2.116:33360) 2021-11-21T22:00:56.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62dd6dd700 1 -- 172.21.2.116:0/1890281123 learned_addr learned my addr 172.21.2.116:0/1890281123 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:56.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178580932 0 0) 0x7f62d8143a80 con 0x7f62d80b2160 2021-11-21T22:00:56.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c4003260 con 0x7f62d80b2160 2021-11-21T22:00:56.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 806630561 0 0) 0x7f62c4003260 con 0x7f62d80b2160 2021-11-21T22:00:56.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 >> v2:172.21.2.117:3300/0 conn(0x7f62d80b2540 msgr2=0x7f62d8143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:56.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 --2- 172.21.2.116:0/1890281123 >> v2:172.21.2.117:3300/0 conn(0x7f62d80b2540 0x7f62d8143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:56.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 >> v1:172.21.2.117:6789/0 conn(0x7f62d80b1870 legacy=0x7f62d8142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:56.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80b0410 con 0x7f62d80b2160 2021-11-21T22:00:56.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f62cc001660 con 0x7f62d80b2160 2021-11-21T22:00:56.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 >> v1:172.21.2.116:6789/0 conn(0x7f62d80b2160 legacy=0x7f62d8140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62c4003820 0x7f62c4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4004500 0x7f62c4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c4005050 0x7f62c400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c400bf10 con 0x7f62c4005050 2021-11-21T22:00:56.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c400c080 con 0x7f62c4004500 2021-11-21T22:00:56.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c400c240 con 0x7f62c4003820 2021-11-21T22:00:56.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c4005050 0x7f62c400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62dcedc700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4004500 0x7f62c4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62dd6dd700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62c4003820 0x7f62c4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62c4003820 msgr2=0x7f62c4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:56.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62c4003820 0x7f62c4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4004500 msgr2=0x7f62c4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:56.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4004500 0x7f62c4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1890281123 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c4003260 con 0x7f62c4005050 2021-11-21T22:00:56.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62dcedc700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4004500 0x7f62c4004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:00:56.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1890281123 reap_dead start 2021-11-21T22:00:56.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c4005050 0x7f62c400b750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f62c8002f70 tx=0x7f62c8006ca0).ready entity=mon.0 client_cookie=3af9f00af215c3f9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:56.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c8012460 con 0x7f62c4005050 2021-11-21T22:00:56.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c80125c0 con 0x7f62c4005050 2021-11-21T22:00:56.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d7fff700 1 -- 172.21.2.116:0/1890281123 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c8012920 con 0x7f62c4005050 2021-11-21T22:00:56.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- 172.21.2.116:0/1890281123 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c4005050 msgr2=0x7f62c400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c4005050 0x7f62c400b750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f62c8002f70 tx=0x7f62c8006ca0).stop 2021-11-21T22:00:56.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- 172.21.2.116:0/1890281123 shutdown_connections 2021-11-21T22:00:56.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 --2- 172.21.2.116:0/1890281123 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c4005050 0x7f62c400b750 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- 172.21.2.116:0/1890281123 >> 172.21.2.116:0/1890281123 conn(0x7f62d805fd00 msgr2=0x7f62d80600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:56.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- 172.21.2.116:0/1890281123 shutdown_connections 2021-11-21T22:00:56.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- 172.21.2.116:0/1890281123 wait complete. 2021-11-21T22:00:56.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 Processor -- start 2021-11-21T22:00:56.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- start start 2021-11-21T22:00:56.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4005050 0x7f62d813a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d8141e90 0x7f62d8155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62d8155e80 0x7f62d8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8140e50 con 0x7f62d8155e80 2021-11-21T22:00:56.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d81555a0 con 0x7f62c4005050 2021-11-21T22:00:56.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62df941700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8141370 con 0x7f62d8141e90 2021-11-21T22:00:56.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62d8155e80 0x7f62d8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62d8155e80 0x7f62d8152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49652/0 (socket says 172.21.2.116:49652) 2021-11-21T22:00:56.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1491141745 learned_addr learned my addr 172.21.2.116:0/1491141745 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:56.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62dcedc700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d8141e90 0x7f62d8155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62dd6dd700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4005050 0x7f62d813a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d8141e90 msgr2=0x7f62d8155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d8141e90 0x7f62d8155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4005050 msgr2=0x7f62d813a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:56.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4005050 0x7f62d813a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:56.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 -- 172.21.2.116:0/1491141745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8141610 con 0x7f62d8155e80 2021-11-21T22:00:56.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62ddede700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62d8155e80 0x7f62d8152310 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f62c8002f70 tx=0x7f62c8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:56.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.410+0000 7f62d67fc700 1 -- 172.21.2.116:0/1491141745 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c8012460 con 0x7f62d8155e80 2021-11-21T22:00:56.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62d67fc700 1 -- 172.21.2.116:0/1491141745 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8000e90 con 0x7f62d8155e80 2021-11-21T22:00:56.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d81418b0 con 0x7f62d8155e80 2021-11-21T22:00:56.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62d67fc700 1 -- 172.21.2.116:0/1491141745 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c8006a80 con 0x7f62d8155e80 2021-11-21T22:00:56.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d8142880 con 0x7f62d8155e80 2021-11-21T22:00:56.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8066f20 con 0x7f62d8155e80 2021-11-21T22:00:56.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62d67fc700 1 -- 172.21.2.116:0/1491141745 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f62c801c070 con 0x7f62d8155e80 2021-11-21T22:00:56.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62d67fc700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b8053640 0x7f62b8055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:56.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62d67fc700 1 -- 172.21.2.116:0/1491141745 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f62c806bf70 con 0x7f62d8155e80 2021-11-21T22:00:56.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62dd6dd700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b8053640 0x7f62b8055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:56.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62dd6dd700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b8053640 0x7f62b8055b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f62cc00d000 tx=0x7f62cc007040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:56.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.414+0000 7f62d67fc700 1 -- 172.21.2.116:0/1491141745 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f62c8043380 con 0x7f62d8155e80 2021-11-21T22:00:56.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:56.526+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f62d8141a40 con 0x7f62d8155e80 2021-11-21T22:00:57.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62d67fc700 1 -- 172.21.2.116:0/1491141745 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v271) v1 ==== 139+0+0 (secure 0 0 0) 0x7f62c80469a0 con 0x7f62d8155e80 2021-11-21T22:00:57.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b8053640 msgr2=0x7f62b8055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:57.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b8053640 0x7f62b8055b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f62cc00d000 tx=0x7f62cc007040).stop 2021-11-21T22:00:57.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62d8155e80 msgr2=0x7f62d8152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:57.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62d8155e80 0x7f62d8152310 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f62c8002f70 tx=0x7f62c8014000).stop 2021-11-21T22:00:57.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 shutdown_connections 2021-11-21T22:00:57.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b8053640 0x7f62b8055b00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:57.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62d8155e80 0x7f62d8152310 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:57.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d8141e90 0x7f62d8155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:57.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 --2- 172.21.2.116:0/1491141745 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c4005050 0x7f62d813a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:57.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 >> 172.21.2.116:0/1491141745 conn(0x7f62d805fd00 msgr2=0x7f62d80b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:57.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 shutdown_connections 2021-11-21T22:00:57.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.206+0000 7f62df941700 1 -- 172.21.2.116:0/1491141745 wait complete. 2021-11-21T22:00:57.216 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2021-11-21T22:00:57.217 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:00:57.217 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2021-11-21T22:00:57.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 Processor -- start 2021-11-21T22:00:57.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- start start 2021-11-21T22:00:57.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f52280b24a0 0x7f52280b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f52280af360 0x7f52280ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52280b00d0 con 0x7f52280aeb70 2021-11-21T22:00:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52280adba0 con 0x7f52280af360 2021-11-21T22:00:57.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52280af7a0 con 0x7f52280b24a0 2021-11-21T22:00:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f52280af360 0x7f52280ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f52280af360 0x7f52280ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49658/0 (socket says 172.21.2.116:49658) 2021-11-21T22:00:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 -- 172.21.2.116:0/1038843849 learned_addr learned my addr 172.21.2.116:0/1038843849 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:57.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 -- 172.21.2.116:0/1038843849 >> v2:172.21.2.117:3300/0 conn(0x7f52280b24a0 msgr2=0x7f52280b1a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:57.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 --2- 172.21.2.116:0/1038843849 >> v2:172.21.2.117:3300/0 conn(0x7f52280b24a0 0x7f52280b1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:57.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 -- 172.21.2.116:0/1038843849 >> v1:172.21.2.117:6789/0 conn(0x7f52280aeb70 legacy=0x7f522813c210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:00:57.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 -- 172.21.2.116:0/1038843849 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280b2880 con 0x7f52280af360 2021-11-21T22:00:57.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f5227fff700 1 --2- 172.21.2.116:0/1038843849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280af360 0x7f52280ae680 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f52280630d0 tx=0x7f5210006f40).ready entity=mon.0 client_cookie=b388ea79f592fa9d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f52277fe700 1 -- 172.21.2.116:0/1038843849 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f521000c900 con 0x7f52280af360 2021-11-21T22:00:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f52277fe700 1 -- 172.21.2.116:0/1038843849 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5210000e90 con 0x7f52280af360 2021-11-21T22:00:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f52277fe700 1 -- 172.21.2.116:0/1038843849 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f521000cc50 con 0x7f52280af360 2021-11-21T22:00:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- 172.21.2.116:0/1038843849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280af360 msgr2=0x7f52280ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:57.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 --2- 172.21.2.116:0/1038843849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280af360 0x7f52280ae680 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f52280630d0 tx=0x7f5210006f40).stop 2021-11-21T22:00:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- 172.21.2.116:0/1038843849 shutdown_connections 2021-11-21T22:00:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 --2- 172.21.2.116:0/1038843849 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280af360 0x7f52280ae680 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 --2- 172.21.2.116:0/1038843849 >> v2:172.21.2.117:3300/0 conn(0x7f52280b24a0 0x7f52280b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:57.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- 172.21.2.116:0/1038843849 >> 172.21.2.116:0/1038843849 conn(0x7f522805fd00 msgr2=0x7f52280600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- 172.21.2.116:0/1038843849 shutdown_connections 2021-11-21T22:00:57.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.266+0000 7f522f1ca700 1 -- 172.21.2.116:0/1038843849 wait complete. 2021-11-21T22:00:57.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 Processor -- start 2021-11-21T22:00:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 -- start start 2021-11-21T22:00:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280aeb70 0x7f5228142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52280af360 0x7f5228066250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:57.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280b24a0 0x7f52280b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228143380 con 0x7f52280aeb70 2021-11-21T22:00:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228064ed0 con 0x7f52280af360 2021-11-21T22:00:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52281430e0 con 0x7f52280b24a0 2021-11-21T22:00:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280aeb70 0x7f5228142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:57.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280aeb70 0x7f5228142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49662/0 (socket says 172.21.2.116:49662) 2021-11-21T22:00:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 -- 172.21.2.116:0/3651455605 learned_addr learned my addr 172.21.2.116:0/3651455605 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 -- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280b24a0 msgr2=0x7f52280b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280b24a0 0x7f52280b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:57.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 -- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52280af360 msgr2=0x7f5228066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52280af360 0x7f5228066250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 -- 172.21.2.116:0/3651455605 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521000a360 con 0x7f52280aeb70 2021-11-21T22:00:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522cf66700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280aeb70 0x7f5228142350 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f5218000ed0 tx=0x7f5218006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f5225ffb700 1 -- 172.21.2.116:0/3651455605 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f521800f7d0 con 0x7f52280aeb70 2021-11-21T22:00:57.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f5225ffb700 1 -- 172.21.2.116:0/3651455605 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521800f930 con 0x7f52280aeb70 2021-11-21T22:00:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5228064c30 con 0x7f52280aeb70 2021-11-21T22:00:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f5225ffb700 1 -- 172.21.2.116:0/3651455605 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f521800fc90 con 0x7f52280aeb70 2021-11-21T22:00:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5228141c30 con 0x7f52280aeb70 2021-11-21T22:00:57.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52280573d0 con 0x7f52280aeb70 2021-11-21T22:00:57.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f5225ffb700 1 -- 172.21.2.116:0/3651455605 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5218010470 con 0x7f52280aeb70 2021-11-21T22:00:57.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f5225ffb700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0536d0 0x7f520c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:57.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f5225ffb700 1 -- 172.21.2.116:0/3651455605 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f521806d5f0 con 0x7f52280aeb70 2021-11-21T22:00:57.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.270+0000 7f5227fff700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0536d0 0x7f520c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:57.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.274+0000 7f5227fff700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0536d0 0x7f520c055b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f52280630d0 tx=0x7f522805cd30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:57.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.274+0000 7f5225ffb700 1 -- 172.21.2.116:0/3651455605 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52180449a0 con 0x7f52280aeb70 2021-11-21T22:00:57.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:57.386+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f5228067590 con 0x7f52280aeb70 2021-11-21T22:00:58.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f5225ffb700 1 -- 172.21.2.116:0/3651455605 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v272) v1 ==== 129+0+0 (secure 0 0 0) 0x7f5218047fc0 con 0x7f52280aeb70 2021-11-21T22:00:58.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0536d0 msgr2=0x7f520c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0536d0 0x7f520c055b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f52280630d0 tx=0x7f522805cd30).stop 2021-11-21T22:00:58.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280aeb70 msgr2=0x7f5228142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280aeb70 0x7f5228142350 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f5218000ed0 tx=0x7f5218006d70).stop 2021-11-21T22:00:58.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 shutdown_connections 2021-11-21T22:00:58.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280b24a0 0x7f52280b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f52280af360 0x7f5228066250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0536d0 0x7f520c055b90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 --2- 172.21.2.116:0/3651455605 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52280aeb70 0x7f5228142350 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 >> 172.21.2.116:0/3651455605 conn(0x7f522805fd00 msgr2=0x7f522813c730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 shutdown_connections 2021-11-21T22:00:58.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.214+0000 7f522f1ca700 1 -- 172.21.2.116:0/3651455605 wait complete. 2021-11-21T22:00:58.225 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T22:00:58.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce025b8700 1 Processor -- start 2021-11-21T22:00:58.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce025b8700 1 -- start start 2021-11-21T22:00:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce025b8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 0x7fcdfc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce025b8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcdfc0ac600 0x7fcdfc0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce025b8700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc147e80 con 0x7fcdfc0abce0 2021-11-21T22:00:58.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce025b8700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc0b1490 con 0x7fcdfc0ac600 2021-11-21T22:00:58.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce025b8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0b1fd0 con 0x7fcdfc144810 2021-11-21T22:00:58.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce00b55700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcdfc0ac600 0x7fcdfc0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce00b55700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcdfc0ac600 0x7fcdfc0aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49670/0 (socket says 172.21.2.116:49670) 2021-11-21T22:00:58.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce00b55700 1 -- 172.21.2.116:0/1850238212 learned_addr learned my addr 172.21.2.116:0/1850238212 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:58.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fcdfb7fe700 1 --1- 172.21.2.116:0/1850238212 >> v1:172.21.2.116:6789/0 conn(0x7fcdfc0abce0 0x7fcdfc0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33398/0 (socket says 172.21.2.116:33398) 2021-11-21T22:00:58.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce00b55700 1 -- 172.21.2.116:0/1850238212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 msgr2=0x7fcdfc1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:58.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce00b55700 1 --2- 172.21.2.116:0/1850238212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 0x7fcdfc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce00b55700 1 -- 172.21.2.116:0/1850238212 >> v1:172.21.2.116:6789/0 conn(0x7fcdfc0abce0 legacy=0x7fcdfc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.286+0000 7fce00b55700 1 -- 172.21.2.116:0/1850238212 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc0af5a0 con 0x7fcdfc0ac600 2021-11-21T22:00:58.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce00b55700 1 --2- 172.21.2.116:0/1850238212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0ac600 0x7fcdfc0aca90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fcdf40032f0 tx=0x7fcdf4009290).ready entity=mon.0 client_cookie=91c1d11450064e7b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfaffd700 1 -- 172.21.2.116:0/1850238212 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf400b8a0 con 0x7fcdfc0ac600 2021-11-21T22:00:58.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfaffd700 1 -- 172.21.2.116:0/1850238212 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf400ba00 con 0x7fcdfc0ac600 2021-11-21T22:00:58.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfaffd700 1 -- 172.21.2.116:0/1850238212 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf400bb70 con 0x7fcdfc0ac600 2021-11-21T22:00:58.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/1850238212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0ac600 msgr2=0x7fcdfc0aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 --2- 172.21.2.116:0/1850238212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0ac600 0x7fcdfc0aca90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fcdf40032f0 tx=0x7fcdf4009290).stop 2021-11-21T22:00:58.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/1850238212 shutdown_connections 2021-11-21T22:00:58.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 --2- 172.21.2.116:0/1850238212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0ac600 0x7fcdfc0aca90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 --2- 172.21.2.116:0/1850238212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 0x7fcdfc1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/1850238212 >> 172.21.2.116:0/1850238212 conn(0x7fcdfc05bd20 msgr2=0x7fcdfc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:58.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/1850238212 shutdown_connections 2021-11-21T22:00:58.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/1850238212 wait complete. 2021-11-21T22:00:58.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 Processor -- start 2021-11-21T22:00:58.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- start start 2021-11-21T22:00:58.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0abce0 0x7fcdfc1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcdfc0ac600 0x7fcdfc143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 0x7fcdfc15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc159ab0 con 0x7fcdfc0abce0 2021-11-21T22:00:58.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0b3110 con 0x7fcdfc0ac600 2021-11-21T22:00:58.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0b33b0 con 0x7fcdfc144810 2021-11-21T22:00:58.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0abce0 0x7fcdfc1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0abce0 0x7fcdfc1564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49680/0 (socket says 172.21.2.116:49680) 2021-11-21T22:00:58.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 -- 172.21.2.116:0/3730925749 learned_addr learned my addr 172.21.2.116:0/3730925749 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:58.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce00b55700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 0x7fcdfc15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 -- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 msgr2=0x7fcdfc15a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 0x7fcdfc15a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 -- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcdfc0ac600 msgr2=0x7fcdfc143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcdfc0ac600 0x7fcdfc143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 -- 172.21.2.116:0/3730925749 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4009530 con 0x7fcdfc0abce0 2021-11-21T22:00:58.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdfbfff700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0abce0 0x7fcdfc1564e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fcde4002f70 tx=0x7fcde4006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdf97fa700 1 -- 172.21.2.116:0/3730925749 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcde4013460 con 0x7fcdfc0abce0 2021-11-21T22:00:58.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdf97fa700 1 -- 172.21.2.116:0/3730925749 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde40135c0 con 0x7fcdfc0abce0 2021-11-21T22:00:58.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdf97fa700 1 -- 172.21.2.116:0/3730925749 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcde4013920 con 0x7fcdfc0abce0 2021-11-21T22:00:58.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc159ce0 con 0x7fcdfc0abce0 2021-11-21T22:00:58.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc146cf0 con 0x7fcdfc0abce0 2021-11-21T22:00:58.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc159e70 con 0x7fcdfc0abce0 2021-11-21T22:00:58.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.290+0000 7fcdf97fa700 1 -- 172.21.2.116:0/3730925749 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcde400f070 con 0x7fcdfc0abce0 2021-11-21T22:00:58.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.294+0000 7fcdf97fa700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcde0053580 0x7fcde0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.294+0000 7fcdf97fa700 1 -- 172.21.2.116:0/3730925749 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fcde406b070 con 0x7fcdfc0abce0 2021-11-21T22:00:58.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.294+0000 7fcdfb7fe700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcde0053580 0x7fcde0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.294+0000 7fcdfb7fe700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcde0053580 0x7fcde0055a40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fcdfc0b2a30 tx=0x7fcdec007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.294+0000 7fcdf97fa700 1 -- 172.21.2.116:0/3730925749 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcde4042310 con 0x7fcdfc0abce0 2021-11-21T22:00:58.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.378+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcdfc0af060 con 0x7fcdfc0abce0 2021-11-21T22:00:58.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fcdf97fa700 1 -- 172.21.2.116:0/3730925749 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcde4045930 con 0x7fcdfc0abce0 2021-11-21T22:00:58.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcde0053580 msgr2=0x7fcde0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcde0053580 0x7fcde0055a40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fcdfc0b2a30 tx=0x7fcdec007800).stop 2021-11-21T22:00:58.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0abce0 msgr2=0x7fcdfc1564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0abce0 0x7fcdfc1564e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fcde4002f70 tx=0x7fcde4006e20).stop 2021-11-21T22:00:58.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 shutdown_connections 2021-11-21T22:00:58.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcdfc144810 0x7fcdfc15a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcdfc0ac600 0x7fcdfc143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcde0053580 0x7fcde0055a40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 --2- 172.21.2.116:0/3730925749 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcdfc0abce0 0x7fcdfc1564e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 >> 172.21.2.116:0/3730925749 conn(0x7fcdfc05bd20 msgr2=0x7fcdfc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:58.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 shutdown_connections 2021-11-21T22:00:58.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.382+0000 7fce025b8700 1 -- 172.21.2.116:0/3730925749 wait complete. 2021-11-21T22:00:58.394 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:00:58.395 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:58.403 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:58.412 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:58.426 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:00:58.440 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:00:58.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:58.441 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:58.441 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:00:58.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:00:58.474 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:00:58.474 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:00:58.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:00:58.475 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:00:58.475 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:00:58.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:00:58.481 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:00:58.481 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:00:58.482 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:00:58.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f759957c700 1 Processor -- start 2021-11-21T22:00:58.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f759957c700 1 -- start start 2021-11-21T22:00:58.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f759957c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 0x7f759414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f759957c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75940a6e80 con 0x7f75940a86b0 2021-11-21T22:00:58.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f759957c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759414a7b0 con 0x7f7594062410 2021-11-21T22:00:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f759957c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75940a8bc0 con 0x7f75940a6aa0 2021-11-21T22:00:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f75927fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f75940a86b0 0x7f75941470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33410/0 (socket says 172.21.2.116:33410) 2021-11-21T22:00:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f75927fc700 1 -- 172.21.2.116:0/2277157073 learned_addr learned my addr 172.21.2.116:0/2277157073 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:00:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098353348 0 0) 0x7f75940a6e80 con 0x7f75940a86b0 2021-11-21T22:00:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7574003260 con 0x7f75940a86b0 2021-11-21T22:00:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 0x7f759414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2345069968 0 0) 0x7f7574003260 con 0x7f75940a86b0 2021-11-21T22:00:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 msgr2=0x7f759414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 0x7f759414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 >> v1:172.21.2.117:6789/0 conn(0x7f7594062410 legacy=0x7f75941443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759414aa10 con 0x7f75940a86b0 2021-11-21T22:00:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f7584001680 con 0x7f75940a86b0 2021-11-21T22:00:58.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 >> v1:172.21.2.116:6789/0 conn(0x7f75940a86b0 legacy=0x7f75941470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7574003800 0x7f7574003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 0x7f7574004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574004b10 0x7f7574004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940a8bc0 con 0x7f7574003800 2021-11-21T22:00:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757400b910 con 0x7f7574004b10 2021-11-21T22:00:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757400bad0 con 0x7f75940a6aa0 2021-11-21T22:00:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7574003800 0x7f7574003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f75927fc700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 0x7f7574004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f75937fe700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574004b10 0x7f7574004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 -- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 msgr2=0x7f7574004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75940a6aa0 0x7f7574004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 -- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574004b10 msgr2=0x7f7574004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574004b10 0x7f7574004f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 -- 172.21.2.116:0/2277157073 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574003260 con 0x7f7574003800 2021-11-21T22:00:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f75937fe700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574004b10 0x7f7574004f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7592ffd700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7574003800 0x7f7574003c00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f757c00a000 tx=0x7f757c008040).ready entity=mon.0 client_cookie=b1a90aa65b530b0e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f757c013670 con 0x7f7574003800 2021-11-21T22:00:58.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c0137d0 con 0x7f7574003800 2021-11-21T22:00:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.550+0000 7f7591ffb700 1 -- 172.21.2.116:0/2277157073 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f757c013b30 con 0x7f7574003800 2021-11-21T22:00:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/2277157073 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7574003800 msgr2=0x7f7574003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 --2- 172.21.2.116:0/2277157073 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7574003800 0x7f7574003c00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f757c00a000 tx=0x7f757c008040).stop 2021-11-21T22:00:58.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75937fe700 1 -- 172.21.2.116:0/2277157073 reap_dead start 2021-11-21T22:00:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/2277157073 shutdown_connections 2021-11-21T22:00:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/2277157073 >> 172.21.2.116:0/2277157073 conn(0x7f759405bd20 msgr2=0x7f759405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/2277157073 shutdown_connections 2021-11-21T22:00:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/2277157073 wait complete. 2021-11-21T22:00:58.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 Processor -- start 2021-11-21T22:00:58.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- start start 2021-11-21T22:00:58.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574003800 0x7f75940b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7594159c50 0x7f759415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75941566f0 0x7f7594156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7594156300 con 0x7f7594159c50 2021-11-21T22:00:58.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940ac600 con 0x7f7574003800 2021-11-21T22:00:58.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75940ac8a0 con 0x7f75941566f0 2021-11-21T22:00:58.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7594159c50 0x7f759415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7594159c50 0x7f759415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49700/0 (socket says 172.21.2.116:49700) 2021-11-21T22:00:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 -- 172.21.2.116:0/4016346080 learned_addr learned my addr 172.21.2.116:0/4016346080 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75937fe700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75941566f0 0x7f7594156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f7592ffd700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574003800 0x7f75940b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 -- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75941566f0 msgr2=0x7f7594156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75941566f0 0x7f7594156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 -- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574003800 msgr2=0x7f75940b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574003800 0x7f75940b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 -- 172.21.2.116:0/4016346080 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75940acb40 con 0x7f7594159c50 2021-11-21T22:00:58.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f75927fc700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7594159c50 0x7f759415a0d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f758400b000 tx=0x7f7584001e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f758bfff700 1 -- 172.21.2.116:0/4016346080 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7584010640 con 0x7f7594159c50 2021-11-21T22:00:58.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f758bfff700 1 -- 172.21.2.116:0/4016346080 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840107a0 con 0x7f7594159c50 2021-11-21T22:00:58.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7594153f40 con 0x7f7594159c50 2021-11-21T22:00:58.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f758bfff700 1 -- 172.21.2.116:0/4016346080 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7584010b00 con 0x7f7594159c50 2021-11-21T22:00:58.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75940ac280 con 0x7f7594159c50 2021-11-21T22:00:58.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75940573d0 con 0x7f7594159c50 2021-11-21T22:00:58.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f758bfff700 1 -- 172.21.2.116:0/4016346080 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f758401a020 con 0x7f7594159c50 2021-11-21T22:00:58.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f758bfff700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75780536a0 0x7f7578055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.554+0000 7f758bfff700 1 -- 172.21.2.116:0/4016346080 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f758406b610 con 0x7f7594159c50 2021-11-21T22:00:58.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.558+0000 7f7592ffd700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75780536a0 0x7f7578055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.558+0000 7f7592ffd700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75780536a0 0x7f7578055b60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f757c019000 tx=0x7f757c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.558+0000 7f758bfff700 1 -- 172.21.2.116:0/4016346080 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f75840429c0 con 0x7f7594159c50 2021-11-21T22:00:58.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.638+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f75940accd0 con 0x7f7594159c50 2021-11-21T22:00:58.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f758bfff700 1 -- 172.21.2.116:0/4016346080 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v80) v1 ==== 81+0+28900 (secure 0 0 0) 0x7f7584045fe0 con 0x7f7594159c50 2021-11-21T22:00:58.645 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:58.645 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:00:58.646 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 80, 2021-11-21T22:00:58.646 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:00:58.646 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:00:58.646 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:00:56.187627+0000", 2021-11-21T22:00:58.647 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:00:58.647 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:00:58.647 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:00:58.647 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:00:58.648 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:00:58.648 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:00:58.648 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:00:58.648 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:00:58.648 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:00:58.649 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:58.649 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:00:58.649 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:00:58.649 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:00:58.650 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:00:58.650 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:00:58.650 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 14, 2021-11-21T22:00:58.650 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:00:58.650 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:00:58.651 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:00:58.651 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:00:58.651 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:00:58.651 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.652 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:00:58.652 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:00:58.652 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:00:58.652 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:00:58.652 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:00:58.653 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:00:58.653 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:00:58.653 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:00:58.653 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:00:58.653 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:00:58.654 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:00:58.654 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:00:58.654 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:00:58.654 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:00:58.655 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:00:58.655 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:00:58.655 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:00:58.655 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:00:58.655 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:00:58.656 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:00:58.656 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:00:58.656 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:00:58.656 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:00:58.656 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.657 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:00:58.657 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:00:58.657 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:00:58.657 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:00:58.658 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:00:58.658 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:00:58.658 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:00:58.658 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:00:58.658 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:00:58.659 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:00:58.659 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:00:58.659 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:00:58.659 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:00:58.660 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:00:58.660 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:00:58.660 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:00:58.660 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:00:58.660 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:00:58.661 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:00:58.661 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:00:58.661 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:00:58.661 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:00:58.661 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:00:58.662 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:00:58.662 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:00:58.662 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:00:58.662 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:00:58.663 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.663 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:00:58.663 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:00:58.663 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:00:58.663 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:00:58.664 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:00:58.664 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:00:58.664 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:00:58.664 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:00:58.664 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:00:58.665 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:00:58.665 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:00:58.665 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:00:58.665 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:00:58.666 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.666 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:00:58.666 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:00:58.666 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.667 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.667 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.667 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:00:58.667 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:00:58.667 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:00:58.668 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:00:58.668 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:00:58.668 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:00:58.668 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:00:58.668 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:00:58.669 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:00:58.669 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:00:58.669 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:00:58.669 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:00:58.670 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:00:58.670 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:00:58.670 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:00:58.670 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:00:58.670 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:00:58.671 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:00:58.671 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:00:58.671 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:00:58.671 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:00:58.672 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:00:58.672 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:00:58.672 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.672 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:00:58.672 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:00:58.673 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:00:58.673 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:00:58.673 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:00:58.673 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:00:58.674 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:00:58.674 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:00:58.674 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:00:58.674 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:00:58.674 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:00:58.675 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:00:58.675 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:00:58.675 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:00:58.675 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:00:58.676 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:00:58.676 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:00:58.676 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:00:58.676 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:00:58.676 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:00:58.677 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:00:58.677 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:00:58.677 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:00:58.677 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:00:58.678 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:00:58.678 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:00:58.678 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:00:58.678 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.678 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:00:58.679 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:00:58.679 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:00:58.679 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:00:58.679 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:00:58.680 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:00:58.680 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:00:58.680 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:00:58.680 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:00:58.680 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:00:58.681 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:00:58.681 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:00:58.681 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:00:58.681 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:00:58.682 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.682 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.682 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:58.682 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:00:58.682 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.683 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:00:58.683 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:00:58.683 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.683 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.684 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.684 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.684 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.685 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.685 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.685 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:58.685 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.686 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.686 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.686 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.686 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.686 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.687 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:00:58.687 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.687 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.688 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.688 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.688 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:00:58.688 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.688 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.689 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.689 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.689 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.689 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.690 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.690 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.690 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:00:58.690 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.690 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.691 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.691 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.691 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:00:58.691 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.692 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.692 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.692 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.692 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.692 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.693 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.693 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.693 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:00:58.693 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.694 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.694 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.694 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.694 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:00:58.694 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.695 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.695 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.695 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.695 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.696 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.696 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.696 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.696 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:00:58.697 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.697 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.697 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.698 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.698 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:00:58.698 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:00:58.698 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.698 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.699 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.699 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:00:58.699 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:00:58.699 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:00:58.700 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:00:58.700 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.700 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.700 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.700 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.701 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.701 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.701 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:00:58.701 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:00:58.702 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.702 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.702 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.702 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.703 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.703 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.703 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.703 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 64, 2021-11-21T22:00:58.703 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.704 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.704 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.704 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.704 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.705 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.705 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:00:58.705 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.705 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.706 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.706 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.706 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:00:58.706 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.706 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.707 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.707 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.707 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.707 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.708 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.708 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.708 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:00:58.708 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.708 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.709 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.709 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.709 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:00:58.709 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.710 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.710 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.710 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.710 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.710 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.711 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.711 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.711 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:00:58.711 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.712 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.712 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.712 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.712 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:00:58.712 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.713 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.713 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.713 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.713 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.714 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.714 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.714 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.714 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:00:58.715 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.715 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.715 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.715 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.715 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:00:58.716 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:00:58.716 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.716 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.716 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.717 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:00:58.717 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:00:58.717 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:00:58.717 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:00:58.718 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.718 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.718 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.718 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.718 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.719 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.719 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:00:58.719 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:00:58.719 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.720 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.720 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.720 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.720 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.720 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.721 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.721 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:58.721 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.721 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.722 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.722 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.722 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.722 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.723 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:00:58.723 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.723 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.723 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.724 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.724 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:00:58.724 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.724 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.725 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.725 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.725 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.725 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.725 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.726 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.726 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:00:58.726 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.726 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.727 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.727 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.727 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:00:58.727 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.727 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.728 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.728 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.728 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.728 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.728 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.729 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.729 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:00:58.729 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.729 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.730 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.730 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.730 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:00:58.730 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.731 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.731 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.731 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.731 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.731 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.732 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.732 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.732 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:00:58.732 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.733 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.733 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.733 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.733 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:00:58.733 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:00:58.734 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.734 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.734 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.734 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:00:58.734 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:00:58.735 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:00:58.735 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:00:58.735 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.735 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.736 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.736 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.736 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.736 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.736 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:00:58.737 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:00:58.737 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.737 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.737 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.738 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.738 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.738 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.738 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.739 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:58.739 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.739 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.739 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.739 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.740 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.740 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.740 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:00:58.740 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.741 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.741 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.741 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.741 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:00:58.741 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.742 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.742 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.742 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.742 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.743 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.743 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.743 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.743 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:00:58.743 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.744 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.744 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.744 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.744 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:00:58.745 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.745 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.745 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.745 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.745 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.746 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.746 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.746 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.746 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:00:58.747 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.747 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.747 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.747 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.747 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:00:58.748 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.748 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.748 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.748 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.749 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.749 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.749 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.749 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.750 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:00:58.750 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.750 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.750 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.750 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.751 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:00:58.751 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:00:58.751 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.751 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.752 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.752 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:00:58.752 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:00:58.752 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:00:58.752 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:00:58.753 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.753 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.753 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.753 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.754 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.754 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.754 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:00:58.754 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:00:58.755 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.755 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.755 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.755 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.755 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.756 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.756 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.756 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 72, 2021-11-21T22:00:58.756 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.757 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.757 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.757 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.757 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.757 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.758 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:00:58.758 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.758 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.758 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.759 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.759 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:00:58.759 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.759 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.759 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.760 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.760 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.760 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.760 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.761 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.761 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:00:58.761 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.761 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.761 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.762 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.762 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:00:58.762 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.762 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.763 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.763 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.763 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.763 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.764 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.764 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.764 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:00:58.764 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.764 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.765 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.765 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.765 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:00:58.765 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.766 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.766 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.766 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.766 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.766 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.767 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.767 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.767 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:00:58.767 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.768 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.768 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.768 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.768 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:00:58.768 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:00:58.769 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.769 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.769 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.769 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:00:58.769 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:00:58.770 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:00:58.770 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:00:58.770 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.770 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.771 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.771 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.771 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.771 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.771 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:00:58.772 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:00:58.772 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.772 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.772 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.772 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.773 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.773 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.773 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.773 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:58.773 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.774 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.774 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.774 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.774 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.775 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.775 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:00:58.775 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.775 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.775 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.776 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.776 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:00:58.776 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.776 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.776 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.777 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.777 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.777 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.777 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.778 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.778 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:00:58.778 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.778 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.778 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.779 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.779 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:00:58.779 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.779 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.779 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.780 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.780 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.780 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.780 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.781 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.781 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:00:58.781 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.781 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.781 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.782 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.782 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:00:58.782 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.782 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.782 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.783 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.783 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.783 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.783 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.784 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.784 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:00:58.784 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.784 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.784 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.785 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.785 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:00:58.785 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:00:58.785 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.785 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.786 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.786 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:00:58.786 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:00:58.786 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:00:58.787 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:00:58.787 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.787 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.787 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.787 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.788 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.788 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.788 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:00:58.788 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:00:58.789 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.789 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.789 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.789 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.789 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.790 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.790 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.790 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 73, 2021-11-21T22:00:58.790 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.790 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.791 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.791 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.791 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.791 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.792 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:00:58.792 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.792 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.792 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.792 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.793 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:00:58.793 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.793 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.793 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.794 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.794 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.794 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.794 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.794 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.795 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:00:58.795 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.795 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.795 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.796 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.796 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:00:58.796 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.796 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.797 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.797 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.797 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.797 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.797 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.798 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.798 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:00:58.798 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.798 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.799 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.799 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.799 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:00:58.799 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.799 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.800 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.800 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.800 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.800 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.800 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.801 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.801 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:00:58.801 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.801 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.802 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.802 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.802 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:00:58.802 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:00:58.802 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.803 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.803 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.803 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:00:58.803 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:00:58.804 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:00:58.804 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:00:58.804 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.804 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.804 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.805 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.805 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.805 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.805 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:00:58.806 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:00:58.806 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:00:58.806 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:00:58.806 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:00:58.807 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:00:58.807 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:00:58.807 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:00:58.807 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:00:58.807 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 72, 2021-11-21T22:00:58.808 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:00:58.808 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:00:58.808 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:00:58.808 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.808 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.809 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.809 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:00:58.809 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.809 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.810 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.810 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.810 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:00:58.810 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.811 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.811 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.811 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.812 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:00:58.812 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.812 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.812 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.813 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:00:58.813 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.813 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.813 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.813 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.814 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:00:58.814 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.814 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.814 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.815 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.815 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:00:58.815 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.815 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.815 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.816 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:00:58.816 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.816 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.816 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.816 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.817 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:00:58.817 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.817 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.817 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.818 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.818 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:00:58.818 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:00:58.818 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.818 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:00:58.819 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:00:58.819 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.819 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.819 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.819 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:00:58.820 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:00:58.820 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:00:58.820 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.820 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.821 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.821 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:00:58.821 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:00:58.821 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:00:58.821 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:00:58.822 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:00:58.822 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:00:58.822 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:00:58.822 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:00:58.822 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.823 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:58.823 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:00:58.823 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.823 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:00:58.824 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.824 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.824 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.824 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.824 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.825 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:00:58.825 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.825 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.825 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.826 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:00:58.826 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.826 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.826 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.826 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.827 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.827 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:00:58.827 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.827 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.828 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.828 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:00:58.828 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.828 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.828 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.829 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.829 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.829 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:00:58.829 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.829 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.830 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.830 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:00:58.830 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.830 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.831 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.831 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.831 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.831 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:00:58.831 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.832 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.832 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.832 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:00:58.832 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.832 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.833 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.833 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.833 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.833 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:00:58.834 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.834 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.834 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.834 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:00:58.834 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.835 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.835 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.835 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.835 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.836 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:00:58.836 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.836 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.836 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.836 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:00:58.837 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.837 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.837 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.837 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.838 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.838 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:00:58.838 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.838 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.839 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:00:58.839 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:00:58.839 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:00:58.839 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:00:58.840 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:00:58.840 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:00:58.840 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:00:58.840 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:00:58.840 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:00:58.841 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.841 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:00:58.841 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:00:58.841 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:00:58.841 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:00:58.842 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:00:58.842 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:00:58.842 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/3502880248": "2021-11-22T22:00:52.736855+0000", 2021-11-21T22:00:58.842 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/3502880248": "2021-11-22T22:00:52.736855+0000", 2021-11-21T22:00:58.843 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.119:0/1262068513": "2021-11-21T23:00:42.232370+0000" 2021-11-21T22:00:58.843 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.843 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:00:58.843 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:00:58.843 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:00:58.844 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:00:58.844 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:00:58.844 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:00:58.844 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:00:58.845 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:00:58.845 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:00:58.845 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:00:58.845 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:00:58.845 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:00:58.846 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:00:58.846 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:00:58.846 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:00:58.846 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:00:58.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75780536a0 msgr2=0x7f7578055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75780536a0 0x7f7578055b60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f757c019000 tx=0x7f757c008040).stop 2021-11-21T22:00:58.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7594159c50 msgr2=0x7f759415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7594159c50 0x7f759415a0d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f758400b000 tx=0x7f7584001e00).stop 2021-11-21T22:00:58.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 shutdown_connections 2021-11-21T22:00:58.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75941566f0 0x7f7594156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75780536a0 0x7f7578055b60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7594159c50 0x7f759415a0d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 --2- 172.21.2.116:0/4016346080 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7574003800 0x7f75940b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 >> 172.21.2.116:0/4016346080 conn(0x7f759405bd20 msgr2=0x7f75941486f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:58.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 shutdown_connections 2021-11-21T22:00:58.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.642+0000 7f759957c700 1 -- 172.21.2.116:0/4016346080 wait complete. 2021-11-21T22:00:58.851 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:58.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.906+0000 7f35e4567700 1 Processor -- start 2021-11-21T22:00:58.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.906+0000 7f35e4567700 1 -- start start 2021-11-21T22:00:58.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f35dc0a6aa0 0x7f35dc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35dc062410 0x7f35dc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc0a6e80 con 0x7f35dc0a86b0 2021-11-21T22:00:58.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc0a8bc0 con 0x7f35dc062410 2021-11-21T22:00:58.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc14a7b0 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35dc062410 0x7f35dc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35dc062410 0x7f35dc1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49704/0 (socket says 172.21.2.116:49704) 2021-11-21T22:00:58.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 -- 172.21.2.116:0/2016553163 learned_addr learned my addr 172.21.2.116:0/2016553163 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:58.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 -- 172.21.2.116:0/2016553163 >> v2:172.21.2.117:3300/0 conn(0x7f35dc0a6aa0 msgr2=0x7f35dc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:58.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 --2- 172.21.2.116:0/2016553163 >> v2:172.21.2.117:3300/0 conn(0x7f35dc0a6aa0 0x7f35dc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 -- 172.21.2.116:0/2016553163 >> v1:172.21.2.116:6789/0 conn(0x7f35dc0a86b0 legacy=0x7f35dc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 -- 172.21.2.116:0/2016553163 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc14aa10 con 0x7f35dc062410 2021-11-21T22:00:58.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e2b04700 1 --2- 172.21.2.116:0/2016553163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc062410 0x7f35dc1443f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f35d40032f0 tx=0x7f35d4009290).ready entity=mon.0 client_cookie=7761c45a09be92b7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1301700 1 -- 172.21.2.116:0/2016553163 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d400b870 con 0x7f35dc062410 2021-11-21T22:00:58.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1301700 1 -- 172.21.2.116:0/2016553163 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d400b9d0 con 0x7f35dc062410 2021-11-21T22:00:58.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1301700 1 -- 172.21.2.116:0/2016553163 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d400bb40 con 0x7f35dc062410 2021-11-21T22:00:58.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- 172.21.2.116:0/2016553163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc062410 msgr2=0x7f35dc1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:58.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- 172.21.2.116:0/2016553163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc062410 0x7f35dc1443f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f35d40032f0 tx=0x7f35d4009290).stop 2021-11-21T22:00:58.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- 172.21.2.116:0/2016553163 shutdown_connections 2021-11-21T22:00:58.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- 172.21.2.116:0/2016553163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc062410 0x7f35dc1443f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:58.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- 172.21.2.116:0/2016553163 >> v2:172.21.2.117:3300/0 conn(0x7f35dc0a6aa0 0x7f35dc14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- 172.21.2.116:0/2016553163 >> 172.21.2.116:0/2016553163 conn(0x7f35dc05bd20 msgr2=0x7f35dc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:58.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- 172.21.2.116:0/2016553163 shutdown_connections 2021-11-21T22:00:58.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- 172.21.2.116:0/2016553163 wait complete. 2021-11-21T22:00:58.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 Processor -- start 2021-11-21T22:00:58.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- start start 2021-11-21T22:00:58.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc062410 0x7f35dc0ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc0a6aa0 0x7f35dc145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc0a86b0 0x7f35dc145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0aa420 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc063850 con 0x7f35dc062410 2021-11-21T22:00:58.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc14d3f0 con 0x7f35dc0a86b0 2021-11-21T22:00:58.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc0a6aa0 0x7f35dc145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc0a6aa0 0x7f35dc145a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49714/0 (socket says 172.21.2.116:49714) 2021-11-21T22:00:58.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 -- 172.21.2.116:0/253292529 learned_addr learned my addr 172.21.2.116:0/253292529 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:58.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 -- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc0a86b0 msgr2=0x7f35dc145f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:58.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc0a86b0 0x7f35dc145f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 -- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc062410 msgr2=0x7f35dc0ab7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:00:58.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc062410 0x7f35dc0ab7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:58.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 -- 172.21.2.116:0/253292529 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4009530 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e1b02700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc0a6aa0 0x7f35dc145a70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f35dc066f30 tx=0x7f35d8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35cb7fe700 1 -- 172.21.2.116:0/253292529 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d800ce20 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35cb7fe700 1 -- 172.21.2.116:0/253292529 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d8013450 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35cb7fe700 1 -- 172.21.2.116:0/253292529 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d80137b0 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc146f60 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.910+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc147190 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.914+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc0573d0 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.914+0000 7f35cb7fe700 1 -- 172.21.2.116:0/253292529 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f35d8010070 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.914+0000 7f35cb7fe700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053610 0x7f35bc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:58.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.914+0000 7f35cb7fe700 1 -- 172.21.2.116:0/253292529 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f35d806aef0 con 0x7f35dc0a6aa0 2021-11-21T22:00:58.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.914+0000 7f35e2303700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053610 0x7f35bc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:58.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.914+0000 7f35e2303700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053610 0x7f35bc055ad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f35cc002f70 tx=0x7f35cc006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:58.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:58.914+0000 7f35cb7fe700 1 -- 172.21.2.116:0/253292529 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35d80421f0 con 0x7f35dc0a6aa0 2021-11-21T22:00:59.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.022+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35dc05db70 con 0x7f35dc0a6aa0 2021-11-21T22:00:59.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.022+0000 7f35cb7fe700 1 -- 172.21.2.116:0/253292529 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f35d8018070 con 0x7f35dc0a6aa0 2021-11-21T22:00:59.028 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:59.028 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6570,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2793092574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2793092574},{"type":"v1","addr":"172.21.2.116:6835","nonce":2793092574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[]} 2021-11-21T22:00:59.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.022+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053610 msgr2=0x7f35bc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.022+0000 7f35e4567700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053610 0x7f35bc055ad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f35cc002f70 tx=0x7f35cc006a00).stop 2021-11-21T22:00:59.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.022+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc0a6aa0 msgr2=0x7f35dc145a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.022+0000 7f35e4567700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc0a6aa0 0x7f35dc145a70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f35dc066f30 tx=0x7f35d8007c20).stop 2021-11-21T22:00:59.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 shutdown_connections 2021-11-21T22:00:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35dc0a86b0 0x7f35dc145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f35bc053610 0x7f35bc055ad0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35dc0a6aa0 0x7f35dc145a70 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 --2- 172.21.2.116:0/253292529 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35dc062410 0x7f35dc0ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:59.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 >> 172.21.2.116:0/253292529 conn(0x7f35dc05bd20 msgr2=0x7f35dc149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:59.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 shutdown_connections 2021-11-21T22:00:59.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.026+0000 7f35e4567700 1 -- 172.21.2.116:0/253292529 wait complete. 2021-11-21T22:00:59.032 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 35 2021-11-21T22:00:59.035 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:00:59.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 Processor -- start 2021-11-21T22:00:59.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- start start 2021-11-21T22:00:59.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2800a6aa0 0x7ff28014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a86b0 0x7ff2801470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2800a6e80 con 0x7ff2801475d0 2021-11-21T22:00:59.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2800a8bc0 con 0x7ff2800a6aa0 2021-11-21T22:00:59.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28014a7b0 con 0x7ff2800a86b0 2021-11-21T22:00:59.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2800a6aa0 0x7ff28014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2800a6aa0 0x7ff28014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49718/0 (socket says 172.21.2.116:49718) 2021-11-21T22:00:59.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 -- 172.21.2.116:0/1212453932 learned_addr learned my addr 172.21.2.116:0/1212453932 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:59.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27f7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff2801475d0 0x7ff2801443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33444/0 (socket says 172.21.2.116:33444) 2021-11-21T22:00:59.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27dffb700 1 -- 172.21.2.116:0/1212453932 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220019131 0 0) 0x7ff2800a6e80 con 0x7ff2801475d0 2021-11-21T22:00:59.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 -- 172.21.2.116:0/1212453932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a86b0 msgr2=0x7ff2801470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:59.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 --2- 172.21.2.116:0/1212453932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a86b0 0x7ff2801470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:59.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 -- 172.21.2.116:0/1212453932 >> v1:172.21.2.116:6789/0 conn(0x7ff2801475d0 legacy=0x7ff2801443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 -- 172.21.2.116:0/1212453932 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28014aa10 con 0x7ff2800a6aa0 2021-11-21T22:00:59.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27effd700 1 --2- 172.21.2.116:0/1212453932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a6aa0 0x7ff28014b150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7ff268002f70 tx=0x7ff268006eb0).ready entity=mon.0 client_cookie=630bbd73a3588bba server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:59.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27dffb700 1 -- 172.21.2.116:0/1212453932 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26800d5c0 con 0x7ff2800a6aa0 2021-11-21T22:00:59.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27dffb700 1 -- 172.21.2.116:0/1212453932 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268000e90 con 0x7ff2800a6aa0 2021-11-21T22:00:59.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27dffb700 1 -- 172.21.2.116:0/1212453932 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26800d950 con 0x7ff2800a6aa0 2021-11-21T22:00:59.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- 172.21.2.116:0/1212453932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a6aa0 msgr2=0x7ff28014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- 172.21.2.116:0/1212453932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a6aa0 0x7ff28014b150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7ff268002f70 tx=0x7ff268006eb0).stop 2021-11-21T22:00:59.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- 172.21.2.116:0/1212453932 shutdown_connections 2021-11-21T22:00:59.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- 172.21.2.116:0/1212453932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a86b0 0x7ff2801470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:59.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- 172.21.2.116:0/1212453932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a6aa0 0x7ff28014b150 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- 172.21.2.116:0/1212453932 >> 172.21.2.116:0/1212453932 conn(0x7ff28005bd20 msgr2=0x7ff28005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:59.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- 172.21.2.116:0/1212453932 shutdown_connections 2021-11-21T22:00:59.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- 172.21.2.116:0/1212453932 wait complete. 2021-11-21T22:00:59.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 Processor -- start 2021-11-21T22:00:59.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- start start 2021-11-21T22:00:59.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a6aa0 0x7ff28013dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a86b0 0x7ff28014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2801475d0 0x7ff2801596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28013d8e0 con 0x7ff2800a86b0 2021-11-21T22:00:59.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28013d640 con 0x7ff2801475d0 2021-11-21T22:00:59.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff28546c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28013cac0 con 0x7ff2800a6aa0 2021-11-21T22:00:59.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.082+0000 7ff27e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a86b0 0x7ff28014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a86b0 0x7ff28014da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49726/0 (socket says 172.21.2.116:49726) 2021-11-21T22:00:59.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 -- 172.21.2.116:0/290011246 learned_addr learned my addr 172.21.2.116:0/290011246 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:59.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27effd700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a6aa0 0x7ff28013dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 -- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a6aa0 msgr2=0x7ff28013dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a6aa0 0x7ff28013dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 -- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2801475d0 msgr2=0x7ff2801596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2801475d0 0x7ff2801596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:59.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 -- 172.21.2.116:0/290011246 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26800b1e0 con 0x7ff2800a86b0 2021-11-21T22:00:59.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27effd700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a6aa0 0x7ff28013dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:59.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27e7fc700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a86b0 0x7ff28014da50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7ff270002f70 tx=0x7ff270006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:59.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff277fff700 1 -- 172.21.2.116:0/290011246 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff270013460 con 0x7ff2800a86b0 2021-11-21T22:00:59.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff277fff700 1 -- 172.21.2.116:0/290011246 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2700135c0 con 0x7ff2800a86b0 2021-11-21T22:00:59.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28013cd60 con 0x7ff2800a86b0 2021-11-21T22:00:59.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28014c280 con 0x7ff2800a86b0 2021-11-21T22:00:59.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff277fff700 1 -- 172.21.2.116:0/290011246 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff270013920 con 0x7ff2800a86b0 2021-11-21T22:00:59.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28013cef0 con 0x7ff2800a86b0 2021-11-21T22:00:59.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff277fff700 1 -- 172.21.2.116:0/290011246 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff27001a060 con 0x7ff2800a86b0 2021-11-21T22:00:59.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff277fff700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff264053640 0x7ff264055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff277fff700 1 -- 172.21.2.116:0/290011246 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7ff27006b1d0 con 0x7ff2800a86b0 2021-11-21T22:00:59.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27effd700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff264053640 0x7ff264055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.086+0000 7ff27effd700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff264053640 0x7ff264055b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff268017000 tx=0x7ff268015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:59.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.090+0000 7ff277fff700 1 -- 172.21.2.116:0/290011246 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff2700424d0 con 0x7ff2800a86b0 2021-11-21T22:00:59.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2800573d0 con 0x7ff2800a86b0 2021-11-21T22:00:59.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff277fff700 1 -- 172.21.2.116:0/290011246 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+1559 (secure 0 0 0) 0x7ff270045af0 con 0x7ff2800a86b0 2021-11-21T22:00:59.202 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:00:59.202 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1535378031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1535378031},{"type":"v1","addr":"172.21.2.117:6835","nonce":1535378031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/370661376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":370661376},{"type":"v1","addr":"172.21.2.116:6833","nonce":370661376}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6570,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2793092574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2793092574},{"type":"v1","addr":"172.21.2.116:6835","nonce":2793092574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[]} 2021-11-21T22:00:59.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff264053640 msgr2=0x7ff264055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff264053640 0x7ff264055b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff268017000 tx=0x7ff268015000).stop 2021-11-21T22:00:59.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a86b0 msgr2=0x7ff28014da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a86b0 0x7ff28014da50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7ff270002f70 tx=0x7ff270006e90).stop 2021-11-21T22:00:59.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 shutdown_connections 2021-11-21T22:00:59.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2801475d0 0x7ff2801596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:00:59.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff264053640 0x7ff264055b00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2800a86b0 0x7ff28014da50 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 --2- 172.21.2.116:0/290011246 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2800a6aa0 0x7ff28013dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 >> 172.21.2.116:0/290011246 conn(0x7ff28005bd20 msgr2=0x7ff280143480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:59.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 shutdown_connections 2021-11-21T22:00:59.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.198+0000 7ff28546c700 1 -- 172.21.2.116:0/290011246 wait complete. 2021-11-21T22:00:59.207 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 35 2021-11-21T22:00:59.209 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T22:00:59.209 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T22:00:59.210 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:00:59.210 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:00:59.210 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T22:00:59.211 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T22:00:59.211 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:00:59.211 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:00:59.211 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T22:00:59.212 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T22:00:59.212 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:00:59.212 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:00:59.215 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:00:59.215 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T22:00:59.217 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:00:59.217 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T22:00:59.218 INFO:tasks.ceph.mds.a:Started 2021-11-21T22:00:59.219 DEBUG:teuthology.parallel:result is None 2021-11-21T22:00:59.220 INFO:tasks.ceph.mds.c:Started 2021-11-21T22:00:59.220 DEBUG:teuthology.parallel:result is None 2021-11-21T22:00:59.224 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:00:59.224 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T22:00:59.227 INFO:tasks.ceph.mds.b:Started 2021-11-21T22:00:59.227 DEBUG:teuthology.parallel:result is None 2021-11-21T22:00:59.228 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T22:00:59.250 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:00:59.251 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:00:59.260 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T22:00:59.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 Processor -- start 2021-11-21T22:00:59.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- start start 2021-11-21T22:00:59.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1c941426e0 0x7f1c94142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c940ad470 0x7f1c940ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940afae0 con 0x7f1c94062e60 2021-11-21T22:00:59.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c940b0f60 con 0x7f1c941426e0 2021-11-21T22:00:59.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94063470 con 0x7f1c940ad470 2021-11-21T22:00:59.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1c941426e0 0x7f1c94142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1c941426e0 0x7f1c94142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49754/0 (socket says 172.21.2.116:49754) 2021-11-21T22:00:59.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 -- 172.21.2.116:0/499789978 learned_addr learned my addr 172.21.2.116:0/499789978 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:59.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c937fe700 1 --2- 172.21.2.116:0/499789978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c940ad470 0x7f1c940ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 -- 172.21.2.116:0/499789978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c940ad470 msgr2=0x7f1c940ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:00:59.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 --2- 172.21.2.116:0/499789978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c940ad470 0x7f1c940ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 -- 172.21.2.116:0/499789978 >> v1:172.21.2.116:6789/0 conn(0x7f1c94062e60 legacy=0x7f1c940af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 -- 172.21.2.116:0/499789978 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94143050 con 0x7f1c941426e0 2021-11-21T22:00:59.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c937fe700 1 --2- 172.21.2.116:0/499789978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c940ad470 0x7f1c940ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:00:59.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c92ffd700 1 --2- 172.21.2.116:0/499789978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c941426e0 0x7f1c94142ac0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f1c84002f70 tx=0x7f1c84006e40).ready entity=mon.0 client_cookie=bfe48320672a5aeb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:59.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c91ffb700 1 -- 172.21.2.116:0/499789978 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8400bb30 con 0x7f1c941426e0 2021-11-21T22:00:59.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c91ffb700 1 -- 172.21.2.116:0/499789978 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c84000e90 con 0x7f1c941426e0 2021-11-21T22:00:59.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c91ffb700 1 -- 172.21.2.116:0/499789978 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8400be80 con 0x7f1c941426e0 2021-11-21T22:00:59.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/499789978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c941426e0 msgr2=0x7f1c94142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/499789978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c941426e0 0x7f1c94142ac0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f1c84002f70 tx=0x7f1c84006e40).stop 2021-11-21T22:00:59.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/499789978 shutdown_connections 2021-11-21T22:00:59.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/499789978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c940ad470 0x7f1c940ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/499789978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c941426e0 0x7f1c94142ac0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/499789978 >> 172.21.2.116:0/499789978 conn(0x7f1c940b5730 msgr2=0x7f1c940b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:00:59.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/499789978 shutdown_connections 2021-11-21T22:00:59.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/499789978 wait complete. 2021-11-21T22:00:59.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 Processor -- start 2021-11-21T22:00:59.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 -- start start 2021-11-21T22:00:59.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c940ad470 0x7f1c94141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c941426e0 0x7f1c9414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c9413fcb0 0x7f1c9414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94066d40 con 0x7f1c940ad470 2021-11-21T22:00:59.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9414e5a0 con 0x7f1c941426e0 2021-11-21T22:00:59.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940673d0 con 0x7f1c9413fcb0 2021-11-21T22:00:59.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c940ad470 0x7f1c94141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c940ad470 0x7f1c94141e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49760/0 (socket says 172.21.2.116:49760) 2021-11-21T22:00:59.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 -- 172.21.2.116:0/1670879842 learned_addr learned my addr 172.21.2.116:0/1670879842 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:00:59.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c937fe700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c9413fcb0 0x7f1c9414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c927fc700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c941426e0 0x7f1c9414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 -- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c9413fcb0 msgr2=0x7f1c9414e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c9413fcb0 0x7f1c9414e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 -- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c941426e0 msgr2=0x7f1c9414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:00:59.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c941426e0 0x7f1c9414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:00:59.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 -- 172.21.2.116:0/1670879842 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8400d4d0 con 0x7f1c940ad470 2021-11-21T22:00:59.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c92ffd700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c940ad470 0x7f1c94141e70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f1c94066130 tx=0x7f1c84006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:59.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c8bfff700 1 -- 172.21.2.116:0/1670879842 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c84020070 con 0x7f1c940ad470 2021-11-21T22:00:59.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c8bfff700 1 -- 172.21.2.116:0/1670879842 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c84012e90 con 0x7f1c940ad470 2021-11-21T22:00:59.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c8bfff700 1 -- 172.21.2.116:0/1670879842 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c84025650 con 0x7f1c940ad470 2021-11-21T22:00:59.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c94067600 con 0x7f1c940ad470 2021-11-21T22:00:59.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9413e250 con 0x7f1c940ad470 2021-11-21T22:00:59.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c940b1e00 con 0x7f1c940ad470 2021-11-21T22:00:59.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c8bfff700 1 -- 172.21.2.116:0/1670879842 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1c840257b0 con 0x7f1c940ad470 2021-11-21T22:00:59.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c8bfff700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c78053580 0x7f1c78055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:00:59.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c8bfff700 1 -- 172.21.2.116:0/1670879842 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f1c84070e90 con 0x7f1c940ad470 2021-11-21T22:00:59.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c927fc700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c78053580 0x7f1c78055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:00:59.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.282+0000 7f1c927fc700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c78053580 0x7f1c78055a40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f1c7c00b000 tx=0x7f1c7c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:00:59.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.286+0000 7f1c8bfff700 1 -- 172.21.2.116:0/1670879842 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c84048190 con 0x7f1c940ad470 2021-11-21T22:00:59.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:00:59.378+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f1c940573d0 con 0x7f1c940ad470 2021-11-21T22:01:00.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.274+0000 7f1c8bfff700 1 -- 172.21.2.116:0/1670879842 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1c8402a030 con 0x7f1c940ad470 2021-11-21T22:01:00.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c78053580 msgr2=0x7f1c78055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c78053580 0x7f1c78055a40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f1c7c00b000 tx=0x7f1c7c009040).stop 2021-11-21T22:01:00.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c940ad470 msgr2=0x7f1c94141e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c940ad470 0x7f1c94141e70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f1c94066130 tx=0x7f1c84006e40).stop 2021-11-21T22:01:00.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 shutdown_connections 2021-11-21T22:01:00.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1c9413fcb0 0x7f1c9414e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1c941426e0 0x7f1c9414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1c78053580 0x7f1c78055a40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 --2- 172.21.2.116:0/1670879842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1c940ad470 0x7f1c94141e70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 >> 172.21.2.116:0/1670879842 conn(0x7f1c940b5730 msgr2=0x7f1c9405bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.278+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 shutdown_connections 2021-11-21T22:01:00.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.282+0000 7f1c9949d700 1 -- 172.21.2.116:0/1670879842 wait complete. 2021-11-21T22:01:00.287 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T22:01:00.311 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T22:01:00.312 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T22:01:00.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 Processor -- start 2021-11-21T22:01:00.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 -- start start 2021-11-21T22:01:00.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0e600a6aa0 0x7f0e6014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e600a86b0 0x7f0e601470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e601475d0 0x7f0e601443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e600a8bc0 con 0x7f0e601475d0 2021-11-21T22:01:00.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e600a6e80 con 0x7f0e600a6aa0 2021-11-21T22:01:00.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e654ef700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6014a7b0 con 0x7f0e600a86b0 2021-11-21T22:01:00.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e601475d0 0x7f0e601443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e601475d0 0x7f0e601443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49772/0 (socket says 172.21.2.116:49772) 2021-11-21T22:01:00.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 -- 172.21.2.116:0/2561164746 learned_addr learned my addr 172.21.2.116:0/2561164746 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:00.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 -- 172.21.2.116:0/2561164746 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e600a86b0 msgr2=0x7f0e601470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:00.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 --2- 172.21.2.116:0/2561164746 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e600a86b0 0x7f0e601470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 -- 172.21.2.116:0/2561164746 >> v2:172.21.2.117:3300/0 conn(0x7f0e600a6aa0 msgr2=0x7f0e6014b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:00.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 --2- 172.21.2.116:0/2561164746 >> v2:172.21.2.117:3300/0 conn(0x7f0e600a6aa0 0x7f0e6014b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 -- 172.21.2.116:0/2561164746 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6014aa10 con 0x7f0e601475d0 2021-11-21T22:01:00.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5f7fe700 1 --2- 172.21.2.116:0/2561164746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e601475d0 0x7f0e601443f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f0e580032f0 tx=0x7f0e58009290).ready entity=mon.0 client_cookie=5136b0eb7659007a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5dffb700 1 -- 172.21.2.116:0/2561164746 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e5800b870 con 0x7f0e601475d0 2021-11-21T22:01:00.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5dffb700 1 -- 172.21.2.116:0/2561164746 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5800b9d0 con 0x7f0e601475d0 2021-11-21T22:01:00.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.402+0000 7f0e5dffb700 1 -- 172.21.2.116:0/2561164746 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e5800bb80 con 0x7f0e601475d0 2021-11-21T22:01:00.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/2561164746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e601475d0 msgr2=0x7f0e601443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 --2- 172.21.2.116:0/2561164746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e601475d0 0x7f0e601443f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f0e580032f0 tx=0x7f0e58009290).stop 2021-11-21T22:01:00.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/2561164746 shutdown_connections 2021-11-21T22:01:00.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 --2- 172.21.2.116:0/2561164746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e601475d0 0x7f0e601443f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 --2- 172.21.2.116:0/2561164746 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e600a86b0 0x7f0e601470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 --2- 172.21.2.116:0/2561164746 >> v2:172.21.2.117:3300/0 conn(0x7f0e600a6aa0 0x7f0e6014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/2561164746 >> 172.21.2.116:0/2561164746 conn(0x7f0e6005bd20 msgr2=0x7f0e6005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/2561164746 shutdown_connections 2021-11-21T22:01:00.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/2561164746 wait complete. 2021-11-21T22:01:00.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 Processor -- start 2021-11-21T22:01:00.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- start start 2021-11-21T22:01:00.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e600a6aa0 0x7f0e601460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e600a86b0 0x7f0e6014bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e601475d0 0x7f0e60141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60145a40 con 0x7f0e600a86b0 2021-11-21T22:01:00.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60146fc0 con 0x7f0e600a6aa0 2021-11-21T22:01:00.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60064760 con 0x7f0e601475d0 2021-11-21T22:01:00.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e600a86b0 0x7f0e6014bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e600a86b0 0x7f0e6014bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49782/0 (socket says 172.21.2.116:49782) 2021-11-21T22:01:00.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 -- 172.21.2.116:0/1207440958 learned_addr learned my addr 172.21.2.116:0/1207440958 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:00.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 -- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e601475d0 msgr2=0x7f0e60141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e601475d0 0x7f0e60141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 -- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e600a6aa0 msgr2=0x7f0e601460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e600a6aa0 0x7f0e601460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 -- 172.21.2.116:0/1207440958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58009530 con 0x7f0e600a86b0 2021-11-21T22:01:00.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e5e7fc700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e600a86b0 0x7f0e6014bc00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f0e50002fb0 tx=0x7f0e50006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e4ffff700 1 -- 172.21.2.116:0/1207440958 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e50009e00 con 0x7f0e600a86b0 2021-11-21T22:01:00.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e4ffff700 1 -- 172.21.2.116:0/1207440958 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50013460 con 0x7f0e600a86b0 2021-11-21T22:01:00.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e4ffff700 1 -- 172.21.2.116:0/1207440958 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e500137c0 con 0x7f0e600a86b0 2021-11-21T22:01:00.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e60063b90 con 0x7f0e600a86b0 2021-11-21T22:01:00.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e600638f0 con 0x7f0e600a86b0 2021-11-21T22:01:00.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e600573d0 con 0x7f0e600a86b0 2021-11-21T22:01:00.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e4ffff700 1 -- 172.21.2.116:0/1207440958 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0e5000f070 con 0x7f0e600a86b0 2021-11-21T22:01:00.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e4ffff700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e3c053720 0x7f0e3c055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.406+0000 7f0e4ffff700 1 -- 172.21.2.116:0/1207440958 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0e5006afb0 con 0x7f0e600a86b0 2021-11-21T22:01:00.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.410+0000 7f0e5effd700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e3c053720 0x7f0e3c055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.410+0000 7f0e5effd700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e3c053720 0x7f0e3c055be0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f0e54002fb0 tx=0x7f0e540069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.410+0000 7f0e4ffff700 1 -- 172.21.2.116:0/1207440958 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e500423e0 con 0x7f0e600a86b0 2021-11-21T22:01:00.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.518+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0e6005db70 con 0x7f0e600a86b0 2021-11-21T22:01:00.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.518+0000 7f0e4ffff700 1 -- 172.21.2.116:0/1207440958 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f0e50045a00 con 0x7f0e600a86b0 2021-11-21T22:01:00.525 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:00.525 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:00.525 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T22:01:00.525 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.526 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T22:01:00.526 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T22:01:00.526 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.526 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:00.527 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.527 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.527 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.527 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.527 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.528 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.528 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T22:01:00.528 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T22:01:00.528 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.529 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:00.529 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.529 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.529 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.529 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.530 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.530 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.530 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T22:01:00.530 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T22:01:00.531 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.531 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:00.531 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.531 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.532 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.532 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.532 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.532 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.532 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T22:01:00.533 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T22:01:00.533 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.533 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.533 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.534 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.534 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.534 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.534 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.534 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T22:01:00.535 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T22:01:00.535 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.535 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.535 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.536 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.536 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.536 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.536 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.537 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T22:01:00.537 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T22:01:00.537 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.537 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.537 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.538 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.538 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.538 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.538 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.539 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T22:01:00.539 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T22:01:00.539 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.539 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.539 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.540 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.540 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.540 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.540 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.541 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T22:01:00.541 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T22:01:00.541 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.542 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.542 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.542 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.542 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.543 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.543 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.543 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T22:01:00.543 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T22:01:00.543 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.544 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.544 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.544 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.544 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.545 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.545 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T22:01:00.545 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T22:01:00.546 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.546 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.546 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.546 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.547 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.547 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.547 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.547 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T22:01:00.547 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T22:01:00.548 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.548 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.548 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.548 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.549 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.549 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.549 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.549 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T22:01:00.549 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T22:01:00.550 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.550 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:00.550 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:00.550 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:00.551 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.551 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.551 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.551 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T22:01:00.552 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T22:01:00.552 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.552 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:00.552 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:00.553 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:00.553 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.553 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.553 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.553 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T22:01:00.554 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T22:01:00.554 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.554 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:00.554 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:00.555 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:00.555 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.555 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.555 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.555 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.556 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T22:01:00.556 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T22:01:00.556 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.557 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:00.557 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:00.557 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.557 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.558 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:00.558 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:00.558 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T22:01:00.558 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T22:01:00.559 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:00.559 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:00.559 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:00.559 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:00.559 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.560 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:00.560 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:00.560 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:00.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e3c053720 msgr2=0x7f0e3c055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e3c053720 0x7f0e3c055be0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f0e54002fb0 tx=0x7f0e540069b0).stop 2021-11-21T22:01:00.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e600a86b0 msgr2=0x7f0e6014bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e600a86b0 0x7f0e6014bc00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f0e50002fb0 tx=0x7f0e50006de0).stop 2021-11-21T22:01:00.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 shutdown_connections 2021-11-21T22:01:00.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e601475d0 0x7f0e60141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e3c053720 0x7f0e3c055be0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e600a86b0 0x7f0e6014bc00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 --2- 172.21.2.116:0/1207440958 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e600a6aa0 0x7f0e601460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 >> 172.21.2.116:0/1207440958 conn(0x7f0e6005bd20 msgr2=0x7f0e601483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 shutdown_connections 2021-11-21T22:01:00.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.522+0000 7f0e654ef700 1 -- 172.21.2.116:0/1207440958 wait complete. 2021-11-21T22:01:00.565 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:00.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.610+0000 7f2d928c9700 1 Processor -- start 2021-11-21T22:01:00.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.610+0000 7f2d928c9700 1 -- start start 2021-11-21T22:01:00.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2d8c0a6aa0 0x7f2d8c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c0a86b0 0x7f2d8c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2d8c1475d0 0x7f2d8c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0a8bc0 con 0x7f2d8c0a6aa0 2021-11-21T22:01:00.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0a6e80 con 0x7f2d8c1475d0 2021-11-21T22:01:00.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c14a7b0 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2d8c0a6aa0 0x7f2d8c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2d8c0a6aa0 0x7f2d8c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49786/0 (socket says 172.21.2.116:49786) 2021-11-21T22:01:00.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 -- 172.21.2.116:0/1473854638 learned_addr learned my addr 172.21.2.116:0/1473854638 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:00.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 -- 172.21.2.116:0/1473854638 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c0a86b0 msgr2=0x7f2d8c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:00.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 --2- 172.21.2.116:0/1473854638 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c0a86b0 0x7f2d8c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 -- 172.21.2.116:0/1473854638 >> v2:172.21.2.117:3300/0 conn(0x7f2d8c1475d0 msgr2=0x7f2d8c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:00.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 --2- 172.21.2.116:0/1473854638 >> v2:172.21.2.117:3300/0 conn(0x7f2d8c1475d0 0x7f2d8c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 -- 172.21.2.116:0/1473854638 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c14aa10 con 0x7f2d8c0a6aa0 2021-11-21T22:01:00.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8bfff700 1 --2- 172.21.2.116:0/1473854638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a6aa0 0x7f2d8c14b150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f2d74002f70 tx=0x7f2d74006eb0).ready entity=mon.0 client_cookie=cde32444bf3aecfa server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8affd700 1 -- 172.21.2.116:0/1473854638 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7400d570 con 0x7f2d8c0a6aa0 2021-11-21T22:01:00.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8affd700 1 -- 172.21.2.116:0/1473854638 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74000e90 con 0x7f2d8c0a6aa0 2021-11-21T22:01:00.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8affd700 1 -- 172.21.2.116:0/1473854638 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7400d900 con 0x7f2d8c0a6aa0 2021-11-21T22:01:00.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- 172.21.2.116:0/1473854638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a6aa0 msgr2=0x7f2d8c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- 172.21.2.116:0/1473854638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a6aa0 0x7f2d8c14b150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f2d74002f70 tx=0x7f2d74006eb0).stop 2021-11-21T22:01:00.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- 172.21.2.116:0/1473854638 shutdown_connections 2021-11-21T22:01:00.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- 172.21.2.116:0/1473854638 >> v2:172.21.2.117:3300/0 conn(0x7f2d8c1475d0 0x7f2d8c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- 172.21.2.116:0/1473854638 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c0a86b0 0x7f2d8c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- 172.21.2.116:0/1473854638 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a6aa0 0x7f2d8c14b150 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- 172.21.2.116:0/1473854638 >> 172.21.2.116:0/1473854638 conn(0x7f2d8c05bd20 msgr2=0x7f2d8c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- 172.21.2.116:0/1473854638 shutdown_connections 2021-11-21T22:01:00.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- 172.21.2.116:0/1473854638 wait complete. 2021-11-21T22:01:00.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 Processor -- start 2021-11-21T22:01:00.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- start start 2021-11-21T22:01:00.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d8c0a6aa0 0x7f2d8c14ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a86b0 0x7f2d8c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c1475d0 0x7f2d8c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b3b30 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c065640 con 0x7f2d8c0a6aa0 2021-11-21T22:01:00.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d928c9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b3890 con 0x7f2d8c1475d0 2021-11-21T22:01:00.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a86b0 0x7f2d8c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a86b0 0x7f2d8c1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49796/0 (socket says 172.21.2.116:49796) 2021-11-21T22:01:00.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 -- 172.21.2.116:0/2894417322 learned_addr learned my addr 172.21.2.116:0/2894417322 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:00.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 -- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c1475d0 msgr2=0x7f2d8c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c1475d0 0x7f2d8c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 -- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d8c0a6aa0 msgr2=0x7f2d8c14ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d8c0a6aa0 0x7f2d8c14ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 -- 172.21.2.116:0/2894417322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7400b1e0 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d8b7fe700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a86b0 0x7f2d8c1597f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f2d7c002f70 tx=0x7f2d7c006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.614+0000 7f2d897fa700 1 -- 172.21.2.116:0/2894417322 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7c013460 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d897fa700 1 -- 172.21.2.116:0/2894417322 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c0135c0 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c0658a0 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d897fa700 1 -- 172.21.2.116:0/2894417322 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7c013920 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c153dd0 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c0b3540 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d897fa700 1 -- 172.21.2.116:0/2894417322 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2d7c01a060 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d897fa700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d700536a0 0x7f2d70055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d897fa700 1 -- 172.21.2.116:0/2894417322 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2d7c06bcc0 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d8bfff700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d700536a0 0x7f2d70055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d8bfff700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d700536a0 0x7f2d70055b60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f2d74002f70 tx=0x7f2d74015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.618+0000 7f2d897fa700 1 -- 172.21.2.116:0/2894417322 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d7c042480 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d8c065a30 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d897fa700 1 -- 172.21.2.116:0/2894417322 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+13182 (secure 0 0 0) 0x7f2d7c045aa0 con 0x7f2d8c0a86b0 2021-11-21T22:01:00.710 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:00.710 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":84,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:00.268294+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:00.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d700536a0 msgr2=0x7f2d70055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d700536a0 0x7f2d70055b60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f2d74002f70 tx=0x7f2d74015000).stop 2021-11-21T22:01:00.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a86b0 msgr2=0x7f2d8c1597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a86b0 0x7f2d8c1597f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f2d7c002f70 tx=0x7f2d7c006e90).stop 2021-11-21T22:01:00.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 shutdown_connections 2021-11-21T22:01:00.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d8c1475d0 0x7f2d8c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d700536a0 0x7f2d70055b60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d8c0a86b0 0x7f2d8c1597f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 --2- 172.21.2.116:0/2894417322 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d8c0a6aa0 0x7f2d8c14ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 >> 172.21.2.116:0/2894417322 conn(0x7f2d8c05bd20 msgr2=0x7f2d8c1490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 shutdown_connections 2021-11-21T22:01:00.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.706+0000 7f2d928c9700 1 -- 172.21.2.116:0/2894417322 wait complete. 2021-11-21T22:01:00.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T22:01:00.718 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:00.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe37ed9700 1 Processor -- start 2021-11-21T22:01:00.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe37ed9700 1 -- start start 2021-11-21T22:01:00.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe37ed9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbe300abce0 0x7fbe301478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe37ed9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe300ac600 0x7fbe300aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe37ed9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe30147e80 con 0x7fbe30144810 2021-11-21T22:01:00.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe37ed9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe300b1490 con 0x7fbe300abce0 2021-11-21T22:01:00.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe37ed9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300b1fd0 con 0x7fbe300ac600 2021-11-21T22:01:00.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35474700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbe300abce0 0x7fbe301478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35c75700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbe30144810 0x7fbe300b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33524/0 (socket says 172.21.2.116:33524) 2021-11-21T22:01:00.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35c75700 1 -- 172.21.2.116:0/1509927955 learned_addr learned my addr 172.21.2.116:0/1509927955 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:00.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35474700 1 -- 172.21.2.116:0/1509927955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe300ac600 msgr2=0x7fbe300aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:00.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35474700 1 --2- 172.21.2.116:0/1509927955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe300ac600 0x7fbe300aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35474700 1 -- 172.21.2.116:0/1509927955 >> v1:172.21.2.116:6789/0 conn(0x7fbe30144810 legacy=0x7fbe300b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35474700 1 -- 172.21.2.116:0/1509927955 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300af5a0 con 0x7fbe300abce0 2021-11-21T22:01:00.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe35474700 1 --2- 172.21.2.116:0/1509927955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300abce0 0x7fbe301478c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fbe2c002f70 tx=0x7fbe2c006df0).ready entity=mon.0 client_cookie=c5665c5d70dde680 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe34c73700 1 -- 172.21.2.116:0/1509927955 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe2c00d570 con 0x7fbe300abce0 2021-11-21T22:01:00.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe34c73700 1 -- 172.21.2.116:0/1509927955 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c000e90 con 0x7fbe300abce0 2021-11-21T22:01:00.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.766+0000 7fbe34c73700 1 -- 172.21.2.116:0/1509927955 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe2c00d8c0 con 0x7fbe300abce0 2021-11-21T22:01:00.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/1509927955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300abce0 msgr2=0x7fbe301478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/1509927955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300abce0 0x7fbe301478c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fbe2c002f70 tx=0x7fbe2c006df0).stop 2021-11-21T22:01:00.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/1509927955 shutdown_connections 2021-11-21T22:01:00.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/1509927955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe300ac600 0x7fbe300aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/1509927955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300abce0 0x7fbe301478c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/1509927955 >> 172.21.2.116:0/1509927955 conn(0x7fbe3005bd20 msgr2=0x7fbe3005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/1509927955 shutdown_connections 2021-11-21T22:01:00.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/1509927955 wait complete. 2021-11-21T22:01:00.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 Processor -- start 2021-11-21T22:01:00.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- start start 2021-11-21T22:01:00.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe300abce0 0x7fbe30143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300ac600 0x7fbe3015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe30144810 0x7fbe3015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301568c0 con 0x7fbe300ac600 2021-11-21T22:01:00.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301453e0 con 0x7fbe300abce0 2021-11-21T22:01:00.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30147060 con 0x7fbe30144810 2021-11-21T22:01:00.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300ac600 0x7fbe3015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300ac600 0x7fbe3015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49808/0 (socket says 172.21.2.116:49808) 2021-11-21T22:01:00.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 -- 172.21.2.116:0/3044813965 learned_addr learned my addr 172.21.2.116:0/3044813965 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:00.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 -- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe30144810 msgr2=0x7fbe3015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe30144810 0x7fbe3015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 -- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe300abce0 msgr2=0x7fbe30143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe300abce0 0x7fbe30143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 -- 172.21.2.116:0/3044813965 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c00b180 con 0x7fbe300ac600 2021-11-21T22:01:00.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe35474700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300ac600 0x7fbe3015a120 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fbe2c002f70 tx=0x7fbe2c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe22ffd700 1 -- 172.21.2.116:0/3044813965 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe2c00d8f0 con 0x7fbe300ac600 2021-11-21T22:01:00.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe22ffd700 1 -- 172.21.2.116:0/3044813965 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c011ce0 con 0x7fbe300ac600 2021-11-21T22:01:00.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe30144ea0 con 0x7fbe300ac600 2021-11-21T22:01:00.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30147280 con 0x7fbe300ac600 2021-11-21T22:01:00.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe22ffd700 1 -- 172.21.2.116:0/3044813965 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe2c01e650 con 0x7fbe300ac600 2021-11-21T22:01:00.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3015aca0 con 0x7fbe300ac600 2021-11-21T22:01:00.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe22ffd700 1 -- 172.21.2.116:0/3044813965 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbe2c025030 con 0x7fbe300ac600 2021-11-21T22:01:00.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe22ffd700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe10053610 0x7fbe10055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.770+0000 7fbe22ffd700 1 -- 172.21.2.116:0/3044813965 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe2c06cb20 con 0x7fbe300ac600 2021-11-21T22:01:00.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.774+0000 7fbe35c75700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe10053610 0x7fbe10055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.774+0000 7fbe35c75700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe10053610 0x7fbe10055ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fbe30062500 tx=0x7fbe24007860).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.774+0000 7fbe22ffd700 1 -- 172.21.2.116:0/3044813965 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbe2c043ea0 con 0x7fbe300ac600 2021-11-21T22:01:00.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbe300af060 con 0x7fbe300ac600 2021-11-21T22:01:00.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe22ffd700 1 -- 172.21.2.116:0/3044813965 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbe2c023020 con 0x7fbe300ac600 2021-11-21T22:01:00.861 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:00.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe10053610 msgr2=0x7fbe10055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe10053610 0x7fbe10055ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fbe30062500 tx=0x7fbe24007860).stop 2021-11-21T22:01:00.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300ac600 msgr2=0x7fbe3015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300ac600 0x7fbe3015a120 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fbe2c002f70 tx=0x7fbe2c015000).stop 2021-11-21T22:01:00.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 shutdown_connections 2021-11-21T22:01:00.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe30144810 0x7fbe3015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe10053610 0x7fbe10055ad0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe300ac600 0x7fbe3015a120 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 --2- 172.21.2.116:0/3044813965 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe300abce0 0x7fbe30143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 >> 172.21.2.116:0/3044813965 conn(0x7fbe3005bd20 msgr2=0x7fbe300ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 shutdown_connections 2021-11-21T22:01:00.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.858+0000 7fbe37ed9700 1 -- 172.21.2.116:0/3044813965 wait complete. 2021-11-21T22:01:00.868 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:00.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 Processor -- start 2021-11-21T22:01:00.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- start start 2021-11-21T22:01:00.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f06c00abce0 0x7f06c01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f06c00ac600 0x7f06c00aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c0147e80 con 0x7f06c0144810 2021-11-21T22:01:00.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00b1490 con 0x7f06c00abce0 2021-11-21T22:01:00.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00b1fd0 con 0x7f06c00ac600 2021-11-21T22:01:00.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f06c00abce0 0x7f06c01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f06c00abce0 0x7f06c01478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49816/0 (socket says 172.21.2.116:49816) 2021-11-21T22:01:00.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 -- 172.21.2.116:0/3281528379 learned_addr learned my addr 172.21.2.116:0/3281528379 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:00.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 -- 172.21.2.116:0/3281528379 >> v2:172.21.2.117:3300/0 conn(0x7f06c00ac600 msgr2=0x7f06c00aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:00.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 --2- 172.21.2.116:0/3281528379 >> v2:172.21.2.117:3300/0 conn(0x7f06c00ac600 0x7f06c00aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 -- 172.21.2.116:0/3281528379 >> v1:172.21.2.116:6789/0 conn(0x7f06c0144810 legacy=0x7f06c00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 -- 172.21.2.116:0/3281528379 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00af5a0 con 0x7f06c00abce0 2021-11-21T22:01:00.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bffff700 1 --2- 172.21.2.116:0/3281528379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01478c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f06a8002f70 tx=0x7f06a8006eb0).ready entity=mon.0 client_cookie=af87a55326d2043b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bf7fe700 1 -- 172.21.2.116:0/3281528379 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a800d610 con 0x7f06c00abce0 2021-11-21T22:01:00.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bf7fe700 1 -- 172.21.2.116:0/3281528379 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8000e90 con 0x7f06c00abce0 2021-11-21T22:01:00.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06bf7fe700 1 -- 172.21.2.116:0/3281528379 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a800d9a0 con 0x7f06c00abce0 2021-11-21T22:01:00.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- 172.21.2.116:0/3281528379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 msgr2=0x7f06c01478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 --2- 172.21.2.116:0/3281528379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01478c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f06a8002f70 tx=0x7f06a8006eb0).stop 2021-11-21T22:01:00.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- 172.21.2.116:0/3281528379 shutdown_connections 2021-11-21T22:01:00.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 --2- 172.21.2.116:0/3281528379 >> v2:172.21.2.117:3300/0 conn(0x7f06c00ac600 0x7f06c00aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 --2- 172.21.2.116:0/3281528379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01478c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:00.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- 172.21.2.116:0/3281528379 >> 172.21.2.116:0/3281528379 conn(0x7f06c005bd20 msgr2=0x7f06c005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:00.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- 172.21.2.116:0/3281528379 shutdown_connections 2021-11-21T22:01:00.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- 172.21.2.116:0/3281528379 wait complete. 2021-11-21T22:01:00.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 Processor -- start 2021-11-21T22:01:00.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.918+0000 7f06c6f54700 1 -- start start 2021-11-21T22:01:00.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06c00ac600 0x7f06c00b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06c0144810 0x7f06c015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0159e10 con 0x7f06c00abce0 2021-11-21T22:01:00.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00b3150 con 0x7f06c0144810 2021-11-21T22:01:00.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c015a040 con 0x7f06c00ac600 2021-11-21T22:01:00.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49820/0 (socket says 172.21.2.116:49820) 2021-11-21T22:01:00.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 -- 172.21.2.116:0/1658631075 learned_addr learned my addr 172.21.2.116:0/1658631075 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:00.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 -- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06c00ac600 msgr2=0x7f06c00b2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:00.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06c00ac600 0x7f06c00b2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 -- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06c0144810 msgr2=0x7f06c015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:00.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06c0144810 0x7f06c015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:00.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 -- 172.21.2.116:0/1658631075 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a800b1e0 con 0x7f06c00abce0 2021-11-21T22:01:00.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c4cf0700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01485c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f06c0062500 tx=0x7f06b0007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bdffb700 1 -- 172.21.2.116:0/1658631075 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b0007030 con 0x7f06c00abce0 2021-11-21T22:01:00.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bdffb700 1 -- 172.21.2.116:0/1658631075 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b0007190 con 0x7f06c00abce0 2021-11-21T22:01:00.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c015a2e0 con 0x7f06c00abce0 2021-11-21T22:01:00.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c015ac70 con 0x7f06c00abce0 2021-11-21T22:01:00.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bdffb700 1 -- 172.21.2.116:0/1658631075 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b0006510 con 0x7f06c00abce0 2021-11-21T22:01:00.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c00b32c0 con 0x7f06c00abce0 2021-11-21T22:01:00.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bdffb700 1 -- 172.21.2.116:0/1658631075 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f06b0010030 con 0x7f06c00abce0 2021-11-21T22:01:00.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bdffb700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f06a40535d0 0x7f06a4055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:00.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bdffb700 1 -- 172.21.2.116:0/1658631075 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f06b006b340 con 0x7f06c00abce0 2021-11-21T22:01:00.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bffff700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f06a40535d0 0x7f06a4055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:00.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.922+0000 7f06bffff700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f06a40535d0 0x7f06a4055a90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f06a8002f70 tx=0x7f06a8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:00.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:00.926+0000 7f06bdffb700 1 -- 172.21.2.116:0/1658631075 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06c00b32c0 con 0x7f06c00abce0 2021-11-21T22:01:01.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f06c00b32c0 con 0x7f06c00abce0 2021-11-21T22:01:01.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06bdffb700 1 -- 172.21.2.116:0/1658631075 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7f06c00b32c0 con 0x7f06c00abce0 2021-11-21T22:01:01.013 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:01.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f06a40535d0 msgr2=0x7f06a4055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f06a40535d0 0x7f06a4055a90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f06a8002f70 tx=0x7f06a8015000).stop 2021-11-21T22:01:01.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 msgr2=0x7f06c01485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01485c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f06c0062500 tx=0x7f06b0007c00).stop 2021-11-21T22:01:01.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 shutdown_connections 2021-11-21T22:01:01.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f06c0144810 0x7f06c015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f06c00ac600 0x7f06c00b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f06a40535d0 0x7f06a4055a90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 --2- 172.21.2.116:0/1658631075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f06c00abce0 0x7f06c01485c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 >> 172.21.2.116:0/1658631075 conn(0x7f06c005bd20 msgr2=0x7f06c00afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:01.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 shutdown_connections 2021-11-21T22:01:01.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.010+0000 7f06c6f54700 1 -- 172.21.2.116:0/1658631075 wait complete. 2021-11-21T22:01:01.021 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T22:01:01.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb7aa5700 1 Processor -- start 2021-11-21T22:01:01.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb7aa5700 1 -- start start 2021-11-21T22:01:01.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb7aa5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbeb00a6aa0 0x7fbeb014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb7aa5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a86b0 0x7fbeb01470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb7aa5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb00a6e80 con 0x7fbeb01475d0 2021-11-21T22:01:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb7aa5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb00a8bc0 con 0x7fbeb00a6aa0 2021-11-21T22:01:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb7aa5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb014a7b0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb5841700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbeb00a6aa0 0x7fbeb014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb5841700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbeb00a6aa0 0x7fbeb014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49832/0 (socket says 172.21.2.116:49832) 2021-11-21T22:01:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb5841700 1 -- 172.21.2.116:0/2671668241 learned_addr learned my addr 172.21.2.116:0/2671668241 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb5841700 1 -- 172.21.2.116:0/2671668241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a86b0 msgr2=0x7fbeb01470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.066+0000 7fbeb5841700 1 --2- 172.21.2.116:0/2671668241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a86b0 0x7fbeb01470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5841700 1 -- 172.21.2.116:0/2671668241 >> v1:172.21.2.116:6789/0 conn(0x7fbeb01475d0 legacy=0x7fbeb01443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5841700 1 -- 172.21.2.116:0/2671668241 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb014aa10 con 0x7fbeb00a6aa0 2021-11-21T22:01:01.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5841700 1 --2- 172.21.2.116:0/2671668241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a6aa0 0x7fbeb014b150 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fbea4002f70 tx=0x7fbea4006eb0).ready entity=mon.0 client_cookie=56bf138ada3d26c2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:01.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb483f700 1 -- 172.21.2.116:0/2671668241 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbea400d5c0 con 0x7fbeb00a6aa0 2021-11-21T22:01:01.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb483f700 1 -- 172.21.2.116:0/2671668241 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea4000e90 con 0x7fbeb00a6aa0 2021-11-21T22:01:01.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb483f700 1 -- 172.21.2.116:0/2671668241 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbea400d950 con 0x7fbeb00a6aa0 2021-11-21T22:01:01.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/2671668241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a6aa0 msgr2=0x7fbeb014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/2671668241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a6aa0 0x7fbeb014b150 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fbea4002f70 tx=0x7fbea4006eb0).stop 2021-11-21T22:01:01.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/2671668241 shutdown_connections 2021-11-21T22:01:01.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/2671668241 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a86b0 0x7fbeb01470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/2671668241 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a6aa0 0x7fbeb014b150 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/2671668241 >> 172.21.2.116:0/2671668241 conn(0x7fbeb005bd20 msgr2=0x7fbeb005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:01.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/2671668241 shutdown_connections 2021-11-21T22:01:01.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/2671668241 wait complete. 2021-11-21T22:01:01.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 Processor -- start 2021-11-21T22:01:01.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- start start 2021-11-21T22:01:01.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a6aa0 0x7fbeb01451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a86b0 0x7fbeb015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbeb01475d0 0x7fbeb0156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00b1710 con 0x7fbeb00a86b0 2021-11-21T22:01:01.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb015a720 con 0x7fbeb01475d0 2021-11-21T22:01:01.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00a8e00 con 0x7fbeb00a6aa0 2021-11-21T22:01:01.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a86b0 0x7fbeb015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a86b0 0x7fbeb015a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49834/0 (socket says 172.21.2.116:49834) 2021-11-21T22:01:01.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 -- 172.21.2.116:0/3661379838 learned_addr learned my addr 172.21.2.116:0/3661379838 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:01.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb6042700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbeb01475d0 0x7fbeb0156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5841700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a6aa0 0x7fbeb01451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 -- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a6aa0 msgr2=0x7fbeb01451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a6aa0 0x7fbeb01451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 -- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbeb01475d0 msgr2=0x7fbeb0156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbeb01475d0 0x7fbeb0156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 -- 172.21.2.116:0/3661379838 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea400b1e0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5841700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a6aa0 0x7fbeb01451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:01.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb5040700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a86b0 0x7fbeb015a230 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fbeac002f70 tx=0x7fbeac006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:01.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbea2ffd700 1 -- 172.21.2.116:0/3661379838 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac013460 con 0x7fbeb00a86b0 2021-11-21T22:01:01.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbea2ffd700 1 -- 172.21.2.116:0/3661379838 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac0135c0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb00a90a0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb0064fe0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbea2ffd700 1 -- 172.21.2.116:0/3661379838 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac013920 con 0x7fbeb00a86b0 2021-11-21T22:01:01.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.070+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb00a9450 con 0x7fbeb00a86b0 2021-11-21T22:01:01.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.074+0000 7fbea2ffd700 1 -- 172.21.2.116:0/3661379838 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbeac01a060 con 0x7fbeb00a86b0 2021-11-21T22:01:01.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.074+0000 7fbea2ffd700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe900536a0 0x7fbe90055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.074+0000 7fbea2ffd700 1 -- 172.21.2.116:0/3661379838 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbeac06b150 con 0x7fbeb00a86b0 2021-11-21T22:01:01.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.074+0000 7fbeb5841700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe900536a0 0x7fbe90055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.074+0000 7fbeb5841700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe900536a0 0x7fbe90055b60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fbea4017000 tx=0x7fbea4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:01.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.074+0000 7fbea2ffd700 1 -- 172.21.2.116:0/3661379838 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbeac0424d0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbeb00573d0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbea2ffd700 1 -- 172.21.2.116:0/3661379838 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) v1 ==== 84+0+143 (secure 0 0 0) 0x7fbeac045af0 con 0x7fbeb00a86b0 2021-11-21T22:01:01.163 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:01.163 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T22:01:01.163 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:01.164 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T22:01:01.164 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T22:01:01.164 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:01.164 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:01.165 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T22:01:01.165 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T22:01:01.165 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:01.165 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T22:01:01.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe900536a0 msgr2=0x7fbe90055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe900536a0 0x7fbe90055b60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fbea4017000 tx=0x7fbea4015000).stop 2021-11-21T22:01:01.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a86b0 msgr2=0x7fbeb015a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a86b0 0x7fbeb015a230 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fbeac002f70 tx=0x7fbeac006e90).stop 2021-11-21T22:01:01.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 shutdown_connections 2021-11-21T22:01:01.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbeb01475d0 0x7fbeb0156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe900536a0 0x7fbe90055b60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbeb00a86b0 0x7fbeb015a230 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 --2- 172.21.2.116:0/3661379838 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbeb00a6aa0 0x7fbeb01451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 >> 172.21.2.116:0/3661379838 conn(0x7fbeb005bd20 msgr2=0x7fbeb00b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:01.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 shutdown_connections 2021-11-21T22:01:01.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.158+0000 7fbeb7aa5700 1 -- 172.21.2.116:0/3661379838 wait complete. 2021-11-21T22:01:01.171 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T22:01:01.171 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:01:01.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 Processor -- start 2021-11-21T22:01:01.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- start start 2021-11-21T22:01:01.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e4144810 0x7f89e41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f89e40abce0 0x7f89e40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f89e40b1db0 0x7f89e40ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e4147e80 con 0x7f89e40b1db0 2021-11-21T22:01:01.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40af5a0 con 0x7f89e40abce0 2021-11-21T22:01:01.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40b1490 con 0x7f89e4144810 2021-11-21T22:01:01.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f89e40b1db0 0x7f89e40ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f89e40b1db0 0x7f89e40ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49844/0 (socket says 172.21.2.116:49844) 2021-11-21T22:01:01.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/224818817 learned_addr learned my addr 172.21.2.116:0/224818817 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:01.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/224818817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e4144810 msgr2=0x7f89e41478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:01.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- 172.21.2.116:0/224818817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e4144810 0x7f89e41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/224818817 >> v2:172.21.2.117:3300/0 conn(0x7f89e40abce0 msgr2=0x7f89e40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:01.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- 172.21.2.116:0/224818817 >> v2:172.21.2.117:3300/0 conn(0x7f89e40abce0 0x7f89e40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/224818817 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e40af060 con 0x7f89e40b1db0 2021-11-21T22:01:01.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- 172.21.2.116:0/224818817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e40b1db0 0x7f89e40ac600 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f89dc0032f0 tx=0x7f89dc009290).ready entity=mon.0 client_cookie=1cc103621486c11f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:01.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89d7fff700 1 -- 172.21.2.116:0/224818817 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89dc00b870 con 0x7f89e40b1db0 2021-11-21T22:01:01.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89d7fff700 1 -- 172.21.2.116:0/224818817 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00b9d0 con 0x7f89e40b1db0 2021-11-21T22:01:01.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89d7fff700 1 -- 172.21.2.116:0/224818817 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89dc00bb40 con 0x7f89e40b1db0 2021-11-21T22:01:01.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- 172.21.2.116:0/224818817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e40b1db0 msgr2=0x7f89e40ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:01.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/224818817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e40b1db0 0x7f89e40ac600 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f89dc0032f0 tx=0x7f89dc009290).stop 2021-11-21T22:01:01.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- 172.21.2.116:0/224818817 shutdown_connections 2021-11-21T22:01:01.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/224818817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e40b1db0 0x7f89e40ac600 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:01.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/224818817 >> v2:172.21.2.117:3300/0 conn(0x7f89e40abce0 0x7f89e40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/224818817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e4144810 0x7f89e41478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- 172.21.2.116:0/224818817 >> 172.21.2.116:0/224818817 conn(0x7f89e405bd20 msgr2=0x7f89e405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:01.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- 172.21.2.116:0/224818817 shutdown_connections 2021-11-21T22:01:01.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- 172.21.2.116:0/224818817 wait complete. 2021-11-21T22:01:01.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 Processor -- start 2021-11-21T22:01:01.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- start start 2021-11-21T22:01:01.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e40abce0 0x7f89e40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89e4144810 0x7f89e4159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e4144d20 0x7f89e415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4145310 con 0x7f89e4144d20 2021-11-21T22:01:01.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4143360 con 0x7f89e4144810 2021-11-21T22:01:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4143600 con 0x7f89e40abce0 2021-11-21T22:01:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e4144d20 0x7f89e415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e4144d20 0x7f89e415a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49850/0 (socket says 172.21.2.116:49850) 2021-11-21T22:01:01.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/2311372532 learned_addr learned my addr 172.21.2.116:0/2311372532 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e40abce0 msgr2=0x7f89e40b3f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e40abce0 0x7f89e40b3f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89e4144810 msgr2=0x7f89e4159f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89e4144810 0x7f89e4159f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:01.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 -- 172.21.2.116:0/2311372532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc009530 con 0x7f89e4144d20 2021-11-21T22:01:01.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89e9d4b700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e4144d20 0x7f89e415a420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f89dc0032f0 tx=0x7f89dc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:01.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89d67fc700 1 -- 172.21.2.116:0/2311372532 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89dc00b870 con 0x7f89e4144d20 2021-11-21T22:01:01.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89d67fc700 1 -- 172.21.2.116:0/2311372532 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc020430 con 0x7f89e4144d20 2021-11-21T22:01:01.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e4159a90 con 0x7f89e4144d20 2021-11-21T22:01:01.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89d67fc700 1 -- 172.21.2.116:0/2311372532 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89dc020790 con 0x7f89e4144d20 2021-11-21T22:01:01.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.222+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e4142fe0 con 0x7f89e4144d20 2021-11-21T22:01:01.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.226+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e4159c20 con 0x7f89e4144d20 2021-11-21T22:01:01.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.226+0000 7f89d67fc700 1 -- 172.21.2.116:0/2311372532 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f89dc00f070 con 0x7f89e4144d20 2021-11-21T22:01:01.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.226+0000 7f89d67fc700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f89c4053580 0x7f89c4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:01.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.226+0000 7f89d67fc700 1 -- 172.21.2.116:0/2311372532 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f89dc06c820 con 0x7f89e4144d20 2021-11-21T22:01:01.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.226+0000 7f89e954a700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f89c4053580 0x7f89c4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:01.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.226+0000 7f89e954a700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f89c4053580 0x7f89c4055a40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f89e0002f70 tx=0x7f89e0006970).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:01.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.226+0000 7f89d67fc700 1 -- 172.21.2.116:0/2311372532 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f89dc043cb0 con 0x7f89e4144d20 2021-11-21T22:01:01.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:01.310+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f89e40af300 con 0x7f89e4144d20 2021-11-21T22:01:02.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89d67fc700 1 -- 172.21.2.116:0/2311372532 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v85) v1 ==== 133+0+0 (secure 0 0 0) 0x7f89dc015070 con 0x7f89e4144d20 2021-11-21T22:01:02.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f89c4053580 msgr2=0x7f89c4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:02.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f89c4053580 0x7f89c4055a40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f89e0002f70 tx=0x7f89e0006970).stop 2021-11-21T22:01:02.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e4144d20 msgr2=0x7f89e415a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:02.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e4144d20 0x7f89e415a420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f89dc0032f0 tx=0x7f89dc016000).stop 2021-11-21T22:01:02.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 shutdown_connections 2021-11-21T22:01:02.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f89c4053580 0x7f89c4055a40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:02.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f89e4144d20 0x7f89e415a420 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:02.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f89e4144810 0x7f89e4159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:02.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 --2- 172.21.2.116:0/2311372532 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f89e40abce0 0x7f89e40b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:02.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 >> 172.21.2.116:0/2311372532 conn(0x7f89e405bd20 msgr2=0x7f89e4148510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:02.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 shutdown_connections 2021-11-21T22:01:02.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.250+0000 7f89eb7ae700 1 -- 172.21.2.116:0/2311372532 wait complete. 2021-11-21T22:01:02.258 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:01:02.261 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:01:02.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb66ce4700 1 Processor -- start 2021-11-21T22:01:02.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb66ce4700 1 -- start start 2021-11-21T22:01:02.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb66ce4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb600b5490 0x7feb600b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:02.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb66ce4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb60097ec0 0x7feb6014a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:02.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb66ce4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb600995c0 con 0x7feb600af720 2021-11-21T22:01:02.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb66ce4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb60144b40 con 0x7feb60097ec0 2021-11-21T22:01:02.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb66ce4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6014b140 con 0x7feb600b5490 2021-11-21T22:01:02.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb60097ec0 0x7feb6014a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:02.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7feb60097ec0 0x7feb6014a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49856/0 (socket says 172.21.2.116:49856) 2021-11-21T22:01:02.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 -- 172.21.2.116:0/3009949171 learned_addr learned my addr 172.21.2.116:0/3009949171 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:02.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 -- 172.21.2.116:0/3009949171 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb600b5490 msgr2=0x7feb600b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:02.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 --2- 172.21.2.116:0/3009949171 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb600b5490 0x7feb600b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:02.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 -- 172.21.2.116:0/3009949171 >> v1:172.21.2.117:6789/0 conn(0x7feb600af720 legacy=0x7feb60146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:02.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 -- 172.21.2.116:0/3009949171 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600997f0 con 0x7feb60097ec0 2021-11-21T22:01:02.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb664e3700 1 --2- 172.21.2.116:0/3009949171 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb60097ec0 0x7feb6014a3b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7feb60099e10 tx=0x7feb58009290).ready entity=mon.0 client_cookie=137f2137b0bf5631 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:02.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb64ce0700 1 -- 172.21.2.116:0/3009949171 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb5800b860 con 0x7feb60097ec0 2021-11-21T22:01:02.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb64ce0700 1 -- 172.21.2.116:0/3009949171 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5800b9c0 con 0x7feb60097ec0 2021-11-21T22:01:02.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.318+0000 7feb64ce0700 1 -- 172.21.2.116:0/3009949171 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb5800bb70 con 0x7feb60097ec0 2021-11-21T22:01:02.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/3009949171 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb60097ec0 msgr2=0x7feb6014a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:02.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 --2- 172.21.2.116:0/3009949171 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb60097ec0 0x7feb6014a3b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7feb60099e10 tx=0x7feb58009290).stop 2021-11-21T22:01:02.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/3009949171 shutdown_connections 2021-11-21T22:01:02.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 --2- 172.21.2.116:0/3009949171 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb60097ec0 0x7feb6014a3b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:02.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 --2- 172.21.2.116:0/3009949171 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb600b5490 0x7feb600b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:02.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/3009949171 >> 172.21.2.116:0/3009949171 conn(0x7feb6005c7a0 msgr2=0x7feb600b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:02.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/3009949171 shutdown_connections 2021-11-21T22:01:02.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/3009949171 wait complete. 2021-11-21T22:01:02.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 Processor -- start 2021-11-21T22:01:02.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- start start 2021-11-21T22:01:02.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb60097ec0 0x7feb6014a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:02.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb600af720 0x7feb600ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:02.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb600b5490 0x7feb60156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:02.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600af080 con 0x7feb600af720 2021-11-21T22:01:02.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60156230 con 0x7feb600b5490 2021-11-21T22:01:02.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb601564d0 con 0x7feb60097ec0 2021-11-21T22:01:02.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb600af720 0x7feb600ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:02.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb600af720 0x7feb600ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49864/0 (socket says 172.21.2.116:49864) 2021-11-21T22:01:02.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 -- 172.21.2.116:0/4031497256 learned_addr learned my addr 172.21.2.116:0/4031497256 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:02.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb65ce2700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb60097ec0 0x7feb6014a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:02.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 -- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb60097ec0 msgr2=0x7feb6014a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:02.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb664e3700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb600b5490 0x7feb60156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:02.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb60097ec0 0x7feb6014a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:02.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 -- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb600b5490 msgr2=0x7feb60156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:02.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb600b5490 0x7feb60156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:02.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 -- 172.21.2.116:0/4031497256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58009530 con 0x7feb600af720 2021-11-21T22:01:02.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb65ce2700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb60097ec0 0x7feb6014a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:02.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb664e3700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb600b5490 0x7feb60156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:02.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb654e1700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb600af720 0x7feb600ae9e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7feb50002f90 tx=0x7feb50006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:02.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb4effd700 1 -- 172.21.2.116:0/4031497256 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb50013460 con 0x7feb600af720 2021-11-21T22:01:02.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb4effd700 1 -- 172.21.2.116:0/4031497256 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb500135c0 con 0x7feb600af720 2021-11-21T22:01:02.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb60156770 con 0x7feb600af720 2021-11-21T22:01:02.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb4effd700 1 -- 172.21.2.116:0/4031497256 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb50013920 con 0x7feb600af720 2021-11-21T22:01:02.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb601533c0 con 0x7feb600af720 2021-11-21T22:01:02.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.322+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb60099a90 con 0x7feb600af720 2021-11-21T22:01:02.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.326+0000 7feb4effd700 1 -- 172.21.2.116:0/4031497256 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7feb5001a060 con 0x7feb600af720 2021-11-21T22:01:02.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.326+0000 7feb4effd700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb400536f0 0x7feb40055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:02.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.326+0000 7feb4effd700 1 -- 172.21.2.116:0/4031497256 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5603+0+0 (secure 0 0 0) 0x7feb5006b200 con 0x7feb600af720 2021-11-21T22:01:02.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.326+0000 7feb65ce2700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb400536f0 0x7feb40055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:02.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.326+0000 7feb65ce2700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb400536f0 0x7feb40055bb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7feb5c007000 tx=0x7feb5c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:02.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.326+0000 7feb4effd700 1 -- 172.21.2.116:0/4031497256 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feb50042420 con 0x7feb600af720 2021-11-21T22:01:02.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:02.410+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7feb600573d0 con 0x7feb600af720 2021-11-21T22:01:03.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.262+0000 7feb4effd700 1 -- 172.21.2.116:0/4031497256 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v86) v1 ==== 125+0+0 (secure 0 0 0) 0x7feb50045a40 con 0x7feb600af720 2021-11-21T22:01:03.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb400536f0 msgr2=0x7feb40055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb400536f0 0x7feb40055bb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7feb5c007000 tx=0x7feb5c008040).stop 2021-11-21T22:01:03.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb600af720 msgr2=0x7feb600ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb600af720 0x7feb600ae9e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7feb50002f90 tx=0x7feb50006de0).stop 2021-11-21T22:01:03.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 shutdown_connections 2021-11-21T22:01:03.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7feb600b5490 0x7feb60156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feb400536f0 0x7feb40055bb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7feb600af720 0x7feb600ae9e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 --2- 172.21.2.116:0/4031497256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7feb60097ec0 0x7feb6014a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 >> 172.21.2.116:0/4031497256 conn(0x7feb6005c7a0 msgr2=0x7feb601471b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 shutdown_connections 2021-11-21T22:01:03.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.266+0000 7feb66ce4700 1 -- 172.21.2.116:0/4031497256 wait complete. 2021-11-21T22:01:03.275 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:01:03.276 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:01:03.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 Processor -- start 2021-11-21T22:01:03.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- start start 2021-11-21T22:01:03.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fac580b5660 0x7fac5813bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fac5805efb0 0x7fac5813c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac580b5a40 con 0x7fac580b1660 2021-11-21T22:01:03.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac580aef20 con 0x7fac5805efb0 2021-11-21T22:01:03.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5813c8f0 con 0x7fac580b5660 2021-11-21T22:01:03.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fac5805efb0 0x7fac5813c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fac5805efb0 0x7fac5813c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49874/0 (socket says 172.21.2.116:49874) 2021-11-21T22:01:03.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 -- 172.21.2.116:0/1290924932 learned_addr learned my addr 172.21.2.116:0/1290924932 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 -- 172.21.2.116:0/1290924932 >> v2:172.21.2.117:3300/0 conn(0x7fac580b5660 msgr2=0x7fac5813bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:03.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 --2- 172.21.2.116:0/1290924932 >> v2:172.21.2.117:3300/0 conn(0x7fac580b5660 0x7fac5813bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 -- 172.21.2.116:0/1290924932 >> v1:172.21.2.117:6789/0 conn(0x7fac580b1660 legacy=0x7fac580b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:03.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 -- 172.21.2.116:0/1290924932 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580ae540 con 0x7fac5805efb0 2021-11-21T22:01:03.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac57fff700 1 --2- 172.21.2.116:0/1290924932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac5805efb0 0x7fac5813c400 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fac580ae2a0 tx=0x7fac40006eb0).ready entity=mon.0 client_cookie=267e94ba66b31832 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac577fe700 1 -- 172.21.2.116:0/1290924932 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac4000d5c0 con 0x7fac5805efb0 2021-11-21T22:01:03.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac577fe700 1 -- 172.21.2.116:0/1290924932 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac40000e90 con 0x7fac5805efb0 2021-11-21T22:01:03.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac577fe700 1 -- 172.21.2.116:0/1290924932 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac4000d910 con 0x7fac5805efb0 2021-11-21T22:01:03.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- 172.21.2.116:0/1290924932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac5805efb0 msgr2=0x7fac5813c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 --2- 172.21.2.116:0/1290924932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac5805efb0 0x7fac5813c400 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fac580ae2a0 tx=0x7fac40006eb0).stop 2021-11-21T22:01:03.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- 172.21.2.116:0/1290924932 shutdown_connections 2021-11-21T22:01:03.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 --2- 172.21.2.116:0/1290924932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac5805efb0 0x7fac5813c400 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 --2- 172.21.2.116:0/1290924932 >> v2:172.21.2.117:3300/0 conn(0x7fac580b5660 0x7fac5813bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- 172.21.2.116:0/1290924932 >> 172.21.2.116:0/1290924932 conn(0x7fac5805f730 msgr2=0x7fac58063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- 172.21.2.116:0/1290924932 shutdown_connections 2021-11-21T22:01:03.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.326+0000 7fac5ea80700 1 -- 172.21.2.116:0/1290924932 wait complete. 2021-11-21T22:01:03.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 Processor -- start 2021-11-21T22:01:03.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 -- start start 2021-11-21T22:01:03.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac5805efb0 0x7fac58064060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac580b1660 0x7fac580a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac580b5660 0x7fac580aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580ab820 con 0x7fac580b5660 2021-11-21T22:01:03.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac581428d0 con 0x7fac5805efb0 2021-11-21T22:01:03.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac58141f20 con 0x7fac580b1660 2021-11-21T22:01:03.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac580b5660 0x7fac580aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac580b5660 0x7fac580aa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49878/0 (socket says 172.21.2.116:49878) 2021-11-21T22:01:03.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 -- 172.21.2.116:0/4010839752 learned_addr learned my addr 172.21.2.116:0/4010839752 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 -- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac580b1660 msgr2=0x7fac580a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac57fff700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac580b1660 0x7fac580a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac580b1660 0x7fac580a9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5c81c700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac5805efb0 0x7fac58064060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 -- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac5805efb0 msgr2=0x7fac58064060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac5805efb0 0x7fac58064060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 -- 172.21.2.116:0/4010839752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4000b1e0 con 0x7fac580b5660 2021-11-21T22:01:03.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac57fff700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac580b1660 0x7fac580a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:03.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5c81c700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac5805efb0 0x7fac58064060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:03.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5d01d700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac580b5660 0x7fac580aa290 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fac50003310 tx=0x7fac500091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac55ffb700 1 -- 172.21.2.116:0/4010839752 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac50001500 con 0x7fac580b5660 2021-11-21T22:01:03.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac55ffb700 1 -- 172.21.2.116:0/4010839752 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50001660 con 0x7fac580b5660 2021-11-21T22:01:03.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac58130ef0 con 0x7fac580b5660 2021-11-21T22:01:03.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac55ffb700 1 -- 172.21.2.116:0/4010839752 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac500019c0 con 0x7fac580b5660 2021-11-21T22:01:03.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac58142da0 con 0x7fac580b5660 2021-11-21T22:01:03.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac58142f60 con 0x7fac580b5660 2021-11-21T22:01:03.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac55ffb700 1 -- 172.21.2.116:0/4010839752 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fac50018030 con 0x7fac580b5660 2021-11-21T22:01:03.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac55ffb700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac3c0535b0 0x7fac3c055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac55ffb700 1 -- 172.21.2.116:0/4010839752 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fac5006b210 con 0x7fac580b5660 2021-11-21T22:01:03.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.330+0000 7fac5c81c700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac3c0535b0 0x7fac3c055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.334+0000 7fac5c81c700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac3c0535b0 0x7fac3c055a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fac48007000 tx=0x7fac48008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.334+0000 7fac55ffb700 1 -- 172.21.2.116:0/4010839752 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac50042490 con 0x7fac580b5660 2021-11-21T22:01:03.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.442+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fac580ae7e0 con 0x7fac580b5660 2021-11-21T22:01:03.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.446+0000 7fac55ffb700 1 -- 172.21.2.116:0/4010839752 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v37) v1 ==== 189+0+0 (secure 0 0 0) 0x7fac50045ab0 con 0x7fac580b5660 2021-11-21T22:01:03.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac3c0535b0 msgr2=0x7fac3c055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac3c0535b0 0x7fac3c055a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fac48007000 tx=0x7fac48008040).stop 2021-11-21T22:01:03.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac580b5660 msgr2=0x7fac580aa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac580b5660 0x7fac580aa290 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fac50003310 tx=0x7fac500091a0).stop 2021-11-21T22:01:03.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 shutdown_connections 2021-11-21T22:01:03.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fac3c0535b0 0x7fac3c055a70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fac580b5660 0x7fac580aa290 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fac580b1660 0x7fac580a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 --2- 172.21.2.116:0/4010839752 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fac5805efb0 0x7fac58064060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 >> 172.21.2.116:0/4010839752 conn(0x7fac5805f730 msgr2=0x7fac58138ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 shutdown_connections 2021-11-21T22:01:03.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.450+0000 7fac5ea80700 1 -- 172.21.2.116:0/4010839752 wait complete. 2021-11-21T22:01:03.459 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 15 and data pool 16 2021-11-21T22:01:03.461 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:03.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.506+0000 7f99764aa700 1 Processor -- start 2021-11-21T22:01:03.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.506+0000 7f99764aa700 1 -- start start 2021-11-21T22:01:03.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.506+0000 7f99764aa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a6aa0 0x7f997014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.506+0000 7f99764aa700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f99700a86b0 0x7f99701470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.506+0000 7f99764aa700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99700a6e80 con 0x7f99701475d0 2021-11-21T22:01:03.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.506+0000 7f99764aa700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99700a8bc0 con 0x7f99700a86b0 2021-11-21T22:01:03.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.506+0000 7f99764aa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997014a7b0 con 0x7f99700a6aa0 2021-11-21T22:01:03.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f99700a86b0 0x7f99701470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f99700a86b0 0x7f99701470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49896/0 (socket says 172.21.2.116:49896) 2021-11-21T22:01:03.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 -- 172.21.2.116:0/311620063 learned_addr learned my addr 172.21.2.116:0/311620063 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 --2- 172.21.2.116:0/311620063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a6aa0 0x7f997014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 -- 172.21.2.116:0/311620063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a6aa0 msgr2=0x7f997014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:03.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 --2- 172.21.2.116:0/311620063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a6aa0 0x7f997014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 -- 172.21.2.116:0/311620063 >> v1:172.21.2.117:6789/0 conn(0x7f99701475d0 legacy=0x7f99701443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:03.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 -- 172.21.2.116:0/311620063 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997014aa10 con 0x7f99700a86b0 2021-11-21T22:01:03.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 --2- 172.21.2.116:0/311620063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a6aa0 0x7f997014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:03.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996f7fe700 1 --2- 172.21.2.116:0/311620063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a86b0 0x7f99701470e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f9970061ab0 tx=0x7f9960006eb0).ready entity=mon.0 client_cookie=47f45001d1913556 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996effd700 1 -- 172.21.2.116:0/311620063 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996000d570 con 0x7f99700a86b0 2021-11-21T22:01:03.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996effd700 1 -- 172.21.2.116:0/311620063 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9960000e90 con 0x7f99700a86b0 2021-11-21T22:01:03.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996effd700 1 -- 172.21.2.116:0/311620063 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996000d8c0 con 0x7f99700a86b0 2021-11-21T22:01:03.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- 172.21.2.116:0/311620063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a86b0 msgr2=0x7f99701470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 --2- 172.21.2.116:0/311620063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a86b0 0x7f99701470e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f9970061ab0 tx=0x7f9960006eb0).stop 2021-11-21T22:01:03.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- 172.21.2.116:0/311620063 shutdown_connections 2021-11-21T22:01:03.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 --2- 172.21.2.116:0/311620063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a86b0 0x7f99701470e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 --2- 172.21.2.116:0/311620063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a6aa0 0x7f997014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- 172.21.2.116:0/311620063 >> 172.21.2.116:0/311620063 conn(0x7f997005bd20 msgr2=0x7f997005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- 172.21.2.116:0/311620063 shutdown_connections 2021-11-21T22:01:03.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- 172.21.2.116:0/311620063 wait complete. 2021-11-21T22:01:03.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 Processor -- start 2021-11-21T22:01:03.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- start start 2021-11-21T22:01:03.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a6aa0 0x7f9970065780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a86b0 0x7f9970144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99701475d0 0x7f9970063cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99701454e0 con 0x7f99700a6aa0 2021-11-21T22:01:03.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970156f90 con 0x7f99701475d0 2021-11-21T22:01:03.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970064880 con 0x7f99700a86b0 2021-11-21T22:01:03.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a6aa0 0x7f9970065780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a6aa0 0x7f9970065780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49900/0 (socket says 172.21.2.116:49900) 2021-11-21T22:01:03.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 -- 172.21.2.116:0/607055763 learned_addr learned my addr 172.21.2.116:0/607055763 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 -- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a86b0 msgr2=0x7f9970144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a86b0 0x7f9970144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 -- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99701475d0 msgr2=0x7f9970063cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99701475d0 0x7f9970063cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 -- 172.21.2.116:0/607055763 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996000b1e0 con 0x7f99700a6aa0 2021-11-21T22:01:03.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996ffff700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a6aa0 0x7f9970065780 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9958007000 tx=0x7f9958008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996d7fa700 1 -- 172.21.2.116:0/607055763 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9958010580 con 0x7f99700a6aa0 2021-11-21T22:01:03.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996d7fa700 1 -- 172.21.2.116:0/607055763 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99580106e0 con 0x7f99700a6aa0 2021-11-21T22:01:03.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f996d7fa700 1 -- 172.21.2.116:0/607055763 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9958010a40 con 0x7f99700a6aa0 2021-11-21T22:01:03.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9970064b20 con 0x7f99700a6aa0 2021-11-21T22:01:03.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.510+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99700641b0 con 0x7f99700a6aa0 2021-11-21T22:01:03.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.514+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9970157190 con 0x7f99700a6aa0 2021-11-21T22:01:03.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.514+0000 7f996d7fa700 1 -- 172.21.2.116:0/607055763 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9958010ba0 con 0x7f99700a6aa0 2021-11-21T22:01:03.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.514+0000 7f996d7fa700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99540535d0 0x7f9954055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.514+0000 7f996d7fa700 1 -- 172.21.2.116:0/607055763 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f995806c320 con 0x7f99700a6aa0 2021-11-21T22:01:03.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.514+0000 7f996f7fe700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99540535d0 0x7f9954055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.514+0000 7f996f7fe700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99540535d0 0x7f9954055a90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f9970061ab0 tx=0x7f9960015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.514+0000 7f996d7fa700 1 -- 172.21.2.116:0/607055763 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9958043310 con 0x7f99700a6aa0 2021-11-21T22:01:03.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9970064cb0 con 0x7f99700a6aa0 2021-11-21T22:01:03.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f996d7fa700 1 -- 172.21.2.116:0/607055763 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f9958046930 con 0x7f99700a6aa0 2021-11-21T22:01:03.604 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:03.604 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":87,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:03.446975+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:01.325473+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:02.426827+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:03.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99540535d0 msgr2=0x7f9954055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99540535d0 0x7f9954055a90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f9970061ab0 tx=0x7f9960015000).stop 2021-11-21T22:01:03.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a6aa0 msgr2=0x7f9970065780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a6aa0 0x7f9970065780 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9958007000 tx=0x7f9958008040).stop 2021-11-21T22:01:03.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 shutdown_connections 2021-11-21T22:01:03.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99701475d0 0x7f9970063cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f99700a86b0 0x7f9970144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99540535d0 0x7f9954055a90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 --2- 172.21.2.116:0/607055763 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99700a6aa0 0x7f9970065780 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.598+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 >> 172.21.2.116:0/607055763 conn(0x7f997005bd20 msgr2=0x7f9970148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.602+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 shutdown_connections 2021-11-21T22:01:03.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.602+0000 7f99764aa700 1 -- 172.21.2.116:0/607055763 wait complete. 2021-11-21T22:01:03.611 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:03.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631bfff700 1 Processor -- start 2021-11-21T22:01:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631bfff700 1 -- start start 2021-11-21T22:01:03.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631bfff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 0x7f631c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f631c097ec0 0x7f631c146e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631bfff700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631c14a560 con 0x7f631c147300 2021-11-21T22:01:03.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631bfff700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c144b40 con 0x7f631c097ec0 2021-11-21T22:01:03.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631bfff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c14a800 con 0x7f631c0b5630 2021-11-21T22:01:03.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f631c097ec0 0x7f631c146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f631c097ec0 0x7f631c146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49908/0 (socket says 172.21.2.116:49908) 2021-11-21T22:01:03.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 -- 172.21.2.116:0/943020706 learned_addr learned my addr 172.21.2.116:0/943020706 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631b7fe700 1 --1- 172.21.2.116:0/943020706 >> v1:172.21.2.116:6789/0 conn(0x7f631c147300 0x7f631c099650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33632/0 (socket says 172.21.2.116:33632) 2021-11-21T22:01:03.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631affd700 1 --2- 172.21.2.116:0/943020706 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 0x7f631c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 -- 172.21.2.116:0/943020706 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 msgr2=0x7f631c14af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:03.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 --2- 172.21.2.116:0/943020706 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 0x7f631c14af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 -- 172.21.2.116:0/943020706 >> v1:172.21.2.116:6789/0 conn(0x7f631c147300 legacy=0x7f631c099650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 -- 172.21.2.116:0/943020706 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c14aaa0 con 0x7f631c097ec0 2021-11-21T22:01:03.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f631a7fc700 1 --2- 172.21.2.116:0/943020706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c146e10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f6304002f70 tx=0x7f6304006eb0).ready entity=mon.0 client_cookie=22efd7e754d044e8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f6319ffb700 1 -- 172.21.2.116:0/943020706 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f630400d5c0 con 0x7f631c097ec0 2021-11-21T22:01:03.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f6319ffb700 1 -- 172.21.2.116:0/943020706 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6304000e90 con 0x7f631c097ec0 2021-11-21T22:01:03.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.658+0000 7f6319ffb700 1 -- 172.21.2.116:0/943020706 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f630400d950 con 0x7f631c097ec0 2021-11-21T22:01:03.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/943020706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 msgr2=0x7f631c146e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 --2- 172.21.2.116:0/943020706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c146e10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f6304002f70 tx=0x7f6304006eb0).stop 2021-11-21T22:01:03.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/943020706 shutdown_connections 2021-11-21T22:01:03.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 --2- 172.21.2.116:0/943020706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c146e10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 --2- 172.21.2.116:0/943020706 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 0x7f631c14af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/943020706 >> 172.21.2.116:0/943020706 conn(0x7f631c05c7f0 msgr2=0x7f631c0b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/943020706 shutdown_connections 2021-11-21T22:01:03.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/943020706 wait complete. 2021-11-21T22:01:03.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 Processor -- start 2021-11-21T22:01:03.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- start start 2021-11-21T22:01:03.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 0x7f631c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f631c147300 0x7f631c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c063660 con 0x7f631c097ec0 2021-11-21T22:01:03.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c063890 con 0x7f631c147300 2021-11-21T22:01:03.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c063b30 con 0x7f631c0b5630 2021-11-21T22:01:03.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49916/0 (socket says 172.21.2.116:49916) 2021-11-21T22:01:03.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 -- 172.21.2.116:0/1798870791 learned_addr learned my addr 172.21.2.116:0/1798870791 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 -- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 msgr2=0x7f631c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 0x7f631c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 -- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f631c147300 msgr2=0x7f631c0ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f631c147300 0x7f631c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 -- 172.21.2.116:0/1798870791 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630400b1e0 con 0x7f631c097ec0 2021-11-21T22:01:03.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631affd700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c149870 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f630c007000 tx=0x7f630c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f6313fff700 1 -- 172.21.2.116:0/1798870791 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f630c010580 con 0x7f631c097ec0 2021-11-21T22:01:03.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f6313fff700 1 -- 172.21.2.116:0/1798870791 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c0106e0 con 0x7f631c097ec0 2021-11-21T22:01:03.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f6313fff700 1 -- 172.21.2.116:0/1798870791 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f630c010a40 con 0x7f631c097ec0 2021-11-21T22:01:03.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c156400 con 0x7f631c097ec0 2021-11-21T22:01:03.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c067420 con 0x7f631c097ec0 2021-11-21T22:01:03.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c156590 con 0x7f631c097ec0 2021-11-21T22:01:03.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f6313fff700 1 -- 172.21.2.116:0/1798870791 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f630c010ba0 con 0x7f631c097ec0 2021-11-21T22:01:03.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f6313fff700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6300053580 0x7f6300055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.662+0000 7f6313fff700 1 -- 172.21.2.116:0/1798870791 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f630c06bb20 con 0x7f631c097ec0 2021-11-21T22:01:03.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.666+0000 7f631a7fc700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6300053580 0x7f6300055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.666+0000 7f631a7fc700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6300053580 0x7f6300055a40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f6304002f70 tx=0x7f6304015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.666+0000 7f6313fff700 1 -- 172.21.2.116:0/1798870791 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f630c042c80 con 0x7f631c097ec0 2021-11-21T22:01:03.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f631c0573d0 con 0x7f631c097ec0 2021-11-21T22:01:03.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f6313fff700 1 -- 172.21.2.116:0/1798870791 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f630c0462a0 con 0x7f631c097ec0 2021-11-21T22:01:03.755 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:03.756 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":87,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:03.446975+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:01.325473+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:02.426827+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:03.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6300053580 msgr2=0x7f6300055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6300053580 0x7f6300055a40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f6304002f70 tx=0x7f6304015000).stop 2021-11-21T22:01:03.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 msgr2=0x7f631c149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c149870 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f630c007000 tx=0x7f630c008040).stop 2021-11-21T22:01:03.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 shutdown_connections 2021-11-21T22:01:03.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f631c147300 0x7f631c0ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f631c0b5630 0x7f631c1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6300053580 0x7f6300055a40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 --2- 172.21.2.116:0/1798870791 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f631c097ec0 0x7f631c149870 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 >> 172.21.2.116:0/1798870791 conn(0x7f631c05c7f0 msgr2=0x7f631c0b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 shutdown_connections 2021-11-21T22:01:03.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.750+0000 7f631bfff700 1 -- 172.21.2.116:0/1798870791 wait complete. 2021-11-21T22:01:03.763 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:01:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.810+0000 7f445ac60700 1 Processor -- start 2021-11-21T22:01:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.810+0000 7f445ac60700 1 -- start start 2021-11-21T22:01:03.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 0x7f44540ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44540ae1c0 0x7f4454142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44541433b0 con 0x7f44540ac2f0 2021-11-21T22:01:03.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4454143110 con 0x7f44540ae1c0 2021-11-21T22:01:03.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540ada80 con 0x7f44541426c0 2021-11-21T22:01:03.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44540ae1c0 0x7f4454142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f44540ae1c0 0x7f4454142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49924/0 (socket says 172.21.2.116:49924) 2021-11-21T22:01:03.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 -- 172.21.2.116:0/1461308247 learned_addr learned my addr 172.21.2.116:0/1461308247 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 -- 172.21.2.116:0/1461308247 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 msgr2=0x7f44540ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:03.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 --2- 172.21.2.116:0/1461308247 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 0x7f44540ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 -- 172.21.2.116:0/1461308247 >> v1:172.21.2.116:6789/0 conn(0x7f44540ac2f0 legacy=0x7f44540af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44537fe700 1 -- 172.21.2.116:0/1461308247 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265969446 0 0) 0x7f44541433b0 con 0x7f44540ac2f0 2021-11-21T22:01:03.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 -- 172.21.2.116:0/1461308247 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540add20 con 0x7f44540ae1c0 2021-11-21T22:01:03.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44591fd700 1 --2- 172.21.2.116:0/1461308247 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ae1c0 0x7f4454142aa0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f444c0032f0 tx=0x7f444c009290).ready entity=mon.0 client_cookie=a1b157847d0eccd9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44537fe700 1 -- 172.21.2.116:0/1461308247 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f444c00b8a0 con 0x7f44540ae1c0 2021-11-21T22:01:03.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44537fe700 1 -- 172.21.2.116:0/1461308247 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c00ba00 con 0x7f44540ae1c0 2021-11-21T22:01:03.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44537fe700 1 -- 172.21.2.116:0/1461308247 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f444c00bb70 con 0x7f44540ae1c0 2021-11-21T22:01:03.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- 172.21.2.116:0/1461308247 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ae1c0 msgr2=0x7f4454142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- 172.21.2.116:0/1461308247 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ae1c0 0x7f4454142aa0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f444c0032f0 tx=0x7f444c009290).stop 2021-11-21T22:01:03.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- 172.21.2.116:0/1461308247 shutdown_connections 2021-11-21T22:01:03.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- 172.21.2.116:0/1461308247 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ae1c0 0x7f4454142aa0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:03.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- 172.21.2.116:0/1461308247 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 0x7f44540ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- 172.21.2.116:0/1461308247 >> 172.21.2.116:0/1461308247 conn(0x7f445405bd20 msgr2=0x7f445405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:03.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- 172.21.2.116:0/1461308247 shutdown_connections 2021-11-21T22:01:03.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- 172.21.2.116:0/1461308247 wait complete. 2021-11-21T22:01:03.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 Processor -- start 2021-11-21T22:01:03.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- start start 2021-11-21T22:01:03.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ac2f0 0x7f445413f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44540ae1c0 0x7f44540ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 0x7f44540ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454156df0 con 0x7f44540ac2f0 2021-11-21T22:01:03.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454157090 con 0x7f44540ae1c0 2021-11-21T22:01:03.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445412e8a0 con 0x7f44541426c0 2021-11-21T22:01:03.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ac2f0 0x7f445413f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ac2f0 0x7f445413f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49930/0 (socket says 172.21.2.116:49930) 2021-11-21T22:01:03.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 -- 172.21.2.116:0/573511856 learned_addr learned my addr 172.21.2.116:0/573511856 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:03.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 -- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 msgr2=0x7f44540ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 0x7f44540ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 -- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44540ae1c0 msgr2=0x7f44540ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:03.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44540ae1c0 0x7f44540ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:03.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 -- 172.21.2.116:0/573511856 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009530 con 0x7f44540ac2f0 2021-11-21T22:01:03.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f44589fc700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ac2f0 0x7f445413f880 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f443c002f70 tx=0x7f443c006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f4451ffb700 1 -- 172.21.2.116:0/573511856 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f443c00e470 con 0x7f44540ac2f0 2021-11-21T22:01:03.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f4451ffb700 1 -- 172.21.2.116:0/573511856 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c00e5d0 con 0x7f44540ac2f0 2021-11-21T22:01:03.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44540af050 con 0x7f44540ac2f0 2021-11-21T22:01:03.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f4451ffb700 1 -- 172.21.2.116:0/573511856 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f443c00e930 con 0x7f44540ac2f0 2021-11-21T22:01:03.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.814+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44540abc30 con 0x7f44540ac2f0 2021-11-21T22:01:03.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.818+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4454130760 con 0x7f44540ac2f0 2021-11-21T22:01:03.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.818+0000 7f4451ffb700 1 -- 172.21.2.116:0/573511856 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f443c018030 con 0x7f44540ac2f0 2021-11-21T22:01:03.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.818+0000 7f4451ffb700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44380536a0 0x7f4438055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:03.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.818+0000 7f4451ffb700 1 -- 172.21.2.116:0/573511856 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f443c06ae10 con 0x7f44540ac2f0 2021-11-21T22:01:03.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.818+0000 7f4453fff700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44380536a0 0x7f4438055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:03.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.818+0000 7f4453fff700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44380536a0 0x7f4438055b60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f4444000ed0 tx=0x7f4444007680).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:03.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.818+0000 7f4451ffb700 1 -- 172.21.2.116:0/573511856 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f443c042020 con 0x7f44540ac2f0 2021-11-21T22:01:03.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:03.926+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f44540573d0 con 0x7f44540ac2f0 2021-11-21T22:01:04.363 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:04.367 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:04.370 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:04.373 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:04.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.458+0000 7f4451ffb700 1 -- 172.21.2.116:0/573511856 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f443c045640 con 0x7f44540ac2f0 2021-11-21T22:01:04.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44380536a0 msgr2=0x7f4438055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44380536a0 0x7f4438055b60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f4444000ed0 tx=0x7f4444007680).stop 2021-11-21T22:01:04.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ac2f0 msgr2=0x7f445413f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ac2f0 0x7f445413f880 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f443c002f70 tx=0x7f443c006e30).stop 2021-11-21T22:01:04.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 shutdown_connections 2021-11-21T22:01:04.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44541426c0 0x7f44540ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44540ae1c0 0x7f44540ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44380536a0 0x7f4438055b60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 --2- 172.21.2.116:0/573511856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44540ac2f0 0x7f445413f880 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 >> 172.21.2.116:0/573511856 conn(0x7f445405bd20 msgr2=0x7f44540b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:04.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 shutdown_connections 2021-11-21T22:01:04.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.462+0000 7f445ac60700 1 -- 172.21.2.116:0/573511856 wait complete. 2021-11-21T22:01:04.473 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:04.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 Processor -- start 2021-11-21T22:01:04.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- start start 2021-11-21T22:01:04.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 0x7f84dc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc0b6aa0 con 0x7f84dc0b5440 2021-11-21T22:01:04.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc0a9420 con 0x7f84dc0b09c0 2021-11-21T22:01:04.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc0a96c0 con 0x7f84dc0b66c0 2021-11-21T22:01:04.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84da7fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f84dc0b5440 0x7f84dc05bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33660/0 (socket says 172.21.2.116:33660) 2021-11-21T22:01:04.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84da7fc700 1 -- 172.21.2.116:0/370909681 learned_addr learned my addr 172.21.2.116:0/370909681 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:04.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578838373 0 0) 0x7f84dc0b6aa0 con 0x7f84dc0b5440 2021-11-21T22:01:04.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c0003260 con 0x7f84dc0b5440 2021-11-21T22:01:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84daffd700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 0x7f84dc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2578900083 0 0) 0x7f84c0003260 con 0x7f84dc0b5440 2021-11-21T22:01:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 msgr2=0x7f84dc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 0x7f84dc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 >> v1:172.21.2.117:6789/0 conn(0x7f84dc0b09c0 legacy=0x7f84dc143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:04.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc0b4a20 con 0x7f84dc0b5440 2021-11-21T22:01:04.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f84d0001680 con 0x7f84dc0b5440 2021-11-21T22:01:04.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 >> v1:172.21.2.116:6789/0 conn(0x7f84dc0b5440 legacy=0x7f84dc05bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c0003780 0x7f84c0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 0x7f84c0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84c00049f0 0x7f84c0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc0a96c0 con 0x7f84c00049f0 2021-11-21T22:01:04.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c000b830 con 0x7f84c0003780 2021-11-21T22:01:04.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c000b9f0 con 0x7f84dc0b66c0 2021-11-21T22:01:04.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84c00049f0 0x7f84c0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84da7fc700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 0x7f84c0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84daffd700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c0003780 0x7f84c0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 -- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 msgr2=0x7f84c0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:04.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc0b66c0 0x7f84c0004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 -- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c0003780 msgr2=0x7f84c0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:04.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c0003780 0x7f84c0003b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 -- 172.21.2.116:0/370909681 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c0003260 con 0x7f84c00049f0 2021-11-21T22:01:04.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84c00049f0 0x7f84c0004e70 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f84d4002320 tx=0x7f84d4007d80).ready entity=mon.0 client_cookie=2782c170d4c927d8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d4006890 con 0x7f84c00049f0 2021-11-21T22:01:04.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d40069f0 con 0x7f84c00049f0 2021-11-21T22:01:04.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84d9ffb700 1 -- 172.21.2.116:0/370909681 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d4006d50 con 0x7f84c00049f0 2021-11-21T22:01:04.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- 172.21.2.116:0/370909681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84c00049f0 msgr2=0x7f84c0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 --2- 172.21.2.116:0/370909681 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84c00049f0 0x7f84c0004e70 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f84d4002320 tx=0x7f84d4007d80).stop 2021-11-21T22:01:04.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84db7fe700 1 -- 172.21.2.116:0/370909681 reap_dead start 2021-11-21T22:01:04.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- 172.21.2.116:0/370909681 shutdown_connections 2021-11-21T22:01:04.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- 172.21.2.116:0/370909681 >> 172.21.2.116:0/370909681 conn(0x7f84dc0b9c50 msgr2=0x7f84dc0b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:04.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- 172.21.2.116:0/370909681 shutdown_connections 2021-11-21T22:01:04.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.522+0000 7f84dbfff700 1 -- 172.21.2.116:0/370909681 wait complete. 2021-11-21T22:01:04.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 Processor -- start 2021-11-21T22:01:04.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 -- start start 2021-11-21T22:01:04.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c00049f0 0x7f84dc0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84dc152240 0x7f84dc155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc146510 0x7f84dc156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1459f0 con 0x7f84dc152240 2021-11-21T22:01:04.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc145c20 con 0x7f84c00049f0 2021-11-21T22:01:04.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc145ec0 con 0x7f84dc146510 2021-11-21T22:01:04.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84dc152240 0x7f84dc155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84dc152240 0x7f84dc155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49946/0 (socket says 172.21.2.116:49946) 2021-11-21T22:01:04.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 -- 172.21.2.116:0/513754082 learned_addr learned my addr 172.21.2.116:0/513754082 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:04.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 -- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc146510 msgr2=0x7f84dc156320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:04.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc146510 0x7f84dc156320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 -- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c00049f0 msgr2=0x7f84dc0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c00049f0 0x7f84dc0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 -- 172.21.2.116:0/513754082 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc155990 con 0x7f84dc152240 2021-11-21T22:01:04.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84da7fc700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84dc152240 0x7f84dc155e30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f84d0008000 tx=0x7f84d0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84cbfff700 1 -- 172.21.2.116:0/513754082 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d0007170 con 0x7f84dc152240 2021-11-21T22:01:04.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84cbfff700 1 -- 172.21.2.116:0/513754082 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d00072d0 con 0x7f84dc152240 2021-11-21T22:01:04.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84cbfff700 1 -- 172.21.2.116:0/513754082 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84d0004920 con 0x7f84dc152240 2021-11-21T22:01:04.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc144e30 con 0x7f84dc152240 2021-11-21T22:01:04.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc0aa6d0 con 0x7f84dc152240 2021-11-21T22:01:04.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc146080 con 0x7f84dc152240 2021-11-21T22:01:04.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84cbfff700 1 -- 172.21.2.116:0/513754082 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f84d0004a80 con 0x7f84dc152240 2021-11-21T22:01:04.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84cbfff700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84b8053600 0x7f84b8055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84cbfff700 1 -- 172.21.2.116:0/513754082 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f84d006ba20 con 0x7f84dc152240 2021-11-21T22:01:04.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84daffd700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84b8053600 0x7f84b8055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.526+0000 7f84daffd700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84b8053600 0x7f84b8055ac0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f84cc008000 tx=0x7f84cc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.530+0000 7f84cbfff700 1 -- 172.21.2.116:0/513754082 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84d0042a70 con 0x7f84dc152240 2021-11-21T22:01:04.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.638+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84dc0573d0 con 0x7f84dc152240 2021-11-21T22:01:04.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.638+0000 7f84cbfff700 1 -- 172.21.2.116:0/513754082 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f84d0046090 con 0x7f84dc152240 2021-11-21T22:01:04.644 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:04.645 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":38,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:03.447073+0000","modified":"2021-11-21T22:01:04.452008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2155808669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2155808669},{"type":"v1","addr":"172.21.2.116:6835","nonce":2155808669}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-11-21T22:01:04.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84b8053600 msgr2=0x7f84b8055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84b8053600 0x7f84b8055ac0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f84cc008000 tx=0x7f84cc008040).stop 2021-11-21T22:01:04.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84dc152240 msgr2=0x7f84dc155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84dc152240 0x7f84dc155e30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f84d0008000 tx=0x7f84d0009040).stop 2021-11-21T22:01:04.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 shutdown_connections 2021-11-21T22:01:04.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84dc146510 0x7f84dc156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84b8053600 0x7f84b8055ac0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84dc152240 0x7f84dc155e30 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 --2- 172.21.2.116:0/513754082 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f84c00049f0 0x7f84dc0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 >> 172.21.2.116:0/513754082 conn(0x7f84dc0b9c50 msgr2=0x7f84dc1430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:04.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 shutdown_connections 2021-11-21T22:01:04.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.642+0000 7f84dbfff700 1 -- 172.21.2.116:0/513754082 wait complete. 2021-11-21T22:01:04.650 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 38 2021-11-21T22:01:04.651 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:04.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 Processor -- start 2021-11-21T22:01:04.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 -- start start 2021-11-21T22:01:04.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a6aa0 0x7f12d814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12d80a86b0 0x7f12d81470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f12d81475d0 0x7f12d81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80a8bc0 con 0x7f12d80a86b0 2021-11-21T22:01:04.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80a6e80 con 0x7f12d81475d0 2021-11-21T22:01:04.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12df528700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d814a7b0 con 0x7f12d80a6aa0 2021-11-21T22:01:04.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12dcac3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12d80a86b0 0x7f12d81470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12dcac3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12d80a86b0 0x7f12d81470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49954/0 (socket says 172.21.2.116:49954) 2021-11-21T22:01:04.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.698+0000 7f12dcac3700 1 -- 172.21.2.116:0/3175533292 learned_addr learned my addr 172.21.2.116:0/3175533292 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:04.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12dcac3700 1 -- 172.21.2.116:0/3175533292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a6aa0 msgr2=0x7f12d814b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:04.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12dcac3700 1 --2- 172.21.2.116:0/3175533292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a6aa0 0x7f12d814b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12dcac3700 1 -- 172.21.2.116:0/3175533292 >> v2:172.21.2.117:3300/0 conn(0x7f12d81475d0 msgr2=0x7f12d81443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:04.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12dcac3700 1 --2- 172.21.2.116:0/3175533292 >> v2:172.21.2.117:3300/0 conn(0x7f12d81475d0 0x7f12d81443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12dcac3700 1 -- 172.21.2.116:0/3175533292 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d814aa10 con 0x7f12d80a86b0 2021-11-21T22:01:04.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12dcac3700 1 --2- 172.21.2.116:0/3175533292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d80a86b0 0x7f12d81470e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f12cc002f70 tx=0x7f12cc006eb0).ready entity=mon.0 client_cookie=15ad840e0ebc72f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12cbfff700 1 -- 172.21.2.116:0/3175533292 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12cc00d570 con 0x7f12d80a86b0 2021-11-21T22:01:04.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12cbfff700 1 -- 172.21.2.116:0/3175533292 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc000e90 con 0x7f12d80a86b0 2021-11-21T22:01:04.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12cbfff700 1 -- 172.21.2.116:0/3175533292 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12cc00d900 con 0x7f12d80a86b0 2021-11-21T22:01:04.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3175533292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d80a86b0 msgr2=0x7f12d81470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 --2- 172.21.2.116:0/3175533292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d80a86b0 0x7f12d81470e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f12cc002f70 tx=0x7f12cc006eb0).stop 2021-11-21T22:01:04.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3175533292 shutdown_connections 2021-11-21T22:01:04.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 --2- 172.21.2.116:0/3175533292 >> v2:172.21.2.117:3300/0 conn(0x7f12d81475d0 0x7f12d81443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 --2- 172.21.2.116:0/3175533292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d80a86b0 0x7f12d81470e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 --2- 172.21.2.116:0/3175533292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a6aa0 0x7f12d814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3175533292 >> 172.21.2.116:0/3175533292 conn(0x7f12d805bd20 msgr2=0x7f12d805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:04.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3175533292 shutdown_connections 2021-11-21T22:01:04.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3175533292 wait complete. 2021-11-21T22:01:04.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 Processor -- start 2021-11-21T22:01:04.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- start start 2021-11-21T22:01:04.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12d80a6aa0 0x7f12d80b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a86b0 0x7f12d815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d81475d0 0x7f12d815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d81458d0 con 0x7f12d81475d0 2021-11-21T22:01:04.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d815aa50 con 0x7f12d80a6aa0 2021-11-21T22:01:04.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8065d50 con 0x7f12d80a86b0 2021-11-21T22:01:04.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d81475d0 0x7f12d815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d81475d0 0x7f12d815a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49960/0 (socket says 172.21.2.116:49960) 2021-11-21T22:01:04.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 -- 172.21.2.116:0/3622077592 learned_addr learned my addr 172.21.2.116:0/3622077592 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:04.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 -- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a86b0 msgr2=0x7f12d815a070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:04.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a86b0 0x7f12d815a070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 -- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12d80a6aa0 msgr2=0x7f12d80b1c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:04.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12d80a6aa0 0x7f12d80b1c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 -- 172.21.2.116:0/3622077592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc00b1e0 con 0x7f12d81475d0 2021-11-21T22:01:04.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ddac5700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d81475d0 0x7f12d815a560 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f12d815ab90 tx=0x7f12d00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ca7fc700 1 -- 172.21.2.116:0/3622077592 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d000d4a0 con 0x7f12d81475d0 2021-11-21T22:01:04.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ca7fc700 1 -- 172.21.2.116:0/3622077592 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d000d600 con 0x7f12d81475d0 2021-11-21T22:01:04.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ca7fc700 1 -- 172.21.2.116:0/3622077592 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d000d960 con 0x7f12d81475d0 2021-11-21T22:01:04.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d8065ff0 con 0x7f12d81475d0 2021-11-21T22:01:04.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d8159bd0 con 0x7f12d81475d0 2021-11-21T22:01:04.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d80573d0 con 0x7f12d81475d0 2021-11-21T22:01:04.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ca7fc700 1 -- 172.21.2.116:0/3622077592 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f12d0018030 con 0x7f12d81475d0 2021-11-21T22:01:04.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ca7fc700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12b8053580 0x7f12b8055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.702+0000 7f12ca7fc700 1 -- 172.21.2.116:0/3622077592 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f12d006b160 con 0x7f12d81475d0 2021-11-21T22:01:04.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.706+0000 7f12dd2c4700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12b8053580 0x7f12b8055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.706+0000 7f12dd2c4700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12b8053580 0x7f12b8055a40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f12d4002fb0 tx=0x7f12d4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.706+0000 7f12ca7fc700 1 -- 172.21.2.116:0/3622077592 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12d0042370 con 0x7f12d81475d0 2021-11-21T22:01:04.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12d80b27f0 con 0x7f12d81475d0 2021-11-21T22:01:04.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12ca7fc700 1 -- 172.21.2.116:0/3622077592 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f12d0023ed0 con 0x7f12d81475d0 2021-11-21T22:01:04.797 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:04.797 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":88,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:04.450266+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:01.325473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:02.426827+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:04.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12b8053580 msgr2=0x7f12b8055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12b8053580 0x7f12b8055a40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f12d4002fb0 tx=0x7f12d4006930).stop 2021-11-21T22:01:04.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d81475d0 msgr2=0x7f12d815a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d81475d0 0x7f12d815a560 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f12d815ab90 tx=0x7f12d00090f0).stop 2021-11-21T22:01:04.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 shutdown_connections 2021-11-21T22:01:04.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12b8053580 0x7f12b8055a40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12d81475d0 0x7f12d815a560 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12d80a86b0 0x7f12d815a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 --2- 172.21.2.116:0/3622077592 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12d80a6aa0 0x7f12d80b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 >> 172.21.2.116:0/3622077592 conn(0x7f12d805bd20 msgr2=0x7f12d8148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:04.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 shutdown_connections 2021-11-21T22:01:04.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.790+0000 7f12df528700 1 -- 172.21.2.116:0/3622077592 wait complete. 2021-11-21T22:01:04.805 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:01:04.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.854+0000 7f537ab96700 1 Processor -- start 2021-11-21T22:01:04.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.854+0000 7f537ab96700 1 -- start start 2021-11-21T22:01:04.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f53740a8670 0x7f5374147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5374143e60 0x7f53741442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740b1940 con 0x7f53740b11c0 2021-11-21T22:01:04.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740b15a0 con 0x7f53740a8670 2021-11-21T22:01:04.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740b1b70 con 0x7f5374143e60 2021-11-21T22:01:04.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f53740a8670 0x7f5374147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f53740a8670 0x7f5374147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49968/0 (socket says 172.21.2.116:49968) 2021-11-21T22:01:04.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 -- 172.21.2.116:0/84765864 learned_addr learned my addr 172.21.2.116:0/84765864 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:04.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 -- 172.21.2.116:0/84765864 >> v2:172.21.2.117:3300/0 conn(0x7f5374143e60 msgr2=0x7f53741442d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:04.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 --2- 172.21.2.116:0/84765864 >> v2:172.21.2.117:3300/0 conn(0x7f5374143e60 0x7f53741442d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 -- 172.21.2.116:0/84765864 >> v1:172.21.2.117:6789/0 conn(0x7f53740b11c0 legacy=0x7f5374147230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:04.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 -- 172.21.2.116:0/84765864 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740b1e10 con 0x7f53740a8670 2021-11-21T22:01:04.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5373fff700 1 --2- 172.21.2.116:0/84765864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53740a8670 0x7f5374147940 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f5374147180 tx=0x7f5364006eb0).ready entity=mon.0 client_cookie=e577bfdd9bcf8c0a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f53737fe700 1 -- 172.21.2.116:0/84765864 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f536400d5c0 con 0x7f53740a8670 2021-11-21T22:01:04.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f53737fe700 1 -- 172.21.2.116:0/84765864 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5364000e90 con 0x7f53740a8670 2021-11-21T22:01:04.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f53737fe700 1 -- 172.21.2.116:0/84765864 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f536400d910 con 0x7f53740a8670 2021-11-21T22:01:04.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- 172.21.2.116:0/84765864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53740a8670 msgr2=0x7f5374147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- 172.21.2.116:0/84765864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53740a8670 0x7f5374147940 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f5374147180 tx=0x7f5364006eb0).stop 2021-11-21T22:01:04.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- 172.21.2.116:0/84765864 shutdown_connections 2021-11-21T22:01:04.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- 172.21.2.116:0/84765864 >> v2:172.21.2.117:3300/0 conn(0x7f5374143e60 0x7f53741442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- 172.21.2.116:0/84765864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53740a8670 0x7f5374147940 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- 172.21.2.116:0/84765864 >> 172.21.2.116:0/84765864 conn(0x7f537405bd20 msgr2=0x7f537405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:04.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- 172.21.2.116:0/84765864 shutdown_connections 2021-11-21T22:01:04.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- 172.21.2.116:0/84765864 wait complete. 2021-11-21T22:01:04.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 Processor -- start 2021-11-21T22:01:04.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- start start 2021-11-21T22:01:04.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53740a8670 0x7f5374064c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53740b11c0 0x7f5374064260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5374143e60 0x7f537414a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740b4e00 con 0x7f5374143e60 2021-11-21T22:01:04.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374149950 con 0x7f53740b11c0 2021-11-21T22:01:04.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537414ac30 con 0x7f53740a8670 2021-11-21T22:01:04.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5374143e60 0x7f537414a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5374143e60 0x7f537414a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49976/0 (socket says 172.21.2.116:49976) 2021-11-21T22:01:04.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 -- 172.21.2.116:0/2626920214 learned_addr learned my addr 172.21.2.116:0/2626920214 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:04.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 -- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53740a8670 msgr2=0x7f5374064c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53740a8670 0x7f5374064c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 -- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53740b11c0 msgr2=0x7f5374064260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53740b11c0 0x7f5374064260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 -- 172.21.2.116:0/2626920214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536400b1e0 con 0x7f5374143e60 2021-11-21T22:01:04.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5379133700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5374143e60 0x7f537414a050 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f536c003270 tx=0x7f536c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5371ffb700 1 -- 172.21.2.116:0/2626920214 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f536c00d4a0 con 0x7f5374143e60 2021-11-21T22:01:04.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5371ffb700 1 -- 172.21.2.116:0/2626920214 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c00d600 con 0x7f5374143e60 2021-11-21T22:01:04.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f5371ffb700 1 -- 172.21.2.116:0/2626920214 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f536c00d960 con 0x7f5374143e60 2021-11-21T22:01:04.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53740b5030 con 0x7f5374143e60 2021-11-21T22:01:04.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.858+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53740b3e10 con 0x7f5374143e60 2021-11-21T22:01:04.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.862+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53740b20b0 con 0x7f5374143e60 2021-11-21T22:01:04.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.862+0000 7f5371ffb700 1 -- 172.21.2.116:0/2626920214 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f536c018030 con 0x7f5374143e60 2021-11-21T22:01:04.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.862+0000 7f5371ffb700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5358053610 0x7f5358055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:04.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.862+0000 7f5371ffb700 1 -- 172.21.2.116:0/2626920214 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f536c06b300 con 0x7f5374143e60 2021-11-21T22:01:04.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.862+0000 7f5378932700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5358053610 0x7f5358055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:04.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.862+0000 7f5378932700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5358053610 0x7f5358055ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f535c002fb0 tx=0x7f535c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:04.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.862+0000 7f5371ffb700 1 -- 172.21.2.116:0/2626920214 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f536c042400 con 0x7f5374143e60 2021-11-21T22:01:04.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.974+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f53740573d0 con 0x7f5374143e60 2021-11-21T22:01:04.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f5371ffb700 1 -- 172.21.2.116:0/2626920214 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f536c045a20 con 0x7f5374143e60 2021-11-21T22:01:04.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5358053610 msgr2=0x7f5358055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5358053610 0x7f5358055ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f535c002fb0 tx=0x7f535c0069b0).stop 2021-11-21T22:01:04.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5374143e60 msgr2=0x7f537414a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:04.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5374143e60 0x7f537414a050 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f536c003270 tx=0x7f536c0090f0).stop 2021-11-21T22:01:04.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 shutdown_connections 2021-11-21T22:01:04.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5358053610 0x7f5358055ad0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5374143e60 0x7f537414a050 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:04.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53740b11c0 0x7f5374064260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 --2- 172.21.2.116:0/2626920214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53740a8670 0x7f5374064c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:04.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 >> 172.21.2.116:0/2626920214 conn(0x7f537405bd20 msgr2=0x7f5374142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:04.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 shutdown_connections 2021-11-21T22:01:04.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:04.978+0000 7f537ab96700 1 -- 172.21.2.116:0/2626920214 wait complete. 2021-11-21T22:01:04.985 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:01:04.989 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:01:05.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 Processor -- start 2021-11-21T22:01:05.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- start start 2021-11-21T22:01:05.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f66180b11c0 0x7f6618147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6618143e60 0x7f66181442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66180b1940 con 0x7f66180a8670 2021-11-21T22:01:05.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66180b15a0 con 0x7f66180b11c0 2021-11-21T22:01:05.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b1b70 con 0x7f6618143e60 2021-11-21T22:01:05.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f66180b11c0 0x7f6618147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f66180b11c0 0x7f6618147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49980/0 (socket says 172.21.2.116:49980) 2021-11-21T22:01:05.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 -- 172.21.2.116:0/2780904545 learned_addr learned my addr 172.21.2.116:0/2780904545 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:05.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 -- 172.21.2.116:0/2780904545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6618143e60 msgr2=0x7f66181442d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:05.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 --2- 172.21.2.116:0/2780904545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6618143e60 0x7f66181442d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 -- 172.21.2.116:0/2780904545 >> v1:172.21.2.116:6789/0 conn(0x7f66180a8670 legacy=0x7f6618147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 -- 172.21.2.116:0/2780904545 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180b1e10 con 0x7f66180b11c0 2021-11-21T22:01:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66177fe700 1 --2- 172.21.2.116:0/2780904545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66180b11c0 0x7f6618147230 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f6608002f70 tx=0x7f6608006eb0).ready entity=mon.0 client_cookie=78c04bb2ce543ed9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66167fc700 1 -- 172.21.2.116:0/2780904545 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f660800d5c0 con 0x7f66180b11c0 2021-11-21T22:01:05.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66167fc700 1 -- 172.21.2.116:0/2780904545 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608000e90 con 0x7f66180b11c0 2021-11-21T22:01:05.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f66167fc700 1 -- 172.21.2.116:0/2780904545 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f660800d950 con 0x7f66180b11c0 2021-11-21T22:01:05.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- 172.21.2.116:0/2780904545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66180b11c0 msgr2=0x7f6618147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- 172.21.2.116:0/2780904545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66180b11c0 0x7f6618147230 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f6608002f70 tx=0x7f6608006eb0).stop 2021-11-21T22:01:05.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- 172.21.2.116:0/2780904545 shutdown_connections 2021-11-21T22:01:05.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- 172.21.2.116:0/2780904545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6618143e60 0x7f66181442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- 172.21.2.116:0/2780904545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66180b11c0 0x7f6618147230 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- 172.21.2.116:0/2780904545 >> 172.21.2.116:0/2780904545 conn(0x7f661805bd20 msgr2=0x7f661805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:05.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- 172.21.2.116:0/2780904545 shutdown_connections 2021-11-21T22:01:05.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- 172.21.2.116:0/2780904545 wait complete. 2021-11-21T22:01:05.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 Processor -- start 2021-11-21T22:01:05.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- start start 2021-11-21T22:01:05.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66180a8670 0x7f661813ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f66180b11c0 0x7f6618064940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6618143e60 0x7f6618064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b2c90 con 0x7f6618143e60 2021-11-21T22:01:05.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b2f30 con 0x7f66180a8670 2021-11-21T22:01:05.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f661dbec700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b36f0 con 0x7f66180b11c0 2021-11-21T22:01:05.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6618143e60 0x7f6618064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6618143e60 0x7f6618064e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49990/0 (socket says 172.21.2.116:49990) 2021-11-21T22:01:05.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 -- 172.21.2.116:0/3497546762 learned_addr learned my addr 172.21.2.116:0/3497546762 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:05.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 -- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f66180b11c0 msgr2=0x7f6618064940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f66180b11c0 0x7f6618064940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 -- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66180a8670 msgr2=0x7f661813ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66180a8670 0x7f661813ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.042+0000 7f6617fff700 1 -- 172.21.2.116:0/3497546762 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660800b1e0 con 0x7f6618143e60 2021-11-21T22:01:05.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6617fff700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6618143e60 0x7f6618064e30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f6610003270 tx=0x7f66100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6614ff9700 1 -- 172.21.2.116:0/3497546762 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f661000d4a0 con 0x7f6618143e60 2021-11-21T22:01:05.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6614ff9700 1 -- 172.21.2.116:0/3497546762 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661000d600 con 0x7f6618143e60 2021-11-21T22:01:05.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6614ff9700 1 -- 172.21.2.116:0/3497546762 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f661000d960 con 0x7f6618143e60 2021-11-21T22:01:05.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66180b3b90 con 0x7f6618143e60 2021-11-21T22:01:05.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618065320 con 0x7f6618143e60 2021-11-21T22:01:05.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66180b38b0 con 0x7f6618143e60 2021-11-21T22:01:05.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6614ff9700 1 -- 172.21.2.116:0/3497546762 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6610018030 con 0x7f6618143e60 2021-11-21T22:01:05.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6614ff9700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6614ff9700 1 -- 172.21.2.116:0/3497546762 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f661006b2b0 con 0x7f6618143e60 2021-11-21T22:01:05.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f66177fe700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f66177fe700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f6608002f70 tx=0x7f6608015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.046+0000 7f6614ff9700 1 -- 172.21.2.116:0/3497546762 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f66100423b0 con 0x7f6618143e60 2021-11-21T22:01:05.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.158+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f66180573d0 con 0x7f6618143e60 2021-11-21T22:01:05.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.162+0000 7f6614ff9700 1 -- 172.21.2.116:0/3497546762 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f66100459d0 con 0x7f6618143e60 2021-11-21T22:01:05.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 msgr2=0x7f65fc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f6608002f70 tx=0x7f6608015000).stop 2021-11-21T22:01:05.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6618143e60 msgr2=0x7f6618064e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6618143e60 0x7f6618064e30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f6610003270 tx=0x7f66100090f0).stop 2021-11-21T22:01:05.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 shutdown_connections 2021-11-21T22:01:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f65fc053610 0x7f65fc055ad0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6618143e60 0x7f6618064e30 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f66180b11c0 0x7f6618064940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 --2- 172.21.2.116:0/3497546762 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66180a8670 0x7f661813ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 >> 172.21.2.116:0/3497546762 conn(0x7f661805bd20 msgr2=0x7f661814b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 shutdown_connections 2021-11-21T22:01:05.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.166+0000 7f661dbec700 1 -- 172.21.2.116:0/3497546762 wait complete. 2021-11-21T22:01:05.173 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:01:05.176 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:05.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944fbe2700 1 Processor -- start 2021-11-21T22:01:05.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944fbe2700 1 -- start start 2021-11-21T22:01:05.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944fbe2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 0x7f944805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944fbe2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f94480af7e0 0x7f944813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944fbe2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9448143f70 con 0x7f94480affc0 2021-11-21T22:01:05.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944fbe2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94480adb60 con 0x7f94480af7e0 2021-11-21T22:01:05.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944fbe2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9448143160 con 0x7f94480ad780 2021-11-21T22:01:05.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f94480af7e0 0x7f944813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f94480af7e0 0x7f944813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:49994/0 (socket says 172.21.2.116:49994) 2021-11-21T22:01:05.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 -- 172.21.2.116:0/514053733 learned_addr learned my addr 172.21.2.116:0/514053733 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:05.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 -- 172.21.2.116:0/514053733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 msgr2=0x7f944805f730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:05.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 --2- 172.21.2.116:0/514053733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 0x7f944805f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 -- 172.21.2.116:0/514053733 >> v1:172.21.2.117:6789/0 conn(0x7f94480affc0 legacy=0x7f9448143860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:05.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 -- 172.21.2.116:0/514053733 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480b0ce0 con 0x7f94480af7e0 2021-11-21T22:01:05.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.222+0000 7f944e17f700 1 --2- 172.21.2.116:0/514053733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f944813fe70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f9448068820 tx=0x7f9440009290).ready entity=mon.0 client_cookie=eeee86719fc2b887 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944c97c700 1 -- 172.21.2.116:0/514053733 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f944000b860 con 0x7f94480af7e0 2021-11-21T22:01:05.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944c97c700 1 -- 172.21.2.116:0/514053733 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944000b9c0 con 0x7f94480af7e0 2021-11-21T22:01:05.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944c97c700 1 -- 172.21.2.116:0/514053733 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f944000bb70 con 0x7f94480af7e0 2021-11-21T22:01:05.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/514053733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 msgr2=0x7f944813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 --2- 172.21.2.116:0/514053733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f944813fe70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f9448068820 tx=0x7f9440009290).stop 2021-11-21T22:01:05.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/514053733 shutdown_connections 2021-11-21T22:01:05.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 --2- 172.21.2.116:0/514053733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f944813fe70 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 --2- 172.21.2.116:0/514053733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 0x7f944805f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/514053733 >> 172.21.2.116:0/514053733 conn(0x7f94480b6000 msgr2=0x7f94480b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:05.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/514053733 shutdown_connections 2021-11-21T22:01:05.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/514053733 wait complete. 2021-11-21T22:01:05.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 Processor -- start 2021-11-21T22:01:05.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- start start 2021-11-21T22:01:05.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 0x7f94480b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f9448155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f94480affc0 0x7f9448155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94480a9eb0 con 0x7f94480af7e0 2021-11-21T22:01:05.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9448141640 con 0x7f94480affc0 2021-11-21T22:01:05.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94481418e0 con 0x7f94480ad780 2021-11-21T22:01:05.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f9448155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f9448155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50002/0 (socket says 172.21.2.116:50002) 2021-11-21T22:01:05.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 -- 172.21.2.116:0/1049088058 learned_addr learned my addr 172.21.2.116:0/1049088058 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:05.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 -- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 msgr2=0x7f94480b5200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 0x7f94480b5200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 -- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f94480affc0 msgr2=0x7f9448155b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f94480affc0 0x7f9448155b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 -- 172.21.2.116:0/1049088058 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440009530 con 0x7f94480af7e0 2021-11-21T22:01:05.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944d17d700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f9448155660 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f9444000ed0 tx=0x7f9444006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f943affd700 1 -- 172.21.2.116:0/1049088058 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9444012410 con 0x7f94480af7e0 2021-11-21T22:01:05.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f943affd700 1 -- 172.21.2.116:0/1049088058 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444012570 con 0x7f94480af7e0 2021-11-21T22:01:05.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9448141b80 con 0x7f94480af7e0 2021-11-21T22:01:05.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94480acc70 con 0x7f94480af7e0 2021-11-21T22:01:05.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f943affd700 1 -- 172.21.2.116:0/1049088058 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94440128d0 con 0x7f94480af7e0 2021-11-21T22:01:05.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94480b0a40 con 0x7f94480af7e0 2021-11-21T22:01:05.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f943affd700 1 -- 172.21.2.116:0/1049088058 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f944400f030 con 0x7f94480af7e0 2021-11-21T22:01:05.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f943affd700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94280535d0 0x7f9428055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.226+0000 7f943affd700 1 -- 172.21.2.116:0/1049088058 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f944406b610 con 0x7f94480af7e0 2021-11-21T22:01:05.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.230+0000 7f944d97e700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94280535d0 0x7f9428055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.230+0000 7f944d97e700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94280535d0 0x7f9428055a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f943c002fb0 tx=0x7f943c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.230+0000 7f943affd700 1 -- 172.21.2.116:0/1049088058 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9444042660 con 0x7f94480af7e0 2021-11-21T22:01:05.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f944805db70 con 0x7f94480af7e0 2021-11-21T22:01:05.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f943affd700 1 -- 172.21.2.116:0/1049088058 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f9444045c80 con 0x7f94480af7e0 2021-11-21T22:01:05.343 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:05.344 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":38,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:03.447073+0000","modified":"2021-11-21T22:01:04.452008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2155808669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2155808669},{"type":"v1","addr":"172.21.2.116:6835","nonce":2155808669}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-11-21T22:01:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94280535d0 msgr2=0x7f9428055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94280535d0 0x7f9428055a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f943c002fb0 tx=0x7f943c006930).stop 2021-11-21T22:01:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 msgr2=0x7f9448155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f9448155660 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f9444000ed0 tx=0x7f9444006d70).stop 2021-11-21T22:01:05.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 shutdown_connections 2021-11-21T22:01:05.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f94480affc0 0x7f9448155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f94280535d0 0x7f9428055a90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f94480af7e0 0x7f9448155660 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 --2- 172.21.2.116:0/1049088058 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f94480ad780 0x7f94480b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 >> 172.21.2.116:0/1049088058 conn(0x7f94480b6000 msgr2=0x7f94480b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:05.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 shutdown_connections 2021-11-21T22:01:05.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.338+0000 7f944fbe2700 1 -- 172.21.2.116:0/1049088058 wait complete. 2021-11-21T22:01:05.347 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 38 2021-11-21T22:01:05.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:03.447073+0000', 'modified': '2021-11-21T22:01:04.452008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6669}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6669': {'gid': 6669, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2155808669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2155808669}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2155808669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:05.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:05.653 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:05.718 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":6669,"want_state":"up:active","state":"up:active","rank_uptime":2.2640591990000001,"mdsmap_epoch":38,"osdmap_epoch":89,"osdmap_epoch_barrier":88,"uptime":6.4612996960000002} 2021-11-21T22:01:05.719 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 6669, "mdsmap_epoch": 38, "osdmap_epoch": 89, "osdmap_epoch_barrier": 88, "rank_uptime": 2.264059199, "state": "up:active", "uptime": 6.461299696, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:05.720 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:05.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c97a5700 1 Processor -- start 2021-11-21T22:01:05.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c97a5700 1 -- start start 2021-11-21T22:01:05.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c97a5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a86b0 0x7ff4c414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c97a5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff4c4062410 0x7ff4c41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c97a5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40a6e80 con 0x7ff4c40a6aa0 2021-11-21T22:01:05.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c97a5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c40a8bc0 con 0x7ff4c4062410 2021-11-21T22:01:05.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c97a5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c414a7b0 con 0x7ff4c40a86b0 2021-11-21T22:01:05.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c2ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff4c40a6aa0 0x7ff4c41470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33732/0 (socket says 172.21.2.116:33732) 2021-11-21T22:01:05.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c2ffd700 1 -- 172.21.2.116:0/3213657075 learned_addr learned my addr 172.21.2.116:0/3213657075 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:05.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.766+0000 7ff4c37fe700 1 --2- 172.21.2.116:0/3213657075 >> v2:172.21.2.116:3300/0 conn(0x7ff4c4062410 0x7ff4c41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c1ffb700 1 -- 172.21.2.116:0/3213657075 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558548268 0 0) 0x7ff4c40a6e80 con 0x7ff4c40a6aa0 2021-11-21T22:01:05.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c1ffb700 1 -- 172.21.2.116:0/3213657075 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a4003260 con 0x7ff4c40a6aa0 2021-11-21T22:01:05.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 -- 172.21.2.116:0/3213657075 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a86b0 msgr2=0x7ff4c414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:05.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 --2- 172.21.2.116:0/3213657075 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a86b0 0x7ff4c414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 -- 172.21.2.116:0/3213657075 >> v1:172.21.2.116:6789/0 conn(0x7ff4c40a6aa0 legacy=0x7ff4c41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 -- 172.21.2.116:0/3213657075 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c414aa10 con 0x7ff4c4062410 2021-11-21T22:01:05.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 --2- 172.21.2.116:0/3213657075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c4062410 0x7ff4c41443f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7ff4bc0032f0 tx=0x7ff4bc0091d0).ready entity=mon.0 client_cookie=d7aace21bede698a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c1ffb700 1 -- 172.21.2.116:0/3213657075 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc00b6c0 con 0x7ff4c4062410 2021-11-21T22:01:05.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c1ffb700 1 -- 172.21.2.116:0/3213657075 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc00b820 con 0x7ff4c4062410 2021-11-21T22:01:05.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c1ffb700 1 -- 172.21.2.116:0/3213657075 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc00b990 con 0x7ff4c4062410 2021-11-21T22:01:05.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- 172.21.2.116:0/3213657075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c4062410 msgr2=0x7ff4c41443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/3213657075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c4062410 0x7ff4c41443f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7ff4bc0032f0 tx=0x7ff4bc0091d0).stop 2021-11-21T22:01:05.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- 172.21.2.116:0/3213657075 shutdown_connections 2021-11-21T22:01:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/3213657075 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c4062410 0x7ff4c41443f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/3213657075 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a86b0 0x7ff4c414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- 172.21.2.116:0/3213657075 >> 172.21.2.116:0/3213657075 conn(0x7ff4c405bd20 msgr2=0x7ff4c405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:05.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- 172.21.2.116:0/3213657075 shutdown_connections 2021-11-21T22:01:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- 172.21.2.116:0/3213657075 wait complete. 2021-11-21T22:01:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 Processor -- start 2021-11-21T22:01:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- start start 2021-11-21T22:01:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4c4062410 0x7ff4c413f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a6aa0 0x7ff4c4139690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c40a86b0 0x7ff4c4139b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4142b20 con 0x7ff4c40a86b0 2021-11-21T22:01:05.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c413a070 con 0x7ff4c4062410 2021-11-21T22:01:05.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4141c90 con 0x7ff4c40a6aa0 2021-11-21T22:01:05.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c40a86b0 0x7ff4c4139b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c40a86b0 0x7ff4c4139b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50016/0 (socket says 172.21.2.116:50016) 2021-11-21T22:01:05.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 -- 172.21.2.116:0/1100497753 learned_addr learned my addr 172.21.2.116:0/1100497753 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:05.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 -- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a6aa0 msgr2=0x7ff4c4139690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a6aa0 0x7ff4c4139690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 -- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4c4062410 msgr2=0x7ff4c413f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4c4062410 0x7ff4c413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 -- 172.21.2.116:0/1100497753 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009470 con 0x7ff4c40a86b0 2021-11-21T22:01:05.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c37fe700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c40a86b0 0x7ff4c4139b80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7ff4c413a1b0 tx=0x7ff4bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4bbfff700 1 -- 172.21.2.116:0/1100497753 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc011e90 con 0x7ff4c40a86b0 2021-11-21T22:01:05.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4bbfff700 1 -- 172.21.2.116:0/1100497753 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc021480 con 0x7ff4c40a86b0 2021-11-21T22:01:05.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4bbfff700 1 -- 172.21.2.116:0/1100497753 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc0217e0 con 0x7ff4c40a86b0 2021-11-21T22:01:05.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c414e1f0 con 0x7ff4c40a86b0 2021-11-21T22:01:05.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.770+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c413ec10 con 0x7ff4c40a86b0 2021-11-21T22:01:05.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.774+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c4141e50 con 0x7ff4c40a86b0 2021-11-21T22:01:05.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.774+0000 7ff4bbfff700 1 -- 172.21.2.116:0/1100497753 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff4bc00f070 con 0x7ff4c40a86b0 2021-11-21T22:01:05.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.774+0000 7ff4bbfff700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4a80536a0 0x7ff4a8055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.774+0000 7ff4bbfff700 1 -- 172.21.2.116:0/1100497753 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff4bc06cc70 con 0x7ff4c40a86b0 2021-11-21T22:01:05.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.774+0000 7ff4c2ffd700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4a80536a0 0x7ff4a8055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.774+0000 7ff4c2ffd700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4a80536a0 0x7ff4a8055b60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7ff4b4001110 tx=0x7ff4b40076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.774+0000 7ff4bbfff700 1 -- 172.21.2.116:0/1100497753 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4bc043dd0 con 0x7ff4c40a86b0 2021-11-21T22:01:05.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4c40573d0 con 0x7ff4c40a86b0 2021-11-21T22:01:05.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4bbfff700 1 -- 172.21.2.116:0/1100497753 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16041 (secure 0 0 0) 0x7ff4bc015020 con 0x7ff4c40a86b0 2021-11-21T22:01:05.863 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:05.863 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":89,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:05.463699+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:01.325473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:02.426827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:05.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4a80536a0 msgr2=0x7ff4a8055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4a80536a0 0x7ff4a8055b60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7ff4b4001110 tx=0x7ff4b40076c0).stop 2021-11-21T22:01:05.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c40a86b0 msgr2=0x7ff4c4139b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c40a86b0 0x7ff4c4139b80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7ff4c413a1b0 tx=0x7ff4bc016000).stop 2021-11-21T22:01:05.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 shutdown_connections 2021-11-21T22:01:05.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff4a80536a0 0x7ff4a8055b60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4c40a86b0 0x7ff4c4139b80 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4c40a6aa0 0x7ff4c4139690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 --2- 172.21.2.116:0/1100497753 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4c4062410 0x7ff4c413f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 >> 172.21.2.116:0/1100497753 conn(0x7ff4c405bd20 msgr2=0x7ff4c4149bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:05.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 shutdown_connections 2021-11-21T22:01:05.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.858+0000 7ff4c97a5700 1 -- 172.21.2.116:0/1100497753 wait complete. 2021-11-21T22:01:05.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:01:01.325473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:01:02.426827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:01:05.871 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:05.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 Processor -- start 2021-11-21T22:01:05.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- start start 2021-11-21T22:01:05.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcb0c144810 0x7fcb0c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c147e80 con 0x7fcb0c0ac600 2021-11-21T22:01:05.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c0af5a0 con 0x7fcb0c0abce0 2021-11-21T22:01:05.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c0b1490 con 0x7fcb0c144810 2021-11-21T22:01:05.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0bfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fcb0c0ac600 0x7fcb0c0aca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33748/0 (socket says 172.21.2.116:33748) 2021-11-21T22:01:05.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0bfff700 1 -- 172.21.2.116:0/2086508462 learned_addr learned my addr 172.21.2.116:0/2086508462 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:05.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876330019 0 0) 0x7fcb0c147e80 con 0x7fcb0c0ac600 2021-11-21T22:01:05.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf8003260 con 0x7fcb0c0ac600 2021-11-21T22:01:05.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0b7fe700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcb0c144810 0x7fcb0c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 551179226 0 0) 0x7fcaf8003260 con 0x7fcb0c0ac600 2021-11-21T22:01:05.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcb0c144810 msgr2=0x7fcb0c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:05.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcb0c144810 0x7fcb0c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 >> v1:172.21.2.117:6789/0 conn(0x7fcb0c0abce0 legacy=0x7fcb0c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:05.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0b1fd0 con 0x7fcb0c0ac600 2021-11-21T22:01:05.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fcb040019e0 con 0x7fcb0c0ac600 2021-11-21T22:01:05.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 >> v1:172.21.2.116:6789/0 conn(0x7fcb0c0ac600 legacy=0x7fcb0c0aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8003800 0x7fcaf8003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaf8004500 0x7fcaf8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaf8004f90 0x7fcaf800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c0b1490 con 0x7fcaf8004500 2021-11-21T22:01:05.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf800bdd0 con 0x7fcaf8004f90 2021-11-21T22:01:05.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf800bf90 con 0x7fcaf8003800 2021-11-21T22:01:05.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0affd700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaf8004500 0x7fcaf8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0affd700 1 -- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8003800 msgr2=0x7fcaf8003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:05.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0affd700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8003800 0x7fcaf8003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0affd700 1 -- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaf8004f90 msgr2=0x7fcaf800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:05.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0affd700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaf8004f90 0x7fcaf800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:05.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0affd700 1 -- 172.21.2.116:0/2086508462 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8003260 con 0x7fcaf8004500 2021-11-21T22:01:05.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0affd700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaf8004500 0x7fcaf8004980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fcb00001fa0 tx=0x7fcb00007a00).ready entity=mon.0 client_cookie=18ca27c09cc75a39 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb00007070 con 0x7fcaf8004500 2021-11-21T22:01:05.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb000071d0 con 0x7fcaf8004500 2021-11-21T22:01:05.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0a7fc700 1 -- 172.21.2.116:0/2086508462 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb00006510 con 0x7fcaf8004500 2021-11-21T22:01:05.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- 172.21.2.116:0/2086508462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaf8004500 msgr2=0x7fcaf8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 --2- 172.21.2.116:0/2086508462 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaf8004500 0x7fcaf8004980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fcb00001fa0 tx=0x7fcb00007a00).stop 2021-11-21T22:01:05.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb0bfff700 1 -- 172.21.2.116:0/2086508462 reap_dead start 2021-11-21T22:01:05.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- 172.21.2.116:0/2086508462 shutdown_connections 2021-11-21T22:01:05.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- 172.21.2.116:0/2086508462 >> 172.21.2.116:0/2086508462 conn(0x7fcb0c05bd20 msgr2=0x7fcb0c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:05.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- 172.21.2.116:0/2086508462 shutdown_connections 2021-11-21T22:01:05.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.922+0000 7fcb11bba700 1 -- 172.21.2.116:0/2086508462 wait complete. 2021-11-21T22:01:05.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 Processor -- start 2021-11-21T22:01:05.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 -- start start 2021-11-21T22:01:05.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8004500 0x7fcb0c146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcb0c15a460 0x7fcb0c156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcb0c157000 0x7fcb0c14a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c156480 con 0x7fcb0c157000 2021-11-21T22:01:05.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c157480 con 0x7fcb0c15a460 2021-11-21T22:01:05.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c13ed80 con 0x7fcaf8004500 2021-11-21T22:01:05.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcb0c157000 0x7fcb0c14a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcb0c157000 0x7fcb0c14a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50034/0 (socket says 172.21.2.116:50034) 2021-11-21T22:01:05.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 -- 172.21.2.116:0/768999756 learned_addr learned my addr 172.21.2.116:0/768999756 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:05.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0b7fe700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8004500 0x7fcb0c146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0affd700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcb0c15a460 0x7fcb0c156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 -- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8004500 msgr2=0x7fcb0c146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8004500 0x7fcb0c146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 -- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcb0c15a460 msgr2=0x7fcb0c156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:05.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcb0c15a460 0x7fcb0c156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:05.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 -- 172.21.2.116:0/768999756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c15a8c0 con 0x7fcb0c157000 2021-11-21T22:01:05.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0bfff700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcb0c157000 0x7fcb0c14a750 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7fcb0c13aed0 tx=0x7fcb0c146e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb08ff9700 1 -- 172.21.2.116:0/768999756 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb04004ab0 con 0x7fcb0c157000 2021-11-21T22:01:05.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb08ff9700 1 -- 172.21.2.116:0/768999756 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04004c10 con 0x7fcb0c157000 2021-11-21T22:01:05.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c13efb0 con 0x7fcb0c157000 2021-11-21T22:01:05.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb08ff9700 1 -- 172.21.2.116:0/768999756 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb040070d0 con 0x7fcb0c157000 2021-11-21T22:01:05.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c156670 con 0x7fcb0c157000 2021-11-21T22:01:05.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c13f450 con 0x7fcb0c157000 2021-11-21T22:01:05.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb08ff9700 1 -- 172.21.2.116:0/768999756 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcb0401c070 con 0x7fcb0c157000 2021-11-21T22:01:05.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb08ff9700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaec053610 0x7fcaec055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:05.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb08ff9700 1 -- 172.21.2.116:0/768999756 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcb0406ada0 con 0x7fcb0c157000 2021-11-21T22:01:05.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0b7fe700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaec053610 0x7fcaec055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:05.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.926+0000 7fcb0b7fe700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaec053610 0x7fcaec055ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fcafc00c000 tx=0x7fcafc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:05.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:05.930+0000 7fcb08ff9700 1 -- 172.21.2.116:0/768999756 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb04041f00 con 0x7fcb0c157000 2021-11-21T22:01:06.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcb0c0af060 con 0x7fcb0c157000 2021-11-21T22:01:06.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb08ff9700 1 -- 172.21.2.116:0/768999756 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v89) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcb04045520 con 0x7fcb0c157000 2021-11-21T22:01:06.018 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:06.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaec053610 msgr2=0x7fcaec055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaec053610 0x7fcaec055ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fcafc00c000 tx=0x7fcafc008040).stop 2021-11-21T22:01:06.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcb0c157000 msgr2=0x7fcb0c14a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcb0c157000 0x7fcb0c14a750 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7fcb0c13aed0 tx=0x7fcb0c146e80).stop 2021-11-21T22:01:06.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 shutdown_connections 2021-11-21T22:01:06.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcaec053610 0x7fcaec055ad0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcb0c157000 0x7fcb0c14a750 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcb0c15a460 0x7fcb0c156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 --2- 172.21.2.116:0/768999756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaf8004500 0x7fcb0c146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 >> 172.21.2.116:0/768999756 conn(0x7fcb0c05bd20 msgr2=0x7fcb0c0b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 shutdown_connections 2021-11-21T22:01:06.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.014+0000 7fcb11bba700 1 -- 172.21.2.116:0/768999756 wait complete. 2021-11-21T22:01:06.024 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:06.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 Processor -- start 2021-11-21T22:01:06.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- start start 2021-11-21T22:01:06.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0abce0 0x7f4f2c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4f2c0ac600 0x7f4f2c0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c147e80 con 0x7f4f2c144810 2021-11-21T22:01:06.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2c0b1490 con 0x7f4f2c0ac600 2021-11-21T22:01:06.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c0b1fd0 con 0x7f4f2c0abce0 2021-11-21T22:01:06.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4f2c0ac600 0x7f4f2c0acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4f2c0ac600 0x7f4f2c0acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50042/0 (socket says 172.21.2.116:50042) 2021-11-21T22:01:06.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 -- 172.21.2.116:0/345510949 learned_addr learned my addr 172.21.2.116:0/345510949 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:06.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 -- 172.21.2.116:0/345510949 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0abce0 msgr2=0x7f4f2c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:06.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 --2- 172.21.2.116:0/345510949 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0abce0 0x7f4f2c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 -- 172.21.2.116:0/345510949 >> v1:172.21.2.116:6789/0 conn(0x7f4f2c144810 legacy=0x7f4f2c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 -- 172.21.2.116:0/345510949 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c0af5a0 con 0x7f4f2c0ac600 2021-11-21T22:01:06.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f31ce7700 1 --2- 172.21.2.116:0/345510949 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0ac600 0x7f4f2c0acab0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f4f240032f0 tx=0x7f4f24009290).ready entity=mon.0 client_cookie=64ae5aedd2ef80ab server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f1ffff700 1 -- 172.21.2.116:0/345510949 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f2400b8a0 con 0x7f4f2c0ac600 2021-11-21T22:01:06.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f1ffff700 1 -- 172.21.2.116:0/345510949 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2400ba00 con 0x7f4f2c0ac600 2021-11-21T22:01:06.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f1ffff700 1 -- 172.21.2.116:0/345510949 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f2400bb70 con 0x7f4f2c0ac600 2021-11-21T22:01:06.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- 172.21.2.116:0/345510949 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0ac600 msgr2=0x7f4f2c0acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- 172.21.2.116:0/345510949 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0ac600 0x7f4f2c0acab0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f4f240032f0 tx=0x7f4f24009290).stop 2021-11-21T22:01:06.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- 172.21.2.116:0/345510949 shutdown_connections 2021-11-21T22:01:06.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- 172.21.2.116:0/345510949 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0ac600 0x7f4f2c0acab0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- 172.21.2.116:0/345510949 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0abce0 0x7f4f2c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- 172.21.2.116:0/345510949 >> 172.21.2.116:0/345510949 conn(0x7f4f2c05bd20 msgr2=0x7f4f2c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- 172.21.2.116:0/345510949 shutdown_connections 2021-11-21T22:01:06.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- 172.21.2.116:0/345510949 wait complete. 2021-11-21T22:01:06.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 Processor -- start 2021-11-21T22:01:06.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- start start 2021-11-21T22:01:06.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0abce0 0x7f4f2c143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0ac600 0x7f4f2c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f2c144810 0x7f4f2c15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c145bb0 con 0x7f4f2c0abce0 2021-11-21T22:01:06.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c156b40 con 0x7f4f2c144810 2021-11-21T22:01:06.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.074+0000 7f4f3374a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c145910 con 0x7f4f2c0ac600 2021-11-21T22:01:06.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0abce0 0x7f4f2c143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0abce0 0x7f4f2c143810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50046/0 (socket says 172.21.2.116:50046) 2021-11-21T22:01:06.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 -- 172.21.2.116:0/923087366 learned_addr learned my addr 172.21.2.116:0/923087366 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:06.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 -- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0ac600 msgr2=0x7f4f2c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0ac600 0x7f4f2c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 -- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f2c144810 msgr2=0x7f4f2c15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f2c144810 0x7f4f2c15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 -- 172.21.2.116:0/923087366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f24009530 con 0x7f4f2c0abce0 2021-11-21T22:01:06.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f314e6700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0abce0 0x7f4f2c143810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f4f2c0b2a30 tx=0x7f4f20007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f1e7fc700 1 -- 172.21.2.116:0/923087366 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f20012400 con 0x7f4f2c0abce0 2021-11-21T22:01:06.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f1e7fc700 1 -- 172.21.2.116:0/923087366 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f20012560 con 0x7f4f2c0abce0 2021-11-21T22:01:06.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c159cb0 con 0x7f4f2c0abce0 2021-11-21T22:01:06.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c0643c0 con 0x7f4f2c0abce0 2021-11-21T22:01:06.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f1e7fc700 1 -- 172.21.2.116:0/923087366 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f200128c0 con 0x7f4f2c0abce0 2021-11-21T22:01:06.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c15a150 con 0x7f4f2c0abce0 2021-11-21T22:01:06.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f1e7fc700 1 -- 172.21.2.116:0/923087366 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4f20019030 con 0x7f4f2c0abce0 2021-11-21T22:01:06.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f1e7fc700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f0c0535d0 0x7f4f0c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f1e7fc700 1 -- 172.21.2.116:0/923087366 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4f2006ae60 con 0x7f4f2c0abce0 2021-11-21T22:01:06.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f30ce5700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f0c0535d0 0x7f4f0c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.078+0000 7f4f30ce5700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f0c0535d0 0x7f4f0c055a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4f28002f70 tx=0x7f4f28006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.082+0000 7f4f1e7fc700 1 -- 172.21.2.116:0/923087366 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f2c15a150 con 0x7f4f2c0abce0 2021-11-21T22:01:06.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.162+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4f2c15a150 con 0x7f4f2c0abce0 2021-11-21T22:01:06.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f1e7fc700 1 -- 172.21.2.116:0/923087366 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4f2c15a150 con 0x7f4f2c0abce0 2021-11-21T22:01:06.169 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:06.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f0c0535d0 msgr2=0x7f4f0c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f0c0535d0 0x7f4f0c055a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4f28002f70 tx=0x7f4f28006a80).stop 2021-11-21T22:01:06.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0abce0 msgr2=0x7f4f2c143810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0abce0 0x7f4f2c143810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f4f2c0b2a30 tx=0x7f4f20007c50).stop 2021-11-21T22:01:06.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 shutdown_connections 2021-11-21T22:01:06.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4f2c144810 0x7f4f2c15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4f2c0ac600 0x7f4f2c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4f0c0535d0 0x7f4f0c055a90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 --2- 172.21.2.116:0/923087366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4f2c0abce0 0x7f4f2c143810 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 >> 172.21.2.116:0/923087366 conn(0x7f4f2c05bd20 msgr2=0x7f4f2c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 shutdown_connections 2021-11-21T22:01:06.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.166+0000 7f4f3374a700 1 -- 172.21.2.116:0/923087366 wait complete. 2021-11-21T22:01:06.176 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:01:06.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 Processor -- start 2021-11-21T22:01:06.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- start start 2021-11-21T22:01:06.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a541426e0 0x7f1a54142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1a540ad470 0x7f1a540ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a540afae0 con 0x7f1a54062e60 2021-11-21T22:01:06.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a540b0f60 con 0x7f1a540ad470 2021-11-21T22:01:06.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a54063470 con 0x7f1a541426e0 2021-11-21T22:01:06.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1a540ad470 0x7f1a540ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1a540ad470 0x7f1a540ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50054/0 (socket says 172.21.2.116:50054) 2021-11-21T22:01:06.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 -- 172.21.2.116:0/1956897020 learned_addr learned my addr 172.21.2.116:0/1956897020 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:06.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 -- 172.21.2.116:0/1956897020 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a541426e0 msgr2=0x7f1a54142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:06.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 --2- 172.21.2.116:0/1956897020 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a541426e0 0x7f1a54142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 -- 172.21.2.116:0/1956897020 >> v1:172.21.2.117:6789/0 conn(0x7f1a54062e60 legacy=0x7f1a540af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:06.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 -- 172.21.2.116:0/1956897020 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54143050 con 0x7f1a540ad470 2021-11-21T22:01:06.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 --2- 172.21.2.116:0/1956897020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a540ad470 0x7f1a540ad920 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f1a541432f0 tx=0x7f1a4c00b530).ready entity=mon.0 client_cookie=2fa59943834c8c2f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a47fff700 1 -- 172.21.2.116:0/1956897020 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4c0155e0 con 0x7f1a540ad470 2021-11-21T22:01:06.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a47fff700 1 -- 172.21.2.116:0/1956897020 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c015740 con 0x7f1a540ad470 2021-11-21T22:01:06.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a47fff700 1 -- 172.21.2.116:0/1956897020 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4c015ae0 con 0x7f1a540ad470 2021-11-21T22:01:06.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1956897020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a540ad470 msgr2=0x7f1a540ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1956897020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a540ad470 0x7f1a540ad920 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f1a541432f0 tx=0x7f1a4c00b530).stop 2021-11-21T22:01:06.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1956897020 shutdown_connections 2021-11-21T22:01:06.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1956897020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a540ad470 0x7f1a540ad920 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1956897020 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a541426e0 0x7f1a54142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1956897020 >> 172.21.2.116:0/1956897020 conn(0x7f1a540b5730 msgr2=0x7f1a540b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1956897020 shutdown_connections 2021-11-21T22:01:06.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1956897020 wait complete. 2021-11-21T22:01:06.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 Processor -- start 2021-11-21T22:01:06.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- start start 2021-11-21T22:01:06.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a540ad470 0x7f1a5413de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a541426e0 0x7f1a5413e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a5412bb80 0x7f1a5412c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5413ff90 con 0x7f1a541426e0 2021-11-21T22:01:06.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5412c520 con 0x7f1a540ad470 2021-11-21T22:01:06.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a5b8e6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5413fae0 con 0x7f1a5412bb80 2021-11-21T22:01:06.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a541426e0 0x7f1a5413e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a541426e0 0x7f1a5413e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50064/0 (socket says 172.21.2.116:50064) 2021-11-21T22:01:06.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 -- 172.21.2.116:0/1389861612 learned_addr learned my addr 172.21.2.116:0/1389861612 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:06.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59e83700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a5412bb80 0x7f1a5412c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a59682700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a540ad470 0x7f1a5413de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 -- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a5412bb80 msgr2=0x7f1a5412c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a5412bb80 0x7f1a5412c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 -- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a540ad470 msgr2=0x7f1a5413de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a540ad470 0x7f1a5413de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.226+0000 7f1a58e81700 1 -- 172.21.2.116:0/1389861612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c00b7d0 con 0x7f1a541426e0 2021-11-21T22:01:06.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a58e81700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a541426e0 0x7f1a5413e310 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f1a48002f90 tx=0x7f1a48006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a467fc700 1 -- 172.21.2.116:0/1389861612 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a48013460 con 0x7f1a541426e0 2021-11-21T22:01:06.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a467fc700 1 -- 172.21.2.116:0/1389861612 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480135c0 con 0x7f1a541426e0 2021-11-21T22:01:06.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a54152c30 con 0x7f1a541426e0 2021-11-21T22:01:06.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a467fc700 1 -- 172.21.2.116:0/1389861612 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a48013920 con 0x7f1a541426e0 2021-11-21T22:01:06.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5413f050 con 0x7f1a541426e0 2021-11-21T22:01:06.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a54152a60 con 0x7f1a541426e0 2021-11-21T22:01:06.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a467fc700 1 -- 172.21.2.116:0/1389861612 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1a4801a060 con 0x7f1a541426e0 2021-11-21T22:01:06.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a467fc700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a340536d0 0x7f1a34055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a467fc700 1 -- 172.21.2.116:0/1389861612 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1a4806b220 con 0x7f1a541426e0 2021-11-21T22:01:06.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a59682700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a340536d0 0x7f1a34055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a59682700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a340536d0 0x7f1a34055b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1a5000a000 tx=0x7f1a50008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.230+0000 7f1a467fc700 1 -- 172.21.2.116:0/1389861612 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a48042380 con 0x7f1a541426e0 2021-11-21T22:01:06.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a54152dc0 con 0x7f1a541426e0 2021-11-21T22:01:06.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a467fc700 1 -- 172.21.2.116:0/1389861612 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1a480459a0 con 0x7f1a541426e0 2021-11-21T22:01:06.318 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:06.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a340536d0 msgr2=0x7f1a34055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a340536d0 0x7f1a34055b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1a5000a000 tx=0x7f1a50008040).stop 2021-11-21T22:01:06.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a541426e0 msgr2=0x7f1a5413e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a541426e0 0x7f1a5413e310 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f1a48002f90 tx=0x7f1a48006df0).stop 2021-11-21T22:01:06.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 shutdown_connections 2021-11-21T22:01:06.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a5412bb80 0x7f1a5412c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a340536d0 0x7f1a34055b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a541426e0 0x7f1a5413e310 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 --2- 172.21.2.116:0/1389861612 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a540ad470 0x7f1a5413de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 >> 172.21.2.116:0/1389861612 conn(0x7f1a540b5730 msgr2=0x7f1a5405c810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 shutdown_connections 2021-11-21T22:01:06.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.314+0000 7f1a5b8e6700 1 -- 172.21.2.116:0/1389861612 wait complete. 2021-11-21T22:01:06.325 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:01:06.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4889dcf700 1 Processor -- start 2021-11-21T22:01:06.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4889dcf700 1 -- start start 2021-11-21T22:01:06.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4889dcf700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f48840abce0 0x7f48841478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4889dcf700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4884147e80 con 0x7f48840ac600 2021-11-21T22:01:06.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4889dcf700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48840af5a0 con 0x7f4884144810 2021-11-21T22:01:06.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4889dcf700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48840b1490 con 0x7f48840abce0 2021-11-21T22:01:06.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4883fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f48840ac600 0x7f48840aca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33792/0 (socket says 172.21.2.116:33792) 2021-11-21T22:01:06.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4883fff700 1 -- 172.21.2.116:0/5865254 learned_addr learned my addr 172.21.2.116:0/5865254 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:06.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609126789 0 0) 0x7f4884147e80 con 0x7f48840ac600 2021-11-21T22:01:06.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4864003260 con 0x7f48840ac600 2021-11-21T22:01:06.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 841586695 0 0) 0x7f4864003260 con 0x7f48840ac600 2021-11-21T22:01:06.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 >> v2:172.21.2.117:3300/0 conn(0x7f48840abce0 msgr2=0x7f48841478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:06.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 --2- 172.21.2.116:0/5865254 >> v2:172.21.2.117:3300/0 conn(0x7f48840abce0 0x7f48841478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 >> v1:172.21.2.117:6789/0 conn(0x7f4884144810 legacy=0x7f48840b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:06.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840b1fd0 con 0x7f48840ac600 2021-11-21T22:01:06.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f487c0019e0 con 0x7f48840ac600 2021-11-21T22:01:06.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 >> v1:172.21.2.116:6789/0 conn(0x7f48840ac600 legacy=0x7f48840aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4864003750 0x7f4864003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4864004500 0x7f4864004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864004f90 0x7f486400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b1490 con 0x7f4864003750 2021-11-21T22:01:06.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486400be80 con 0x7f4864004f90 2021-11-21T22:01:06.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486400c040 con 0x7f4864004500 2021-11-21T22:01:06.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48837fe700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4864003750 0x7f4864003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4883fff700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864004f90 0x7f486400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4882ffd700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4864004500 0x7f4864004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48837fe700 1 -- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4864004500 msgr2=0x7f4864004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:06.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48837fe700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4864004500 0x7f4864004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48837fe700 1 -- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864004f90 msgr2=0x7f486400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:06.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48837fe700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864004f90 0x7f486400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4883fff700 1 -- 172.21.2.116:0/5865254 reap_dead start 2021-11-21T22:01:06.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48837fe700 1 -- 172.21.2.116:0/5865254 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4864003260 con 0x7f4864003750 2021-11-21T22:01:06.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f4882ffd700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4864004500 0x7f4864004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:06.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.374+0000 7f48837fe700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4864003750 0x7f4864003b50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f4874000f70 tx=0x7f48740079e0).ready entity=mon.0 client_cookie=28a103572341291f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4874007cf0 con 0x7f4864003750 2021-11-21T22:01:06.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874014410 con 0x7f4864003750 2021-11-21T22:01:06.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f48827fc700 1 -- 172.21.2.116:0/5865254 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4874014770 con 0x7f4864003750 2021-11-21T22:01:06.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/5865254 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4864003750 msgr2=0x7f4864003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4864003750 0x7f4864003b50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f4874000f70 tx=0x7f48740079e0).stop 2021-11-21T22:01:06.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/5865254 shutdown_connections 2021-11-21T22:01:06.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 --2- 172.21.2.116:0/5865254 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4864003750 0x7f4864003b50 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/5865254 >> 172.21.2.116:0/5865254 conn(0x7f488405bd20 msgr2=0x7f488405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/5865254 shutdown_connections 2021-11-21T22:01:06.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/5865254 wait complete. 2021-11-21T22:01:06.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 Processor -- start 2021-11-21T22:01:06.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- start start 2021-11-21T22:01:06.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864003750 0x7f488413f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48840b41f0 0x7f48840b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48840b3bc0 0x7f4884159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b32b0 con 0x7f48840b3bc0 2021-11-21T22:01:06.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488413c8a0 con 0x7f4864003750 2021-11-21T22:01:06.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b4040 con 0x7f48840b41f0 2021-11-21T22:01:06.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48840b3bc0 0x7f4884159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48840b3bc0 0x7f4884159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50084/0 (socket says 172.21.2.116:50084) 2021-11-21T22:01:06.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 -- 172.21.2.116:0/725801817 learned_addr learned my addr 172.21.2.116:0/725801817 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:06.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4882ffd700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48840b41f0 0x7f48840b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f48837fe700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864003750 0x7f488413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 -- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48840b41f0 msgr2=0x7f48840b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48840b41f0 0x7f48840b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 -- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864003750 msgr2=0x7f488413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864003750 0x7f488413f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 -- 172.21.2.116:0/725801817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488413cfa0 con 0x7f48840b3bc0 2021-11-21T22:01:06.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f48837fe700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864003750 0x7f488413f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:06.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4883fff700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48840b3bc0 0x7f4884159c70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f48841496c0 tx=0x7f4884146e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4880ff9700 1 -- 172.21.2.116:0/725801817 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487c0107e0 con 0x7f48840b3bc0 2021-11-21T22:01:06.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4880ff9700 1 -- 172.21.2.116:0/725801817 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c010940 con 0x7f48840b3bc0 2021-11-21T22:01:06.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488413c3a0 con 0x7f48840b3bc0 2021-11-21T22:01:06.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488414a140 con 0x7f48840b3bc0 2021-11-21T22:01:06.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4880ff9700 1 -- 172.21.2.116:0/725801817 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487c010ca0 con 0x7f48840b3bc0 2021-11-21T22:01:06.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.378+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488414a610 con 0x7f48840b3bc0 2021-11-21T22:01:06.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.382+0000 7f4880ff9700 1 -- 172.21.2.116:0/725801817 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f487c014430 con 0x7f48840b3bc0 2021-11-21T22:01:06.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.382+0000 7f4880ff9700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4868053660 0x7f4868055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.382+0000 7f4880ff9700 1 -- 172.21.2.116:0/725801817 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f487c06b920 con 0x7f48840b3bc0 2021-11-21T22:01:06.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.382+0000 7f48837fe700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4868053660 0x7f4868055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.382+0000 7f48837fe700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4868053660 0x7f4868055b20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f4874017000 tx=0x7f4874015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.382+0000 7f4880ff9700 1 -- 172.21.2.116:0/725801817 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f487c042a20 con 0x7f48840b3bc0 2021-11-21T22:01:06.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.466+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48840af060 con 0x7f48840b3bc0 2021-11-21T22:01:06.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.466+0000 7f4880ff9700 1 -- 172.21.2.116:0/725801817 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+10 (secure 0 0 0) 0x7f487c046040 con 0x7f48840b3bc0 2021-11-21T22:01:06.472 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:06.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4868053660 msgr2=0x7f4868055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4868053660 0x7f4868055b20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f4874017000 tx=0x7f4874015000).stop 2021-11-21T22:01:06.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48840b3bc0 msgr2=0x7f4884159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48840b3bc0 0x7f4884159c70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f48841496c0 tx=0x7f4884146e80).stop 2021-11-21T22:01:06.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 shutdown_connections 2021-11-21T22:01:06.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4868053660 0x7f4868055b20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48840b3bc0 0x7f4884159c70 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48840b41f0 0x7f48840b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 --2- 172.21.2.116:0/725801817 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4864003750 0x7f488413f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 >> 172.21.2.116:0/725801817 conn(0x7f488405bd20 msgr2=0x7f48840b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 shutdown_connections 2021-11-21T22:01:06.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.470+0000 7f4889dcf700 1 -- 172.21.2.116:0/725801817 wait complete. 2021-11-21T22:01:06.479 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:01:06.480 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:06.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 Processor -- start 2021-11-21T22:01:06.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- start start 2021-11-21T22:01:06.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f891c0a86b0 0x7f891c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c062410 0x7f891c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c0a6e80 con 0x7f891c0a6aa0 2021-11-21T22:01:06.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0a8bc0 con 0x7f891c0a86b0 2021-11-21T22:01:06.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c14a7b0 con 0x7f891c062410 2021-11-21T22:01:06.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f891c0a86b0 0x7f891c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f891c0a86b0 0x7f891c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50092/0 (socket says 172.21.2.116:50092) 2021-11-21T22:01:06.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/4125071302 learned_addr learned my addr 172.21.2.116:0/4125071302 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:06.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/4125071302 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c062410 msgr2=0x7f891c1443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:06.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- 172.21.2.116:0/4125071302 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c062410 0x7f891c1443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/4125071302 >> v1:172.21.2.117:6789/0 conn(0x7f891c0a6aa0 legacy=0x7f891c1470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:06.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/4125071302 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c14aa10 con 0x7f891c0a86b0 2021-11-21T22:01:06.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- 172.21.2.116:0/4125071302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a86b0 0x7f891c14b150 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f891c061ab0 tx=0x7f8904006eb0).ready entity=mon.0 client_cookie=8b64024c78182a23 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891b7fe700 1 -- 172.21.2.116:0/4125071302 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f890400d570 con 0x7f891c0a86b0 2021-11-21T22:01:06.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891b7fe700 1 -- 172.21.2.116:0/4125071302 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904000e90 con 0x7f891c0a86b0 2021-11-21T22:01:06.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891b7fe700 1 -- 172.21.2.116:0/4125071302 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f890400d8c0 con 0x7f891c0a86b0 2021-11-21T22:01:06.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- 172.21.2.116:0/4125071302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a86b0 msgr2=0x7f891c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- 172.21.2.116:0/4125071302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a86b0 0x7f891c14b150 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f891c061ab0 tx=0x7f8904006eb0).stop 2021-11-21T22:01:06.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- 172.21.2.116:0/4125071302 shutdown_connections 2021-11-21T22:01:06.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- 172.21.2.116:0/4125071302 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c062410 0x7f891c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- 172.21.2.116:0/4125071302 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a86b0 0x7f891c14b150 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- 172.21.2.116:0/4125071302 >> 172.21.2.116:0/4125071302 conn(0x7f891c05bd20 msgr2=0x7f891c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- 172.21.2.116:0/4125071302 shutdown_connections 2021-11-21T22:01:06.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- 172.21.2.116:0/4125071302 wait complete. 2021-11-21T22:01:06.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 Processor -- start 2021-11-21T22:01:06.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- start start 2021-11-21T22:01:06.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f891c062410 0x7f891c147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a6aa0 0x7f891c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c0a86b0 0x7f891c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0accd0 con 0x7f891c0a6aa0 2021-11-21T22:01:06.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0ac790 con 0x7f891c062410 2021-11-21T22:01:06.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f892300e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0aca30 con 0x7f891c0a86b0 2021-11-21T22:01:06.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a6aa0 0x7f891c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a6aa0 0x7f891c159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50096/0 (socket says 172.21.2.116:50096) 2021-11-21T22:01:06.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/2058103764 learned_addr learned my addr 172.21.2.116:0/2058103764 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:06.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c0a86b0 msgr2=0x7f891c15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c0a86b0 0x7f891c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f891c062410 msgr2=0x7f891c147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f891c062410 0x7f891c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 -- 172.21.2.116:0/2058103764 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890400b1e0 con 0x7f891c0a6aa0 2021-11-21T22:01:06.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f891bfff700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a6aa0 0x7f891c159d40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f891c061ab0 tx=0x7f8904015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f8919ffb700 1 -- 172.21.2.116:0/2058103764 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8904011990 con 0x7f891c0a6aa0 2021-11-21T22:01:06.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f8919ffb700 1 -- 172.21.2.116:0/2058103764 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904011af0 con 0x7f891c0a6aa0 2021-11-21T22:01:06.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.530+0000 7f8919ffb700 1 -- 172.21.2.116:0/2058103764 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8904011e50 con 0x7f891c0a6aa0 2021-11-21T22:01:06.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c1598a0 con 0x7f891c0a6aa0 2021-11-21T22:01:06.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c143740 con 0x7f891c0a6aa0 2021-11-21T22:01:06.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c159a30 con 0x7f891c0a6aa0 2021-11-21T22:01:06.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f8919ffb700 1 -- 172.21.2.116:0/2058103764 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8904025030 con 0x7f891c0a6aa0 2021-11-21T22:01:06.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f8919ffb700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8900053640 0x7f8900055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:06.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f8919ffb700 1 -- 172.21.2.116:0/2058103764 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f890406cdb0 con 0x7f891c0a6aa0 2021-11-21T22:01:06.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f8920daa700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8900053640 0x7f8900055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:06.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f8920daa700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8900053640 0x7f8900055b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f890c002fb0 tx=0x7f890c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:06.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.534+0000 7f8919ffb700 1 -- 172.21.2.116:0/2058103764 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8904071070 con 0x7f891c0a6aa0 2021-11-21T22:01:06.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.642+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f891c05db70 con 0x7f891c0a6aa0 2021-11-21T22:01:06.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f8919ffb700 1 -- 172.21.2.116:0/2058103764 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f8904023020 con 0x7f891c0a6aa0 2021-11-21T22:01:06.649 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:06.649 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":38,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:03.447073+0000","modified":"2021-11-21T22:01:04.452008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2155808669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2155808669},{"type":"v1","addr":"172.21.2.116:6835","nonce":2155808669}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-11-21T22:01:06.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8900053640 msgr2=0x7f8900055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8900053640 0x7f8900055b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f890c002fb0 tx=0x7f890c0069b0).stop 2021-11-21T22:01:06.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a6aa0 msgr2=0x7f891c159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:06.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a6aa0 0x7f891c159d40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f891c061ab0 tx=0x7f8904015000).stop 2021-11-21T22:01:06.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 shutdown_connections 2021-11-21T22:01:06.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f891c0a86b0 0x7f891c15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8900053640 0x7f8900055b00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f891c0a6aa0 0x7f891c159d40 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:06.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 --2- 172.21.2.116:0/2058103764 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f891c062410 0x7f891c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:06.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 >> 172.21.2.116:0/2058103764 conn(0x7f891c05bd20 msgr2=0x7f891c0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:06.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 shutdown_connections 2021-11-21T22:01:06.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:06.646+0000 7f892300e700 1 -- 172.21.2.116:0/2058103764 wait complete. 2021-11-21T22:01:06.653 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 38 2021-11-21T22:01:06.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:03.447073+0000', 'modified': '2021-11-21T22:01:04.452008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6669}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6669': {'gid': 6669, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2155808669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2155808669}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2155808669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:06.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:06.960 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:07.024 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":6669,"want_state":"up:active","state":"up:active","rank_uptime":3.5701110549999999,"mdsmap_epoch":38,"osdmap_epoch":89,"osdmap_epoch_barrier":88,"uptime":7.7673511120000001} 2021-11-21T22:01:07.024 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 6669, "mdsmap_epoch": 38, "osdmap_epoch": 89, "osdmap_epoch_barrier": 88, "rank_uptime": 3.570111055, "state": "up:active", "uptime": 7.767351112, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:07.025 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:01:07.025 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:01:07.026 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:01:07.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:07.027 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:07.038 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:07.092 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:07.092 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:07.098 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:07.099 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:07.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:07.100 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:07.104 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:01:07.105 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:07.147 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:07.156 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:07.157 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:07.162 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:07.162 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:07.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:07.164 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:07.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:01:07.185+0000 7f54b4871080 -1 init, newargv = 0x55e183a429d0 newargc=9 2021-11-21T22:01:07.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14783]: starting ceph client 2021-11-21T22:01:07.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14783]: starting fuse 2021-11-21T22:01:08.268 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:08.281 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:08.281 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:08.332 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:08.333 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:08.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:08.334 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:08.340 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:01:08.340 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:01:08.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:08.342 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:01:08.342 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:01:08.343 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:01:08.343 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:01:08.344 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:01:08.344 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:08.344 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:01:08.345 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:01:08.345 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:01:08.346 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:08.346 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:01:08.347 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:01:08.347 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:01:08.347 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:01:08.348 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:08.348 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:01:08.349 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:01:08.349 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:01:08.350 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:01:08.350 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:01:08.351 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:08.352 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:01:08.352 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:01:08.416 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14783.asok 2021-11-21T22:01:08.419 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14783.asok 2021-11-21T22:01:08.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:08.419 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14783.asok status 2021-11-21T22:01:08.468 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:01:08.468 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:01:08.469 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:01:08.469 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:01:08.469 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:01:08.469 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:01:08.470 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:01:08.470 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14783", 2021-11-21T22:01:08.470 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:01:08.470 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:08.471 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:01:08.471 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:01:08.471 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4900, 2021-11-21T22:01:08.471 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:01:08.472 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:01:08.472 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:01:08.472 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4900 2021-11-21T22:01:08.472 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:08.473 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:01:08.473 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:01:08.473 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:01:08.473 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3683508435 2021-11-21T22:01:08.473 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:01:08.474 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:08.474 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:01:08.474 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:01:08.474 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:01:08.474 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3683508435 2021-11-21T22:01:08.475 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:08.475 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4900 172.21.2.119:0/3683508435", 2021-11-21T22:01:08.475 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3683508435", 2021-11-21T22:01:08.475 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:01:08.476 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 38, 2021-11-21T22:01:08.476 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 89, 2021-11-21T22:01:08.476 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:01:08.476 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:01:08.476 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:01:08.477 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:01:08.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:08.479 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:08.479 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:08.485 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:01:08.485 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:08.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:08.486 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:08.538 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:08.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 Processor -- start 2021-11-21T22:01:08.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- start start 2021-11-21T22:01:08.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 0x7f756414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7564062410 0x7f75641443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75640a6e80 con 0x7f75640a6aa0 2021-11-21T22:01:08.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75640a8bc0 con 0x7f7564062410 2021-11-21T22:01:08.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756414a7b0 con 0x7f75640a86b0 2021-11-21T22:01:08.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7564062410 0x7f75641443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7564062410 0x7f75641443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50106/0 (socket says 172.21.2.116:50106) 2021-11-21T22:01:08.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 -- 172.21.2.116:0/1638989892 learned_addr learned my addr 172.21.2.116:0/1638989892 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:08.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 -- 172.21.2.116:0/1638989892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 msgr2=0x7f756414b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:08.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 --2- 172.21.2.116:0/1638989892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 0x7f756414b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 -- 172.21.2.116:0/1638989892 >> v1:172.21.2.117:6789/0 conn(0x7f75640a6aa0 legacy=0x7f75641470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:08.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 -- 172.21.2.116:0/1638989892 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756414aa10 con 0x7f7564062410 2021-11-21T22:01:08.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756881a700 1 --2- 172.21.2.116:0/1638989892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75641443f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f7564061ab0 tx=0x7f755c009290).ready entity=mon.0 client_cookie=98a64ba138c7e94f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f7562ffd700 1 -- 172.21.2.116:0/1638989892 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f755c00b830 con 0x7f7564062410 2021-11-21T22:01:08.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f7562ffd700 1 -- 172.21.2.116:0/1638989892 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c00b990 con 0x7f7564062410 2021-11-21T22:01:08.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f7562ffd700 1 -- 172.21.2.116:0/1638989892 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f755c00bb40 con 0x7f7564062410 2021-11-21T22:01:08.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- 172.21.2.116:0/1638989892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 msgr2=0x7f75641443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- 172.21.2.116:0/1638989892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75641443f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f7564061ab0 tx=0x7f755c009290).stop 2021-11-21T22:01:08.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- 172.21.2.116:0/1638989892 shutdown_connections 2021-11-21T22:01:08.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- 172.21.2.116:0/1638989892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75641443f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- 172.21.2.116:0/1638989892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 0x7f756414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- 172.21.2.116:0/1638989892 >> 172.21.2.116:0/1638989892 conn(0x7f756405bd20 msgr2=0x7f756405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:08.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- 172.21.2.116:0/1638989892 shutdown_connections 2021-11-21T22:01:08.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- 172.21.2.116:0/1638989892 wait complete. 2021-11-21T22:01:08.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 Processor -- start 2021-11-21T22:01:08.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- start start 2021-11-21T22:01:08.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75640a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75640a6aa0 0x7f7564063850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 0x7f7564065f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564159c10 con 0x7f7564062410 2021-11-21T22:01:08.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564159e40 con 0x7f75640a6aa0 2021-11-21T22:01:08.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f756a27d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756415a0e0 con 0x7f75640a86b0 2021-11-21T22:01:08.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f7563fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75640a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f7563fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75640a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50110/0 (socket says 172.21.2.116:50110) 2021-11-21T22:01:08.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.590+0000 7f7563fff700 1 -- 172.21.2.116:0/1901448133 learned_addr learned my addr 172.21.2.116:0/1901448133 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:08.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f7563fff700 1 -- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 msgr2=0x7f7564065f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f7563fff700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 0x7f7564065f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f7563fff700 1 -- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75640a6aa0 msgr2=0x7f7564063850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f7563fff700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75640a6aa0 0x7f7564063850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f7563fff700 1 -- 172.21.2.116:0/1901448133 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009530 con 0x7f7564062410 2021-11-21T22:01:08.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f7563fff700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75640a9ab0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f7554002fb0 tx=0x7f7554006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75617fa700 1 -- 172.21.2.116:0/1901448133 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7554013460 con 0x7f7564062410 2021-11-21T22:01:08.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75617fa700 1 -- 172.21.2.116:0/1901448133 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75540135c0 con 0x7f7564062410 2021-11-21T22:01:08.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756415a380 con 0x7f7564062410 2021-11-21T22:01:08.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75617fa700 1 -- 172.21.2.116:0/1901448133 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7554013920 con 0x7f7564062410 2021-11-21T22:01:08.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75640aa1c0 con 0x7f7564062410 2021-11-21T22:01:08.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756415a510 con 0x7f7564062410 2021-11-21T22:01:08.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75617fa700 1 -- 172.21.2.116:0/1901448133 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f755400f070 con 0x7f7564062410 2021-11-21T22:01:08.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75617fa700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75480536f0 0x7f7548055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75617fa700 1 -- 172.21.2.116:0/1901448133 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f755406b4e0 con 0x7f7564062410 2021-11-21T22:01:08.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75637fe700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75480536f0 0x7f7548055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.594+0000 7f75637fe700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75480536f0 0x7f7548055bb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f754c002ef0 tx=0x7f754c006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.598+0000 7f75617fa700 1 -- 172.21.2.116:0/1901448133 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7554042530 con 0x7f7564062410 2021-11-21T22:01:08.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.678+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75640573d0 con 0x7f7564062410 2021-11-21T22:01:08.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.678+0000 7f75617fa700 1 -- 172.21.2.116:0/1901448133 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f7554045b50 con 0x7f7564062410 2021-11-21T22:01:08.684 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:08.684 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":89,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:05.463699+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:01.325473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:02.426827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:08.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75480536f0 msgr2=0x7f7548055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75480536f0 0x7f7548055bb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f754c002ef0 tx=0x7f754c006910).stop 2021-11-21T22:01:08.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 msgr2=0x7f75640a9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75640a9ab0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f7554002fb0 tx=0x7f7554006df0).stop 2021-11-21T22:01:08.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 shutdown_connections 2021-11-21T22:01:08.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f75640a86b0 0x7f7564065f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f75640a6aa0 0x7f7564063850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75480536f0 0x7f7548055bb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 --2- 172.21.2.116:0/1901448133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7564062410 0x7f75640a9ab0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 >> 172.21.2.116:0/1901448133 conn(0x7f756405bd20 msgr2=0x7f75640b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:08.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 shutdown_connections 2021-11-21T22:01:08.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.682+0000 7f756a27d700 1 -- 172.21.2.116:0/1901448133 wait complete. 2021-11-21T22:01:08.691 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:01:01.325473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:01:02.426827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:01:08.692 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:08.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 Processor -- start 2021-11-21T22:01:08.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- start start 2021-11-21T22:01:08.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2700144810 0x7f27001478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f27000abce0 0x7f27000b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27000b1db0 0x7f27000ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2700147e80 con 0x7f2700144810 2021-11-21T22:01:08.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27000af5a0 con 0x7f27000abce0 2021-11-21T22:01:08.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27000b1490 con 0x7f27000b1db0 2021-11-21T22:01:08.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2700144810 0x7f27001478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2700144810 0x7f27001478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50116/0 (socket says 172.21.2.116:50116) 2021-11-21T22:01:08.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 -- 172.21.2.116:0/2776162405 learned_addr learned my addr 172.21.2.116:0/2776162405 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:08.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 -- 172.21.2.116:0/2776162405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27000b1db0 msgr2=0x7f27000ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:08.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 --2- 172.21.2.116:0/2776162405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27000b1db0 0x7f27000ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 -- 172.21.2.116:0/2776162405 >> v2:172.21.2.117:3300/0 conn(0x7f27000abce0 msgr2=0x7f27000b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:08.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 --2- 172.21.2.116:0/2776162405 >> v2:172.21.2.117:3300/0 conn(0x7f27000abce0 0x7f27000b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 -- 172.21.2.116:0/2776162405 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000af060 con 0x7f2700144810 2021-11-21T22:01:08.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26feffd700 1 --2- 172.21.2.116:0/2776162405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f27001478c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f26f0002f70 tx=0x7f26f0006eb0).ready entity=mon.0 client_cookie=2382e169bd2eccb8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26fdffb700 1 -- 172.21.2.116:0/2776162405 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26f000d570 con 0x7f2700144810 2021-11-21T22:01:08.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26fdffb700 1 -- 172.21.2.116:0/2776162405 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f0000e90 con 0x7f2700144810 2021-11-21T22:01:08.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f26fdffb700 1 -- 172.21.2.116:0/2776162405 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26f000d900 con 0x7f2700144810 2021-11-21T22:01:08.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- 172.21.2.116:0/2776162405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 msgr2=0x7f27001478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 --2- 172.21.2.116:0/2776162405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f27001478c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f26f0002f70 tx=0x7f26f0006eb0).stop 2021-11-21T22:01:08.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- 172.21.2.116:0/2776162405 shutdown_connections 2021-11-21T22:01:08.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 --2- 172.21.2.116:0/2776162405 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27000b1db0 0x7f27000ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 --2- 172.21.2.116:0/2776162405 >> v2:172.21.2.117:3300/0 conn(0x7f27000abce0 0x7f27000b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 --2- 172.21.2.116:0/2776162405 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f27001478c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- 172.21.2.116:0/2776162405 >> 172.21.2.116:0/2776162405 conn(0x7f270005bd20 msgr2=0x7f270005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:08.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- 172.21.2.116:0/2776162405 shutdown_connections 2021-11-21T22:01:08.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.742+0000 7f2705348700 1 -- 172.21.2.116:0/2776162405 wait complete. 2021-11-21T22:01:08.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 Processor -- start 2021-11-21T22:01:08.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 -- start start 2021-11-21T22:01:08.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27000abce0 0x7f2700140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f2700156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2700156fe0 0x7f27000b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27001564a0 con 0x7f2700144810 2021-11-21T22:01:08.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27000b4b20 con 0x7f27000abce0 2021-11-21T22:01:08.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27000b42f0 con 0x7f2700156fe0 2021-11-21T22:01:08.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f2700156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f2700156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50122/0 (socket says 172.21.2.116:50122) 2021-11-21T22:01:08.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 -- 172.21.2.116:0/4117346387 learned_addr learned my addr 172.21.2.116:0/4117346387 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:08.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 -- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2700156fe0 msgr2=0x7f27000b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2700156fe0 0x7f27000b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 -- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27000abce0 msgr2=0x7f2700140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27000abce0 0x7f2700140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 -- 172.21.2.116:0/4117346387 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f000b1e0 con 0x7f2700144810 2021-11-21T22:01:08.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26fe7fc700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f2700156af0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f26f4002f70 tx=0x7f26f4006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26effff700 1 -- 172.21.2.116:0/4117346387 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26f4013460 con 0x7f2700144810 2021-11-21T22:01:08.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26effff700 1 -- 172.21.2.116:0/4117346387 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f40135c0 con 0x7f2700144810 2021-11-21T22:01:08.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2700154150 con 0x7f2700144810 2021-11-21T22:01:08.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27000b4050 con 0x7f2700144810 2021-11-21T22:01:08.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26effff700 1 -- 172.21.2.116:0/4117346387 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26f4013920 con 0x7f2700144810 2021-11-21T22:01:08.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2700156830 con 0x7f2700144810 2021-11-21T22:01:08.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26effff700 1 -- 172.21.2.116:0/4117346387 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f26f401a060 con 0x7f2700144810 2021-11-21T22:01:08.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26effff700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26dc0536f0 0x7f26dc055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26effff700 1 -- 172.21.2.116:0/4117346387 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f26f406b530 con 0x7f2700144810 2021-11-21T22:01:08.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.746+0000 7f26feffd700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26dc0536f0 0x7f26dc055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.750+0000 7f26feffd700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26dc0536f0 0x7f26dc055bb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f26f0002f70 tx=0x7f26f0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.750+0000 7f26effff700 1 -- 172.21.2.116:0/4117346387 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2700156830 con 0x7f2700144810 2021-11-21T22:01:08.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.830+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2700156830 con 0x7f2700144810 2021-11-21T22:01:08.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.830+0000 7f26effff700 1 -- 172.21.2.116:0/4117346387 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v89) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2700156830 con 0x7f2700144810 2021-11-21T22:01:08.837 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:08.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26dc0536f0 msgr2=0x7f26dc055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26dc0536f0 0x7f26dc055bb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f26f0002f70 tx=0x7f26f0015000).stop 2021-11-21T22:01:08.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 msgr2=0x7f2700156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f2700156af0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f26f4002f70 tx=0x7f26f4006ec0).stop 2021-11-21T22:01:08.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 shutdown_connections 2021-11-21T22:01:08.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2700156fe0 0x7f27000b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26dc0536f0 0x7f26dc055bb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2700144810 0x7f2700156af0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 --2- 172.21.2.116:0/4117346387 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27000abce0 0x7f2700140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 >> 172.21.2.116:0/4117346387 conn(0x7f270005bd20 msgr2=0x7f2700141860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:08.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 shutdown_connections 2021-11-21T22:01:08.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.834+0000 7f2705348700 1 -- 172.21.2.116:0/4117346387 wait complete. 2021-11-21T22:01:08.845 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:08.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 Processor -- start 2021-11-21T22:01:08.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- start start 2021-11-21T22:01:08.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144810 0x7f55cc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55cc0abce0 0x7f55cc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f55cc0b1db0 0x7f55cc0ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc147e80 con 0x7f55cc0abce0 2021-11-21T22:01:08.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc0af5a0 con 0x7f55cc0b1db0 2021-11-21T22:01:08.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0b1490 con 0x7f55cc144810 2021-11-21T22:01:08.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55cc0abce0 0x7f55cc0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55cc0abce0 0x7f55cc0b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50134/0 (socket says 172.21.2.116:50134) 2021-11-21T22:01:08.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 -- 172.21.2.116:0/3363757919 learned_addr learned my addr 172.21.2.116:0/3363757919 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:08.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 -- 172.21.2.116:0/3363757919 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144810 msgr2=0x7f55cc1478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:08.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 --2- 172.21.2.116:0/3363757919 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144810 0x7f55cc1478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 -- 172.21.2.116:0/3363757919 >> v2:172.21.2.117:3300/0 conn(0x7f55cc0b1db0 msgr2=0x7f55cc0ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:08.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 --2- 172.21.2.116:0/3363757919 >> v2:172.21.2.117:3300/0 conn(0x7f55cc0b1db0 0x7f55cc0ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 -- 172.21.2.116:0/3363757919 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0af060 con 0x7f55cc0abce0 2021-11-21T22:01:08.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d112e700 1 --2- 172.21.2.116:0/3363757919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc0b18c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f55c8002f70 tx=0x7f55c8006eb0).ready entity=mon.0 client_cookie=a943cb749eb827fd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d092d700 1 -- 172.21.2.116:0/3363757919 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c800d570 con 0x7f55cc0abce0 2021-11-21T22:01:08.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d092d700 1 -- 172.21.2.116:0/3363757919 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8000e90 con 0x7f55cc0abce0 2021-11-21T22:01:08.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d092d700 1 -- 172.21.2.116:0/3363757919 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c800d900 con 0x7f55cc0abce0 2021-11-21T22:01:08.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- 172.21.2.116:0/3363757919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 msgr2=0x7f55cc0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 --2- 172.21.2.116:0/3363757919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc0b18c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f55c8002f70 tx=0x7f55c8006eb0).stop 2021-11-21T22:01:08.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- 172.21.2.116:0/3363757919 shutdown_connections 2021-11-21T22:01:08.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 --2- 172.21.2.116:0/3363757919 >> v2:172.21.2.117:3300/0 conn(0x7f55cc0b1db0 0x7f55cc0ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 --2- 172.21.2.116:0/3363757919 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc0b18c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 --2- 172.21.2.116:0/3363757919 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144810 0x7f55cc1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- 172.21.2.116:0/3363757919 >> 172.21.2.116:0/3363757919 conn(0x7f55cc05bd20 msgr2=0x7f55cc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:08.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- 172.21.2.116:0/3363757919 shutdown_connections 2021-11-21T22:01:08.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.894+0000 7f55d3b93700 1 -- 172.21.2.116:0/3363757919 wait complete. 2021-11-21T22:01:08.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 Processor -- start 2021-11-21T22:01:08.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 -- start start 2021-11-21T22:01:08.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55cc144810 0x7f55cc1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144f30 0x7f55cc146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0b48e0 con 0x7f55cc0abce0 2021-11-21T22:01:08.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0b4b80 con 0x7f55cc144810 2021-11-21T22:01:08.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc145ec0 con 0x7f55cc144f30 2021-11-21T22:01:08.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc14a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50136/0 (socket says 172.21.2.116:50136) 2021-11-21T22:01:08.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 -- 172.21.2.116:0/4043394375 learned_addr learned my addr 172.21.2.116:0/4043394375 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:08.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 -- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144f30 msgr2=0x7f55cc146600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:08.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144f30 0x7f55cc146600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 -- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55cc144810 msgr2=0x7f55cc1566b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:08.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55cc144810 0x7f55cc1566b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 -- 172.21.2.116:0/4043394375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c800b1e0 con 0x7f55cc0abce0 2021-11-21T22:01:08.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d192f700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc14a780 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f55bc000ed0 tx=0x7f55bc006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55baffd700 1 -- 172.21.2.116:0/4043394375 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55bc012410 con 0x7f55cc0abce0 2021-11-21T22:01:08.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55baffd700 1 -- 172.21.2.116:0/4043394375 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc012570 con 0x7f55cc0abce0 2021-11-21T22:01:08.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc146160 con 0x7f55cc0abce0 2021-11-21T22:01:08.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc145b30 con 0x7f55cc0abce0 2021-11-21T22:01:08.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55baffd700 1 -- 172.21.2.116:0/4043394375 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55bc0128d0 con 0x7f55cc0abce0 2021-11-21T22:01:08.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc0af300 con 0x7f55cc0abce0 2021-11-21T22:01:08.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55baffd700 1 -- 172.21.2.116:0/4043394375 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f55bc00f030 con 0x7f55cc0abce0 2021-11-21T22:01:08.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55baffd700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55ac053720 0x7f55ac055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:08.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55baffd700 1 -- 172.21.2.116:0/4043394375 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f55bc06b4f0 con 0x7f55cc0abce0 2021-11-21T22:01:08.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.898+0000 7f55d112e700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55ac053720 0x7f55ac055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:08.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.902+0000 7f55d112e700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55ac053720 0x7f55ac055be0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f55c8002f70 tx=0x7f55c8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:08.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.902+0000 7f55baffd700 1 -- 172.21.2.116:0/4043394375 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55bc042650 con 0x7f55cc0abce0 2021-11-21T22:01:08.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.982+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f55cc0ad630 con 0x7f55cc0abce0 2021-11-21T22:01:08.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.982+0000 7f55baffd700 1 -- 172.21.2.116:0/4043394375 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7f55bc045c70 con 0x7f55cc0abce0 2021-11-21T22:01:08.988 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:08.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55ac053720 msgr2=0x7f55ac055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55ac053720 0x7f55ac055be0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f55c8002f70 tx=0x7f55c8015000).stop 2021-11-21T22:01:08.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 msgr2=0x7f55cc14a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:08.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc14a780 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f55bc000ed0 tx=0x7f55bc006d70).stop 2021-11-21T22:01:08.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 shutdown_connections 2021-11-21T22:01:08.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55cc144f30 0x7f55cc146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55cc144810 0x7f55cc1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:08.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55ac053720 0x7f55ac055be0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 --2- 172.21.2.116:0/4043394375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55cc0abce0 0x7f55cc14a780 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:08.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 >> 172.21.2.116:0/4043394375 conn(0x7f55cc05bd20 msgr2=0x7f55cc0acce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:08.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 shutdown_connections 2021-11-21T22:01:08.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:08.986+0000 7f55d3b93700 1 -- 172.21.2.116:0/4043394375 wait complete. 2021-11-21T22:01:08.995 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:01:09.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.042+0000 7f7a723b5700 1 Processor -- start 2021-11-21T22:01:09.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.042+0000 7f7a723b5700 1 -- start start 2021-11-21T22:01:09.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.042+0000 7f7a723b5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0abce0 0x7f7a6c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.042+0000 7f7a723b5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a6c0ac600 0x7f7a6c0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.042+0000 7f7a723b5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c147e80 con 0x7f7a6c144810 2021-11-21T22:01:09.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.042+0000 7f7a723b5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6c0b1490 con 0x7f7a6c0ac600 2021-11-21T22:01:09.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.042+0000 7f7a723b5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0b1fd0 con 0x7f7a6c0abce0 2021-11-21T22:01:09.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a6c0ac600 0x7f7a6c0acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a6c0ac600 0x7f7a6c0acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50146/0 (socket says 172.21.2.116:50146) 2021-11-21T22:01:09.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 -- 172.21.2.116:0/3615883004 learned_addr learned my addr 172.21.2.116:0/3615883004 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:09.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 -- 172.21.2.116:0/3615883004 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0abce0 msgr2=0x7f7a6c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:09.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 --2- 172.21.2.116:0/3615883004 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0abce0 0x7f7a6c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 -- 172.21.2.116:0/3615883004 >> v1:172.21.2.117:6789/0 conn(0x7f7a6c144810 legacy=0x7f7a6c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:09.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 -- 172.21.2.116:0/3615883004 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0af5a0 con 0x7f7a6c0ac600 2021-11-21T22:01:09.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a70952700 1 --2- 172.21.2.116:0/3615883004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0ac600 0x7f7a6c0acab0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f7a6c0b2a30 tx=0x7f7a64009290).ready entity=mon.0 client_cookie=a49e61aca1c63769 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6affd700 1 -- 172.21.2.116:0/3615883004 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6400b860 con 0x7f7a6c0ac600 2021-11-21T22:01:09.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6affd700 1 -- 172.21.2.116:0/3615883004 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6400b9c0 con 0x7f7a6c0ac600 2021-11-21T22:01:09.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6affd700 1 -- 172.21.2.116:0/3615883004 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6400bb70 con 0x7f7a6c0ac600 2021-11-21T22:01:09.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- 172.21.2.116:0/3615883004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0ac600 msgr2=0x7f7a6c0acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 --2- 172.21.2.116:0/3615883004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0ac600 0x7f7a6c0acab0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f7a6c0b2a30 tx=0x7f7a64009290).stop 2021-11-21T22:01:09.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- 172.21.2.116:0/3615883004 shutdown_connections 2021-11-21T22:01:09.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 --2- 172.21.2.116:0/3615883004 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0ac600 0x7f7a6c0acab0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 --2- 172.21.2.116:0/3615883004 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0abce0 0x7f7a6c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- 172.21.2.116:0/3615883004 >> 172.21.2.116:0/3615883004 conn(0x7f7a6c05bd20 msgr2=0x7f7a6c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:09.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- 172.21.2.116:0/3615883004 shutdown_connections 2021-11-21T22:01:09.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- 172.21.2.116:0/3615883004 wait complete. 2021-11-21T22:01:09.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 Processor -- start 2021-11-21T22:01:09.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- start start 2021-11-21T22:01:09.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0abce0 0x7f7a6c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0ac600 0x7f7a6c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a6c144810 0x7f7a6c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c145b50 con 0x7f7a6c0abce0 2021-11-21T22:01:09.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c1471c0 con 0x7f7a6c144810 2021-11-21T22:01:09.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c147460 con 0x7f7a6c0ac600 2021-11-21T22:01:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0abce0 0x7f7a6c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0abce0 0x7f7a6c156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50154/0 (socket says 172.21.2.116:50154) 2021-11-21T22:01:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 -- 172.21.2.116:0/503677973 learned_addr learned my addr 172.21.2.116:0/503677973 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:09.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 -- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0ac600 msgr2=0x7f7a6c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0ac600 0x7f7a6c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 -- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a6c144810 msgr2=0x7f7a6c0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a6c144810 0x7f7a6c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 -- 172.21.2.116:0/503677973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64009530 con 0x7f7a6c0abce0 2021-11-21T22:01:09.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a6bfff700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0abce0 0x7f7a6c156ba0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f7a54002f10 tx=0x7f7a54006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a697fa700 1 -- 172.21.2.116:0/503677973 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5400f510 con 0x7f7a6c0abce0 2021-11-21T22:01:09.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a697fa700 1 -- 172.21.2.116:0/503677973 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5400f670 con 0x7f7a6c0abce0 2021-11-21T22:01:09.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a697fa700 1 -- 172.21.2.116:0/503677973 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5400f7e0 con 0x7f7a6c0abce0 2021-11-21T22:01:09.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c145840 con 0x7f7a6c0abce0 2021-11-21T22:01:09.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.046+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c144f40 con 0x7f7a6c0abce0 2021-11-21T22:01:09.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.050+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c145d50 con 0x7f7a6c0abce0 2021-11-21T22:01:09.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.050+0000 7f7a697fa700 1 -- 172.21.2.116:0/503677973 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7a54017020 con 0x7f7a6c0abce0 2021-11-21T22:01:09.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.050+0000 7f7a697fa700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a50053540 0x7f7a50055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.050+0000 7f7a697fa700 1 -- 172.21.2.116:0/503677973 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7a5406afe0 con 0x7f7a6c0abce0 2021-11-21T22:01:09.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.050+0000 7f7a6b7fe700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a50053540 0x7f7a50055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.050+0000 7f7a6b7fe700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a50053540 0x7f7a50055a00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f7a5c002f70 tx=0x7f7a5c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.050+0000 7f7a697fa700 1 -- 172.21.2.116:0/503677973 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a54042050 con 0x7f7a6c0abce0 2021-11-21T22:01:09.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a6c144f40 con 0x7f7a6c0abce0 2021-11-21T22:01:09.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a697fa700 1 -- 172.21.2.116:0/503677973 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7a6c144f40 con 0x7f7a6c0abce0 2021-11-21T22:01:09.140 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:09.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a50053540 msgr2=0x7f7a50055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a50053540 0x7f7a50055a00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f7a5c002f70 tx=0x7f7a5c006a80).stop 2021-11-21T22:01:09.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0abce0 msgr2=0x7f7a6c156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0abce0 0x7f7a6c156ba0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f7a54002f10 tx=0x7f7a54006d70).stop 2021-11-21T22:01:09.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 shutdown_connections 2021-11-21T22:01:09.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a6c144810 0x7f7a6c0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6c0ac600 0x7f7a6c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a50053540 0x7f7a50055a00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 --2- 172.21.2.116:0/503677973 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a6c0abce0 0x7f7a6c156ba0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 >> 172.21.2.116:0/503677973 conn(0x7f7a6c05bd20 msgr2=0x7f7a6c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:09.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 shutdown_connections 2021-11-21T22:01:09.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.134+0000 7f7a723b5700 1 -- 172.21.2.116:0/503677973 wait complete. 2021-11-21T22:01:09.147 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:01:09.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.194+0000 7f538ff1d700 1 Processor -- start 2021-11-21T22:01:09.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.194+0000 7f538ff1d700 1 -- start start 2021-11-21T22:01:09.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.194+0000 7f538ff1d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f53881426e0 0x7f5388142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.194+0000 7f538ff1d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53880ad470 0x7f53880ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.194+0000 7f538ff1d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880afae0 con 0x7f5388062e60 2021-11-21T22:01:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.194+0000 7f538ff1d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53880b0f60 con 0x7f53881426e0 2021-11-21T22:01:09.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.194+0000 7f538ff1d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5388063470 con 0x7f53880ad470 2021-11-21T22:01:09.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53880ad470 0x7f53880ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53880ad470 0x7f53880ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:53504/0 (socket says 172.21.2.116:53504) 2021-11-21T22:01:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 -- 172.21.2.116:0/358902894 learned_addr learned my addr 172.21.2.116:0/358902894 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538d4b8700 1 --1- 172.21.2.116:0/358902894 >> v1:172.21.2.117:6789/0 conn(0x7f5388062e60 0x7f53880af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:32796/0 (socket says 172.21.2.116:32796) 2021-11-21T22:01:09.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538dcb9700 1 --2- 172.21.2.116:0/358902894 >> v2:172.21.2.117:3300/0 conn(0x7f53881426e0 0x7f5388142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 -- 172.21.2.116:0/358902894 >> v2:172.21.2.117:3300/0 conn(0x7f53881426e0 msgr2=0x7f5388142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 --2- 172.21.2.116:0/358902894 >> v2:172.21.2.117:3300/0 conn(0x7f53881426e0 0x7f5388142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 -- 172.21.2.116:0/358902894 >> v1:172.21.2.117:6789/0 conn(0x7f5388062e60 legacy=0x7f53880af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 -- 172.21.2.116:0/358902894 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388143050 con 0x7f53880ad470 2021-11-21T22:01:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538e4ba700 1 --2- 172.21.2.116:0/358902894 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53880ad470 0x7f53880ad900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f53800032f0 tx=0x7f5380009290).ready entity=mon.2 client_cookie=a3bb2e101cb2430 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ccb7700 1 -- 172.21.2.116:0/358902894 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538000b8a0 con 0x7f53880ad470 2021-11-21T22:01:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ccb7700 1 -- 172.21.2.116:0/358902894 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538000ba00 con 0x7f53880ad470 2021-11-21T22:01:09.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ccb7700 1 -- 172.21.2.116:0/358902894 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538000bb70 con 0x7f53880ad470 2021-11-21T22:01:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 -- 172.21.2.116:0/358902894 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53880ad470 msgr2=0x7f53880ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 --2- 172.21.2.116:0/358902894 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53880ad470 0x7f53880ad900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f53800032f0 tx=0x7f5380009290).stop 2021-11-21T22:01:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 -- 172.21.2.116:0/358902894 shutdown_connections 2021-11-21T22:01:09.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 --2- 172.21.2.116:0/358902894 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53880ad470 0x7f53880ad900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 --2- 172.21.2.116:0/358902894 >> v2:172.21.2.117:3300/0 conn(0x7f53881426e0 0x7f5388142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 -- 172.21.2.116:0/358902894 >> 172.21.2.116:0/358902894 conn(0x7f53880b5730 msgr2=0x7f53880b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 -- 172.21.2.116:0/358902894 shutdown_connections 2021-11-21T22:01:09.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 -- 172.21.2.116:0/358902894 wait complete. 2021-11-21T22:01:09.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 Processor -- start 2021-11-21T22:01:09.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.198+0000 7f538ff1d700 1 -- start start 2021-11-21T22:01:09.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53880ad470 0x7f538811efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53881426e0 0x7f5388141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53881417f0 0x7f538813ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388063ce0 con 0x7f53880ad470 2021-11-21T22:01:09.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538813e730 con 0x7f53881417f0 2021-11-21T22:01:09.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538813fbf0 con 0x7f53881426e0 2021-11-21T22:01:09.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53880ad470 0x7f538811efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53880ad470 0x7f538811efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50166/0 (socket says 172.21.2.116:50166) 2021-11-21T22:01:09.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 -- 172.21.2.116:0/2746900772 learned_addr learned my addr 172.21.2.116:0/2746900772 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:09.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 -- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53881426e0 msgr2=0x7f5388141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53881426e0 0x7f5388141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 -- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53881417f0 msgr2=0x7f538813ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53881417f0 0x7f538813ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 -- 172.21.2.116:0/2746900772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5380009530 con 0x7f53880ad470 2021-11-21T22:01:09.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538dcb9700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53880ad470 0x7f538811efe0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f5388141c80 tx=0x7f5384006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f5376ffd700 1 -- 172.21.2.116:0/2746900772 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5384010e60 con 0x7f53880ad470 2021-11-21T22:01:09.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f5376ffd700 1 -- 172.21.2.116:0/2746900772 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538400e870 con 0x7f53880ad470 2021-11-21T22:01:09.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538813fe90 con 0x7f53880ad470 2021-11-21T22:01:09.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538813d300 con 0x7f53880ad470 2021-11-21T22:01:09.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f5376ffd700 1 -- 172.21.2.116:0/2746900772 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538400ebd0 con 0x7f53880ad470 2021-11-21T22:01:09.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53880b1e00 con 0x7f53880ad470 2021-11-21T22:01:09.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f5376ffd700 1 -- 172.21.2.116:0/2746900772 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5384024070 con 0x7f53880ad470 2021-11-21T22:01:09.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f5376ffd700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5368053600 0x7f5368055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f5376ffd700 1 -- 172.21.2.116:0/2746900772 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f538406de70 con 0x7f53880ad470 2021-11-21T22:01:09.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538d4b8700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5368053600 0x7f5368055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.202+0000 7f538d4b8700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5368053600 0x7f5368055ac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f5388062d10 tx=0x7f5378007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.206+0000 7f5376ffd700 1 -- 172.21.2.116:0/2746900772 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5384048e60 con 0x7f53880ad470 2021-11-21T22:01:09.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.286+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53880573d0 con 0x7f53880ad470 2021-11-21T22:01:09.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.286+0000 7f5376ffd700 1 -- 172.21.2.116:0/2746900772 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5384044f70 con 0x7f53880ad470 2021-11-21T22:01:09.292 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:09.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5368053600 msgr2=0x7f5368055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5368053600 0x7f5368055ac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f5388062d10 tx=0x7f5378007800).stop 2021-11-21T22:01:09.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53880ad470 msgr2=0x7f538811efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53880ad470 0x7f538811efe0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f5388141c80 tx=0x7f5384006df0).stop 2021-11-21T22:01:09.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 shutdown_connections 2021-11-21T22:01:09.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53881417f0 0x7f538813ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53881426e0 0x7f5388141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5368053600 0x7f5368055ac0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 --2- 172.21.2.116:0/2746900772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53880ad470 0x7f538811efe0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 >> 172.21.2.116:0/2746900772 conn(0x7f53880b5730 msgr2=0x7f53880ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:09.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 shutdown_connections 2021-11-21T22:01:09.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.290+0000 7f538ff1d700 1 -- 172.21.2.116:0/2746900772 wait complete. 2021-11-21T22:01:09.299 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:01:09.300 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:09.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f8b41700 1 Processor -- start 2021-11-21T22:01:09.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f8b41700 1 -- start start 2021-11-21T22:01:09.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f8b41700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f83f40a6aa0 0x7f83f414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f8b41700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40a6e80 con 0x7f83f40a86b0 2021-11-21T22:01:09.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f8b41700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f414a7b0 con 0x7f83f4062410 2021-11-21T22:01:09.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f8b41700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f40a8bc0 con 0x7f83f40a6aa0 2021-11-21T22:01:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f83f40a6aa0 0x7f83f414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f1d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f83f40a86b0 0x7f83f41470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33898/0 (socket says 172.21.2.116:33898) 2021-11-21T22:01:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f1d9b700 1 -- 172.21.2.116:0/1247534579 learned_addr learned my addr 172.21.2.116:0/1247534579 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:09.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f259c700 1 -- 172.21.2.116:0/1247534579 >> v1:172.21.2.117:6789/0 conn(0x7f83f4062410 legacy=0x7f83f41443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f259c700 1 -- 172.21.2.116:0/1247534579 >> v1:172.21.2.116:6789/0 conn(0x7f83f40a86b0 legacy=0x7f83f41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f259c700 1 -- 172.21.2.116:0/1247534579 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f414aa10 con 0x7f83f40a6aa0 2021-11-21T22:01:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f259c700 1 --2- 172.21.2.116:0/1247534579 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a6aa0 0x7f83f414b150 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f83f4061b50 tx=0x7f83dc006df0).ready entity=mon.0 client_cookie=d03731a2ee5e282f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f159a700 1 -- 172.21.2.116:0/1247534579 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83dc00d5c0 con 0x7f83f40a6aa0 2021-11-21T22:01:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f159a700 1 -- 172.21.2.116:0/1247534579 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc000e90 con 0x7f83f40a6aa0 2021-11-21T22:01:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.346+0000 7f83f159a700 1 -- 172.21.2.116:0/1247534579 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83dc00d910 con 0x7f83f40a6aa0 2021-11-21T22:01:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/1247534579 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a6aa0 msgr2=0x7f83f414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 --2- 172.21.2.116:0/1247534579 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a6aa0 0x7f83f414b150 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f83f4061b50 tx=0x7f83dc006df0).stop 2021-11-21T22:01:09.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/1247534579 shutdown_connections 2021-11-21T22:01:09.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 --2- 172.21.2.116:0/1247534579 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a6aa0 0x7f83f414b150 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/1247534579 >> 172.21.2.116:0/1247534579 conn(0x7f83f405bd20 msgr2=0x7f83f405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:09.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/1247534579 shutdown_connections 2021-11-21T22:01:09.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/1247534579 wait complete. 2021-11-21T22:01:09.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 Processor -- start 2021-11-21T22:01:09.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- start start 2021-11-21T22:01:09.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83f4062410 0x7f83f4156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83f40a6aa0 0x7f83f4142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a86b0 0x7f83f40638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40aa850 con 0x7f83f40a86b0 2021-11-21T22:01:09.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40aaa30 con 0x7f83f4062410 2021-11-21T22:01:09.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40aacd0 con 0x7f83f40a6aa0 2021-11-21T22:01:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a86b0 0x7f83f40638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a86b0 0x7f83f40638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50178/0 (socket says 172.21.2.116:50178) 2021-11-21T22:01:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 -- 172.21.2.116:0/2010615128 learned_addr learned my addr 172.21.2.116:0/2010615128 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 -- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83f40a6aa0 msgr2=0x7f83f4142160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:09.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83f40a6aa0 0x7f83f4142160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 -- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83f4062410 msgr2=0x7f83f4156330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:09.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83f4062410 0x7f83f4156330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 -- 172.21.2.116:0/2010615128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc00b180 con 0x7f83f40a86b0 2021-11-21T22:01:09.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f2d9d700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a86b0 0x7f83f40638c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f83ec002ef0 tx=0x7f83ec006cd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83eb7fe700 1 -- 172.21.2.116:0/2010615128 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ec009e00 con 0x7f83f40a86b0 2021-11-21T22:01:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83eb7fe700 1 -- 172.21.2.116:0/2010615128 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec002320 con 0x7f83f40a86b0 2021-11-21T22:01:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83eb7fe700 1 -- 172.21.2.116:0/2010615128 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ec002680 con 0x7f83f40a86b0 2021-11-21T22:01:09.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f4066760 con 0x7f83f40a86b0 2021-11-21T22:01:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f40ab2b0 con 0x7f83f40a86b0 2021-11-21T22:01:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f40573d0 con 0x7f83f40a86b0 2021-11-21T22:01:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83eb7fe700 1 -- 172.21.2.116:0/2010615128 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f83ec00f030 con 0x7f83f40a86b0 2021-11-21T22:01:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83eb7fe700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83d80534f0 0x7f83d80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:09.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.350+0000 7f83eb7fe700 1 -- 172.21.2.116:0/2010615128 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f83ec06b5d0 con 0x7f83f40a86b0 2021-11-21T22:01:09.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.354+0000 7f83f259c700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83d80534f0 0x7f83d80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:09.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.354+0000 7f83f259c700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83d80534f0 0x7f83d80559b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f83f4061b50 tx=0x7f83dc012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:09.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.354+0000 7f83eb7fe700 1 -- 172.21.2.116:0/2010615128 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83ec042730 con 0x7f83f40a86b0 2021-11-21T22:01:09.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83f405db70 con 0x7f83f40a86b0 2021-11-21T22:01:09.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83eb7fe700 1 -- 172.21.2.116:0/2010615128 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f83ec045d50 con 0x7f83f40a86b0 2021-11-21T22:01:09.467 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:09.467 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":38,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:03.447073+0000","modified":"2021-11-21T22:01:04.452008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2155808669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2155808669},{"type":"v1","addr":"172.21.2.116:6835","nonce":2155808669}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-11-21T22:01:09.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83d80534f0 msgr2=0x7f83d80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83d80534f0 0x7f83d80559b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f83f4061b50 tx=0x7f83dc012000).stop 2021-11-21T22:01:09.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a86b0 msgr2=0x7f83f40638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:09.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a86b0 0x7f83f40638c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f83ec002ef0 tx=0x7f83ec006cd0).stop 2021-11-21T22:01:09.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 shutdown_connections 2021-11-21T22:01:09.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f83d80534f0 0x7f83d80559b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f83f40a86b0 0x7f83f40638c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:09.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83f40a6aa0 0x7f83f4142160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 --2- 172.21.2.116:0/2010615128 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f83f4062410 0x7f83f4156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:09.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 >> 172.21.2.116:0/2010615128 conn(0x7f83f405bd20 msgr2=0x7f83f41416a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:09.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 shutdown_connections 2021-11-21T22:01:09.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:09.462+0000 7f83f8b41700 1 -- 172.21.2.116:0/2010615128 wait complete. 2021-11-21T22:01:09.471 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 38 2021-11-21T22:01:09.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:03.447073+0000', 'modified': '2021-11-21T22:01:04.452008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6669}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6669': {'gid': 6669, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/2155808669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 2155808669}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 2155808669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:09.777 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:09.777 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:09.857 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":6669,"want_state":"up:active","state":"up:active","rank_uptime":6.403668143,"mdsmap_epoch":38,"osdmap_epoch":89,"osdmap_epoch_barrier":88,"uptime":10.600907717} 2021-11-21T22:01:09.858 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 6669, "mdsmap_epoch": 38, "osdmap_epoch": 89, "osdmap_epoch_barrier": 88, "rank_uptime": 6.403668143, "state": "up:active", "uptime": 10.600907717, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:09.859 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:01:09.859 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:01:09.859 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:01:09.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:09.860 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:09.871 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:09.927 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:09.927 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:09.933 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:09.934 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:09.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:09.935 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:09.940 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:01:09.940 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:09.986 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:09.996 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:09.996 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:10.002 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:10.004 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:10.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:10.005 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:10.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:01:10.025+0000 7f733424c080 -1 init, newargv = 0x5583fedfbf10 newargc=9ceph-fuse[14268 2021-11-21T22:01:10.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:]: starting ceph client 2021-11-21T22:01:10.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14268]: starting fuse 2021-11-21T22:01:11.130 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:11.143 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:11.144 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:11.194 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:11.195 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:11.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.197 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:11.204 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:01:11.204 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:01:11.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.206 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:01:11.207 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:01:11.208 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:01:11.208 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:01:11.209 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:01:11.210 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:11.210 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:01:11.211 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:01:11.212 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:01:11.212 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:11.212 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:01:11.213 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:01:11.214 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:01:11.214 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:01:11.215 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:11.215 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:01:11.216 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:01:11.217 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:01:11.217 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:01:11.218 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:01:11.218 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:11.219 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:01:11.219 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:01:11.253 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14268.asok 2021-11-21T22:01:11.255 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14268.asok 2021-11-21T22:01:11.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.257 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14268.asok status 2021-11-21T22:01:11.350 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:01:11.350 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:01:11.350 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:01:11.351 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:01:11.351 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:01:11.351 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:01:11.351 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:01:11.352 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14268", 2021-11-21T22:01:11.352 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:01:11.352 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:11.352 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:01:11.352 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:01:11.353 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 4912, 2021-11-21T22:01:11.353 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:01:11.353 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:01:11.353 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:01:11.354 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 4912 2021-11-21T22:01:11.354 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:11.354 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:01:11.354 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:01:11.354 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:01:11.355 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 2089883301 2021-11-21T22:01:11.355 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:01:11.355 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:11.355 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:01:11.356 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:01:11.356 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:01:11.356 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 2089883301 2021-11-21T22:01:11.356 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:11.356 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.4912 172.21.2.136:0/2089883301", 2021-11-21T22:01:11.357 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/2089883301", 2021-11-21T22:01:11.357 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:01:11.357 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 38, 2021-11-21T22:01:11.357 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 89, 2021-11-21T22:01:11.358 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:01:11.358 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:01:11.358 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:01:11.358 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:01:11.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.360 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:11.360 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:11.369 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:01:11.370 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:11.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.371 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:11.437 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:11.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 Processor -- start 2021-11-21T22:01:11.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 -- start start 2021-11-21T22:01:11.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ef40979a0 0x7f0ef4142930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:11.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef405efb0 0x7f0ef413ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:11.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0ef40b5190 0x7f0ef40b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:11.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef4142000 con 0x7f0ef40979a0 2021-11-21T22:01:11.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef413c5b0 con 0x7f0ef40b5190 2021-11-21T22:01:11.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef8f3c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4061100 con 0x7f0ef405efb0 2021-11-21T22:01:11.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ef40979a0 0x7f0ef4142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:11.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ef40979a0 0x7f0ef4142930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50186/0 (socket says 172.21.2.116:50186) 2021-11-21T22:01:11.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.486+0000 7f0ef37fe700 1 -- 172.21.2.116:0/1056337015 learned_addr learned my addr 172.21.2.116:0/1056337015 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:11.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef37fe700 1 -- 172.21.2.116:0/1056337015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef405efb0 msgr2=0x7f0ef413ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:11.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef37fe700 1 --2- 172.21.2.116:0/1056337015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef405efb0 0x7f0ef413ea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef37fe700 1 -- 172.21.2.116:0/1056337015 >> v2:172.21.2.117:3300/0 conn(0x7f0ef40b5190 msgr2=0x7f0ef40b1b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:11.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef37fe700 1 --2- 172.21.2.116:0/1056337015 >> v2:172.21.2.117:3300/0 conn(0x7f0ef40b5190 0x7f0ef40b1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef37fe700 1 -- 172.21.2.116:0/1056337015 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef41421f0 con 0x7f0ef40979a0 2021-11-21T22:01:11.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef37fe700 1 --2- 172.21.2.116:0/1056337015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40979a0 0x7f0ef4142930 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f0ee4002f70 tx=0x7f0ee4006eb0).ready entity=mon.0 client_cookie=a8551a593517e295 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:11.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef27fc700 1 -- 172.21.2.116:0/1056337015 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ee400d570 con 0x7f0ef40979a0 2021-11-21T22:01:11.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef27fc700 1 -- 172.21.2.116:0/1056337015 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee4000e90 con 0x7f0ef40979a0 2021-11-21T22:01:11.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef27fc700 1 -- 172.21.2.116:0/1056337015 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ee400d900 con 0x7f0ef40979a0 2021-11-21T22:01:11.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/1056337015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40979a0 msgr2=0x7f0ef4142930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:11.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/1056337015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40979a0 0x7f0ef4142930 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f0ee4002f70 tx=0x7f0ee4006eb0).stop 2021-11-21T22:01:11.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/1056337015 shutdown_connections 2021-11-21T22:01:11.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/1056337015 >> v2:172.21.2.117:3300/0 conn(0x7f0ef40b5190 0x7f0ef40b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/1056337015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef405efb0 0x7f0ef413ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/1056337015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40979a0 0x7f0ef4142930 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:11.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/1056337015 >> 172.21.2.116:0/1056337015 conn(0x7f0ef4062e60 msgr2=0x7f0ef4063240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:11.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/1056337015 shutdown_connections 2021-11-21T22:01:11.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/1056337015 wait complete. 2021-11-21T22:01:11.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 Processor -- start 2021-11-21T22:01:11.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- start start 2021-11-21T22:01:11.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ef405efb0 0x7f0ef4133630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:11.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef40979a0 0x7f0ef40aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:11.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40b5190 0x7f0ef4098f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:11.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40af2d0 con 0x7f0ef40b5190 2021-11-21T22:01:11.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4060d20 con 0x7f0ef405efb0 2021-11-21T22:01:11.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40ad700 con 0x7f0ef40979a0 2021-11-21T22:01:11.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40b5190 0x7f0ef4098f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:11.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40b5190 0x7f0ef4098f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50196/0 (socket says 172.21.2.116:50196) 2021-11-21T22:01:11.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 -- 172.21.2.116:0/131348122 learned_addr learned my addr 172.21.2.116:0/131348122 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:11.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 -- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef40979a0 msgr2=0x7f0ef40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:11.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef40979a0 0x7f0ef40aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 -- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ef405efb0 msgr2=0x7f0ef4133630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:11.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ef405efb0 0x7f0ef4133630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 -- 172.21.2.116:0/131348122 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee400b1e0 con 0x7f0ef40b5190 2021-11-21T22:01:11.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef3fff700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40b5190 0x7f0ef4098f60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f0eec001250 tx=0x7f0eec0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:11.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef0ff9700 1 -- 172.21.2.116:0/131348122 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eec0014e0 con 0x7f0ef40b5190 2021-11-21T22:01:11.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef0ff9700 1 -- 172.21.2.116:0/131348122 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec001640 con 0x7f0ef40b5190 2021-11-21T22:01:11.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef0ff9700 1 -- 172.21.2.116:0/131348122 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eec0019a0 con 0x7f0ef40b5190 2021-11-21T22:01:11.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef40ad9a0 con 0x7f0ef40b5190 2021-11-21T22:01:11.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.490+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef40adfd0 con 0x7f0ef40b5190 2021-11-21T22:01:11.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.494+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4142490 con 0x7f0ef40b5190 2021-11-21T22:01:11.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.494+0000 7f0ef0ff9700 1 -- 172.21.2.116:0/131348122 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0eec018030 con 0x7f0ef40b5190 2021-11-21T22:01:11.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.494+0000 7f0ef0ff9700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ed4057990 0x7f0ed4059e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:11.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.494+0000 7f0ef0ff9700 1 -- 172.21.2.116:0/131348122 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0eec06b360 con 0x7f0ef40b5190 2021-11-21T22:01:11.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.494+0000 7f0ef37fe700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ed4057990 0x7f0ed4059e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:11.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.494+0000 7f0ef37fe700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ed4057990 0x7f0ed4059e50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0ee4002f70 tx=0x7f0ee4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:11.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.494+0000 7f0ef0ff9700 1 -- 172.21.2.116:0/131348122 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0eec042460 con 0x7f0ef40b5190 2021-11-21T22:01:11.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ef40573d0 con 0x7f0ef40b5190 2021-11-21T22:01:11.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef0ff9700 1 -- 172.21.2.116:0/131348122 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f0eec045a80 con 0x7f0ef40b5190 2021-11-21T22:01:11.607 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:11.608 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":38,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:03.447073+0000","modified":"2021-11-21T22:01:04.452008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2155808669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2155808669},{"type":"v1","addr":"172.21.2.116:6835","nonce":2155808669}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-11-21T22:01:11.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ed4057990 msgr2=0x7f0ed4059e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:11.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ed4057990 0x7f0ed4059e50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0ee4002f70 tx=0x7f0ee4015000).stop 2021-11-21T22:01:11.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40b5190 msgr2=0x7f0ef4098f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:11.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40b5190 0x7f0ef4098f60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f0eec001250 tx=0x7f0eec0090f0).stop 2021-11-21T22:01:11.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 shutdown_connections 2021-11-21T22:01:11.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ed4057990 0x7f0ed4059e50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:11.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ef40b5190 0x7f0ef4098f60 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:11.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ef40979a0 0x7f0ef40aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 --2- 172.21.2.116:0/131348122 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ef405efb0 0x7f0ef4133630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:11.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 >> 172.21.2.116:0/131348122 conn(0x7f0ef4062e60 msgr2=0x7f0ef4140900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:11.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 shutdown_connections 2021-11-21T22:01:11.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:11.602+0000 7f0ef8f3c700 1 -- 172.21.2.116:0/131348122 wait complete. 2021-11-21T22:01:11.613 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 38 2021-11-21T22:01:11.614 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump inode 1 2021-11-21T22:01:11.672 INFO:teuthology.orchestra.run.gibba016.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2021-11-21T22:01:08.537625+0000","btime":"2021-11-21T22:01:03.450880+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2021-11-21T22:01:03.450880+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2021-11-21T22:01:08.537625+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2021-11-21T22:01:03.450880+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":4900,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":4912,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2021-11-21T22:01:11.674 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2021-11-21T22:01:03.450880+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2021-11-21T22:01:03.450880+0000", "change_attr": 1, "client_caps": [ { "client_id": 4900, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 4912, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2021-11-21T22:01:08.537625+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2021-11-21T22:01:03.450880+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2021-11-21T22:01:08.537625+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2021-11-21T22:01:11.674 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:01:11.675 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:11.683 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:11.697 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:11.711 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:11.725 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:01:11.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.727 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:11.727 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:11.759 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:01:11.760 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:11.761 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:01:11.761 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:01:11.761 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:11.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14783]: fuse finished with error 0 and tester_r 0 2021-11-21T22:01:11.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.774 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:11.774 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:11.781 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:01:11.782 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:01:11.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:01:11.796 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:11.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.842 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:11.842 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:11.854 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:01:11.855 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:11.855 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:01:11.856 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:01:11.856 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:11.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14268]: fuse finished with error 0 and tester_r 0 2021-11-21T22:01:11.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:11.915 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:11.916 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:11.919 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:01:11.920 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:01:12.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:01:12.040 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:12.093 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:01:12.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.142+0000 7ff2f8401700 1 Processor -- start 2021-11-21T22:01:12.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.142+0000 7ff2f8401700 1 -- start start 2021-11-21T22:01:12.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f00b3320 0x7ff2f00ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2f01443a0 0x7ff2f0144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f00b29b0 con 0x7ff2f0067850 2021-11-21T22:01:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f0067c30 con 0x7ff2f01443a0 2021-11-21T22:01:12.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00b3780 con 0x7ff2f00b3320 2021-11-21T22:01:12.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2f01443a0 0x7ff2f0144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2f01443a0 0x7ff2f0144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50204/0 (socket says 172.21.2.116:50204) 2021-11-21T22:01:12.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/4229466986 learned_addr learned my addr 172.21.2.116:0/4229466986 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/4229466986 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f00b3320 msgr2=0x7ff2f00ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- 172.21.2.116:0/4229466986 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f00b3320 0x7ff2f00ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/4229466986 >> v1:172.21.2.117:6789/0 conn(0x7ff2f0067850 legacy=0x7ff2f00b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:12.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/4229466986 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00b7910 con 0x7ff2f01443a0 2021-11-21T22:01:12.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- 172.21.2.116:0/4229466986 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f0144830 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7ff2f0066f90 tx=0x7ff2e8009290).ready entity=mon.0 client_cookie=67d5011b83251733 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f519b700 1 -- 172.21.2.116:0/4229466986 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e800b860 con 0x7ff2f01443a0 2021-11-21T22:01:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f519b700 1 -- 172.21.2.116:0/4229466986 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e800b9c0 con 0x7ff2f01443a0 2021-11-21T22:01:12.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f519b700 1 -- 172.21.2.116:0/4229466986 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e800bb70 con 0x7ff2f01443a0 2021-11-21T22:01:12.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- 172.21.2.116:0/4229466986 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 msgr2=0x7ff2f0144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- 172.21.2.116:0/4229466986 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f0144830 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7ff2f0066f90 tx=0x7ff2e8009290).stop 2021-11-21T22:01:12.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- 172.21.2.116:0/4229466986 shutdown_connections 2021-11-21T22:01:12.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- 172.21.2.116:0/4229466986 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f0144830 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- 172.21.2.116:0/4229466986 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f00b3320 0x7ff2f00ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- 172.21.2.116:0/4229466986 >> 172.21.2.116:0/4229466986 conn(0x7ff2f0064d60 msgr2=0x7ff2f0065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:12.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- 172.21.2.116:0/4229466986 shutdown_connections 2021-11-21T22:01:12.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- 172.21.2.116:0/4229466986 wait complete. 2021-11-21T22:01:12.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 Processor -- start 2021-11-21T22:01:12.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- start start 2021-11-21T22:01:12.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f0067850 0x7ff2f014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2f00b3320 0x7ff2f00b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f009eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f014e070 con 0x7ff2f01443a0 2021-11-21T22:01:12.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f014eed0 con 0x7ff2f00b3320 2021-11-21T22:01:12.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f8401700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f009e460 con 0x7ff2f0067850 2021-11-21T22:01:12.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f009eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f009eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50210/0 (socket says 172.21.2.116:50210) 2021-11-21T22:01:12.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/1342055978 learned_addr learned my addr 172.21.2.116:0/1342055978 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f0067850 msgr2=0x7ff2f014e9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:12.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f0067850 0x7ff2f014e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2f00b3320 msgr2=0x7ff2f00b0e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:12.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2f00b3320 0x7ff2f00b0e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.146+0000 7ff2f699e700 1 -- 172.21.2.116:0/1342055978 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8009530 con 0x7ff2f01443a0 2021-11-21T22:01:12.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2f699e700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f009eca0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7ff2f0066f90 tx=0x7ff2e8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2df7fe700 1 -- 172.21.2.116:0/1342055978 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e800b860 con 0x7ff2f01443a0 2021-11-21T22:01:12.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2df7fe700 1 -- 172.21.2.116:0/1342055978 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e8020620 con 0x7ff2f01443a0 2021-11-21T22:01:12.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f014e2a0 con 0x7ff2f01443a0 2021-11-21T22:01:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f00b1370 con 0x7ff2f01443a0 2021-11-21T22:01:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2df7fe700 1 -- 172.21.2.116:0/1342055978 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e8020980 con 0x7ff2f01443a0 2021-11-21T22:01:12.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f014e6e0 con 0x7ff2f01443a0 2021-11-21T22:01:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2df7fe700 1 -- 172.21.2.116:0/1342055978 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff2e800f070 con 0x7ff2f01443a0 2021-11-21T22:01:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2df7fe700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff2d0053610 0x7ff2d0055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2df7fe700 1 -- 172.21.2.116:0/1342055978 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff2e806cc60 con 0x7ff2f01443a0 2021-11-21T22:01:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2f619d700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff2d0053610 0x7ff2d0055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.150+0000 7ff2f619d700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff2d0053610 0x7ff2d0055ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7ff2ec002f90 tx=0x7ff2ec006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.154+0000 7ff2df7fe700 1 -- 172.21.2.116:0/1342055978 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff2e8043cf0 con 0x7ff2f01443a0 2021-11-21T22:01:12.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff2f00b2be0 con 0x7ff2f01443a0 2021-11-21T22:01:12.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2df7fe700 1 -- 172.21.2.116:0/1342055978 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v89) v1 ==== 81+0+33648 (secure 0 0 0) 0x7ff2e8015070 con 0x7ff2f01443a0 2021-11-21T22:01:12.238 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:12.238 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:12.238 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 89, 2021-11-21T22:01:12.239 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:01:12.239 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:01:12.240 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:01:05.463699+0000", 2021-11-21T22:01:12.240 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:12.240 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:12.240 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:01:12.241 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:01:12.241 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:01:12.241 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:01:12.241 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:01:12.242 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:01:12.242 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:01:12.242 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:12.242 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:01:12.242 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:01:12.243 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:01:12.243 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:01:12.243 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:01:12.243 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 16, 2021-11-21T22:01:12.244 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:01:12.244 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:01:12.244 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:01:12.244 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:01:12.244 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:01:12.245 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.245 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:01:12.245 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:01:12.245 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:01:12.245 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:12.246 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:12.246 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:12.246 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:12.246 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:12.247 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:12.247 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:12.247 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:12.247 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:01:12.247 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:01:12.248 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:01:12.248 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:01:12.248 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:01:12.248 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:12.249 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:12.249 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:12.249 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:12.250 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:12.250 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:12.250 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:12.250 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.251 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:01:12.251 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:12.251 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:12.251 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:12.251 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:12.252 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:12.252 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:12.252 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:12.252 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:12.253 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:12.253 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:12.253 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:12.253 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:12.253 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:12.254 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:12.254 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:12.254 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:12.254 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:12.255 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:12.255 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:12.256 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:12.256 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:12.257 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:12.257 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:12.258 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:12.258 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:12.259 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:12.259 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.259 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:12.260 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:12.260 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:12.261 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:12.261 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:12.262 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:12.262 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:12.263 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:12.263 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:12.263 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:12.264 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:12.264 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:12.264 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:01:12.265 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.265 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:12.265 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:01:12.266 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.266 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.266 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.267 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:01:12.267 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:01:12.267 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:01:12.267 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:12.268 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:12.268 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:12.268 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:12.269 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:12.269 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:12.269 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:12.270 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:12.270 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:12.270 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:12.271 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:12.271 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:12.271 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:12.272 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:12.272 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:12.272 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:12.272 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:12.273 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:12.273 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:12.274 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:12.274 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.275 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:01:12.275 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:12.275 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:12.276 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:12.276 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:12.277 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:12.277 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:12.277 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:12.278 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:12.278 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:12.279 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:12.279 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:12.279 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:12.280 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:12.280 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:12.280 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:12.280 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:12.281 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:12.281 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:12.281 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:12.281 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:12.282 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:12.282 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:12.282 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:12.282 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:12.282 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:12.283 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:12.283 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.283 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:12.283 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:12.283 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:12.284 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:12.284 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:12.284 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:12.284 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:12.285 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:12.285 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:12.285 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:12.285 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:12.285 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:12.286 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:12.286 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:01:12.286 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.286 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.287 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.287 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 15, 2021-11-21T22:01:12.287 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:01:12.287 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:01:01.325473+0000", 2021-11-21T22:01:12.287 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:12.288 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:12.288 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:12.288 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:12.288 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:12.289 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:12.289 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:12.289 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:12.289 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:12.289 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:12.290 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:12.290 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:12.290 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:12.290 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:12.290 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:12.291 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:12.291 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:12.291 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:12.291 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:12.292 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:12.292 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.292 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "88", 2021-11-21T22:01:12.292 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:12.292 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:12.293 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:12.293 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:12.293 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:12.293 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:12.293 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:12.294 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:12.294 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:12.294 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:12.294 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:12.295 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:12.295 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:12.295 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:12.295 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:12.295 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:12.296 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:12.296 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:12.296 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:12.296 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:12.296 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:12.297 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:12.297 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:12.297 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:12.297 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:12.298 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:12.298 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.298 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:12.298 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:12.298 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:12.299 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:12.299 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:12.299 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:12.299 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:12.300 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:12.300 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:12.300 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:12.300 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:12.300 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:12.301 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:01:12.301 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:01:12.301 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:01:12.301 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.301 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:12.302 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:01:12.302 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:01:12.302 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.302 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.303 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.303 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.303 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 16, 2021-11-21T22:01:12.303 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:01:12.303 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:01:02.426827+0000", 2021-11-21T22:01:12.304 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:12.304 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:12.304 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:12.304 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:12.305 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:12.305 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:12.305 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:12.305 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:12.305 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:12.306 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:12.306 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:12.306 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:12.306 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:12.307 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:12.307 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:12.307 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:12.307 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:12.307 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:12.308 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:12.308 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:12.308 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.308 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "89", 2021-11-21T22:01:12.308 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:12.309 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:12.309 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:12.309 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:12.309 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:12.310 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:12.310 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:12.310 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:12.310 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:12.310 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:12.311 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:12.311 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:12.311 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:12.311 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:12.312 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:12.312 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:12.312 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:12.312 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:12.312 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:12.313 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:12.313 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:12.313 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:12.313 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:12.313 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:12.314 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:12.314 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:12.314 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.314 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:12.315 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:12.315 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:12.315 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:12.315 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:12.315 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:12.316 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:12.316 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:12.316 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:12.317 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:12.317 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:12.317 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:12.317 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:12.317 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:01:12.318 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:01:12.318 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.318 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.318 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.319 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:12.319 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:01:12.319 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.319 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:12.319 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:01:12.320 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.320 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.320 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.320 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.320 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.321 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.321 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.321 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:12.321 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.322 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.322 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.322 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.322 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.323 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.323 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:01:12.323 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.323 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.324 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.324 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.324 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:01:12.324 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.324 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.325 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.325 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.325 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.325 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.325 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.326 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.326 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:01:12.326 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.326 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.327 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.327 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.327 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:01:12.327 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.327 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.328 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.328 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.328 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.328 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.329 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.329 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.329 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:01:12.329 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.329 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.330 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.330 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.330 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:01:12.330 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.331 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.331 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.331 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.331 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.331 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.332 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.332 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.332 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:01:12.332 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.332 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.333 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.333 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.333 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:01:12.333 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:12.334 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.334 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.334 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.334 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:01:12.334 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:01:12.335 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:01:12.335 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:01:12.335 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.335 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.335 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.336 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.336 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.336 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.336 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:12.337 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:01:12.337 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.337 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.337 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.337 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.338 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.338 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.338 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.338 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 85, 2021-11-21T22:01:12.339 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.339 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.339 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.339 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.339 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.340 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.340 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:01:12.340 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.340 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.340 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.341 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.341 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:01:12.341 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.341 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.342 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.342 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.342 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.342 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.342 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.343 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.343 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:01:12.343 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.343 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.344 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.344 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.344 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:01:12.344 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.344 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.345 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.345 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.345 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.345 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.345 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.346 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.346 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:01:12.346 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.346 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.347 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.347 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.347 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:01:12.347 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.347 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.348 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.348 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.348 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.348 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.348 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.349 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.349 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:01:12.349 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.349 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.350 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.350 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.350 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:01:12.350 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:12.350 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.351 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.351 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.351 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:01:12.351 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:01:12.351 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:01:12.352 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:01:12.352 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.352 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.352 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.353 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.353 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.353 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.353 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:12.353 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:01:12.354 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.354 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.354 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.354 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.355 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.355 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.355 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.355 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:12.355 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.356 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.356 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.356 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.356 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.357 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.357 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:01:12.357 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.357 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.357 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.358 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.358 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:01:12.358 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.358 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.359 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.359 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.359 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.359 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.359 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.360 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.360 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:01:12.360 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.360 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.360 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.361 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.361 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:01:12.361 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.361 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.362 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.362 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.362 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.362 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.362 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.363 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.363 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:01:12.363 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.363 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.364 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.364 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.364 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:01:12.364 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.364 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.365 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.365 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.365 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.365 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.366 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.366 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.366 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:01:12.366 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.366 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.367 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.367 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.367 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:01:12.367 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:12.368 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.368 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.368 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.368 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:01:12.368 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:01:12.369 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:01:12.369 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:01:12.369 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.369 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.370 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.370 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.370 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.370 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.371 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:12.371 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:01:12.371 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.371 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.371 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.372 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.372 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.372 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.372 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.373 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 87, 2021-11-21T22:01:12.373 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.373 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.373 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.373 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.374 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.374 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.374 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:01:12.374 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.375 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.375 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.375 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.375 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:01:12.375 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.376 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.376 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.376 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.376 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.377 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.377 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.377 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.377 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:01:12.377 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.378 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.378 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.378 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.378 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:01:12.378 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.379 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.379 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.379 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.379 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.380 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.380 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.380 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.380 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:01:12.380 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.381 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.381 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.381 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.381 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:01:12.381 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.382 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.382 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.382 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.382 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.383 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.383 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.383 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.383 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:01:12.383 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.384 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.384 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.384 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.384 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:01:12.385 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:12.385 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.385 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.385 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.385 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:01:12.386 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:01:12.386 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:01:12.386 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:01:12.386 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.387 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.387 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.387 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.387 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.387 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.388 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:12.388 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:01:12.388 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.388 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.389 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.389 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.389 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.389 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.389 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.390 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:12.390 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.390 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.390 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.390 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.391 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.391 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.391 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:01:12.391 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.392 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.392 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.392 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.392 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:01:12.392 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.393 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.393 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.393 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.393 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.394 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.394 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.394 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.394 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:01:12.394 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.395 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.395 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.395 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.395 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:01:12.395 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.396 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.396 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.396 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.396 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.397 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.397 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.397 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.397 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:01:12.397 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.398 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.398 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.398 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.398 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:01:12.399 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.399 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.399 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.399 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.399 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.400 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.400 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.400 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.400 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:01:12.401 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.401 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.401 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.401 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.401 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:01:12.402 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:12.402 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.402 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.402 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.403 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:01:12.403 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:01:12.403 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:01:12.403 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:01:12.403 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.404 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.404 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.404 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.404 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.405 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.405 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:12.405 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:01:12.405 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.405 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.406 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.406 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.406 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.406 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.406 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.407 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:12.407 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.407 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.407 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.408 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.408 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.408 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.408 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:01:12.408 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.409 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.409 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.409 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.409 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:01:12.410 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.410 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.410 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.410 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.410 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.411 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.411 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.411 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.411 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:01:12.412 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.412 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.412 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.412 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.413 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:01:12.413 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.413 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.413 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.414 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.414 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.414 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.414 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.415 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.415 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:01:12.415 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.415 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.416 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.416 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.416 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:01:12.416 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.417 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.417 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.417 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.417 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.418 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.418 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.418 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.418 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:01:12.418 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.419 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.419 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.419 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:01:12.420 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:12.420 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.420 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.420 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.420 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:01:12.421 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:01:12.421 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:01:12.421 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:01:12.421 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.421 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.422 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.422 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.422 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.422 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.423 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:12.423 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:01:12.423 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.423 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.423 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.424 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.424 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.424 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.424 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.425 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:12.425 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.425 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.425 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.425 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.426 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.426 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.426 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:01:12.426 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.427 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.427 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.427 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:01:12.427 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.428 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.428 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.428 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.428 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.428 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.429 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.429 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.429 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:01:12.429 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.430 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.430 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.430 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.430 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:01:12.430 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.431 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.431 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.431 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.431 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.431 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.432 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.432 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:01:12.432 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.433 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.433 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.433 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:01:12.433 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.434 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.434 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.434 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.434 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.434 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.435 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.435 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.435 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:01:12.435 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.436 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.436 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.436 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.436 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:01:12.436 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:12.437 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.437 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.437 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.437 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:01:12.438 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:01:12.438 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:01:12.438 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:01:12.438 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.438 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.439 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.439 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.439 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.439 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.439 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:12.440 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:01:12.440 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:12.440 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:12.440 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:12.441 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:12.441 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:12.441 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:12.441 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:12.441 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 72, 2021-11-21T22:01:12.442 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:12.442 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:12.442 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:12.442 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.442 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.443 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.443 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:01:12.443 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.443 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.444 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.444 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.444 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:01:12.444 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.444 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.445 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.445 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.445 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:12.445 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.445 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.446 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.446 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:01:12.446 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.446 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.447 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.447 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.447 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:01:12.447 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.447 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.448 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.448 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.448 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:12.448 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.449 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.449 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:01:12.449 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.449 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.450 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.450 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.450 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:01:12.450 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.450 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.451 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.451 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.451 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:12.451 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:12.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.452 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:12.452 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:01:12.452 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.453 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.453 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.453 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:12.453 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:01:12.453 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:12.454 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.454 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.454 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.454 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:01:12.454 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:01:12.455 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:01:12.455 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:01:12.455 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:12.455 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:12.456 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:12.456 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:12.456 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.456 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:12.456 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:01:12.457 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.457 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:12.457 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.457 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.458 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.458 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.458 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.458 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:01:12.458 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.459 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.459 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.459 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:12.459 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.459 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.460 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.460 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.460 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.460 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:01:12.461 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.461 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.461 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.461 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:12.461 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.462 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.466 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.466 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.466 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.467 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:01:12.467 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.467 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.467 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.468 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:12.468 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.468 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.468 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.468 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.469 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.469 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:01:12.469 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.469 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.470 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.470 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:12.470 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.470 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.470 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.471 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.471 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.471 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:01:12.471 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.472 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.472 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.472 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:12.472 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.472 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.473 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.473 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.473 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.473 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:01:12.473 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.474 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.474 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:12.474 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.475 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.475 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.475 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.475 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.475 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:01:12.476 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.476 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:12.476 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:12.477 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:12.477 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:12.477 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:12.477 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:12.477 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:12.478 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:01:12.478 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:12.478 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.478 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:12.479 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:01:12.479 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:01:12.479 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:01:12.479 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:01:12.479 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T22:01:12.480 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:01:12.480 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:01:12.480 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:01:12.480 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:01:12.481 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:01:12.481 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:01:12.481 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:01:12.481 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:01:12.481 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:12.482 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:12.482 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:01:12.482 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:01:12.482 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:01:12.483 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:01:12.483 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:01:12.483 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:12.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff2d0053610 msgr2=0x7ff2d0055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff2d0053610 0x7ff2d0055ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7ff2ec002f90 tx=0x7ff2ec006930).stop 2021-11-21T22:01:12.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 msgr2=0x7ff2f009eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f009eca0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7ff2f0066f90 tx=0x7ff2e8016000).stop 2021-11-21T22:01:12.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 shutdown_connections 2021-11-21T22:01:12.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff2d0053610 0x7ff2d0055ad0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2f01443a0 0x7ff2f009eca0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2f00b3320 0x7ff2f00b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 --2- 172.21.2.116:0/1342055978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2f0067850 0x7ff2f014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 >> 172.21.2.116:0/1342055978 conn(0x7ff2f0064d60 msgr2=0x7ff2f0061760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:12.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 shutdown_connections 2021-11-21T22:01:12.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.234+0000 7ff2f8401700 1 -- 172.21.2.116:0/1342055978 wait complete. 2021-11-21T22:01:12.488 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:12.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.554+0000 7f40fa369700 1 Processor -- start 2021-11-21T22:01:12.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.554+0000 7f40fa369700 1 -- start start 2021-11-21T22:01:12.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40f40abcb0 0x7f40f40af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f40adb90 0x7f40f413cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f40ac090 con 0x7f40f413d250 2021-11-21T22:01:12.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f40b1590 con 0x7f40f40abcb0 2021-11-21T22:01:12.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40ae7c0 con 0x7f40f40adb90 2021-11-21T22:01:12.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40f40abcb0 0x7f40f40af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f40f40abcb0 0x7f40f40af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50214/0 (socket says 172.21.2.116:50214) 2021-11-21T22:01:12.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 -- 172.21.2.116:0/1452910133 learned_addr learned my addr 172.21.2.116:0/1452910133 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f8906700 1 --1- 172.21.2.116:0/1452910133 >> v1:172.21.2.116:6789/0 conn(0x7f40f413d250 0x7f40f413fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:33940/0 (socket says 172.21.2.116:33940) 2021-11-21T22:01:12.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 -- 172.21.2.116:0/1452910133 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f40adb90 msgr2=0x7f40f413cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:12.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 --2- 172.21.2.116:0/1452910133 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f40adb90 0x7f40f413cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 -- 172.21.2.116:0/1452910133 >> v1:172.21.2.116:6789/0 conn(0x7f40f413d250 legacy=0x7f40f413fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 -- 172.21.2.116:0/1452910133 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f413d700 con 0x7f40f40abcb0 2021-11-21T22:01:12.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f3fff700 1 --2- 172.21.2.116:0/1452910133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40abcb0 0x7f40f40af290 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f40e4002f70 tx=0x7f40e4006eb0).ready entity=mon.0 client_cookie=68064e6ccbf5e3cf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f2ffd700 1 -- 172.21.2.116:0/1452910133 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e400d5c0 con 0x7f40f40abcb0 2021-11-21T22:01:12.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f2ffd700 1 -- 172.21.2.116:0/1452910133 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e4000e90 con 0x7f40f40abcb0 2021-11-21T22:01:12.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f2ffd700 1 -- 172.21.2.116:0/1452910133 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e400d950 con 0x7f40f40abcb0 2021-11-21T22:01:12.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- 172.21.2.116:0/1452910133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40abcb0 msgr2=0x7f40f40af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- 172.21.2.116:0/1452910133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40abcb0 0x7f40f40af290 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f40e4002f70 tx=0x7f40e4006eb0).stop 2021-11-21T22:01:12.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- 172.21.2.116:0/1452910133 shutdown_connections 2021-11-21T22:01:12.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- 172.21.2.116:0/1452910133 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f40adb90 0x7f40f413cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- 172.21.2.116:0/1452910133 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40abcb0 0x7f40f40af290 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- 172.21.2.116:0/1452910133 >> 172.21.2.116:0/1452910133 conn(0x7f40f405bd20 msgr2=0x7f40f405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:12.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- 172.21.2.116:0/1452910133 shutdown_connections 2021-11-21T22:01:12.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- 172.21.2.116:0/1452910133 wait complete. 2021-11-21T22:01:12.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 Processor -- start 2021-11-21T22:01:12.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- start start 2021-11-21T22:01:12.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40f40abcb0 0x7f40f415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40adb90 0x7f40f4137720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f413d250 0x7f40f4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4159d50 con 0x7f40f40adb90 2021-11-21T22:01:12.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40b3200 con 0x7f40f40abcb0 2021-11-21T22:01:12.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40fa369700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4159f80 con 0x7f40f413d250 2021-11-21T22:01:12.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40adb90 0x7f40f4137720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40adb90 0x7f40f4137720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50224/0 (socket says 172.21.2.116:50224) 2021-11-21T22:01:12.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.558+0000 7f40f37fe700 1 -- 172.21.2.116:0/1834455768 learned_addr learned my addr 172.21.2.116:0/1834455768 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f37fe700 1 -- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f413d250 msgr2=0x7f40f4156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f37fe700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f413d250 0x7f40f4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f37fe700 1 -- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40f40abcb0 msgr2=0x7f40f415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f37fe700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40f40abcb0 0x7f40f415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f37fe700 1 -- 172.21.2.116:0/1834455768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e400b1e0 con 0x7f40f40adb90 2021-11-21T22:01:12.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f37fe700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40adb90 0x7f40f4137720 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f40e8002f70 tx=0x7f40e8006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f17fa700 1 -- 172.21.2.116:0/1834455768 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e8013460 con 0x7f40f40adb90 2021-11-21T22:01:12.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f17fa700 1 -- 172.21.2.116:0/1834455768 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e80135c0 con 0x7f40f40adb90 2021-11-21T22:01:12.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f415a220 con 0x7f40f40adb90 2021-11-21T22:01:12.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f17fa700 1 -- 172.21.2.116:0/1834455768 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e8013920 con 0x7f40f40adb90 2021-11-21T22:01:12.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f413e4b0 con 0x7f40f40adb90 2021-11-21T22:01:12.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f413a3d0 con 0x7f40f40adb90 2021-11-21T22:01:12.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f17fa700 1 -- 172.21.2.116:0/1834455768 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f40e801a060 con 0x7f40f40adb90 2021-11-21T22:01:12.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f17fa700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40d4053580 0x7f40d4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f17fa700 1 -- 172.21.2.116:0/1834455768 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f40e806b460 con 0x7f40f40adb90 2021-11-21T22:01:12.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f3fff700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40d4053580 0x7f40d4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.562+0000 7f40f3fff700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40d4053580 0x7f40d4055a40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f40e4002f70 tx=0x7f40e4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.566+0000 7f40f17fa700 1 -- 172.21.2.116:0/1834455768 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40e80424b0 con 0x7f40f40adb90 2021-11-21T22:01:12.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.670+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40f415a3b0 con 0x7f40f40adb90 2021-11-21T22:01:12.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.670+0000 7f40f17fa700 1 -- 172.21.2.116:0/1834455768 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f40e8045ad0 con 0x7f40f40adb90 2021-11-21T22:01:12.676 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:12.676 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":38,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:03.447073+0000","modified":"2021-11-21T22:01:04.452008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/2155808669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2155808669},{"type":"v1","addr":"172.21.2.116:6835","nonce":2155808669}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-11-21T22:01:12.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40d4053580 msgr2=0x7f40d4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40d4053580 0x7f40d4055a40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f40e4002f70 tx=0x7f40e4015000).stop 2021-11-21T22:01:12.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40adb90 msgr2=0x7f40f4137720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40adb90 0x7f40f4137720 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f40e8002f70 tx=0x7f40e8006e90).stop 2021-11-21T22:01:12.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 shutdown_connections 2021-11-21T22:01:12.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f40f413d250 0x7f40f4156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f40d4053580 0x7f40d4055a40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f40f40adb90 0x7f40f4137720 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 --2- 172.21.2.116:0/1834455768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f40f40abcb0 0x7f40f415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 >> 172.21.2.116:0/1834455768 conn(0x7f40f405bd20 msgr2=0x7f40f4147b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:12.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 shutdown_connections 2021-11-21T22:01:12.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.674+0000 7f40fa369700 1 -- 172.21.2.116:0/1834455768 wait complete. 2021-11-21T22:01:12.680 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 38 2021-11-21T22:01:12.683 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:01:12.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 Processor -- start 2021-11-21T22:01:12.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- start start 2021-11-21T22:01:12.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff5400abce0 0x7ff5401478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff540147e80 con 0x7ff540144810 2021-11-21T22:01:12.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5400af5a0 con 0x7ff5400ac600 2021-11-21T22:01:12.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5400b1490 con 0x7ff5400abce0 2021-11-21T22:01:12.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff545a8d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff5400abce0 0x7ff5401478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff545a8d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff5400abce0 0x7ff5401478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50230/0 (socket says 172.21.2.116:50230) 2021-11-21T22:01:12.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff545a8d700 1 -- 172.21.2.116:0/829579116 learned_addr learned my addr 172.21.2.116:0/829579116 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff545a8d700 1 -- 172.21.2.116:0/829579116 >> v1:172.21.2.117:6789/0 conn(0x7ff5400ac600 legacy=0x7ff5400aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:12.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff545a8d700 1 -- 172.21.2.116:0/829579116 >> v1:172.21.2.116:6789/0 conn(0x7ff540144810 legacy=0x7ff5400b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff545a8d700 1 -- 172.21.2.116:0/829579116 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400b1fd0 con 0x7ff5400abce0 2021-11-21T22:01:12.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff545a8d700 1 --2- 172.21.2.116:0/829579116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff5400abce0 0x7ff5401478c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7ff5400b4e00 tx=0x7ff53c006eb0).ready entity=mon.0 client_cookie=40e3523414c7a830 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff54528c700 1 -- 172.21.2.116:0/829579116 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff53c00d610 con 0x7ff5400abce0 2021-11-21T22:01:12.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff54528c700 1 -- 172.21.2.116:0/829579116 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c000e90 con 0x7ff5400abce0 2021-11-21T22:01:12.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff54528c700 1 -- 172.21.2.116:0/829579116 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff53c00d960 con 0x7ff5400abce0 2021-11-21T22:01:12.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- 172.21.2.116:0/829579116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff5400abce0 msgr2=0x7ff5401478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 --2- 172.21.2.116:0/829579116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff5400abce0 0x7ff5401478c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7ff5400b4e00 tx=0x7ff53c006eb0).stop 2021-11-21T22:01:12.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- 172.21.2.116:0/829579116 shutdown_connections 2021-11-21T22:01:12.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 --2- 172.21.2.116:0/829579116 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff5400abce0 0x7ff5401478c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- 172.21.2.116:0/829579116 >> 172.21.2.116:0/829579116 conn(0x7ff54005bd20 msgr2=0x7ff54005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:12.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- 172.21.2.116:0/829579116 shutdown_connections 2021-11-21T22:01:12.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.730+0000 7ff5484f2700 1 -- 172.21.2.116:0/829579116 wait complete. 2021-11-21T22:01:12.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 Processor -- start 2021-11-21T22:01:12.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 -- start start 2021-11-21T22:01:12.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff5400abce0 0x7ff540067720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff5400ac600 0x7ff540143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff540144810 0x7ff5401465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540063780 con 0x7ff540144810 2021-11-21T22:01:12.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540145c70 con 0x7ff5400ac600 2021-11-21T22:01:12.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5401451c0 con 0x7ff5400abce0 2021-11-21T22:01:12.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff540144810 0x7ff5401465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff540144810 0x7ff5401465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50234/0 (socket says 172.21.2.116:50234) 2021-11-21T22:01:12.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 -- 172.21.2.116:0/3260904746 learned_addr learned my addr 172.21.2.116:0/3260904746 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff545a8d700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff5400ac600 0x7ff540143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 -- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff5400abce0 msgr2=0x7ff540067720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff5400abce0 0x7ff540067720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 -- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff5400ac600 msgr2=0x7ff540143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff5400ac600 0x7ff540143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 -- 172.21.2.116:0/3260904746 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c00b1e0 con 0x7ff540144810 2021-11-21T22:01:12.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff546a8f700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff540144810 0x7ff5401465c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7ff538003310 tx=0x7ff5380091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5337fe700 1 -- 172.21.2.116:0/3260904746 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff538001500 con 0x7ff540144810 2021-11-21T22:01:12.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5337fe700 1 -- 172.21.2.116:0/3260904746 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538001660 con 0x7ff540144810 2021-11-21T22:01:12.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540145460 con 0x7ff540144810 2021-11-21T22:01:12.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5337fe700 1 -- 172.21.2.116:0/3260904746 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5380019c0 con 0x7ff540144810 2021-11-21T22:01:12.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5400629c0 con 0x7ff540144810 2021-11-21T22:01:12.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5401455f0 con 0x7ff540144810 2021-11-21T22:01:12.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5337fe700 1 -- 172.21.2.116:0/3260904746 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff538018030 con 0x7ff540144810 2021-11-21T22:01:12.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5337fe700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff520053610 0x7ff520055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff5337fe700 1 -- 172.21.2.116:0/3260904746 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff53806b2c0 con 0x7ff540144810 2021-11-21T22:01:12.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.734+0000 7ff54628e700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff520053610 0x7ff520055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.738+0000 7ff54628e700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff520053610 0x7ff520055ad0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7ff540062500 tx=0x7ff5340077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.738+0000 7ff5337fe700 1 -- 172.21.2.116:0/3260904746 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5380424d0 con 0x7ff540144810 2021-11-21T22:01:12.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.846+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff5400af060 con 0x7ff540144810 2021-11-21T22:01:12.894 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:01:12.890+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:01:12.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5337fe700 1 -- 172.21.2.116:0/3260904746 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff538045af0 con 0x7ff540144810 2021-11-21T22:01:12.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff520053610 msgr2=0x7ff520055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff520053610 0x7ff520055ad0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7ff540062500 tx=0x7ff5340077f0).stop 2021-11-21T22:01:12.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff540144810 msgr2=0x7ff5401465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff540144810 0x7ff5401465c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7ff538003310 tx=0x7ff5380091a0).stop 2021-11-21T22:01:12.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 shutdown_connections 2021-11-21T22:01:12.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff520053610 0x7ff520055ad0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff540144810 0x7ff5401465c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff5400ac600 0x7ff540143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 --2- 172.21.2.116:0/3260904746 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff5400abce0 0x7ff540067720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 >> 172.21.2.116:0/3260904746 conn(0x7ff54005bd20 msgr2=0x7ff54005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:12.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 shutdown_connections 2021-11-21T22:01:12.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.894+0000 7ff5484f2700 1 -- 172.21.2.116:0/3260904746 wait complete. 2021-11-21T22:01:12.902 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:01:12.907 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:12.911 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:01:12.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 Processor -- start 2021-11-21T22:01:12.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- start start 2021-11-21T22:01:12.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e880a6aa0 0x7f0e8814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 0x7f0e881470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e880a6e80 con 0x7f0e881475d0 2021-11-21T22:01:12.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e880a8bc0 con 0x7f0e880a6aa0 2021-11-21T22:01:12.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8814a7b0 con 0x7f0e880a86b0 2021-11-21T22:01:12.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e880a6aa0 0x7f0e8814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e880a6aa0 0x7f0e8814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50258/0 (socket says 172.21.2.116:50258) 2021-11-21T22:01:12.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 -- 172.21.2.116:0/4178905453 learned_addr learned my addr 172.21.2.116:0/4178905453 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 -- 172.21.2.116:0/4178905453 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 msgr2=0x7f0e881470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:12.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 --2- 172.21.2.116:0/4178905453 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 0x7f0e881470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 -- 172.21.2.116:0/4178905453 >> v1:172.21.2.117:6789/0 conn(0x7f0e881475d0 legacy=0x7f0e881443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:12.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 -- 172.21.2.116:0/4178905453 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8814aa10 con 0x7f0e880a6aa0 2021-11-21T22:01:12.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8d278700 1 --2- 172.21.2.116:0/4178905453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e8814b150 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f0e88061ab0 tx=0x7f0e78006eb0).ready entity=mon.0 client_cookie=123142817745d23f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e77fff700 1 -- 172.21.2.116:0/4178905453 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7800d570 con 0x7f0e880a6aa0 2021-11-21T22:01:12.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e77fff700 1 -- 172.21.2.116:0/4178905453 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78000e90 con 0x7f0e880a6aa0 2021-11-21T22:01:12.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e77fff700 1 -- 172.21.2.116:0/4178905453 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7800d8c0 con 0x7f0e880a6aa0 2021-11-21T22:01:12.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/4178905453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 msgr2=0x7f0e8814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:12.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/4178905453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e8814b150 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f0e88061ab0 tx=0x7f0e78006eb0).stop 2021-11-21T22:01:12.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/4178905453 shutdown_connections 2021-11-21T22:01:12.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/4178905453 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 0x7f0e881470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/4178905453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e8814b150 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:12.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/4178905453 >> 172.21.2.116:0/4178905453 conn(0x7f0e8805bd20 msgr2=0x7f0e8805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:12.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/4178905453 shutdown_connections 2021-11-21T22:01:12.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.954+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/4178905453 wait complete. 2021-11-21T22:01:12.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 Processor -- start 2021-11-21T22:01:12.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 -- start start 2021-11-21T22:01:12.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e88065da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 0x7f0e88156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e881475d0 0x7f0e880a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88065be0 con 0x7f0e880a6aa0 2021-11-21T22:01:12.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88156510 con 0x7f0e881475d0 2021-11-21T22:01:12.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e881567b0 con 0x7f0e880a86b0 2021-11-21T22:01:12.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e88065da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e88065da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50264/0 (socket says 172.21.2.116:50264) 2021-11-21T22:01:12.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 -- 172.21.2.116:0/3178908089 learned_addr learned my addr 172.21.2.116:0/3178908089 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:12.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 -- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 msgr2=0x7f0e88156ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:12.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 0x7f0e88156ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 -- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e881475d0 msgr2=0x7f0e880a8ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:12.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e881475d0 0x7f0e880a8ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:12.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 -- 172.21.2.116:0/3178908089 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7800b1e0 con 0x7f0e880a6aa0 2021-11-21T22:01:12.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8d278700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e88065da0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f0e88061ab0 tx=0x7f0e78015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e767fc700 1 -- 172.21.2.116:0/3178908089 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e78006b60 con 0x7f0e880a6aa0 2021-11-21T22:01:12.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e767fc700 1 -- 172.21.2.116:0/3178908089 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78002f70 con 0x7f0e880a6aa0 2021-11-21T22:01:12.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e88156a50 con 0x7f0e880a6aa0 2021-11-21T22:01:12.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e767fc700 1 -- 172.21.2.116:0/3178908089 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7800dea0 con 0x7f0e880a6aa0 2021-11-21T22:01:12.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e88146a10 con 0x7f0e880a6aa0 2021-11-21T22:01:12.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e880573d0 con 0x7f0e880a6aa0 2021-11-21T22:01:12.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e767fc700 1 -- 172.21.2.116:0/3178908089 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0e78025030 con 0x7f0e880a6aa0 2021-11-21T22:01:12.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e767fc700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e68053610 0x7f0e68055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:12.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e767fc700 1 -- 172.21.2.116:0/3178908089 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f0e7806ce90 con 0x7f0e880a6aa0 2021-11-21T22:01:12.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.958+0000 7f0e8ca77700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e68053610 0x7f0e68055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:12.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.962+0000 7f0e8ca77700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e68053610 0x7f0e68055ad0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f0e80003330 tx=0x7f0e80006cd0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:12.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:12.962+0000 7f0e767fc700 1 -- 172.21.2.116:0/3178908089 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e78043e90 con 0x7f0e880a6aa0 2021-11-21T22:01:13.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e88156be0 con 0x7f0e880a6aa0 2021-11-21T22:01:13.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e767fc700 1 -- 172.21.2.116:0/3178908089 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f0e78023070 con 0x7f0e880a6aa0 2021-11-21T22:01:13.075 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:13.075 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:03.447073+0000","modified":"2021-11-21T22:01:12.895424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-11-21T22:01:13.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e68053610 msgr2=0x7f0e68055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e68053610 0x7f0e68055ad0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f0e80003330 tx=0x7f0e80006cd0).stop 2021-11-21T22:01:13.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 msgr2=0x7f0e88065da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e88065da0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f0e88061ab0 tx=0x7f0e78015000).stop 2021-11-21T22:01:13.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 shutdown_connections 2021-11-21T22:01:13.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e881475d0 0x7f0e880a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e880a86b0 0x7f0e88156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e68053610 0x7f0e68055ad0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:13.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 --2- 172.21.2.116:0/3178908089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e880a6aa0 0x7f0e88065da0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:13.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 >> 172.21.2.116:0/3178908089 conn(0x7f0e8805bd20 msgr2=0x7f0e88149b00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:13.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 shutdown_connections 2021-11-21T22:01:13.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.070+0000 7f0e8f4dc700 1 -- 172.21.2.116:0/3178908089 wait complete. 2021-11-21T22:01:13.082 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 39 2021-11-21T22:01:13.084 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:01:13.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b71d6e700 1 Processor -- start 2021-11-21T22:01:13.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b71d6e700 1 -- start start 2021-11-21T22:01:13.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b71d6e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 0x7f8b6c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b71d6e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8b6c0abce0 0x7f8b6c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b71d6e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6c147e80 con 0x7f8b6c0b1db0 2021-11-21T22:01:13.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b71d6e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c0b1490 con 0x7f8b6c0abce0 2021-11-21T22:01:13.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b71d6e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0af060 con 0x7f8b6c144810 2021-11-21T22:01:13.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8b6c0abce0 0x7f8b6c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:13.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8b6c0abce0 0x7f8b6c0b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50276/0 (socket says 172.21.2.116:50276) 2021-11-21T22:01:13.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 -- 172.21.2.116:0/3249656604 learned_addr learned my addr 172.21.2.116:0/3249656604 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:13.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 -- 172.21.2.116:0/3249656604 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 msgr2=0x7f8b6c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:13.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 --2- 172.21.2.116:0/3249656604 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 0x7f8b6c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 -- 172.21.2.116:0/3249656604 >> v1:172.21.2.116:6789/0 conn(0x7f8b6c0b1db0 legacy=0x7f8b6c0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 -- 172.21.2.116:0/3249656604 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c0af5a0 con 0x7f8b6c0abce0 2021-11-21T22:01:13.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6affd700 1 --2- 172.21.2.116:0/3249656604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0abce0 0x7f8b6c0b18c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f8b54002f70 tx=0x7f8b54006eb0).ready entity=mon.0 client_cookie=229a141358d02d48 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:13.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6a7fc700 1 -- 172.21.2.116:0/3249656604 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5400d5c0 con 0x7f8b6c0abce0 2021-11-21T22:01:13.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6a7fc700 1 -- 172.21.2.116:0/3249656604 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54000e90 con 0x7f8b6c0abce0 2021-11-21T22:01:13.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.186+0000 7f8b6a7fc700 1 -- 172.21.2.116:0/3249656604 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5400d950 con 0x7f8b6c0abce0 2021-11-21T22:01:13.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/3249656604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0abce0 msgr2=0x7f8b6c0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/3249656604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0abce0 0x7f8b6c0b18c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f8b54002f70 tx=0x7f8b54006eb0).stop 2021-11-21T22:01:13.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/3249656604 shutdown_connections 2021-11-21T22:01:13.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/3249656604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0abce0 0x7f8b6c0b18c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:13.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/3249656604 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 0x7f8b6c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/3249656604 >> 172.21.2.116:0/3249656604 conn(0x7f8b6c05bd20 msgr2=0x7f8b6c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:13.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/3249656604 shutdown_connections 2021-11-21T22:01:13.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/3249656604 wait complete. 2021-11-21T22:01:13.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 Processor -- start 2021-11-21T22:01:13.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- start start 2021-11-21T22:01:13.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b6c0abce0 0x7f8b6c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 0x7f8b6c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0b4860 0x7f8b6c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c143f20 con 0x7f8b6c0b4860 2021-11-21T22:01:13.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1439e0 con 0x7f8b6c0abce0 2021-11-21T22:01:13.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c143c80 con 0x7f8b6c144810 2021-11-21T22:01:13.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0b4860 0x7f8b6c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:13.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0b4860 0x7f8b6c15a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50280/0 (socket says 172.21.2.116:50280) 2021-11-21T22:01:13.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 -- 172.21.2.116:0/828407084 learned_addr learned my addr 172.21.2.116:0/828407084 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:13.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 -- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 msgr2=0x7f8b6c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 0x7f8b6c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 -- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b6c0abce0 msgr2=0x7f8b6c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b6c0abce0 0x7f8b6c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 -- 172.21.2.116:0/828407084 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5400b1e0 con 0x7f8b6c0b4860 2021-11-21T22:01:13.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b6bfff700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0b4860 0x7f8b6c15a850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f8b6c0b2a30 tx=0x7f8b64007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:13.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b68ff9700 1 -- 172.21.2.116:0/828407084 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b6400e4a0 con 0x7f8b6c0b4860 2021-11-21T22:01:13.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b68ff9700 1 -- 172.21.2.116:0/828407084 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6400e600 con 0x7f8b6c0b4860 2021-11-21T22:01:13.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b68ff9700 1 -- 172.21.2.116:0/828407084 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b6400e960 con 0x7f8b6c0b4860 2021-11-21T22:01:13.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c159ec0 con 0x7f8b6c0b4860 2021-11-21T22:01:13.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c144fd0 con 0x7f8b6c0b4860 2021-11-21T22:01:13.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6c15a050 con 0x7f8b6c0b4860 2021-11-21T22:01:13.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.190+0000 7f8b68ff9700 1 -- 172.21.2.116:0/828407084 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8b64019030 con 0x7f8b6c0b4860 2021-11-21T22:01:13.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.194+0000 7f8b68ff9700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b500536f0 0x7f8b50055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.194+0000 7f8b68ff9700 1 -- 172.21.2.116:0/828407084 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8b6406b2a0 con 0x7f8b6c0b4860 2021-11-21T22:01:13.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.194+0000 7f8b6b7fe700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b500536f0 0x7f8b50055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:13.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.194+0000 7f8b6b7fe700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b500536f0 0x7f8b50055bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f8b5c002f70 tx=0x7f8b5c0069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:13.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.194+0000 7f8b68ff9700 1 -- 172.21.2.116:0/828407084 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b640422a0 con 0x7f8b6c0b4860 2021-11-21T22:01:13.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.302+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8b6c0af300 con 0x7f8b6c0b4860 2021-11-21T22:01:13.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b68ff9700 1 -- 172.21.2.116:0/828407084 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b640458c0 con 0x7f8b6c0b4860 2021-11-21T22:01:13.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b500536f0 msgr2=0x7f8b50055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b500536f0 0x7f8b50055bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f8b5c002f70 tx=0x7f8b5c0069e0).stop 2021-11-21T22:01:13.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0b4860 msgr2=0x7f8b6c15a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0b4860 0x7f8b6c15a850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f8b6c0b2a30 tx=0x7f8b64007f10).stop 2021-11-21T22:01:13.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 shutdown_connections 2021-11-21T22:01:13.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b500536f0 0x7f8b50055bb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:13.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b6c0b4860 0x7f8b6c15a850 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:13.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b6c144810 0x7f8b6c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 --2- 172.21.2.116:0/828407084 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b6c0abce0 0x7f8b6c0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 >> 172.21.2.116:0/828407084 conn(0x7f8b6c05bd20 msgr2=0x7f8b6c0b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:13.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 shutdown_connections 2021-11-21T22:01:13.916 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.906+0000 7f8b71d6e700 1 -- 172.21.2.116:0/828407084 wait complete. 2021-11-21T22:01:13.918 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:01:13.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 Processor -- start 2021-11-21T22:01:13.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- start start 2021-11-21T22:01:13.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fddbc14fbe0 0x7fddbc151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc0b1940 con 0x7fddbc0a8670 2021-11-21T22:01:13.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc0b15a0 con 0x7fddbc0b11c0 2021-11-21T22:01:13.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddbc0b1b70 con 0x7fddbc14fbe0 2021-11-21T22:01:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddbbfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fddbc14fbe0 0x7fddbc151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddbbfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fddbc14fbe0 0x7fddbc151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50286/0 (socket says 172.21.2.116:50286) 2021-11-21T22:01:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddbbfff700 1 -- 172.21.2.116:0/3249146918 learned_addr learned my addr 172.21.2.116:0/3249146918 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddbbfff700 1 -- 172.21.2.116:0/3249146918 >> v1:172.21.2.117:6789/0 conn(0x7fddbc0b11c0 legacy=0x7fddbc147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddbbfff700 1 -- 172.21.2.116:0/3249146918 >> v1:172.21.2.116:6789/0 conn(0x7fddbc0a8670 legacy=0x7fddbc147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddbbfff700 1 -- 172.21.2.116:0/3249146918 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc0b1e10 con 0x7fddbc14fbe0 2021-11-21T22:01:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddbbfff700 1 --2- 172.21.2.116:0/3249146918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc14fbe0 0x7fddbc151fd0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fddbc061da0 tx=0x7fddb4009290).ready entity=mon.0 client_cookie=c3c64e91118a4238 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddba7fc700 1 -- 172.21.2.116:0/3249146918 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddb400b860 con 0x7fddbc14fbe0 2021-11-21T22:01:13.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddba7fc700 1 -- 172.21.2.116:0/3249146918 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb400b9c0 con 0x7fddbc14fbe0 2021-11-21T22:01:13.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddba7fc700 1 -- 172.21.2.116:0/3249146918 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddb400bb30 con 0x7fddbc14fbe0 2021-11-21T22:01:13.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- 172.21.2.116:0/3249146918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc14fbe0 msgr2=0x7fddbc151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 --2- 172.21.2.116:0/3249146918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc14fbe0 0x7fddbc151fd0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fddbc061da0 tx=0x7fddb4009290).stop 2021-11-21T22:01:13.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- 172.21.2.116:0/3249146918 shutdown_connections 2021-11-21T22:01:13.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 --2- 172.21.2.116:0/3249146918 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc14fbe0 0x7fddbc151fd0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:13.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- 172.21.2.116:0/3249146918 >> 172.21.2.116:0/3249146918 conn(0x7fddbc05bd20 msgr2=0x7fddbc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:13.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- 172.21.2.116:0/3249146918 shutdown_connections 2021-11-21T22:01:13.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- 172.21.2.116:0/3249146918 wait complete. 2021-11-21T22:01:13.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 Processor -- start 2021-11-21T22:01:13.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.970+0000 7fddc2164700 1 -- start start 2021-11-21T22:01:13.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fddbc0a8670 0x7fddbc146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc0b11c0 0x7fddbc13d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fddbc14fbe0 0x7fddbc1440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc156610 con 0x7fddbc0b11c0 2021-11-21T22:01:13.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc156800 con 0x7fddbc0a8670 2021-11-21T22:01:13.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc156aa0 con 0x7fddbc14fbe0 2021-11-21T22:01:13.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc0b11c0 0x7fddbc13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:13.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc0b11c0 0x7fddbc13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50294/0 (socket says 172.21.2.116:50294) 2021-11-21T22:01:13.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 -- 172.21.2.116:0/3221889857 learned_addr learned my addr 172.21.2.116:0/3221889857 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:13.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 -- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fddbc14fbe0 msgr2=0x7fddbc1440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fddbc14fbe0 0x7fddbc1440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 -- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fddbc0a8670 msgr2=0x7fddbc146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:13.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fddbc0a8670 0x7fddbc146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:13.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 -- 172.21.2.116:0/3221889857 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb4009530 con 0x7fddbc0b11c0 2021-11-21T22:01:13.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbaffd700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc0b11c0 0x7fddbc13d340 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7fddbc068600 tx=0x7fddac007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:13.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddb8ff9700 1 -- 172.21.2.116:0/3221889857 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddac00ce30 con 0x7fddbc0b11c0 2021-11-21T22:01:13.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddb8ff9700 1 -- 172.21.2.116:0/3221889857 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac00f470 con 0x7fddbc0b11c0 2021-11-21T22:01:13.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddb8ff9700 1 -- 172.21.2.116:0/3221889857 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddac00f7d0 con 0x7fddbc0b11c0 2021-11-21T22:01:13.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc156d40 con 0x7fddbc0b11c0 2021-11-21T22:01:13.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc157480 con 0x7fddbc0b11c0 2021-11-21T22:01:13.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc0b20b0 con 0x7fddbc0b11c0 2021-11-21T22:01:13.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddb8ff9700 1 -- 172.21.2.116:0/3221889857 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fddac019030 con 0x7fddbc0b11c0 2021-11-21T22:01:13.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddb8ff9700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdda00536a0 0x7fdda0055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:13.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddb8ff9700 1 -- 172.21.2.116:0/3221889857 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fddac06ae60 con 0x7fddbc0b11c0 2021-11-21T22:01:13.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbb7fe700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdda00536a0 0x7fdda0055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:13.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.974+0000 7fddbb7fe700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdda00536a0 0x7fdda0055b60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fdda4002f90 tx=0x7fdda40069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:13.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:13.978+0000 7fddb8ff9700 1 -- 172.21.2.116:0/3221889857 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fddac045e60 con 0x7fddbc0b11c0 2021-11-21T22:01:14.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.062+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fddbc0573d0 con 0x7fddbc0b11c0 2021-11-21T22:01:14.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddb8ff9700 1 -- 172.21.2.116:0/3221889857 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 185+0+0 (secure 0 0 0) 0x7fddac0459a0 con 0x7fddbc0b11c0 2021-11-21T22:01:14.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdda00536a0 msgr2=0x7fdda0055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:14.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdda00536a0 0x7fdda0055b60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fdda4002f90 tx=0x7fdda40069b0).stop 2021-11-21T22:01:14.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc0b11c0 msgr2=0x7fddbc13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:14.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc0b11c0 0x7fddbc13d340 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7fddbc068600 tx=0x7fddac007c50).stop 2021-11-21T22:01:14.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 shutdown_connections 2021-11-21T22:01:14.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fddbc14fbe0 0x7fddbc1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:14.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdda00536a0 0x7fdda0055b60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:14.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fddbc0b11c0 0x7fddbc13d340 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:14.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 --2- 172.21.2.116:0/3221889857 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fddbc0a8670 0x7fddbc146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:14.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 >> 172.21.2.116:0/3221889857 conn(0x7fddbc05bd20 msgr2=0x7fddbc13e310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:14.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 shutdown_connections 2021-11-21T22:01:14.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.922+0000 7fddc2164700 1 -- 172.21.2.116:0/3221889857 wait complete. 2021-11-21T22:01:14.930 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:01:14.935 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:01:14.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 Processor -- start 2021-11-21T22:01:14.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- start start 2021-11-21T22:01:14.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0b24a0 0x7f6dbc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:14.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6dbc0af360 0x7f6dbc0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:14.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6dbc0aeb70 0x7f6dbc13c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:15.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0adba0 con 0x7f6dbc0aeb70 2021-11-21T22:01:15.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0af7a0 con 0x7f6dbc0af360 2021-11-21T22:01:15.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b00d0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6dbc0aeb70 0x7f6dbc13c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:15.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6dbc0aeb70 0x7f6dbc13c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50302/0 (socket says 172.21.2.116:50302) 2021-11-21T22:01:15.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/1915602937 learned_addr learned my addr 172.21.2.116:0/1915602937 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:15.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbaffd700 1 --2- 172.21.2.116:0/1915602937 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0b24a0 0x7f6dbc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:15.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dba7fc700 1 --2- 172.21.2.116:0/1915602937 >> v2:172.21.2.117:3300/0 conn(0x7f6dbc0af360 0x7f6dbc0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:15.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/1915602937 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0b24a0 msgr2=0x7f6dbc0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:15.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 --2- 172.21.2.116:0/1915602937 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0b24a0 0x7f6dbc0b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/1915602937 >> v2:172.21.2.117:3300/0 conn(0x7f6dbc0af360 msgr2=0x7f6dbc0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:15.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 --2- 172.21.2.116:0/1915602937 >> v2:172.21.2.117:3300/0 conn(0x7f6dbc0af360 0x7f6dbc0ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/1915602937 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0b2880 con 0x7f6dbc0aeb70 2021-11-21T22:01:15.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dbb7fe700 1 --2- 172.21.2.116:0/1915602937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0aeb70 0x7f6dbc13c210 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f6db40032f0 tx=0x7f6db400b530).ready entity=mon.0 client_cookie=90835c717e492d16 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:15.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6db9ffb700 1 -- 172.21.2.116:0/1915602937 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db40153f0 con 0x7f6dbc0aeb70 2021-11-21T22:01:15.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6db9ffb700 1 -- 172.21.2.116:0/1915602937 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db4015550 con 0x7f6dbc0aeb70 2021-11-21T22:01:15.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6db9ffb700 1 -- 172.21.2.116:0/1915602937 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db40158b0 con 0x7f6dbc0aeb70 2021-11-21T22:01:15.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- 172.21.2.116:0/1915602937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0aeb70 msgr2=0x7f6dbc13c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:15.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 --2- 172.21.2.116:0/1915602937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0aeb70 0x7f6dbc13c210 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f6db40032f0 tx=0x7f6db400b530).stop 2021-11-21T22:01:15.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- 172.21.2.116:0/1915602937 shutdown_connections 2021-11-21T22:01:15.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 --2- 172.21.2.116:0/1915602937 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0aeb70 0x7f6dbc13c210 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 --2- 172.21.2.116:0/1915602937 >> v2:172.21.2.117:3300/0 conn(0x7f6dbc0af360 0x7f6dbc0ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 --2- 172.21.2.116:0/1915602937 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0b24a0 0x7f6dbc0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- 172.21.2.116:0/1915602937 >> 172.21.2.116:0/1915602937 conn(0x7f6dbc05fd00 msgr2=0x7f6dbc0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:15.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- 172.21.2.116:0/1915602937 shutdown_connections 2021-11-21T22:01:15.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.986+0000 7f6dc1810700 1 -- 172.21.2.116:0/1915602937 wait complete. 2021-11-21T22:01:15.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 Processor -- start 2021-11-21T22:01:15.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 -- start start 2021-11-21T22:01:15.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0aeb70 0x7f6dbc142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:15.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6dbc0af360 0x7f6dbc13afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:15.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0b24a0 0x7f6dbc13b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:15.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc1115c0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc110be0 con 0x7f6dbc0af360 2021-11-21T22:01:15.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc110e80 con 0x7f6dbc0aeb70 2021-11-21T22:01:15.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0b24a0 0x7f6dbc13b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:15.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0b24a0 0x7f6dbc13b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50310/0 (socket says 172.21.2.116:50310) 2021-11-21T22:01:15.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/2417692781 learned_addr learned my addr 172.21.2.116:0/2417692781 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:15.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dba7fc700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6dbc0af360 0x7f6dbc13afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:15.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbaffd700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0aeb70 0x7f6dbc142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:15.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0aeb70 msgr2=0x7f6dbc142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:15.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0aeb70 0x7f6dbc142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6dbc0af360 msgr2=0x7f6dbc13afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:15.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6dbc0af360 0x7f6dbc13afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 -- 172.21.2.116:0/2417692781 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db400b7d0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dba7fc700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6dbc0af360 0x7f6dbc13afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:01:15.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbaffd700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0aeb70 0x7f6dbc142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:01:15.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbb7fe700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0b24a0 0x7f6dbc13b4b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f6db40032f0 tx=0x7f6db4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:15.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6db3fff700 1 -- 172.21.2.116:0/2417692781 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db40153f0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6db3fff700 1 -- 172.21.2.116:0/2417692781 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db400b1f0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc111120 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc13b9a0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6db3fff700 1 -- 172.21.2.116:0/2417692781 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db4014e20 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc1112b0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6db3fff700 1 -- 172.21.2.116:0/2417692781 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6db4025030 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6db3fff700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6da0053540 0x7f6da0055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:15.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6db3fff700 1 -- 172.21.2.116:0/2417692781 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f6db400b030 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbaffd700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6da0053540 0x7f6da0055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:15.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.990+0000 7f6dbaffd700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6da0053540 0x7f6da0055a00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f6dac00b000 tx=0x7f6dac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:15.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:14.994+0000 7f6db3fff700 1 -- 172.21.2.116:0/2417692781 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6db40442e0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.078+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6dbc0573d0 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.934+0000 7f6db3fff700 1 -- 172.21.2.116:0/2417692781 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 173+0+0 (secure 0 0 0) 0x7f6db4047900 con 0x7f6dbc0b24a0 2021-11-21T22:01:15.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6da0053540 msgr2=0x7f6da0055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:15.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6da0053540 0x7f6da0055a00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f6dac00b000 tx=0x7f6dac008040).stop 2021-11-21T22:01:15.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0b24a0 msgr2=0x7f6dbc13b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:15.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0b24a0 0x7f6dbc13b4b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f6db40032f0 tx=0x7f6db4019000).stop 2021-11-21T22:01:15.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 shutdown_connections 2021-11-21T22:01:15.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6da0053540 0x7f6da0055a00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6dbc0b24a0 0x7f6dbc13b4b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6dbc0af360 0x7f6dbc13afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 --2- 172.21.2.116:0/2417692781 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6dbc0aeb70 0x7f6dbc142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:15.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 >> 172.21.2.116:0/2417692781 conn(0x7f6dbc05fd00 msgr2=0x7f6dbc0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:15.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 shutdown_connections 2021-11-21T22:01:15.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.938+0000 7f6dc1810700 1 -- 172.21.2.116:0/2417692781 wait complete. 2021-11-21T22:01:15.944 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:01:15.950 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T22:01:16.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.998+0000 7fabadb82700 1 Processor -- start 2021-11-21T22:01:16.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.998+0000 7fabadb82700 1 -- start start 2021-11-21T22:01:16.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:15.998+0000 7fabadb82700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7faba81478c0 0x7faba80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba8147e80 con 0x7faba8144810 2021-11-21T22:01:16.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba80af5a0 con 0x7faba80abce0 2021-11-21T22:01:16.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba80b1490 con 0x7faba81478c0 2021-11-21T22:01:16.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7faba8144810 0x7faba80b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34038/0 (socket says 172.21.2.116:34038) 2021-11-21T22:01:16.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/478491198 learned_addr learned my addr 172.21.2.116:0/478491198 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:16.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114804282 0 0) 0x7faba8147e80 con 0x7faba8144810 2021-11-21T22:01:16.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab94003260 con 0x7faba8144810 2021-11-21T22:01:16.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762414592 0 0) 0x7fab94003260 con 0x7faba8144810 2021-11-21T22:01:16.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 >> v2:172.21.2.117:3300/0 conn(0x7faba81478c0 msgr2=0x7faba80acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:16.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 --2- 172.21.2.116:0/478491198 >> v2:172.21.2.117:3300/0 conn(0x7faba81478c0 0x7faba80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 >> v1:172.21.2.117:6789/0 conn(0x7faba80abce0 legacy=0x7faba80ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:16.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80b1fd0 con 0x7faba8144810 2021-11-21T22:01:16.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fab9c001660 con 0x7faba8144810 2021-11-21T22:01:16.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 >> v1:172.21.2.116:6789/0 conn(0x7faba8144810 legacy=0x7faba80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7fab94003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab94004500 0x7fab94004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94005050 0x7fab9400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80b1490 con 0x7fab94003820 2021-11-21T22:01:16.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9400bf40 con 0x7fab94004500 2021-11-21T22:01:16.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9400c100 con 0x7fab94005050 2021-11-21T22:01:16.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7fab94003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:16.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/478491198 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94005050 msgr2=0x7fab9400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:16.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94005050 0x7fab9400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/478491198 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab94004500 msgr2=0x7fab94004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:16.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab94004500 0x7fab94004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/478491198 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab94003260 con 0x7fab94003820 2021-11-21T22:01:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba7fff700 1 -- 172.21.2.116:0/478491198 reap_dead start 2021-11-21T22:01:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7fab94003c20 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7faba80b4e00 tx=0x7faba80aee70).ready entity=mon.0 client_cookie=e638e7ecab24ce26 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c0070d0 con 0x7fab94003820 2021-11-21T22:01:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c007230 con 0x7fab94003820 2021-11-21T22:01:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba67fc700 1 -- 172.21.2.116:0/478491198 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c00c8f0 con 0x7fab94003820 2021-11-21T22:01:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- 172.21.2.116:0/478491198 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 msgr2=0x7fab94003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7fab94003c20 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7faba80b4e00 tx=0x7faba80aee70).stop 2021-11-21T22:01:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- 172.21.2.116:0/478491198 shutdown_connections 2021-11-21T22:01:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 --2- 172.21.2.116:0/478491198 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7fab94003c20 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- 172.21.2.116:0/478491198 >> 172.21.2.116:0/478491198 conn(0x7faba805bd20 msgr2=0x7faba805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- 172.21.2.116:0/478491198 shutdown_connections 2021-11-21T22:01:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- 172.21.2.116:0/478491198 wait complete. 2021-11-21T22:01:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 Processor -- start 2021-11-21T22:01:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- start start 2021-11-21T22:01:16.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7faba80b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba815a160 0x7faba815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba80642f0 0x7faba8064770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba815aad0 con 0x7fab94003820 2021-11-21T22:01:16.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba8159b90 con 0x7faba80642f0 2021-11-21T22:01:16.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba80b3340 con 0x7faba815a160 2021-11-21T22:01:16.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7faba80b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:16.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7faba80b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50326/0 (socket says 172.21.2.116:50326) 2021-11-21T22:01:16.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/950492112 learned_addr learned my addr 172.21.2.116:0/950492112 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:16.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba815a160 msgr2=0x7faba815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba6ffd700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba815a160 0x7faba815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:16.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba815a160 0x7faba815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba80642f0 msgr2=0x7faba8064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba80642f0 0x7faba8064770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 -- 172.21.2.116:0/950492112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8062b50 con 0x7fab94003820 2021-11-21T22:01:16.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba6ffd700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba815a160 0x7faba815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:16.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba77fe700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7faba80b0a80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7faba80b4e00 tx=0x7fab9c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:16.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba4ff9700 1 -- 172.21.2.116:0/950492112 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c01a070 con 0x7fab94003820 2021-11-21T22:01:16.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba4ff9700 1 -- 172.21.2.116:0/950492112 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c004920 con 0x7fab94003820 2021-11-21T22:01:16.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba80b2b50 con 0x7fab94003820 2021-11-21T22:01:16.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7faba4ff9700 1 -- 172.21.2.116:0/950492112 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c0070d0 con 0x7fab94003820 2021-11-21T22:01:16.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.002+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba8146680 con 0x7fab94003820 2021-11-21T22:01:16.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.006+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba80b2ce0 con 0x7fab94003820 2021-11-21T22:01:16.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.006+0000 7faba4ff9700 1 -- 172.21.2.116:0/950492112 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fab9c010070 con 0x7fab94003820 2021-11-21T22:01:16.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.006+0000 7faba4ff9700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab880535b0 0x7fab88055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.006+0000 7faba4ff9700 1 -- 172.21.2.116:0/950492112 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fab9c06b8f0 con 0x7fab94003820 2021-11-21T22:01:16.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.006+0000 7faba6ffd700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab880535b0 0x7fab88055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:16.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.006+0000 7faba6ffd700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab880535b0 0x7fab88055a70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7faba0007000 tx=0x7faba0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:16.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.006+0000 7faba4ff9700 1 -- 172.21.2.116:0/950492112 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab9c042c20 con 0x7fab94003820 2021-11-21T22:01:16.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.090+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faba8067cc0 con 0x7fab94003820 2021-11-21T22:01:16.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7faba4ff9700 1 -- 172.21.2.116:0/950492112 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fab9c046240 con 0x7fab94003820 2021-11-21T22:01:16.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab880535b0 msgr2=0x7fab88055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab880535b0 0x7fab88055a70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7faba0007000 tx=0x7faba0008040).stop 2021-11-21T22:01:16.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 msgr2=0x7faba80b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7faba80b0a80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7faba80b4e00 tx=0x7fab9c013000).stop 2021-11-21T22:01:16.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 shutdown_connections 2021-11-21T22:01:16.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faba80642f0 0x7faba8064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faba815a160 0x7faba815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab880535b0 0x7fab88055a70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 --2- 172.21.2.116:0/950492112 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003820 0x7faba80b0a80 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 >> 172.21.2.116:0/950492112 conn(0x7faba805bd20 msgr2=0x7faba8148280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:16.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 shutdown_connections 2021-11-21T22:01:16.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.098+0000 7fabadb82700 1 -- 172.21.2.116:0/950492112 wait complete. 2021-11-21T22:01:16.108 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2021-11-21T22:01:16.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45f743700 1 Processor -- start 2021-11-21T22:01:16.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45f743700 1 -- start start 2021-11-21T22:01:16.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45f743700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580b24a0 0x7ff4580b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45f743700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff4580af360 0x7ff4580ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45f743700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4580b00d0 con 0x7ff4580aeb70 2021-11-21T22:01:16.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45f743700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4580adba0 con 0x7ff4580af360 2021-11-21T22:01:16.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45f743700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580af7a0 con 0x7ff4580b24a0 2021-11-21T22:01:16.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45ccde700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff4580af360 0x7ff4580ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:16.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45ccde700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff4580af360 0x7ff4580ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50336/0 (socket says 172.21.2.116:50336) 2021-11-21T22:01:16.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.154+0000 7ff45ccde700 1 -- 172.21.2.116:0/423534309 learned_addr learned my addr 172.21.2.116:0/423534309 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:16.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 -- 172.21.2.116:0/423534309 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580b24a0 msgr2=0x7ff4580b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:16.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 --2- 172.21.2.116:0/423534309 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580b24a0 0x7ff4580b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 -- 172.21.2.116:0/423534309 >> v1:172.21.2.117:6789/0 conn(0x7ff4580aeb70 legacy=0x7ff45813c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:16.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 -- 172.21.2.116:0/423534309 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4580b2880 con 0x7ff4580af360 2021-11-21T22:01:16.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 --2- 172.21.2.116:0/423534309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff4580ae680 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7ff4580630d0 tx=0x7ff44c006e40).ready entity=mon.0 client_cookie=291053c7b997fb96 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:16.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff44bfff700 1 -- 172.21.2.116:0/423534309 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff44c00bce0 con 0x7ff4580af360 2021-11-21T22:01:16.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff44bfff700 1 -- 172.21.2.116:0/423534309 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c000e90 con 0x7ff4580af360 2021-11-21T22:01:16.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff44bfff700 1 -- 172.21.2.116:0/423534309 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff44c0155e0 con 0x7ff4580af360 2021-11-21T22:01:16.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/423534309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 msgr2=0x7ff4580ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 --2- 172.21.2.116:0/423534309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff4580ae680 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7ff4580630d0 tx=0x7ff44c006e40).stop 2021-11-21T22:01:16.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/423534309 shutdown_connections 2021-11-21T22:01:16.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 --2- 172.21.2.116:0/423534309 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff4580ae680 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 --2- 172.21.2.116:0/423534309 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580b24a0 0x7ff4580b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/423534309 >> 172.21.2.116:0/423534309 conn(0x7ff45805fd00 msgr2=0x7ff4580600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:16.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/423534309 shutdown_connections 2021-11-21T22:01:16.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/423534309 wait complete. 2021-11-21T22:01:16.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 Processor -- start 2021-11-21T22:01:16.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- start start 2021-11-21T22:01:16.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580aeb70 0x7ff458142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff458140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4580b24a0 0x7ff45813f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581566c0 con 0x7ff4580af360 2021-11-21T22:01:16.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff458156960 con 0x7ff4580b24a0 2021-11-21T22:01:16.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff458156c00 con 0x7ff4580aeb70 2021-11-21T22:01:16.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff458140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:16.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff458140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50344/0 (socket says 172.21.2.116:50344) 2021-11-21T22:01:16.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 -- 172.21.2.116:0/551882771 learned_addr learned my addr 172.21.2.116:0/551882771 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:16.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 -- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580aeb70 msgr2=0x7ff458142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580aeb70 0x7ff458142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 -- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4580b24a0 msgr2=0x7ff45813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4580b24a0 0x7ff45813f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 -- 172.21.2.116:0/551882771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c00d4d0 con 0x7ff4580af360 2021-11-21T22:01:16.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45ccde700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff458140ab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7ff4580630d0 tx=0x7ff44c018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:16.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff44a7fc700 1 -- 172.21.2.116:0/551882771 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff44c00b040 con 0x7ff4580af360 2021-11-21T22:01:16.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff44a7fc700 1 -- 172.21.2.116:0/551882771 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c00be90 con 0x7ff4580af360 2021-11-21T22:01:16.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff44a7fc700 1 -- 172.21.2.116:0/551882771 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff44c015ea0 con 0x7ff4580af360 2021-11-21T22:01:16.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff458156ea0 con 0x7ff4580af360 2021-11-21T22:01:16.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4580667b0 con 0x7ff4580af360 2021-11-21T22:01:16.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff458157030 con 0x7ff4580af360 2021-11-21T22:01:16.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.158+0000 7ff44a7fc700 1 -- 172.21.2.116:0/551882771 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff44c025020 con 0x7ff4580af360 2021-11-21T22:01:16.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.162+0000 7ff44a7fc700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff438053660 0x7ff438055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:16.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.162+0000 7ff44a7fc700 1 -- 172.21.2.116:0/551882771 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff44c06c7f0 con 0x7ff4580af360 2021-11-21T22:01:16.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.162+0000 7ff45d4df700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff438053660 0x7ff438055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:16.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.162+0000 7ff45d4df700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff438053660 0x7ff438055b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7ff454002f70 tx=0x7ff4540069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:16.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.162+0000 7ff44a7fc700 1 -- 172.21.2.116:0/551882771 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff44c043c30 con 0x7ff4580af360 2021-11-21T22:01:16.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.274+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7ff458157030 con 0x7ff4580af360 2021-11-21T22:01:16.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.942+0000 7ff44a7fc700 1 -- 172.21.2.116:0/551882771 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v290) v1 ==== 126+0+0 (secure 0 0 0) 0x7ff44c047250 con 0x7ff4580af360 2021-11-21T22:01:16.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff438053660 msgr2=0x7ff438055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff438053660 0x7ff438055b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7ff454002f70 tx=0x7ff4540069e0).stop 2021-11-21T22:01:16.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 msgr2=0x7ff458140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:16.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff458140ab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7ff4580630d0 tx=0x7ff44c018000).stop 2021-11-21T22:01:16.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 shutdown_connections 2021-11-21T22:01:16.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff4580b24a0 0x7ff45813f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff438053660 0x7ff438055b20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff4580af360 0x7ff458140ab0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:16.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 --2- 172.21.2.116:0/551882771 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff4580aeb70 0x7ff458142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:16.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 >> 172.21.2.116:0/551882771 conn(0x7ff45805fd00 msgr2=0x7ff4580b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:16.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 shutdown_connections 2021-11-21T22:01:16.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:16.946+0000 7ff45f743700 1 -- 172.21.2.116:0/551882771 wait complete. 2021-11-21T22:01:16.956 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2021-11-21T22:01:16.957 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:01:16.957 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2021-11-21T22:01:17.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.002+0000 7fc9d39c3700 1 Processor -- start 2021-11-21T22:01:17.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.002+0000 7fc9d39c3700 1 -- start start 2021-11-21T22:01:17.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc9cc062410 0x7fc9cc144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:17.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc0a6e80 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc14a7b0 con 0x7fc9cc0a86b0 2021-11-21T22:01:17.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc0a8bc0 con 0x7fc9cc062410 2021-11-21T22:01:17.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d1f60700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc9cc062410 0x7fc9cc144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:17.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d175f700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc9cc0a6aa0 0x7fc9cc1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34072/0 (socket says 172.21.2.116:34072) 2021-11-21T22:01:17.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d175f700 1 -- 172.21.2.116:0/3623979456 learned_addr learned my addr 172.21.2.116:0/3623979456 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:17.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d1f60700 1 -- 172.21.2.116:0/3623979456 >> v1:172.21.2.117:6789/0 conn(0x7fc9cc0a86b0 legacy=0x7fc9cc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:17.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d1f60700 1 -- 172.21.2.116:0/3623979456 >> v1:172.21.2.116:6789/0 conn(0x7fc9cc0a6aa0 legacy=0x7fc9cc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:17.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9bbfff700 1 -- 172.21.2.116:0/3623979456 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274932931 0 0) 0x7fc9cc0a6e80 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d1f60700 1 -- 172.21.2.116:0/3623979456 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc14aa10 con 0x7fc9cc062410 2021-11-21T22:01:17.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d1f60700 1 --2- 172.21.2.116:0/3623979456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc062410 0x7fc9cc144b00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7fc9cc061b50 tx=0x7fc9c40091d0).ready entity=mon.0 client_cookie=9ccc7536731731a2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:17.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9bbfff700 1 -- 172.21.2.116:0/3623979456 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9c4009ea0 con 0x7fc9cc062410 2021-11-21T22:01:17.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9bbfff700 1 -- 172.21.2.116:0/3623979456 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c400b860 con 0x7fc9cc062410 2021-11-21T22:01:17.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9bbfff700 1 -- 172.21.2.116:0/3623979456 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9c400b9d0 con 0x7fc9cc062410 2021-11-21T22:01:17.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- 172.21.2.116:0/3623979456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc062410 msgr2=0x7fc9cc144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:17.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/3623979456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc062410 0x7fc9cc144b00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7fc9cc061b50 tx=0x7fc9c40091d0).stop 2021-11-21T22:01:17.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- 172.21.2.116:0/3623979456 shutdown_connections 2021-11-21T22:01:17.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/3623979456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc062410 0x7fc9cc144b00 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:17.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- 172.21.2.116:0/3623979456 >> 172.21.2.116:0/3623979456 conn(0x7fc9cc05bd20 msgr2=0x7fc9cc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:17.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- 172.21.2.116:0/3623979456 shutdown_connections 2021-11-21T22:01:17.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- 172.21.2.116:0/3623979456 wait complete. 2021-11-21T22:01:17.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 Processor -- start 2021-11-21T22:01:17.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- start start 2021-11-21T22:01:17.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc9cc062410 0x7fc9cc067320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:17.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc0a6aa0 0x7fc9cc142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:17.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc9cc0a86b0 0x7fc9cc063fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:17.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15a5c0 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0a9060 con 0x7fc9cc0a86b0 2021-11-21T22:01:17.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d39c3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc1564d0 con 0x7fc9cc062410 2021-11-21T22:01:17.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc0a6aa0 0x7fc9cc142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:17.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc0a6aa0 0x7fc9cc142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50356/0 (socket says 172.21.2.116:50356) 2021-11-21T22:01:17.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 -- 172.21.2.116:0/4225926987 learned_addr learned my addr 172.21.2.116:0/4225926987 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:17.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d175f700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc9cc062410 0x7fc9cc067320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:17.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d1f60700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc9cc0a86b0 0x7fc9cc063fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:17.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 -- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc9cc062410 msgr2=0x7fc9cc067320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:17.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc9cc062410 0x7fc9cc067320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:17.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 -- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc9cc0a86b0 msgr2=0x7fc9cc063fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:17.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc9cc0a86b0 0x7fc9cc063fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:17.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 -- 172.21.2.116:0/4225926987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c4009470 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d175f700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc9cc062410 0x7fc9cc067320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:17.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d1f60700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc9cc0a86b0 0x7fc9cc063fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:17.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.006+0000 7fc9d0f5e700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc0a6aa0 0x7fc9cc142590 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7fc9c8000f70 tx=0x7fc9c8007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:17.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9ba7fc700 1 -- 172.21.2.116:0/4225926987 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9c800ce80 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9ba7fc700 1 -- 172.21.2.116:0/4225926987 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8013450 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0a8dc0 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9ba7fc700 1 -- 172.21.2.116:0/4225926987 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9c80137b0 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc143160 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc15a900 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9ba7fc700 1 -- 172.21.2.116:0/4225926987 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc9c801a060 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9ba7fc700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9ac053640 0x7fc9ac055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:17.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9ba7fc700 1 -- 172.21.2.116:0/4225926987 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc9c806b020 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9d175f700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9ac053640 0x7fc9ac055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:17.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9d175f700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9ac053640 0x7fc9ac055b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fc9bc008000 tx=0x7fc9bc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:17.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.010+0000 7fc9ba7fc700 1 -- 172.21.2.116:0/4225926987 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9c8042350 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.122+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fc9cc0573d0 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9ba7fc700 1 -- 172.21.2.116:0/4225926987 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v291) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc9c8018070 con 0x7fc9cc0a6aa0 2021-11-21T22:01:17.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9ac053640 msgr2=0x7fc9ac055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:17.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9ac053640 0x7fc9ac055b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fc9bc008000 tx=0x7fc9bc009040).stop 2021-11-21T22:01:17.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc0a6aa0 msgr2=0x7fc9cc142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:17.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc0a6aa0 0x7fc9cc142590 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7fc9c8000f70 tx=0x7fc9c8007c30).stop 2021-11-21T22:01:17.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 shutdown_connections 2021-11-21T22:01:17.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc9cc0a86b0 0x7fc9cc063fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:17.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9ac053640 0x7fc9ac055b00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:17.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc9cc0a6aa0 0x7fc9cc142590 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:17.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 --2- 172.21.2.116:0/4225926987 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc9cc062410 0x7fc9cc067320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:17.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 >> 172.21.2.116:0/4225926987 conn(0x7fc9cc05bd20 msgr2=0x7fc9cc0b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:17.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 shutdown_connections 2021-11-21T22:01:17.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:17.950+0000 7fc9d39c3700 1 -- 172.21.2.116:0/4225926987 wait complete. 2021-11-21T22:01:17.962 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T22:01:18.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 Processor -- start 2021-11-21T22:01:18.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- start start 2021-11-21T22:01:18.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9c280603b0 0x7f9c280af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c28064260 con 0x7f9c28063e80 2021-11-21T22:01:18.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c28140550 con 0x7f9c2813d410 2021-11-21T22:01:18.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c281407f0 con 0x7f9c280603b0 2021-11-21T22:01:18.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c277fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9c28063e80 0x7f9c2813cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34088/0 (socket says 172.21.2.116:34088) 2021-11-21T22:01:18.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c277fe700 1 -- 172.21.2.116:0/1922948549 learned_addr learned my addr 172.21.2.116:0/1922948549 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:18.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787027251 0 0) 0x7f9c28064260 con 0x7f9c28063e80 2021-11-21T22:01:18.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1922948549 >> v2:172.21.2.117:3300/0 conn(0x7f9c280603b0 0x7f9c280af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c14003260 con 0x7f9c28063e80 2021-11-21T22:01:18.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4288724462 0 0) 0x7f9c14003260 con 0x7f9c28063e80 2021-11-21T22:01:18.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 >> v2:172.21.2.117:3300/0 conn(0x7f9c280603b0 msgr2=0x7f9c280af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 --2- 172.21.2.116:0/1922948549 >> v2:172.21.2.117:3300/0 conn(0x7f9c280603b0 0x7f9c280af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 >> v1:172.21.2.117:6789/0 conn(0x7f9c2813d410 legacy=0x7f9c280af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c280ae920 con 0x7f9c28063e80 2021-11-21T22:01:18.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9c18001680 con 0x7f9c28063e80 2021-11-21T22:01:18.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 >> v1:172.21.2.116:6789/0 conn(0x7f9c28063e80 legacy=0x7f9c2813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c14003820 0x7f9c14003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004500 0x7f9c14004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c14004ee0 0x7f9c1400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281407f0 con 0x7f9c14004ee0 2021-11-21T22:01:18.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1400bdd0 con 0x7f9c14003820 2021-11-21T22:01:18.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1400bf90 con 0x7f9c14004500 2021-11-21T22:01:18.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c14004ee0 0x7f9c1400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c277fe700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c14003820 0x7f9c14003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c26ffd700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004500 0x7f9c14004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 -- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004500 msgr2=0x7f9c14004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004500 0x7f9c14004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 -- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c14003820 msgr2=0x7f9c14003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c14003820 0x7f9c14003c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 -- 172.21.2.116:0/1922948549 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c14003260 con 0x7f9c14004ee0 2021-11-21T22:01:18.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 -- 172.21.2.116:0/1922948549 reap_dead start 2021-11-21T22:01:18.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c14004ee0 0x7f9c1400b750 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f9c2000c000 tx=0x7f9c200091d0).ready entity=mon.0 client_cookie=84b0601d9c2d34ad server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c2000e6e0 con 0x7f9c14004ee0 2021-11-21T22:01:18.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2000e840 con 0x7f9c14004ee0 2021-11-21T22:01:18.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c267fc700 1 -- 172.21.2.116:0/1922948549 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c2000eba0 con 0x7f9c14004ee0 2021-11-21T22:01:18.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1922948549 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c14004ee0 msgr2=0x7f9c1400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c14004ee0 0x7f9c1400b750 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f9c2000c000 tx=0x7f9c200091d0).stop 2021-11-21T22:01:18.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1922948549 shutdown_connections 2021-11-21T22:01:18.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1922948549 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c14004ee0 0x7f9c1400b750 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1922948549 >> 172.21.2.116:0/1922948549 conn(0x7f9c28061d40 msgr2=0x7f9c28062120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1922948549 shutdown_connections 2021-11-21T22:01:18.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.018+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1922948549 wait complete. 2021-11-21T22:01:18.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 Processor -- start 2021-11-21T22:01:18.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 -- start start 2021-11-21T22:01:18.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004ee0 0x7f9c280b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c28132720 0x7f9c28066cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c280671b0 0x7f9c280b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280b4ec0 con 0x7f9c280671b0 2021-11-21T22:01:18.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c28132140 con 0x7f9c28132720 2021-11-21T22:01:18.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c28132370 con 0x7f9c14004ee0 2021-11-21T22:01:18.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c280671b0 0x7f9c280b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c280671b0 0x7f9c280b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50378/0 (socket says 172.21.2.116:50378) 2021-11-21T22:01:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 -- 172.21.2.116:0/1379499225 learned_addr learned my addr 172.21.2.116:0/1379499225 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 -- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004ee0 msgr2=0x7f9c280b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004ee0 0x7f9c280b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 -- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c28132720 msgr2=0x7f9c28066cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c28132720 0x7f9c28066cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 -- 172.21.2.116:0/1379499225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c28131b30 con 0x7f9c280671b0 2021-11-21T22:01:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c27fff700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c280671b0 0x7f9c280b1670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f9c2000c000 tx=0x7f9c2001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c24ff9700 1 -- 172.21.2.116:0/1379499225 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c20006ea0 con 0x7f9c280671b0 2021-11-21T22:01:18.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c24ff9700 1 -- 172.21.2.116:0/1379499225 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2000ee50 con 0x7f9c280671b0 2021-11-21T22:01:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c280b4790 con 0x7f9c280671b0 2021-11-21T22:01:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c28130bd0 con 0x7f9c280671b0 2021-11-21T22:01:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c24ff9700 1 -- 172.21.2.116:0/1379499225 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c2000e6e0 con 0x7f9c280671b0 2021-11-21T22:01:18.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c280677e0 con 0x7f9c280671b0 2021-11-21T22:01:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c24ff9700 1 -- 172.21.2.116:0/1379499225 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9c200245e0 con 0x7f9c280671b0 2021-11-21T22:01:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c24ff9700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c080535d0 0x7f9c08055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c24ff9700 1 -- 172.21.2.116:0/1379499225 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9c2006d2c0 con 0x7f9c280671b0 2021-11-21T22:01:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.022+0000 7f9c277fe700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c080535d0 0x7f9c08055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.026+0000 7f9c277fe700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c080535d0 0x7f9c08055a90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f9c1800d000 tx=0x7f9c280ae7a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.026+0000 7f9c24ff9700 1 -- 172.21.2.116:0/1379499225 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c280677e0 con 0x7f9c280671b0 2021-11-21T22:01:18.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.110+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9c280677e0 con 0x7f9c280671b0 2021-11-21T22:01:18.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.114+0000 7f9c24ff9700 1 -- 172.21.2.116:0/1379499225 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c280677e0 con 0x7f9c280671b0 2021-11-21T22:01:18.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c080535d0 msgr2=0x7f9c08055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c080535d0 0x7f9c08055a90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f9c1800d000 tx=0x7f9c280ae7a0).stop 2021-11-21T22:01:18.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c280671b0 msgr2=0x7f9c280b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c280671b0 0x7f9c280b1670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f9c2000c000 tx=0x7f9c2001a000).stop 2021-11-21T22:01:18.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 shutdown_connections 2021-11-21T22:01:18.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c080535d0 0x7f9c08055a90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c280671b0 0x7f9c280b1670 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c28132720 0x7f9c28066cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 --2- 172.21.2.116:0/1379499225 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c14004ee0 0x7f9c280b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 >> 172.21.2.116:0/1379499225 conn(0x7f9c28061d40 msgr2=0x7f9c2813cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 shutdown_connections 2021-11-21T22:01:18.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.118+0000 7f9c2dba5700 1 -- 172.21.2.116:0/1379499225 wait complete. 2021-11-21T22:01:18.127 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:01:18.127 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:18.135 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:18.143 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:18.157 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:18.172 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:01:18.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:18.173 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:18.173 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:18.210 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:01:18.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:01:18.211 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:18.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:18.212 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:18.213 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:18.224 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:01:18.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:01:18.225 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:18.226 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:01:18.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 Processor -- start 2021-11-21T22:01:18.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- start start 2021-11-21T22:01:18.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f16740a86b0 0x7f167414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 0x7f16741443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16740a6e80 con 0x7f16740a6aa0 2021-11-21T22:01:18.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16740a8bc0 con 0x7f16740a86b0 2021-11-21T22:01:18.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167414a7b0 con 0x7f1674062410 2021-11-21T22:01:18.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f16740a86b0 0x7f167414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f16740a86b0 0x7f167414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50388/0 (socket says 172.21.2.116:50388) 2021-11-21T22:01:18.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 -- 172.21.2.116:0/3925146798 learned_addr learned my addr 172.21.2.116:0/3925146798 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:18.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 -- 172.21.2.116:0/3925146798 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 msgr2=0x7f16741443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:18.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 --2- 172.21.2.116:0/3925146798 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 0x7f16741443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 -- 172.21.2.116:0/3925146798 >> v1:172.21.2.117:6789/0 conn(0x7f16740a6aa0 legacy=0x7f16741470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:18.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 -- 172.21.2.116:0/3925146798 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167414aa10 con 0x7f16740a86b0 2021-11-21T22:01:18.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f16727fc700 1 --2- 172.21.2.116:0/3925146798 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a86b0 0x7f167414b150 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f1674061ab0 tx=0x7f165c006eb0).ready entity=mon.0 client_cookie=2adcf148f356c95e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1671ffb700 1 -- 172.21.2.116:0/3925146798 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f165c00d570 con 0x7f16740a86b0 2021-11-21T22:01:18.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1671ffb700 1 -- 172.21.2.116:0/3925146798 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c000e90 con 0x7f16740a86b0 2021-11-21T22:01:18.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1671ffb700 1 -- 172.21.2.116:0/3925146798 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f165c00d8c0 con 0x7f16740a86b0 2021-11-21T22:01:18.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- 172.21.2.116:0/3925146798 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a86b0 msgr2=0x7f167414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3925146798 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a86b0 0x7f167414b150 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f1674061ab0 tx=0x7f165c006eb0).stop 2021-11-21T22:01:18.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- 172.21.2.116:0/3925146798 shutdown_connections 2021-11-21T22:01:18.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3925146798 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 0x7f16741443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3925146798 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a86b0 0x7f167414b150 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- 172.21.2.116:0/3925146798 >> 172.21.2.116:0/3925146798 conn(0x7f167405bd20 msgr2=0x7f167405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- 172.21.2.116:0/3925146798 shutdown_connections 2021-11-21T22:01:18.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- 172.21.2.116:0/3925146798 wait complete. 2021-11-21T22:01:18.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 Processor -- start 2021-11-21T22:01:18.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.278+0000 7f1679a0c700 1 -- start start 2021-11-21T22:01:18.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 0x7f1674147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a6aa0 0x7f1674159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16740a86b0 0x7f167415a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740accd0 con 0x7f16740a6aa0 2021-11-21T22:01:18.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740ac790 con 0x7f16740a86b0 2021-11-21T22:01:18.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740aca30 con 0x7f1674062410 2021-11-21T22:01:18.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a6aa0 0x7f1674159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a6aa0 0x7f1674159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50390/0 (socket says 172.21.2.116:50390) 2021-11-21T22:01:18.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 -- 172.21.2.116:0/3503647863 learned_addr learned my addr 172.21.2.116:0/3503647863 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:18.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 -- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 msgr2=0x7f1674147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 0x7f1674147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 -- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16740a86b0 msgr2=0x7f167415a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16740a86b0 0x7f167415a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 -- 172.21.2.116:0/3503647863 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c00b1e0 con 0x7f16740a6aa0 2021-11-21T22:01:18.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f16727fc700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a6aa0 0x7f1674159d40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f1674061ab0 tx=0x7f165c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f166bfff700 1 -- 172.21.2.116:0/3503647863 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f165c011990 con 0x7f16740a6aa0 2021-11-21T22:01:18.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f166bfff700 1 -- 172.21.2.116:0/3503647863 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c011af0 con 0x7f16740a6aa0 2021-11-21T22:01:18.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f166bfff700 1 -- 172.21.2.116:0/3503647863 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f165c011e50 con 0x7f16740a6aa0 2021-11-21T22:01:18.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16741598a0 con 0x7f16740a6aa0 2021-11-21T22:01:18.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1674143740 con 0x7f16740a6aa0 2021-11-21T22:01:18.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1674159a30 con 0x7f16740a6aa0 2021-11-21T22:01:18.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f166bfff700 1 -- 172.21.2.116:0/3503647863 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f165c025030 con 0x7f16740a6aa0 2021-11-21T22:01:18.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f166bfff700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1658053640 0x7f1658055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f166bfff700 1 -- 172.21.2.116:0/3503647863 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f165c06c9c0 con 0x7f16740a6aa0 2021-11-21T22:01:18.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.282+0000 7f1672ffd700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1658053640 0x7f1658055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.286+0000 7f1672ffd700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1658053640 0x7f1658055b00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f1664002fb0 tx=0x7f16640069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.286+0000 7f166bfff700 1 -- 172.21.2.116:0/3503647863 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f165c043e00 con 0x7f16740a6aa0 2021-11-21T22:01:18.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.366+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f167405db70 con 0x7f16740a6aa0 2021-11-21T22:01:18.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.366+0000 7f166bfff700 1 -- 172.21.2.116:0/3503647863 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v92) v1 ==== 81+0+28828 (secure 0 0 0) 0x7f165c023020 con 0x7f16740a6aa0 2021-11-21T22:01:18.373 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:18.373 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:18.373 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 92, 2021-11-21T22:01:18.373 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:01:18.374 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:01:18.374 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:01:15.933096+0000", 2021-11-21T22:01:18.374 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:18.374 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:18.374 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:01:18.375 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:01:18.375 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:01:18.375 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:01:18.375 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:01:18.376 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:01:18.376 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:01:18.376 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:18.376 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:01:18.376 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:01:18.377 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:01:18.377 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:01:18.377 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:01:18.377 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 16, 2021-11-21T22:01:18.378 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:01:18.378 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:01:18.378 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:01:18.378 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:01:18.378 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:01:18.379 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.379 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:01:18.379 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:01:18.379 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:01:18.380 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:18.380 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:18.380 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:18.380 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:18.380 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:18.381 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:18.381 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:18.381 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:18.381 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:01:18.382 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:01:18.382 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:01:18.382 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:01:18.382 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:01:18.383 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:18.383 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:18.383 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:18.383 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:18.383 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:18.384 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:18.384 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:18.384 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.384 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:01:18.385 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:18.385 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:18.385 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:18.385 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:18.385 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:18.386 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:18.386 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:18.386 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:18.386 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:18.387 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:18.387 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:18.387 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:18.387 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:18.387 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:18.388 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:18.388 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:18.388 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:18.388 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:18.388 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:18.389 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:18.389 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:18.389 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:18.389 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:18.390 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:18.390 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:18.390 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:18.390 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.390 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:18.391 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:18.391 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:18.391 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:18.391 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:18.392 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:18.392 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:18.392 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:18.392 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:18.392 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:18.393 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:18.393 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:18.393 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:01:18.393 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.394 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:18.394 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:01:18.394 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.394 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.394 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.395 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:01:18.395 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:01:18.395 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:01:18.395 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:18.396 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:18.396 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:18.396 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:18.396 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:18.396 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:18.397 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:18.397 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:18.397 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:18.397 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:18.398 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:18.398 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:18.398 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:18.398 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:18.398 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:18.399 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:18.399 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:18.399 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:18.399 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:18.400 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:18.400 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.400 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:01:18.400 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:18.400 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:18.401 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:18.401 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:18.401 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:18.401 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:18.401 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:18.402 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:18.402 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:18.402 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:18.402 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:18.403 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:18.403 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:18.403 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:18.403 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:18.403 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:18.404 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:18.404 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:18.404 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:18.404 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:18.405 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:18.405 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:18.405 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:18.405 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:18.405 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:18.406 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:18.406 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.406 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:18.406 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:18.407 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:18.407 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:18.407 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:18.407 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:18.407 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:18.408 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:18.408 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:18.408 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:18.408 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:18.408 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:18.409 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:18.409 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:01:18.409 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.409 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.410 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:18.410 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:01:18.410 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.410 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:18.410 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:01:18.411 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.411 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.411 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.411 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.412 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.412 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.412 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.412 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:18.412 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.413 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.413 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.413 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.413 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.414 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.414 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:01:18.414 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.414 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.414 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.415 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.415 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:01:18.415 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.415 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.416 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.416 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.416 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.416 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.416 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.417 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.417 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:01:18.417 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.418 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.418 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.418 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.418 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:01:18.418 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.419 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.419 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.419 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.420 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.420 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.420 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.420 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:01:18.420 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.421 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.421 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.421 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.421 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:01:18.421 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.422 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.422 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.422 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.422 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.423 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.423 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.423 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.423 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:01:18.424 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.424 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.424 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.424 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.424 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:01:18.425 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:18.425 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.425 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.425 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.425 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:01:18.426 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:01:18.426 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:01:18.426 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:01:18.426 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.427 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.427 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.427 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.427 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.428 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:18.428 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:01:18.428 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.428 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.429 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.429 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.429 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.429 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.429 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.430 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 85, 2021-11-21T22:01:18.430 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.430 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.430 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.431 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.431 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.431 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.431 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:01:18.432 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.432 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.432 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.432 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:01:18.433 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.433 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.433 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.434 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.434 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.434 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.434 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.434 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:01:18.435 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.435 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.435 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.435 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.436 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:01:18.436 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.436 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.436 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.436 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.437 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.437 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.437 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.437 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.438 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:01:18.438 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.438 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.438 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.438 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.439 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:01:18.439 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.439 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.439 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.440 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.440 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.440 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.440 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.440 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.441 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:01:18.441 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.441 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.441 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.442 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.442 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:01:18.442 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:18.442 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.442 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.443 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.443 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:01:18.443 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:01:18.443 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:01:18.444 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:01:18.444 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.444 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.444 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.444 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.445 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.445 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.445 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:18.445 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:01:18.445 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.446 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.446 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.446 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.446 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.447 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.447 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.447 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:18.447 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.447 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.448 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.448 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.448 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.448 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.449 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:01:18.449 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.449 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.449 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.450 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:01:18.450 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.450 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.450 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.451 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.451 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.451 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.451 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.451 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.452 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:01:18.452 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.452 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.452 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.453 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:01:18.453 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.453 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.453 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.454 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.454 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.454 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.454 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.455 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.455 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:01:18.455 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.455 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.455 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.456 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.456 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:01:18.456 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.456 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.456 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.457 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.457 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.457 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.457 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.458 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.458 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:01:18.458 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.458 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.458 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.459 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.459 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:01:18.459 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:18.459 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.460 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.460 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.460 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:01:18.460 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:01:18.460 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:01:18.461 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:01:18.461 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.461 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.461 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.461 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.462 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.462 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:18.462 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:01:18.463 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.463 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.463 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.463 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.463 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.464 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.464 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.464 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 87, 2021-11-21T22:01:18.464 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.465 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.465 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.465 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.465 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.465 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.466 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:01:18.466 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.466 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.466 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.467 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.467 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:01:18.467 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.467 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.467 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.468 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.468 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.468 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.468 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.468 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.469 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:01:18.469 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.469 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.469 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.470 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.470 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:01:18.470 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.470 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.470 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.471 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.471 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.471 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.471 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.472 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.472 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:01:18.472 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.472 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.472 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.473 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.473 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:01:18.473 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.473 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.473 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.474 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.474 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.474 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.475 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.475 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:01:18.475 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.475 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.475 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.476 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.476 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:01:18.476 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:18.476 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.476 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.477 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.477 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:01:18.477 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:01:18.477 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:01:18.478 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:01:18.478 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.478 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.478 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.478 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.479 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.479 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:18.479 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:01:18.480 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.480 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.480 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.480 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.480 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.481 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.481 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.481 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:18.481 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.481 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.482 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.482 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.482 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.482 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.483 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:01:18.483 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.483 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.483 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.483 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.484 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:01:18.484 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.484 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.484 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.485 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.485 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.485 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.485 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.485 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.486 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:01:18.486 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.486 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.486 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.487 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.487 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:01:18.487 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.487 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.487 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.488 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.488 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.488 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.488 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.488 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.489 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:01:18.489 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.489 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.489 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.490 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.490 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:01:18.490 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.490 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.490 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.491 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.491 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.491 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.491 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.492 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.492 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:01:18.492 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.492 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.492 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.493 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.493 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:01:18.493 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:18.493 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.493 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.494 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.494 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:01:18.494 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:01:18.494 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:01:18.495 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:01:18.495 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.495 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.495 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.496 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.496 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.496 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.496 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:18.496 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:01:18.497 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.497 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.497 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.497 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.497 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.498 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.498 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.498 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:18.498 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.499 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.499 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.499 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.499 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.500 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:01:18.500 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.500 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.500 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.501 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.501 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:01:18.501 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.501 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.501 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.502 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.502 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.502 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.502 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.502 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.503 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:01:18.503 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.503 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.503 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.504 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.504 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:01:18.504 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.504 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.504 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.505 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.505 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.505 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.505 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.505 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.506 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:01:18.506 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.506 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.506 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.507 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.507 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:01:18.507 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.507 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.507 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.508 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.508 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.508 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.508 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.508 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:01:18.509 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.509 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.509 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.510 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:01:18.510 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:18.510 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.512 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.512 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.512 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:01:18.513 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:01:18.513 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:01:18.513 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:01:18.513 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.514 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.514 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.514 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.514 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.514 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.515 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:18.515 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:01:18.515 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.515 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.516 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.516 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.516 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.516 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.516 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.517 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:18.517 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.517 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.517 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.517 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.518 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.518 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.518 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:01:18.518 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.519 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.519 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.519 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.519 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:01:18.519 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.520 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.520 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.520 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.520 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.520 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.521 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.521 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.521 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:01:18.521 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.522 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.522 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.522 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.522 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:01:18.523 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.523 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.523 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.523 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.523 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.524 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.524 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.524 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.524 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:01:18.524 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.525 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.525 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.525 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.525 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:01:18.526 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.526 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.526 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.526 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.526 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.527 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:01:18.528 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.528 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.528 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.528 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:01:18.529 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:18.529 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.529 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.529 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.530 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:01:18.530 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:01:18.530 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:01:18.530 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:01:18.530 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.531 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.531 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.531 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.531 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.532 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.532 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:18.532 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:01:18.532 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:18.533 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:18.533 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:18.533 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:18.533 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:18.533 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:18.534 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:18.534 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 72, 2021-11-21T22:01:18.534 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:18.534 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:18.535 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:18.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.535 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.535 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:01:18.536 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.536 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.536 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.536 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.537 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:01:18.537 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.537 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.537 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.538 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.538 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:18.538 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.538 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.539 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.539 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:01:18.539 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.539 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.539 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.540 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.540 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:01:18.540 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.540 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.541 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.541 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.541 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:18.541 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.541 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.542 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.542 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:01:18.542 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.542 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.543 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.543 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.543 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:01:18.543 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.544 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.544 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.544 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.544 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:18.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:18.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.545 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:18.545 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:01:18.545 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.546 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.546 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.546 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:18.546 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:01:18.546 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:18.547 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.547 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.547 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.547 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:01:18.547 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:01:18.548 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:01:18.548 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:01:18.548 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:18.548 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:18.549 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:18.549 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:18.549 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.549 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:18.550 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:01:18.550 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.550 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:18.550 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.550 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.551 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.551 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.551 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.551 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:01:18.552 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.552 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.552 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.552 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:18.552 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.553 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.553 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.553 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.553 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.554 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:01:18.554 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.554 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.554 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.554 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:18.555 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.555 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.555 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.555 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.556 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.556 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:01:18.556 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.556 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.556 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.557 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:18.557 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.557 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.557 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.558 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.558 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.558 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:01:18.558 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.559 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.559 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.559 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:18.559 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.559 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.560 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.560 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.560 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.560 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:01:18.561 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.561 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.561 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.561 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:18.561 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.562 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.562 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.562 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.562 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.563 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:01:18.563 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.563 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.563 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.564 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:18.564 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.564 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.564 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.564 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.565 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.565 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:01:18.565 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.565 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.566 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:18.566 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:18.566 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:18.566 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:18.566 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:18.567 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:18.567 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:18.567 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:01:18.567 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:18.568 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.568 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:18.568 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:01:18.568 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:01:18.568 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:01:18.569 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:01:18.569 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:01:18.569 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/2155808669": "2021-11-22T22:01:12.849668+0000", 2021-11-21T22:01:18.569 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/2155808669": "2021-11-22T22:01:12.849668+0000" 2021-11-21T22:01:18.569 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.570 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:01:18.570 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:01:18.570 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:01:18.570 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:01:18.571 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:01:18.571 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:01:18.571 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:01:18.571 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:01:18.572 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:18.572 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:18.572 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:01:18.572 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:01:18.572 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:01:18.573 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:01:18.573 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:01:18.573 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:18.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1658053640 msgr2=0x7f1658055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1658053640 0x7f1658055b00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f1664002fb0 tx=0x7f16640069b0).stop 2021-11-21T22:01:18.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a6aa0 msgr2=0x7f1674159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a6aa0 0x7f1674159d40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f1674061ab0 tx=0x7f165c015000).stop 2021-11-21T22:01:18.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 shutdown_connections 2021-11-21T22:01:18.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16740a86b0 0x7f167415a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1658053640 0x7f1658055b00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16740a6aa0 0x7f1674159d40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 --2- 172.21.2.116:0/3503647863 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1674062410 0x7f1674147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 >> 172.21.2.116:0/3503647863 conn(0x7f167405bd20 msgr2=0x7f16740b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 shutdown_connections 2021-11-21T22:01:18.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.370+0000 7f1679a0c700 1 -- 172.21.2.116:0/3503647863 wait complete. 2021-11-21T22:01:18.578 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:18.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c8d922700 1 Processor -- start 2021-11-21T22:01:18.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c8d922700 1 -- start start 2021-11-21T22:01:18.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c8d922700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880aed90 0x7f9c880b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c8d922700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9c880af360 0x7f9c88069e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c8d922700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c880b00d0 con 0x7f9c880b24a0 2021-11-21T22:01:18.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c8d922700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880adba0 con 0x7f9c880af360 2021-11-21T22:01:18.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c8d922700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b2880 con 0x7f9c880aed90 2021-11-21T22:01:18.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c867fc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880aed90 0x7f9c880b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c877fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9c880af360 0x7f9c88069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c867fc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880aed90 0x7f9c880b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:53744/0 (socket says 172.21.2.116:53744) 2021-11-21T22:01:18.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c877fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9c880af360 0x7f9c88069e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:51570/0 (socket says 172.21.2.116:51570) 2021-11-21T22:01:18.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c867fc700 1 -- 172.21.2.116:0/3160183422 learned_addr learned my addr 172.21.2.116:0/3160183422 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:18.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c877fe700 1 -- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880aed90 msgr2=0x7f9c880b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c877fe700 1 --2- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880aed90 0x7f9c880b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c877fe700 1 -- 172.21.2.116:0/3160183422 >> v1:172.21.2.117:6789/0 conn(0x7f9c880b24a0 legacy=0x7f9c880ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c877fe700 1 -- 172.21.2.116:0/3160183422 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880b14a0 con 0x7f9c880af360 2021-11-21T22:01:18.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c867fc700 1 --2- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880aed90 0x7f9c880b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:01:18.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.646+0000 7f9c877fe700 1 --2- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af360 0x7f9c88069e70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9c800032f0 tx=0x7f9c800091d0).ready entity=mon.1 client_cookie=d46b482b4cf48974 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c85ffb700 1 -- 172.21.2.116:0/3160183422 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8000b6c0 con 0x7f9c880af360 2021-11-21T22:01:18.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c85ffb700 1 -- 172.21.2.116:0/3160183422 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8000b820 con 0x7f9c880af360 2021-11-21T22:01:18.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c85ffb700 1 -- 172.21.2.116:0/3160183422 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8000b990 con 0x7f9c880af360 2021-11-21T22:01:18.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af360 msgr2=0x7f9c88069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 --2- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af360 0x7f9c88069e70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9c800032f0 tx=0x7f9c800091d0).stop 2021-11-21T22:01:18.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- 172.21.2.116:0/3160183422 shutdown_connections 2021-11-21T22:01:18.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 --2- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880af360 0x7f9c88069e70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 --2- 172.21.2.116:0/3160183422 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880aed90 0x7f9c880b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- 172.21.2.116:0/3160183422 >> 172.21.2.116:0/3160183422 conn(0x7f9c8805fd00 msgr2=0x7f9c880600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- 172.21.2.116:0/3160183422 shutdown_connections 2021-11-21T22:01:18.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- 172.21.2.116:0/3160183422 wait complete. 2021-11-21T22:01:18.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 Processor -- start 2021-11-21T22:01:18.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- start start 2021-11-21T22:01:18.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880aed90 0x7f9c880666f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af360 0x7f9c88066be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880b24a0 0x7f9c8810f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8813ab40 con 0x7f9c880af360 2021-11-21T22:01:18.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880670d0 con 0x7f9c880aed90 2021-11-21T22:01:18.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88156d50 con 0x7f9c880b24a0 2021-11-21T22:01:18.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af360 0x7f9c88066be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af360 0x7f9c88066be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50406/0 (socket says 172.21.2.116:50406) 2021-11-21T22:01:18.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 -- 172.21.2.116:0/1044697976 learned_addr learned my addr 172.21.2.116:0/1044697976 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:18.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c877fe700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880b24a0 0x7f9c8810f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c86ffd700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880aed90 0x7f9c880666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 -- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880b24a0 msgr2=0x7f9c8810f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880b24a0 0x7f9c8810f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 -- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880aed90 msgr2=0x7f9c880666f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880aed90 0x7f9c880666f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 -- 172.21.2.116:0/1044697976 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c00b1e0 con 0x7f9c880af360 2021-11-21T22:01:18.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c86ffd700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880aed90 0x7f9c880666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:18.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c867fc700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af360 0x7f9c88066be0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f9c7c00c000 tx=0x7f9c7c006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c77fff700 1 -- 172.21.2.116:0/1044697976 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c7c00c6a0 con 0x7f9c880af360 2021-11-21T22:01:18.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c77fff700 1 -- 172.21.2.116:0/1044697976 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c00c800 con 0x7f9c880af360 2021-11-21T22:01:18.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c77fff700 1 -- 172.21.2.116:0/1044697976 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c7c00cb60 con 0x7f9c880af360 2021-11-21T22:01:18.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c80009470 con 0x7f9c880af360 2021-11-21T22:01:18.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.650+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8813f330 con 0x7f9c880af360 2021-11-21T22:01:18.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.654+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c881418d0 con 0x7f9c880af360 2021-11-21T22:01:18.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.654+0000 7f9c77fff700 1 -- 172.21.2.116:0/1044697976 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9c7c022070 con 0x7f9c880af360 2021-11-21T22:01:18.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.654+0000 7f9c77fff700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c64053720 0x7f9c64055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.654+0000 7f9c77fff700 1 -- 172.21.2.116:0/1044697976 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9c7c06bd00 con 0x7f9c880af360 2021-11-21T22:01:18.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.654+0000 7f9c86ffd700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c64053720 0x7f9c64055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.654+0000 7f9c86ffd700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c64053720 0x7f9c64055be0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f9c78008000 tx=0x7f9c78009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.654+0000 7f9c77fff700 1 -- 172.21.2.116:0/1044697976 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c881418d0 con 0x7f9c880af360 2021-11-21T22:01:18.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c880648b0 con 0x7f9c880af360 2021-11-21T22:01:18.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c77fff700 1 -- 172.21.2.116:0/1044697976 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f9c7c043030 con 0x7f9c880af360 2021-11-21T22:01:18.766 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:18.767 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/491943077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":491943077},{"type":"v1","addr":"172.21.2.116:6835","nonce":491943077}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":41}],"filesystems":[]} 2021-11-21T22:01:18.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c64053720 msgr2=0x7f9c64055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c64053720 0x7f9c64055be0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f9c78008000 tx=0x7f9c78009040).stop 2021-11-21T22:01:18.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af360 msgr2=0x7f9c88066be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af360 0x7f9c88066be0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f9c7c00c000 tx=0x7f9c7c006eb0).stop 2021-11-21T22:01:18.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 shutdown_connections 2021-11-21T22:01:18.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9c880b24a0 0x7f9c8810f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9c64053720 0x7f9c64055be0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9c880af360 0x7f9c88066be0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 --2- 172.21.2.116:0/1044697976 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9c880aed90 0x7f9c880666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 >> 172.21.2.116:0/1044697976 conn(0x7f9c8805fd00 msgr2=0x7f9c88069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 shutdown_connections 2021-11-21T22:01:18.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.762+0000 7f9c8d922700 1 -- 172.21.2.116:0/1044697976 wait complete. 2021-11-21T22:01:18.771 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 41 2021-11-21T22:01:18.773 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:18.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 Processor -- start 2021-11-21T22:01:18.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- start start 2021-11-21T22:01:18.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181443a0 0x7fe418144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180b29b0 con 0x7fe418067850 2021-11-21T22:01:18.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe418067c30 con 0x7fe4180b3320 2021-11-21T22:01:18.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180ab720 con 0x7fe4181443a0 2021-11-21T22:01:18.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe418067850 0x7fe4180b7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34138/0 (socket says 172.21.2.116:34138) 2021-11-21T22:01:18.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 -- 172.21.2.116:0/3927009544 learned_addr learned my addr 172.21.2.116:0/3927009544 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:18.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000006728 0 0) 0x7fe4180b29b0 con 0x7fe418067850 2021-11-21T22:01:18.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f8003260 con 0x7fe418067850 2021-11-21T22:01:18.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe4177fe700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181443a0 0x7fe418144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445457419 0 0) 0x7fe3f8003260 con 0x7fe418067850 2021-11-21T22:01:18.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181443a0 msgr2=0x7fe418144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181443a0 0x7fe418144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 >> v1:172.21.2.117:6789/0 conn(0x7fe4180b3320 legacy=0x7fe4180ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180b3780 con 0x7fe418067850 2021-11-21T22:01:18.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe408001660 con 0x7fe418067850 2021-11-21T22:01:18.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 >> v1:172.21.2.116:6789/0 conn(0x7fe418067850 legacy=0x7fe4180b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f8003800 0x7fe3f8003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181443a0 0x7fe3f8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8004bf0 0x7fe3f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180ab720 con 0x7fe3f8003800 2021-11-21T22:01:18.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8005250 con 0x7fe3f8004bf0 2021-11-21T22:01:18.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f800bc40 con 0x7fe4181443a0 2021-11-21T22:01:18.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f8003800 0x7fe3f8003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 -- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181443a0 msgr2=0x7fe3f8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181443a0 0x7fe3f8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 -- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8004bf0 msgr2=0x7fe3f800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:18.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8004bf0 0x7fe3f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 -- 172.21.2.116:0/3927009544 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180b3780 con 0x7fe3f8003800 2021-11-21T22:01:18.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe416ffd700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f8003800 0x7fe3f8003c00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fe408001020 tx=0x7fe408001e20).ready entity=mon.0 client_cookie=46d682503eb1ea62 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4080070d0 con 0x7fe3f8003800 2021-11-21T22:01:18.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408007230 con 0x7fe3f8003800 2021-11-21T22:01:18.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe415ffb700 1 -- 172.21.2.116:0/3927009544 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40800c9c0 con 0x7fe3f8003800 2021-11-21T22:01:18.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- 172.21.2.116:0/3927009544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f8003800 msgr2=0x7fe3f8003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 --2- 172.21.2.116:0/3927009544 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f8003800 0x7fe3f8003c00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fe408001020 tx=0x7fe408001e20).stop 2021-11-21T22:01:18.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe4177fe700 1 -- 172.21.2.116:0/3927009544 reap_dead start 2021-11-21T22:01:18.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- 172.21.2.116:0/3927009544 shutdown_connections 2021-11-21T22:01:18.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- 172.21.2.116:0/3927009544 >> 172.21.2.116:0/3927009544 conn(0x7fe418064d60 msgr2=0x7fe418065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- 172.21.2.116:0/3927009544 shutdown_connections 2021-11-21T22:01:18.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.822+0000 7fe41d365700 1 -- 172.21.2.116:0/3927009544 wait complete. 2021-11-21T22:01:18.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 Processor -- start 2021-11-21T22:01:18.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 -- start start 2021-11-21T22:01:18.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8003800 0x7fe4180b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe41809df50 0x7fe4181434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181439e0 0x7fe418151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41814dbd0 con 0x7fe41809df50 2021-11-21T22:01:18.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe418143e90 con 0x7fe3f8003800 2021-11-21T22:01:18.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41814ddc0 con 0x7fe4181439e0 2021-11-21T22:01:18.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe41809df50 0x7fe4181434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe41809df50 0x7fe4181434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50424/0 (socket says 172.21.2.116:50424) 2021-11-21T22:01:18.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 -- 172.21.2.116:0/528022997 learned_addr learned my addr 172.21.2.116:0/528022997 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:18.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 -- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181439e0 msgr2=0x7fe418151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181439e0 0x7fe418151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 -- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8003800 msgr2=0x7fe4180b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8003800 0x7fe4180b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 -- 172.21.2.116:0/528022997 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41814e060 con 0x7fe41809df50 2021-11-21T22:01:18.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe4167fc700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe41809df50 0x7fe4181434f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fe400000f70 tx=0x7fe400006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe40ffff700 1 -- 172.21.2.116:0/528022997 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe400009e70 con 0x7fe41809df50 2021-11-21T22:01:18.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe40ffff700 1 -- 172.21.2.116:0/528022997 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400013460 con 0x7fe41809df50 2021-11-21T22:01:18.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41814e300 con 0x7fe41809df50 2021-11-21T22:01:18.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe40ffff700 1 -- 172.21.2.116:0/528022997 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4000137c0 con 0x7fe41809df50 2021-11-21T22:01:18.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41814ea40 con 0x7fe41809df50 2021-11-21T22:01:18.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41809e550 con 0x7fe41809df50 2021-11-21T22:01:18.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe40ffff700 1 -- 172.21.2.116:0/528022997 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe40001a060 con 0x7fe41809df50 2021-11-21T22:01:18.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe40ffff700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3fc053560 0x7fe3fc055a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:18.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe40ffff700 1 -- 172.21.2.116:0/528022997 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe40006af50 con 0x7fe41809df50 2021-11-21T22:01:18.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.826+0000 7fe416ffd700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3fc053560 0x7fe3fc055a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:18.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.830+0000 7fe416ffd700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3fc053560 0x7fe3fc055a20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fe408001020 tx=0x7fe408015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:18.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.830+0000 7fe40ffff700 1 -- 172.21.2.116:0/528022997 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe400042390 con 0x7fe41809df50 2021-11-21T22:01:18.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4180b7910 con 0x7fe41809df50 2021-11-21T22:01:18.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe40ffff700 1 -- 172.21.2.116:0/528022997 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+1556 (secure 0 0 0) 0x7fe400018090 con 0x7fe41809df50 2021-11-21T22:01:18.943 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:18.943 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4888,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3566559907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3566559907},{"type":"v1","addr":"172.21.2.117:6835","nonce":3566559907}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/275870239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":275870239},{"type":"v1","addr":"172.21.2.116:6833","nonce":275870239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/491943077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":491943077},{"type":"v1","addr":"172.21.2.116:6835","nonce":491943077}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":41}],"filesystems":[]} 2021-11-21T22:01:18.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3fc053560 msgr2=0x7fe3fc055a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3fc053560 0x7fe3fc055a20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fe408001020 tx=0x7fe408015000).stop 2021-11-21T22:01:18.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe41809df50 msgr2=0x7fe4181434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:18.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe41809df50 0x7fe4181434f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fe400000f70 tx=0x7fe400006df0).stop 2021-11-21T22:01:18.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 shutdown_connections 2021-11-21T22:01:18.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4181439e0 0x7fe418151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3fc053560 0x7fe3fc055a20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe41809df50 0x7fe4181434f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:18.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 --2- 172.21.2.116:0/528022997 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8003800 0x7fe4180b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:18.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 >> 172.21.2.116:0/528022997 conn(0x7fe418064d60 msgr2=0x7fe4180aaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:18.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 shutdown_connections 2021-11-21T22:01:18.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:18.938+0000 7fe41d365700 1 -- 172.21.2.116:0/528022997 wait complete. 2021-11-21T22:01:18.948 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 41 2021-11-21T22:01:18.950 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T22:01:18.950 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T22:01:18.951 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:01:18.951 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:18.951 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T22:01:18.952 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T22:01:18.952 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:01:18.952 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:18.953 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T22:01:18.953 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T22:01:18.953 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:01:18.954 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:18.956 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:01:18.957 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T22:01:18.958 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:01:18.959 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T22:01:18.960 INFO:tasks.ceph.mds.a:Started 2021-11-21T22:01:18.960 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:18.961 INFO:tasks.ceph.mds.c:Started 2021-11-21T22:01:18.962 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:18.963 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:01:18.963 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T22:01:18.965 INFO:tasks.ceph.mds.b:Started 2021-11-21T22:01:18.965 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:18.966 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T22:01:18.992 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:01:18.992 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:01:19.001 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T22:01:19.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f195ba04700 1 Processor -- start 2021-11-21T22:01:19.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f195ba04700 1 -- start start 2021-11-21T22:01:19.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f195ba04700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f19540abce0 0x7f19541478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:19.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f195ba04700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1954147e80 con 0x7f19540ac600 2021-11-21T22:01:19.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f195ba04700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19540af5a0 con 0x7f1954144810 2021-11-21T22:01:19.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f195ba04700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19540b1490 con 0x7f19540abce0 2021-11-21T22:01:19.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f1959fa1700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f19540ac600 0x7f19540aca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34182/0 (socket says 172.21.2.116:34182) 2021-11-21T22:01:19.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f1958f9f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f19540abce0 0x7f19541478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:19.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f1959fa1700 1 -- 172.21.2.116:0/322841759 learned_addr learned my addr 172.21.2.116:0/322841759 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:19.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f1958f9f700 1 -- 172.21.2.116:0/322841759 >> v1:172.21.2.117:6789/0 conn(0x7f1954144810 legacy=0x7f19540b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:19.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f1958f9f700 1 -- 172.21.2.116:0/322841759 >> v1:172.21.2.116:6789/0 conn(0x7f19540ac600 legacy=0x7f19540aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:19.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f1958f9f700 1 -- 172.21.2.116:0/322841759 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19540b1fd0 con 0x7f19540abce0 2021-11-21T22:01:19.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f1958f9f700 1 --2- 172.21.2.116:0/322841759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540abce0 0x7f19541478c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f1950002f70 tx=0x7f1950006df0).ready entity=mon.1 client_cookie=cb53c062a1ab6506 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:19.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f194bfff700 1 -- 172.21.2.116:0/322841759 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195000d610 con 0x7f19540abce0 2021-11-21T22:01:19.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f194bfff700 1 -- 172.21.2.116:0/322841759 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1950000e90 con 0x7f19540abce0 2021-11-21T22:01:19.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.014+0000 7f194bfff700 1 -- 172.21.2.116:0/322841759 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195000d960 con 0x7f19540abce0 2021-11-21T22:01:19.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/322841759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540abce0 msgr2=0x7f19541478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:19.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 --2- 172.21.2.116:0/322841759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540abce0 0x7f19541478c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f1950002f70 tx=0x7f1950006df0).stop 2021-11-21T22:01:19.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/322841759 shutdown_connections 2021-11-21T22:01:19.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 --2- 172.21.2.116:0/322841759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540abce0 0x7f19541478c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:19.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/322841759 >> 172.21.2.116:0/322841759 conn(0x7f195405bd20 msgr2=0x7f195405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:19.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/322841759 shutdown_connections 2021-11-21T22:01:19.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/322841759 wait complete. 2021-11-21T22:01:19.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 Processor -- start 2021-11-21T22:01:19.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- start start 2021-11-21T22:01:19.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19540abce0 0x7f1954063d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:19.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540ac600 0x7f1954146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:19.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1954144810 0x7f1954146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:19.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541571a0 con 0x7f1954144810 2021-11-21T22:01:19.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954146e10 con 0x7f19540ac600 2021-11-21T22:01:19.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19541573d0 con 0x7f19540abce0 2021-11-21T22:01:19.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f1958f9f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540ac600 0x7f1954146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:19.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f1959fa1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1954144810 0x7f1954146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:19.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f19597a0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19540abce0 0x7f1954063d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:19.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f1958f9f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540ac600 0x7f1954146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:51632/0 (socket says 172.21.2.116:51632) 2021-11-21T22:01:19.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f1959fa1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1954144810 0x7f1954146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50466/0 (socket says 172.21.2.116:50466) 2021-11-21T22:01:19.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f1958f9f700 1 -- 172.21.2.116:0/3757464355 learned_addr learned my addr 172.21.2.116:0/3757464355 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:19.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f19597a0700 1 -- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540ac600 msgr2=0x7f1954146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:19.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f19597a0700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540ac600 0x7f1954146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:19.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f19597a0700 1 -- 172.21.2.116:0/3757464355 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1954144810 msgr2=0x7f1954146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:19.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f19597a0700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1954144810 0x7f1954146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:19.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f19597a0700 1 -- 172.21.2.116:0/3757464355 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195000b180 con 0x7f19540abce0 2021-11-21T22:01:19.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f1958f9f700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540ac600 0x7f1954146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:01:19.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f19597a0700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19540abce0 0x7f1954063d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f19540b4e00 tx=0x7f1944007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:19.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f194a7fc700 1 -- 172.21.2.116:0/3757464355 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f194400ccd0 con 0x7f19540abce0 2021-11-21T22:01:19.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f194a7fc700 1 -- 172.21.2.116:0/3757464355 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400ce30 con 0x7f19540abce0 2021-11-21T22:01:19.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1954144e30 con 0x7f19540abce0 2021-11-21T22:01:19.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954143cb0 con 0x7f19540abce0 2021-11-21T22:01:19.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f194a7fc700 1 -- 172.21.2.116:0/3757464355 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19440125f0 con 0x7f19540abce0 2021-11-21T22:01:19.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.018+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19540af060 con 0x7f19540abce0 2021-11-21T22:01:19.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.022+0000 7f194a7fc700 1 -- 172.21.2.116:0/3757464355 <== mon.2 v2:172.21.2.117:3301/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1944019030 con 0x7f19540abce0 2021-11-21T22:01:19.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.022+0000 7f194a7fc700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f193c053690 0x7f193c055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:19.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.022+0000 7f194a7fc700 1 -- 172.21.2.116:0/3757464355 <== mon.2 v2:172.21.2.117:3301/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f194406a9c0 con 0x7f19540abce0 2021-11-21T22:01:19.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.022+0000 7f1958f9f700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f193c053690 0x7f193c055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:19.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.022+0000 7f1958f9f700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f193c053690 0x7f193c055b50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f195001c000 tx=0x7f1950015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:19.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.022+0000 7f194a7fc700 1 -- 172.21.2.116:0/3757464355 <== mon.2 v2:172.21.2.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1944041cf0 con 0x7f19540abce0 2021-11-21T22:01:19.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:19.118+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f19540ad630 con 0x7f19540abce0 2021-11-21T22:01:20.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.034+0000 7f194a7fc700 1 -- 172.21.2.116:0/3757464355 <== mon.2 v2:172.21.2.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v96) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1944045310 con 0x7f19540abce0 2021-11-21T22:01:20.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f193c053690 msgr2=0x7f193c055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f193c053690 0x7f193c055b50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f195001c000 tx=0x7f1950015000).stop 2021-11-21T22:01:20.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19540abce0 msgr2=0x7f1954063d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19540abce0 0x7f1954063d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f19540b4e00 tx=0x7f1944007b90).stop 2021-11-21T22:01:20.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 shutdown_connections 2021-11-21T22:01:20.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f193c053690 0x7f193c055b50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1954144810 0x7f1954146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19540ac600 0x7f1954146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 --2- 172.21.2.116:0/3757464355 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19540abce0 0x7f1954063d30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 >> 172.21.2.116:0/3757464355 conn(0x7f195405bd20 msgr2=0x7f19541486e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 shutdown_connections 2021-11-21T22:01:20.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.038+0000 7f195ba04700 1 -- 172.21.2.116:0/3757464355 wait complete. 2021-11-21T22:01:20.048 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T22:01:20.055 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T22:01:20.056 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T22:01:20.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 Processor -- start 2021-11-21T22:01:20.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- start start 2021-11-21T22:01:20.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85980a86b0 0x7f859814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8598062410 0x7f85981443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85980a6e80 con 0x7f85980a6aa0 2021-11-21T22:01:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85980a8bc0 con 0x7f8598062410 2021-11-21T22:01:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859814a7b0 con 0x7f85980a86b0 2021-11-21T22:01:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859e3f8700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f85980a6aa0 0x7f85981470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34198/0 (socket says 172.21.2.116:34198) 2021-11-21T22:01:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859e3f8700 1 -- 172.21.2.116:0/2603369145 learned_addr learned my addr 172.21.2.116:0/2603369145 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859ebf9700 1 --2- 172.21.2.116:0/2603369145 >> v2:172.21.2.116:3300/0 conn(0x7f8598062410 0x7f85981443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859ebf9700 1 -- 172.21.2.116:0/2603369145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85980a86b0 msgr2=0x7f859814b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859ebf9700 1 --2- 172.21.2.116:0/2603369145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85980a86b0 0x7f859814b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859ebf9700 1 -- 172.21.2.116:0/2603369145 >> v1:172.21.2.116:6789/0 conn(0x7f85980a6aa0 legacy=0x7f85981470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859ebf9700 1 -- 172.21.2.116:0/2603369145 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859814aa10 con 0x7f8598062410 2021-11-21T22:01:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859ebf9700 1 --2- 172.21.2.116:0/2603369145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8598062410 0x7f85981443f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f85900032f0 tx=0x7f85900091d0).ready entity=mon.0 client_cookie=945d5198a3bd034f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859d3f6700 1 -- 172.21.2.116:0/2603369145 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8590009ea0 con 0x7f8598062410 2021-11-21T22:01:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859d3f6700 1 -- 172.21.2.116:0/2603369145 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859000b8a0 con 0x7f8598062410 2021-11-21T22:01:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f859d3f6700 1 -- 172.21.2.116:0/2603369145 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f859000ba50 con 0x7f8598062410 2021-11-21T22:01:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- 172.21.2.116:0/2603369145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8598062410 msgr2=0x7f85981443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 --2- 172.21.2.116:0/2603369145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8598062410 0x7f85981443f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f85900032f0 tx=0x7f85900091d0).stop 2021-11-21T22:01:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- 172.21.2.116:0/2603369145 shutdown_connections 2021-11-21T22:01:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 --2- 172.21.2.116:0/2603369145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8598062410 0x7f85981443f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 --2- 172.21.2.116:0/2603369145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85980a86b0 0x7f859814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- 172.21.2.116:0/2603369145 >> 172.21.2.116:0/2603369145 conn(0x7f859805bd20 msgr2=0x7f859805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- 172.21.2.116:0/2603369145 shutdown_connections 2021-11-21T22:01:20.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.146+0000 7f85a065c700 1 -- 172.21.2.116:0/2603369145 wait complete. 2021-11-21T22:01:20.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 Processor -- start 2021-11-21T22:01:20.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 -- start start 2021-11-21T22:01:20.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8598062410 0x7f8598064260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85980a6aa0 0x7f85980a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85980a86b0 0x7f85980aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85980a9dc0 con 0x7f85980a86b0 2021-11-21T22:01:20.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598147840 con 0x7f85980a6aa0 2021-11-21T22:01:20.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8598147ae0 con 0x7f8598062410 2021-11-21T22:01:20.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85980a86b0 0x7f85980aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85980a86b0 0x7f85980aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50482/0 (socket says 172.21.2.116:50482) 2021-11-21T22:01:20.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 -- 172.21.2.116:0/2242574138 learned_addr learned my addr 172.21.2.116:0/2242574138 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 -- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8598062410 msgr2=0x7f8598064260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8598062410 0x7f8598064260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 -- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85980a6aa0 msgr2=0x7f85980a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85980a6aa0 0x7f85980a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 -- 172.21.2.116:0/2242574138 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8590009470 con 0x7f85980a86b0 2021-11-21T22:01:20.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f859ebf9700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85980a86b0 0x7f85980aa4e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f85900032f0 tx=0x7f8590016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f858b7fe700 1 -- 172.21.2.116:0/2242574138 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8590009ea0 con 0x7f85980a86b0 2021-11-21T22:01:20.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f858b7fe700 1 -- 172.21.2.116:0/2242574138 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859001f5e0 con 0x7f85980a86b0 2021-11-21T22:01:20.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85980aa9d0 con 0x7f85980a86b0 2021-11-21T22:01:20.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f858b7fe700 1 -- 172.21.2.116:0/2242574138 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f859000bee0 con 0x7f85980a86b0 2021-11-21T22:01:20.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8598157470 con 0x7f85980a86b0 2021-11-21T22:01:20.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85980573d0 con 0x7f85980a86b0 2021-11-21T22:01:20.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f858b7fe700 1 -- 172.21.2.116:0/2242574138 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8590026030 con 0x7f85980a86b0 2021-11-21T22:01:20.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f858b7fe700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85780536a0 0x7f8578055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.150+0000 7f858b7fe700 1 -- 172.21.2.116:0/2242574138 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f859006c320 con 0x7f85980a86b0 2021-11-21T22:01:20.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.154+0000 7f859e3f8700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85780536a0 0x7f8578055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.154+0000 7f859e3f8700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85780536a0 0x7f8578055b60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8598061ab0 tx=0x7f858c0077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.154+0000 7f858b7fe700 1 -- 172.21.2.116:0/2242574138 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85900437b0 con 0x7f85980a86b0 2021-11-21T22:01:20.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f85980a7ea0 con 0x7f85980a86b0 2021-11-21T22:01:20.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f858b7fe700 1 -- 172.21.2.116:0/2242574138 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f8590015090 con 0x7f85980a86b0 2021-11-21T22:01:20.267 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:20.267 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:20.268 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T22:01:20.268 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.268 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T22:01:20.268 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T22:01:20.269 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.269 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:20.269 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.269 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.269 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.270 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.270 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.270 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.270 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T22:01:20.271 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T22:01:20.271 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.271 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:20.271 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.271 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.272 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.272 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.272 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.272 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.273 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T22:01:20.273 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T22:01:20.273 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.273 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:20.273 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.274 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.274 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.274 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.274 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.275 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.275 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T22:01:20.275 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T22:01:20.275 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.275 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.276 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.276 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.276 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.276 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.277 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.277 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T22:01:20.277 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T22:01:20.277 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.277 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.278 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.278 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.278 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.278 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.279 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.279 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T22:01:20.279 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T22:01:20.279 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.279 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.280 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.280 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.280 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.280 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.280 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.281 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T22:01:20.281 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T22:01:20.281 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.281 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.282 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.282 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.282 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.282 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.282 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.283 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T22:01:20.283 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T22:01:20.283 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.283 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.284 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.284 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.284 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.284 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.284 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.285 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T22:01:20.285 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T22:01:20.285 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.285 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.286 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.286 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.286 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.286 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.287 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.287 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T22:01:20.287 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T22:01:20.287 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.287 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.288 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.288 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.288 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.288 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.289 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.289 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T22:01:20.289 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T22:01:20.289 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.289 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.290 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.290 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.290 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.290 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.291 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.291 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T22:01:20.291 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T22:01:20.291 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.291 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:20.292 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:20.292 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:20.292 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.292 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.293 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.293 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T22:01:20.293 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T22:01:20.293 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.293 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:20.294 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:20.294 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:20.294 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.294 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.295 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.295 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T22:01:20.295 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T22:01:20.295 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.295 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:20.296 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:20.296 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:20.296 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.296 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.297 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.297 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.297 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T22:01:20.297 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T22:01:20.298 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.298 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:20.298 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:20.298 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.299 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.299 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.299 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.299 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T22:01:20.299 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T22:01:20.300 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:20.300 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:20.300 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:20.300 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:20.301 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.301 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.301 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:20.301 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:20.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85780536a0 msgr2=0x7f8578055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85780536a0 0x7f8578055b60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8598061ab0 tx=0x7f858c0077f0).stop 2021-11-21T22:01:20.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85980a86b0 msgr2=0x7f85980aa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85980a86b0 0x7f85980aa4e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f85900032f0 tx=0x7f8590016000).stop 2021-11-21T22:01:20.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 shutdown_connections 2021-11-21T22:01:20.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85780536a0 0x7f8578055b60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85980a86b0 0x7f85980aa4e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85980a6aa0 0x7f85980a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 --2- 172.21.2.116:0/2242574138 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8598062410 0x7f8598064260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 >> 172.21.2.116:0/2242574138 conn(0x7f859805bd20 msgr2=0x7f85980b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 shutdown_connections 2021-11-21T22:01:20.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.262+0000 7f85a065c700 1 -- 172.21.2.116:0/2242574138 wait complete. 2021-11-21T22:01:20.306 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:20.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3df0843700 1 Processor -- start 2021-11-21T22:01:20.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3df0843700 1 -- start start 2021-11-21T22:01:20.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3df0843700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec0a86b0 0x7f3dec14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3df0843700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec0a6e80 con 0x7f3dec0a6aa0 2021-11-21T22:01:20.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3df0843700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec14a7b0 con 0x7f3dec062410 2021-11-21T22:01:20.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3df0843700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0a8bc0 con 0x7f3dec0a86b0 2021-11-21T22:01:20.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3dea59c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f3dec0a6aa0 0x7f3dec1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34212/0 (socket says 172.21.2.116:34212) 2021-11-21T22:01:20.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3dea59c700 1 -- 172.21.2.116:0/3707243050 learned_addr learned my addr 172.21.2.116:0/3707243050 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:20.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060276229 0 0) 0x7f3dec0a6e80 con 0x7f3dec0a6aa0 2021-11-21T22:01:20.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dcc003260 con 0x7f3dec0a6aa0 2021-11-21T22:01:20.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3692957885 0 0) 0x7f3dcc003260 con 0x7f3dec0a6aa0 2021-11-21T22:01:20.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec0a86b0 msgr2=0x7f3dec14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec0a86b0 0x7f3dec14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 >> v1:172.21.2.117:6789/0 conn(0x7f3dec062410 legacy=0x7f3dec1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec14aa10 con 0x7f3dec0a6aa0 2021-11-21T22:01:20.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f3dd4001680 con 0x7f3dec0a6aa0 2021-11-21T22:01:20.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 >> v1:172.21.2.116:6789/0 conn(0x7f3dec0a6aa0 legacy=0x7f3dec1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.362+0000 7f3de959a700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dcc003780 0x7f3dcc003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc004500 0x7f3dcc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec0a86b0 0x7f3dcc004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0a8bc0 con 0x7f3dcc003780 2021-11-21T22:01:20.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc00b910 con 0x7f3dcc004500 2021-11-21T22:01:20.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc00bad0 con 0x7f3dec0a86b0 2021-11-21T22:01:20.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dcc003780 0x7f3dcc003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 -- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec0a86b0 msgr2=0x7f3dcc004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec0a86b0 0x7f3dcc004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 -- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc004500 msgr2=0x7f3dcc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc004500 0x7f3dcc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 -- 172.21.2.116:0/3707243050 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc003260 con 0x7f3dcc003780 2021-11-21T22:01:20.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dcc003780 0x7f3dcc003b80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f3dec061b50 tx=0x7f3dd4001e50).ready entity=mon.0 client_cookie=12e77021e54824df server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd40070d0 con 0x7f3dcc003780 2021-11-21T22:01:20.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd4007230 con 0x7f3dcc003780 2021-11-21T22:01:20.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de959a700 1 -- 172.21.2.116:0/3707243050 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd400c8c0 con 0x7f3dcc003780 2021-11-21T22:01:20.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- 172.21.2.116:0/3707243050 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dcc003780 msgr2=0x7f3dcc003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 --2- 172.21.2.116:0/3707243050 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dcc003780 0x7f3dcc003b80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f3dec061b50 tx=0x7f3dd4001e50).stop 2021-11-21T22:01:20.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dead9d700 1 -- 172.21.2.116:0/3707243050 reap_dead start 2021-11-21T22:01:20.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- 172.21.2.116:0/3707243050 shutdown_connections 2021-11-21T22:01:20.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- 172.21.2.116:0/3707243050 >> 172.21.2.116:0/3707243050 conn(0x7f3dec05bd20 msgr2=0x7f3dec05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- 172.21.2.116:0/3707243050 shutdown_connections 2021-11-21T22:01:20.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- 172.21.2.116:0/3707243050 wait complete. 2021-11-21T22:01:20.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 Processor -- start 2021-11-21T22:01:20.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- start start 2021-11-21T22:01:20.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc003780 0x7f3dec147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dec1438b0 0x7f3dec159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec063cb0 0x7f3dec156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0646d0 con 0x7f3dec1438b0 2021-11-21T22:01:20.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec064430 con 0x7f3dcc003780 2021-11-21T22:01:20.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec156220 con 0x7f3dec063cb0 2021-11-21T22:01:20.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dec1438b0 0x7f3dec159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dec1438b0 0x7f3dec159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50504/0 (socket says 172.21.2.116:50504) 2021-11-21T22:01:20.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 -- 172.21.2.116:0/1514454873 learned_addr learned my addr 172.21.2.116:0/1514454873 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3dea59c700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc003780 0x7f3dec147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 -- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec063cb0 msgr2=0x7f3dec156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec063cb0 0x7f3dec156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 -- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc003780 msgr2=0x7f3dec147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc003780 0x7f3dec147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 -- 172.21.2.116:0/1514454873 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec1564c0 con 0x7f3dec1438b0 2021-11-21T22:01:20.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de9d9b700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dec1438b0 0x7f3dec159610 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f3de4002f70 tx=0x7f3de4006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de37fe700 1 -- 172.21.2.116:0/1514454873 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3de400e470 con 0x7f3dec1438b0 2021-11-21T22:01:20.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de37fe700 1 -- 172.21.2.116:0/1514454873 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4000e90 con 0x7f3dec1438b0 2021-11-21T22:01:20.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec064130 con 0x7f3dec1438b0 2021-11-21T22:01:20.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3de37fe700 1 -- 172.21.2.116:0/1514454873 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3de400e7c0 con 0x7f3dec1438b0 2021-11-21T22:01:20.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.366+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec066b20 con 0x7f3dec1438b0 2021-11-21T22:01:20.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.370+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dec0573d0 con 0x7f3dec1438b0 2021-11-21T22:01:20.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.370+0000 7f3de37fe700 1 -- 172.21.2.116:0/1514454873 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3de400f070 con 0x7f3dec1438b0 2021-11-21T22:01:20.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.370+0000 7f3de37fe700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3dd0053610 0x7f3dd0055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.370+0000 7f3de37fe700 1 -- 172.21.2.116:0/1514454873 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3de406a9b0 con 0x7f3dec1438b0 2021-11-21T22:01:20.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.370+0000 7f3dea59c700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3dd0053610 0x7f3dd0055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.370+0000 7f3dea59c700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3dd0053610 0x7f3dd0055ad0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3dd401a000 tx=0x7f3dd4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.370+0000 7f3de37fe700 1 -- 172.21.2.116:0/1514454873 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3de4041e40 con 0x7f3dec1438b0 2021-11-21T22:01:20.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dec05db70 con 0x7f3dec1438b0 2021-11-21T22:01:20.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3de37fe700 1 -- 172.21.2.116:0/1514454873 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+13182 (secure 0 0 0) 0x7f3de4045460 con 0x7f3dec1438b0 2021-11-21T22:01:20.458 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:20.458 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":96,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:20.026161+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:20.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3dd0053610 msgr2=0x7f3dd0055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3dd0053610 0x7f3dd0055ad0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3dd401a000 tx=0x7f3dd4015000).stop 2021-11-21T22:01:20.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dec1438b0 msgr2=0x7f3dec159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dec1438b0 0x7f3dec159610 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f3de4002f70 tx=0x7f3de4006e40).stop 2021-11-21T22:01:20.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 shutdown_connections 2021-11-21T22:01:20.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3dec063cb0 0x7f3dec156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3dd0053610 0x7f3dd0055ad0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3dec1438b0 0x7f3dec159610 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 --2- 172.21.2.116:0/1514454873 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3dcc003780 0x7f3dec147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 >> 172.21.2.116:0/1514454873 conn(0x7f3dec05bd20 msgr2=0x7f3dec0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 shutdown_connections 2021-11-21T22:01:20.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.454+0000 7f3df0843700 1 -- 172.21.2.116:0/1514454873 wait complete. 2021-11-21T22:01:20.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T22:01:20.466 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:20.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72cc1e700 1 Processor -- start 2021-11-21T22:01:20.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72cc1e700 1 -- start start 2021-11-21T22:01:20.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72cc1e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc728144810 0x7fc7281478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72cc1e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 0x7fc7280aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72cc1e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc728147e80 con 0x7fc7280abce0 2021-11-21T22:01:20.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72cc1e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7280b1490 con 0x7fc728144810 2021-11-21T22:01:20.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72cc1e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280b1fd0 con 0x7fc7280ac600 2021-11-21T22:01:20.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc728144810 0x7fc7281478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc728144810 0x7fc7281478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50508/0 (socket says 172.21.2.116:50508) 2021-11-21T22:01:20.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 -- 172.21.2.116:0/3675680150 learned_addr learned my addr 172.21.2.116:0/3675680150 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 -- 172.21.2.116:0/3675680150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 msgr2=0x7fc7280aca90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:20.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 --2- 172.21.2.116:0/3675680150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 0x7fc7280aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 -- 172.21.2.116:0/3675680150 >> v1:172.21.2.117:6789/0 conn(0x7fc7280abce0 legacy=0x7fc7280b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:20.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 -- 172.21.2.116:0/3675680150 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280af5a0 con 0x7fc728144810 2021-11-21T22:01:20.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72659c700 1 --2- 172.21.2.116:0/3675680150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728144810 0x7fc7281478c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7fc7280b4e00 tx=0x7fc718006eb0).ready entity=mon.0 client_cookie=617275d23d6230e7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72559a700 1 -- 172.21.2.116:0/3675680150 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc71800d570 con 0x7fc728144810 2021-11-21T22:01:20.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72559a700 1 -- 172.21.2.116:0/3675680150 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718000e90 con 0x7fc728144810 2021-11-21T22:01:20.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.518+0000 7fc72559a700 1 -- 172.21.2.116:0/3675680150 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc71800d8c0 con 0x7fc728144810 2021-11-21T22:01:20.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3675680150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728144810 msgr2=0x7fc7281478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3675680150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728144810 0x7fc7281478c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7fc7280b4e00 tx=0x7fc718006eb0).stop 2021-11-21T22:01:20.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3675680150 shutdown_connections 2021-11-21T22:01:20.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3675680150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 0x7fc7280aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3675680150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc728144810 0x7fc7281478c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3675680150 >> 172.21.2.116:0/3675680150 conn(0x7fc72805bd20 msgr2=0x7fc72805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3675680150 shutdown_connections 2021-11-21T22:01:20.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3675680150 wait complete. 2021-11-21T22:01:20.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 Processor -- start 2021-11-21T22:01:20.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- start start 2021-11-21T22:01:20.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7280abce0 0x7fc7280b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 0x7fc72813c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728144810 0x7fc728159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728143680 con 0x7fc7280abce0 2021-11-21T22:01:20.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728149a50 con 0x7fc728144810 2021-11-21T22:01:20.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281438b0 con 0x7fc7280ac600 2021-11-21T22:01:20.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7280abce0 0x7fc7280b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7280abce0 0x7fc7280b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50516/0 (socket says 172.21.2.116:50516) 2021-11-21T22:01:20.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 -- 172.21.2.116:0/3731039823 learned_addr learned my addr 172.21.2.116:0/3731039823 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 -- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 msgr2=0x7fc72813c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 0x7fc72813c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 -- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728144810 msgr2=0x7fc728159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728144810 0x7fc728159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 -- 172.21.2.116:0/3731039823 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71800b1e0 con 0x7fc7280abce0 2021-11-21T22:01:20.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72659c700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7280abce0 0x7fc7280b4290 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7fc7280b4e00 tx=0x7fc718015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc71f7fe700 1 -- 172.21.2.116:0/3731039823 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc71800d8f0 con 0x7fc7280abce0 2021-11-21T22:01:20.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc71f7fe700 1 -- 172.21.2.116:0/3731039823 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718011ce0 con 0x7fc7280abce0 2021-11-21T22:01:20.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc728149c80 con 0x7fc7280abce0 2021-11-21T22:01:20.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7280b2b20 con 0x7fc7280abce0 2021-11-21T22:01:20.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc71f7fe700 1 -- 172.21.2.116:0/3731039823 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc71801e650 con 0x7fc7280abce0 2021-11-21T22:01:20.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7280af060 con 0x7fc7280abce0 2021-11-21T22:01:20.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc71f7fe700 1 -- 172.21.2.116:0/3731039823 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc718025030 con 0x7fc7280abce0 2021-11-21T22:01:20.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc71f7fe700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc70c0535d0 0x7fc70c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.522+0000 7fc71f7fe700 1 -- 172.21.2.116:0/3731039823 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc71806caa0 con 0x7fc7280abce0 2021-11-21T22:01:20.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.526+0000 7fc725d9b700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc70c0535d0 0x7fc70c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.526+0000 7fc725d9b700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc70c0535d0 0x7fc70c055a90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fc710002fb0 tx=0x7fc7100069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.526+0000 7fc71f7fe700 1 -- 172.21.2.116:0/3731039823 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc718043f30 con 0x7fc7280abce0 2021-11-21T22:01:20.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc7280573d0 con 0x7fc7280abce0 2021-11-21T22:01:20.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc71f7fe700 1 -- 172.21.2.116:0/3731039823 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v96) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc7180230b0 con 0x7fc7280abce0 2021-11-21T22:01:20.615 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:20.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc70c0535d0 msgr2=0x7fc70c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc70c0535d0 0x7fc70c055a90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fc710002fb0 tx=0x7fc7100069b0).stop 2021-11-21T22:01:20.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7280abce0 msgr2=0x7fc7280b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7280abce0 0x7fc7280b4290 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7fc7280b4e00 tx=0x7fc718015000).stop 2021-11-21T22:01:20.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 shutdown_connections 2021-11-21T22:01:20.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc728144810 0x7fc728159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7280ac600 0x7fc72813c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc70c0535d0 0x7fc70c055a90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 --2- 172.21.2.116:0/3731039823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7280abce0 0x7fc7280b4290 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 >> 172.21.2.116:0/3731039823 conn(0x7fc72805bd20 msgr2=0x7fc7280b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 shutdown_connections 2021-11-21T22:01:20.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.610+0000 7fc72cc1e700 1 -- 172.21.2.116:0/3731039823 wait complete. 2021-11-21T22:01:20.621 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:20.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 Processor -- start 2021-11-21T22:01:20.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- start start 2021-11-21T22:01:20.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55381426e0 0x7f5538142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 0x7f55380ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55380afae0 con 0x7f5538062e60 2021-11-21T22:01:20.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55380b0f60 con 0x7f55381426e0 2021-11-21T22:01:20.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5538063470 con 0x7f55380ad470 2021-11-21T22:01:20.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55381426e0 0x7f5538142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55381426e0 0x7f5538142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50522/0 (socket says 172.21.2.116:50522) 2021-11-21T22:01:20.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 -- 172.21.2.116:0/2704613103 learned_addr learned my addr 172.21.2.116:0/2704613103 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 -- 172.21.2.116:0/2704613103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 msgr2=0x7f55380ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 --2- 172.21.2.116:0/2704613103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 0x7f55380ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 -- 172.21.2.116:0/2704613103 >> v1:172.21.2.116:6789/0 conn(0x7f5538062e60 legacy=0x7f55380af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 -- 172.21.2.116:0/2704613103 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538143050 con 0x7f55381426e0 2021-11-21T22:01:20.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553cada700 1 --2- 172.21.2.116:0/2704613103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538142ac0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f5528002f70 tx=0x7f5528006f40).ready entity=mon.0 client_cookie=413468b5aabc6e72 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f55377fe700 1 -- 172.21.2.116:0/2704613103 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f552800dc70 con 0x7f55381426e0 2021-11-21T22:01:20.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f55377fe700 1 -- 172.21.2.116:0/2704613103 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5528000e90 con 0x7f55381426e0 2021-11-21T22:01:20.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f55377fe700 1 -- 172.21.2.116:0/2704613103 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f552800be90 con 0x7f55381426e0 2021-11-21T22:01:20.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- 172.21.2.116:0/2704613103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 msgr2=0x7f5538142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 --2- 172.21.2.116:0/2704613103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538142ac0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f5528002f70 tx=0x7f5528006f40).stop 2021-11-21T22:01:20.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- 172.21.2.116:0/2704613103 shutdown_connections 2021-11-21T22:01:20.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 --2- 172.21.2.116:0/2704613103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 0x7f55380ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 --2- 172.21.2.116:0/2704613103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538142ac0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- 172.21.2.116:0/2704613103 >> 172.21.2.116:0/2704613103 conn(0x7f55380b5730 msgr2=0x7f55380b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- 172.21.2.116:0/2704613103 shutdown_connections 2021-11-21T22:01:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.670+0000 7f553ed3e700 1 -- 172.21.2.116:0/2704613103 wait complete. 2021-11-21T22:01:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 Processor -- start 2021-11-21T22:01:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 -- start start 2021-11-21T22:01:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 0x7f5538145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5538066a60 0x7f5538152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381443a0 con 0x7f55381426e0 2021-11-21T22:01:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5538066020 con 0x7f5538066a60 2021-11-21T22:01:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55381440d0 con 0x7f55380ad470 2021-11-21T22:01:20.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50526/0 (socket says 172.21.2.116:50526) 2021-11-21T22:01:20.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 -- 172.21.2.116:0/1227483737 learned_addr learned my addr 172.21.2.116:0/1227483737 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 -- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 msgr2=0x7f5538145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 0x7f5538145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 -- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5538066a60 msgr2=0x7f5538152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5538066a60 0x7f5538152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 -- 172.21.2.116:0/1227483737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552800b460 con 0x7f55381426e0 2021-11-21T22:01:20.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5537fff700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538151e20 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f5538062d10 tx=0x7f5520007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5535ffb700 1 -- 172.21.2.116:0/1227483737 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f552000ce30 con 0x7f55381426e0 2021-11-21T22:01:20.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5535ffb700 1 -- 172.21.2.116:0/1227483737 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552000f470 con 0x7f55381426e0 2021-11-21T22:01:20.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5535ffb700 1 -- 172.21.2.116:0/1227483737 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f552000f7d0 con 0x7f55381426e0 2021-11-21T22:01:20.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55380664c0 con 0x7f55381426e0 2021-11-21T22:01:20.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55380652e0 con 0x7f55381426e0 2021-11-21T22:01:20.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538146510 con 0x7f55381426e0 2021-11-21T22:01:20.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5535ffb700 1 -- 172.21.2.116:0/1227483737 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5520019030 con 0x7f55381426e0 2021-11-21T22:01:20.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5535ffb700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f551c053660 0x7f551c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f5535ffb700 1 -- 172.21.2.116:0/1227483737 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f552006abe0 con 0x7f55381426e0 2021-11-21T22:01:20.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.674+0000 7f553cada700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f551c053660 0x7f551c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.678+0000 7f553cada700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f551c053660 0x7f551c055b20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f5528002f70 tx=0x7f5528006f40).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.678+0000 7f5535ffb700 1 -- 172.21.2.116:0/1227483737 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5520041f60 con 0x7f55381426e0 2021-11-21T22:01:20.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5538144570 con 0x7f55381426e0 2021-11-21T22:01:20.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f5535ffb700 1 -- 172.21.2.116:0/1227483737 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5520045580 con 0x7f55381426e0 2021-11-21T22:01:20.765 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:20.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f551c053660 msgr2=0x7f551c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f551c053660 0x7f551c055b20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f5528002f70 tx=0x7f5528006f40).stop 2021-11-21T22:01:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 msgr2=0x7f5538151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538151e20 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f5538062d10 tx=0x7f5520007c50).stop 2021-11-21T22:01:20.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 shutdown_connections 2021-11-21T22:01:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5538066a60 0x7f5538152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f551c053660 0x7f551c055b20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55381426e0 0x7f5538151e20 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 --2- 172.21.2.116:0/1227483737 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55380ad470 0x7f5538145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 >> 172.21.2.116:0/1227483737 conn(0x7f55380b5730 msgr2=0x7f55380b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 shutdown_connections 2021-11-21T22:01:20.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.762+0000 7f553ed3e700 1 -- 172.21.2.116:0/1227483737 wait complete. 2021-11-21T22:01:20.772 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T22:01:20.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.818+0000 7fe97aa15700 1 Processor -- start 2021-11-21T22:01:20.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.818+0000 7fe97aa15700 1 -- start start 2021-11-21T22:01:20.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740aed90 0x7fe9740b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9740b00d0 con 0x7fe9740b24a0 2021-11-21T22:01:20.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9740b2880 con 0x7fe9740af360 2021-11-21T22:01:20.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740adba0 con 0x7fe9740aed90 2021-11-21T22:01:20.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe973fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe9740b24a0 0x7fe9740ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34258/0 (socket says 172.21.2.116:34258) 2021-11-21T22:01:20.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe973fff700 1 -- 172.21.2.116:0/2746660015 learned_addr learned my addr 172.21.2.116:0/2746660015 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:20.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528474433 0 0) 0x7fe9740b00d0 con 0x7fe9740b24a0 2021-11-21T22:01:20.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe960003260 con 0x7fe9740b24a0 2021-11-21T22:01:20.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182082547 0 0) 0x7fe960003260 con 0x7fe9740b24a0 2021-11-21T22:01:20.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740aed90 msgr2=0x7fe9740b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740aed90 0x7fe9740b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 >> v1:172.21.2.117:6789/0 conn(0x7fe9740af360 legacy=0x7fe974069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740b14a0 con 0x7fe9740b24a0 2021-11-21T22:01:20.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe968001660 con 0x7fe9740b24a0 2021-11-21T22:01:20.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 >> v1:172.21.2.116:6789/0 conn(0x7fe9740b24a0 legacy=0x7fe9740ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740aed90 0x7fe960003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe960003ef0 0x7fe960004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe960004bf0 0x7fe96000d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740adba0 con 0x7fe960004bf0 2021-11-21T22:01:20.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600052f0 con 0x7fe960003ef0 2021-11-21T22:01:20.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96000de50 con 0x7fe9740aed90 2021-11-21T22:01:20.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe960004bf0 0x7fe96000d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 -- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740aed90 msgr2=0x7fe960003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe9740aed90 0x7fe960003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 -- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe960003ef0 msgr2=0x7fe960004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe960003ef0 0x7fe960004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 -- 172.21.2.116:0/2746660015 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960003260 con 0x7fe960004bf0 2021-11-21T22:01:20.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe960004bf0 0x7fe96000d960 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fe96c0012f0 tx=0x7fe96c007d60).ready entity=mon.0 client_cookie=6e3eb621f55790c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe96c0014c0 con 0x7fe960004bf0 2021-11-21T22:01:20.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c006870 con 0x7fe960004bf0 2021-11-21T22:01:20.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe972ffd700 1 -- 172.21.2.116:0/2746660015 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe96c006bd0 con 0x7fe960004bf0 2021-11-21T22:01:20.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- 172.21.2.116:0/2746660015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe960004bf0 msgr2=0x7fe96000d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 --2- 172.21.2.116:0/2746660015 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe960004bf0 0x7fe96000d960 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fe96c0012f0 tx=0x7fe96c007d60).stop 2021-11-21T22:01:20.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe978fb2700 1 -- 172.21.2.116:0/2746660015 reap_dead start 2021-11-21T22:01:20.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- 172.21.2.116:0/2746660015 shutdown_connections 2021-11-21T22:01:20.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- 172.21.2.116:0/2746660015 >> 172.21.2.116:0/2746660015 conn(0x7fe97405fd00 msgr2=0x7fe9740600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- 172.21.2.116:0/2746660015 shutdown_connections 2021-11-21T22:01:20.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- 172.21.2.116:0/2746660015 wait complete. 2021-11-21T22:01:20.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 Processor -- start 2021-11-21T22:01:20.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- start start 2021-11-21T22:01:20.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe960004bf0 0x7fe974110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9741124c0 0x7fe97414d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe974066760 0x7fe974111aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974110af0 con 0x7fe9741124c0 2021-11-21T22:01:20.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974110d90 con 0x7fe974066760 2021-11-21T22:01:20.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe974111030 con 0x7fe960004bf0 2021-11-21T22:01:20.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9741124c0 0x7fe97414d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9741124c0 0x7fe97414d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50550/0 (socket says 172.21.2.116:50550) 2021-11-21T22:01:20.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 -- 172.21.2.116:0/2541032520 learned_addr learned my addr 172.21.2.116:0/2541032520 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 -- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe960004bf0 msgr2=0x7fe974110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe960004bf0 0x7fe974110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 -- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe974066760 msgr2=0x7fe974111aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe974066760 0x7fe974111aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 -- 172.21.2.116:0/2541032520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe974111600 con 0x7fe9741124c0 2021-11-21T22:01:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9737fe700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9741124c0 0x7fe97414d770 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fe964002f70 tx=0x7fe964006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9717fa700 1 -- 172.21.2.116:0/2541032520 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe964010960 con 0x7fe9741124c0 2021-11-21T22:01:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9717fa700 1 -- 172.21.2.116:0/2541032520 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe964010ac0 con 0x7fe9741124c0 2021-11-21T22:01:20.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe9717fa700 1 -- 172.21.2.116:0/2541032520 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe964010e20 con 0x7fe9741124c0 2021-11-21T22:01:20.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe974114600 con 0x7fe9741124c0 2021-11-21T22:01:20.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.822+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741121b0 con 0x7fe9741124c0 2021-11-21T22:01:20.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.826+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9740573d0 con 0x7fe9741124c0 2021-11-21T22:01:20.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.826+0000 7fe9717fa700 1 -- 172.21.2.116:0/2541032520 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe96401a420 con 0x7fe9741124c0 2021-11-21T22:01:20.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.826+0000 7fe9717fa700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9540536d0 0x7fe954055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.826+0000 7fe9717fa700 1 -- 172.21.2.116:0/2541032520 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe964015020 con 0x7fe9741124c0 2021-11-21T22:01:20.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.826+0000 7fe973fff700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9540536d0 0x7fe954055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.826+0000 7fe973fff700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9540536d0 0x7fe954055b90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fe9740b5680 tx=0x7fe968001660).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.826+0000 7fe9717fa700 1 -- 172.21.2.116:0/2541032520 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe964046a80 con 0x7fe9741124c0 2021-11-21T22:01:20.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe97405f2d0 con 0x7fe9741124c0 2021-11-21T22:01:20.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe9717fa700 1 -- 172.21.2.116:0/2541032520 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe96404a0a0 con 0x7fe9741124c0 2021-11-21T22:01:20.915 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:20.916 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T22:01:20.916 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.916 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T22:01:20.916 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T22:01:20.917 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:20.917 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:20.917 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T22:01:20.917 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T22:01:20.917 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:20.918 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T22:01:20.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9540536d0 msgr2=0x7fe954055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9540536d0 0x7fe954055b90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fe9740b5680 tx=0x7fe968001660).stop 2021-11-21T22:01:20.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9741124c0 msgr2=0x7fe97414d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9741124c0 0x7fe97414d770 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fe964002f70 tx=0x7fe964006e70).stop 2021-11-21T22:01:20.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 shutdown_connections 2021-11-21T22:01:20.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe974066760 0x7fe974111aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe9540536d0 0x7fe954055b90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe9741124c0 0x7fe97414d770 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 --2- 172.21.2.116:0/2541032520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe960004bf0 0x7fe974110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:20.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 >> 172.21.2.116:0/2541032520 conn(0x7fe97405fd00 msgr2=0x7fe97413fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.910+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 shutdown_connections 2021-11-21T22:01:20.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.914+0000 7fe97aa15700 1 -- 172.21.2.116:0/2541032520 wait complete. 2021-11-21T22:01:20.922 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T22:01:20.923 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:01:20.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90b06e9700 1 Processor -- start 2021-11-21T22:01:20.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90b06e9700 1 -- start start 2021-11-21T22:01:20.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90b06e9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f90a8144810 0x7f90a81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90b06e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a80abce0 0x7f90a80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90b06e9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a8147e80 con 0x7f90a80b1db0 2021-11-21T22:01:20.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90b06e9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a80b1490 con 0x7f90a8144810 2021-11-21T22:01:20.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90b06e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80af060 con 0x7f90a80abce0 2021-11-21T22:01:20.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90ae485700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f90a8144810 0x7f90a81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90aec86700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f90a80b1db0 0x7f90a80ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34280/0 (socket says 172.21.2.116:34280) 2021-11-21T22:01:20.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90aec86700 1 -- 172.21.2.116:0/3580371730 learned_addr learned my addr 172.21.2.116:0/3580371730 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:20.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90adc84700 1 --2- 172.21.2.116:0/3580371730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a80abce0 0x7f90a80b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90ae485700 1 -- 172.21.2.116:0/3580371730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a80abce0 msgr2=0x7f90a80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:20.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90ae485700 1 --2- 172.21.2.116:0/3580371730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a80abce0 0x7f90a80b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90ae485700 1 -- 172.21.2.116:0/3580371730 >> v1:172.21.2.116:6789/0 conn(0x7f90a80b1db0 legacy=0x7f90a80ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90ad483700 1 -- 172.21.2.116:0/3580371730 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040295811 0 0) 0x7f90a8147e80 con 0x7f90a80b1db0 2021-11-21T22:01:20.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90ae485700 1 -- 172.21.2.116:0/3580371730 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a80af5a0 con 0x7f90a8144810 2021-11-21T22:01:20.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.970+0000 7f90ae485700 1 --2- 172.21.2.116:0/3580371730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a8144810 0x7f90a81478c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f90a4002f70 tx=0x7f90a4006df0).ready entity=mon.0 client_cookie=3066d84a1e691856 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90ad483700 1 -- 172.21.2.116:0/3580371730 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90a400d570 con 0x7f90a8144810 2021-11-21T22:01:20.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90ad483700 1 -- 172.21.2.116:0/3580371730 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a4000e90 con 0x7f90a8144810 2021-11-21T22:01:20.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90ad483700 1 -- 172.21.2.116:0/3580371730 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90a400d8c0 con 0x7f90a8144810 2021-11-21T22:01:20.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/3580371730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a8144810 msgr2=0x7f90a81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 --2- 172.21.2.116:0/3580371730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a8144810 0x7f90a81478c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f90a4002f70 tx=0x7f90a4006df0).stop 2021-11-21T22:01:20.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/3580371730 shutdown_connections 2021-11-21T22:01:20.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 --2- 172.21.2.116:0/3580371730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a80abce0 0x7f90a80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 --2- 172.21.2.116:0/3580371730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a8144810 0x7f90a81478c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/3580371730 >> 172.21.2.116:0/3580371730 conn(0x7f90a805bd20 msgr2=0x7f90a805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:20.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/3580371730 shutdown_connections 2021-11-21T22:01:20.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/3580371730 wait complete. 2021-11-21T22:01:20.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 Processor -- start 2021-11-21T22:01:20.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- start start 2021-11-21T22:01:20.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a80abce0 0x7f90a8063130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a8144810 0x7f90a81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90a80b2f30 0x7f90a8153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8062a60 con 0x7f90a80abce0 2021-11-21T22:01:20.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8063620 con 0x7f90a80b2f30 2021-11-21T22:01:20.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a814ae10 con 0x7f90a8144810 2021-11-21T22:01:20.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90ae485700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a80abce0 0x7f90a8063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90aec86700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90a80b2f30 0x7f90a8153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a8144810 0x7f90a81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90aec86700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90a80b2f30 0x7f90a8153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:51732/0 (socket says 172.21.2.116:51732) 2021-11-21T22:01:20.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90ae485700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a80abce0 0x7f90a8063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50560/0 (socket says 172.21.2.116:50560) 2021-11-21T22:01:20.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a8144810 0x7f90a81564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:53906/0 (socket says 172.21.2.116:53906) 2021-11-21T22:01:20.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90aec86700 1 -- 172.21.2.116:0/2915901725 learned_addr learned my addr 172.21.2.116:0/2915901725 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:20.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 -- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90a80b2f30 msgr2=0x7f90a8153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90a80b2f30 0x7f90a8153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 -- 172.21.2.116:0/2915901725 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a80abce0 msgr2=0x7f90a8063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:20.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a80abce0 0x7f90a8063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:20.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 -- 172.21.2.116:0/2915901725 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a400b180 con 0x7f90a8144810 2021-11-21T22:01:20.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90aec86700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90a80b2f30 0x7f90a8153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:01:20.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90adc84700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a8144810 0x7f90a81564b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f90a80637b0 tx=0x7f9098008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90977fe700 1 -- 172.21.2.116:0/2915901725 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9098011590 con 0x7f90a8144810 2021-11-21T22:01:20.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90977fe700 1 -- 172.21.2.116:0/2915901725 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90980116f0 con 0x7f90a8144810 2021-11-21T22:01:20.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a8062c90 con 0x7f90a8144810 2021-11-21T22:01:20.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90977fe700 1 -- 172.21.2.116:0/2915901725 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9098011a50 con 0x7f90a8144810 2021-11-21T22:01:20.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a81569a0 con 0x7f90a8144810 2021-11-21T22:01:20.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a814a270 con 0x7f90a8144810 2021-11-21T22:01:20.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.974+0000 7f90977fe700 1 -- 172.21.2.116:0/2915901725 <== mon.2 v2:172.21.2.117:3301/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9098011bb0 con 0x7f90a8144810 2021-11-21T22:01:20.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.978+0000 7f90977fe700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9088053580 0x7f9088055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:20.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.978+0000 7f90977fe700 1 -- 172.21.2.116:0/2915901725 <== mon.2 v2:172.21.2.117:3301/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f909806b580 con 0x7f90a8144810 2021-11-21T22:01:20.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.978+0000 7f90ae485700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9088053580 0x7f9088055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:20.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.978+0000 7f90ae485700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9088053580 0x7f9088055a40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f90a401c000 tx=0x7f90a4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:20.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:20.978+0000 7f90977fe700 1 -- 172.21.2.116:0/2915901725 <== mon.2 v2:172.21.2.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f90980429b0 con 0x7f90a8144810 2021-11-21T22:01:21.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.062+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f90a8062e20 con 0x7f90a8144810 2021-11-21T22:01:21.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90977fe700 1 -- 172.21.2.116:0/2915901725 <== mon.2 v2:172.21.2.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v97) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9098045fd0 con 0x7f90a8144810 2021-11-21T22:01:21.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9088053580 msgr2=0x7f9088055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:21.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9088053580 0x7f9088055a40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f90a401c000 tx=0x7f90a4015000).stop 2021-11-21T22:01:21.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a8144810 msgr2=0x7f90a81564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:21.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a8144810 0x7f90a81564b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f90a80637b0 tx=0x7f9098008040).stop 2021-11-21T22:01:21.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 shutdown_connections 2021-11-21T22:01:21.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90a80b2f30 0x7f90a8153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:21.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f90a8144810 0x7f90a81564b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:21.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9088053580 0x7f9088055a40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:21.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 --2- 172.21.2.116:0/2915901725 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90a80abce0 0x7f90a8063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:21.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 >> 172.21.2.116:0/2915901725 conn(0x7f90a805bd20 msgr2=0x7f90a813f590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:21.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 shutdown_connections 2021-11-21T22:01:21.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:21.982+0000 7f90b06e9700 1 -- 172.21.2.116:0/2915901725 wait complete. 2021-11-21T22:01:21.990 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:01:21.992 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:01:22.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 Processor -- start 2021-11-21T22:01:22.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- start start 2021-11-21T22:01:22.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 0x7fc5441478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:22.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc5440abce0 0x7fc5440b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:22.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc5440b1db0 0x7fc5440ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc544147e80 con 0x7fc5440b1db0 2021-11-21T22:01:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440af5a0 con 0x7fc5440abce0 2021-11-21T22:01:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440b1490 con 0x7fc544144810 2021-11-21T22:01:22.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc5440b1db0 0x7fc5440ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:22.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc5440b1db0 0x7fc5440ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50568/0 (socket says 172.21.2.116:50568) 2021-11-21T22:01:22.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 -- 172.21.2.116:0/3380324099 learned_addr learned my addr 172.21.2.116:0/3380324099 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:22.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 -- 172.21.2.116:0/3380324099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 msgr2=0x7fc5441478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:22.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 --2- 172.21.2.116:0/3380324099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 0x7fc5441478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:22.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 -- 172.21.2.116:0/3380324099 >> v2:172.21.2.117:3300/0 conn(0x7fc5440abce0 msgr2=0x7fc5440b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:22.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 --2- 172.21.2.116:0/3380324099 >> v2:172.21.2.117:3300/0 conn(0x7fc5440abce0 0x7fc5440b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:22.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 -- 172.21.2.116:0/3380324099 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440af060 con 0x7fc5440b1db0 2021-11-21T22:01:22.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc549fd3700 1 --2- 172.21.2.116:0/3380324099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b1db0 0x7fc5440ac600 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fc53c0032f0 tx=0x7fc53c009290).ready entity=mon.0 client_cookie=2400775aa53677e3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:22.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc537fff700 1 -- 172.21.2.116:0/3380324099 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc53c00b870 con 0x7fc5440b1db0 2021-11-21T22:01:22.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc537fff700 1 -- 172.21.2.116:0/3380324099 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c00b9d0 con 0x7fc5440b1db0 2021-11-21T22:01:22.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc537fff700 1 -- 172.21.2.116:0/3380324099 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc53c00bb80 con 0x7fc5440b1db0 2021-11-21T22:01:22.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- 172.21.2.116:0/3380324099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b1db0 msgr2=0x7fc5440ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:22.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 --2- 172.21.2.116:0/3380324099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b1db0 0x7fc5440ac600 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fc53c0032f0 tx=0x7fc53c009290).stop 2021-11-21T22:01:22.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- 172.21.2.116:0/3380324099 shutdown_connections 2021-11-21T22:01:22.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 --2- 172.21.2.116:0/3380324099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b1db0 0x7fc5440ac600 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:22.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 --2- 172.21.2.116:0/3380324099 >> v2:172.21.2.117:3300/0 conn(0x7fc5440abce0 0x7fc5440b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:22.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 --2- 172.21.2.116:0/3380324099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 0x7fc5441478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:22.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- 172.21.2.116:0/3380324099 >> 172.21.2.116:0/3380324099 conn(0x7fc54405bd20 msgr2=0x7fc54405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:22.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- 172.21.2.116:0/3380324099 shutdown_connections 2021-11-21T22:01:22.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- 172.21.2.116:0/3380324099 wait complete. 2021-11-21T22:01:22.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 Processor -- start 2021-11-21T22:01:22.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.042+0000 7fc54ba36700 1 -- start start 2021-11-21T22:01:22.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5440abce0 0x7fc54413c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:22.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 0x7fc5441565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:22.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b3760 0x7fc544147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:22.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544145850 con 0x7fc5440b3760 2021-11-21T22:01:22.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440b2ff0 con 0x7fc5440abce0 2021-11-21T22:01:22.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440b3290 con 0x7fc544144810 2021-11-21T22:01:22.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b3760 0x7fc544147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:22.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b3760 0x7fc544147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50578/0 (socket says 172.21.2.116:50578) 2021-11-21T22:01:22.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 -- 172.21.2.116:0/1319522546 learned_addr learned my addr 172.21.2.116:0/1319522546 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:22.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 -- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 msgr2=0x7fc5441565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:22.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 0x7fc5441565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:22.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 -- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5440abce0 msgr2=0x7fc54413c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:22.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5440abce0 0x7fc54413c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:22.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 -- 172.21.2.116:0/1319522546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c009530 con 0x7fc5440b3760 2021-11-21T22:01:22.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc549fd3700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b3760 0x7fc544147540 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fc53c0032f0 tx=0x7fc53c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:22.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc5367fc700 1 -- 172.21.2.116:0/1319522546 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc53c00b870 con 0x7fc5440b3760 2021-11-21T22:01:22.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc5367fc700 1 -- 172.21.2.116:0/1319522546 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c020430 con 0x7fc5440b3760 2021-11-21T22:01:22.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc544146e00 con 0x7fc5440b3760 2021-11-21T22:01:22.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc5367fc700 1 -- 172.21.2.116:0/1319522546 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc53c020790 con 0x7fc5440b3760 2021-11-21T22:01:22.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc544156ad0 con 0x7fc5440b3760 2021-11-21T22:01:22.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5440af300 con 0x7fc5440b3760 2021-11-21T22:01:22.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc5367fc700 1 -- 172.21.2.116:0/1319522546 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc53c028070 con 0x7fc5440b3760 2021-11-21T22:01:22.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc5367fc700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc524053640 0x7fc524055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:22.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc5367fc700 1 -- 172.21.2.116:0/1319522546 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fc53c06c5a0 con 0x7fc5440b3760 2021-11-21T22:01:22.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.046+0000 7fc5497d2700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc524053640 0x7fc524055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:22.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.050+0000 7fc5497d2700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc524053640 0x7fc524055b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fc538002ef0 tx=0x7fc538006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:22.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.050+0000 7fc5367fc700 1 -- 172.21.2.116:0/1319522546 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc53c0438d0 con 0x7fc5440b3760 2021-11-21T22:01:22.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.130+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fc5440573d0 con 0x7fc5440b3760 2021-11-21T22:01:22.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.994+0000 7fc5367fc700 1 -- 172.21.2.116:0/1319522546 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v98) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc53c046ef0 con 0x7fc5440b3760 2021-11-21T22:01:23.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc524053640 msgr2=0x7fc524055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc524053640 0x7fc524055b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fc538002ef0 tx=0x7fc538006910).stop 2021-11-21T22:01:23.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b3760 msgr2=0x7fc544147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b3760 0x7fc544147540 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fc53c0032f0 tx=0x7fc53c016000).stop 2021-11-21T22:01:23.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 shutdown_connections 2021-11-21T22:01:23.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc524053640 0x7fc524055b00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc5440b3760 0x7fc544147540 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc544144810 0x7fc5441565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 --2- 172.21.2.116:0/1319522546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc5440abce0 0x7fc54413c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 >> 172.21.2.116:0/1319522546 conn(0x7fc54405bd20 msgr2=0x7fc54413ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 shutdown_connections 2021-11-21T22:01:23.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:22.998+0000 7fc54ba36700 1 -- 172.21.2.116:0/1319522546 wait complete. 2021-11-21T22:01:23.007 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:01:23.009 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:01:23.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 Processor -- start 2021-11-21T22:01:23.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- start start 2021-11-21T22:01:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa08414fbe0 0x7fa084151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0840b1940 con 0x7fa0840a8670 2021-11-21T22:01:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0840b15a0 con 0x7fa0840b11c0 2021-11-21T22:01:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0840b1b70 con 0x7fa08414fbe0 2021-11-21T22:01:23.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa0840a8670 0x7fa084147940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34310/0 (socket says 172.21.2.116:34310) 2021-11-21T22:01:23.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 -- 172.21.2.116:0/3450687747 learned_addr learned my addr 172.21.2.116:0/3450687747 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:23.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396018349 0 0) 0x7fa0840b1940 con 0x7fa0840a8670 2021-11-21T22:01:23.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa064003260 con 0x7fa0840a8670 2021-11-21T22:01:23.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 864392953 0 0) 0x7fa064003260 con 0x7fa0840a8670 2021-11-21T22:01:23.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 >> v2:172.21.2.117:3300/0 conn(0x7fa08414fbe0 msgr2=0x7fa084151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:23.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 --2- 172.21.2.116:0/3450687747 >> v2:172.21.2.117:3300/0 conn(0x7fa08414fbe0 0x7fa084151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 >> v1:172.21.2.117:6789/0 conn(0x7fa0840b11c0 legacy=0x7fa084147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840b1e10 con 0x7fa0840a8670 2021-11-21T22:01:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fa0740019e0 con 0x7fa0840a8670 2021-11-21T22:01:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 >> v1:172.21.2.116:6789/0 conn(0x7fa0840a8670 legacy=0x7fa084147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa064003820 0x7fa064003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa064004500 0x7fa064004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa064005050 0x7fa06400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0840b1b70 con 0x7fa064004500 2021-11-21T22:01:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06400bfe0 con 0x7fa064003820 2021-11-21T22:01:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06400c1a0 con 0x7fa064005050 2021-11-21T22:01:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa064004500 0x7fa064004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 -- 172.21.2.116:0/3450687747 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa064005050 msgr2=0x7fa06400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa064005050 0x7fa06400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 -- 172.21.2.116:0/3450687747 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa064003820 msgr2=0x7fa064003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa064003820 0x7fa064003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 -- 172.21.2.116:0/3450687747 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064003260 con 0x7fa064004500 2021-11-21T22:01:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa083fff700 1 -- 172.21.2.116:0/3450687747 reap_dead start 2021-11-21T22:01:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa082ffd700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa064004500 0x7fa064004980 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fa084061da0 tx=0x7fa0840b1fa0).ready entity=mon.0 client_cookie=3d61f226dd3ab689 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa07400cc60 con 0x7fa064004500 2021-11-21T22:01:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07400cdc0 con 0x7fa064004500 2021-11-21T22:01:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa0827fc700 1 -- 172.21.2.116:0/3450687747 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa074005b80 con 0x7fa064004500 2021-11-21T22:01:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- 172.21.2.116:0/3450687747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa064004500 msgr2=0x7fa064004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa064004500 0x7fa064004980 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fa084061da0 tx=0x7fa0840b1fa0).stop 2021-11-21T22:01:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- 172.21.2.116:0/3450687747 shutdown_connections 2021-11-21T22:01:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 --2- 172.21.2.116:0/3450687747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa064004500 0x7fa064004980 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- 172.21.2.116:0/3450687747 >> 172.21.2.116:0/3450687747 conn(0x7fa08405bd20 msgr2=0x7fa08405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- 172.21.2.116:0/3450687747 shutdown_connections 2021-11-21T22:01:23.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- 172.21.2.116:0/3450687747 wait complete. 2021-11-21T22:01:23.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 Processor -- start 2021-11-21T22:01:23.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.062+0000 7fa089f00700 1 -- start start 2021-11-21T22:01:23.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa064004500 0x7fa0840b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0840b34b0 0x7fa084144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa084144a00 0x7fa084159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0841435d0 con 0x7fa0840b34b0 2021-11-21T22:01:23.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa084143870 con 0x7fa084144a00 2021-11-21T22:01:23.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa084143be0 con 0x7fa064004500 2021-11-21T22:01:23.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0840b34b0 0x7fa084144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0840b34b0 0x7fa084144510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50594/0 (socket says 172.21.2.116:50594) 2021-11-21T22:01:23.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 -- 172.21.2.116:0/3959889016 learned_addr learned my addr 172.21.2.116:0/3959889016 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:23.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 -- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa064004500 msgr2=0x7fa0840b2b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:23.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa064004500 0x7fa0840b2b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 -- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa084144a00 msgr2=0x7fa084159780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:23.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa084144a00 0x7fa084159780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 -- 172.21.2.116:0/3959889016 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa084146e80 con 0x7fa0840b34b0 2021-11-21T22:01:23.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa082ffd700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0840b34b0 0x7fa084144510 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7fa084061da0 tx=0x7fa074015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa080ff9700 1 -- 172.21.2.116:0/3959889016 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa074017070 con 0x7fa0840b34b0 2021-11-21T22:01:23.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa080ff9700 1 -- 172.21.2.116:0/3959889016 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa074004ab0 con 0x7fa0840b34b0 2021-11-21T22:01:23.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa080ff9700 1 -- 172.21.2.116:0/3959889016 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa074001ea0 con 0x7fa0840b34b0 2021-11-21T22:01:23.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0840b55f0 con 0x7fa0840b34b0 2021-11-21T22:01:23.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0840b31a0 con 0x7fa0840b34b0 2021-11-21T22:01:23.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0840b20b0 con 0x7fa0840b34b0 2021-11-21T22:01:23.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa080ff9700 1 -- 172.21.2.116:0/3959889016 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa074010070 con 0x7fa0840b34b0 2021-11-21T22:01:23.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa080ff9700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa068053660 0x7fa068055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa080ff9700 1 -- 172.21.2.116:0/3959889016 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fa07406cce0 con 0x7fa0840b34b0 2021-11-21T22:01:23.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa0837fe700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa068053660 0x7fa068055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.066+0000 7fa0837fe700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa068053660 0x7fa068055b20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fa07c000f70 tx=0x7fa07c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.070+0000 7fa080ff9700 1 -- 172.21.2.116:0/3959889016 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa074043da0 con 0x7fa0840b34b0 2021-11-21T22:01:23.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.178+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa0840573d0 con 0x7fa0840b34b0 2021-11-21T22:01:23.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.182+0000 7fa080ff9700 1 -- 172.21.2.116:0/3959889016 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v43) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa07402c0d0 con 0x7fa0840b34b0 2021-11-21T22:01:23.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa068053660 msgr2=0x7fa068055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa068053660 0x7fa068055b20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fa07c000f70 tx=0x7fa07c006930).stop 2021-11-21T22:01:23.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0840b34b0 msgr2=0x7fa084144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0840b34b0 0x7fa084144510 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7fa084061da0 tx=0x7fa074015000).stop 2021-11-21T22:01:23.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 shutdown_connections 2021-11-21T22:01:23.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa084144a00 0x7fa084159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa068053660 0x7fa068055b20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0840b34b0 0x7fa084144510 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 --2- 172.21.2.116:0/3959889016 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa064004500 0x7fa0840b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 >> 172.21.2.116:0/3959889016 conn(0x7fa08405bd20 msgr2=0x7fa084148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 shutdown_connections 2021-11-21T22:01:23.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.186+0000 7fa089f00700 1 -- 172.21.2.116:0/3959889016 wait complete. 2021-11-21T22:01:23.192 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 17 and data pool 18 2021-11-21T22:01:23.196 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:23.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a937d700 1 Processor -- start 2021-11-21T22:01:23.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a937d700 1 -- start start 2021-11-21T22:01:23.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a937d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4141530 0x7f00a4060b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a937d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f00a40b1a50 0x7f00a4067200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a937d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a4144410 con 0x7f00a4060740 2021-11-21T22:01:23.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a937d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a41446b0 con 0x7f00a40b1a50 2021-11-21T22:01:23.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a937d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4144950 con 0x7f00a4141530 2021-11-21T22:01:23.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a27fc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4141530 0x7f00a4060b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a2ffd700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f00a4060740 0x7f00a4140e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:33244/0 (socket says 172.21.2.116:33244) 2021-11-21T22:01:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a2ffd700 1 -- 172.21.2.116:0/2618597669 learned_addr learned my addr 172.21.2.116:0/2618597669 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.242+0000 7f00a37fe700 1 --2- 172.21.2.116:0/2618597669 >> v2:172.21.2.117:3300/0 conn(0x7f00a40b1a50 0x7f00a4067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a1ffb700 1 -- 172.21.2.116:0/2618597669 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144919244 0 0) 0x7f00a4144410 con 0x7f00a4060740 2021-11-21T22:01:23.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a1ffb700 1 -- 172.21.2.116:0/2618597669 --> v1:172.21.2.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0084003260 con 0x7f00a4060740 2021-11-21T22:01:23.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a37fe700 1 -- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4141530 msgr2=0x7f00a4060b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:23.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a37fe700 1 --2- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4141530 0x7f00a4060b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a37fe700 1 -- 172.21.2.116:0/2618597669 >> v1:172.21.2.117:6789/0 conn(0x7f00a4060740 legacy=0x7f00a4140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a37fe700 1 -- 172.21.2.116:0/2618597669 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a4144bf0 con 0x7f00a40b1a50 2021-11-21T22:01:23.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 --2- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4141530 0x7f00a4060b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:01:23.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a37fe700 1 --2- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a40b1a50 0x7f00a4067200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f00a405d460 tx=0x7f009c0091d0).ready entity=mon.1 client_cookie=b11ff7648ed8b36d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a1ffb700 1 -- 172.21.2.116:0/2618597669 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f009c00b600 con 0x7f00a40b1a50 2021-11-21T22:01:23.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a1ffb700 1 -- 172.21.2.116:0/2618597669 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c00b760 con 0x7f00a40b1a50 2021-11-21T22:01:23.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a1ffb700 1 -- 172.21.2.116:0/2618597669 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f009c00b8d0 con 0x7f00a40b1a50 2021-11-21T22:01:23.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a40b1a50 msgr2=0x7f00a4067200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 --2- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a40b1a50 0x7f00a4067200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f00a405d460 tx=0x7f009c0091d0).stop 2021-11-21T22:01:23.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/2618597669 shutdown_connections 2021-11-21T22:01:23.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 --2- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a40b1a50 0x7f00a4067200 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 --2- 172.21.2.116:0/2618597669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4141530 0x7f00a4060b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/2618597669 >> 172.21.2.116:0/2618597669 conn(0x7f00a40b7b80 msgr2=0x7f00a40b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/2618597669 shutdown_connections 2021-11-21T22:01:23.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/2618597669 wait complete. 2021-11-21T22:01:23.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 Processor -- start 2021-11-21T22:01:23.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- start start 2021-11-21T22:01:23.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4060740 0x7f00a40ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f00a40b1a50 0x7f00a409b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a4141530 0x7f00a409bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a409c120 con 0x7f00a40b1a50 2021-11-21T22:01:23.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a409a800 con 0x7f00a4141530 2021-11-21T22:01:23.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4141080 con 0x7f00a4060740 2021-11-21T22:01:23.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f00a40b1a50 0x7f00a409b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f00a40b1a50 0x7f00a409b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50620/0 (socket says 172.21.2.116:50620) 2021-11-21T22:01:23.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 -- 172.21.2.116:0/1459007996 learned_addr learned my addr 172.21.2.116:0/1459007996 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:23.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a2ffd700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4060740 0x7f00a40ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 -- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4060740 msgr2=0x7f00a40ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4060740 0x7f00a40ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 -- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a4141530 msgr2=0x7f00a409bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a4141530 0x7f00a409bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 -- 172.21.2.116:0/1459007996 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009400b180 con 0x7f00a40b1a50 2021-11-21T22:01:23.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a27fc700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f00a40b1a50 0x7f00a409b740 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f009400c000 tx=0x7f0094006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f009bfff700 1 -- 172.21.2.116:0/1459007996 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f009400c860 con 0x7f00a40b1a50 2021-11-21T22:01:23.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f009bfff700 1 -- 172.21.2.116:0/1459007996 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400c9c0 con 0x7f00a40b1a50 2021-11-21T22:01:23.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c009470 con 0x7f00a40b1a50 2021-11-21T22:01:23.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a40b0b00 con 0x7f00a40b1a50 2021-11-21T22:01:23.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f009bfff700 1 -- 172.21.2.116:0/1459007996 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f009400cd20 con 0x7f00a40b1a50 2021-11-21T22:01:23.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.246+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a4140ee0 con 0x7f00a40b1a50 2021-11-21T22:01:23.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.250+0000 7f009bfff700 1 -- 172.21.2.116:0/1459007996 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f009401c460 con 0x7f00a40b1a50 2021-11-21T22:01:23.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.250+0000 7f009bfff700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0088053610 0x7f0088055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.250+0000 7f009bfff700 1 -- 172.21.2.116:0/1459007996 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f009406c0e0 con 0x7f00a40b1a50 2021-11-21T22:01:23.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.250+0000 7f00a2ffd700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0088053610 0x7f0088055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.250+0000 7f00a2ffd700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0088053610 0x7f0088055ad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f008c008000 tx=0x7f008c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.250+0000 7f009bfff700 1 -- 172.21.2.116:0/1459007996 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00a4140ee0 con 0x7f00a40b1a50 2021-11-21T22:01:23.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.334+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00a4140ee0 con 0x7f00a40b1a50 2021-11-21T22:01:23.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f009bfff700 1 -- 172.21.2.116:0/1459007996 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f00a4140ee0 con 0x7f00a40b1a50 2021-11-21T22:01:23.342 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:23.342 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":99,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:23.183092+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:21.079308+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:22.145654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:23.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0088053610 msgr2=0x7f0088055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0088053610 0x7f0088055ad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f008c008000 tx=0x7f008c009040).stop 2021-11-21T22:01:23.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f00a40b1a50 msgr2=0x7f00a409b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f00a40b1a50 0x7f00a409b740 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f009400c000 tx=0x7f0094006df0).stop 2021-11-21T22:01:23.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 shutdown_connections 2021-11-21T22:01:23.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f00a4141530 0x7f00a409bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0088053610 0x7f0088055ad0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f00a40b1a50 0x7f00a409b740 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 --2- 172.21.2.116:0/1459007996 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f00a4060740 0x7f00a40ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 >> 172.21.2.116:0/1459007996 conn(0x7f00a40b7b80 msgr2=0x7f00a40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 shutdown_connections 2021-11-21T22:01:23.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.338+0000 7f00a937d700 1 -- 172.21.2.116:0/1459007996 wait complete. 2021-11-21T22:01:23.348 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:23.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3fc98700 1 Processor -- start 2021-11-21T22:01:23.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3fc98700 1 -- start start 2021-11-21T22:01:23.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3fc98700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc38061760 0x7fdc38148470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3fc98700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc380b1ad0 0x7fdc38148960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3fc98700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38062f00 con 0x7fdc380b7210 2021-11-21T22:01:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3fc98700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc38148e50 con 0x7fdc380b1ad0 2021-11-21T22:01:23.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3fc98700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38144c20 con 0x7fdc38061760 2021-11-21T22:01:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3e235700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc380b1ad0 0x7fdc38148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3e235700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc380b1ad0 0x7fdc38148960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50628/0 (socket says 172.21.2.116:50628) 2021-11-21T22:01:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.394+0000 7fdc3e235700 1 -- 172.21.2.116:0/359085151 learned_addr learned my addr 172.21.2.116:0/359085151 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:23.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 -- 172.21.2.116:0/359085151 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc38061760 msgr2=0x7fdc38148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 --2- 172.21.2.116:0/359085151 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc38061760 0x7fdc38148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 -- 172.21.2.116:0/359085151 >> v1:172.21.2.117:6789/0 conn(0x7fdc380b7210 legacy=0x7fdc381470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 -- 172.21.2.116:0/359085151 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38147ba0 con 0x7fdc380b1ad0 2021-11-21T22:01:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 --2- 172.21.2.116:0/359085151 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b1ad0 0x7fdc38148960 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fdc380aae10 tx=0x7fdc30009290).ready entity=mon.0 client_cookie=db36ea9cf7f199bb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3ca32700 1 -- 172.21.2.116:0/359085151 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc3000b860 con 0x7fdc380b1ad0 2021-11-21T22:01:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3ca32700 1 -- 172.21.2.116:0/359085151 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3000b9c0 con 0x7fdc380b1ad0 2021-11-21T22:01:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3ca32700 1 -- 172.21.2.116:0/359085151 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc3000bb70 con 0x7fdc380b1ad0 2021-11-21T22:01:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/359085151 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b1ad0 msgr2=0x7fdc38148960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/359085151 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b1ad0 0x7fdc38148960 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fdc380aae10 tx=0x7fdc30009290).stop 2021-11-21T22:01:23.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/359085151 shutdown_connections 2021-11-21T22:01:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/359085151 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b1ad0 0x7fdc38148960 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/359085151 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc38061760 0x7fdc38148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/359085151 >> 172.21.2.116:0/359085151 conn(0x7fdc38064d60 msgr2=0x7fdc38065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/359085151 shutdown_connections 2021-11-21T22:01:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/359085151 wait complete. 2021-11-21T22:01:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 Processor -- start 2021-11-21T22:01:23.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- start start 2021-11-21T22:01:23.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc38061760 0x7fdc38144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc380b1ad0 0x7fdc381401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b7210 0x7fdc381406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38141710 con 0x7fdc380b7210 2021-11-21T22:01:23.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380ace70 con 0x7fdc38061760 2021-11-21T22:01:23.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38142ca0 con 0x7fdc380b1ad0 2021-11-21T22:01:23.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b7210 0x7fdc381406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b7210 0x7fdc381406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50634/0 (socket says 172.21.2.116:50634) 2021-11-21T22:01:23.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 -- 172.21.2.116:0/942274055 learned_addr learned my addr 172.21.2.116:0/942274055 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:23.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 -- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc380b1ad0 msgr2=0x7fdc381401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc380b1ad0 0x7fdc381401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 -- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc38061760 msgr2=0x7fdc38144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc38061760 0x7fdc38144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 -- 172.21.2.116:0/942274055 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30009530 con 0x7fdc380b7210 2021-11-21T22:01:23.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3e235700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b7210 0x7fdc381406b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fdc381432c0 tx=0x7fdc30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc2affd700 1 -- 172.21.2.116:0/942274055 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc3000bb30 con 0x7fdc380b7210 2021-11-21T22:01:23.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc2affd700 1 -- 172.21.2.116:0/942274055 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3001f420 con 0x7fdc380b7210 2021-11-21T22:01:23.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc380ad0a0 con 0x7fdc380b7210 2021-11-21T22:01:23.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc2affd700 1 -- 172.21.2.116:0/942274055 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc3001f780 con 0x7fdc380b7210 2021-11-21T22:01:23.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc381429b0 con 0x7fdc380b7210 2021-11-21T22:01:23.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc380ad540 con 0x7fdc380b7210 2021-11-21T22:01:23.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc2affd700 1 -- 172.21.2.116:0/942274055 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdc3000f070 con 0x7fdc380b7210 2021-11-21T22:01:23.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.398+0000 7fdc2affd700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc18053660 0x7fdc18055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.402+0000 7fdc2affd700 1 -- 172.21.2.116:0/942274055 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdc3006c660 con 0x7fdc380b7210 2021-11-21T22:01:23.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.402+0000 7fdc3da34700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc18053660 0x7fdc18055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.402+0000 7fdc3da34700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc18053660 0x7fdc18055b20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fdc2c002f10 tx=0x7fdc2c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.402+0000 7fdc2affd700 1 -- 172.21.2.116:0/942274055 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc380ad540 con 0x7fdc380b7210 2021-11-21T22:01:23.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc380ad540 con 0x7fdc380b7210 2021-11-21T22:01:23.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc2affd700 1 -- 172.21.2.116:0/942274055 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16067 (secure 0 0 0) 0x7fdc30030ee0 con 0x7fdc380b7210 2021-11-21T22:01:23.490 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:23.490 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":99,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:23.183092+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:21.079308+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:22.145654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:23.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc18053660 msgr2=0x7fdc18055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc18053660 0x7fdc18055b20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fdc2c002f10 tx=0x7fdc2c006930).stop 2021-11-21T22:01:23.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b7210 msgr2=0x7fdc381406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b7210 0x7fdc381406b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fdc381432c0 tx=0x7fdc30016000).stop 2021-11-21T22:01:23.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 shutdown_connections 2021-11-21T22:01:23.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc18053660 0x7fdc18055b20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc380b7210 0x7fdc381406b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc380b1ad0 0x7fdc381401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 --2- 172.21.2.116:0/942274055 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc38061760 0x7fdc38144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 >> 172.21.2.116:0/942274055 conn(0x7fdc38064d60 msgr2=0x7fdc381470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 shutdown_connections 2021-11-21T22:01:23.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.486+0000 7fdc3fc98700 1 -- 172.21.2.116:0/942274055 wait complete. 2021-11-21T22:01:23.497 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:01:23.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 Processor -- start 2021-11-21T22:01:23.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- start start 2021-11-21T22:01:23.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f4bd8144810 0x7f4bd81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4bd80abce0 0x7f4bd80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd8147e80 con 0x7f4bd80b1db0 2021-11-21T22:01:23.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd80b1490 con 0x7f4bd80abce0 2021-11-21T22:01:23.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd80af060 con 0x7f4bd8144810 2021-11-21T22:01:23.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4bd80abce0 0x7f4bd80b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4bd80abce0 0x7f4bd80b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50640/0 (socket says 172.21.2.116:50640) 2021-11-21T22:01:23.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 -- 172.21.2.116:0/2774772744 learned_addr learned my addr 172.21.2.116:0/2774772744 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:23.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 -- 172.21.2.116:0/2774772744 >> v2:172.21.2.117:3300/0 conn(0x7f4bd8144810 msgr2=0x7f4bd81478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:23.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 --2- 172.21.2.116:0/2774772744 >> v2:172.21.2.117:3300/0 conn(0x7f4bd8144810 0x7f4bd81478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 -- 172.21.2.116:0/2774772744 >> v1:172.21.2.116:6789/0 conn(0x7f4bd80b1db0 legacy=0x7f4bd80ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 -- 172.21.2.116:0/2774772744 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80af5a0 con 0x7f4bd80abce0 2021-11-21T22:01:23.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdd582700 1 --2- 172.21.2.116:0/2774772744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd80abce0 0x7f4bd80b18c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f4bc8002f70 tx=0x7f4bc8006eb0).ready entity=mon.0 client_cookie=aa92570eb0dc51d8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdcd81700 1 -- 172.21.2.116:0/2774772744 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc800d610 con 0x7f4bd80abce0 2021-11-21T22:01:23.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdcd81700 1 -- 172.21.2.116:0/2774772744 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8000e90 con 0x7f4bd80abce0 2021-11-21T22:01:23.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdcd81700 1 -- 172.21.2.116:0/2774772744 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc800d9a0 con 0x7f4bd80abce0 2021-11-21T22:01:23.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- 172.21.2.116:0/2774772744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd80abce0 msgr2=0x7f4bd80b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/2774772744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd80abce0 0x7f4bd80b18c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f4bc8002f70 tx=0x7f4bc8006eb0).stop 2021-11-21T22:01:23.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- 172.21.2.116:0/2774772744 shutdown_connections 2021-11-21T22:01:23.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/2774772744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd80abce0 0x7f4bd80b18c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:23.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/2774772744 >> v2:172.21.2.117:3300/0 conn(0x7f4bd8144810 0x7f4bd81478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- 172.21.2.116:0/2774772744 >> 172.21.2.116:0/2774772744 conn(0x7f4bd805bd20 msgr2=0x7f4bd805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:23.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- 172.21.2.116:0/2774772744 shutdown_connections 2021-11-21T22:01:23.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 -- 172.21.2.116:0/2774772744 wait complete. 2021-11-21T22:01:23.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.546+0000 7f4bdffe7700 1 Processor -- start 2021-11-21T22:01:23.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 -- start start 2021-11-21T22:01:23.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bd80abce0 0x7f4bd8143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd8144810 0x7f4bd8146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bd8145ed0 0x7f4bd8146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8144ef0 con 0x7f4bd8144810 2021-11-21T22:01:23.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd81465d0 con 0x7f4bd8145ed0 2021-11-21T22:01:23.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8145120 con 0x7f4bd80abce0 2021-11-21T22:01:23.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd8144810 0x7f4bd8146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd8144810 0x7f4bd8146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50644/0 (socket says 172.21.2.116:50644) 2021-11-21T22:01:23.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 -- 172.21.2.116:0/4110362344 learned_addr learned my addr 172.21.2.116:0/4110362344 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:23.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 -- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bd80abce0 msgr2=0x7f4bd8143a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bd80abce0 0x7f4bd8143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 -- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bd8145ed0 msgr2=0x7f4bd8146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:23.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bd8145ed0 0x7f4bd8146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:23.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 -- 172.21.2.116:0/4110362344 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc800b1e0 con 0x7f4bd8144810 2021-11-21T22:01:23.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdd582700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd8144810 0x7f4bd8146a00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f4bc8002f70 tx=0x7f4bc8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bc6ffd700 1 -- 172.21.2.116:0/4110362344 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc8011a00 con 0x7f4bd8144810 2021-11-21T22:01:23.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bc6ffd700 1 -- 172.21.2.116:0/4110362344 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8011b60 con 0x7f4bd8144810 2021-11-21T22:01:23.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bc6ffd700 1 -- 172.21.2.116:0/4110362344 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc801e460 con 0x7f4bd8144810 2021-11-21T22:01:23.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd81453c0 con 0x7f4bd8144810 2021-11-21T22:01:23.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd80a81a0 con 0x7f4bd8144810 2021-11-21T22:01:23.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd8146740 con 0x7f4bd8144810 2021-11-21T22:01:23.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bc6ffd700 1 -- 172.21.2.116:0/4110362344 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4bc8025030 con 0x7f4bd8144810 2021-11-21T22:01:23.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bc6ffd700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bb80534f0 0x7f4bb80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:23.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bc6ffd700 1 -- 172.21.2.116:0/4110362344 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4bc806ce40 con 0x7f4bd8144810 2021-11-21T22:01:23.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.550+0000 7f4bddd83700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bb80534f0 0x7f4bb80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:23.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.554+0000 7f4bddd83700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bb80534f0 0x7f4bb80559b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f4bd4002ef0 tx=0x7f4bd4006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:23.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.554+0000 7f4bc6ffd700 1 -- 172.21.2.116:0/4110362344 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4bd8146740 con 0x7f4bd8144810 2021-11-21T22:01:23.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:23.662+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4bd8146740 con 0x7f4bd8144810 2021-11-21T22:01:24.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.194+0000 7f4bc6ffd700 1 -- 172.21.2.116:0/4110362344 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4bd8146740 con 0x7f4bd8144810 2021-11-21T22:01:24.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bb80534f0 msgr2=0x7f4bb80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bb80534f0 0x7f4bb80559b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f4bd4002ef0 tx=0x7f4bd4006910).stop 2021-11-21T22:01:24.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd8144810 msgr2=0x7f4bd8146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd8144810 0x7f4bd8146a00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f4bc8002f70 tx=0x7f4bc8015000).stop 2021-11-21T22:01:24.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 shutdown_connections 2021-11-21T22:01:24.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bd8145ed0 0x7f4bd8146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bb80534f0 0x7f4bb80559b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bd8144810 0x7f4bd8146a00 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 --2- 172.21.2.116:0/4110362344 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bd80abce0 0x7f4bd8143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 >> 172.21.2.116:0/4110362344 conn(0x7f4bd805bd20 msgr2=0x7f4bd805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 shutdown_connections 2021-11-21T22:01:24.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.198+0000 7f4bdffe7700 1 -- 172.21.2.116:0/4110362344 wait complete. 2021-11-21T22:01:24.208 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:24.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.254+0000 7fabb91cc700 1 Processor -- start 2021-11-21T22:01:24.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.254+0000 7fabb91cc700 1 -- start start 2021-11-21T22:01:24.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.254+0000 7fabb91cc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fabb4062410 0x7fabb4144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.254+0000 7fabb91cc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb40a6e80 con 0x7fabb40a86b0 2021-11-21T22:01:24.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.254+0000 7fabb91cc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb414a7b0 con 0x7fabb40a6aa0 2021-11-21T22:01:24.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.254+0000 7fabb91cc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb40a8bc0 con 0x7fabb4062410 2021-11-21T22:01:24.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fabb40a86b0 0x7fabb41443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34378/0 (socket says 172.21.2.116:34378) 2021-11-21T22:01:24.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 -- 172.21.2.116:0/1644214341 learned_addr learned my addr 172.21.2.116:0/1644214341 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:24.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021664740 0 0) 0x7fabb40a6e80 con 0x7fabb40a86b0 2021-11-21T22:01:24.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab94003260 con 0x7fabb40a86b0 2021-11-21T22:01:24.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb359e700 1 --2- 172.21.2.116:0/1644214341 >> v2:172.21.2.117:3300/0 conn(0x7fabb4062410 0x7fabb4144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3947092449 0 0) 0x7fab94003260 con 0x7fabb40a86b0 2021-11-21T22:01:24.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 >> v2:172.21.2.117:3300/0 conn(0x7fabb4062410 msgr2=0x7fabb4144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 --2- 172.21.2.116:0/1644214341 >> v2:172.21.2.117:3300/0 conn(0x7fabb4062410 0x7fabb4144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 >> v1:172.21.2.117:6789/0 conn(0x7fabb40a6aa0 legacy=0x7fabb41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb414aa10 con 0x7fabb40a86b0 2021-11-21T22:01:24.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7faba4001680 con 0x7fabb40a86b0 2021-11-21T22:01:24.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 >> v1:172.21.2.116:6789/0 conn(0x7fabb40a86b0 legacy=0x7fabb41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003750 0x7fab94003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab94004500 0x7fab94004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94004ec0 0x7fab9400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40a8bc0 con 0x7fab94003750 2021-11-21T22:01:24.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9400bda0 con 0x7fab94004500 2021-11-21T22:01:24.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9400bf60 con 0x7fab94004ec0 2021-11-21T22:01:24.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003750 0x7fab94003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab94004500 0x7fab94004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb359e700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94004ec0 0x7fab9400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 -- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94004ec0 msgr2=0x7fab9400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94004ec0 0x7fab9400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 -- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab94004500 msgr2=0x7fab94004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fab94004500 0x7fab94004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 -- 172.21.2.116:0/1644214341 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab94003260 con 0x7fab94003750 2021-11-21T22:01:24.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb359e700 1 -- 172.21.2.116:0/1644214341 reap_dead start 2021-11-21T22:01:24.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003750 0x7fab94003b70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7fabb4061b50 tx=0x7fab9c007a00).ready entity=mon.0 client_cookie=90cab93d97666457 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c007d10 con 0x7fab94003750 2021-11-21T22:01:24.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c007030 con 0x7fab94003750 2021-11-21T22:01:24.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb1d9b700 1 -- 172.21.2.116:0/1644214341 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9c007390 con 0x7fab94003750 2021-11-21T22:01:24.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- 172.21.2.116:0/1644214341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003750 msgr2=0x7fab94003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003750 0x7fab94003b70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7fabb4061b50 tx=0x7fab9c007a00).stop 2021-11-21T22:01:24.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- 172.21.2.116:0/1644214341 shutdown_connections 2021-11-21T22:01:24.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1644214341 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fab94003750 0x7fab94003b70 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- 172.21.2.116:0/1644214341 >> 172.21.2.116:0/1644214341 conn(0x7fabb405bd20 msgr2=0x7fabb405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- 172.21.2.116:0/1644214341 shutdown_connections 2021-11-21T22:01:24.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- 172.21.2.116:0/1644214341 wait complete. 2021-11-21T22:01:24.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 Processor -- start 2021-11-21T22:01:24.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- start start 2021-11-21T22:01:24.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94003750 0x7fabb40b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb4146960 0x7fabb4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fabb4156a40 0x7fabb413f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40b3330 con 0x7fabb4146960 2021-11-21T22:01:24.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41560f0 con 0x7fabb4156a40 2021-11-21T22:01:24.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40b3ed0 con 0x7fab94003750 2021-11-21T22:01:24.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb4146960 0x7fabb4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb4146960 0x7fabb4156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50666/0 (socket says 172.21.2.116:50666) 2021-11-21T22:01:24.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 -- 172.21.2.116:0/1287979046 learned_addr learned my addr 172.21.2.116:0/1287979046 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:24.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94003750 0x7fabb40b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb359e700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fabb4156a40 0x7fabb413f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 -- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94003750 msgr2=0x7fabb40b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94003750 0x7fabb40b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 -- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fabb4156a40 msgr2=0x7fabb413f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fabb4156a40 0x7fabb413f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 -- 172.21.2.116:0/1287979046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb415a170 con 0x7fabb4146960 2021-11-21T22:01:24.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb359e700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fabb4156a40 0x7fabb413f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:24.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb259c700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb4146960 0x7fabb4156550 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7faba400b000 tx=0x7fabb40a8520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94003750 0x7fabb40b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:24.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fababfff700 1 -- 172.21.2.116:0/1287979046 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba4010710 con 0x7fabb4146960 2021-11-21T22:01:24.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fababfff700 1 -- 172.21.2.116:0/1287979046 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4010870 con 0x7fabb4146960 2021-11-21T22:01:24.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb415a410 con 0x7fabb4146960 2021-11-21T22:01:24.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fababfff700 1 -- 172.21.2.116:0/1287979046 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba4010bd0 con 0x7fabb4146960 2021-11-21T22:01:24.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.258+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb40b25f0 con 0x7fabb4146960 2021-11-21T22:01:24.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.262+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb415a5a0 con 0x7fabb4146960 2021-11-21T22:01:24.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.262+0000 7fababfff700 1 -- 172.21.2.116:0/1287979046 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7faba4010d30 con 0x7fabb4146960 2021-11-21T22:01:24.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.262+0000 7fababfff700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab980534f0 0x7fab980559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.262+0000 7fababfff700 1 -- 172.21.2.116:0/1287979046 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faba406bef0 con 0x7fabb4146960 2021-11-21T22:01:24.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.262+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab980534f0 0x7fab980559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.262+0000 7fabb2d9d700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab980534f0 0x7fab980559b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fab9c016000 tx=0x7fab9c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.262+0000 7fababfff700 1 -- 172.21.2.116:0/1287979046 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faba4042ff0 con 0x7fabb4146960 2021-11-21T22:01:24.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.370+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabb4156260 con 0x7fabb4146960 2021-11-21T22:01:24.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fababfff700 1 -- 172.21.2.116:0/1287979046 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+2493 (secure 0 0 0) 0x7faba4046610 con 0x7fabb4146960 2021-11-21T22:01:24.378 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:24.378 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:23.183200+0000","modified":"2021-11-21T22:01:24.187992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3417150056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3417150056},{"type":"v1","addr":"172.21.2.116:6835","nonce":3417150056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-11-21T22:01:24.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab980534f0 msgr2=0x7fab980559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab980534f0 0x7fab980559b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fab9c016000 tx=0x7fab9c015000).stop 2021-11-21T22:01:24.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb4146960 msgr2=0x7fabb4156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb4146960 0x7fabb4156550 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7faba400b000 tx=0x7fabb40a8520).stop 2021-11-21T22:01:24.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 shutdown_connections 2021-11-21T22:01:24.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fabb4156a40 0x7fabb413f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fab980534f0 0x7fab980559b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fabb4146960 0x7fabb4156550 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 --2- 172.21.2.116:0/1287979046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fab94003750 0x7fabb40b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 >> 172.21.2.116:0/1287979046 conn(0x7fabb405bd20 msgr2=0x7fabb41421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 shutdown_connections 2021-11-21T22:01:24.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.374+0000 7fabb91cc700 1 -- 172.21.2.116:0/1287979046 wait complete. 2021-11-21T22:01:24.382 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 44 2021-11-21T22:01:24.385 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:24.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.430+0000 7f3622587700 1 Processor -- start 2021-11-21T22:01:24.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.430+0000 7f3622587700 1 -- start start 2021-11-21T22:01:24.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 0x7f361c140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c05c440 con 0x7f361c0a9500 2021-11-21T22:01:24.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c143630 con 0x7f361c05c060 2021-11-21T22:01:24.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0a9c10 con 0x7f361c060c50 2021-11-21T22:01:24.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3620b24700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f361c0a9500 0x7f361c05fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34398/0 (socket says 172.21.2.116:34398) 2021-11-21T22:01:24.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3620b24700 1 -- 172.21.2.116:0/1483696233 learned_addr learned my addr 172.21.2.116:0/1483696233 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:24.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793415630 0 0) 0x7f361c05c440 con 0x7f361c0a9500 2021-11-21T22:01:24.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35fc003260 con 0x7f361c0a9500 2021-11-21T22:01:24.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 6791713 0 0) 0x7f35fc003260 con 0x7f361c0a9500 2021-11-21T22:01:24.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 msgr2=0x7f361c140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361b7fe700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 0x7f361c140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 0x7f361c140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 >> v1:172.21.2.117:6789/0 conn(0x7f361c05c060 legacy=0x7f361c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c1438a0 con 0x7f361c0a9500 2021-11-21T22:01:24.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f3614001660 con 0x7f361c0a9500 2021-11-21T22:01:24.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 >> v1:172.21.2.116:6789/0 conn(0x7f361c0a9500 legacy=0x7f361c05fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35fc003800 0x7f35fc003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc004500 0x7f35fc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 0x7f35fc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0a9c10 con 0x7f35fc003800 2021-11-21T22:01:24.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc005290 con 0x7f35fc004500 2021-11-21T22:01:24.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc00bc40 con 0x7f361c060c50 2021-11-21T22:01:24.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361bfff700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35fc003800 0x7f35fc003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361b7fe700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc004500 0x7f35fc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3620b24700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 0x7f35fc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361bfff700 1 -- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 msgr2=0x7f35fc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361bfff700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c060c50 0x7f35fc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361bfff700 1 -- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc004500 msgr2=0x7f35fc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361bfff700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc004500 0x7f35fc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361bfff700 1 -- 172.21.2.116:0/1483696233 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc003260 con 0x7f35fc003800 2021-11-21T22:01:24.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361bfff700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35fc003800 0x7f35fc003c00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f3604001fa0 tx=0x7f3604007a00).ready entity=mon.0 client_cookie=7e53e42df205491f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3604007070 con 0x7f35fc003800 2021-11-21T22:01:24.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36040071d0 con 0x7f35fc003800 2021-11-21T22:01:24.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f361affd700 1 -- 172.21.2.116:0/1483696233 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3604006510 con 0x7f35fc003800 2021-11-21T22:01:24.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- 172.21.2.116:0/1483696233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35fc003800 msgr2=0x7f35fc003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 --2- 172.21.2.116:0/1483696233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35fc003800 0x7f35fc003c00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f3604001fa0 tx=0x7f3604007a00).stop 2021-11-21T22:01:24.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3620b24700 1 -- 172.21.2.116:0/1483696233 reap_dead start 2021-11-21T22:01:24.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- 172.21.2.116:0/1483696233 shutdown_connections 2021-11-21T22:01:24.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- 172.21.2.116:0/1483696233 >> 172.21.2.116:0/1483696233 conn(0x7f361c0b6000 msgr2=0x7f361c0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- 172.21.2.116:0/1483696233 shutdown_connections 2021-11-21T22:01:24.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- 172.21.2.116:0/1483696233 wait complete. 2021-11-21T22:01:24.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 Processor -- start 2021-11-21T22:01:24.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.434+0000 7f3622587700 1 -- start start 2021-11-21T22:01:24.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc003800 0x7f361c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c144f10 0x7f361c145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f361c145880 0x7f361c13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c155da0 con 0x7f361c145880 2021-11-21T22:01:24.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c067bb0 con 0x7f35fc003800 2021-11-21T22:01:24.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c146ba0 con 0x7f361c144f10 2021-11-21T22:01:24.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f361c145880 0x7f361c13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f361c145880 0x7f361c13a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50686/0 (socket says 172.21.2.116:50686) 2021-11-21T22:01:24.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 -- 172.21.2.116:0/3477782923 learned_addr learned my addr 172.21.2.116:0/3477782923 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:24.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f361b7fe700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c144f10 0x7f361c145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f361bfff700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc003800 0x7f361c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 -- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c144f10 msgr2=0x7f361c145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c144f10 0x7f361c145390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 -- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc003800 msgr2=0x7f361c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc003800 0x7f361c155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 -- 172.21.2.116:0/3477782923 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c146e40 con 0x7f361c145880 2021-11-21T22:01:24.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f361bfff700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc003800 0x7f361c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:24.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3620b24700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f361c145880 0x7f361c13a160 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f3614000e90 tx=0x7f3614001d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f36197fa700 1 -- 172.21.2.116:0/3477782923 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3614010640 con 0x7f361c145880 2021-11-21T22:01:24.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f36197fa700 1 -- 172.21.2.116:0/3477782923 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140107a0 con 0x7f361c145880 2021-11-21T22:01:24.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f36197fa700 1 -- 172.21.2.116:0/3477782923 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3614010b00 con 0x7f361c145880 2021-11-21T22:01:24.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c13cf30 con 0x7f361c145880 2021-11-21T22:01:24.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c1449c0 con 0x7f361c145880 2021-11-21T22:01:24.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c147040 con 0x7f361c145880 2021-11-21T22:01:24.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f36197fa700 1 -- 172.21.2.116:0/3477782923 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3614010c60 con 0x7f361c145880 2021-11-21T22:01:24.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f36197fa700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f36000535b0 0x7f3600055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f36197fa700 1 -- 172.21.2.116:0/3477782923 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f361406af00 con 0x7f361c145880 2021-11-21T22:01:24.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.438+0000 7f361bfff700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f36000535b0 0x7f3600055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.442+0000 7f361bfff700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f36000535b0 0x7f3600055a70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f3604016000 tx=0x7f3604015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.442+0000 7f36197fa700 1 -- 172.21.2.116:0/3477782923 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3614042060 con 0x7f361c145880 2021-11-21T22:01:24.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.522+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f361c0573d0 con 0x7f361c145880 2021-11-21T22:01:24.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f36197fa700 1 -- 172.21.2.116:0/3477782923 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16044 (secure 0 0 0) 0x7f3614045680 con 0x7f361c145880 2021-11-21T22:01:24.529 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:24.529 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":100,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:24.187518+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:21.079308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:22.145654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:24.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f36000535b0 msgr2=0x7f3600055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f36000535b0 0x7f3600055a70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f3604016000 tx=0x7f3604015000).stop 2021-11-21T22:01:24.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f361c145880 msgr2=0x7f361c13a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f361c145880 0x7f361c13a160 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f3614000e90 tx=0x7f3614001d10).stop 2021-11-21T22:01:24.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 shutdown_connections 2021-11-21T22:01:24.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f36000535b0 0x7f3600055a70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f361c145880 0x7f361c13a160 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f361c144f10 0x7f361c145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 --2- 172.21.2.116:0/3477782923 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35fc003800 0x7f361c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 >> 172.21.2.116:0/3477782923 conn(0x7f361c0b6000 msgr2=0x7f361c148430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 shutdown_connections 2021-11-21T22:01:24.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.526+0000 7f3622587700 1 -- 172.21.2.116:0/3477782923 wait complete. 2021-11-21T22:01:24.536 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:01:24.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 Processor -- start 2021-11-21T22:01:24.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- start start 2021-11-21T22:01:24.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd14c0aed90 0x7fd14c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c0af360 0x7fd14c13c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0b00d0 con 0x7fd14c0b24a0 2021-11-21T22:01:24.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c0adba0 con 0x7fd14c0aed90 2021-11-21T22:01:24.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b2880 con 0x7fd14c0af360 2021-11-21T22:01:24.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd14c0b24a0 0x7fd14c0ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34416/0 (socket says 172.21.2.116:34416) 2021-11-21T22:01:24.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 -- 172.21.2.116:0/767682847 learned_addr learned my addr 172.21.2.116:0/767682847 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:24.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138816213 0 0) 0x7fd14c0b00d0 con 0x7fd14c0b24a0 2021-11-21T22:01:24.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd134003260 con 0x7fd14c0b24a0 2021-11-21T22:01:24.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119862061 0 0) 0x7fd134003260 con 0x7fd14c0b24a0 2021-11-21T22:01:24.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c0af360 msgr2=0x7fd14c13c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c0af360 0x7fd14c13c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 >> v2:172.21.2.117:3300/0 conn(0x7fd14c0aed90 msgr2=0x7fd14c0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 --2- 172.21.2.116:0/767682847 >> v2:172.21.2.117:3300/0 conn(0x7fd14c0aed90 0x7fd14c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0b14a0 con 0x7fd14c0b24a0 2021-11-21T22:01:24.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd148001660 con 0x7fd14c0b24a0 2021-11-21T22:01:24.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 >> v1:172.21.2.116:6789/0 conn(0x7fd14c0b24a0 legacy=0x7fd14c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1340037d0 0x7fd134003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd134004500 0x7fd134004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c0af360 0x7fd134007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0adba0 con 0x7fd1340037d0 2021-11-21T22:01:24.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b2880 con 0x7fd134004500 2021-11-21T22:01:24.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd134005300 con 0x7fd14c0af360 2021-11-21T22:01:24.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1340037d0 0x7fd134003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 -- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c0af360 msgr2=0x7fd134007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c0af360 0x7fd134007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 -- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd134004500 msgr2=0x7fd134004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd134004500 0x7fd134004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 -- 172.21.2.116:0/767682847 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134003260 con 0x7fd1340037d0 2021-11-21T22:01:24.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1340037d0 0x7fd134003bf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fd148001370 tx=0x7fd148001d80).ready entity=mon.0 client_cookie=1d7383ab9fc57395 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1480070d0 con 0x7fd1340037d0 2021-11-21T22:01:24.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd148007230 con 0x7fd1340037d0 2021-11-21T22:01:24.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd13bfff700 1 -- 172.21.2.116:0/767682847 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd14800c8f0 con 0x7fd1340037d0 2021-11-21T22:01:24.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- 172.21.2.116:0/767682847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1340037d0 msgr2=0x7fd134003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 --2- 172.21.2.116:0/767682847 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1340037d0 0x7fd134003bf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fd148001370 tx=0x7fd148001d80).stop 2021-11-21T22:01:24.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151988700 1 -- 172.21.2.116:0/767682847 reap_dead start 2021-11-21T22:01:24.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- 172.21.2.116:0/767682847 shutdown_connections 2021-11-21T22:01:24.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- 172.21.2.116:0/767682847 >> 172.21.2.116:0/767682847 conn(0x7fd14c05fd00 msgr2=0x7fd14c0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- 172.21.2.116:0/767682847 shutdown_connections 2021-11-21T22:01:24.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- 172.21.2.116:0/767682847 wait complete. 2021-11-21T22:01:24.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 Processor -- start 2021-11-21T22:01:24.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- start start 2021-11-21T22:01:24.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd14c0b47f0 0x7fd14c065b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c1424a0 0x7fd14c063ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd14c064390 0x7fd14c064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c064d30 con 0x7fd14c0b47f0 2021-11-21T22:01:24.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c142070 con 0x7fd14c064390 2021-11-21T22:01:24.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd1533eb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b4bd0 con 0x7fd14c1424a0 2021-11-21T22:01:24.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd14c0b47f0 0x7fd14c065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd14c0b47f0 0x7fd14c065b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50704/0 (socket says 172.21.2.116:50704) 2021-11-21T22:01:24.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.590+0000 7fd151187700 1 -- 172.21.2.116:0/2567166543 learned_addr learned my addr 172.21.2.116:0/2567166543 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:24.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd151187700 1 -- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c1424a0 msgr2=0x7fd14c063ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd151187700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c1424a0 0x7fd14c063ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd151187700 1 -- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd14c064390 msgr2=0x7fd14c064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd151187700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd14c064390 0x7fd14c064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd151187700 1 -- 172.21.2.116:0/2567166543 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0b5490 con 0x7fd14c0b47f0 2021-11-21T22:01:24.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd151187700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd14c0b47f0 0x7fd14c065b90 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fd148001370 tx=0x7fd148015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd13a7fc700 1 -- 172.21.2.116:0/2567166543 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1480070d0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd13a7fc700 1 -- 172.21.2.116:0/2567166543 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd148018070 con 0x7fd14c0b47f0 2021-11-21T22:01:24.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd13a7fc700 1 -- 172.21.2.116:0/2567166543 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1480049e0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c0657e0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c142cd0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c1421e0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd13a7fc700 1 -- 172.21.2.116:0/2567166543 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd148024020 con 0x7fd14c0b47f0 2021-11-21T22:01:24.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd13a7fc700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053720 0x7fd12c055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd13a7fc700 1 -- 172.21.2.116:0/2567166543 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd14806c680 con 0x7fd14c0b47f0 2021-11-21T22:01:24.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd150986700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053720 0x7fd12c055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.594+0000 7fd150986700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053720 0x7fd12c055be0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fd13c002f70 tx=0x7fd13c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.598+0000 7fd13a7fc700 1 -- 172.21.2.116:0/2567166543 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd14c1421e0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.710+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd14c1421e0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd13a7fc700 1 -- 172.21.2.116:0/2567166543 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd14c1421e0 con 0x7fd14c0b47f0 2021-11-21T22:01:24.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053720 msgr2=0x7fd12c055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053720 0x7fd12c055be0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fd13c002f70 tx=0x7fd13c006a80).stop 2021-11-21T22:01:24.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd14c0b47f0 msgr2=0x7fd14c065b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd14c0b47f0 0x7fd14c065b90 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fd148001370 tx=0x7fd148015000).stop 2021-11-21T22:01:24.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 shutdown_connections 2021-11-21T22:01:24.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd14c064390 0x7fd14c064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd14c1424a0 0x7fd14c063ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd12c053720 0x7fd12c055be0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 --2- 172.21.2.116:0/2567166543 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd14c0b47f0 0x7fd14c065b90 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 >> 172.21.2.116:0/2567166543 conn(0x7fd14c05fd00 msgr2=0x7fd14c13f820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 shutdown_connections 2021-11-21T22:01:24.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.714+0000 7fd1533eb700 1 -- 172.21.2.116:0/2567166543 wait complete. 2021-11-21T22:01:24.722 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:01:24.726 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:01:24.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 Processor -- start 2021-11-21T22:01:24.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- start start 2021-11-21T22:01:24.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8b680a8670 0x7f8b68147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8b68143e60 0x7f8b681442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b680b1940 con 0x7f8b680b11c0 2021-11-21T22:01:24.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b680b15a0 con 0x7f8b68143e60 2021-11-21T22:01:24.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b680b1b70 con 0x7f8b680a8670 2021-11-21T22:01:24.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6659c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8b680b11c0 0x7f8b68147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34436/0 (socket says 172.21.2.116:34436) 2021-11-21T22:01:24.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6659c700 1 -- 172.21.2.116:0/214525948 learned_addr learned my addr 172.21.2.116:0/214525948 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:24.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b66d9d700 1 --2- 172.21.2.116:0/214525948 >> v2:172.21.2.116:3300/0 conn(0x7f8b68143e60 0x7f8b681442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6559a700 1 -- 172.21.2.116:0/214525948 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723986106 0 0) 0x7f8b680b1940 con 0x7f8b680b11c0 2021-11-21T22:01:24.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6559a700 1 -- 172.21.2.116:0/214525948 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b48003260 con 0x7f8b680b11c0 2021-11-21T22:01:24.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b66d9d700 1 -- 172.21.2.116:0/214525948 >> v2:172.21.2.117:3300/0 conn(0x7f8b680a8670 msgr2=0x7f8b68147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:24.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b66d9d700 1 --2- 172.21.2.116:0/214525948 >> v2:172.21.2.117:3300/0 conn(0x7f8b680a8670 0x7f8b68147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b66d9d700 1 -- 172.21.2.116:0/214525948 >> v1:172.21.2.116:6789/0 conn(0x7f8b680b11c0 legacy=0x7f8b68147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b66d9d700 1 -- 172.21.2.116:0/214525948 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680b1e10 con 0x7f8b68143e60 2021-11-21T22:01:24.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b66d9d700 1 --2- 172.21.2.116:0/214525948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b681442d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f8b600032f0 tx=0x7f8b600091d0).ready entity=mon.0 client_cookie=26f437273ad19187 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6559a700 1 -- 172.21.2.116:0/214525948 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b6000b680 con 0x7f8b68143e60 2021-11-21T22:01:24.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6559a700 1 -- 172.21.2.116:0/214525948 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6000b7e0 con 0x7f8b68143e60 2021-11-21T22:01:24.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6559a700 1 -- 172.21.2.116:0/214525948 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b6000b950 con 0x7f8b68143e60 2021-11-21T22:01:24.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- 172.21.2.116:0/214525948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 msgr2=0x7f8b681442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/214525948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b681442d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f8b600032f0 tx=0x7f8b600091d0).stop 2021-11-21T22:01:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- 172.21.2.116:0/214525948 shutdown_connections 2021-11-21T22:01:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/214525948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b681442d0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/214525948 >> v2:172.21.2.117:3300/0 conn(0x7f8b680a8670 0x7f8b68147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- 172.21.2.116:0/214525948 >> 172.21.2.116:0/214525948 conn(0x7f8b6805bd20 msgr2=0x7f8b6805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- 172.21.2.116:0/214525948 shutdown_connections 2021-11-21T22:01:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- 172.21.2.116:0/214525948 wait complete. 2021-11-21T22:01:24.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 Processor -- start 2021-11-21T22:01:24.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.778+0000 7f8b6cd54700 1 -- start start 2021-11-21T22:01:24.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b680a8670 0x7f8b680afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b680b11c0 0x7f8b6814a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b6814a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b680676e0 con 0x7f8b68143e60 2021-11-21T22:01:24.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6814beb0 con 0x7f8b680b11c0 2021-11-21T22:01:24.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6814c110 con 0x7f8b680a8670 2021-11-21T22:01:24.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b6814a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b6814a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50722/0 (socket says 172.21.2.116:50722) 2021-11-21T22:01:24.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 -- 172.21.2.116:0/2299707939 learned_addr learned my addr 172.21.2.116:0/2299707939 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:24.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 -- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b680a8670 msgr2=0x7f8b680afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b680a8670 0x7f8b680afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 -- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b680b11c0 msgr2=0x7f8b6814a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b680b11c0 0x7f8b6814a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 -- 172.21.2.116:0/2299707939 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60009470 con 0x7f8b68143e60 2021-11-21T22:01:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b66d9d700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b6814a730 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f8b600032f0 tx=0x7f8b60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b5f7fe700 1 -- 172.21.2.116:0/2299707939 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b60006ea0 con 0x7f8b68143e60 2021-11-21T22:01:24.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b5f7fe700 1 -- 172.21.2.116:0/2299707939 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b60021480 con 0x7f8b68143e60 2021-11-21T22:01:24.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b5f7fe700 1 -- 172.21.2.116:0/2299707939 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b6000b680 con 0x7f8b68143e60 2021-11-21T22:01:24.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6814c3b0 con 0x7f8b68143e60 2021-11-21T22:01:24.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b68149140 con 0x7f8b68143e60 2021-11-21T22:01:24.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b68148810 con 0x7f8b68143e60 2021-11-21T22:01:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b5f7fe700 1 -- 172.21.2.116:0/2299707939 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8b6000f070 con 0x7f8b68143e60 2021-11-21T22:01:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b5f7fe700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b4c0535d0 0x7f8b4c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b5f7fe700 1 -- 172.21.2.116:0/2299707939 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8b6006cc80 con 0x7f8b68143e60 2021-11-21T22:01:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6659c700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b4c0535d0 0x7f8b4c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.782+0000 7f8b6659c700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b4c0535d0 0x7f8b4c055a90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8b58001110 tx=0x7f8b580076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.786+0000 7f8b5f7fe700 1 -- 172.21.2.116:0/2299707939 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b68148810 con 0x7f8b68143e60 2021-11-21T22:01:24.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.898+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8b68148810 con 0x7f8b68143e60 2021-11-21T22:01:24.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b5f7fe700 1 -- 172.21.2.116:0/2299707939 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8b68148810 con 0x7f8b68143e60 2021-11-21T22:01:24.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b4c0535d0 msgr2=0x7f8b4c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b4c0535d0 0x7f8b4c055a90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8b58001110 tx=0x7f8b580076c0).stop 2021-11-21T22:01:24.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 msgr2=0x7f8b6814a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b6814a730 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f8b600032f0 tx=0x7f8b60016000).stop 2021-11-21T22:01:24.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 shutdown_connections 2021-11-21T22:01:24.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8b4c0535d0 0x7f8b4c055a90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8b68143e60 0x7f8b6814a730 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8b680b11c0 0x7f8b6814a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 --2- 172.21.2.116:0/2299707939 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8b680a8670 0x7f8b680afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 >> 172.21.2.116:0/2299707939 conn(0x7f8b6805bd20 msgr2=0x7f8b68146700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 shutdown_connections 2021-11-21T22:01:24.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.902+0000 7f8b6cd54700 1 -- 172.21.2.116:0/2299707939 wait complete. 2021-11-21T22:01:24.913 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:01:24.914 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:24.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 Processor -- start 2021-11-21T22:01:24.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- start start 2021-11-21T22:01:24.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f15900ad780 0x7f159005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f15900af7e0 0x7f159013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1590143f70 con 0x7f15900affc0 2021-11-21T22:01:24.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15900adb60 con 0x7f15900af7e0 2021-11-21T22:01:24.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1590143160 con 0x7f15900ad780 2021-11-21T22:01:24.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f15900af7e0 0x7f159013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f15900af7e0 0x7f159013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50726/0 (socket says 172.21.2.116:50726) 2021-11-21T22:01:24.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 -- 172.21.2.116:0/1094180542 learned_addr learned my addr 172.21.2.116:0/1094180542 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 -- 172.21.2.116:0/1094180542 >> v2:172.21.2.117:3300/0 conn(0x7f15900ad780 msgr2=0x7f159005f730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 --2- 172.21.2.116:0/1094180542 >> v2:172.21.2.117:3300/0 conn(0x7f15900ad780 0x7f159005f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 -- 172.21.2.116:0/1094180542 >> v1:172.21.2.116:6789/0 conn(0x7f15900affc0 legacy=0x7f1590143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 -- 172.21.2.116:0/1094180542 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900b0ce0 con 0x7f15900af7e0 2021-11-21T22:01:24.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f1596bb9700 1 --2- 172.21.2.116:0/1094180542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f159013fe70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f15880032f0 tx=0x7f158800b530).ready entity=mon.0 client_cookie=2fec2f843696fc9c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f15953b6700 1 -- 172.21.2.116:0/1094180542 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158800be70 con 0x7f15900af7e0 2021-11-21T22:01:24.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f15953b6700 1 -- 172.21.2.116:0/1094180542 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1588010470 con 0x7f15900af7e0 2021-11-21T22:01:24.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f15953b6700 1 -- 172.21.2.116:0/1094180542 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1588010810 con 0x7f15900af7e0 2021-11-21T22:01:24.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- 172.21.2.116:0/1094180542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 msgr2=0x7f159013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 --2- 172.21.2.116:0/1094180542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f159013fe70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f15880032f0 tx=0x7f158800b530).stop 2021-11-21T22:01:24.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- 172.21.2.116:0/1094180542 shutdown_connections 2021-11-21T22:01:24.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 --2- 172.21.2.116:0/1094180542 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f159013fe70 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 --2- 172.21.2.116:0/1094180542 >> v2:172.21.2.117:3300/0 conn(0x7f15900ad780 0x7f159005f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:24.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- 172.21.2.116:0/1094180542 >> 172.21.2.116:0/1094180542 conn(0x7f15900b6000 msgr2=0x7f15900b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:24.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- 172.21.2.116:0/1094180542 shutdown_connections 2021-11-21T22:01:24.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.962+0000 7f159861c700 1 -- 172.21.2.116:0/1094180542 wait complete. 2021-11-21T22:01:24.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 Processor -- start 2021-11-21T22:01:24.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 -- start start 2021-11-21T22:01:24.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15900ad780 0x7f1590067190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f1590065f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15900affc0 0x7f15900663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:24.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900a9550 con 0x7f15900af7e0 2021-11-21T22:01:24.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900a9f90 con 0x7f15900affc0 2021-11-21T22:01:24.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900aa6b0 con 0x7f15900ad780 2021-11-21T22:01:24.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f1590065f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f1590065f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50734/0 (socket says 172.21.2.116:50734) 2021-11-21T22:01:24.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 -- 172.21.2.116:0/380100717 learned_addr learned my addr 172.21.2.116:0/380100717 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:24.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15963b8700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15900ad780 0x7f1590067190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1596bb9700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15900affc0 0x7f15900663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:24.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 -- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15900ad780 msgr2=0x7f1590067190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15900ad780 0x7f1590067190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 -- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15900affc0 msgr2=0x7f15900663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:24.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15900affc0 0x7f15900663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:24.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 -- 172.21.2.116:0/380100717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158800b7d0 con 0x7f15900af7e0 2021-11-21T22:01:24.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15963b8700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15900ad780 0x7f1590067190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:24.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1596bb9700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15900affc0 0x7f15900663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:24.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f1595bb7700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f1590065f00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f1590068820 tx=0x7f158c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:24.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15837fe700 1 -- 172.21.2.116:0/380100717 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158c00ce30 con 0x7f15900af7e0 2021-11-21T22:01:24.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15837fe700 1 -- 172.21.2.116:0/380100717 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c012400 con 0x7f15900af7e0 2021-11-21T22:01:24.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15900aac10 con 0x7f15900af7e0 2021-11-21T22:01:24.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15837fe700 1 -- 172.21.2.116:0/380100717 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158c012760 con 0x7f15900af7e0 2021-11-21T22:01:24.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15900678a0 con 0x7f15900af7e0 2021-11-21T22:01:24.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15900aa870 con 0x7f15900af7e0 2021-11-21T22:01:24.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15837fe700 1 -- 172.21.2.116:0/380100717 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f158c019030 con 0x7f15900af7e0 2021-11-21T22:01:24.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15837fe700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1570053640 0x7f1570055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15837fe700 1 -- 172.21.2.116:0/380100717 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f158c06aeb0 con 0x7f15900af7e0 2021-11-21T22:01:25.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15963b8700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1570053640 0x7f1570055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.966+0000 7f15963b8700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1570053640 0x7f1570055b00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f1584007000 tx=0x7f1584008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:24.970+0000 7f15837fe700 1 -- 172.21.2.116:0/380100717 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f158c042010 con 0x7f15900af7e0 2021-11-21T22:01:25.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15900a9720 con 0x7f15900af7e0 2021-11-21T22:01:25.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f15837fe700 1 -- 172.21.2.116:0/380100717 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f158c045630 con 0x7f15900af7e0 2021-11-21T22:01:25.083 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:25.083 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:23.183200+0000","modified":"2021-11-21T22:01:24.187992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3417150056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3417150056},{"type":"v1","addr":"172.21.2.116:6835","nonce":3417150056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-11-21T22:01:25.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1570053640 msgr2=0x7f1570055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1570053640 0x7f1570055b00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f1584007000 tx=0x7f1584008040).stop 2021-11-21T22:01:25.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 msgr2=0x7f1590065f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f1590065f00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f1590068820 tx=0x7f158c007c20).stop 2021-11-21T22:01:25.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 shutdown_connections 2021-11-21T22:01:25.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15900affc0 0x7f15900663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1570053640 0x7f1570055b00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15900af7e0 0x7f1590065f00 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 --2- 172.21.2.116:0/380100717 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15900ad780 0x7f1590067190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 >> 172.21.2.116:0/380100717 conn(0x7f15900b6000 msgr2=0x7f1590069680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 shutdown_connections 2021-11-21T22:01:25.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.078+0000 7f159861c700 1 -- 172.21.2.116:0/380100717 wait complete. 2021-11-21T22:01:25.087 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 44 2021-11-21T22:01:25.392 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:23.183200+0000', 'modified': '2021-11-21T22:01:24.187992+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3417150056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3417150056}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3417150056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:25.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:25.394 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:25.460 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7011,"want_state":"up:active","state":"up:active","rank_uptime":2.268927277,"mdsmap_epoch":44,"osdmap_epoch":100,"osdmap_epoch_barrier":99,"uptime":6.4607715739999998} 2021-11-21T22:01:25.460 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7011, "mdsmap_epoch": 44, "osdmap_epoch": 100, "osdmap_epoch_barrier": 99, "rank_uptime": 2.268927277, "state": "up:active", "uptime": 6.460771574, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:25.462 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:25.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 Processor -- start 2021-11-21T22:01:25.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- start start 2021-11-21T22:01:25.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9d700b5630 0x7f9d7014af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d70099650 0x7f9d70099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7014a560 con 0x7f9d70097ec0 2021-11-21T22:01:25.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d70144b40 con 0x7f9d700b5630 2021-11-21T22:01:25.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7014a800 con 0x7f9d70099650 2021-11-21T22:01:25.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9d70097ec0 0x7f9d70146e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34464/0 (socket says 172.21.2.116:34464) 2021-11-21T22:01:25.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 -- 172.21.2.116:0/2585753846 learned_addr learned my addr 172.21.2.116:0/2585753846 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:25.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345738474 0 0) 0x7f9d7014a560 con 0x7f9d70097ec0 2021-11-21T22:01:25.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d54003260 con 0x7f9d70097ec0 2021-11-21T22:01:25.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836689595 0 0) 0x7f9d54003260 con 0x7f9d70097ec0 2021-11-21T22:01:25.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d70099650 msgr2=0x7f9d70099ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:25.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d70099650 0x7f9d70099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 >> v2:172.21.2.117:3300/0 conn(0x7f9d700b5630 msgr2=0x7f9d7014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:25.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 --2- 172.21.2.116:0/2585753846 >> v2:172.21.2.117:3300/0 conn(0x7f9d700b5630 0x7f9d7014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7014aaa0 con 0x7f9d70097ec0 2021-11-21T22:01:25.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9d64001680 con 0x7f9d70097ec0 2021-11-21T22:01:25.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 >> v1:172.21.2.116:6789/0 conn(0x7f9d70097ec0 legacy=0x7f9d70146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d540037d0 0x7f9d54003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d54004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d70099650 0x7f9d5400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70144b40 con 0x7f9d54004500 2021-11-21T22:01:25.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7014a800 con 0x7f9d540037d0 2021-11-21T22:01:25.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54005330 con 0x7f9d70099650 2021-11-21T22:01:25.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d54004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 -- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d70099650 msgr2=0x7f9d5400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:25.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d70099650 0x7f9d5400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 -- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d540037d0 msgr2=0x7f9d54003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:25.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d540037d0 0x7f9d54003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 -- 172.21.2.116:0/2585753846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d54003260 con 0x7f9d54004500 2021-11-21T22:01:25.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d54004980 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f9d64001390 tx=0x7f9d7014a6f0).ready entity=mon.0 client_cookie=176a8e364b9358ed server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d64007170 con 0x7f9d54004500 2021-11-21T22:01:25.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d640072d0 con 0x7f9d54004500 2021-11-21T22:01:25.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6dffb700 1 -- 172.21.2.116:0/2585753846 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6400c9c0 con 0x7f9d54004500 2021-11-21T22:01:25.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- 172.21.2.116:0/2585753846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 msgr2=0x7f9d54004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 --2- 172.21.2.116:0/2585753846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d54004980 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f9d64001390 tx=0x7f9d7014a6f0).stop 2021-11-21T22:01:25.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6f7fe700 1 -- 172.21.2.116:0/2585753846 reap_dead start 2021-11-21T22:01:25.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- 172.21.2.116:0/2585753846 shutdown_connections 2021-11-21T22:01:25.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- 172.21.2.116:0/2585753846 >> 172.21.2.116:0/2585753846 conn(0x7f9d7005c7f0 msgr2=0x7f9d700b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.510+0000 7f9d6ffff700 1 -- 172.21.2.116:0/2585753846 shutdown_connections 2021-11-21T22:01:25.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- 172.21.2.116:0/2585753846 wait complete. 2021-11-21T22:01:25.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 Processor -- start 2021-11-21T22:01:25.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- start start 2021-11-21T22:01:25.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d70063970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d700ac300 0x7f9d70153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d70063e60 0x7f9d70153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70153f20 con 0x7f9d54004500 2021-11-21T22:01:25.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700ace30 con 0x7f9d70063e60 2021-11-21T22:01:25.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70156de0 con 0x7f9d700ac300 2021-11-21T22:01:25.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d70063970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d70063970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50756/0 (socket says 172.21.2.116:50756) 2021-11-21T22:01:25.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 -- 172.21.2.116:0/886192820 learned_addr learned my addr 172.21.2.116:0/886192820 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:25.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d700ac300 0x7f9d70153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6f7fe700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d70063e60 0x7f9d70153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 -- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d700ac300 msgr2=0x7f9d70153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d700ac300 0x7f9d70153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 -- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d70063e60 msgr2=0x7f9d70153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d70063e60 0x7f9d70153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 -- 172.21.2.116:0/886192820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70157080 con 0x7f9d54004500 2021-11-21T22:01:25.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d700ac300 0x7f9d70153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:25.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6effd700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d70063970 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f9d60002f70 tx=0x7f9d60006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d5ffff700 1 -- 172.21.2.116:0/886192820 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d60013460 con 0x7f9d54004500 2021-11-21T22:01:25.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d5ffff700 1 -- 172.21.2.116:0/886192820 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d600135c0 con 0x7f9d54004500 2021-11-21T22:01:25.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d700ac990 con 0x7f9d54004500 2021-11-21T22:01:25.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d5ffff700 1 -- 172.21.2.116:0/886192820 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d60013920 con 0x7f9d54004500 2021-11-21T22:01:25.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d700ad4c0 con 0x7f9d54004500 2021-11-21T22:01:25.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d70157210 con 0x7f9d54004500 2021-11-21T22:01:25.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d5ffff700 1 -- 172.21.2.116:0/886192820 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9d6000f070 con 0x7f9d54004500 2021-11-21T22:01:25.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d5ffff700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0536d0 0x7f9d4c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.514+0000 7f9d5ffff700 1 -- 172.21.2.116:0/886192820 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9d6006b270 con 0x7f9d54004500 2021-11-21T22:01:25.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.518+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0536d0 0x7f9d4c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.518+0000 7f9d6e7fc700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0536d0 0x7f9d4c055b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f9d64015000 tx=0x7f9d64013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.518+0000 7f9d5ffff700 1 -- 172.21.2.116:0/886192820 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d600423d0 con 0x7f9d54004500 2021-11-21T22:01:25.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.602+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d700acb20 con 0x7f9d54004500 2021-11-21T22:01:25.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d5ffff700 1 -- 172.21.2.116:0/886192820 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16044 (secure 0 0 0) 0x7f9d600459f0 con 0x7f9d54004500 2021-11-21T22:01:25.609 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:25.609 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":100,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:24.187518+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:21.079308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:22.145654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:25.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0536d0 msgr2=0x7f9d4c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0536d0 0x7f9d4c055b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f9d64015000 tx=0x7f9d64013000).stop 2021-11-21T22:01:25.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 msgr2=0x7f9d70063970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d70063970 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f9d60002f70 tx=0x7f9d60006ec0).stop 2021-11-21T22:01:25.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 shutdown_connections 2021-11-21T22:01:25.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9d70063e60 0x7f9d70153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9d700ac300 0x7f9d70153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9d4c0536d0 0x7f9d4c055b90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 --2- 172.21.2.116:0/886192820 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9d54004500 0x7f9d70063970 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 >> 172.21.2.116:0/886192820 conn(0x7f9d7005c7f0 msgr2=0x7f9d7005f730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 shutdown_connections 2021-11-21T22:01:25.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.606+0000 7f9d6ffff700 1 -- 172.21.2.116:0/886192820 wait complete. 2021-11-21T22:01:25.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:01:21.079308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:01:22.145654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:01:25.617 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:25.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 Processor -- start 2021-11-21T22:01:25.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- start start 2021-11-21T22:01:25.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f65a80abce0 0x7f65a81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a8147e80 con 0x7f65a80ac600 2021-11-21T22:01:25.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a80af5a0 con 0x7f65a8144810 2021-11-21T22:01:25.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a80b1490 con 0x7f65a80abce0 2021-11-21T22:01:25.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a67fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f65a80abce0 0x7f65a81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a67fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f65a80abce0 0x7f65a81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50762/0 (socket says 172.21.2.116:50762) 2021-11-21T22:01:25.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a67fc700 1 -- 172.21.2.116:0/4153551908 learned_addr learned my addr 172.21.2.116:0/4153551908 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:25.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a67fc700 1 -- 172.21.2.116:0/4153551908 >> v1:172.21.2.117:6789/0 conn(0x7f65a8144810 legacy=0x7f65a80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:25.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a67fc700 1 -- 172.21.2.116:0/4153551908 >> v1:172.21.2.116:6789/0 conn(0x7f65a80ac600 legacy=0x7f65a80aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a67fc700 1 -- 172.21.2.116:0/4153551908 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a80b1fd0 con 0x7f65a80abce0 2021-11-21T22:01:25.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a67fc700 1 --2- 172.21.2.116:0/4153551908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a81478c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f65a80b4e00 tx=0x7f6598006eb0).ready entity=mon.0 client_cookie=c668e6c2886b97bb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a5ffb700 1 -- 172.21.2.116:0/4153551908 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659800d610 con 0x7f65a80abce0 2021-11-21T22:01:25.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a5ffb700 1 -- 172.21.2.116:0/4153551908 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6598000e90 con 0x7f65a80abce0 2021-11-21T22:01:25.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65a5ffb700 1 -- 172.21.2.116:0/4153551908 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659800d960 con 0x7f65a80abce0 2021-11-21T22:01:25.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- 172.21.2.116:0/4153551908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 msgr2=0x7f65a81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 --2- 172.21.2.116:0/4153551908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a81478c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f65a80b4e00 tx=0x7f6598006eb0).stop 2021-11-21T22:01:25.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- 172.21.2.116:0/4153551908 shutdown_connections 2021-11-21T22:01:25.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 --2- 172.21.2.116:0/4153551908 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a81478c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- 172.21.2.116:0/4153551908 >> 172.21.2.116:0/4153551908 conn(0x7f65a805bd20 msgr2=0x7f65a805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- 172.21.2.116:0/4153551908 shutdown_connections 2021-11-21T22:01:25.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- 172.21.2.116:0/4153551908 wait complete. 2021-11-21T22:01:25.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 Processor -- start 2021-11-21T22:01:25.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.666+0000 7f65ad36c700 1 -- start start 2021-11-21T22:01:25.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a8145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f65a80ac600 0x7f65a8146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f65a8144810 0x7f65a80b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8146ff0 con 0x7f65a80abce0 2021-11-21T22:01:25.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8157890 con 0x7f65a80ac600 2021-11-21T22:01:25.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8145a80 con 0x7f65a8144810 2021-11-21T22:01:25.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a8145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a8145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50766/0 (socket says 172.21.2.116:50766) 2021-11-21T22:01:25.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 -- 172.21.2.116:0/2929286199 learned_addr learned my addr 172.21.2.116:0/2929286199 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:25.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 -- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f65a8144810 msgr2=0x7f65a80b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f65a8144810 0x7f65a80b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 -- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f65a80ac600 msgr2=0x7f65a8146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f65a80ac600 0x7f65a8146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 -- 172.21.2.116:0/2929286199 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659800b1e0 con 0x7f65a80abce0 2021-11-21T22:01:25.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a6ffd700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a8145340 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f6590002ef0 tx=0x7f6590006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f659ffff700 1 -- 172.21.2.116:0/2929286199 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6590012410 con 0x7f65a80abce0 2021-11-21T22:01:25.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f659ffff700 1 -- 172.21.2.116:0/2929286199 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590012570 con 0x7f65a80abce0 2021-11-21T22:01:25.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a8145d20 con 0x7f65a80abce0 2021-11-21T22:01:25.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f659ffff700 1 -- 172.21.2.116:0/2929286199 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65900128d0 con 0x7f65a80abce0 2021-11-21T22:01:25.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a8147420 con 0x7f65a80abce0 2021-11-21T22:01:25.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a805cb80 con 0x7f65a80abce0 2021-11-21T22:01:25.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f659ffff700 1 -- 172.21.2.116:0/2929286199 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f659000f030 con 0x7f65a80abce0 2021-11-21T22:01:25.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f659ffff700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f658c0535b0 0x7f658c055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f659ffff700 1 -- 172.21.2.116:0/2929286199 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f659006b630 con 0x7f65a80abce0 2021-11-21T22:01:25.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a67fc700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f658c0535b0 0x7f658c055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.670+0000 7f65a67fc700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f658c0535b0 0x7f658c055a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f65a80b4e00 tx=0x7f6598015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.674+0000 7f659ffff700 1 -- 172.21.2.116:0/2929286199 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6590042790 con 0x7f65a80abce0 2021-11-21T22:01:25.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f65a8145eb0 con 0x7f65a80abce0 2021-11-21T22:01:25.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f659ffff700 1 -- 172.21.2.116:0/2929286199 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v100) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6590045db0 con 0x7f65a80abce0 2021-11-21T22:01:25.760 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:25.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f658c0535b0 msgr2=0x7f658c055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f658c0535b0 0x7f658c055a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f65a80b4e00 tx=0x7f6598015000).stop 2021-11-21T22:01:25.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 msgr2=0x7f65a8145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a8145340 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f6590002ef0 tx=0x7f6590006d50).stop 2021-11-21T22:01:25.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 shutdown_connections 2021-11-21T22:01:25.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f65a8144810 0x7f65a80b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f65a80ac600 0x7f65a8146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f658c0535b0 0x7f658c055a70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 --2- 172.21.2.116:0/2929286199 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f65a80abce0 0x7f65a8145340 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 >> 172.21.2.116:0/2929286199 conn(0x7f65a805bd20 msgr2=0x7f65a80af780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 shutdown_connections 2021-11-21T22:01:25.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.754+0000 7f65ad36c700 1 -- 172.21.2.116:0/2929286199 wait complete. 2021-11-21T22:01:25.767 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:25.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.814+0000 7f080e66b700 1 Processor -- start 2021-11-21T22:01:25.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.814+0000 7f080e66b700 1 -- start start 2021-11-21T22:01:25.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0808144810 0x7f08081478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f08080ac600 0x7f08080aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0808147e80 con 0x7f08080abce0 2021-11-21T22:01:25.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080b1490 con 0x7f08080ac600 2021-11-21T22:01:25.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080b1fd0 con 0x7f0808144810 2021-11-21T22:01:25.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f08080ac600 0x7f08080aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f08080ac600 0x7f08080aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50774/0 (socket says 172.21.2.116:50774) 2021-11-21T22:01:25.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/3453397680 learned_addr learned my addr 172.21.2.116:0/3453397680 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:25.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/3453397680 >> v2:172.21.2.117:3300/0 conn(0x7f0808144810 msgr2=0x7f08081478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:25.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- 172.21.2.116:0/3453397680 >> v2:172.21.2.117:3300/0 conn(0x7f0808144810 0x7f08081478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/3453397680 >> v1:172.21.2.117:6789/0 conn(0x7f08080abce0 legacy=0x7f08080b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:25.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/3453397680 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08080af5a0 con 0x7f08080ac600 2021-11-21T22:01:25.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- 172.21.2.116:0/3453397680 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08080ac600 0x7f08080aca50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f08080b4e00 tx=0x7f0800009290).ready entity=mon.0 client_cookie=166faa48206a697a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f0806ffd700 1 -- 172.21.2.116:0/3453397680 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080000b830 con 0x7f08080ac600 2021-11-21T22:01:25.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f0806ffd700 1 -- 172.21.2.116:0/3453397680 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080000b990 con 0x7f08080ac600 2021-11-21T22:01:25.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f0806ffd700 1 -- 172.21.2.116:0/3453397680 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080000bb40 con 0x7f08080ac600 2021-11-21T22:01:25.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- 172.21.2.116:0/3453397680 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08080ac600 msgr2=0x7f08080aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- 172.21.2.116:0/3453397680 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08080ac600 0x7f08080aca50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f08080b4e00 tx=0x7f0800009290).stop 2021-11-21T22:01:25.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- 172.21.2.116:0/3453397680 shutdown_connections 2021-11-21T22:01:25.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- 172.21.2.116:0/3453397680 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08080ac600 0x7f08080aca50 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- 172.21.2.116:0/3453397680 >> v2:172.21.2.117:3300/0 conn(0x7f0808144810 0x7f08081478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- 172.21.2.116:0/3453397680 >> 172.21.2.116:0/3453397680 conn(0x7f080805bd20 msgr2=0x7f080805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- 172.21.2.116:0/3453397680 shutdown_connections 2021-11-21T22:01:25.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- 172.21.2.116:0/3453397680 wait complete. 2021-11-21T22:01:25.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 Processor -- start 2021-11-21T22:01:25.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- start start 2021-11-21T22:01:25.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08080abce0 0x7f0808148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08080ac600 0x7f0808143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0808144810 0x7f0808062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08081497e0 con 0x7f0808144810 2021-11-21T22:01:25.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08080b3d30 con 0x7f08080ac600 2021-11-21T22:01:25.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08080b3fd0 con 0x7f08080abce0 2021-11-21T22:01:25.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0808144810 0x7f0808062fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0808144810 0x7f0808062fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50778/0 (socket says 172.21.2.116:50778) 2021-11-21T22:01:25.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/2855939413 learned_addr learned my addr 172.21.2.116:0/2855939413 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:25.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08080abce0 msgr2=0x7f0808148e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:25.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08080abce0 0x7f0808148e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08080ac600 msgr2=0x7f0808143320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:25.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08080ac600 0x7f0808143320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 -- 172.21.2.116:0/2855939413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009530 con 0x7f0808144810 2021-11-21T22:01:25.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080cc08700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0808144810 0x7f0808062fb0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f08080b4e00 tx=0x7f0800016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f08057fa700 1 -- 172.21.2.116:0/2855939413 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080000b830 con 0x7f0808144810 2021-11-21T22:01:25.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f08057fa700 1 -- 172.21.2.116:0/2855939413 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0800020620 con 0x7f0808144810 2021-11-21T22:01:25.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08080b4270 con 0x7f0808144810 2021-11-21T22:01:25.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f08057fa700 1 -- 172.21.2.116:0/2855939413 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0800020980 con 0x7f0808144810 2021-11-21T22:01:25.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.818+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08081493b0 con 0x7f0808144810 2021-11-21T22:01:25.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.822+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0808149600 con 0x7f0808144810 2021-11-21T22:01:25.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.822+0000 7f08057fa700 1 -- 172.21.2.116:0/2855939413 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f080000f070 con 0x7f0808144810 2021-11-21T22:01:25.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.822+0000 7f08057fa700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07ec0536a0 0x7f07ec055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.822+0000 7f08057fa700 1 -- 172.21.2.116:0/2855939413 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f080006cbf0 con 0x7f0808144810 2021-11-21T22:01:25.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.822+0000 7f0807fff700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07ec0536a0 0x7f07ec055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.822+0000 7f0807fff700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07ec0536a0 0x7f07ec055b60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f07f8002f70 tx=0x7f07f8006970).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.822+0000 7f08057fa700 1 -- 172.21.2.116:0/2855939413 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0800043d50 con 0x7f0808144810 2021-11-21T22:01:25.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f08080af060 con 0x7f0808144810 2021-11-21T22:01:25.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f08057fa700 1 -- 172.21.2.116:0/2855939413 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0800015070 con 0x7f0808144810 2021-11-21T22:01:25.910 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:25.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07ec0536a0 msgr2=0x7f07ec055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07ec0536a0 0x7f07ec055b60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f07f8002f70 tx=0x7f07f8006970).stop 2021-11-21T22:01:25.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0808144810 msgr2=0x7f0808062fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0808144810 0x7f0808062fb0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f08080b4e00 tx=0x7f0800016000).stop 2021-11-21T22:01:25.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 shutdown_connections 2021-11-21T22:01:25.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07ec0536a0 0x7f07ec055b60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0808144810 0x7f0808062fb0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08080ac600 0x7f0808143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 --2- 172.21.2.116:0/2855939413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08080abce0 0x7f0808148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 >> 172.21.2.116:0/2855939413 conn(0x7f080805bd20 msgr2=0x7f0808144d20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 shutdown_connections 2021-11-21T22:01:25.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.906+0000 7f080e66b700 1 -- 172.21.2.116:0/2855939413 wait complete. 2021-11-21T22:01:25.917 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:01:25.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 Processor -- start 2021-11-21T22:01:25.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- start start 2021-11-21T22:01:25.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7b68144810 0x7f7b681478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b680abce0 0x7f7b680b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b68147e80 con 0x7f7b680b1db0 2021-11-21T22:01:25.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b680b1490 con 0x7f7b68144810 2021-11-21T22:01:25.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680af060 con 0x7f7b680abce0 2021-11-21T22:01:25.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7b68144810 0x7f7b681478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7b68144810 0x7f7b681478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50786/0 (socket says 172.21.2.116:50786) 2021-11-21T22:01:25.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 -- 172.21.2.116:0/839243765 learned_addr learned my addr 172.21.2.116:0/839243765 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:25.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 -- 172.21.2.116:0/839243765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b680abce0 msgr2=0x7f7b680b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:25.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 --2- 172.21.2.116:0/839243765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b680abce0 0x7f7b680b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 -- 172.21.2.116:0/839243765 >> v1:172.21.2.116:6789/0 conn(0x7f7b680b1db0 legacy=0x7f7b680ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 -- 172.21.2.116:0/839243765 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680af5a0 con 0x7f7b68144810 2021-11-21T22:01:25.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b66ffd700 1 --2- 172.21.2.116:0/839243765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b68144810 0x7f7b681478c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f7b50002f70 tx=0x7f7b50006eb0).ready entity=mon.0 client_cookie=c656d13fc231226 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b65ffb700 1 -- 172.21.2.116:0/839243765 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b5000d5c0 con 0x7f7b68144810 2021-11-21T22:01:25.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b65ffb700 1 -- 172.21.2.116:0/839243765 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50000e90 con 0x7f7b68144810 2021-11-21T22:01:25.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b65ffb700 1 -- 172.21.2.116:0/839243765 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b5000d950 con 0x7f7b68144810 2021-11-21T22:01:25.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/839243765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b68144810 msgr2=0x7f7b681478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/839243765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b68144810 0x7f7b681478c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f7b50002f70 tx=0x7f7b50006eb0).stop 2021-11-21T22:01:25.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/839243765 shutdown_connections 2021-11-21T22:01:25.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/839243765 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b680abce0 0x7f7b680b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/839243765 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b68144810 0x7f7b681478c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:25.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/839243765 >> 172.21.2.116:0/839243765 conn(0x7f7b6805bd20 msgr2=0x7f7b6805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:25.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/839243765 shutdown_connections 2021-11-21T22:01:25.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.966+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/839243765 wait complete. 2021-11-21T22:01:25.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 Processor -- start 2021-11-21T22:01:25.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 -- start start 2021-11-21T22:01:25.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7b680abce0 0x7f7b681437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b68144810 0x7f7b6813c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b681469b0 0x7f7b6813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6813d130 con 0x7f7b681469b0 2021-11-21T22:01:25.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68156700 con 0x7f7b680abce0 2021-11-21T22:01:25.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6815a000 con 0x7f7b68144810 2021-11-21T22:01:25.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b681469b0 0x7f7b6813cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b681469b0 0x7f7b6813cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50794/0 (socket says 172.21.2.116:50794) 2021-11-21T22:01:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 -- 172.21.2.116:0/3500674439 learned_addr learned my addr 172.21.2.116:0/3500674439 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 -- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b68144810 msgr2=0x7f7b6813c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b68144810 0x7f7b6813c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 -- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7b680abce0 msgr2=0x7f7b681437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:25.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7b680abce0 0x7f7b681437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:25.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 -- 172.21.2.116:0/3500674439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5000b1e0 con 0x7f7b681469b0 2021-11-21T22:01:25.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b677fe700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b681469b0 0x7f7b6813cc40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f7b680b2a30 tx=0x7f7b60007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b5ffff700 1 -- 172.21.2.116:0/3500674439 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6000e4a0 con 0x7f7b681469b0 2021-11-21T22:01:25.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b5ffff700 1 -- 172.21.2.116:0/3500674439 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000e600 con 0x7f7b681469b0 2021-11-21T22:01:25.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b68145f10 con 0x7f7b681469b0 2021-11-21T22:01:25.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b5ffff700 1 -- 172.21.2.116:0/3500674439 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6000e960 con 0x7f7b681469b0 2021-11-21T22:01:25.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b68145a70 con 0x7f7b681469b0 2021-11-21T22:01:25.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b680b2ef0 con 0x7f7b681469b0 2021-11-21T22:01:25.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b5ffff700 1 -- 172.21.2.116:0/3500674439 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7b60010070 con 0x7f7b681469b0 2021-11-21T22:01:25.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b5ffff700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7b4c0536a0 0x7f7b4c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:25.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b5ffff700 1 -- 172.21.2.116:0/3500674439 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7b6006b130 con 0x7f7b681469b0 2021-11-21T22:01:25.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b66ffd700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7b4c0536a0 0x7f7b4c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:25.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.970+0000 7f7b66ffd700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7b4c0536a0 0x7f7b4c055b60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f7b50002f70 tx=0x7f7b50015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:25.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:25.974+0000 7f7b5ffff700 1 -- 172.21.2.116:0/3500674439 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b60042230 con 0x7f7b681469b0 2021-11-21T22:01:26.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b680af300 con 0x7f7b681469b0 2021-11-21T22:01:26.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b5ffff700 1 -- 172.21.2.116:0/3500674439 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7b60045850 con 0x7f7b681469b0 2021-11-21T22:01:26.062 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:26.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7b4c0536a0 msgr2=0x7f7b4c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7b4c0536a0 0x7f7b4c055b60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f7b50002f70 tx=0x7f7b50015000).stop 2021-11-21T22:01:26.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b681469b0 msgr2=0x7f7b6813cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b681469b0 0x7f7b6813cc40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f7b680b2a30 tx=0x7f7b60007f10).stop 2021-11-21T22:01:26.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 shutdown_connections 2021-11-21T22:01:26.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7b4c0536a0 0x7f7b4c055b60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7b681469b0 0x7f7b6813cc40 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7b68144810 0x7f7b6813c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 --2- 172.21.2.116:0/3500674439 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7b680abce0 0x7f7b681437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 >> 172.21.2.116:0/3500674439 conn(0x7f7b6805bd20 msgr2=0x7f7b681400c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:26.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 shutdown_connections 2021-11-21T22:01:26.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.058+0000 7f7b6d4dc700 1 -- 172.21.2.116:0/3500674439 wait complete. 2021-11-21T22:01:26.069 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:01:26.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 Processor -- start 2021-11-21T22:01:26.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- start start 2021-11-21T22:01:26.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49b4144810 0x7f49b41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f49b40abce0 0x7f49b40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b4147e80 con 0x7f49b40b1db0 2021-11-21T22:01:26.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b40b1490 con 0x7f49b4144810 2021-11-21T22:01:26.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b40af060 con 0x7f49b40abce0 2021-11-21T22:01:26.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49b4144810 0x7f49b41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:26.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49b4144810 0x7f49b41478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50800/0 (socket says 172.21.2.116:50800) 2021-11-21T22:01:26.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 -- 172.21.2.116:0/1986604089 learned_addr learned my addr 172.21.2.116:0/1986604089 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:26.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 -- 172.21.2.116:0/1986604089 >> v2:172.21.2.117:3300/0 conn(0x7f49b40abce0 msgr2=0x7f49b40b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:26.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 --2- 172.21.2.116:0/1986604089 >> v2:172.21.2.117:3300/0 conn(0x7f49b40abce0 0x7f49b40b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 -- 172.21.2.116:0/1986604089 >> v1:172.21.2.117:6789/0 conn(0x7f49b40b1db0 legacy=0x7f49b40ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:26.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 -- 172.21.2.116:0/1986604089 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b40af5a0 con 0x7f49b4144810 2021-11-21T22:01:26.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b37fe700 1 --2- 172.21.2.116:0/1986604089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b4144810 0x7f49b41478c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f49b40b4e00 tx=0x7f49a4006eb0).ready entity=mon.0 client_cookie=d8573697aa28fe57 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:26.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b27fc700 1 -- 172.21.2.116:0/1986604089 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a400d5c0 con 0x7f49b4144810 2021-11-21T22:01:26.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b27fc700 1 -- 172.21.2.116:0/1986604089 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a4000e90 con 0x7f49b4144810 2021-11-21T22:01:26.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b27fc700 1 -- 172.21.2.116:0/1986604089 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a400d910 con 0x7f49b4144810 2021-11-21T22:01:26.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- 172.21.2.116:0/1986604089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b4144810 msgr2=0x7f49b41478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/1986604089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b4144810 0x7f49b41478c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f49b40b4e00 tx=0x7f49a4006eb0).stop 2021-11-21T22:01:26.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- 172.21.2.116:0/1986604089 shutdown_connections 2021-11-21T22:01:26.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/1986604089 >> v2:172.21.2.117:3300/0 conn(0x7f49b40abce0 0x7f49b40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/1986604089 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b4144810 0x7f49b41478c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- 172.21.2.116:0/1986604089 >> 172.21.2.116:0/1986604089 conn(0x7f49b405bd20 msgr2=0x7f49b405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:26.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- 172.21.2.116:0/1986604089 shutdown_connections 2021-11-21T22:01:26.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- 172.21.2.116:0/1986604089 wait complete. 2021-11-21T22:01:26.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 Processor -- start 2021-11-21T22:01:26.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.118+0000 7f49b9ce5700 1 -- start start 2021-11-21T22:01:26.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b40abce0 0x7f49b4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49b4144810 0x7f49b40b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49b413cd00 0x7f49b413d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4144db0 con 0x7f49b40abce0 2021-11-21T22:01:26.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40b2ab0 con 0x7f49b413cd00 2021-11-21T22:01:26.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40b4b20 con 0x7f49b4144810 2021-11-21T22:01:26.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b40abce0 0x7f49b4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:26.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b40abce0 0x7f49b4143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50806/0 (socket says 172.21.2.116:50806) 2021-11-21T22:01:26.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 -- 172.21.2.116:0/449218222 learned_addr learned my addr 172.21.2.116:0/449218222 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:26.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 -- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49b4144810 msgr2=0x7f49b40b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49b4144810 0x7f49b40b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 -- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49b413cd00 msgr2=0x7f49b413d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49b413cd00 0x7f49b413d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 -- 172.21.2.116:0/449218222 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a400b1e0 con 0x7f49b40abce0 2021-11-21T22:01:26.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b37fe700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b40abce0 0x7f49b4143a80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f49b40b4e00 tx=0x7f49a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:26.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b0ff9700 1 -- 172.21.2.116:0/449218222 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a4002f70 con 0x7f49b40abce0 2021-11-21T22:01:26.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b0ff9700 1 -- 172.21.2.116:0/449218222 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a400de70 con 0x7f49b40abce0 2021-11-21T22:01:26.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b40b44b0 con 0x7f49b40abce0 2021-11-21T22:01:26.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b0ff9700 1 -- 172.21.2.116:0/449218222 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a401e460 con 0x7f49b40abce0 2021-11-21T22:01:26.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b40b2d10 con 0x7f49b40abce0 2021-11-21T22:01:26.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b40b2ed0 con 0x7f49b40abce0 2021-11-21T22:01:26.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b0ff9700 1 -- 172.21.2.116:0/449218222 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f49a4025030 con 0x7f49b40abce0 2021-11-21T22:01:26.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b0ff9700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49980536a0 0x7f4998055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b0ff9700 1 -- 172.21.2.116:0/449218222 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f49a406ccf0 con 0x7f49b40abce0 2021-11-21T22:01:26.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.122+0000 7f49b2ffd700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49980536a0 0x7f4998055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:26.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.126+0000 7f49b2ffd700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49980536a0 0x7f4998055b60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f499c002fb0 tx=0x7f499c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:26.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.126+0000 7f49b0ff9700 1 -- 172.21.2.116:0/449218222 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49a4043dd0 con 0x7f49b40abce0 2021-11-21T22:01:26.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.206+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49b40573d0 con 0x7f49b40abce0 2021-11-21T22:01:26.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b0ff9700 1 -- 172.21.2.116:0/449218222 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+10 (secure 0 0 0) 0x7f49a4023020 con 0x7f49b40abce0 2021-11-21T22:01:26.213 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:26.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49980536a0 msgr2=0x7f4998055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49980536a0 0x7f4998055b60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f499c002fb0 tx=0x7f499c0069b0).stop 2021-11-21T22:01:26.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b40abce0 msgr2=0x7f49b4143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b40abce0 0x7f49b4143a80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f49b40b4e00 tx=0x7f49a4015000).stop 2021-11-21T22:01:26.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 shutdown_connections 2021-11-21T22:01:26.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49b413cd00 0x7f49b413d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49b4144810 0x7f49b40b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49980536a0 0x7f4998055b60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 --2- 172.21.2.116:0/449218222 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49b40abce0 0x7f49b4143a80 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 >> 172.21.2.116:0/449218222 conn(0x7f49b405bd20 msgr2=0x7f49b405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:26.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 shutdown_connections 2021-11-21T22:01:26.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.210+0000 7f49b9ce5700 1 -- 172.21.2.116:0/449218222 wait complete. 2021-11-21T22:01:26.220 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:01:26.220 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:26.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 Processor -- start 2021-11-21T22:01:26.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 -- start start 2021-11-21T22:01:26.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f171c0a86b0 0x7f171c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c0a6e80 con 0x7f171c0a6aa0 2021-11-21T22:01:26.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c14a7b0 con 0x7f171c062410 2021-11-21T22:01:26.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c0a8bc0 con 0x7f171c0a86b0 2021-11-21T22:01:26.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1720c14700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f171c0a86b0 0x7f171c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:26.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1720c14700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f171c0a86b0 0x7f171c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50816/0 (socket says 172.21.2.116:50816) 2021-11-21T22:01:26.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1720c14700 1 -- 172.21.2.116:0/1118121407 learned_addr learned my addr 172.21.2.116:0/1118121407 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:26.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1720c14700 1 -- 172.21.2.116:0/1118121407 >> v1:172.21.2.117:6789/0 conn(0x7f171c062410 legacy=0x7f171c1443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:26.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1720c14700 1 -- 172.21.2.116:0/1118121407 >> v1:172.21.2.116:6789/0 conn(0x7f171c0a6aa0 legacy=0x7f171c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1720c14700 1 -- 172.21.2.116:0/1118121407 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c14aa10 con 0x7f171c0a86b0 2021-11-21T22:01:26.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1720c14700 1 --2- 172.21.2.116:0/1118121407 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c14b150 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f171c061b50 tx=0x7f1718006eb0).ready entity=mon.0 client_cookie=fda9282462741e61 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:26.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f170ffff700 1 -- 172.21.2.116:0/1118121407 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171800d610 con 0x7f171c0a86b0 2021-11-21T22:01:26.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f170ffff700 1 -- 172.21.2.116:0/1118121407 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718000e90 con 0x7f171c0a86b0 2021-11-21T22:01:26.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f170ffff700 1 -- 172.21.2.116:0/1118121407 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171800d960 con 0x7f171c0a86b0 2021-11-21T22:01:26.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 -- 172.21.2.116:0/1118121407 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 msgr2=0x7f171c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 --2- 172.21.2.116:0/1118121407 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c14b150 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f171c061b50 tx=0x7f1718006eb0).stop 2021-11-21T22:01:26.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 -- 172.21.2.116:0/1118121407 shutdown_connections 2021-11-21T22:01:26.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 --2- 172.21.2.116:0/1118121407 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c14b150 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.266+0000 7f1723679700 1 -- 172.21.2.116:0/1118121407 >> 172.21.2.116:0/1118121407 conn(0x7f171c05bd20 msgr2=0x7f171c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:26.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- 172.21.2.116:0/1118121407 shutdown_connections 2021-11-21T22:01:26.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- 172.21.2.116:0/1118121407 wait complete. 2021-11-21T22:01:26.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 Processor -- start 2021-11-21T22:01:26.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- start start 2021-11-21T22:01:26.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f171c062410 0x7f171c148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f171c0a6aa0 0x7f171c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c0649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1599b0 con 0x7f171c0a86b0 2021-11-21T22:01:26.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c067130 con 0x7f171c0a6aa0 2021-11-21T22:01:26.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c159be0 con 0x7f171c062410 2021-11-21T22:01:26.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c0649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:26.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c0649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50822/0 (socket says 172.21.2.116:50822) 2021-11-21T22:01:26.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 -- 172.21.2.116:0/2649551494 learned_addr learned my addr 172.21.2.116:0/2649551494 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:26.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 -- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f171c062410 msgr2=0x7f171c148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f171c062410 0x7f171c148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 -- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f171c0a6aa0 msgr2=0x7f171c15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f171c0a6aa0 0x7f171c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 -- 172.21.2.116:0/2649551494 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171800b1e0 con 0x7f171c0a86b0 2021-11-21T22:01:26.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1721c16700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c0649d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f1714003270 tx=0x7f17140090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:26.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f170e7fc700 1 -- 172.21.2.116:0/2649551494 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171400d4a0 con 0x7f171c0a86b0 2021-11-21T22:01:26.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f170e7fc700 1 -- 172.21.2.116:0/2649551494 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400d600 con 0x7f171c0a86b0 2021-11-21T22:01:26.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c1492f0 con 0x7f171c0a86b0 2021-11-21T22:01:26.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c0673d0 con 0x7f171c0a86b0 2021-11-21T22:01:26.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f170e7fc700 1 -- 172.21.2.116:0/2649551494 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171400d960 con 0x7f171c0a86b0 2021-11-21T22:01:26.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f170e7fc700 1 -- 172.21.2.116:0/2649551494 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1714018030 con 0x7f171c0a86b0 2021-11-21T22:01:26.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c159da0 con 0x7f171c0a86b0 2021-11-21T22:01:26.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f170e7fc700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16fc053580 0x7f16fc055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:26.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.270+0000 7f170e7fc700 1 -- 172.21.2.116:0/2649551494 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f171406b1c0 con 0x7f171c0a86b0 2021-11-21T22:01:26.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.274+0000 7f1721415700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16fc053580 0x7f16fc055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:26.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.274+0000 7f1721415700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16fc053580 0x7f16fc055a40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f171c061ab0 tx=0x7f17100077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:26.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.274+0000 7f170e7fc700 1 -- 172.21.2.116:0/2649551494 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f17140423d0 con 0x7f171c0a86b0 2021-11-21T22:01:26.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f171c0573d0 con 0x7f171c0a86b0 2021-11-21T22:01:26.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f170e7fc700 1 -- 172.21.2.116:0/2649551494 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f17140459f0 con 0x7f171c0a86b0 2021-11-21T22:01:26.387 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:26.387 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:23.183200+0000","modified":"2021-11-21T22:01:24.187992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3417150056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3417150056},{"type":"v1","addr":"172.21.2.116:6835","nonce":3417150056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-11-21T22:01:26.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16fc053580 msgr2=0x7f16fc055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16fc053580 0x7f16fc055a40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f171c061ab0 tx=0x7f17100077f0).stop 2021-11-21T22:01:26.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 msgr2=0x7f171c0649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:26.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c0649d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f1714003270 tx=0x7f17140090f0).stop 2021-11-21T22:01:26.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 shutdown_connections 2021-11-21T22:01:26.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16fc053580 0x7f16fc055a40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f171c0a86b0 0x7f171c0649d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:26.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f171c0a6aa0 0x7f171c15a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 --2- 172.21.2.116:0/2649551494 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f171c062410 0x7f171c148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:26.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 >> 172.21.2.116:0/2649551494 conn(0x7f171c05bd20 msgr2=0x7f171c142690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:26.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 shutdown_connections 2021-11-21T22:01:26.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:26.382+0000 7f1723679700 1 -- 172.21.2.116:0/2649551494 wait complete. 2021-11-21T22:01:26.394 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 44 2021-11-21T22:01:26.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:23.183200+0000', 'modified': '2021-11-21T22:01:24.187992+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3417150056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3417150056}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3417150056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:26.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:26.700 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:26.766 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7011,"want_state":"up:active","state":"up:active","rank_uptime":3.5757093040000001,"mdsmap_epoch":44,"osdmap_epoch":100,"osdmap_epoch_barrier":99,"uptime":7.7675557939999997} 2021-11-21T22:01:26.766 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7011, "mdsmap_epoch": 44, "osdmap_epoch": 100, "osdmap_epoch_barrier": 99, "rank_uptime": 3.575709304, "state": "up:active", "uptime": 7.767555794, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:26.767 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:01:26.767 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:01:26.768 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:01:26.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:26.769 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:26.780 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:26.832 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:26.832 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:26.838 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:26.839 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:26.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:26.840 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:26.844 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:01:26.845 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:26.887 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:26.896 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:26.896 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:26.901 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:26.902 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:26.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:26.903 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:26.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:01:26.925+0000 7fc13ce12080 -1 init, newargv = 0x56241de80c30 newargc=9 2021-11-21T22:01:26.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14867]: starting ceph client 2021-11-21T22:01:26.937 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14867]: starting fuse 2021-11-21T22:01:28.008 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:28.021 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:28.021 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:28.073 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:28.073 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:28.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:28.075 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:28.081 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:01:28.081 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:01:28.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:28.083 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:01:28.083 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:01:28.084 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:01:28.085 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:01:28.086 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:01:28.086 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:28.087 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:01:28.088 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:01:28.089 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:01:28.089 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:28.090 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:01:28.091 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:01:28.092 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:01:28.092 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:01:28.092 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:28.093 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:01:28.093 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:01:28.094 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:01:28.094 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:01:28.094 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:01:28.095 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:28.095 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:01:28.096 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:01:28.132 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14867.asok 2021-11-21T22:01:28.134 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14867.asok 2021-11-21T22:01:28.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:28.135 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14867.asok status 2021-11-21T22:01:28.242 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:01:28.243 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:01:28.243 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:01:28.243 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:01:28.243 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:01:28.244 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:01:28.244 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:01:28.244 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14867", 2021-11-21T22:01:28.244 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:01:28.245 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:28.245 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:01:28.245 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:01:28.245 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4987, 2021-11-21T22:01:28.245 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:01:28.246 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:01:28.246 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:01:28.246 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4987 2021-11-21T22:01:28.246 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:28.247 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:01:28.247 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:01:28.247 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:01:28.247 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 375157747 2021-11-21T22:01:28.247 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:01:28.248 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:28.248 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:01:28.248 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:01:28.248 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:01:28.249 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 375157747 2021-11-21T22:01:28.249 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:28.249 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4987 172.21.2.119:0/375157747", 2021-11-21T22:01:28.249 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/375157747", 2021-11-21T22:01:28.249 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:01:28.250 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 44, 2021-11-21T22:01:28.250 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 100, 2021-11-21T22:01:28.250 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:01:28.250 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:01:28.251 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:01:28.251 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:01:28.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:28.253 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:28.253 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:28.259 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:01:28.259 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:28.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:28.260 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:28.269 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:28.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 Processor -- start 2021-11-21T22:01:28.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- start start 2021-11-21T22:01:28.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf400a6aa0 0x7faf4014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7faf40062410 0x7faf401443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf400a6e80 con 0x7faf400a86b0 2021-11-21T22:01:28.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf400a8bc0 con 0x7faf400a6aa0 2021-11-21T22:01:28.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4014a7b0 con 0x7faf40062410 2021-11-21T22:01:28.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf400a6aa0 0x7faf4014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf400a6aa0 0x7faf4014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50826/0 (socket says 172.21.2.116:50826) 2021-11-21T22:01:28.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 -- 172.21.2.116:0/2131698281 learned_addr learned my addr 172.21.2.116:0/2131698281 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 -- 172.21.2.116:0/2131698281 >> v2:172.21.2.117:3300/0 conn(0x7faf40062410 msgr2=0x7faf401443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:28.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 --2- 172.21.2.116:0/2131698281 >> v2:172.21.2.117:3300/0 conn(0x7faf40062410 0x7faf401443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 -- 172.21.2.116:0/2131698281 >> v1:172.21.2.116:6789/0 conn(0x7faf400a86b0 legacy=0x7faf401470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3dffb700 1 -- 172.21.2.116:0/2131698281 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635901890 0 0) 0x7faf400a6e80 con 0x7faf400a86b0 2021-11-21T22:01:28.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 -- 172.21.2.116:0/2131698281 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4014aa10 con 0x7faf400a6aa0 2021-11-21T22:01:28.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3effd700 1 --2- 172.21.2.116:0/2131698281 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4014b150 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7faf30002f70 tx=0x7faf30006eb0).ready entity=mon.0 client_cookie=729228f4e936f396 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3dffb700 1 -- 172.21.2.116:0/2131698281 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf3000d610 con 0x7faf400a6aa0 2021-11-21T22:01:28.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3dffb700 1 -- 172.21.2.116:0/2131698281 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30000e90 con 0x7faf400a6aa0 2021-11-21T22:01:28.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf3dffb700 1 -- 172.21.2.116:0/2131698281 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf3000d960 con 0x7faf400a6aa0 2021-11-21T22:01:28.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- 172.21.2.116:0/2131698281 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 msgr2=0x7faf4014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 --2- 172.21.2.116:0/2131698281 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4014b150 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7faf30002f70 tx=0x7faf30006eb0).stop 2021-11-21T22:01:28.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- 172.21.2.116:0/2131698281 shutdown_connections 2021-11-21T22:01:28.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 --2- 172.21.2.116:0/2131698281 >> v2:172.21.2.117:3300/0 conn(0x7faf40062410 0x7faf401443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 --2- 172.21.2.116:0/2131698281 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4014b150 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- 172.21.2.116:0/2131698281 >> 172.21.2.116:0/2131698281 conn(0x7faf4005bd20 msgr2=0x7faf4005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- 172.21.2.116:0/2131698281 shutdown_connections 2021-11-21T22:01:28.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.322+0000 7faf4591b700 1 -- 172.21.2.116:0/2131698281 wait complete. 2021-11-21T22:01:28.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 Processor -- start 2021-11-21T22:01:28.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 -- start start 2021-11-21T22:01:28.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf40062410 0x7faf40141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4013bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf400a86b0 0x7faf40159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4014c5c0 con 0x7faf400a6aa0 2021-11-21T22:01:28.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400b4ac0 con 0x7faf400a86b0 2021-11-21T22:01:28.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf40142780 con 0x7faf40062410 2021-11-21T22:01:28.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4013bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4013bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50836/0 (socket says 172.21.2.116:50836) 2021-11-21T22:01:28.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 -- 172.21.2.116:0/1797374334 learned_addr learned my addr 172.21.2.116:0/1797374334 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 -- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf40062410 msgr2=0x7faf40141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf40062410 0x7faf40141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 -- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf400a86b0 msgr2=0x7faf40159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf400a86b0 0x7faf40159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 -- 172.21.2.116:0/1797374334 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3000b1e0 con 0x7faf400a6aa0 2021-11-21T22:01:28.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3e7fc700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4013bb40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7faf28000ed0 tx=0x7faf28007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf37fff700 1 -- 172.21.2.116:0/1797374334 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf2800ce80 con 0x7faf400a6aa0 2021-11-21T22:01:28.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf37fff700 1 -- 172.21.2.116:0/1797374334 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28013450 con 0x7faf400a6aa0 2021-11-21T22:01:28.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf400b40e0 con 0x7faf400a6aa0 2021-11-21T22:01:28.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf400b4620 con 0x7faf400a6aa0 2021-11-21T22:01:28.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf37fff700 1 -- 172.21.2.116:0/1797374334 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf280137b0 con 0x7faf400a6aa0 2021-11-21T22:01:28.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf400573d0 con 0x7faf400a6aa0 2021-11-21T22:01:28.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf37fff700 1 -- 172.21.2.116:0/1797374334 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7faf2801a060 con 0x7faf400a6aa0 2021-11-21T22:01:28.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf37fff700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf24053660 0x7faf24055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf37fff700 1 -- 172.21.2.116:0/1797374334 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faf2806b200 con 0x7faf400a6aa0 2021-11-21T22:01:28.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.326+0000 7faf3effd700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf24053660 0x7faf24055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.330+0000 7faf3effd700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf24053660 0x7faf24055b20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7faf30002f70 tx=0x7faf30015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.330+0000 7faf37fff700 1 -- 172.21.2.116:0/1797374334 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faf28042300 con 0x7faf400a6aa0 2021-11-21T22:01:28.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf400b4c80 con 0x7faf400a6aa0 2021-11-21T22:01:28.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf37fff700 1 -- 172.21.2.116:0/1797374334 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16044 (secure 0 0 0) 0x7faf28018020 con 0x7faf400a6aa0 2021-11-21T22:01:28.418 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:28.419 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":100,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:24.187518+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:21.079308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:22.145654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:28.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf24053660 msgr2=0x7faf24055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf24053660 0x7faf24055b20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7faf30002f70 tx=0x7faf30015000).stop 2021-11-21T22:01:28.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 msgr2=0x7faf4013bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4013bb40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7faf28000ed0 tx=0x7faf28007c20).stop 2021-11-21T22:01:28.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 shutdown_connections 2021-11-21T22:01:28.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf400a86b0 0x7faf40159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf24053660 0x7faf24055b20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf400a6aa0 0x7faf4013bb40 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 --2- 172.21.2.116:0/1797374334 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf40062410 0x7faf40141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 >> 172.21.2.116:0/1797374334 conn(0x7faf4005bd20 msgr2=0x7faf40149b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 shutdown_connections 2021-11-21T22:01:28.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.414+0000 7faf4591b700 1 -- 172.21.2.116:0/1797374334 wait complete. 2021-11-21T22:01:28.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:01:21.079308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:01:22.145654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:01:28.426 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:28.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff88a373700 1 Processor -- start 2021-11-21T22:01:28.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff88a373700 1 -- start start 2021-11-21T22:01:28.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff88a373700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff884144810 0x7ff8841478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff88a373700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff8840abce0 0x7ff8840b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff88a373700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff884147e80 con 0x7ff8840b1db0 2021-11-21T22:01:28.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff88a373700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8840b1490 con 0x7ff884144810 2021-11-21T22:01:28.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff88a373700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8840af060 con 0x7ff8840abce0 2021-11-21T22:01:28.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff884144810 0x7ff8841478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff884144810 0x7ff8841478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50840/0 (socket says 172.21.2.116:50840) 2021-11-21T22:01:28.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 -- 172.21.2.116:0/1001117604 learned_addr learned my addr 172.21.2.116:0/1001117604 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 -- 172.21.2.116:0/1001117604 >> v2:172.21.2.117:3300/0 conn(0x7ff8840abce0 msgr2=0x7ff8840b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:28.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff882ffd700 1 -- 172.21.2.116:0/1001117604 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396658918 0 0) 0x7ff884147e80 con 0x7ff8840b1db0 2021-11-21T22:01:28.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 --2- 172.21.2.116:0/1001117604 >> v2:172.21.2.117:3300/0 conn(0x7ff8840abce0 0x7ff8840b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 -- 172.21.2.116:0/1001117604 >> v1:172.21.2.116:6789/0 conn(0x7ff8840b1db0 legacy=0x7ff8840ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 -- 172.21.2.116:0/1001117604 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840af5a0 con 0x7ff884144810 2021-11-21T22:01:28.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.474+0000 7ff883fff700 1 --2- 172.21.2.116:0/1001117604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff884144810 0x7ff8841478c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7ff86c002f70 tx=0x7ff86c006eb0).ready entity=mon.0 client_cookie=bbdb9a852d8fc8fa server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff882ffd700 1 -- 172.21.2.116:0/1001117604 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86c00d610 con 0x7ff884144810 2021-11-21T22:01:28.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff882ffd700 1 -- 172.21.2.116:0/1001117604 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c000e90 con 0x7ff884144810 2021-11-21T22:01:28.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff882ffd700 1 -- 172.21.2.116:0/1001117604 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86c00d9a0 con 0x7ff884144810 2021-11-21T22:01:28.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1001117604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff884144810 msgr2=0x7ff8841478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 --2- 172.21.2.116:0/1001117604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff884144810 0x7ff8841478c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7ff86c002f70 tx=0x7ff86c006eb0).stop 2021-11-21T22:01:28.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1001117604 shutdown_connections 2021-11-21T22:01:28.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 --2- 172.21.2.116:0/1001117604 >> v2:172.21.2.117:3300/0 conn(0x7ff8840abce0 0x7ff8840b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 --2- 172.21.2.116:0/1001117604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff884144810 0x7ff8841478c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1001117604 >> 172.21.2.116:0/1001117604 conn(0x7ff88405bd20 msgr2=0x7ff88405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1001117604 shutdown_connections 2021-11-21T22:01:28.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1001117604 wait complete. 2021-11-21T22:01:28.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 Processor -- start 2021-11-21T22:01:28.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- start start 2021-11-21T22:01:28.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8840abce0 0x7ff884156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff884144810 0x7ff884145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff884062f60 0x7ff884065c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b44a0 con 0x7ff8840abce0 2021-11-21T22:01:28.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840639e0 con 0x7ff884062f60 2021-11-21T22:01:28.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b46d0 con 0x7ff884144810 2021-11-21T22:01:28.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8840abce0 0x7ff884156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8840abce0 0x7ff884156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50850/0 (socket says 172.21.2.116:50850) 2021-11-21T22:01:28.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 -- 172.21.2.116:0/1938470141 learned_addr learned my addr 172.21.2.116:0/1938470141 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 -- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff884144810 msgr2=0x7ff884145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff884144810 0x7ff884145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 -- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff884062f60 msgr2=0x7ff884065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff884062f60 0x7ff884065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 -- 172.21.2.116:0/1938470141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c00b1e0 con 0x7ff8840abce0 2021-11-21T22:01:28.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff883fff700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8840abce0 0x7ff884156700 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7ff86c002f70 tx=0x7ff86c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff8817fa700 1 -- 172.21.2.116:0/1938470141 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86c006b60 con 0x7ff8840abce0 2021-11-21T22:01:28.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff8817fa700 1 -- 172.21.2.116:0/1938470141 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c00d290 con 0x7ff8840abce0 2021-11-21T22:01:28.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff8817fa700 1 -- 172.21.2.116:0/1938470141 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86c01e460 con 0x7ff8840abce0 2021-11-21T22:01:28.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8840634a0 con 0x7ff8840abce0 2021-11-21T22:01:28.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884146b00 con 0x7ff8840abce0 2021-11-21T22:01:28.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.478+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8840af300 con 0x7ff8840abce0 2021-11-21T22:01:28.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.482+0000 7ff8817fa700 1 -- 172.21.2.116:0/1938470141 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff86c025030 con 0x7ff8840abce0 2021-11-21T22:01:28.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.482+0000 7ff8817fa700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8680536f0 0x7ff868055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.482+0000 7ff8817fa700 1 -- 172.21.2.116:0/1938470141 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff86c06cdd0 con 0x7ff8840abce0 2021-11-21T22:01:28.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.482+0000 7ff8837fe700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8680536f0 0x7ff868055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.482+0000 7ff8837fe700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8680536f0 0x7ff868055bb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff874002f70 tx=0x7ff874006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.482+0000 7ff8817fa700 1 -- 172.21.2.116:0/1938470141 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff86c070030 con 0x7ff8840abce0 2021-11-21T22:01:28.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.566+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff884067cc0 con 0x7ff8840abce0 2021-11-21T22:01:28.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.566+0000 7ff8817fa700 1 -- 172.21.2.116:0/1938470141 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v100) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff86c0230b0 con 0x7ff8840abce0 2021-11-21T22:01:28.572 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:28.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8680536f0 msgr2=0x7ff868055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8680536f0 0x7ff868055bb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff874002f70 tx=0x7ff874006a80).stop 2021-11-21T22:01:28.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8840abce0 msgr2=0x7ff884156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8840abce0 0x7ff884156700 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7ff86c002f70 tx=0x7ff86c015000).stop 2021-11-21T22:01:28.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 shutdown_connections 2021-11-21T22:01:28.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff884062f60 0x7ff884065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff884144810 0x7ff884145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff8680536f0 0x7ff868055bb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 --2- 172.21.2.116:0/1938470141 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff8840abce0 0x7ff884156700 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 >> 172.21.2.116:0/1938470141 conn(0x7ff88405bd20 msgr2=0x7ff8841407c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 shutdown_connections 2021-11-21T22:01:28.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.570+0000 7ff88a373700 1 -- 172.21.2.116:0/1938470141 wait complete. 2021-11-21T22:01:28.579 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:28.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f76163db700 1 Processor -- start 2021-11-21T22:01:28.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f76163db700 1 -- start start 2021-11-21T22:01:28.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f76163db700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 0x7f76101478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f76163db700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f76100ac600 0x7f76100aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f76163db700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7610147e80 con 0x7f76100abce0 2021-11-21T22:01:28.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f76163db700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76100b1490 con 0x7f76100ac600 2021-11-21T22:01:28.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f76163db700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76100b1fd0 con 0x7f7610144810 2021-11-21T22:01:28.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f7614978700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f76100ac600 0x7f76100aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f760ffff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 0x7f76101478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f7614978700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f76100ac600 0x7f76100aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:52026/0 (socket says 172.21.2.116:52026) 2021-11-21T22:01:28.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f7614978700 1 -- 172.21.2.116:0/3420187905 learned_addr learned my addr 172.21.2.116:0/3420187905 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f760f7fe700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f76100abce0 0x7f76100b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:33490/0 (socket says 172.21.2.116:33490) 2021-11-21T22:01:28.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f7614978700 1 -- 172.21.2.116:0/3420187905 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 msgr2=0x7f76101478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:28.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f7614978700 1 --2- 172.21.2.116:0/3420187905 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 0x7f76101478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f7614978700 1 -- 172.21.2.116:0/3420187905 >> v1:172.21.2.117:6789/0 conn(0x7f76100abce0 legacy=0x7f76100b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.630+0000 7f7614978700 1 -- 172.21.2.116:0/3420187905 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76100af5a0 con 0x7f76100ac600 2021-11-21T22:01:28.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f7614978700 1 --2- 172.21.2.116:0/3420187905 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100ac600 0x7f76100aca90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f76080032f0 tx=0x7f7608009290).ready entity=mon.1 client_cookie=f6a26b1fe7c8265f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760effd700 1 -- 172.21.2.116:0/3420187905 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f760800b8a0 con 0x7f76100ac600 2021-11-21T22:01:28.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760effd700 1 -- 172.21.2.116:0/3420187905 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760800ba00 con 0x7f76100ac600 2021-11-21T22:01:28.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760effd700 1 -- 172.21.2.116:0/3420187905 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f760800bb70 con 0x7f76100ac600 2021-11-21T22:01:28.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- 172.21.2.116:0/3420187905 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100ac600 msgr2=0x7f76100aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 --2- 172.21.2.116:0/3420187905 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100ac600 0x7f76100aca90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f76080032f0 tx=0x7f7608009290).stop 2021-11-21T22:01:28.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- 172.21.2.116:0/3420187905 shutdown_connections 2021-11-21T22:01:28.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 --2- 172.21.2.116:0/3420187905 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100ac600 0x7f76100aca90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 --2- 172.21.2.116:0/3420187905 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 0x7f76101478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- 172.21.2.116:0/3420187905 >> 172.21.2.116:0/3420187905 conn(0x7f761005bd20 msgr2=0x7f761005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- 172.21.2.116:0/3420187905 shutdown_connections 2021-11-21T22:01:28.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- 172.21.2.116:0/3420187905 wait complete. 2021-11-21T22:01:28.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 Processor -- start 2021-11-21T22:01:28.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- start start 2021-11-21T22:01:28.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100abce0 0x7f761013c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76100ac600 0x7f7610146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 0x7f7610159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76100b2e30 con 0x7f76100ac600 2021-11-21T22:01:28.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7610143290 con 0x7f76100abce0 2021-11-21T22:01:28.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7610143530 con 0x7f7610144810 2021-11-21T22:01:28.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76100ac600 0x7f7610146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76100ac600 0x7f7610146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50862/0 (socket says 172.21.2.116:50862) 2021-11-21T22:01:28.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 -- 172.21.2.116:0/2402225345 learned_addr learned my addr 172.21.2.116:0/2402225345 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 -- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 msgr2=0x7f7610159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 0x7f7610159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 -- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100abce0 msgr2=0x7f761013c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100abce0 0x7f761013c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 -- 172.21.2.116:0/2402225345 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7608009530 con 0x7f76100ac600 2021-11-21T22:01:28.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760f7fe700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76100ac600 0x7f7610146770 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f76100b2a30 tx=0x7f7600007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760d7fa700 1 -- 172.21.2.116:0/2402225345 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f760000ce30 con 0x7f76100ac600 2021-11-21T22:01:28.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760d7fa700 1 -- 172.21.2.116:0/2402225345 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76000072e0 con 0x7f76100ac600 2021-11-21T22:01:28.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76101437d0 con 0x7f76100ac600 2021-11-21T22:01:28.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f760d7fa700 1 -- 172.21.2.116:0/2402225345 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7600007450 con 0x7f76100ac600 2021-11-21T22:01:28.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.634+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76100b3190 con 0x7f76100ac600 2021-11-21T22:01:28.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.638+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76100b3410 con 0x7f76100ac600 2021-11-21T22:01:28.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.638+0000 7f760d7fa700 1 -- 172.21.2.116:0/2402225345 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7600010070 con 0x7f76100ac600 2021-11-21T22:01:28.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.638+0000 7f760d7fa700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75f4053540 0x7f75f4055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.638+0000 7f760d7fa700 1 -- 172.21.2.116:0/2402225345 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f760006b0a0 con 0x7f76100ac600 2021-11-21T22:01:28.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.638+0000 7f760ffff700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75f4053540 0x7f75f4055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.638+0000 7f760ffff700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75f4053540 0x7f75f4055a00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f75f800c000 tx=0x7f75f8006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.638+0000 7f760d7fa700 1 -- 172.21.2.116:0/2402225345 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76100b3410 con 0x7f76100ac600 2021-11-21T22:01:28.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f76100b3410 con 0x7f76100ac600 2021-11-21T22:01:28.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f760d7fa700 1 -- 172.21.2.116:0/2402225345 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7f76100b3410 con 0x7f76100ac600 2021-11-21T22:01:28.725 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:28.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75f4053540 msgr2=0x7f75f4055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75f4053540 0x7f75f4055a00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f75f800c000 tx=0x7f75f8006df0).stop 2021-11-21T22:01:28.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76100ac600 msgr2=0x7f7610146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76100ac600 0x7f7610146770 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f76100b2a30 tx=0x7f7600007c40).stop 2021-11-21T22:01:28.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 shutdown_connections 2021-11-21T22:01:28.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7610144810 0x7f7610159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75f4053540 0x7f75f4055a00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f76100ac600 0x7f7610146770 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 --2- 172.21.2.116:0/2402225345 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f76100abce0 0x7f761013c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 >> 172.21.2.116:0/2402225345 conn(0x7f761005bd20 msgr2=0x7f7610148280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 shutdown_connections 2021-11-21T22:01:28.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.722+0000 7f76163db700 1 -- 172.21.2.116:0/2402225345 wait complete. 2021-11-21T22:01:28.733 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:01:28.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 Processor -- start 2021-11-21T22:01:28.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- start start 2021-11-21T22:01:28.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d4144810 0x7fa7d41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa7d40abce0 0x7fa7d40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d4147e80 con 0x7fa7d40b1db0 2021-11-21T22:01:28.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d40b1490 con 0x7fa7d40abce0 2021-11-21T22:01:28.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40af060 con 0x7fa7d4144810 2021-11-21T22:01:28.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d9da5700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa7d40b1db0 0x7fa7d40ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34592/0 (socket says 172.21.2.116:34592) 2021-11-21T22:01:28.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d9da5700 1 -- 172.21.2.116:0/1544499696 learned_addr learned my addr 172.21.2.116:0/1544499696 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:28.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d8da3700 1 --2- 172.21.2.116:0/1544499696 >> v2:172.21.2.116:3300/0 conn(0x7fa7d40abce0 0x7fa7d40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d8da3700 1 -- 172.21.2.116:0/1544499696 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d4144810 msgr2=0x7fa7d41478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:28.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d8da3700 1 --2- 172.21.2.116:0/1544499696 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d4144810 0x7fa7d41478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d8da3700 1 -- 172.21.2.116:0/1544499696 >> v1:172.21.2.116:6789/0 conn(0x7fa7d40b1db0 legacy=0x7fa7d40ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d8da3700 1 -- 172.21.2.116:0/1544499696 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40af5a0 con 0x7fa7d40abce0 2021-11-21T22:01:28.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7d8da3700 1 --2- 172.21.2.116:0/1544499696 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d40abce0 0x7fa7d40b18c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7fa7d0002f70 tx=0x7fa7d0006df0).ready entity=mon.0 client_cookie=fb744f87345527bc server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7c7fff700 1 -- 172.21.2.116:0/1544499696 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d000d570 con 0x7fa7d40abce0 2021-11-21T22:01:28.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7c7fff700 1 -- 172.21.2.116:0/1544499696 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d0000e90 con 0x7fa7d40abce0 2021-11-21T22:01:28.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7c7fff700 1 -- 172.21.2.116:0/1544499696 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d000d900 con 0x7fa7d40abce0 2021-11-21T22:01:28.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- 172.21.2.116:0/1544499696 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d40abce0 msgr2=0x7fa7d40b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 --2- 172.21.2.116:0/1544499696 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d40abce0 0x7fa7d40b18c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7fa7d0002f70 tx=0x7fa7d0006df0).stop 2021-11-21T22:01:28.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- 172.21.2.116:0/1544499696 shutdown_connections 2021-11-21T22:01:28.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 --2- 172.21.2.116:0/1544499696 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d40abce0 0x7fa7d40b18c0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 --2- 172.21.2.116:0/1544499696 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d4144810 0x7fa7d41478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- 172.21.2.116:0/1544499696 >> 172.21.2.116:0/1544499696 conn(0x7fa7d405bd20 msgr2=0x7fa7d405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- 172.21.2.116:0/1544499696 shutdown_connections 2021-11-21T22:01:28.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.782+0000 7fa7db808700 1 -- 172.21.2.116:0/1544499696 wait complete. 2021-11-21T22:01:28.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 Processor -- start 2021-11-21T22:01:28.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 -- start start 2021-11-21T22:01:28.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d40abce0 0x7fa7d413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.795 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d4144810 0x7fa7d41573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7d415a820 0x7fa7d413dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4156a40 con 0x7fa7d4144810 2021-11-21T22:01:28.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d415acc0 con 0x7fa7d415a820 2021-11-21T22:01:28.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4156ca0 con 0x7fa7d40abce0 2021-11-21T22:01:28.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d4144810 0x7fa7d41573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d4144810 0x7fa7d41573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50876/0 (socket says 172.21.2.116:50876) 2021-11-21T22:01:28.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 -- 172.21.2.116:0/342297955 learned_addr learned my addr 172.21.2.116:0/342297955 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 -- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d40abce0 msgr2=0x7fa7d413bea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:28.797 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d40abce0 0x7fa7d413bea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 -- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7d415a820 msgr2=0x7fa7d413dec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:28.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7d415a820 0x7fa7d413dec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 -- 172.21.2.116:0/342297955 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d000b180 con 0x7fa7d4144810 2021-11-21T22:01:28.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d8da3700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d4144810 0x7fa7d41573e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fa7d0002f70 tx=0x7fa7d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7c67fc700 1 -- 172.21.2.116:0/342297955 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d0011a40 con 0x7fa7d4144810 2021-11-21T22:01:28.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7c67fc700 1 -- 172.21.2.116:0/342297955 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d0011ba0 con 0x7fa7d4144810 2021-11-21T22:01:28.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7c67fc700 1 -- 172.21.2.116:0/342297955 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d001e460 con 0x7fa7d4144810 2021-11-21T22:01:28.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d4156f40 con 0x7fa7d4144810 2021-11-21T22:01:28.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d413e3b0 con 0x7fa7d4144810 2021-11-21T22:01:28.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d413ca60 con 0x7fa7d4144810 2021-11-21T22:01:28.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7c67fc700 1 -- 172.21.2.116:0/342297955 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa7d001a040 con 0x7fa7d4144810 2021-11-21T22:01:28.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7c67fc700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7b4053640 0x7fa7b4055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7c67fc700 1 -- 172.21.2.116:0/342297955 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa7d006c940 con 0x7fa7d4144810 2021-11-21T22:01:28.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.786+0000 7fa7d95a4700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7b4053640 0x7fa7b4055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.790+0000 7fa7d95a4700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7b4053640 0x7fa7b4055b00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fa7c8002ef0 tx=0x7fa7c8006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.790+0000 7fa7c67fc700 1 -- 172.21.2.116:0/342297955 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7d413ca60 con 0x7fa7d4144810 2021-11-21T22:01:28.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.870+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7d413ca60 con 0x7fa7d4144810 2021-11-21T22:01:28.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.870+0000 7fa7c67fc700 1 -- 172.21.2.116:0/342297955 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa7d413ca60 con 0x7fa7d4144810 2021-11-21T22:01:28.877 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:28.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7b4053640 msgr2=0x7fa7b4055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7b4053640 0x7fa7b4055b00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fa7c8002ef0 tx=0x7fa7c8006890).stop 2021-11-21T22:01:28.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d4144810 msgr2=0x7fa7d41573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d4144810 0x7fa7d41573e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fa7d0002f70 tx=0x7fa7d0015000).stop 2021-11-21T22:01:28.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 shutdown_connections 2021-11-21T22:01:28.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7d415a820 0x7fa7d413dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa7b4053640 0x7fa7b4055b00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7d4144810 0x7fa7d41573e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 --2- 172.21.2.116:0/342297955 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa7d40abce0 0x7fa7d413bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 >> 172.21.2.116:0/342297955 conn(0x7fa7d405bd20 msgr2=0x7fa7d414ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 shutdown_connections 2021-11-21T22:01:28.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.874+0000 7fa7db808700 1 -- 172.21.2.116:0/342297955 wait complete. 2021-11-21T22:01:28.884 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:01:28.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.930+0000 7f93a3669700 1 Processor -- start 2021-11-21T22:01:28.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.930+0000 7f93a3669700 1 -- start start 2021-11-21T22:01:28.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.930+0000 7f93a3669700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f939c0abce0 0x7f939c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.930+0000 7f93a3669700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c0ac600 0x7f939c0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.930+0000 7f93a3669700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c147e80 con 0x7f939c144810 2021-11-21T22:01:28.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.930+0000 7f93a3669700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c0b1490 con 0x7f939c0abce0 2021-11-21T22:01:28.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.930+0000 7f93a3669700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0b1fd0 con 0x7f939c0ac600 2021-11-21T22:01:28.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a0c04700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f939c0abce0 0x7f939c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1405700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f939c144810 0x7f939c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:33516/0 (socket says 172.21.2.116:33516) 2021-11-21T22:01:28.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1c06700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c0ac600 0x7f939c0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1405700 1 -- 172.21.2.116:0/384775920 learned_addr learned my addr 172.21.2.116:0/384775920 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:28.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1c06700 1 -- 172.21.2.116:0/384775920 >> v2:172.21.2.117:3300/0 conn(0x7f939c0abce0 msgr2=0x7f939c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:28.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1c06700 1 --2- 172.21.2.116:0/384775920 >> v2:172.21.2.117:3300/0 conn(0x7f939c0abce0 0x7f939c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1c06700 1 -- 172.21.2.116:0/384775920 >> v1:172.21.2.117:6789/0 conn(0x7f939c144810 legacy=0x7f939c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1c06700 1 -- 172.21.2.116:0/384775920 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c0af5a0 con 0x7f939c0ac600 2021-11-21T22:01:28.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a1c06700 1 --2- 172.21.2.116:0/384775920 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c0ac600 0x7f939c0aca90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f93940032f0 tx=0x7f93940091d0).ready entity=mon.2 client_cookie=f12462b6530bb5be server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f938ffff700 1 -- 172.21.2.116:0/384775920 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939400b6c0 con 0x7f939c0ac600 2021-11-21T22:01:28.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f938ffff700 1 -- 172.21.2.116:0/384775920 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939400b820 con 0x7f939c0ac600 2021-11-21T22:01:28.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f938ffff700 1 -- 172.21.2.116:0/384775920 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939400b990 con 0x7f939c0ac600 2021-11-21T22:01:28.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 -- 172.21.2.116:0/384775920 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c0ac600 msgr2=0x7f939c0aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 --2- 172.21.2.116:0/384775920 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c0ac600 0x7f939c0aca90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f93940032f0 tx=0x7f93940091d0).stop 2021-11-21T22:01:28.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 -- 172.21.2.116:0/384775920 shutdown_connections 2021-11-21T22:01:28.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 --2- 172.21.2.116:0/384775920 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c0ac600 0x7f939c0aca90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 --2- 172.21.2.116:0/384775920 >> v2:172.21.2.117:3300/0 conn(0x7f939c0abce0 0x7f939c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:28.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 -- 172.21.2.116:0/384775920 >> 172.21.2.116:0/384775920 conn(0x7f939c05bd20 msgr2=0x7f939c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:28.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 -- 172.21.2.116:0/384775920 shutdown_connections 2021-11-21T22:01:28.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.934+0000 7f93a3669700 1 -- 172.21.2.116:0/384775920 wait complete. 2021-11-21T22:01:28.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 Processor -- start 2021-11-21T22:01:28.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 -- start start 2021-11-21T22:01:28.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f939c0abce0 0x7f939c13e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f939c0ac600 0x7f939c156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c144810 0x7f939c156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c145a40 con 0x7f939c0ac600 2021-11-21T22:01:28.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c15ac60 con 0x7f939c0abce0 2021-11-21T22:01:28.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c145ca0 con 0x7f939c144810 2021-11-21T22:01:28.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f939c0ac600 0x7f939c156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f939c0ac600 0x7f939c156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50890/0 (socket says 172.21.2.116:50890) 2021-11-21T22:01:28.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 -- 172.21.2.116:0/3062345657 learned_addr learned my addr 172.21.2.116:0/3062345657 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:28.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 -- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c144810 msgr2=0x7f939c156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c144810 0x7f939c156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 -- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f939c0abce0 msgr2=0x7f939c13e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:28.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f939c0abce0 0x7f939c13e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:28.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 -- 172.21.2.116:0/3062345657 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9394009470 con 0x7f939c0ac600 2021-11-21T22:01:28.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a0c04700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f939c0ac600 0x7f939c156860 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f939000c000 tx=0x7f9390006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f938e7fc700 1 -- 172.21.2.116:0/3062345657 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9390010590 con 0x7f939c0ac600 2021-11-21T22:01:28.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f938e7fc700 1 -- 172.21.2.116:0/3062345657 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93900106f0 con 0x7f939c0ac600 2021-11-21T22:01:28.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c145f40 con 0x7f939c0ac600 2021-11-21T22:01:28.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f938e7fc700 1 -- 172.21.2.116:0/3062345657 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9390010a50 con 0x7f939c0ac600 2021-11-21T22:01:28.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c157240 con 0x7f939c0ac600 2021-11-21T22:01:28.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c1460d0 con 0x7f939c0ac600 2021-11-21T22:01:28.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f938e7fc700 1 -- 172.21.2.116:0/3062345657 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9390010bb0 con 0x7f939c0ac600 2021-11-21T22:01:28.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f938e7fc700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f937c053660 0x7f937c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:28.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f938e7fc700 1 -- 172.21.2.116:0/3062345657 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f939006ba30 con 0x7f939c0ac600 2021-11-21T22:01:28.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.938+0000 7f93a1405700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f937c053660 0x7f937c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:28.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.942+0000 7f93a1405700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f937c053660 0x7f937c055b20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f939c062500 tx=0x7f9398007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:28.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:28.942+0000 7f938e7fc700 1 -- 172.21.2.116:0/3062345657 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9390042b30 con 0x7f939c0ac600 2021-11-21T22:01:29.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.022+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f939c0af060 con 0x7f939c0ac600 2021-11-21T22:01:29.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f938e7fc700 1 -- 172.21.2.116:0/3062345657 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9390046150 con 0x7f939c0ac600 2021-11-21T22:01:29.029 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:29.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f937c053660 msgr2=0x7f937c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f937c053660 0x7f937c055b20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f939c062500 tx=0x7f9398007820).stop 2021-11-21T22:01:29.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f939c0ac600 msgr2=0x7f939c156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f939c0ac600 0x7f939c156860 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f939000c000 tx=0x7f9390006df0).stop 2021-11-21T22:01:29.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 shutdown_connections 2021-11-21T22:01:29.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f939c144810 0x7f939c156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:29.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f937c053660 0x7f937c055b20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:29.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f939c0ac600 0x7f939c156860 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:29.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 --2- 172.21.2.116:0/3062345657 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f939c0abce0 0x7f939c13e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:29.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 >> 172.21.2.116:0/3062345657 conn(0x7f939c05bd20 msgr2=0x7f939c0ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:29.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 shutdown_connections 2021-11-21T22:01:29.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.026+0000 7f93a3669700 1 -- 172.21.2.116:0/3062345657 wait complete. 2021-11-21T22:01:29.036 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:01:29.037 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:29.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.082+0000 7f4884e8f700 1 Processor -- start 2021-11-21T22:01:29.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.082+0000 7f4884e8f700 1 -- start start 2021-11-21T22:01:29.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.082+0000 7f4884e8f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f48800b5660 0x7f488013bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:29.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.082+0000 7f4884e8f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48800b5a40 con 0x7f48800b1660 2021-11-21T22:01:29.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48800ae540 con 0x7f48800aff60 2021-11-21T22:01:29.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800aef20 con 0x7f48800b5660 2021-11-21T22:01:29.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487e59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f48800b5660 0x7f488013bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:29.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487e59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f48800b5660 0x7f488013bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50898/0 (socket says 172.21.2.116:50898) 2021-11-21T22:01:29.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487e59c700 1 -- 172.21.2.116:0/1728982173 learned_addr learned my addr 172.21.2.116:0/1728982173 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:29.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487e59c700 1 -- 172.21.2.116:0/1728982173 >> v1:172.21.2.117:6789/0 conn(0x7f48800aff60 legacy=0x7f48801495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:29.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487e59c700 1 -- 172.21.2.116:0/1728982173 >> v1:172.21.2.116:6789/0 conn(0x7f48800b1660 legacy=0x7f48800b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487e59c700 1 -- 172.21.2.116:0/1728982173 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800ae7e0 con 0x7f48800b5660 2021-11-21T22:01:29.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487e59c700 1 --2- 172.21.2.116:0/1728982173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b5660 0x7f488013bf10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f4880063a90 tx=0x7f4870006eb0).ready entity=mon.0 client_cookie=f5a08905a6893a2d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:29.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487d59a700 1 -- 172.21.2.116:0/1728982173 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487000d610 con 0x7f48800b5660 2021-11-21T22:01:29.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487d59a700 1 -- 172.21.2.116:0/1728982173 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870000e90 con 0x7f48800b5660 2021-11-21T22:01:29.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487d59a700 1 -- 172.21.2.116:0/1728982173 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487000d960 con 0x7f48800b5660 2021-11-21T22:01:29.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- 172.21.2.116:0/1728982173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b5660 msgr2=0x7f488013bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 --2- 172.21.2.116:0/1728982173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b5660 0x7f488013bf10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f4880063a90 tx=0x7f4870006eb0).stop 2021-11-21T22:01:29.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- 172.21.2.116:0/1728982173 shutdown_connections 2021-11-21T22:01:29.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 --2- 172.21.2.116:0/1728982173 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b5660 0x7f488013bf10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:29.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- 172.21.2.116:0/1728982173 >> 172.21.2.116:0/1728982173 conn(0x7f488005f730 msgr2=0x7f4880063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:29.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- 172.21.2.116:0/1728982173 shutdown_connections 2021-11-21T22:01:29.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- 172.21.2.116:0/1728982173 wait complete. 2021-11-21T22:01:29.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 Processor -- start 2021-11-21T22:01:29.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- start start 2021-11-21T22:01:29.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48800aff60 0x7f4880140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:29.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b1660 0x7f48800b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:29.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48800b5660 0x7f48800b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:29.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48800b6240 con 0x7f48800b1660 2021-11-21T22:01:29.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488006aa70 con 0x7f48800b5660 2021-11-21T22:01:29.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48800aa070 con 0x7f48800aff60 2021-11-21T22:01:29.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b1660 0x7f48800b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:29.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b1660 0x7f48800b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50906/0 (socket says 172.21.2.116:50906) 2021-11-21T22:01:29.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 -- 172.21.2.116:0/2932392676 learned_addr learned my addr 172.21.2.116:0/2932392676 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:29.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 -- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48800aff60 msgr2=0x7f4880140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48800aff60 0x7f4880140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:29.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 -- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48800b5660 msgr2=0x7f48800b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48800b5660 0x7f48800b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:29.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 -- 172.21.2.116:0/2932392676 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487000b1e0 con 0x7f48800b1660 2021-11-21T22:01:29.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f487dd9b700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b1660 0x7f48800b39a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f48800a9230 tx=0x7f4868007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:29.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f48777fe700 1 -- 172.21.2.116:0/2932392676 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48680108a0 con 0x7f48800b1660 2021-11-21T22:01:29.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48800a9c40 con 0x7f48800b1660 2021-11-21T22:01:29.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f48777fe700 1 -- 172.21.2.116:0/2932392676 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4868010a00 con 0x7f48800b1660 2021-11-21T22:01:29.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f48777fe700 1 -- 172.21.2.116:0/2932392676 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4868010d60 con 0x7f48800b1660 2021-11-21T22:01:29.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.086+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48800b4f90 con 0x7f48800b1660 2021-11-21T22:01:29.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.090+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48800aea80 con 0x7f48800b1660 2021-11-21T22:01:29.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.090+0000 7f48777fe700 1 -- 172.21.2.116:0/2932392676 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4868021070 con 0x7f48800b1660 2021-11-21T22:01:29.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.090+0000 7f48777fe700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4864053610 0x7f4864055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:29.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.090+0000 7f48777fe700 1 -- 172.21.2.116:0/2932392676 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f486806de70 con 0x7f48800b1660 2021-11-21T22:01:29.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.090+0000 7f487e59c700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4864053610 0x7f4864055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:29.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.090+0000 7f487e59c700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4864053610 0x7f4864055ad0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f4880063a90 tx=0x7f4870015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:29.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.090+0000 7f48777fe700 1 -- 172.21.2.116:0/2932392676 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4868044ed0 con 0x7f48800b1660 2021-11-21T22:01:29.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.198+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48800573d0 con 0x7f48800b1660 2021-11-21T22:01:29.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f48777fe700 1 -- 172.21.2.116:0/2932392676 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f486801f020 con 0x7f48800b1660 2021-11-21T22:01:29.205 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:29.205 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:23.183200+0000","modified":"2021-11-21T22:01:24.187992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3417150056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3417150056},{"type":"v1","addr":"172.21.2.116:6835","nonce":3417150056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-11-21T22:01:29.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4864053610 msgr2=0x7f4864055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4864053610 0x7f4864055ad0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f4880063a90 tx=0x7f4870015000).stop 2021-11-21T22:01:29.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b1660 msgr2=0x7f48800b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:29.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b1660 0x7f48800b39a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f48800a9230 tx=0x7f4868007ce0).stop 2021-11-21T22:01:29.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 shutdown_connections 2021-11-21T22:01:29.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48800b5660 0x7f48800b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:29.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4864053610 0x7f4864055ad0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:29.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48800b1660 0x7f48800b39a0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:29.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 --2- 172.21.2.116:0/2932392676 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48800aff60 0x7f4880140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:29.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 >> 172.21.2.116:0/2932392676 conn(0x7f488005f730 msgr2=0x7f488013f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:29.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 shutdown_connections 2021-11-21T22:01:29.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:29.202+0000 7f4884e8f700 1 -- 172.21.2.116:0/2932392676 wait complete. 2021-11-21T22:01:29.209 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 44 2021-11-21T22:01:29.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:23.183200+0000', 'modified': '2021-11-21T22:01:24.187992+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/3417150056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 3417150056}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 3417150056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:29.516 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:29.516 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:29.582 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7011,"want_state":"up:active","state":"up:active","rank_uptime":6.3920821549999998,"mdsmap_epoch":44,"osdmap_epoch":100,"osdmap_epoch_barrier":99,"uptime":10.583913746} 2021-11-21T22:01:29.583 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7011, "mdsmap_epoch": 44, "osdmap_epoch": 100, "osdmap_epoch_barrier": 99, "rank_uptime": 6.392082155, "state": "up:active", "uptime": 10.583913746, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:29.583 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:01:29.584 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:01:29.584 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:01:29.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:29.585 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:29.596 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:29.651 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:29.652 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:29.658 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:29.659 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:29.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:29.660 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:29.708 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:01:29.709 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:29.711 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:29.721 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:29.722 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:29.727 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:29.728 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:29.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:29.729 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:29.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:01:29.749+0000 7f8cb8fbc080 -1 init, newargv = 0x55c6671f0c20 newargc=9 2021-11-21T22:01:29.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14351]: starting ceph client 2021-11-21T22:01:29.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14351]: starting fuse 2021-11-21T22:01:30.856 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:30.868 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:30.868 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:30.918 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:30.919 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:30.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:30.921 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:30.926 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:01:30.926 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:01:30.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:30.927 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:01:30.927 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:01:30.928 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:01:30.928 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:01:30.929 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:01:30.929 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:30.930 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:01:30.930 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:01:30.931 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:01:30.931 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:30.931 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:01:30.932 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:01:30.932 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:01:30.933 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:01:30.933 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:30.933 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:01:30.934 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:01:30.934 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:01:30.935 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:01:30.935 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:01:30.935 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:30.936 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:01:30.936 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:01:31.002 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14351.asok 2021-11-21T22:01:31.005 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14351.asok 2021-11-21T22:01:31.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:31.006 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14351.asok status 2021-11-21T22:01:31.053 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:01:31.054 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:01:31.054 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:01:31.054 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:01:31.055 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:01:31.055 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:01:31.055 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:01:31.055 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14351", 2021-11-21T22:01:31.056 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:01:31.056 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:31.056 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:01:31.056 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:01:31.056 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 7233, 2021-11-21T22:01:31.057 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:01:31.057 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:01:31.057 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:01:31.057 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 7233 2021-11-21T22:01:31.058 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:31.058 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:01:31.058 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:01:31.058 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:01:31.058 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1502885988 2021-11-21T22:01:31.059 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:01:31.059 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:31.059 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:01:31.059 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:01:31.060 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:01:31.060 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1502885988 2021-11-21T22:01:31.060 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:31.060 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.7233 172.21.2.136:0/1502885988", 2021-11-21T22:01:31.060 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/1502885988", 2021-11-21T22:01:31.061 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:01:31.061 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 44, 2021-11-21T22:01:31.061 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 100, 2021-11-21T22:01:31.061 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:01:31.061 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:01:31.062 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:01:31.062 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:01:31.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:31.064 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:31.064 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:31.070 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:01:31.070 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:31.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:31.071 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:31.120 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-11-21T22:01:31.121 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2021-11-21T22:01:31.153 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T22:01:31.154 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:01:31.155 DEBUG:teuthology.orchestra.run.gibba019:> import stat 2021-11-21T22:01:31.155 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:31.155 DEBUG:teuthology.orchestra.run.gibba019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2021-11-21T22:01:31.156 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:01:31.191 INFO:teuthology.orchestra.run.gibba019.stdout:1099511627776 2021-11-21T22:01:31.400 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:31.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.450+0000 7f4370f51700 1 Processor -- start 2021-11-21T22:01:31.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.450+0000 7f4370f51700 1 -- start start 2021-11-21T22:01:31.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a6aa0 0x7f436c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:31.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f436c062410 0x7f436c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:31.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c0a6e80 con 0x7f436c0a86b0 2021-11-21T22:01:31.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c0a8bc0 con 0x7f436c062410 2021-11-21T22:01:31.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c14a7b0 con 0x7f436c0a6aa0 2021-11-21T22:01:31.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f436ad9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f436c062410 0x7f436c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:31.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4361d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f436c0a86b0 0x7f436c1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34634/0 (socket says 172.21.2.116:34634) 2021-11-21T22:01:31.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4361d9b700 1 -- 172.21.2.116:0/1844071488 learned_addr learned my addr 172.21.2.116:0/1844071488 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:31.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f436ad9d700 1 -- 172.21.2.116:0/1844071488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a6aa0 msgr2=0x7f436c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:31.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f436ad9d700 1 --2- 172.21.2.116:0/1844071488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a6aa0 0x7f436c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:31.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f436ad9d700 1 -- 172.21.2.116:0/1844071488 >> v1:172.21.2.116:6789/0 conn(0x7f436c0a86b0 legacy=0x7f436c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:31.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f436ad9d700 1 -- 172.21.2.116:0/1844071488 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c14aa10 con 0x7f436c062410 2021-11-21T22:01:31.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f436ad9d700 1 --2- 172.21.2.116:0/1844071488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c062410 0x7f436c1443f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f43640032f0 tx=0x7f43640091d0).ready entity=mon.0 client_cookie=3921e5fddc7e3353 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:31.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4369d9b700 1 -- 172.21.2.116:0/1844071488 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4364009ea0 con 0x7f436c062410 2021-11-21T22:01:31.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4369d9b700 1 -- 172.21.2.116:0/1844071488 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436400b8a0 con 0x7f436c062410 2021-11-21T22:01:31.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4369d9b700 1 -- 172.21.2.116:0/1844071488 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436400ba10 con 0x7f436c062410 2021-11-21T22:01:31.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- 172.21.2.116:0/1844071488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c062410 msgr2=0x7f436c1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:31.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- 172.21.2.116:0/1844071488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c062410 0x7f436c1443f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f43640032f0 tx=0x7f43640091d0).stop 2021-11-21T22:01:31.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- 172.21.2.116:0/1844071488 shutdown_connections 2021-11-21T22:01:31.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- 172.21.2.116:0/1844071488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c062410 0x7f436c1443f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:31.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- 172.21.2.116:0/1844071488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a6aa0 0x7f436c14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:31.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- 172.21.2.116:0/1844071488 >> 172.21.2.116:0/1844071488 conn(0x7f436c05bd20 msgr2=0x7f436c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:31.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- 172.21.2.116:0/1844071488 shutdown_connections 2021-11-21T22:01:31.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- 172.21.2.116:0/1844071488 wait complete. 2021-11-21T22:01:31.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 Processor -- start 2021-11-21T22:01:31.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- start start 2021-11-21T22:01:31.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f436c062410 0x7f436c0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:31.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c0a6aa0 0x7f436c0a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:31.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a86b0 0x7f436c0ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:31.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c156f20 con 0x7f436c0a6aa0 2021-11-21T22:01:31.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0aa1b0 con 0x7f436c062410 2021-11-21T22:01:31.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4370f51700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c063bd0 con 0x7f436c0a86b0 2021-11-21T22:01:31.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4361d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c0a6aa0 0x7f436c0a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:31.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4361d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c0a6aa0 0x7f436c0a97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50918/0 (socket says 172.21.2.116:50918) 2021-11-21T22:01:31.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4361d9b700 1 -- 172.21.2.116:0/1136568196 learned_addr learned my addr 172.21.2.116:0/1136568196 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:31.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.454+0000 7f4361d9b700 1 -- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a86b0 msgr2=0x7f436c0ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:31.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4361d9b700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a86b0 0x7f436c0ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:31.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4361d9b700 1 -- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f436c062410 msgr2=0x7f436c0aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:31.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4361d9b700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f436c062410 0x7f436c0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:31.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4361d9b700 1 -- 172.21.2.116:0/1136568196 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364009470 con 0x7f436c0a6aa0 2021-11-21T22:01:31.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4361d9b700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c0a6aa0 0x7f436c0a97c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f436c061ab0 tx=0x7f4354007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:31.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4363fff700 1 -- 172.21.2.116:0/1136568196 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f435400ce30 con 0x7f436c0a6aa0 2021-11-21T22:01:31.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4363fff700 1 -- 172.21.2.116:0/1136568196 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540072e0 con 0x7f436c0a6aa0 2021-11-21T22:01:31.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4363fff700 1 -- 172.21.2.116:0/1136568196 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4354005800 con 0x7f436c0a6aa0 2021-11-21T22:01:31.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c0674c0 con 0x7f436c0a6aa0 2021-11-21T22:01:31.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c0a8d20 con 0x7f436c0a6aa0 2021-11-21T22:01:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c067320 con 0x7f436c0a6aa0 2021-11-21T22:01:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4363fff700 1 -- 172.21.2.116:0/1136568196 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4354010070 con 0x7f436c0a6aa0 2021-11-21T22:01:31.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4363fff700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43500536a0 0x7f4350055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:31.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f4363fff700 1 -- 172.21.2.116:0/1136568196 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f435406af30 con 0x7f436c0a6aa0 2021-11-21T22:01:31.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f436a59c700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43500536a0 0x7f4350055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:31.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.458+0000 7f436a59c700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43500536a0 0x7f4350055b60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f435c002f70 tx=0x7f435c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:31.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.462+0000 7f4363fff700 1 -- 172.21.2.116:0/1136568196 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f436c067320 con 0x7f436c0a6aa0 2021-11-21T22:01:31.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.566+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f436c067320 con 0x7f436c0a6aa0 2021-11-21T22:01:31.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.566+0000 7f4363fff700 1 -- 172.21.2.116:0/1136568196 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f436c067320 con 0x7f436c0a6aa0 2021-11-21T22:01:31.572 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:31.572 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:23.183200+0000","modified":"2021-11-21T22:01:24.187992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3417150056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3417150056},{"type":"v1","addr":"172.21.2.116:6835","nonce":3417150056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-11-21T22:01:31.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43500536a0 msgr2=0x7f4350055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:31.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43500536a0 0x7f4350055b60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f435c002f70 tx=0x7f435c006a80).stop 2021-11-21T22:01:31.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c0a6aa0 msgr2=0x7f436c0a97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:31.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c0a6aa0 0x7f436c0a97c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f436c061ab0 tx=0x7f4354007c10).stop 2021-11-21T22:01:31.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 shutdown_connections 2021-11-21T22:01:31.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f436c0a86b0 0x7f436c0ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:31.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f43500536a0 0x7f4350055b60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:31.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f436c0a6aa0 0x7f436c0a97c0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:31.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 --2- 172.21.2.116:0/1136568196 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f436c062410 0x7f436c0aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:31.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 >> 172.21.2.116:0/1136568196 conn(0x7f436c05bd20 msgr2=0x7f436c1441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:31.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 shutdown_connections 2021-11-21T22:01:31.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.570+0000 7f4370f51700 1 -- 172.21.2.116:0/1136568196 wait complete. 2021-11-21T22:01:31.577 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 44 2021-11-21T22:01:31.580 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump inode 0x10000000000 2021-11-21T22:01:31.639 INFO:teuthology.orchestra.run.gibba016.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2021-11-21T22:01:31.146103+0000","btime":"2021-11-21T22:01:31.146103+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2021-11-21T22:01:31.146103+0000","atime":"2021-11-21T22:01:31.146103+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":4987,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":4987,"want_loner":4987,"mds_caps_wanted":[]} 2021-11-21T22:01:31.640 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2021-11-21T22:01:31.146103+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2021-11-21T22:01:31.146103+0000", "change_attr": 0, "client_caps": [ { "client_id": 4987, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2021-11-21T22:01:31.146103+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 4987, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2021-11-21T22:01:31.146103+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 4987, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2021-11-21T22:01:31.641 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:01:31.642 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:31.650 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:31.664 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:31.679 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:31.693 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:01:31.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:31.695 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:31.696 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:31.703 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:01:31.703 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:31.704 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:01:31.705 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:01:31.706 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:31.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14867]: fuse finished with error 0 and tester_r 0 2021-11-21T22:01:31.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:31.715 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:31.716 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:31.780 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:01:31.781 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:01:31.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:01:31.782 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:31.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:31.787 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:31.788 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:31.799 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:01:31.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:31.801 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:01:31.801 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:01:31.802 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:31.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14351]: fuse finished with error 0 and tester_r 0 2021-11-21T22:01:31.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:31.818 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:31.819 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:31.823 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:01:31.824 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:01:31.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:01:31.935 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:31.939 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:01:31.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 Processor -- start 2021-11-21T22:01:31.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 -- start start 2021-11-21T22:01:32.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5c580a86b0 0x7f5c5814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5c58062410 0x7f5c581443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580a6e80 con 0x7f5c580a6aa0 2021-11-21T22:01:32.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c580a8bc0 con 0x7f5c580a86b0 2021-11-21T22:01:32.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5814a7b0 con 0x7f5c58062410 2021-11-21T22:01:32.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5c580a86b0 0x7f5c5814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5c580a86b0 0x7f5c5814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50928/0 (socket says 172.21.2.116:50928) 2021-11-21T22:01:32.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 -- 172.21.2.116:0/1848488608 learned_addr learned my addr 172.21.2.116:0/1848488608 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:32.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 -- 172.21.2.116:0/1848488608 >> v2:172.21.2.117:3300/0 conn(0x7f5c58062410 msgr2=0x7f5c581443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:32.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 --2- 172.21.2.116:0/1848488608 >> v2:172.21.2.117:3300/0 conn(0x7f5c58062410 0x7f5c581443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 -- 172.21.2.116:0/1848488608 >> v1:172.21.2.117:6789/0 conn(0x7f5c580a6aa0 legacy=0x7f5c581470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:32.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 -- 172.21.2.116:0/1848488608 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5814aa10 con 0x7f5c580a86b0 2021-11-21T22:01:32.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c56ffd700 1 --2- 172.21.2.116:0/1848488608 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c5814b150 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f5c58061ab0 tx=0x7f5c48006eb0).ready entity=mon.0 client_cookie=6a3ec357d41bf304 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c567fc700 1 -- 172.21.2.116:0/1848488608 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c4800d5c0 con 0x7f5c580a86b0 2021-11-21T22:01:32.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c567fc700 1 -- 172.21.2.116:0/1848488608 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48000e90 con 0x7f5c580a86b0 2021-11-21T22:01:32.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c567fc700 1 -- 172.21.2.116:0/1848488608 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c4800d910 con 0x7f5c580a86b0 2021-11-21T22:01:32.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 -- 172.21.2.116:0/1848488608 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 msgr2=0x7f5c5814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.990+0000 7f5c5e206700 1 --2- 172.21.2.116:0/1848488608 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c5814b150 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f5c58061ab0 tx=0x7f5c48006eb0).stop 2021-11-21T22:01:32.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- 172.21.2.116:0/1848488608 shutdown_connections 2021-11-21T22:01:32.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 --2- 172.21.2.116:0/1848488608 >> v2:172.21.2.117:3300/0 conn(0x7f5c58062410 0x7f5c581443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 --2- 172.21.2.116:0/1848488608 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c5814b150 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- 172.21.2.116:0/1848488608 >> 172.21.2.116:0/1848488608 conn(0x7f5c5805bd20 msgr2=0x7f5c5805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:32.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- 172.21.2.116:0/1848488608 shutdown_connections 2021-11-21T22:01:32.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- 172.21.2.116:0/1848488608 wait complete. 2021-11-21T22:01:32.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 Processor -- start 2021-11-21T22:01:32.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- start start 2021-11-21T22:01:32.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c58062410 0x7f5c58065080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c580a6aa0 0x7f5c58065570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c580aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58065a60 con 0x7f5c580a86b0 2021-11-21T22:01:32.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58157340 con 0x7f5c580a6aa0 2021-11-21T22:01:32.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581570a0 con 0x7f5c58062410 2021-11-21T22:01:32.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c580aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c580aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50934/0 (socket says 172.21.2.116:50934) 2021-11-21T22:01:32.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 -- 172.21.2.116:0/2050573099 learned_addr learned my addr 172.21.2.116:0/2050573099 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:32.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 -- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c58062410 msgr2=0x7f5c58065080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c58062410 0x7f5c58065080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 -- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c580a6aa0 msgr2=0x7f5c58065570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c580a6aa0 0x7f5c58065570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 -- 172.21.2.116:0/2050573099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4800b1e0 con 0x7f5c580a86b0 2021-11-21T22:01:32.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c57fff700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c580aab70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f5c50003330 tx=0x7f5c500091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c54ff9700 1 -- 172.21.2.116:0/2050573099 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c50001520 con 0x7f5c580a86b0 2021-11-21T22:01:32.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c54ff9700 1 -- 172.21.2.116:0/2050573099 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c50001680 con 0x7f5c580a86b0 2021-11-21T22:01:32.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c54ff9700 1 -- 172.21.2.116:0/2050573099 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c500019e0 con 0x7f5c580a86b0 2021-11-21T22:01:32.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c58156d90 con 0x7f5c580a86b0 2021-11-21T22:01:32.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c580aa030 con 0x7f5c580a86b0 2021-11-21T22:01:32.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c580573d0 con 0x7f5c580a86b0 2021-11-21T22:01:32.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c54ff9700 1 -- 172.21.2.116:0/2050573099 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5c50018030 con 0x7f5c580a86b0 2021-11-21T22:01:32.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c54ff9700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c3c053720 0x7f5c3c055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.994+0000 7f5c54ff9700 1 -- 172.21.2.116:0/2050573099 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5c5006b440 con 0x7f5c580a86b0 2021-11-21T22:01:32.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.998+0000 7f5c577fe700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c3c053720 0x7f5c3c055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.998+0000 7f5c577fe700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c3c053720 0x7f5c3c055be0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f5c40002fb0 tx=0x7f5c400069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:31.998+0000 7f5c54ff9700 1 -- 172.21.2.116:0/2050573099 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c50042540 con 0x7f5c580a86b0 2021-11-21T22:01:32.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5c5805db70 con 0x7f5c580a86b0 2021-11-21T22:01:32.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c54ff9700 1 -- 172.21.2.116:0/2050573099 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v100) v1 ==== 81+0+33651 (secure 0 0 0) 0x7f5c50045b60 con 0x7f5c580a86b0 2021-11-21T22:01:32.086 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:32.086 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:32.087 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 100, 2021-11-21T22:01:32.087 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:01:32.087 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:01:32.087 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:01:24.187518+0000", 2021-11-21T22:01:32.088 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:32.088 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:32.088 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:01:32.088 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:01:32.088 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:01:32.089 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:01:32.089 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:01:32.089 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:01:32.089 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:01:32.090 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:32.090 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:01:32.090 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:01:32.090 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:01:32.091 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:01:32.091 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:01:32.091 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 18, 2021-11-21T22:01:32.091 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:01:32.091 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:01:32.092 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:01:32.092 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:01:32.092 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:01:32.092 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.092 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:01:32.093 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:01:32.093 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:01:32.093 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:32.093 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:32.094 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:32.094 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:32.094 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:32.094 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:32.095 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:32.095 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:32.095 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:01:32.095 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:01:32.095 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:01:32.096 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:01:32.096 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:01:32.096 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:32.096 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:32.096 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:32.097 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:32.097 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:32.097 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:32.097 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:32.098 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.098 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:01:32.098 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:32.098 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:32.099 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:32.099 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:32.099 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:32.099 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:32.099 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:32.100 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:32.100 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:32.100 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:32.100 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:32.101 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:32.101 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:32.101 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:32.101 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:32.101 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:32.102 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:32.102 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:32.102 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:32.102 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:32.102 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:32.103 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:32.103 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:32.103 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:32.103 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:32.104 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:32.104 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.104 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:32.104 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:32.104 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:32.105 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:32.105 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:32.105 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:32.105 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:32.106 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:32.106 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:32.106 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:32.106 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:32.106 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:32.107 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:01:32.107 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.107 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:32.107 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:01:32.108 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.108 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.108 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.108 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:01:32.108 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:01:32.109 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:01:32.109 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:32.109 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:32.109 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:32.110 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:32.110 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:32.110 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:32.110 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:32.110 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:32.111 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:32.111 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:32.111 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:32.111 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:32.112 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:32.112 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:32.112 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:32.112 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:32.112 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:32.113 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:32.113 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:32.113 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:32.113 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.113 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:01:32.114 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:32.114 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:32.114 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:32.114 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:32.115 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:32.115 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:32.115 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:32.115 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:32.115 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:32.116 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:32.116 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:32.116 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:32.116 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:32.117 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:32.117 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:32.117 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:32.117 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:32.117 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:32.118 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:32.118 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:32.118 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:32.118 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:32.119 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:32.119 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:32.119 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:32.119 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:32.119 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.120 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:32.120 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:32.120 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:32.120 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:32.120 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:32.121 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:32.121 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:32.121 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:32.121 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:32.122 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:32.122 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:32.122 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:32.122 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:32.122 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:01:32.123 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.123 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.123 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.123 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 17, 2021-11-21T22:01:32.124 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:01:32.124 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:01:21.079308+0000", 2021-11-21T22:01:32.124 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:32.124 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:32.124 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:32.125 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:32.125 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:32.125 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:32.125 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:32.125 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:32.126 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:32.126 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:32.126 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:32.126 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:32.127 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:32.127 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:32.127 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:32.127 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:32.127 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:32.128 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:32.128 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:32.128 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:32.128 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.128 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "100", 2021-11-21T22:01:32.129 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:32.129 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:32.129 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:32.129 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:32.130 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:32.130 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:32.130 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:32.130 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:32.130 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:32.131 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:32.131 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:32.131 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:32.131 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:32.131 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:32.132 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:32.132 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:32.132 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:32.132 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:32.133 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:32.133 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:32.133 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:32.133 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:32.133 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:32.134 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:32.134 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:32.134 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:32.134 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.135 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:32.135 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:32.135 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:32.135 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:32.135 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:32.136 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:32.136 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:32.136 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:32.136 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:32.136 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:32.137 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:32.137 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:32.137 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:01:32.137 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:01:32.138 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:01:32.138 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.138 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:32.138 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:01:32.138 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:01:32.139 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.139 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.139 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.139 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.140 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 18, 2021-11-21T22:01:32.140 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:01:32.140 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:01:22.145654+0000", 2021-11-21T22:01:32.140 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:32.141 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:32.141 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:32.141 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:32.141 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:32.141 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:32.142 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:32.142 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:32.142 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:32.142 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:32.142 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:32.143 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:32.143 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:32.143 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:32.143 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:32.144 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:32.144 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:32.144 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:32.144 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:32.145 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:32.145 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.145 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "100", 2021-11-21T22:01:32.145 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:32.145 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:32.146 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:32.146 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:32.146 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:32.146 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:32.147 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:32.147 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:32.147 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:32.147 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:32.147 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:32.148 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:32.148 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:32.148 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:32.148 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:32.148 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:32.149 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:32.149 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:32.149 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:32.149 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:32.150 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:32.150 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:32.150 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:32.150 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:32.150 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:32.151 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:32.151 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.151 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:32.151 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:32.152 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:32.152 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:32.152 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:32.152 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:32.152 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:32.153 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:32.153 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:32.153 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:32.153 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:32.154 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:32.154 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:32.154 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:01:32.154 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:01:32.154 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.155 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.155 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.155 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:32.155 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:01:32.156 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.156 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:32.156 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:01:32.156 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.156 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.157 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.157 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.157 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.157 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.158 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.158 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:32.158 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.158 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.158 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.159 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.159 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.159 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.159 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:01:32.159 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.160 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.160 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.160 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.160 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:01:32.161 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.161 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.161 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.161 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.161 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.162 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.162 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.162 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.162 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:01:32.162 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.163 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.163 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.163 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.163 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:01:32.164 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.164 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.164 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.164 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.164 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.165 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.165 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.165 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.165 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:01:32.166 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.166 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.166 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.166 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.166 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:01:32.167 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.167 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.167 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.167 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.167 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.168 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.168 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.168 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.168 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:01:32.169 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.169 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.169 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.169 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.169 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:01:32.170 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:32.170 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.170 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.170 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.171 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:01:32.171 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:01:32.171 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:01:32.171 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:01:32.171 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.172 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.172 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.172 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.172 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.172 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.173 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:32.173 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:01:32.173 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.173 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.174 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.174 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.174 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.174 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.174 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.175 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:32.175 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.175 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.175 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.175 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.176 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.176 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.176 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:01:32.176 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.177 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.177 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.177 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.177 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:01:32.177 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.178 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.178 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.178 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.178 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.179 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.179 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.179 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.179 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:01:32.179 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.180 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.180 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.180 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.180 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:01:32.180 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.181 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.181 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.181 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.181 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.182 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.182 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.182 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.182 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:01:32.182 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.183 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.183 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.183 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.183 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:01:32.184 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.184 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.184 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.184 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.185 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.185 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.185 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.185 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.186 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:01:32.186 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.186 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.186 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.186 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.187 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:01:32.187 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:32.187 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.187 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.188 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.188 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:01:32.188 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:01:32.188 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:01:32.188 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:01:32.189 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.189 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.189 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.189 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.190 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.190 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.190 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:32.190 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:01:32.190 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.191 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.191 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.191 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.191 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.191 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.192 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.192 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:32.192 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.192 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.193 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.193 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.193 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.193 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.193 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:01:32.194 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.194 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.194 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.194 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.195 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:01:32.195 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.195 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.195 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.195 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.196 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.196 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.196 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.196 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.197 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:01:32.197 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.197 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.197 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.197 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.198 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:01:32.198 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.198 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.198 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.199 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.199 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.199 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.199 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.200 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.200 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:01:32.200 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.200 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.200 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.201 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.201 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:01:32.201 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.201 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.202 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.202 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.202 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.202 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.202 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.203 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.203 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:01:32.203 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.203 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.204 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.204 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.204 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:01:32.204 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:32.204 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.205 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.205 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.205 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:01:32.205 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:01:32.206 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:01:32.206 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:01:32.206 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.206 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.206 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.207 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.207 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.207 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.207 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:32.208 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:01:32.208 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.208 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.208 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.208 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.209 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.209 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.209 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.209 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:32.210 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.210 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.210 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.210 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.210 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.211 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.211 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:01:32.211 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.211 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.211 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.212 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.212 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:01:32.212 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.212 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.213 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.213 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.213 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.213 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.213 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.214 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.214 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:01:32.214 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.214 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.215 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.215 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.215 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:01:32.215 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.215 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.216 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.216 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.216 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.216 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.216 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.217 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.217 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:01:32.217 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.217 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.218 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.218 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.218 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:01:32.218 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.218 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.219 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.219 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.219 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.219 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.220 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.220 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.220 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:01:32.220 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.220 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.221 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.221 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.221 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:01:32.221 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:32.222 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.222 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.222 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.222 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:01:32.222 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:01:32.223 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:01:32.223 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:01:32.223 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.223 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.224 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.224 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.224 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.224 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.224 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:32.225 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:01:32.225 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.225 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.225 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.226 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.226 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.226 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.226 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.226 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 97, 2021-11-21T22:01:32.227 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.227 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.227 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.227 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.228 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.228 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.228 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:01:32.228 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.228 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.229 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.229 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.229 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:01:32.229 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.230 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.230 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.230 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.230 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.230 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.231 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.231 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.231 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:01:32.231 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.232 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.232 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.232 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.232 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:01:32.233 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.233 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.233 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.233 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.233 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.234 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.234 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.234 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.234 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:01:32.235 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.235 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.235 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.235 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.235 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:01:32.236 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.236 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.236 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.236 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.237 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.237 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.237 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.237 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.237 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:01:32.238 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.238 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.238 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.238 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.238 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:01:32.239 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:32.239 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.239 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.239 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.240 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:01:32.240 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:01:32.240 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:01:32.240 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:01:32.240 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.241 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.241 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.241 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.241 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.241 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.242 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:32.242 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:01:32.242 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.242 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.243 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.243 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.243 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.243 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.243 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.244 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:32.244 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.244 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.244 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.244 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.245 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.245 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.245 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:01:32.245 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.246 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.246 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.246 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.246 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:01:32.246 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.247 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.247 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.247 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.247 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.248 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.248 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.248 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.248 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:01:32.248 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.249 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.249 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.249 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.249 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:01:32.249 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.250 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.250 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.250 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.250 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.251 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.251 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.251 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.251 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:01:32.251 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.252 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.252 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.252 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.252 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:01:32.253 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.253 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.253 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.253 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.253 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.254 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.254 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.254 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.254 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:01:32.255 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.255 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.255 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.255 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.255 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:01:32.256 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:32.256 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.256 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.256 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.257 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:01:32.257 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:01:32.257 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:01:32.257 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:01:32.257 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.258 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.258 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.258 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.258 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.259 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.259 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:32.259 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:01:32.259 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.260 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.260 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.260 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.260 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.260 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.261 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.261 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 97, 2021-11-21T22:01:32.261 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.261 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.262 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.262 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.262 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.262 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.262 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:01:32.263 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.263 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.263 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.263 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.264 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:01:32.264 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.264 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.264 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.265 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.265 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.265 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.265 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.265 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.266 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:01:32.266 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.266 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.266 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.267 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.267 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:01:32.267 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.267 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.267 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.268 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.268 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.268 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.268 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.268 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.269 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:01:32.269 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.269 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.269 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.269 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.270 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:01:32.270 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.270 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.270 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.271 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.271 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.271 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.271 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.271 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.272 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:01:32.272 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.272 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.272 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.272 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.273 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:01:32.273 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:32.273 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.273 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.274 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.274 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:01:32.274 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:01:32.274 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:01:32.274 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:01:32.275 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.275 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.275 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.275 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.276 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.276 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.276 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:32.276 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:01:32.276 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:32.277 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:32.277 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:32.277 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:32.277 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:32.277 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:32.278 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:32.278 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:32.278 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:32.278 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:32.279 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:32.279 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.279 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.279 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.279 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:01:32.280 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.280 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.280 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.280 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.280 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:01:32.281 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.281 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.281 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.281 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.282 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:32.282 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.282 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.282 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.282 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:01:32.283 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.283 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.283 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.283 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.284 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:01:32.284 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.284 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.284 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.284 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.285 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:32.285 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.285 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.285 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.285 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:01:32.286 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.286 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.286 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.286 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.287 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:01:32.287 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.287 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.287 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.287 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.288 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:32.288 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:32.288 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.288 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:32.288 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:01:32.289 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.289 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.289 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.289 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:32.290 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:01:32.290 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:32.290 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.290 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.290 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.291 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:01:32.291 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:01:32.291 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:01:32.291 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:01:32.292 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:32.292 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:32.292 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:32.292 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:32.292 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.293 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:32.293 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:01:32.293 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.293 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:32.294 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.294 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.294 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.294 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.294 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.295 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:01:32.295 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.295 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.295 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.296 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:32.296 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.296 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.296 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.297 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.297 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.297 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:01:32.297 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.297 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.298 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.298 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:32.298 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.298 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.299 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.299 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.299 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.299 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:01:32.300 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.300 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.300 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.300 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:32.300 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.301 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.301 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.301 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.301 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.301 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:01:32.302 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.302 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.302 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.302 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:32.303 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.303 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.303 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.303 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.303 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.304 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:01:32.304 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.304 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.304 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.305 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:32.305 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.305 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.305 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.305 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.306 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.306 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:01:32.306 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.306 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.307 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.307 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:32.307 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.307 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.307 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.308 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.308 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.308 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:01:32.308 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.308 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.309 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:32.309 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:32.309 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:32.309 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:32.310 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:32.310 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:32.310 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:32.310 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:01:32.310 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:32.311 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.311 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:32.311 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:01:32.311 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:01:32.312 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:01:32.312 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:01:32.312 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T22:01:32.312 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:01:32.312 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:01:32.313 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:01:32.313 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:01:32.313 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:01:32.313 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:01:32.314 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:01:32.314 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:01:32.314 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:32.314 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:32.314 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:01:32.315 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:01:32.315 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:01:32.315 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:01:32.315 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:01:32.316 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:32.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c3c053720 msgr2=0x7f5c3c055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c3c053720 0x7f5c3c055be0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f5c40002fb0 tx=0x7f5c400069b0).stop 2021-11-21T22:01:32.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 msgr2=0x7f5c580aab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c580aab70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f5c50003330 tx=0x7f5c500091a0).stop 2021-11-21T22:01:32.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 shutdown_connections 2021-11-21T22:01:32.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5c3c053720 0x7f5c3c055be0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5c580a86b0 0x7f5c580aab70 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5c580a6aa0 0x7f5c58065570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 --2- 172.21.2.116:0/2050573099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5c58062410 0x7f5c58065080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 >> 172.21.2.116:0/2050573099 conn(0x7f5c5805bd20 msgr2=0x7f5c581437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:32.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 shutdown_connections 2021-11-21T22:01:32.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.082+0000 7f5c5e206700 1 -- 172.21.2.116:0/2050573099 wait complete. 2021-11-21T22:01:32.320 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:32.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 Processor -- start 2021-11-21T22:01:32.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- start start 2021-11-21T22:01:32.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0600b5630 0x7fc06014af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc060097ec0 0x7fc060146e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc060147300 0x7fc060099650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc060144b40 con 0x7fc060147300 2021-11-21T22:01:32.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06014a560 con 0x7fc060097ec0 2021-11-21T22:01:32.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06014a800 con 0x7fc0600b5630 2021-11-21T22:01:32.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc060147300 0x7fc060099650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc060147300 0x7fc060099650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50942/0 (socket says 172.21.2.116:50942) 2021-11-21T22:01:32.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 -- 172.21.2.116:0/2348028823 learned_addr learned my addr 172.21.2.116:0/2348028823 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:32.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 -- 172.21.2.116:0/2348028823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0600b5630 msgr2=0x7fc06014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:32.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 --2- 172.21.2.116:0/2348028823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0600b5630 0x7fc06014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 -- 172.21.2.116:0/2348028823 >> v2:172.21.2.117:3300/0 conn(0x7fc060097ec0 msgr2=0x7fc060146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:32.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 --2- 172.21.2.116:0/2348028823 >> v2:172.21.2.117:3300/0 conn(0x7fc060097ec0 0x7fc060146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 -- 172.21.2.116:0/2348028823 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06014aaa0 con 0x7fc060147300 2021-11-21T22:01:32.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc064f79700 1 --2- 172.21.2.116:0/2348028823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060147300 0x7fc060099650 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fc0580032f0 tx=0x7fc058009290).ready entity=mon.0 client_cookie=73337b6da50c7490 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc05effd700 1 -- 172.21.2.116:0/2348028823 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05800b870 con 0x7fc060147300 2021-11-21T22:01:32.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc05effd700 1 -- 172.21.2.116:0/2348028823 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05800b9d0 con 0x7fc060147300 2021-11-21T22:01:32.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc05effd700 1 -- 172.21.2.116:0/2348028823 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05800bb40 con 0x7fc060147300 2021-11-21T22:01:32.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- 172.21.2.116:0/2348028823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060147300 msgr2=0x7fc060099650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 --2- 172.21.2.116:0/2348028823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060147300 0x7fc060099650 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fc0580032f0 tx=0x7fc058009290).stop 2021-11-21T22:01:32.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- 172.21.2.116:0/2348028823 shutdown_connections 2021-11-21T22:01:32.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 --2- 172.21.2.116:0/2348028823 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060147300 0x7fc060099650 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 --2- 172.21.2.116:0/2348028823 >> v2:172.21.2.117:3300/0 conn(0x7fc060097ec0 0x7fc060146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 --2- 172.21.2.116:0/2348028823 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc0600b5630 0x7fc06014af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- 172.21.2.116:0/2348028823 >> 172.21.2.116:0/2348028823 conn(0x7fc06005c7f0 msgr2=0x7fc0600b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:32.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- 172.21.2.116:0/2348028823 shutdown_connections 2021-11-21T22:01:32.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.374+0000 7fc06577a700 1 -- 172.21.2.116:0/2348028823 wait complete. 2021-11-21T22:01:32.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 Processor -- start 2021-11-21T22:01:32.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 -- start start 2021-11-21T22:01:32.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060097ec0 0x7fc0600ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc0600b5630 0x7fc060062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc060147300 0x7fc060159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06015a720 con 0x7fc060097ec0 2021-11-21T22:01:32.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0600ad610 con 0x7fc0600b5630 2021-11-21T22:01:32.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06015a480 con 0x7fc060147300 2021-11-21T22:01:32.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060097ec0 0x7fc0600ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060097ec0 0x7fc0600ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50944/0 (socket says 172.21.2.116:50944) 2021-11-21T22:01:32.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 -- 172.21.2.116:0/31097936 learned_addr learned my addr 172.21.2.116:0/31097936 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:32.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 -- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc060147300 msgr2=0x7fc060159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc060147300 0x7fc060159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 -- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc0600b5630 msgr2=0x7fc060062f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc0600b5630 0x7fc060062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 -- 172.21.2.116:0/31097936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058009530 con 0x7fc060097ec0 2021-11-21T22:01:32.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05ffff700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060097ec0 0x7fc0600ad120 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fc050002f70 tx=0x7fc050006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05d7fa700 1 -- 172.21.2.116:0/31097936 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc050013460 con 0x7fc060097ec0 2021-11-21T22:01:32.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05d7fa700 1 -- 172.21.2.116:0/31097936 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0500135c0 con 0x7fc060097ec0 2021-11-21T22:01:32.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0600ac9e0 con 0x7fc060097ec0 2021-11-21T22:01:32.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05d7fa700 1 -- 172.21.2.116:0/31097936 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc050013920 con 0x7fc060097ec0 2021-11-21T22:01:32.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0600ad8e0 con 0x7fc060097ec0 2021-11-21T22:01:32.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06015a1f0 con 0x7fc060097ec0 2021-11-21T22:01:32.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05d7fa700 1 -- 172.21.2.116:0/31097936 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc05000f070 con 0x7fc060097ec0 2021-11-21T22:01:32.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05d7fa700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc040053520 0x7fc0400559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.378+0000 7fc05d7fa700 1 -- 172.21.2.116:0/31097936 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc05006b210 con 0x7fc060097ec0 2021-11-21T22:01:32.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.382+0000 7fc05f7fe700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc040053520 0x7fc0400559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.382+0000 7fc05f7fe700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc040053520 0x7fc0400559e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc054002f70 tx=0x7fc054006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.382+0000 7fc05d7fa700 1 -- 172.21.2.116:0/31097936 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc050042260 con 0x7fc060097ec0 2021-11-21T22:01:32.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0600573d0 con 0x7fc060097ec0 2021-11-21T22:01:32.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc05d7fa700 1 -- 172.21.2.116:0/31097936 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fc050045880 con 0x7fc060097ec0 2021-11-21T22:01:32.494 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:32.494 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":44,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:23.183200+0000","modified":"2021-11-21T22:01:24.187992+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/3417150056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":3417150056},{"type":"v1","addr":"172.21.2.116:6835","nonce":3417150056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-11-21T22:01:32.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc040053520 msgr2=0x7fc0400559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc040053520 0x7fc0400559e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc054002f70 tx=0x7fc054006a80).stop 2021-11-21T22:01:32.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060097ec0 msgr2=0x7fc0600ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060097ec0 0x7fc0600ad120 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fc050002f70 tx=0x7fc050006e40).stop 2021-11-21T22:01:32.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 shutdown_connections 2021-11-21T22:01:32.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc060147300 0x7fc060159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc0600b5630 0x7fc060062f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc040053520 0x7fc0400559e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 --2- 172.21.2.116:0/31097936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc060097ec0 0x7fc0600ad120 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 >> 172.21.2.116:0/31097936 conn(0x7fc06005c7f0 msgr2=0x7fc0601493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:32.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 shutdown_connections 2021-11-21T22:01:32.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.490+0000 7fc06577a700 1 -- 172.21.2.116:0/31097936 wait complete. 2021-11-21T22:01:32.498 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 44 2021-11-21T22:01:32.501 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:01:32.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 Processor -- start 2021-11-21T22:01:32.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 -- start start 2021-11-21T22:01:32.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcf78144810 0x7fcf781478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 0x7fcf780b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcf780b1db0 0x7fcf780ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf78147e80 con 0x7fcf78144810 2021-11-21T22:01:32.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf780af5a0 con 0x7fcf780b1db0 2021-11-21T22:01:32.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf7ea0e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780b1490 con 0x7fcf780abce0 2021-11-21T22:01:32.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf77fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcf78144810 0x7fcf781478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf77fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcf78144810 0x7fcf781478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50952/0 (socket says 172.21.2.116:50952) 2021-11-21T22:01:32.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.546+0000 7fcf77fff700 1 -- 172.21.2.116:0/3197758485 learned_addr learned my addr 172.21.2.116:0/3197758485 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:32.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf77fff700 1 -- 172.21.2.116:0/3197758485 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 msgr2=0x7fcf780b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:32.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf77fff700 1 --2- 172.21.2.116:0/3197758485 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 0x7fcf780b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf77fff700 1 -- 172.21.2.116:0/3197758485 >> v2:172.21.2.117:3300/0 conn(0x7fcf780b1db0 msgr2=0x7fcf780ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:32.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf77fff700 1 --2- 172.21.2.116:0/3197758485 >> v2:172.21.2.117:3300/0 conn(0x7fcf780b1db0 0x7fcf780ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf77fff700 1 -- 172.21.2.116:0/3197758485 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf780af060 con 0x7fcf78144810 2021-11-21T22:01:32.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf77fff700 1 --2- 172.21.2.116:0/3197758485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf781478c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fcf60002f70 tx=0x7fcf60006eb0).ready entity=mon.0 client_cookie=b1b2a42dc7a9c2c3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf76ffd700 1 -- 172.21.2.116:0/3197758485 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6000d570 con 0x7fcf78144810 2021-11-21T22:01:32.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf76ffd700 1 -- 172.21.2.116:0/3197758485 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60000e90 con 0x7fcf78144810 2021-11-21T22:01:32.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf76ffd700 1 -- 172.21.2.116:0/3197758485 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6000d900 con 0x7fcf78144810 2021-11-21T22:01:32.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/3197758485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 msgr2=0x7fcf781478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/3197758485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf781478c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fcf60002f70 tx=0x7fcf60006eb0).stop 2021-11-21T22:01:32.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/3197758485 shutdown_connections 2021-11-21T22:01:32.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/3197758485 >> v2:172.21.2.117:3300/0 conn(0x7fcf780b1db0 0x7fcf780ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/3197758485 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 0x7fcf780b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/3197758485 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf781478c0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/3197758485 >> 172.21.2.116:0/3197758485 conn(0x7fcf7805bd20 msgr2=0x7fcf7805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:32.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/3197758485 shutdown_connections 2021-11-21T22:01:32.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/3197758485 wait complete. 2021-11-21T22:01:32.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 Processor -- start 2021-11-21T22:01:32.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- start start 2021-11-21T22:01:32.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 0x7fcf78143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf78156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcf78064320 0x7fcf78156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78063240 con 0x7fcf78144810 2021-11-21T22:01:32.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7815a590 con 0x7fcf78064320 2021-11-21T22:01:32.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7815a830 con 0x7fcf780abce0 2021-11-21T22:01:32.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf78156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf78156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50958/0 (socket says 172.21.2.116:50958) 2021-11-21T22:01:32.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 -- 172.21.2.116:0/786992326 learned_addr learned my addr 172.21.2.116:0/786992326 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:32.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 -- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 msgr2=0x7fcf78143510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:32.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 0x7fcf78143510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 -- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcf78064320 msgr2=0x7fcf78156b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:32.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcf78064320 0x7fcf78156b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 -- 172.21.2.116:0/786992326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6000b1e0 con 0x7fcf78144810 2021-11-21T22:01:32.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf777fe700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf78156630 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7fcf68002fb0 tx=0x7fcf68006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf757fa700 1 -- 172.21.2.116:0/786992326 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf68013460 con 0x7fcf78144810 2021-11-21T22:01:32.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf757fa700 1 -- 172.21.2.116:0/786992326 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf680135c0 con 0x7fcf78144810 2021-11-21T22:01:32.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf78063be0 con 0x7fcf78144810 2021-11-21T22:01:32.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf780b3c00 con 0x7fcf78144810 2021-11-21T22:01:32.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf757fa700 1 -- 172.21.2.116:0/786992326 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf68013920 con 0x7fcf78144810 2021-11-21T22:01:32.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.550+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf50003410 con 0x7fcf78144810 2021-11-21T22:01:32.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.554+0000 7fcf757fa700 1 -- 172.21.2.116:0/786992326 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcf6800f070 con 0x7fcf78144810 2021-11-21T22:01:32.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.554+0000 7fcf757fa700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcf5c0536f0 0x7fcf5c055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.554+0000 7fcf757fa700 1 -- 172.21.2.116:0/786992326 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcf6806b460 con 0x7fcf78144810 2021-11-21T22:01:32.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.554+0000 7fcf77fff700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcf5c0536f0 0x7fcf5c055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.554+0000 7fcf77fff700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcf5c0536f0 0x7fcf5c055bb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fcf60002f70 tx=0x7fcf60015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:32.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.554+0000 7fcf757fa700 1 -- 172.21.2.116:0/786992326 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcf68042450 con 0x7fcf78144810 2021-11-21T22:01:32.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.662+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcf500036c0 con 0x7fcf78144810 2021-11-21T22:01:32.906 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:01:32.902+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:01:32.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.914+0000 7fcf757fa700 1 -- 172.21.2.116:0/786992326 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcf68045a70 con 0x7fcf78144810 2021-11-21T22:01:32.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcf5c0536f0 msgr2=0x7fcf5c055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcf5c0536f0 0x7fcf5c055bb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fcf60002f70 tx=0x7fcf60015000).stop 2021-11-21T22:01:32.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 msgr2=0x7fcf78156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf78156630 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7fcf68002fb0 tx=0x7fcf68006de0).stop 2021-11-21T22:01:32.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 shutdown_connections 2021-11-21T22:01:32.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcf78064320 0x7fcf78156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcf5c0536f0 0x7fcf5c055bb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcf78144810 0x7fcf78156630 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 --2- 172.21.2.116:0/786992326 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcf780abce0 0x7fcf78143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:32.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 >> 172.21.2.116:0/786992326 conn(0x7fcf7805bd20 msgr2=0x7fcf780af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:32.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 shutdown_connections 2021-11-21T22:01:32.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.918+0000 7fcf7ea0e700 1 -- 172.21.2.116:0/786992326 wait complete. 2021-11-21T22:01:32.926 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:01:32.930 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:32.934 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:01:32.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31bcadc700 1 Processor -- start 2021-11-21T22:01:32.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31bcadc700 1 -- start start 2021-11-21T22:01:32.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31bcadc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 0x7f31b80b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31bcadc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f31b81443a0 0x7f31b8144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31bcadc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b80b29b0 con 0x7f31b80b7070 2021-11-21T22:01:32.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31bcadc700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b80ab720 con 0x7f31b81443a0 2021-11-21T22:01:32.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31bcadc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8067c30 con 0x7f31b8067850 2021-11-21T22:01:32.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31b5d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f31b80b7070 0x7f31b80ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34706/0 (socket says 172.21.2.116:34706) 2021-11-21T22:01:32.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31b5d9b700 1 -- 172.21.2.116:0/3617461480 learned_addr learned my addr 172.21.2.116:0/3617461480 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:32.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.978+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494314894 0 0) 0x7f31b80b29b0 con 0x7f31b80b7070 2021-11-21T22:01:32.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3198003260 con 0x7f31b80b7070 2021-11-21T22:01:32.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 0x7f31b80b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b6d9d700 1 --2- 172.21.2.116:0/3617461480 >> v2:172.21.2.117:3300/0 conn(0x7f31b81443a0 0x7f31b8144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 601082058 0 0) 0x7f3198003260 con 0x7f31b80b7070 2021-11-21T22:01:32.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 msgr2=0x7f31b80b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:32.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 0x7f31b80b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 >> v2:172.21.2.117:3300/0 conn(0x7f31b81443a0 msgr2=0x7f31b8144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:32.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 --2- 172.21.2.116:0/3617461480 >> v2:172.21.2.117:3300/0 conn(0x7f31b81443a0 0x7f31b8144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b80b7910 con 0x7f31b80b7070 2021-11-21T22:01:32.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f31a0001660 con 0x7f31b80b7070 2021-11-21T22:01:32.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 >> v1:172.21.2.116:6789/0 conn(0x7f31b80b7070 legacy=0x7f31b80ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:32.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3198003750 0x7f3198003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3198004500 0x7f3198004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 0x7f3198004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:32.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80ab720 con 0x7f3198003750 2021-11-21T22:01:32.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8067c30 con 0x7f3198004500 2021-11-21T22:01:32.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319800b7a0 con 0x7f31b8067850 2021-11-21T22:01:32.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3198003750 0x7f3198003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b6d9d700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 0x7f3198004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3198004500 0x7f3198004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:32.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 -- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 msgr2=0x7f3198004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:32.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f31b8067850 0x7f3198004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:32.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 -- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3198004500 msgr2=0x7f3198004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:33.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3198004500 0x7f3198004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 -- 172.21.2.116:0/3617461480 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198003260 con 0x7f3198003750 2021-11-21T22:01:33.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3198003750 0x7f3198003b70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f31a800a000 tx=0x7f31a8008040).ready entity=mon.0 client_cookie=b851e7d1b15042f6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:33.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a8013670 con 0x7f3198003750 2021-11-21T22:01:33.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a80137d0 con 0x7f3198003750 2021-11-21T22:01:33.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b559a700 1 -- 172.21.2.116:0/3617461480 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a8013b30 con 0x7f3198003750 2021-11-21T22:01:33.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- 172.21.2.116:0/3617461480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3198003750 msgr2=0x7f3198003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 --2- 172.21.2.116:0/3617461480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3198003750 0x7f3198003b70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f31a800a000 tx=0x7f31a8008040).stop 2021-11-21T22:01:33.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b6d9d700 1 -- 172.21.2.116:0/3617461480 reap_dead start 2021-11-21T22:01:33.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- 172.21.2.116:0/3617461480 shutdown_connections 2021-11-21T22:01:33.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- 172.21.2.116:0/3617461480 >> 172.21.2.116:0/3617461480 conn(0x7f31b8064d60 msgr2=0x7f31b8065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:33.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- 172.21.2.116:0/3617461480 shutdown_connections 2021-11-21T22:01:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- 172.21.2.116:0/3617461480 wait complete. 2021-11-21T22:01:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 Processor -- start 2021-11-21T22:01:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- start start 2021-11-21T22:01:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3198003750 0x7f31b809e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31b8063140 0x7f31b8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31b8062a20 0x7f31b8143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80aa060 con 0x7f31b8063140 2021-11-21T22:01:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b809ea10 con 0x7f31b8062a20 2021-11-21T22:01:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8063650 con 0x7f3198003750 2021-11-21T22:01:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31b8063140 0x7f31b8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31b8063140 0x7f31b8156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:50992/0 (socket says 172.21.2.116:50992) 2021-11-21T22:01:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 -- 172.21.2.116:0/3890946614 learned_addr learned my addr 172.21.2.116:0/3890946614 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b659c700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3198003750 0x7f31b809e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b6d9d700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31b8062a20 0x7f31b8143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 -- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3198003750 msgr2=0x7f31b809e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3198003750 0x7f31b809e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 -- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31b8062a20 msgr2=0x7f31b8143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31b8062a20 0x7f31b8143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 -- 172.21.2.116:0/3890946614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b81564c0 con 0x7f31b8063140 2021-11-21T22:01:33.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31b5d9b700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31b8063140 0x7f31b8156ea0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f31a0008000 tx=0x7f31a0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:33.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31af7fe700 1 -- 172.21.2.116:0/3890946614 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a0007170 con 0x7f31b8063140 2021-11-21T22:01:33.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31af7fe700 1 -- 172.21.2.116:0/3890946614 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a00072d0 con 0x7f31b8063140 2021-11-21T22:01:33.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.982+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b8156760 con 0x7f31b8063140 2021-11-21T22:01:33.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31af7fe700 1 -- 172.21.2.116:0/3890946614 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a0004920 con 0x7f31b8063140 2021-11-21T22:01:33.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b8157390 con 0x7f31b8063140 2021-11-21T22:01:33.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b80b2be0 con 0x7f31b8063140 2021-11-21T22:01:33.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31af7fe700 1 -- 172.21.2.116:0/3890946614 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f31a001b060 con 0x7f31b8063140 2021-11-21T22:01:33.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31af7fe700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f319c0536d0 0x7f319c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31af7fe700 1 -- 172.21.2.116:0/3890946614 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f31a006b5d0 con 0x7f31b8063140 2021-11-21T22:01:33.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31b659c700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f319c0536d0 0x7f319c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31b659c700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f319c0536d0 0x7f319c055b90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f31a8019000 tx=0x7f31a8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:33.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:32.986+0000 7f31af7fe700 1 -- 172.21.2.116:0/3890946614 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f31a0042570 con 0x7f31b8063140 2021-11-21T22:01:33.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31b80573d0 con 0x7f31b8063140 2021-11-21T22:01:33.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31af7fe700 1 -- 172.21.2.116:0/3890946614 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f31a0019030 con 0x7f31b8063140 2021-11-21T22:01:33.104 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:33.104 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:23.183200+0000","modified":"2021-11-21T22:01:32.906513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-11-21T22:01:33.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f319c0536d0 msgr2=0x7f319c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f319c0536d0 0x7f319c055b90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f31a8019000 tx=0x7f31a8008040).stop 2021-11-21T22:01:33.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31b8063140 msgr2=0x7f31b8156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31b8063140 0x7f31b8156ea0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f31a0008000 tx=0x7f31a0009040).stop 2021-11-21T22:01:33.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 shutdown_connections 2021-11-21T22:01:33.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31b8062a20 0x7f31b8143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f319c0536d0 0x7f319c055b90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31b8063140 0x7f31b8156ea0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 --2- 172.21.2.116:0/3890946614 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3198003750 0x7f31b809e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.098+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 >> 172.21.2.116:0/3890946614 conn(0x7f31b8064d60 msgr2=0x7f31b8142820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:33.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.102+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 shutdown_connections 2021-11-21T22:01:33.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.102+0000 7f31bcadc700 1 -- 172.21.2.116:0/3890946614 wait complete. 2021-11-21T22:01:33.109 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 45 2021-11-21T22:01:33.111 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:01:33.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53e4d20700 1 Processor -- start 2021-11-21T22:01:33.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53e4d20700 1 -- start start 2021-11-21T22:01:33.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53e4d20700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f53e0144810 0x7f53e01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53e4d20700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e00abce0 0x7f53e00b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53e4d20700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e0147e80 con 0x7f53e00b1db0 2021-11-21T22:01:33.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53e4d20700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00b1490 con 0x7f53e0144810 2021-11-21T22:01:33.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53e4d20700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00af060 con 0x7f53e00abce0 2021-11-21T22:01:33.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53ded9d700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f53e00b1db0 0x7f53e00ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:33634/0 (socket says 172.21.2.116:33634) 2021-11-21T22:01:33.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53ded9d700 1 -- 172.21.2.116:0/1588545878 learned_addr learned my addr 172.21.2.116:0/1588545878 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:33.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53ddd9b700 1 --2- 172.21.2.116:0/1588545878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e00abce0 0x7f53e00b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.202+0000 7f53de59c700 1 --2- 172.21.2.116:0/1588545878 >> v2:172.21.2.117:3300/0 conn(0x7f53e0144810 0x7f53e01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ddd9b700 1 -- 172.21.2.116:0/1588545878 >> v2:172.21.2.117:3300/0 conn(0x7f53e0144810 msgr2=0x7f53e01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:33.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ddd9b700 1 --2- 172.21.2.116:0/1588545878 >> v2:172.21.2.117:3300/0 conn(0x7f53e0144810 0x7f53e01478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ddd9b700 1 -- 172.21.2.116:0/1588545878 >> v1:172.21.2.117:6789/0 conn(0x7f53e00b1db0 legacy=0x7f53e00ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ddd9b700 1 -- 172.21.2.116:0/1588545878 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00af5a0 con 0x7f53e00abce0 2021-11-21T22:01:33.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ddd9b700 1 --2- 172.21.2.116:0/1588545878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e00abce0 0x7f53e00b18c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f53d8001490 tx=0x7f53d0006df0).ready entity=mon.2 client_cookie=3a8219defcd956c6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:33.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53dd59a700 1 -- 172.21.2.116:0/1588545878 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53d000d4a0 con 0x7f53e00abce0 2021-11-21T22:01:33.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53dd59a700 1 -- 172.21.2.116:0/1588545878 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d0000e90 con 0x7f53e00abce0 2021-11-21T22:01:33.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53dd59a700 1 -- 172.21.2.116:0/1588545878 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53d000d7f0 con 0x7f53e00abce0 2021-11-21T22:01:33.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- 172.21.2.116:0/1588545878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e00abce0 msgr2=0x7f53e00b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 --2- 172.21.2.116:0/1588545878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e00abce0 0x7f53e00b18c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f53d8001490 tx=0x7f53d0006df0).stop 2021-11-21T22:01:33.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- 172.21.2.116:0/1588545878 shutdown_connections 2021-11-21T22:01:33.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 --2- 172.21.2.116:0/1588545878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e00abce0 0x7f53e00b18c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 --2- 172.21.2.116:0/1588545878 >> v2:172.21.2.117:3300/0 conn(0x7f53e0144810 0x7f53e01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- 172.21.2.116:0/1588545878 >> 172.21.2.116:0/1588545878 conn(0x7f53e005bd20 msgr2=0x7f53e005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:33.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- 172.21.2.116:0/1588545878 shutdown_connections 2021-11-21T22:01:33.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- 172.21.2.116:0/1588545878 wait complete. 2021-11-21T22:01:33.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 Processor -- start 2021-11-21T22:01:33.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- start start 2021-11-21T22:01:33.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53e00abce0 0x7f53e0143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e0144810 0x7f53e00b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53e01464a0 0x7f53e00b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b3410 con 0x7f53e01464a0 2021-11-21T22:01:33.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b31b0 con 0x7f53e00abce0 2021-11-21T22:01:33.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53e4d20700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0145840 con 0x7f53e0144810 2021-11-21T22:01:33.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53e01464a0 0x7f53e00b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53e01464a0 0x7f53e00b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51008/0 (socket says 172.21.2.116:51008) 2021-11-21T22:01:33.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 -- 172.21.2.116:0/3556497932 learned_addr learned my addr 172.21.2.116:0/3556497932 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:33.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 -- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e0144810 msgr2=0x7f53e00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e0144810 0x7f53e00b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:33.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 -- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53e00abce0 msgr2=0x7f53e0143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53e00abce0 0x7f53e0143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:33.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.206+0000 7f53ded9d700 1 -- 172.21.2.116:0/3556497932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d000b180 con 0x7f53e01464a0 2021-11-21T22:01:33.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53ded9d700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53e01464a0 0x7f53e00b3c70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f53d8001550 tx=0x7f53d8007ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:33.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53d77fe700 1 -- 172.21.2.116:0/3556497932 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53d8005ba0 con 0x7f53e01464a0 2021-11-21T22:01:33.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53d77fe700 1 -- 172.21.2.116:0/3556497932 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d8005d00 con 0x7f53e01464a0 2021-11-21T22:01:33.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53d77fe700 1 -- 172.21.2.116:0/3556497932 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53d8007300 con 0x7f53e01464a0 2021-11-21T22:01:33.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e0146130 con 0x7f53e01464a0 2021-11-21T22:01:33.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e00b2c50 con 0x7f53e01464a0 2021-11-21T22:01:33.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0145bc0 con 0x7f53e01464a0 2021-11-21T22:01:33.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53d77fe700 1 -- 172.21.2.116:0/3556497932 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f53d8010030 con 0x7f53e01464a0 2021-11-21T22:01:33.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53d77fe700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f53c4053690 0x7f53c4055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53d77fe700 1 -- 172.21.2.116:0/3556497932 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f53d806bb90 con 0x7f53e01464a0 2021-11-21T22:01:33.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53de59c700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f53c4053690 0x7f53c4055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53de59c700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f53c4053690 0x7f53c4055b50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f53c800c000 tx=0x7f53c8006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:33.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.210+0000 7f53d77fe700 1 -- 172.21.2.116:0/3556497932 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f53d8042b90 con 0x7f53e01464a0 2021-11-21T22:01:33.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.318+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f53e00af300 con 0x7f53e01464a0 2021-11-21T22:01:33.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53d77fe700 1 -- 172.21.2.116:0/3556497932 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f53d80461b0 con 0x7f53e01464a0 2021-11-21T22:01:33.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f53c4053690 msgr2=0x7f53c4055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f53c4053690 0x7f53c4055b50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f53c800c000 tx=0x7f53c8006df0).stop 2021-11-21T22:01:33.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53e01464a0 msgr2=0x7f53e00b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:33.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53e01464a0 0x7f53e00b3c70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f53d8001550 tx=0x7f53d8007ea0).stop 2021-11-21T22:01:33.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 shutdown_connections 2021-11-21T22:01:33.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f53c4053690 0x7f53c4055b50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f53e01464a0 0x7f53e00b3c70 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:33.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f53e0144810 0x7f53e00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:33.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 --2- 172.21.2.116:0/3556497932 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f53e00abce0 0x7f53e0143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:33.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 >> 172.21.2.116:0/3556497932 conn(0x7f53e005bd20 msgr2=0x7f53e01400a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:33.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 shutdown_connections 2021-11-21T22:01:33.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.922+0000 7f53e4d20700 1 -- 172.21.2.116:0/3556497932 wait complete. 2021-11-21T22:01:33.933 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:01:33.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73fa8e0700 1 Processor -- start 2021-11-21T22:01:33.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73fa8e0700 1 -- start start 2021-11-21T22:01:33.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73fa8e0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f73f40a8670 0x7f73f4147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73fa8e0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f4143e60 0x7f73f41442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:33.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73fa8e0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f40b1940 con 0x7f73f40b11c0 2021-11-21T22:01:33.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73fa8e0700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f40b15a0 con 0x7f73f40a8670 2021-11-21T22:01:33.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73fa8e0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40b1b70 con 0x7f73f4143e60 2021-11-21T22:01:33.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73f3fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f73f40b11c0 0x7f73f4147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34738/0 (socket says 172.21.2.116:34738) 2021-11-21T22:01:33.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73f3fff700 1 -- 172.21.2.116:0/3685866904 learned_addr learned my addr 172.21.2.116:0/3685866904 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:33.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73f37fe700 1 --2- 172.21.2.116:0/3685866904 >> v2:172.21.2.116:3300/0 conn(0x7f73f40a8670 0x7f73f4147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:33.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73f2ffd700 1 -- 172.21.2.116:0/3685866904 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317856530 0 0) 0x7f73f40b1940 con 0x7f73f40b11c0 2021-11-21T22:01:33.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73f2ffd700 1 -- 172.21.2.116:0/3685866904 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e0003260 con 0x7f73f40b11c0 2021-11-21T22:01:33.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73f37fe700 1 -- 172.21.2.116:0/3685866904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f4143e60 msgr2=0x7f73f41442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:33.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.982+0000 7f73f37fe700 1 --2- 172.21.2.116:0/3685866904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f4143e60 0x7f73f41442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:34.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f37fe700 1 -- 172.21.2.116:0/3685866904 >> v1:172.21.2.116:6789/0 conn(0x7f73f40b11c0 legacy=0x7f73f4147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:34.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f37fe700 1 -- 172.21.2.116:0/3685866904 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f40b1e10 con 0x7f73f40a8670 2021-11-21T22:01:34.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f37fe700 1 --2- 172.21.2.116:0/3685866904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f40a8670 0x7f73f4147940 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f73e8002f70 tx=0x7f73e8006df0).ready entity=mon.0 client_cookie=868534868c6457e8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:34.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f2ffd700 1 -- 172.21.2.116:0/3685866904 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73e800d570 con 0x7f73f40a8670 2021-11-21T22:01:34.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f2ffd700 1 -- 172.21.2.116:0/3685866904 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e8000e90 con 0x7f73f40a8670 2021-11-21T22:01:34.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f2ffd700 1 -- 172.21.2.116:0/3685866904 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73e800d8c0 con 0x7f73f40a8670 2021-11-21T22:01:34.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/3685866904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f40a8670 msgr2=0x7f73f4147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:34.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/3685866904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f40a8670 0x7f73f4147940 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f73e8002f70 tx=0x7f73e8006df0).stop 2021-11-21T22:01:34.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/3685866904 shutdown_connections 2021-11-21T22:01:34.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/3685866904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f4143e60 0x7f73f41442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:34.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/3685866904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f40a8670 0x7f73f4147940 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:34.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/3685866904 >> 172.21.2.116:0/3685866904 conn(0x7f73f405bd20 msgr2=0x7f73f405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:34.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/3685866904 shutdown_connections 2021-11-21T22:01:34.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/3685866904 wait complete. 2021-11-21T22:01:34.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 Processor -- start 2021-11-21T22:01:34.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- start start 2021-11-21T22:01:34.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f40a8670 0x7f73f414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:34.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f73f40b11c0 0x7f73f4141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:34.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f4143e60 0x7f73f41422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:34.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40676c0 con 0x7f73f4143e60 2021-11-21T22:01:34.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f4159840 con 0x7f73f40b11c0 2021-11-21T22:01:34.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f4149450 con 0x7f73f40a8670 2021-11-21T22:01:34.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f4143e60 0x7f73f41422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:34.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f4143e60 0x7f73f41422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51022/0 (socket says 172.21.2.116:51022) 2021-11-21T22:01:34.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 -- 172.21.2.116:0/1733149850 learned_addr learned my addr 172.21.2.116:0/1733149850 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:34.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f37fe700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f73f40b11c0 0x7f73f4141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:34.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f3fff700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f40a8670 0x7f73f414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:34.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 -- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f40a8670 msgr2=0x7f73f414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:34.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f40a8670 0x7f73f414bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:34.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 -- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f73f40b11c0 msgr2=0x7f73f4141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:34.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f73f40b11c0 0x7f73f4141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:34.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 -- 172.21.2.116:0/1733149850 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e800b180 con 0x7f73f4143e60 2021-11-21T22:01:34.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f8e7d700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f4143e60 0x7f73f41422a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f73ec0032f0 tx=0x7f73ec0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:34.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f17fa700 1 -- 172.21.2.116:0/1733149850 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73ec00d4a0 con 0x7f73f4143e60 2021-11-21T22:01:34.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f17fa700 1 -- 172.21.2.116:0/1733149850 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec00d600 con 0x7f73f4143e60 2021-11-21T22:01:34.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f41488f0 con 0x7f73f4143e60 2021-11-21T22:01:34.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f17fa700 1 -- 172.21.2.116:0/1733149850 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73ec00d960 con 0x7f73f4143e60 2021-11-21T22:01:34.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f40b00c0 con 0x7f73f4143e60 2021-11-21T22:01:34.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f4149290 con 0x7f73f4143e60 2021-11-21T22:01:34.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f17fa700 1 -- 172.21.2.116:0/1733149850 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f73ec018030 con 0x7f73f4143e60 2021-11-21T22:01:34.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f17fa700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f73d4053580 0x7f73d4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:34.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f17fa700 1 -- 172.21.2.116:0/1733149850 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f73ec06b370 con 0x7f73f4143e60 2021-11-21T22:01:34.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.986+0000 7f73f3fff700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f73d4053580 0x7f73d4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:34.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.990+0000 7f73f3fff700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f73d4053580 0x7f73d4055a40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f73e400d000 tx=0x7f73e4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:34.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:33.990+0000 7f73f17fa700 1 -- 172.21.2.116:0/1733149850 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73ec042420 con 0x7f73f4143e60 2021-11-21T22:01:34.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.074+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73f4149610 con 0x7f73f4143e60 2021-11-21T22:01:34.411 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:34.415 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:34.420 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:34.423 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:01:34.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73f17fa700 1 -- 172.21.2.116:0/1733149850 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 185+0+0 (secure 0 0 0) 0x7f73ec045a40 con 0x7f73f4143e60 2021-11-21T22:01:34.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f73d4053580 msgr2=0x7f73d4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:34.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f73d4053580 0x7f73d4055a40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f73e400d000 tx=0x7f73e4009040).stop 2021-11-21T22:01:34.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f4143e60 msgr2=0x7f73f41422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:34.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f4143e60 0x7f73f41422a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f73ec0032f0 tx=0x7f73ec0091a0).stop 2021-11-21T22:01:34.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 shutdown_connections 2021-11-21T22:01:34.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f73d4053580 0x7f73d4055a40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:34.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f73f4143e60 0x7f73f41422a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:34.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f73f40b11c0 0x7f73f4141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:34.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 --2- 172.21.2.116:0/1733149850 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f73f40a8670 0x7f73f414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:34.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 >> 172.21.2.116:0/1733149850 conn(0x7f73f405bd20 msgr2=0x7f73f4145420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:34.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 shutdown_connections 2021-11-21T22:01:34.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:34.970+0000 7f73fa8e0700 1 -- 172.21.2.116:0/1733149850 wait complete. 2021-11-21T22:01:34.977 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:01:34.981 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:01:35.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 Processor -- start 2021-11-21T22:01:35.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 -- start start 2021-11-21T22:01:35.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80b11c0 0x7f1bd8147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:35.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1bd80a8670 0x7f1bd8147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:35.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1bd8147c10 0x7f1bd814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:35.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd80b1940 con 0x7f1bd80a8670 2021-11-21T22:01:35.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd80b15a0 con 0x7f1bd8147c10 2021-11-21T22:01:35.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdf735700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd80b1b70 con 0x7f1bd80b11c0 2021-11-21T22:01:35.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1bd80a8670 0x7f1bd8147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:35.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1bd80a8670 0x7f1bd8147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51032/0 (socket says 172.21.2.116:51032) 2021-11-21T22:01:35.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 -- 172.21.2.116:0/1021074959 learned_addr learned my addr 172.21.2.116:0/1021074959 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:35.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 -- 172.21.2.116:0/1021074959 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80b11c0 msgr2=0x7f1bd8147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:35.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 --2- 172.21.2.116:0/1021074959 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80b11c0 0x7f1bd8147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 -- 172.21.2.116:0/1021074959 >> v2:172.21.2.117:3300/0 conn(0x7f1bd8147c10 msgr2=0x7f1bd814a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:35.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 --2- 172.21.2.116:0/1021074959 >> v2:172.21.2.117:3300/0 conn(0x7f1bd8147c10 0x7f1bd814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 -- 172.21.2.116:0/1021074959 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd80b1e10 con 0x7f1bd80a8670 2021-11-21T22:01:35.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bdccd0700 1 --2- 172.21.2.116:0/1021074959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd80a8670 0x7f1bd8147720 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f1bcc002f70 tx=0x7f1bcc006eb0).ready entity=mon.0 client_cookie=7bf855bae9a94076 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:35.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bcbfff700 1 -- 172.21.2.116:0/1021074959 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bcc00d570 con 0x7f1bd80a8670 2021-11-21T22:01:35.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bcbfff700 1 -- 172.21.2.116:0/1021074959 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc000e90 con 0x7f1bd80a8670 2021-11-21T22:01:35.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.030+0000 7f1bcbfff700 1 -- 172.21.2.116:0/1021074959 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bcc00d900 con 0x7f1bd80a8670 2021-11-21T22:01:35.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/1021074959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd80a8670 msgr2=0x7f1bd8147720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:35.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 --2- 172.21.2.116:0/1021074959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd80a8670 0x7f1bd8147720 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f1bcc002f70 tx=0x7f1bcc006eb0).stop 2021-11-21T22:01:35.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/1021074959 shutdown_connections 2021-11-21T22:01:35.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 --2- 172.21.2.116:0/1021074959 >> v2:172.21.2.117:3300/0 conn(0x7f1bd8147c10 0x7f1bd814a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 --2- 172.21.2.116:0/1021074959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd80a8670 0x7f1bd8147720 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:35.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 --2- 172.21.2.116:0/1021074959 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80b11c0 0x7f1bd8147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/1021074959 >> 172.21.2.116:0/1021074959 conn(0x7f1bd805bd20 msgr2=0x7f1bd805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:35.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/1021074959 shutdown_connections 2021-11-21T22:01:35.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/1021074959 wait complete. 2021-11-21T22:01:35.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 Processor -- start 2021-11-21T22:01:35.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- start start 2021-11-21T22:01:35.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80a8670 0x7f1bd814a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:35.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bd80b11c0 0x7f1bd813ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:35.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd8147c10 0x7f1bd8156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:35.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81563c0 con 0x7f1bd8147c10 2021-11-21T22:01:35.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8145770 con 0x7f1bd80b11c0 2021-11-21T22:01:35.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81489e0 con 0x7f1bd80a8670 2021-11-21T22:01:35.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd8147c10 0x7f1bd8156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:35.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd8147c10 0x7f1bd8156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51038/0 (socket says 172.21.2.116:51038) 2021-11-21T22:01:35.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 -- 172.21.2.116:0/2655866777 learned_addr learned my addr 172.21.2.116:0/2655866777 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:35.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 -- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80a8670 msgr2=0x7f1bd814a000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:35.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80a8670 0x7f1bd814a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 -- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bd80b11c0 msgr2=0x7f1bd813ed80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:35.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bd80b11c0 0x7f1bd813ed80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 -- 172.21.2.116:0/2655866777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc00b1e0 con 0x7f1bd8147c10 2021-11-21T22:01:35.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bddcd2700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd8147c10 0x7f1bd8156a50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f1bd00032f0 tx=0x7f1bd00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:35.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bca7fc700 1 -- 172.21.2.116:0/2655866777 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bd000f760 con 0x7f1bd8147c10 2021-11-21T22:01:35.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bca7fc700 1 -- 172.21.2.116:0/2655866777 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd000f8c0 con 0x7f1bd8147c10 2021-11-21T22:01:35.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bca7fc700 1 -- 172.21.2.116:0/2655866777 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bd000fc20 con 0x7f1bd8147c10 2021-11-21T22:01:35.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd815a3a0 con 0x7f1bd8147c10 2021-11-21T22:01:35.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd81565b0 con 0x7f1bd8147c10 2021-11-21T22:01:35.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd8156770 con 0x7f1bd8147c10 2021-11-21T22:01:35.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bca7fc700 1 -- 172.21.2.116:0/2655866777 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1bd001a460 con 0x7f1bd8147c10 2021-11-21T22:01:35.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bca7fc700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1bb80536f0 0x7f1bb8055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:35.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bca7fc700 1 -- 172.21.2.116:0/2655866777 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f1bd006bcf0 con 0x7f1bd8147c10 2021-11-21T22:01:35.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.034+0000 7f1bdd4d1700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1bb80536f0 0x7f1bb8055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:35.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.038+0000 7f1bdd4d1700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1bb80536f0 0x7f1bb8055bb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f1bd4002f70 tx=0x7f1bd4006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:35.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.038+0000 7f1bca7fc700 1 -- 172.21.2.116:0/2655866777 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1bd0042e40 con 0x7f1bd8147c10 2021-11-21T22:01:35.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.122+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1bd80b20b0 con 0x7f1bd8147c10 2021-11-21T22:01:35.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.970+0000 7f1bca7fc700 1 -- 172.21.2.116:0/2655866777 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 173+0+0 (secure 0 0 0) 0x7f1bd0046460 con 0x7f1bd8147c10 2021-11-21T22:01:35.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1bb80536f0 msgr2=0x7f1bb8055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:35.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1bb80536f0 0x7f1bb8055bb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f1bd4002f70 tx=0x7f1bd4006a00).stop 2021-11-21T22:01:35.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd8147c10 msgr2=0x7f1bd8156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:35.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd8147c10 0x7f1bd8156a50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f1bd00032f0 tx=0x7f1bd00091d0).stop 2021-11-21T22:01:35.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 shutdown_connections 2021-11-21T22:01:35.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1bb80536f0 0x7f1bb8055bb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:35.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1bd8147c10 0x7f1bd8156a50 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:35.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bd80b11c0 0x7f1bd813ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 --2- 172.21.2.116:0/2655866777 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bd80a8670 0x7f1bd814a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:35.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 >> 172.21.2.116:0/2655866777 conn(0x7f1bd805bd20 msgr2=0x7f1bd80b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:35.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 shutdown_connections 2021-11-21T22:01:35.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:35.974+0000 7f1bdf735700 1 -- 172.21.2.116:0/2655866777 wait complete. 2021-11-21T22:01:35.981 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:01:35.987 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T22:01:36.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.034+0000 7fe171ed9700 1 Processor -- start 2021-11-21T22:01:36.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.034+0000 7fe171ed9700 1 -- start start 2021-11-21T22:01:36.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.034+0000 7fe171ed9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0abce0 0x7fe16c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.034+0000 7fe171ed9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c147e80 con 0x7fe16c0ac600 2021-11-21T22:01:36.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.034+0000 7fe171ed9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c0af5a0 con 0x7fe16c144810 2021-11-21T22:01:36.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.034+0000 7fe171ed9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0b1490 con 0x7fe16c0abce0 2021-11-21T22:01:36.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe16c0ac600 0x7fe16c0acab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34766/0 (socket says 172.21.2.116:34766) 2021-11-21T22:01:36.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 -- 172.21.2.116:0/3819179110 learned_addr learned my addr 172.21.2.116:0/3819179110 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:36.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919891537 0 0) 0x7fe16c147e80 con 0x7fe16c0ac600 2021-11-21T22:01:36.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe14c003260 con 0x7fe16c0ac600 2021-11-21T22:01:36.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16affd700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0abce0 0x7fe16c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:36.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391251063 0 0) 0x7fe14c003260 con 0x7fe16c0ac600 2021-11-21T22:01:36.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0abce0 msgr2=0x7fe16c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:36.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0abce0 0x7fe16c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:36.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 >> v1:172.21.2.117:6789/0 conn(0x7fe16c144810 legacy=0x7fe16c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:36.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0b1fd0 con 0x7fe16c0ac600 2021-11-21T22:01:36.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe1640019e0 con 0x7fe16c0ac600 2021-11-21T22:01:36.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 >> v1:172.21.2.116:6789/0 conn(0x7fe16c0ac600 legacy=0x7fe16c0acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe14c003780 0x7fe14c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0abce0 0x7fe14c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe14c0049f0 0x7fe14c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0b1490 con 0x7fe14c0049f0 2021-11-21T22:01:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c00b860 con 0x7fe14c003780 2021-11-21T22:01:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c00ba20 con 0x7fe16c0abce0 2021-11-21T22:01:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe14c0049f0 0x7fe14c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:36.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 -- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0abce0 msgr2=0x7fe14c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0abce0 0x7fe14c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 -- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe14c003780 msgr2=0x7fe14c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe14c003780 0x7fe14c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 -- 172.21.2.116:0/3819179110 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c003260 con 0x7fe14c0049f0 2021-11-21T22:01:36.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe14c0049f0 0x7fe14c004e70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fe16c0ac4e0 tx=0x7fe16c146e80).ready entity=mon.0 client_cookie=6ab3dfc46381f1ff server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:36.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe16400cd40 con 0x7fe14c0049f0 2021-11-21T22:01:36.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe164005990 con 0x7fe14c0049f0 2021-11-21T22:01:36.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16a7fc700 1 -- 172.21.2.116:0/3819179110 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe16400cea0 con 0x7fe14c0049f0 2021-11-21T22:01:36.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/3819179110 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe14c0049f0 msgr2=0x7fe14c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 --2- 172.21.2.116:0/3819179110 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe14c0049f0 0x7fe14c004e70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fe16c0ac4e0 tx=0x7fe16c146e80).stop 2021-11-21T22:01:36.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16bfff700 1 -- 172.21.2.116:0/3819179110 reap_dead start 2021-11-21T22:01:36.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/3819179110 shutdown_connections 2021-11-21T22:01:36.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/3819179110 >> 172.21.2.116:0/3819179110 conn(0x7fe16c05bd20 msgr2=0x7fe16c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:36.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/3819179110 shutdown_connections 2021-11-21T22:01:36.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/3819179110 wait complete. 2021-11-21T22:01:36.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 Processor -- start 2021-11-21T22:01:36.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- start start 2021-11-21T22:01:36.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe16c13eb80 0x7fe16c15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0b4380 0x7fe16c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe16c0b4800 0x7fe16c145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c13ef60 con 0x7fe16c13eb80 2021-11-21T22:01:36.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c159a00 con 0x7fe16c0b4800 2021-11-21T22:01:36.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c159ca0 con 0x7fe16c0b4380 2021-11-21T22:01:36.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe16c13eb80 0x7fe16c15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:36.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe16c13eb80 0x7fe16c15a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51058/0 (socket says 172.21.2.116:51058) 2021-11-21T22:01:36.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 -- 172.21.2.116:0/807533369 learned_addr learned my addr 172.21.2.116:0/807533369 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:36.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 -- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0b4380 msgr2=0x7fe16c1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0b4380 0x7fe16c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 -- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe16c0b4800 msgr2=0x7fe16c145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe16c0b4800 0x7fe16c145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 -- 172.21.2.116:0/807533369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c159f40 con 0x7fe16c13eb80 2021-11-21T22:01:36.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe16b7fe700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe16c13eb80 0x7fe16c15a3e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fe15c001fa0 tx=0x7fe15c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:36.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe168ff9700 1 -- 172.21.2.116:0/807533369 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15c00ce80 con 0x7fe16c13eb80 2021-11-21T22:01:36.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe168ff9700 1 -- 172.21.2.116:0/807533369 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c000e90 con 0x7fe16c13eb80 2021-11-21T22:01:36.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c142790 con 0x7fe16c13eb80 2021-11-21T22:01:36.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe168ff9700 1 -- 172.21.2.116:0/807533369 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15c00f660 con 0x7fe16c13eb80 2021-11-21T22:01:36.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c1569c0 con 0x7fe16c13eb80 2021-11-21T22:01:36.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.038+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c159810 con 0x7fe16c13eb80 2021-11-21T22:01:36.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.042+0000 7fe168ff9700 1 -- 172.21.2.116:0/807533369 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe15c010070 con 0x7fe16c13eb80 2021-11-21T22:01:36.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.042+0000 7fe168ff9700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe150053600 0x7fe150055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.042+0000 7fe168ff9700 1 -- 172.21.2.116:0/807533369 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe15c06ab30 con 0x7fe16c13eb80 2021-11-21T22:01:36.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.042+0000 7fe16affd700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe150053600 0x7fe150055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:36.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.042+0000 7fe16affd700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe150053600 0x7fe150055ac0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fe15400a000 tx=0x7fe154008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:36.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.042+0000 7fe168ff9700 1 -- 172.21.2.116:0/807533369 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe15c045e60 con 0x7fe16c13eb80 2021-11-21T22:01:36.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.126+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe16c0af060 con 0x7fe16c13eb80 2021-11-21T22:01:36.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.130+0000 7fe168ff9700 1 -- 172.21.2.116:0/807533369 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe15c041f70 con 0x7fe16c13eb80 2021-11-21T22:01:36.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe150053600 msgr2=0x7fe150055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe150053600 0x7fe150055ac0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fe15400a000 tx=0x7fe154008040).stop 2021-11-21T22:01:36.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe16c13eb80 msgr2=0x7fe16c15a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe16c13eb80 0x7fe16c15a3e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fe15c001fa0 tx=0x7fe15c007c50).stop 2021-11-21T22:01:36.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 shutdown_connections 2021-11-21T22:01:36.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe16c0b4800 0x7fe16c145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe16c0b4380 0x7fe16c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe150053600 0x7fe150055ac0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:36.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 --2- 172.21.2.116:0/807533369 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe16c13eb80 0x7fe16c15a3e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:36.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 >> 172.21.2.116:0/807533369 conn(0x7fe16c05bd20 msgr2=0x7fe16c148390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:36.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 shutdown_connections 2021-11-21T22:01:36.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.134+0000 7fe171ed9700 1 -- 172.21.2.116:0/807533369 wait complete. 2021-11-21T22:01:36.144 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2021-11-21T22:01:36.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff01e9d1700 1 Processor -- start 2021-11-21T22:01:36.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff01e9d1700 1 -- start start 2021-11-21T22:01:36.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff01e9d1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff0180a86b0 0x7ff01814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff01e9d1700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff018062410 0x7ff0181443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff01e9d1700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0180a6e80 con 0x7ff0180a6aa0 2021-11-21T22:01:36.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff01e9d1700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0180a8bc0 con 0x7ff0180a86b0 2021-11-21T22:01:36.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff01e9d1700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01814a7b0 con 0x7ff018062410 2021-11-21T22:01:36.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff0180a86b0 0x7ff01814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:36.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff0180a86b0 0x7ff01814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51066/0 (socket says 172.21.2.116:51066) 2021-11-21T22:01:36.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 -- 172.21.2.116:0/3759933390 learned_addr learned my addr 172.21.2.116:0/3759933390 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:36.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 -- 172.21.2.116:0/3759933390 >> v2:172.21.2.117:3300/0 conn(0x7ff018062410 msgr2=0x7ff0181443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:36.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 --2- 172.21.2.116:0/3759933390 >> v2:172.21.2.117:3300/0 conn(0x7ff018062410 0x7ff0181443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 -- 172.21.2.116:0/3759933390 >> v1:172.21.2.117:6789/0 conn(0x7ff0180a6aa0 legacy=0x7ff0181470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:36.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 -- 172.21.2.116:0/3759933390 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01814aa10 con 0x7ff0180a86b0 2021-11-21T22:01:36.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.190+0000 7ff0177fe700 1 --2- 172.21.2.116:0/3759933390 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a86b0 0x7ff01814b150 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7ff018061ab0 tx=0x7ff008006eb0).ready entity=mon.0 client_cookie=1e9c7487e8dba85d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:36.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff016ffd700 1 -- 172.21.2.116:0/3759933390 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff00800d5c0 con 0x7ff0180a86b0 2021-11-21T22:01:36.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff016ffd700 1 -- 172.21.2.116:0/3759933390 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008000e90 con 0x7ff0180a86b0 2021-11-21T22:01:36.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff016ffd700 1 -- 172.21.2.116:0/3759933390 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff00800d910 con 0x7ff0180a86b0 2021-11-21T22:01:36.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/3759933390 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a86b0 msgr2=0x7ff01814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/3759933390 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a86b0 0x7ff01814b150 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7ff018061ab0 tx=0x7ff008006eb0).stop 2021-11-21T22:01:36.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/3759933390 shutdown_connections 2021-11-21T22:01:36.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/3759933390 >> v2:172.21.2.117:3300/0 conn(0x7ff018062410 0x7ff0181443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/3759933390 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a86b0 0x7ff01814b150 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:36.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/3759933390 >> 172.21.2.116:0/3759933390 conn(0x7ff01805bd20 msgr2=0x7ff01805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/3759933390 shutdown_connections 2021-11-21T22:01:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/3759933390 wait complete. 2021-11-21T22:01:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 Processor -- start 2021-11-21T22:01:36.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- start start 2021-11-21T22:01:36.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff018062410 0x7ff018144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a6aa0 0x7ff0180aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0180a86b0 0x7ff0180ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180639d0 con 0x7ff0180a6aa0 2021-11-21T22:01:36.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff018063c70 con 0x7ff0180a86b0 2021-11-21T22:01:36.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff018063f10 con 0x7ff018062410 2021-11-21T22:01:36.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a6aa0 0x7ff0180aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:36.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a6aa0 0x7ff0180aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51068/0 (socket says 172.21.2.116:51068) 2021-11-21T22:01:36.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 -- 172.21.2.116:0/57467318 learned_addr learned my addr 172.21.2.116:0/57467318 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:36.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 -- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff018062410 msgr2=0x7ff018144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff018062410 0x7ff018144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 -- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0180a86b0 msgr2=0x7ff0180ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0180a86b0 0x7ff0180ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 -- 172.21.2.116:0/57467318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00800b1e0 con 0x7ff0180a6aa0 2021-11-21T22:01:36.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0177fe700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a6aa0 0x7ff0180aaea0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7ff018061ab0 tx=0x7ff008015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:36.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0157fa700 1 -- 172.21.2.116:0/57467318 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff008011920 con 0x7ff0180a6aa0 2021-11-21T22:01:36.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0157fa700 1 -- 172.21.2.116:0/57467318 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008011a80 con 0x7ff0180a6aa0 2021-11-21T22:01:36.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0180660e0 con 0x7ff0180a6aa0 2021-11-21T22:01:36.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0180aa2c0 con 0x7ff0180a6aa0 2021-11-21T22:01:36.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0157fa700 1 -- 172.21.2.116:0/57467318 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff008011de0 con 0x7ff0180a6aa0 2021-11-21T22:01:36.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff018063830 con 0x7ff0180a6aa0 2021-11-21T22:01:36.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0157fa700 1 -- 172.21.2.116:0/57467318 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff008025030 con 0x7ff0180a6aa0 2021-11-21T22:01:36.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0157fa700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feffc053660 0x7feffc055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:36.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.194+0000 7ff0157fa700 1 -- 172.21.2.116:0/57467318 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff00806cb70 con 0x7ff0180a6aa0 2021-11-21T22:01:36.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.198+0000 7ff017fff700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feffc053660 0x7feffc055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:36.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.198+0000 7ff017fff700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feffc053660 0x7feffc055b20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7ff000002fb0 tx=0x7ff0000069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:36.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.198+0000 7ff0157fa700 1 -- 172.21.2.116:0/57467318 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff018063830 con 0x7ff0180a6aa0 2021-11-21T22:01:36.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.310+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7ff0180640d0 con 0x7ff0180a6aa0 2021-11-21T22:01:36.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.978+0000 7ff0157fa700 1 -- 172.21.2.116:0/57467318 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v309) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff0080230d0 con 0x7ff0180a6aa0 2021-11-21T22:01:36.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feffc053660 msgr2=0x7feffc055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feffc053660 0x7feffc055b20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7ff000002fb0 tx=0x7ff0000069b0).stop 2021-11-21T22:01:36.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a6aa0 msgr2=0x7ff0180aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:36.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a6aa0 0x7ff0180aaea0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7ff018061ab0 tx=0x7ff008015000).stop 2021-11-21T22:01:36.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 shutdown_connections 2021-11-21T22:01:36.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0180a86b0 0x7ff0180ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7feffc053660 0x7feffc055b20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:36.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0180a6aa0 0x7ff0180aaea0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:36.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 --2- 172.21.2.116:0/57467318 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff018062410 0x7ff018144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:36.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 >> 172.21.2.116:0/57467318 conn(0x7ff01805bd20 msgr2=0x7ff018149140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:36.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 shutdown_connections 2021-11-21T22:01:36.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:36.982+0000 7ff01e9d1700 1 -- 172.21.2.116:0/57467318 wait complete. 2021-11-21T22:01:36.992 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2021-11-21T22:01:36.993 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:01:36.993 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2021-11-21T22:01:37.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 Processor -- start 2021-11-21T22:01:37.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- start start 2021-11-21T22:01:37.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0dc00a9d00 0x7f0dc013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:37.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0dc005c840 0x7f0dc013e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:37.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc0144970 con 0x7f0dc005cef0 2021-11-21T22:01:37.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc013e750 con 0x7f0dc005c840 2021-11-21T22:01:37.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc00b1200 con 0x7f0dc00a9d00 2021-11-21T22:01:37.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0dc005c840 0x7f0dc013e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:37.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0dc005c840 0x7f0dc013e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51078/0 (socket says 172.21.2.116:51078) 2021-11-21T22:01:37.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/3051880747 learned_addr learned my addr 172.21.2.116:0/3051880747 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:37.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/3051880747 >> v2:172.21.2.117:3300/0 conn(0x7f0dc00a9d00 msgr2=0x7f0dc013dd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:37.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- 172.21.2.116:0/3051880747 >> v2:172.21.2.117:3300/0 conn(0x7f0dc00a9d00 0x7f0dc013dd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:37.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/3051880747 >> v1:172.21.2.117:6789/0 conn(0x7f0dc005cef0 legacy=0x7f0dc00b17b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:37.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/3051880747 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc00aa0e0 con 0x7f0dc005c840 2021-11-21T22:01:37.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- 172.21.2.116:0/3051880747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005c840 0x7f0dc013e260 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f0dc0061ab0 tx=0x7f0da8006eb0).ready entity=mon.0 client_cookie=213e95602287f326 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:37.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbeffd700 1 -- 172.21.2.116:0/3051880747 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da800d5c0 con 0x7f0dc005c840 2021-11-21T22:01:37.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbeffd700 1 -- 172.21.2.116:0/3051880747 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da8000e90 con 0x7f0dc005c840 2021-11-21T22:01:37.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbeffd700 1 -- 172.21.2.116:0/3051880747 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da800d910 con 0x7f0dc005c840 2021-11-21T22:01:37.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- 172.21.2.116:0/3051880747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005c840 msgr2=0x7f0dc013e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:37.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- 172.21.2.116:0/3051880747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005c840 0x7f0dc013e260 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f0dc0061ab0 tx=0x7f0da8006eb0).stop 2021-11-21T22:01:37.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- 172.21.2.116:0/3051880747 shutdown_connections 2021-11-21T22:01:37.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- 172.21.2.116:0/3051880747 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005c840 0x7f0dc013e260 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:37.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- 172.21.2.116:0/3051880747 >> v2:172.21.2.117:3300/0 conn(0x7f0dc00a9d00 0x7f0dc013dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:37.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- 172.21.2.116:0/3051880747 >> 172.21.2.116:0/3051880747 conn(0x7f0dc005bd20 msgr2=0x7f0dc005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:37.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- 172.21.2.116:0/3051880747 shutdown_connections 2021-11-21T22:01:37.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- 172.21.2.116:0/3051880747 wait complete. 2021-11-21T22:01:37.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 Processor -- start 2021-11-21T22:01:37.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- start start 2021-11-21T22:01:37.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0dc005c840 0x7f0dc00ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:37.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005cef0 0x7f0dc0066d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:37.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0dc00a9d00 0x7f0dc0065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:37.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00ae8f0 con 0x7f0dc005cef0 2021-11-21T22:01:37.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0067270 con 0x7f0dc00a9d00 2021-11-21T22:01:37.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dc651a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00adf10 con 0x7f0dc005c840 2021-11-21T22:01:37.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005cef0 0x7f0dc0066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:37.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005cef0 0x7f0dc0066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51082/0 (socket says 172.21.2.116:51082) 2021-11-21T22:01:37.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/2356585204 learned_addr learned my addr 172.21.2.116:0/2356585204 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:37.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0dc005c840 msgr2=0x7f0dc00ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:37.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0dc005c840 0x7f0dc00ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:37.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0dc00a9d00 msgr2=0x7f0dc0065ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:37.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.042+0000 7f0dbf7fe700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0dc00a9d00 0x7f0dc0065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:37.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbf7fe700 1 -- 172.21.2.116:0/2356585204 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da800b1e0 con 0x7f0dc005cef0 2021-11-21T22:01:37.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbf7fe700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005cef0 0x7f0dc0066d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f0dc0061ab0 tx=0x7f0da8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:37.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbd7fa700 1 -- 172.21.2.116:0/2356585204 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da8002f50 con 0x7f0dc005cef0 2021-11-21T22:01:37.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbd7fa700 1 -- 172.21.2.116:0/2356585204 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da8006780 con 0x7f0dc005cef0 2021-11-21T22:01:37.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbd7fa700 1 -- 172.21.2.116:0/2356585204 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da801e460 con 0x7f0dc005cef0 2021-11-21T22:01:37.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc00ae1b0 con 0x7f0dc005cef0 2021-11-21T22:01:37.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc0067600 con 0x7f0dc005cef0 2021-11-21T22:01:37.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc00abb80 con 0x7f0dc005cef0 2021-11-21T22:01:37.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbd7fa700 1 -- 172.21.2.116:0/2356585204 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0da8025030 con 0x7f0dc005cef0 2021-11-21T22:01:37.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbd7fa700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0da4053610 0x7f0da4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:37.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbd7fa700 1 -- 172.21.2.116:0/2356585204 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0da806c9e0 con 0x7f0dc005cef0 2021-11-21T22:01:37.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbffff700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0da4053610 0x7f0da4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:37.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.046+0000 7f0dbffff700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0da4053610 0x7f0da4055ad0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f0db0002fb0 tx=0x7f0db00069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:37.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.050+0000 7f0dbd7fa700 1 -- 172.21.2.116:0/2356585204 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0da8043d10 con 0x7f0dc005cef0 2021-11-21T22:01:37.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.158+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f0dc00573d0 con 0x7f0dc005cef0 2021-11-21T22:01:37.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dbd7fa700 1 -- 172.21.2.116:0/2356585204 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v310) v1 ==== 130+0+0 (secure 0 0 0) 0x7f0da8023020 con 0x7f0dc005cef0 2021-11-21T22:01:37.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0da4053610 msgr2=0x7f0da4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:37.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0da4053610 0x7f0da4055ad0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f0db0002fb0 tx=0x7f0db00069d0).stop 2021-11-21T22:01:37.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005cef0 msgr2=0x7f0dc0066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:37.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005cef0 0x7f0dc0066d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f0dc0061ab0 tx=0x7f0da8015000).stop 2021-11-21T22:01:37.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 shutdown_connections 2021-11-21T22:01:37.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0dc00a9d00 0x7f0dc0065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:37.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0da4053610 0x7f0da4055ad0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:37.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0dc005cef0 0x7f0dc0066d80 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:37.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 --2- 172.21.2.116:0/2356585204 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0dc005c840 0x7f0dc00ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:37.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 >> 172.21.2.116:0/2356585204 conn(0x7f0dc005bd20 msgr2=0x7f0dc00b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:37.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 shutdown_connections 2021-11-21T22:01:37.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:37.990+0000 7f0dc651a700 1 -- 172.21.2.116:0/2356585204 wait complete. 2021-11-21T22:01:38.001 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T22:01:38.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f759295a700 1 Processor -- start 2021-11-21T22:01:38.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f759295a700 1 -- start start 2021-11-21T22:01:38.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f759295a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f758c144810 0x7f758c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f759295a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 0x7f758c0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f759295a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c147e80 con 0x7f758c0abce0 2021-11-21T22:01:38.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f759295a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0b1490 con 0x7f758c144810 2021-11-21T22:01:38.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f759295a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b1fd0 con 0x7f758c0ac600 2021-11-21T22:01:38.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f758b7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f758c0abce0 0x7f758c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34814/0 (socket says 172.21.2.116:34814) 2021-11-21T22:01:38.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f758b7fe700 1 -- 172.21.2.116:0/2637294305 learned_addr learned my addr 172.21.2.116:0/2637294305 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:38.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821127805 0 0) 0x7f758c147e80 con 0x7f758c0abce0 2021-11-21T22:01:38.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f756c003260 con 0x7f758c0abce0 2021-11-21T22:01:38.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f758bfff700 1 --2- 172.21.2.116:0/2637294305 >> v2:172.21.2.117:3300/0 conn(0x7f758c144810 0x7f758c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f7590ef7700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 0x7f758c0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.050+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3802281549 0 0) 0x7f756c003260 con 0x7f758c0abce0 2021-11-21T22:01:38.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 msgr2=0x7f758c0aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 0x7f758c0aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 >> v2:172.21.2.117:3300/0 conn(0x7f758c144810 msgr2=0x7f758c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 --2- 172.21.2.116:0/2637294305 >> v2:172.21.2.117:3300/0 conn(0x7f758c144810 0x7f758c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0af5a0 con 0x7f758c0abce0 2021-11-21T22:01:38.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f7574001660 con 0x7f758c0abce0 2021-11-21T22:01:38.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 >> v1:172.21.2.116:6789/0 conn(0x7f758c0abce0 legacy=0x7f758c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f756c003750 0x7f756c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 0x7f756c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c0049f0 0x7f756c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b1490 con 0x7f756c003750 2021-11-21T22:01:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b1fd0 con 0x7f756c0049f0 2021-11-21T22:01:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c00b7a0 con 0x7f758c0ac600 2021-11-21T22:01:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758bfff700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f756c003750 0x7f756c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f7590ef7700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c0049f0 0x7f756c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 0x7f756c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758bfff700 1 -- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 msgr2=0x7f756c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758bfff700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 0x7f756c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758bfff700 1 -- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c0049f0 msgr2=0x7f756c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758bfff700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c0049f0 0x7f756c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758bfff700 1 -- 172.21.2.116:0/2637294305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c003260 con 0x7f756c003750 2021-11-21T22:01:38.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c0ac600 0x7f756c004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:38.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f7590ef7700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c0049f0 0x7f756c004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:38.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758bfff700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f756c003750 0x7f756c003b70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f757c00a000 tx=0x7f757c008040).ready entity=mon.0 client_cookie=7d6d25ce1be76218 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f757c013670 con 0x7f756c003750 2021-11-21T22:01:38.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c0137d0 con 0x7f756c003750 2021-11-21T22:01:38.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758affd700 1 -- 172.21.2.116:0/2637294305 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f757c013b30 con 0x7f756c003750 2021-11-21T22:01:38.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- 172.21.2.116:0/2637294305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f756c003750 msgr2=0x7f756c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 --2- 172.21.2.116:0/2637294305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f756c003750 0x7f756c003b70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f757c00a000 tx=0x7f757c008040).stop 2021-11-21T22:01:38.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f7590ef7700 1 -- 172.21.2.116:0/2637294305 reap_dead start 2021-11-21T22:01:38.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- 172.21.2.116:0/2637294305 shutdown_connections 2021-11-21T22:01:38.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- 172.21.2.116:0/2637294305 >> 172.21.2.116:0/2637294305 conn(0x7f758c05bd20 msgr2=0x7f758c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- 172.21.2.116:0/2637294305 shutdown_connections 2021-11-21T22:01:38.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- 172.21.2.116:0/2637294305 wait complete. 2021-11-21T22:01:38.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 Processor -- start 2021-11-21T22:01:38.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- start start 2021-11-21T22:01:38.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c003750 0x7f758c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0b49d0 0x7f758c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c13cd00 0x7f758c13d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b3490 con 0x7f758c0b49d0 2021-11-21T22:01:38.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c13d680 con 0x7f756c003750 2021-11-21T22:01:38.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b3f40 con 0x7f758c13cd00 2021-11-21T22:01:38.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0b49d0 0x7f758c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0b49d0 0x7f758c156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51102/0 (socket says 172.21.2.116:51102) 2021-11-21T22:01:38.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 -- 172.21.2.116:0/112017803 learned_addr learned my addr 172.21.2.116:0/112017803 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:38.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 -- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c13cd00 msgr2=0x7f758c13d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c13cd00 0x7f758c13d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 -- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c003750 msgr2=0x7f758c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c003750 0x7f758c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 -- 172.21.2.116:0/112017803 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0b36c0 con 0x7f758c0b49d0 2021-11-21T22:01:38.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f758b7fe700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0b49d0 0x7f758c156ae0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f7574008000 tx=0x7f7574009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f75897fa700 1 -- 172.21.2.116:0/112017803 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7574007170 con 0x7f758c0b49d0 2021-11-21T22:01:38.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c14aab0 con 0x7f758c0b49d0 2021-11-21T22:01:38.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c0b2b20 con 0x7f758c0b49d0 2021-11-21T22:01:38.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f75897fa700 1 -- 172.21.2.116:0/112017803 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75740072d0 con 0x7f758c0b49d0 2021-11-21T22:01:38.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.054+0000 7f75897fa700 1 -- 172.21.2.116:0/112017803 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7574004920 con 0x7f758c0b49d0 2021-11-21T22:01:38.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.058+0000 7f75897fa700 1 -- 172.21.2.116:0/112017803 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7574004a80 con 0x7f758c0b49d0 2021-11-21T22:01:38.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.058+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c0af060 con 0x7f758c0b49d0 2021-11-21T22:01:38.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.058+0000 7f75897fa700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75700536a0 0x7f7570055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.058+0000 7f75897fa700 1 -- 172.21.2.116:0/112017803 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f757406bc50 con 0x7f758c0b49d0 2021-11-21T22:01:38.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.058+0000 7f758bfff700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75700536a0 0x7f7570055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.058+0000 7f758bfff700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75700536a0 0x7f7570055b60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f757c00a000 tx=0x7f757c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.058+0000 7f75897fa700 1 -- 172.21.2.116:0/112017803 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7574042f20 con 0x7f758c0b49d0 2021-11-21T22:01:38.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.142+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f758c0573d0 con 0x7f758c0b49d0 2021-11-21T22:01:38.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f75897fa700 1 -- 172.21.2.116:0/112017803 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7574046540 con 0x7f758c0b49d0 2021-11-21T22:01:38.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75700536a0 msgr2=0x7f7570055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75700536a0 0x7f7570055b60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f757c00a000 tx=0x7f757c008040).stop 2021-11-21T22:01:38.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0b49d0 msgr2=0x7f758c156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0b49d0 0x7f758c156ae0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f7574008000 tx=0x7f7574009040).stop 2021-11-21T22:01:38.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 shutdown_connections 2021-11-21T22:01:38.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f758c13cd00 0x7f758c13d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f75700536a0 0x7f7570055b60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f758c0b49d0 0x7f758c156ae0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 --2- 172.21.2.116:0/112017803 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f756c003750 0x7f758c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 >> 172.21.2.116:0/112017803 conn(0x7f758c05bd20 msgr2=0x7f758c149000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 shutdown_connections 2021-11-21T22:01:38.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.146+0000 7f759295a700 1 -- 172.21.2.116:0/112017803 wait complete. 2021-11-21T22:01:38.158 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:01:38.159 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:38.174 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:38.185 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:38.199 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:38.213 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:01:38.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:38.214 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:38.214 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:38.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:01:38.219 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:01:38.220 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:38.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:38.221 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:38.221 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:38.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:01:38.226 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:01:38.227 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:38.227 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:01:38.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e71971700 1 Processor -- start 2021-11-21T22:01:38.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e71971700 1 -- start start 2021-11-21T22:01:38.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e71971700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c0a86b0 0x7f0e6c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e71971700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e6c062410 0x7f0e6c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e71971700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c0a6e80 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e71971700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6c0a8bc0 con 0x7f0e6c062410 2021-11-21T22:01:38.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e71971700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c14a7b0 con 0x7f0e6c0a86b0 2021-11-21T22:01:38.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e6c062410 0x7f0e6c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0e6c062410 0x7f0e6c1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51114/0 (socket says 172.21.2.116:51114) 2021-11-21T22:01:38.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 -- 172.21.2.116:0/590458316 learned_addr learned my addr 172.21.2.116:0/590458316 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:38.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 -- 172.21.2.116:0/590458316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c0a86b0 msgr2=0x7f0e6c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 --2- 172.21.2.116:0/590458316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c0a86b0 0x7f0e6c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 -- 172.21.2.116:0/590458316 >> v1:172.21.2.117:6789/0 conn(0x7f0e6c0a6aa0 legacy=0x7f0e6c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 -- 172.21.2.116:0/590458316 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c14aa10 con 0x7f0e6c062410 2021-11-21T22:01:38.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e6b7fe700 1 --2- 172.21.2.116:0/590458316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c062410 0x7f0e6c1443f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f0e6c061ab0 tx=0x7f0e5c009290).ready entity=mon.0 client_cookie=9351154993572666 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e69ffb700 1 -- 172.21.2.116:0/590458316 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e5c00b830 con 0x7f0e6c062410 2021-11-21T22:01:38.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.278+0000 7f0e69ffb700 1 -- 172.21.2.116:0/590458316 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5c00b990 con 0x7f0e6c062410 2021-11-21T22:01:38.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e69ffb700 1 -- 172.21.2.116:0/590458316 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e5c00bb00 con 0x7f0e6c062410 2021-11-21T22:01:38.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/590458316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c062410 msgr2=0x7f0e6c1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 --2- 172.21.2.116:0/590458316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c062410 0x7f0e6c1443f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f0e6c061ab0 tx=0x7f0e5c009290).stop 2021-11-21T22:01:38.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/590458316 shutdown_connections 2021-11-21T22:01:38.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 --2- 172.21.2.116:0/590458316 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c062410 0x7f0e6c1443f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 --2- 172.21.2.116:0/590458316 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c0a86b0 0x7f0e6c14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/590458316 >> 172.21.2.116:0/590458316 conn(0x7f0e6c05bd20 msgr2=0x7f0e6c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/590458316 shutdown_connections 2021-11-21T22:01:38.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/590458316 wait complete. 2021-11-21T22:01:38.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 Processor -- start 2021-11-21T22:01:38.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- start start 2021-11-21T22:01:38.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c062410 0x7f0e6c14bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c0a6aa0 0x7f0e6c066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e6c0a86b0 0x7f0e6c066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0ab060 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0ab290 con 0x7f0e6c0a86b0 2021-11-21T22:01:38.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0671c0 con 0x7f0e6c062410 2021-11-21T22:01:38.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c0a6aa0 0x7f0e6c066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c0a6aa0 0x7f0e6c066420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51120/0 (socket says 172.21.2.116:51120) 2021-11-21T22:01:38.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 -- 172.21.2.116:0/1224677312 learned_addr learned my addr 172.21.2.116:0/1224677312 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:38.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 -- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c062410 msgr2=0x7f0e6c14bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c062410 0x7f0e6c14bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 -- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e6c0a86b0 msgr2=0x7f0e6c066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e6c0a86b0 0x7f0e6c066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 -- 172.21.2.116:0/1224677312 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c009530 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e6a7fc700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c0a6aa0 0x7f0e6c066420 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f0e64002f70 tx=0x7f0e64006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e63fff700 1 -- 172.21.2.116:0/1224677312 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e64013460 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e63fff700 1 -- 172.21.2.116:0/1224677312 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e640135c0 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c0aba80 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c063c90 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e63fff700 1 -- 172.21.2.116:0/1224677312 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e64013920 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44003410 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.282+0000 7f0e63fff700 1 -- 172.21.2.116:0/1224677312 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0e6401a060 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.286+0000 7f0e63fff700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e500536f0 0x7f0e50055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.286+0000 7f0e63fff700 1 -- 172.21.2.116:0/1224677312 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0e6406b7e0 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.286+0000 7f0e6affd700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e500536f0 0x7f0e50055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.286+0000 7f0e6affd700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e500536f0 0x7f0e50055bb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f0e54002f90 tx=0x7f0e540069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.286+0000 7f0e63fff700 1 -- 172.21.2.116:0/1224677312 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e64042ab0 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0e440036c0 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e63fff700 1 -- 172.21.2.116:0/1224677312 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+28829 (secure 0 0 0) 0x7f0e64018070 con 0x7f0e6c0a6aa0 2021-11-21T22:01:38.375 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:38.375 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:38.375 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 103, 2021-11-21T22:01:38.376 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:01:38.376 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:01:38.376 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:01:35.967494+0000", 2021-11-21T22:01:38.376 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:38.376 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:38.377 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:01:38.377 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:01:38.377 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:01:38.377 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:01:38.378 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:01:38.378 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:01:38.378 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:01:38.378 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:38.379 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:01:38.379 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:01:38.379 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:01:38.379 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:01:38.379 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:01:38.380 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 18, 2021-11-21T22:01:38.380 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:01:38.380 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:01:38.380 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:01:38.380 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:01:38.381 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:01:38.381 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.381 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:01:38.381 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:01:38.382 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:01:38.382 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:38.382 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:38.382 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:38.382 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:38.383 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:38.383 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:38.383 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:38.383 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:38.384 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:01:38.384 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:01:38.384 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:01:38.384 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:01:38.385 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:01:38.385 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:38.385 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:38.385 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:38.385 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:38.386 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:38.386 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:38.386 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:38.386 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.387 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:01:38.387 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:38.387 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:38.387 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:38.388 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:38.388 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:38.388 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:38.388 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:38.388 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:38.389 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:38.389 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:38.389 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:38.389 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:38.390 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:38.390 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:38.390 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:38.390 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:38.390 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:38.391 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:38.391 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:38.391 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:38.391 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:38.392 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:38.392 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:38.392 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:38.392 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:38.392 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:38.393 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.393 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:38.393 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:38.393 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:38.394 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:38.394 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:38.394 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:38.394 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:38.394 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:38.395 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:38.395 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:38.395 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:38.395 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:38.396 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:01:38.396 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.396 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:38.396 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:01:38.397 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.397 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.397 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.397 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:01:38.397 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:01:38.398 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:01:38.398 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:38.398 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:38.398 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:38.398 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:38.399 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:38.399 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:38.399 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:38.399 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:38.400 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:38.400 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:38.400 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:38.400 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:38.401 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:38.401 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:38.401 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:38.401 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:38.401 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:38.402 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:38.402 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:38.402 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:38.402 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.402 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:01:38.403 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:38.403 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:38.403 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:38.403 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:38.404 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:38.404 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:38.404 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:38.404 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:38.404 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:38.405 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:38.405 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:38.405 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:38.405 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:38.406 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:38.406 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:38.406 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:38.406 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:38.407 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:38.407 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:38.407 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:38.407 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:38.407 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:38.408 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:38.408 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:38.408 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:38.408 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:38.409 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.409 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:38.409 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:38.409 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:38.409 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:38.410 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:38.410 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:38.410 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:38.410 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:38.411 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:38.411 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:38.411 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:38.411 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:38.411 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:38.412 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:01:38.412 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.412 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.412 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:38.413 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:01:38.413 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.413 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:38.413 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:01:38.413 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.414 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.414 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.414 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.414 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.415 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.415 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.415 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:38.416 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.416 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.416 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.416 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.417 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.417 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.417 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:01:38.417 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.418 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.418 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.418 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.418 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:01:38.418 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.419 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.419 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.419 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.420 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.420 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.420 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.420 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:01:38.421 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.421 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.421 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.421 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.421 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:01:38.422 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.422 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.422 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.422 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.423 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.423 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.423 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.423 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.424 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:01:38.424 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.424 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.424 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.424 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.425 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:01:38.425 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.425 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.425 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.426 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.426 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.426 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.426 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.426 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.427 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:01:38.427 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.427 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.428 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.428 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:01:38.428 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:38.428 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.429 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.429 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.429 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:01:38.429 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:01:38.429 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:01:38.430 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:01:38.430 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.430 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.430 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.431 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.431 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.431 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.431 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:38.432 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:01:38.432 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.432 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.432 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.433 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.433 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.433 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.433 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.434 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:38.434 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.434 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.434 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.435 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.435 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.435 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.435 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:01:38.435 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.436 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.436 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.436 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.436 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:01:38.437 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.437 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.437 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.437 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.437 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.438 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.438 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.438 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.438 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:01:38.439 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.439 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.439 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.439 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.439 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:01:38.440 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.440 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.440 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.440 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.441 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.441 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.441 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.441 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.441 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:01:38.442 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.442 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.442 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.442 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.443 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:01:38.443 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.443 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.443 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.444 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.444 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.444 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.444 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.444 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.445 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:01:38.445 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.445 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.445 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.446 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.446 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:01:38.446 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:38.446 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.447 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.447 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.447 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:01:38.447 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:01:38.447 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:01:38.448 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:01:38.448 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.448 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.448 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.449 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.449 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.449 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:38.449 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:01:38.450 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.450 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.450 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.450 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.451 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.451 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.451 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.451 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:38.451 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.452 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.452 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.452 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.453 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.453 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:01:38.453 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.453 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.454 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.454 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.454 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:01:38.454 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.455 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.455 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.455 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.455 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.456 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.456 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.456 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.456 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:01:38.456 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.457 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.457 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.457 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.457 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:01:38.458 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.458 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.458 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.458 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.458 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.459 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.459 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.459 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.459 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:01:38.460 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.460 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.460 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.460 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.460 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:01:38.461 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.461 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.461 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.461 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.462 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.462 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.462 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.463 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:01:38.463 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.463 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.463 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.463 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.464 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:01:38.464 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:38.464 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.464 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.465 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.465 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:01:38.465 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:01:38.465 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:01:38.466 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:01:38.466 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.466 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.466 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.467 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.467 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.467 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.467 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:38.467 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:01:38.468 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.468 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.468 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.468 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.469 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.469 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.469 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.469 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:38.469 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.470 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.470 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.470 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.470 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.470 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.471 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:01:38.471 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.471 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.471 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.472 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.472 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:01:38.472 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.472 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.472 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.473 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.473 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.473 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.473 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.474 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.474 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:01:38.474 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.474 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.475 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.475 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:01:38.475 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.475 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.476 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.476 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.476 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.476 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.477 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.477 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:01:38.477 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.477 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.478 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.478 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:01:38.478 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.478 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.479 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.479 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.479 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.480 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.480 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.480 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:01:38.480 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.481 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.481 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.481 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.481 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:01:38.481 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:38.482 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.482 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.482 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.483 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:01:38.483 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:01:38.483 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:01:38.484 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:01:38.484 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.484 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.484 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.485 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.485 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.485 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.485 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:38.486 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:01:38.486 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.486 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.486 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.487 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.487 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.487 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.487 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.487 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 97, 2021-11-21T22:01:38.488 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.488 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.488 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.488 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.489 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.489 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.489 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:01:38.489 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.489 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.490 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.490 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.490 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:01:38.490 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.491 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.491 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.491 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.491 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.492 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.492 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.492 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.492 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:01:38.493 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.493 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.493 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.493 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.494 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:01:38.494 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.494 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.494 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.495 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.495 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.495 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.495 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.496 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.496 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:01:38.496 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.496 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.496 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.497 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.497 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:01:38.497 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.497 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.498 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.498 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.498 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.498 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.498 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.499 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:01:38.499 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.499 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.500 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.500 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.500 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:01:38.500 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:38.500 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.501 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.501 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.501 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:01:38.501 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:01:38.502 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:01:38.502 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:01:38.502 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.502 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.503 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.503 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.503 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.503 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.503 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:38.504 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:01:38.504 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.504 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.504 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.505 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.505 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.505 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.505 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.505 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 86, 2021-11-21T22:01:38.506 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.506 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.506 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.506 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.507 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.507 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.507 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:01:38.507 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.507 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.508 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.508 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.508 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:01:38.508 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.509 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.509 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.509 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.509 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.510 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.510 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:01:38.510 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.511 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.511 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.511 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.511 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:01:38.512 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.512 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.512 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.512 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.512 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.513 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.513 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.513 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.513 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:01:38.514 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.514 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.514 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.514 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.514 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:01:38.515 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.515 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.515 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.515 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.516 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.516 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.516 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.516 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.517 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:01:38.517 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.517 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.517 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.517 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.518 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:01:38.518 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:38.518 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.518 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.519 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.519 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:01:38.519 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:01:38.519 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:01:38.520 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:01:38.520 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.520 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.520 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.521 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.521 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.521 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.521 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:38.521 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:01:38.522 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.522 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.522 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.522 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.523 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.523 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.523 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.523 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 97, 2021-11-21T22:01:38.523 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.524 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.524 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.524 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.524 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.525 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.525 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:01:38.525 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.525 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.525 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.526 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.526 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:01:38.526 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.526 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.527 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.527 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.527 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.528 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:01:38.528 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.528 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.528 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.529 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.529 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:01:38.529 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.529 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.530 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.530 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.530 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.530 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.531 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.531 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.531 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:01:38.531 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.531 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.532 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.532 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.532 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:01:38.532 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.533 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.533 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.533 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.533 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.534 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.534 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.534 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.534 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:01:38.534 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.535 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.535 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.535 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:01:38.536 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:38.536 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.536 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.536 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.536 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:01:38.537 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:01:38.537 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:01:38.537 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:01:38.537 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.538 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.538 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.538 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.538 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.539 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.539 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:38.539 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:01:38.539 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:38.539 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:38.540 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:38.540 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:38.540 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:38.540 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:38.540 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:38.541 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 98, 2021-11-21T22:01:38.541 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:38.541 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:38.541 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:38.542 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.542 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.542 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.542 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:01:38.542 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.543 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.543 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.543 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.543 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:01:38.544 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.544 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.544 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.544 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.545 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:38.545 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.545 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.546 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:01:38.546 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.546 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.546 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.546 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.547 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:01:38.547 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.547 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.547 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.548 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.548 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:38.548 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.548 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.548 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.549 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:01:38.549 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.549 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.549 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.549 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.550 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:01:38.550 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.550 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.550 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.551 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.551 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:38.551 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:38.551 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.551 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:38.552 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:01:38.552 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.552 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.552 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.553 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:38.553 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:01:38.553 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:38.553 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.553 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.554 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.554 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:01:38.554 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:01:38.554 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:01:38.554 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:01:38.555 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:38.555 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:38.555 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:38.555 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:38.556 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.556 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:38.556 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:01:38.556 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.556 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:38.557 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.557 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.557 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.557 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.557 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.558 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:01:38.558 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.558 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.558 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.559 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:38.559 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.559 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.559 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.559 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.560 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.560 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:01:38.560 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.560 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.561 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.561 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:38.561 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.561 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.561 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.562 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.562 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.562 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:01:38.562 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.563 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.563 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.563 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:38.563 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.563 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.564 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.564 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.564 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.564 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:01:38.565 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.565 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.565 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.565 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:38.565 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.566 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.566 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.566 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.566 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.567 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:01:38.567 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.567 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.568 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:38.568 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.568 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.568 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.569 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.569 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.569 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:01:38.569 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.570 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.570 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.570 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:38.570 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.571 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.571 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.571 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.571 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.571 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:01:38.572 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.572 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.572 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:38.572 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:38.573 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:38.573 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:38.573 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:38.573 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:38.573 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:38.574 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:01:38.574 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:38.574 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.574 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:38.575 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:01:38.575 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:01:38.575 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:01:38.575 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:01:38.575 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:01:38.576 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/3417150056": "2021-11-22T22:01:32.667094+0000", 2021-11-21T22:01:38.576 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/3417150056": "2021-11-22T22:01:32.667094+0000" 2021-11-21T22:01:38.576 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.576 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:01:38.577 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:01:38.577 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:01:38.577 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:01:38.577 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:01:38.577 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:01:38.578 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:01:38.578 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:01:38.578 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:38.578 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:38.579 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:01:38.579 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:01:38.579 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:01:38.579 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:01:38.579 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:01:38.580 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:38.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e500536f0 msgr2=0x7f0e50055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e500536f0 0x7f0e50055bb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f0e54002f90 tx=0x7f0e540069b0).stop 2021-11-21T22:01:38.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c0a6aa0 msgr2=0x7f0e6c066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c0a6aa0 0x7f0e6c066420 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f0e64002f70 tx=0x7f0e64006df0).stop 2021-11-21T22:01:38.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 shutdown_connections 2021-11-21T22:01:38.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0e6c0a86b0 0x7f0e6c066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0e500536f0 0x7f0e50055bb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0e6c0a6aa0 0x7f0e6c066420 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 --2- 172.21.2.116:0/1224677312 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0e6c062410 0x7f0e6c14bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 >> 172.21.2.116:0/1224677312 conn(0x7f0e6c05bd20 msgr2=0x7f0e6c149020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 shutdown_connections 2021-11-21T22:01:38.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.370+0000 7f0e71971700 1 -- 172.21.2.116:0/1224677312 wait complete. 2021-11-21T22:01:38.585 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:38.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d965c700 1 Processor -- start 2021-11-21T22:01:38.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d965c700 1 -- start start 2021-11-21T22:01:38.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d965c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4097ec0 0x7f62d414af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d965c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d414a560 con 0x7f62d4099650 2021-11-21T22:01:38.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d965c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d414a800 con 0x7f62d40b5630 2021-11-21T22:01:38.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d965c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4144b40 con 0x7f62d4097ec0 2021-11-21T22:01:38.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d8e5b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f62d4099650 0x7f62d4099b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34852/0 (socket says 172.21.2.116:34852) 2021-11-21T22:01:38.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d8e5b700 1 -- 172.21.2.116:0/2431518398 learned_addr learned my addr 172.21.2.116:0/2431518398 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:38.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580309452 0 0) 0x7f62d414a560 con 0x7f62d4099650 2021-11-21T22:01:38.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c0003260 con 0x7f62d4099650 2021-11-21T22:01:38.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3657262214 0 0) 0x7f62c0003260 con 0x7f62d4099650 2021-11-21T22:01:38.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4097ec0 msgr2=0x7f62d414af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4097ec0 0x7f62d414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 >> v1:172.21.2.117:6789/0 conn(0x7f62d40b5630 legacy=0x7f62d4146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d414aaa0 con 0x7f62d4099650 2021-11-21T22:01:38.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f62cc0019e0 con 0x7f62d4099650 2021-11-21T22:01:38.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 >> v1:172.21.2.116:6789/0 conn(0x7f62d4099650 legacy=0x7f62d4099b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62c0003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c0004500 0x7f62c0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4097ec0 0x7f62c0004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4144b40 con 0x7f62c0003780 2021-11-21T22:01:38.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c000b960 con 0x7f62c0004500 2021-11-21T22:01:38.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c000bb20 con 0x7f62d4097ec0 2021-11-21T22:01:38.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d3fff700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62c0003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d3fff700 1 -- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4097ec0 msgr2=0x7f62c0004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d3fff700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4097ec0 0x7f62c0004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d3fff700 1 -- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c0004500 msgr2=0x7f62c0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:38.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d3fff700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62c0004500 0x7f62c0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d3fff700 1 -- 172.21.2.116:0/2431518398 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0003260 con 0x7f62c0003780 2021-11-21T22:01:38.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.638+0000 7f62d3fff700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62c0003ba0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f62c4001fa0 tx=0x7f62c4007a00).ready entity=mon.0 client_cookie=d73fea896d819587 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c4007070 con 0x7f62c0003780 2021-11-21T22:01:38.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c40071d0 con 0x7f62c0003780 2021-11-21T22:01:38.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d2ffd700 1 -- 172.21.2.116:0/2431518398 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c4006510 con 0x7f62c0003780 2021-11-21T22:01:38.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2431518398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 msgr2=0x7f62c0003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 --2- 172.21.2.116:0/2431518398 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62c0003ba0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f62c4001fa0 tx=0x7f62c4007a00).stop 2021-11-21T22:01:38.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d8e5b700 1 -- 172.21.2.116:0/2431518398 reap_dead start 2021-11-21T22:01:38.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2431518398 shutdown_connections 2021-11-21T22:01:38.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2431518398 >> 172.21.2.116:0/2431518398 conn(0x7f62d405c7f0 msgr2=0x7f62d40b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2431518398 shutdown_connections 2021-11-21T22:01:38.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2431518398 wait complete. 2021-11-21T22:01:38.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 Processor -- start 2021-11-21T22:01:38.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- start start 2021-11-21T22:01:38.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62d4063da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62d4067380 0x7f62d40ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4064fa0 0x7f62d40ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4064e20 con 0x7f62c0003780 2021-11-21T22:01:38.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d40aea70 con 0x7f62d4067380 2021-11-21T22:01:38.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d40629c0 con 0x7f62d4064fa0 2021-11-21T22:01:38.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62d4063da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62d4063da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51138/0 (socket says 172.21.2.116:51138) 2021-11-21T22:01:38.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 -- 172.21.2.116:0/2121472941 learned_addr learned my addr 172.21.2.116:0/2121472941 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:38.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 -- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4064fa0 msgr2=0x7f62d40ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4064fa0 0x7f62d40ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 -- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62d4067380 msgr2=0x7f62d40ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62d4067380 0x7f62d40ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 -- 172.21.2.116:0/2121472941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d4063060 con 0x7f62c0003780 2021-11-21T22:01:38.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d3fff700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62d4063da0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f62c4001fa0 tx=0x7f62c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d17fa700 1 -- 172.21.2.116:0/2121472941 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c4014810 con 0x7f62c0003780 2021-11-21T22:01:38.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d17fa700 1 -- 172.21.2.116:0/2121472941 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c4014970 con 0x7f62c0003780 2021-11-21T22:01:38.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d40666a0 con 0x7f62c0003780 2021-11-21T22:01:38.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d4066900 con 0x7f62c0003780 2021-11-21T22:01:38.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d17fa700 1 -- 172.21.2.116:0/2121472941 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c4014cd0 con 0x7f62c0003780 2021-11-21T22:01:38.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac003410 con 0x7f62c0003780 2021-11-21T22:01:38.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d17fa700 1 -- 172.21.2.116:0/2121472941 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f62c401d070 con 0x7f62c0003780 2021-11-21T22:01:38.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d17fa700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b4053570 0x7f62b4055a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.642+0000 7f62d17fa700 1 -- 172.21.2.116:0/2121472941 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f62c406c220 con 0x7f62c0003780 2021-11-21T22:01:38.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.646+0000 7f62d37fe700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b4053570 0x7f62b4055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.646+0000 7f62d37fe700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b4053570 0x7f62b4055a30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f62c8002f70 tx=0x7f62c8006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.646+0000 7f62d17fa700 1 -- 172.21.2.116:0/2121472941 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f62c4043590 con 0x7f62c0003780 2021-11-21T22:01:38.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.750+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62ac003c40 con 0x7f62c0003780 2021-11-21T22:01:38.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.750+0000 7f62d17fa700 1 -- 172.21.2.116:0/2121472941 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f62c4046b70 con 0x7f62c0003780 2021-11-21T22:01:38.756 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:38.757 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2736301137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2736301137},{"type":"v1","addr":"172.21.2.116:6835","nonce":2736301137}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[]} 2021-11-21T22:01:38.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b4053570 msgr2=0x7f62b4055a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b4053570 0x7f62b4055a30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f62c8002f70 tx=0x7f62c8006a10).stop 2021-11-21T22:01:38.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 msgr2=0x7f62d4063da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62d4063da0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f62c4001fa0 tx=0x7f62c4015000).stop 2021-11-21T22:01:38.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 shutdown_connections 2021-11-21T22:01:38.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f62d4064fa0 0x7f62d40ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f62d4067380 0x7f62d40ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f62b4053570 0x7f62b4055a30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 --2- 172.21.2.116:0/2121472941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f62c0003780 0x7f62d4063da0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 >> 172.21.2.116:0/2121472941 conn(0x7f62d405c7f0 msgr2=0x7f62d40b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 shutdown_connections 2021-11-21T22:01:38.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.754+0000 7f62d965c700 1 -- 172.21.2.116:0/2121472941 wait complete. 2021-11-21T22:01:38.760 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 46 2021-11-21T22:01:38.763 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:38.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 Processor -- start 2021-11-21T22:01:38.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- start start 2021-11-21T22:01:38.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6900a86b0 0x7fb69014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900a6e80 con 0x7fb690062410 2021-11-21T22:01:38.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69014a7b0 con 0x7fb6900a6aa0 2021-11-21T22:01:38.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900a8bc0 con 0x7fb6900a86b0 2021-11-21T22:01:38.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695a96700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6900a86b0 0x7fb69014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695a96700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6900a86b0 0x7fb69014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51148/0 (socket says 172.21.2.116:51148) 2021-11-21T22:01:38.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695a96700 1 -- 172.21.2.116:0/1691763589 learned_addr learned my addr 172.21.2.116:0/1691763589 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:38.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695a96700 1 -- 172.21.2.116:0/1691763589 >> v1:172.21.2.117:6789/0 conn(0x7fb6900a6aa0 legacy=0x7fb6901470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:38.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695a96700 1 -- 172.21.2.116:0/1691763589 >> v1:172.21.2.116:6789/0 conn(0x7fb690062410 legacy=0x7fb6901443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695a96700 1 -- 172.21.2.116:0/1691763589 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69014aa10 con 0x7fb6900a86b0 2021-11-21T22:01:38.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695a96700 1 --2- 172.21.2.116:0/1691763589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb69014b150 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fb690061b50 tx=0x7fb68c006eb0).ready entity=mon.0 client_cookie=16a68ec5c734f01f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695295700 1 -- 172.21.2.116:0/1691763589 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb68c00d610 con 0x7fb6900a86b0 2021-11-21T22:01:38.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695295700 1 -- 172.21.2.116:0/1691763589 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c000e90 con 0x7fb6900a86b0 2021-11-21T22:01:38.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb695295700 1 -- 172.21.2.116:0/1691763589 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb68c00d960 con 0x7fb6900a86b0 2021-11-21T22:01:38.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- 172.21.2.116:0/1691763589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 msgr2=0x7fb69014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 --2- 172.21.2.116:0/1691763589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb69014b150 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fb690061b50 tx=0x7fb68c006eb0).stop 2021-11-21T22:01:38.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- 172.21.2.116:0/1691763589 shutdown_connections 2021-11-21T22:01:38.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 --2- 172.21.2.116:0/1691763589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb69014b150 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- 172.21.2.116:0/1691763589 >> 172.21.2.116:0/1691763589 conn(0x7fb69005bd20 msgr2=0x7fb69005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- 172.21.2.116:0/1691763589 shutdown_connections 2021-11-21T22:01:38.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- 172.21.2.116:0/1691763589 wait complete. 2021-11-21T22:01:38.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 Processor -- start 2021-11-21T22:01:38.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.810+0000 7fb6984fb700 1 -- start start 2021-11-21T22:01:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb690062410 0x7fb6900a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6900a6aa0 0x7fb69015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb690146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b3bb0 con 0x7fb6900a86b0 2021-11-21T22:01:38.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b3e50 con 0x7fb690062410 2021-11-21T22:01:38.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb690159920 con 0x7fb6900a6aa0 2021-11-21T22:01:38.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb690146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb690146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51154/0 (socket says 172.21.2.116:51154) 2021-11-21T22:01:38.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 -- 172.21.2.116:0/4158936695 learned_addr learned my addr 172.21.2.116:0/4158936695 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:38.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 -- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6900a6aa0 msgr2=0x7fb69015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6900a6aa0 0x7fb69015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 -- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb690062410 msgr2=0x7fb6900a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb690062410 0x7fb6900a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 -- 172.21.2.116:0/4158936695 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c00b1e0 con 0x7fb6900a86b0 2021-11-21T22:01:38.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696a98700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb690146170 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fb690061ab0 tx=0x7fb688007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6837fe700 1 -- 172.21.2.116:0/4158936695 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb688006aa0 con 0x7fb6900a86b0 2021-11-21T22:01:38.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6837fe700 1 -- 172.21.2.116:0/4158936695 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688006c00 con 0x7fb6900a86b0 2021-11-21T22:01:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb690159bc0 con 0x7fb6900a86b0 2021-11-21T22:01:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6837fe700 1 -- 172.21.2.116:0/4158936695 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb688007030 con 0x7fb6900a86b0 2021-11-21T22:01:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6900b3780 con 0x7fb6900a86b0 2021-11-21T22:01:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb690159d50 con 0x7fb6900a86b0 2021-11-21T22:01:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6837fe700 1 -- 172.21.2.116:0/4158936695 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb688010030 con 0x7fb6900a86b0 2021-11-21T22:01:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6837fe700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6700535b0 0x7fb670055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb6837fe700 1 -- 172.21.2.116:0/4158936695 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb68806b500 con 0x7fb6900a86b0 2021-11-21T22:01:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696297700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6700535b0 0x7fb670055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.814+0000 7fb696297700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6700535b0 0x7fb670055a70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fb684002fb0 tx=0x7fb6840069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.818+0000 7fb6837fe700 1 -- 172.21.2.116:0/4158936695 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb688042940 con 0x7fb6900a86b0 2021-11-21T22:01:38.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6900573d0 con 0x7fb6900a86b0 2021-11-21T22:01:38.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6837fe700 1 -- 172.21.2.116:0/4158936695 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fb688045f60 con 0x7fb6900a86b0 2021-11-21T22:01:38.930 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:38.931 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3510733876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3510733876},{"type":"v1","addr":"172.21.2.117:6835","nonce":3510733876}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/1058507343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":1058507343},{"type":"v1","addr":"172.21.2.116:6833","nonce":1058507343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2736301137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2736301137},{"type":"v1","addr":"172.21.2.116:6835","nonce":2736301137}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[]} 2021-11-21T22:01:38.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6700535b0 msgr2=0x7fb670055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6700535b0 0x7fb670055a70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fb684002fb0 tx=0x7fb6840069b0).stop 2021-11-21T22:01:38.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 msgr2=0x7fb690146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:38.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb690146170 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fb690061ab0 tx=0x7fb688007f70).stop 2021-11-21T22:01:38.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 shutdown_connections 2021-11-21T22:01:38.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6700535b0 0x7fb670055a70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6900a86b0 0x7fb690146170 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:38.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6900a6aa0 0x7fb69015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 --2- 172.21.2.116:0/4158936695 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb690062410 0x7fb6900a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:38.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 >> 172.21.2.116:0/4158936695 conn(0x7fb69005bd20 msgr2=0x7fb690148660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:38.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 shutdown_connections 2021-11-21T22:01:38.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:38.926+0000 7fb6984fb700 1 -- 172.21.2.116:0/4158936695 wait complete. 2021-11-21T22:01:38.935 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 46 2021-11-21T22:01:38.937 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T22:01:38.938 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T22:01:38.938 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:01:38.939 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:38.939 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T22:01:38.939 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T22:01:38.940 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:01:38.940 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:38.940 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T22:01:38.941 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T22:01:38.941 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:01:38.941 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:38.944 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:01:38.944 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T22:01:38.947 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:01:38.947 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T22:01:38.952 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:01:38.953 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T22:01:38.955 INFO:tasks.ceph.mds.b:Started 2021-11-21T22:01:38.956 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:38.988 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T22:01:38.989 INFO:tasks.ceph.mds.a:Started 2021-11-21T22:01:38.989 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:38.990 INFO:tasks.ceph.mds.c:Started 2021-11-21T22:01:38.991 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:38.992 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T22:01:39.024 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:01:39.024 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:01:39.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f118b9ca700 1 Processor -- start 2021-11-21T22:01:39.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f118b9ca700 1 -- start start 2021-11-21T22:01:39.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f118b9ca700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f118405ced0 0x7f11840acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:39.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f118b9ca700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118409b180 con 0x7f118409aa10 2021-11-21T22:01:39.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f118b9ca700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118409aea0 con 0x7f1184061060 2021-11-21T22:01:39.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f118b9ca700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118409b3e0 con 0x7f118405ced0 2021-11-21T22:01:39.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f1189f67700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f118409aa10 0x7f1184146070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34906/0 (socket says 172.21.2.116:34906) 2021-11-21T22:01:39.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f1189f67700 1 -- 172.21.2.116:0/465296469 learned_addr learned my addr 172.21.2.116:0/465296469 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:39.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f1189766700 1 --2- 172.21.2.116:0/465296469 >> v2:172.21.2.116:3300/0 conn(0x7f118405ced0 0x7f11840acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:39.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f1189766700 1 -- 172.21.2.116:0/465296469 >> v1:172.21.2.117:6789/0 conn(0x7f1184061060 legacy=0x7f118409a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:39.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f1189766700 1 -- 172.21.2.116:0/465296469 >> v1:172.21.2.116:6789/0 conn(0x7f118409aa10 legacy=0x7f1184146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:39.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f1189766700 1 -- 172.21.2.116:0/465296469 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118409b680 con 0x7f118405ced0 2021-11-21T22:01:39.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.042+0000 7f1189766700 1 --2- 172.21.2.116:0/465296469 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f11840acb80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f1184062400 tx=0x7f1174006df0).ready entity=mon.0 client_cookie=2cda2733991f60b1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:39.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f117bfff700 1 -- 172.21.2.116:0/465296469 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117400d5c0 con 0x7f118405ced0 2021-11-21T22:01:39.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f117bfff700 1 -- 172.21.2.116:0/465296469 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1174000e90 con 0x7f118405ced0 2021-11-21T22:01:39.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f117bfff700 1 -- 172.21.2.116:0/465296469 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117400d910 con 0x7f118405ced0 2021-11-21T22:01:39.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/465296469 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 msgr2=0x7f11840acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:39.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 --2- 172.21.2.116:0/465296469 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f11840acb80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f1184062400 tx=0x7f1174006df0).stop 2021-11-21T22:01:39.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/465296469 shutdown_connections 2021-11-21T22:01:39.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 --2- 172.21.2.116:0/465296469 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f11840acb80 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:39.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/465296469 >> 172.21.2.116:0/465296469 conn(0x7f11840b5de0 msgr2=0x7f11840b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:39.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/465296469 shutdown_connections 2021-11-21T22:01:39.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/465296469 wait complete. 2021-11-21T22:01:39.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 Processor -- start 2021-11-21T22:01:39.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- start start 2021-11-21T22:01:39.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f1184097cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:39.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1184061060 0x7f11840adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:39.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f118409aa10 0x7f11840ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:39.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184140f00 con 0x7f118405ced0 2021-11-21T22:01:39.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841574d0 con 0x7f118409aa10 2021-11-21T22:01:39.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840adad0 con 0x7f1184061060 2021-11-21T22:01:39.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f1184097cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:39.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f1184097cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51188/0 (socket says 172.21.2.116:51188) 2021-11-21T22:01:39.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 -- 172.21.2.116:0/1368798296 learned_addr learned my addr 172.21.2.116:0/1368798296 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:39.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1188f65700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1184061060 0x7f11840adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:39.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189f67700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f118409aa10 0x7f11840ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 -- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1184061060 msgr2=0x7f11840adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1184061060 0x7f11840adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 -- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f118409aa10 msgr2=0x7f11840ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f118409aa10 0x7f11840ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 -- 172.21.2.116:0/1368798296 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117400b180 con 0x7f118405ced0 2021-11-21T22:01:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f1189766700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f1184097cf0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f1184062400 tx=0x7f1174012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f117a7fc700 1 -- 172.21.2.116:0/1368798296 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1174016070 con 0x7f118405ced0 2021-11-21T22:01:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f117a7fc700 1 -- 172.21.2.116:0/1368798296 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1174002f50 con 0x7f118405ced0 2021-11-21T22:01:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1184156850 con 0x7f118405ced0 2021-11-21T22:01:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f117a7fc700 1 -- 172.21.2.116:0/1368798296 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117401e450 con 0x7f118405ced0 2021-11-21T22:01:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11841565b0 con 0x7f118405ced0 2021-11-21T22:01:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118409b920 con 0x7f118405ced0 2021-11-21T22:01:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.046+0000 7f117a7fc700 1 -- 172.21.2.116:0/1368798296 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1174025030 con 0x7f118405ced0 2021-11-21T22:01:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.050+0000 7f117a7fc700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f116c0534f0 0x7f116c0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:39.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.050+0000 7f1188f65700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f116c0534f0 0x7f116c0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:39.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.050+0000 7f117a7fc700 1 -- 172.21.2.116:0/1368798296 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f11740477a0 con 0x7f118405ced0 2021-11-21T22:01:39.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.050+0000 7f1188f65700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f116c0534f0 0x7f116c0559b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f118000b000 tx=0x7f1180009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:39.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.050+0000 7f117a7fc700 1 -- 172.21.2.116:0/1368798296 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1174043b40 con 0x7f118405ced0 2021-11-21T22:01:39.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:39.146+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f11840573d0 con 0x7f118405ced0 2021-11-21T22:01:40.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f117a7fc700 1 -- 172.21.2.116:0/1368798296 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v106) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1174023070 con 0x7f118405ced0 2021-11-21T22:01:40.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f116c0534f0 msgr2=0x7f116c0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f116c0534f0 0x7f116c0559b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f118000b000 tx=0x7f1180009040).stop 2021-11-21T22:01:40.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 msgr2=0x7f1184097cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f1184097cf0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f1184062400 tx=0x7f1174012000).stop 2021-11-21T22:01:40.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 shutdown_connections 2021-11-21T22:01:40.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f118409aa10 0x7f11840ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1184061060 0x7f11840adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f116c0534f0 0x7f116c0559b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 --2- 172.21.2.116:0/1368798296 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f118405ced0 0x7f1184097cf0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 >> 172.21.2.116:0/1368798296 conn(0x7f11840b5de0 msgr2=0x7f118405f730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 shutdown_connections 2021-11-21T22:01:40.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.054+0000 7f118b9ca700 1 -- 172.21.2.116:0/1368798296 wait complete. 2021-11-21T22:01:40.063 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T22:01:40.069 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T22:01:40.069 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T22:01:40.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 Processor -- start 2021-11-21T22:01:40.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- start start 2021-11-21T22:01:40.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 0x7f14cc0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc0b29b0 con 0x7f14cc0b7070 2021-11-21T22:01:40.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc067c30 con 0x7f14cc1443a0 2021-11-21T22:01:40.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0ab720 con 0x7f14cc067850 2021-11-21T22:01:40.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14caffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f14cc0b7070 0x7f14cc0ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34928/0 (socket says 172.21.2.116:34928) 2021-11-21T22:01:40.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14caffd700 1 -- 172.21.2.116:0/3526344767 learned_addr learned my addr 172.21.2.116:0/3526344767 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:40.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461691034 0 0) 0x7f14cc0b29b0 con 0x7f14cc0b7070 2021-11-21T22:01:40.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ac003260 con 0x7f14cc0b7070 2021-11-21T22:01:40.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cb7fe700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 0x7f14cc0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 696032839 0 0) 0x7f14ac003260 con 0x7f14cc0b7070 2021-11-21T22:01:40.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 msgr2=0x7f14cc0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 0x7f14cc0b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 >> v1:172.21.2.117:6789/0 conn(0x7f14cc1443a0 legacy=0x7f14cc144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0b7910 con 0x7f14cc0b7070 2021-11-21T22:01:40.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f14b4001660 con 0x7f14cc0b7070 2021-11-21T22:01:40.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cb7fe700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 0x7f14cc0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:40.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 >> v1:172.21.2.116:6789/0 conn(0x7f14cc0b7070 legacy=0x7f14cc0ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14ac003800 0x7f14ac003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 0x7f14ac004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14ac004bf0 0x7f14ac00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0ab720 con 0x7f14ac004bf0 2021-11-21T22:01:40.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac005270 con 0x7f14ac003800 2021-11-21T22:01:40.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac00bc40 con 0x7f14cc067850 2021-11-21T22:01:40.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14ac004bf0 0x7f14ac00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 -- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 msgr2=0x7f14ac004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14cc067850 0x7f14ac004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 -- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14ac003800 msgr2=0x7f14ac003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14ac003800 0x7f14ac003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 -- 172.21.2.116:0/3526344767 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac003260 con 0x7f14ac004bf0 2021-11-21T22:01:40.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14ac004bf0 0x7f14ac00b750 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f14c4002320 tx=0x7f14c4007d80).ready entity=mon.0 client_cookie=a1886a5affde0230 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c4002510 con 0x7f14ac004bf0 2021-11-21T22:01:40.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4006890 con 0x7f14ac004bf0 2021-11-21T22:01:40.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14ca7fc700 1 -- 172.21.2.116:0/3526344767 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c4006bf0 con 0x7f14ac004bf0 2021-11-21T22:01:40.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- 172.21.2.116:0/3526344767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14ac004bf0 msgr2=0x7f14ac00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 --2- 172.21.2.116:0/3526344767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14ac004bf0 0x7f14ac00b750 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f14c4002320 tx=0x7f14c4007d80).stop 2021-11-21T22:01:40.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14cbfff700 1 -- 172.21.2.116:0/3526344767 reap_dead start 2021-11-21T22:01:40.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- 172.21.2.116:0/3526344767 shutdown_connections 2021-11-21T22:01:40.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- 172.21.2.116:0/3526344767 >> 172.21.2.116:0/3526344767 conn(0x7f14cc064d60 msgr2=0x7f14cc065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- 172.21.2.116:0/3526344767 shutdown_connections 2021-11-21T22:01:40.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.122+0000 7f14d1f18700 1 -- 172.21.2.116:0/3526344767 wait complete. 2021-11-21T22:01:40.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 Processor -- start 2021-11-21T22:01:40.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 -- start start 2021-11-21T22:01:40.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14ac004bf0 0x7f14cc153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14cc0a9c90 0x7f14cc0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14cc0aebc0 0x7f14cc0af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0a18c0 con 0x7f14cc0a9c90 2021-11-21T22:01:40.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0a1b50 con 0x7f14cc0aebc0 2021-11-21T22:01:40.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0a1df0 con 0x7f14ac004bf0 2021-11-21T22:01:40.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14cc0a9c90 0x7f14cc0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14cc0a9c90 0x7f14cc0a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51212/0 (socket says 172.21.2.116:51212) 2021-11-21T22:01:40.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 -- 172.21.2.116:0/1134184546 learned_addr learned my addr 172.21.2.116:0/1134184546 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 -- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14ac004bf0 msgr2=0x7f14cc153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14ac004bf0 0x7f14cc153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 -- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14cc0aebc0 msgr2=0x7f14cc0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14cc0aebc0 0x7f14cc0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 -- 172.21.2.116:0/1134184546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0ae480 con 0x7f14cc0a9c90 2021-11-21T22:01:40.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14caffd700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14cc0a9c90 0x7f14cc0a13d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f14cc0ae000 tx=0x7f14b4001a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14c8ff9700 1 -- 172.21.2.116:0/1134184546 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b4007170 con 0x7f14cc0a9c90 2021-11-21T22:01:40.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14c8ff9700 1 -- 172.21.2.116:0/1134184546 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b40072d0 con 0x7f14cc0a9c90 2021-11-21T22:01:40.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc0ae720 con 0x7f14cc0a9c90 2021-11-21T22:01:40.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc0af750 con 0x7f14cc0a9c90 2021-11-21T22:01:40.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14c8ff9700 1 -- 172.21.2.116:0/1134184546 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b4004920 con 0x7f14cc0a9c90 2021-11-21T22:01:40.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc0ae8b0 con 0x7f14cc0a9c90 2021-11-21T22:01:40.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14c8ff9700 1 -- 172.21.2.116:0/1134184546 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f14b4004a80 con 0x7f14cc0a9c90 2021-11-21T22:01:40.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14c8ff9700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14b0053690 0x7f14b0055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14c8ff9700 1 -- 172.21.2.116:0/1134184546 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f14b406b330 con 0x7f14cc0a9c90 2021-11-21T22:01:40.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.126+0000 7f14cb7fe700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14b0053690 0x7f14b0055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.130+0000 7f14cb7fe700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14b0053690 0x7f14b0055b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f14bc007000 tx=0x7f14bc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.130+0000 7f14c8ff9700 1 -- 172.21.2.116:0/1134184546 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f14b4042650 con 0x7f14cc0a9c90 2021-11-21T22:01:40.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f14cc068740 con 0x7f14cc0a9c90 2021-11-21T22:01:40.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14c8ff9700 1 -- 172.21.2.116:0/1134184546 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f14b4045c70 con 0x7f14cc0a9c90 2021-11-21T22:01:40.248 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:40.248 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:40.249 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T22:01:40.249 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.249 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T22:01:40.250 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T22:01:40.250 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.250 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:40.251 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.251 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.251 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.252 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.252 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.252 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.253 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T22:01:40.253 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T22:01:40.254 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.254 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:40.254 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.255 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.255 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.256 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.256 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.256 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.257 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T22:01:40.257 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T22:01:40.257 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.258 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:40.258 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.258 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.259 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.259 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.259 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.259 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.259 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T22:01:40.260 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T22:01:40.260 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.260 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.260 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.260 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.261 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.261 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.261 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.261 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T22:01:40.262 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T22:01:40.262 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.262 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.262 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.263 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.263 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.263 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.263 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.263 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T22:01:40.264 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T22:01:40.264 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.264 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.264 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.264 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.265 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.265 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.265 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.265 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T22:01:40.266 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T22:01:40.266 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.266 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.266 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.266 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.267 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.267 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.267 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.267 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T22:01:40.268 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T22:01:40.268 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.268 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.269 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.269 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.270 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.270 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.270 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.271 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T22:01:40.271 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T22:01:40.272 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.272 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.273 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.273 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.273 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.274 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.274 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.275 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T22:01:40.275 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T22:01:40.275 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.276 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.276 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.277 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.277 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.278 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.278 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.278 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T22:01:40.279 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T22:01:40.279 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.279 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.279 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.279 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.280 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.280 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.280 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.280 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T22:01:40.281 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T22:01:40.281 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.281 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:40.281 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:40.281 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:40.282 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.282 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.282 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.283 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T22:01:40.283 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T22:01:40.283 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.283 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:40.284 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:40.284 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:40.284 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.284 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.285 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.285 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T22:01:40.285 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T22:01:40.285 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.285 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:40.286 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:40.286 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:40.286 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.286 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.287 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.287 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.287 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T22:01:40.287 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T22:01:40.287 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.288 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:40.288 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:40.288 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.288 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.288 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.289 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.289 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T22:01:40.289 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T22:01:40.289 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:40.289 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:40.290 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:40.290 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:40.290 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.290 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.291 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:40.291 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:40.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14b0053690 msgr2=0x7f14b0055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14b0053690 0x7f14b0055b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f14bc007000 tx=0x7f14bc008040).stop 2021-11-21T22:01:40.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14cc0a9c90 msgr2=0x7f14cc0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14cc0a9c90 0x7f14cc0a13d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f14cc0ae000 tx=0x7f14b4001a90).stop 2021-11-21T22:01:40.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 shutdown_connections 2021-11-21T22:01:40.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14cc0aebc0 0x7f14cc0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14b0053690 0x7f14b0055b50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14cc0a9c90 0x7f14cc0a13d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 --2- 172.21.2.116:0/1134184546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14ac004bf0 0x7f14cc153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 >> 172.21.2.116:0/1134184546 conn(0x7f14cc064d60 msgr2=0x7f14cc0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 shutdown_connections 2021-11-21T22:01:40.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.242+0000 7f14d1f18700 1 -- 172.21.2.116:0/1134184546 wait complete. 2021-11-21T22:01:40.296 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:40.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.342+0000 7f2faf960700 1 Processor -- start 2021-11-21T22:01:40.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.342+0000 7f2faf960700 1 -- start start 2021-11-21T22:01:40.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 0x7f2fa80adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa80ac090 con 0x7f2fa813d470 2021-11-21T22:01:40.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa8062590 con 0x7f2fa80abcb0 2021-11-21T22:01:40.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80b1590 con 0x7f2fa80af290 2021-11-21T22:01:40.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2facefb700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f2fa813d470 0x7f2fa813fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34946/0 (socket says 172.21.2.116:34946) 2021-11-21T22:01:40.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2facefb700 1 -- 172.21.2.116:0/3333617111 learned_addr learned my addr 172.21.2.116:0/3333617111 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:40.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106543444 0 0) 0x7f2fa80ac090 con 0x7f2fa813d470 2021-11-21T22:01:40.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f90003260 con 0x7f2fa813d470 2021-11-21T22:01:40.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017695219 0 0) 0x7f2f90003260 con 0x7f2fa813d470 2021-11-21T22:01:40.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 msgr2=0x7f2fa80adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 0x7f2fa80adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 >> v1:172.21.2.117:6789/0 conn(0x7f2fa80abcb0 legacy=0x7f2fa813cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8062830 con 0x7f2fa813d470 2021-11-21T22:01:40.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f2f9c001660 con 0x7f2fa813d470 2021-11-21T22:01:40.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 >> v1:172.21.2.116:6789/0 conn(0x7f2fa813d470 legacy=0x7f2fa813fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2f90003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 0x7f2f90004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2f90004bf0 0x7f2f9000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f90005270 con 0x7f2f90003800 2021-11-21T22:01:40.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9000bc40 con 0x7f2f90004bf0 2021-11-21T22:01:40.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9000bd80 con 0x7f2fa80af290 2021-11-21T22:01:40.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2f90003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2facefb700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 0x7f2f90004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 -- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 msgr2=0x7f2f90004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 0x7f2f90004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 -- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2f90004bf0 msgr2=0x7f2f9000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2f90004bf0 0x7f2f9000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 -- 172.21.2.116:0/3333617111 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90003260 con 0x7f2f90003800 2021-11-21T22:01:40.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2facefb700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa80af290 0x7f2f90004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:40.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2f90003c00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f2fa4000f70 tx=0x7f2fa40079e0).ready entity=mon.0 client_cookie=31d6cc7c52bb87e9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fa4007cf0 con 0x7f2f90003800 2021-11-21T22:01:40.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa4014410 con 0x7f2f90003800 2021-11-21T22:01:40.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2f9bfff700 1 -- 172.21.2.116:0/3333617111 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fa4014770 con 0x7f2f90003800 2021-11-21T22:01:40.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- 172.21.2.116:0/3333617111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 msgr2=0x7f2f90003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 --2- 172.21.2.116:0/3333617111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2f90003c00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f2fa4000f70 tx=0x7f2fa40079e0).stop 2021-11-21T22:01:40.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fadefd700 1 -- 172.21.2.116:0/3333617111 reap_dead start 2021-11-21T22:01:40.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- 172.21.2.116:0/3333617111 shutdown_connections 2021-11-21T22:01:40.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- 172.21.2.116:0/3333617111 >> 172.21.2.116:0/3333617111 conn(0x7f2fa805bd20 msgr2=0x7f2fa805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- 172.21.2.116:0/3333617111 shutdown_connections 2021-11-21T22:01:40.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- 172.21.2.116:0/3333617111 wait complete. 2021-11-21T22:01:40.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 Processor -- start 2021-11-21T22:01:40.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- start start 2021-11-21T22:01:40.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2fa813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fa8143800 0x7f2fa813ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa813a2c0 0x7f2fa813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8141a50 con 0x7f2f90003800 2021-11-21T22:01:40.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81433a0 con 0x7f2fa8143800 2021-11-21T22:01:40.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2faf960700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8140ec0 con 0x7f2fa813a2c0 2021-11-21T22:01:40.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2fa813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2fa813c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51234/0 (socket says 172.21.2.116:51234) 2021-11-21T22:01:40.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.346+0000 7f2fad6fc700 1 -- 172.21.2.116:0/470524323 learned_addr learned my addr 172.21.2.116:0/470524323 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2fad6fc700 1 -- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa813a2c0 msgr2=0x7f2fa813a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2fad6fc700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa813a2c0 0x7f2fa813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2fad6fc700 1 -- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fa8143800 msgr2=0x7f2fa813ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2fad6fc700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fa8143800 0x7f2fa813ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2fad6fc700 1 -- 172.21.2.116:0/470524323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8141160 con 0x7f2f90003800 2021-11-21T22:01:40.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2fad6fc700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2fa813c7b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f2fa4000f70 tx=0x7f2fa4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2f9a7fc700 1 -- 172.21.2.116:0/470524323 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fa4013840 con 0x7f2f90003800 2021-11-21T22:01:40.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2f9a7fc700 1 -- 172.21.2.116:0/470524323 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa40139a0 con 0x7f2f90003800 2021-11-21T22:01:40.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa8143c80 con 0x7f2f90003800 2021-11-21T22:01:40.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa8142ef0 con 0x7f2f90003800 2021-11-21T22:01:40.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2f9a7fc700 1 -- 172.21.2.116:0/470524323 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fa4013d00 con 0x7f2f90003800 2021-11-21T22:01:40.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa8143200 con 0x7f2f90003800 2021-11-21T22:01:40.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2f9a7fc700 1 -- 172.21.2.116:0/470524323 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2fa4022470 con 0x7f2f90003800 2021-11-21T22:01:40.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2f9a7fc700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2f88053540 0x7f2f88055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2f9a7fc700 1 -- 172.21.2.116:0/470524323 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2fa401c040 con 0x7f2f90003800 2021-11-21T22:01:40.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2facefb700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2f88053540 0x7f2f88055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.350+0000 7f2facefb700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2f88053540 0x7f2f88055a00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f2f9c008000 tx=0x7f2f9c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.354+0000 7f2f9a7fc700 1 -- 172.21.2.116:0/470524323 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fa8143200 con 0x7f2f90003800 2021-11-21T22:01:40.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.434+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fa8143200 con 0x7f2f90003800 2021-11-21T22:01:40.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.434+0000 7f2f9a7fc700 1 -- 172.21.2.116:0/470524323 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13183 (secure 0 0 0) 0x7f2fa8143200 con 0x7f2f90003800 2021-11-21T22:01:40.441 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:40.442 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":106,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:40.052957+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:40.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2f88053540 msgr2=0x7f2f88055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2f88053540 0x7f2f88055a00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f2f9c008000 tx=0x7f2f9c009040).stop 2021-11-21T22:01:40.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 msgr2=0x7f2fa813c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2fa813c7b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f2fa4000f70 tx=0x7f2fa4015000).stop 2021-11-21T22:01:40.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 shutdown_connections 2021-11-21T22:01:40.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2fa813a2c0 0x7f2fa813a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2fa8143800 0x7f2fa813ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2f88053540 0x7f2f88055a00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 --2- 172.21.2.116:0/470524323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2f90003800 0x7f2fa813c7b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 >> 172.21.2.116:0/470524323 conn(0x7f2fa805bd20 msgr2=0x7f2fa80b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 shutdown_connections 2021-11-21T22:01:40.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.438+0000 7f2faf960700 1 -- 172.21.2.116:0/470524323 wait complete. 2021-11-21T22:01:40.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T22:01:40.448 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:40.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 Processor -- start 2021-11-21T22:01:40.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- start start 2021-11-21T22:01:40.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981426e0 0x7f0c98142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0c98062e60 0x7f0c980af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0c980ad470 0x7f0c980ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c980b0f60 con 0x7f0c980ad470 2021-11-21T22:01:40.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c98063470 con 0x7f0c98062e60 2021-11-21T22:01:40.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98143050 con 0x7f0c981426e0 2021-11-21T22:01:40.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0c980ad470 0x7f0c980ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0c980ad470 0x7f0c980ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51244/0 (socket says 172.21.2.116:51244) 2021-11-21T22:01:40.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 -- 172.21.2.116:0/699068237 learned_addr learned my addr 172.21.2.116:0/699068237 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 -- 172.21.2.116:0/699068237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981426e0 msgr2=0x7f0c98142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:40.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 --2- 172.21.2.116:0/699068237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981426e0 0x7f0c98142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 -- 172.21.2.116:0/699068237 >> v2:172.21.2.117:3300/0 conn(0x7f0c98062e60 msgr2=0x7f0c980af3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:40.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 --2- 172.21.2.116:0/699068237 >> v2:172.21.2.117:3300/0 conn(0x7f0c98062e60 0x7f0c980af3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 -- 172.21.2.116:0/699068237 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980b1e00 con 0x7f0c980ad470 2021-11-21T22:01:40.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9e991700 1 --2- 172.21.2.116:0/699068237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c980ad470 0x7f0c980ad920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f0c900032f0 tx=0x7f0c9000b530).ready entity=mon.0 client_cookie=4bf0b78e21840d8a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9d18e700 1 -- 172.21.2.116:0/699068237 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c900155e0 con 0x7f0c980ad470 2021-11-21T22:01:40.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9d18e700 1 -- 172.21.2.116:0/699068237 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90015740 con 0x7f0c980ad470 2021-11-21T22:01:40.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0c9d18e700 1 -- 172.21.2.116:0/699068237 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c90015ae0 con 0x7f0c980ad470 2021-11-21T22:01:40.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- 172.21.2.116:0/699068237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c980ad470 msgr2=0x7f0c980ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/699068237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c980ad470 0x7f0c980ad920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f0c900032f0 tx=0x7f0c9000b530).stop 2021-11-21T22:01:40.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- 172.21.2.116:0/699068237 shutdown_connections 2021-11-21T22:01:40.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/699068237 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c980ad470 0x7f0c980ad920 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/699068237 >> v2:172.21.2.117:3300/0 conn(0x7f0c98062e60 0x7f0c980af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/699068237 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981426e0 0x7f0c98142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- 172.21.2.116:0/699068237 >> 172.21.2.116:0/699068237 conn(0x7f0c980b5730 msgr2=0x7f0c980b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- 172.21.2.116:0/699068237 shutdown_connections 2021-11-21T22:01:40.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- 172.21.2.116:0/699068237 wait complete. 2021-11-21T22:01:40.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 Processor -- start 2021-11-21T22:01:40.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.498+0000 7f0ca03f4700 1 -- start start 2021-11-21T22:01:40.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c980ad470 0x7f0c98145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c981426e0 0x7f0c98151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981461a0 0x7f0c98152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98067100 con 0x7f0c981426e0 2021-11-21T22:01:40.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98155ef0 con 0x7f0c980ad470 2021-11-21T22:01:40.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98156190 con 0x7f0c981461a0 2021-11-21T22:01:40.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c981426e0 0x7f0c98151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c981426e0 0x7f0c98151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51250/0 (socket says 172.21.2.116:51250) 2021-11-21T22:01:40.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 -- 172.21.2.116:0/3648775568 learned_addr learned my addr 172.21.2.116:0/3648775568 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 -- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981461a0 msgr2=0x7f0c98152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981461a0 0x7f0c98152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 -- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c980ad470 msgr2=0x7f0c98145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c980ad470 0x7f0c98145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 -- 172.21.2.116:0/3648775568 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9000b7d0 con 0x7f0c981426e0 2021-11-21T22:01:40.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9d98f700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c981426e0 0x7f0c98151e10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f0c94002ef0 tx=0x7f0c94006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c8b7fe700 1 -- 172.21.2.116:0/3648775568 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c94009ea0 con 0x7f0c981426e0 2021-11-21T22:01:40.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c8b7fe700 1 -- 172.21.2.116:0/3648775568 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c94002320 con 0x7f0c981426e0 2021-11-21T22:01:40.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98065ef0 con 0x7f0c981426e0 2021-11-21T22:01:40.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c981527f0 con 0x7f0c981426e0 2021-11-21T22:01:40.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c8b7fe700 1 -- 172.21.2.116:0/3648775568 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c94002680 con 0x7f0c981426e0 2021-11-21T22:01:40.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c980573d0 con 0x7f0c981426e0 2021-11-21T22:01:40.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c8b7fe700 1 -- 172.21.2.116:0/3648775568 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0c9400f030 con 0x7f0c981426e0 2021-11-21T22:01:40.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c8b7fe700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c78053610 0x7f0c78055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c8b7fe700 1 -- 172.21.2.116:0/3648775568 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0c9406b360 con 0x7f0c981426e0 2021-11-21T22:01:40.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.502+0000 7f0c9e190700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c78053610 0x7f0c78055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.506+0000 7f0c9e190700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c78053610 0x7f0c78055ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f0c8c002fb0 tx=0x7f0c8c0069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.506+0000 7f0c8b7fe700 1 -- 172.21.2.116:0/3648775568 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0c940427f0 con 0x7f0c981426e0 2021-11-21T22:01:40.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0c9805f2d0 con 0x7f0c981426e0 2021-11-21T22:01:40.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0c8b7fe700 1 -- 172.21.2.116:0/3648775568 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v106) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0c94045e10 con 0x7f0c981426e0 2021-11-21T22:01:40.593 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:40.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c78053610 msgr2=0x7f0c78055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c78053610 0x7f0c78055ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f0c8c002fb0 tx=0x7f0c8c0069d0).stop 2021-11-21T22:01:40.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c981426e0 msgr2=0x7f0c98151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c981426e0 0x7f0c98151e10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f0c94002ef0 tx=0x7f0c94006d50).stop 2021-11-21T22:01:40.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 shutdown_connections 2021-11-21T22:01:40.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0c981461a0 0x7f0c98152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0c78053610 0x7f0c78055ad0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0c981426e0 0x7f0c98151e10 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 --2- 172.21.2.116:0/3648775568 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0c980ad470 0x7f0c98145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 >> 172.21.2.116:0/3648775568 conn(0x7f0c980b5730 msgr2=0x7f0c98069680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 shutdown_connections 2021-11-21T22:01:40.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.590+0000 7f0ca03f4700 1 -- 172.21.2.116:0/3648775568 wait complete. 2021-11-21T22:01:40.600 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:40.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 Processor -- start 2021-11-21T22:01:40.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- start start 2021-11-21T22:01:40.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2cd00abce0 0x7f2cd01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd0147e80 con 0x7f2cd0144810 2021-11-21T22:01:40.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd00af5a0 con 0x7f2cd00ac600 2021-11-21T22:01:40.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd00b1490 con 0x7f2cd00abce0 2021-11-21T22:01:40.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2ccffff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f2cd0144810 0x7f2cd00b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:34978/0 (socket says 172.21.2.116:34978) 2021-11-21T22:01:40.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2ccffff700 1 -- 172.21.2.116:0/2438580292 learned_addr learned my addr 172.21.2.116:0/2438580292 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:40.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409463665 0 0) 0x7f2cd0147e80 con 0x7f2cd0144810 2021-11-21T22:01:40.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cb0003260 con 0x7f2cd0144810 2021-11-21T22:01:40.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2ccf7fe700 1 --2- 172.21.2.116:0/2438580292 >> v2:172.21.2.117:3300/0 conn(0x7f2cd00abce0 0x7f2cd01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898120060 0 0) 0x7f2cb0003260 con 0x7f2cd0144810 2021-11-21T22:01:40.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 >> v2:172.21.2.117:3300/0 conn(0x7f2cd00abce0 msgr2=0x7f2cd01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 --2- 172.21.2.116:0/2438580292 >> v2:172.21.2.117:3300/0 conn(0x7f2cd00abce0 0x7f2cd01478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 >> v1:172.21.2.117:6789/0 conn(0x7f2cd00ac600 legacy=0x7f2cd00aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd00b1fd0 con 0x7f2cd0144810 2021-11-21T22:01:40.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f2cc80019e0 con 0x7f2cd0144810 2021-11-21T22:01:40.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 >> v1:172.21.2.116:6789/0 conn(0x7f2cd0144810 legacy=0x7f2cd00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cb0003820 0x7f2cb0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2cb0004500 0x7f2cb0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cb000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00b1490 con 0x7f2cb0005050 2021-11-21T22:01:40.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb000bf40 con 0x7f2cb0003820 2021-11-21T22:01:40.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb000c100 con 0x7f2cb0004500 2021-11-21T22:01:40.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2ccf7fe700 1 --2- 172.21.2.116:0/2438580292 >> v2:172.21.2.117:3300/0 conn(0x7f2cd00abce0 0x7f2cd01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:40.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cb000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2ccffff700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cb0003820 0x7f2cb0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 -- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2cb0004500 msgr2=0x7f2cb0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2cb0004500 0x7f2cb0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 -- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cb0003820 msgr2=0x7f2cb0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cb0003820 0x7f2cb0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 -- 172.21.2.116:0/2438580292 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0003260 con 0x7f2cb0005050 2021-11-21T22:01:40.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2ccffff700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cb0003820 0x7f2cb0003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:40.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 -- 172.21.2.116:0/2438580292 reap_dead start 2021-11-21T22:01:40.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd4edf700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cb000b750 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f2cb8000f70 tx=0x7f2cb80079e0).ready entity=mon.0 client_cookie=b0116df550512b42 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cb8007cf0 con 0x7f2cb0005050 2021-11-21T22:01:40.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb8014410 con 0x7f2cb0005050 2021-11-21T22:01:40.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cceffd700 1 -- 172.21.2.116:0/2438580292 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cb8014790 con 0x7f2cb0005050 2021-11-21T22:01:40.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- 172.21.2.116:0/2438580292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 msgr2=0x7f2cb000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cb000b750 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f2cb8000f70 tx=0x7f2cb80079e0).stop 2021-11-21T22:01:40.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- 172.21.2.116:0/2438580292 shutdown_connections 2021-11-21T22:01:40.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2438580292 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cb000b750 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- 172.21.2.116:0/2438580292 >> 172.21.2.116:0/2438580292 conn(0x7f2cd005bd20 msgr2=0x7f2cd005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- 172.21.2.116:0/2438580292 shutdown_connections 2021-11-21T22:01:40.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 -- 172.21.2.116:0/2438580292 wait complete. 2021-11-21T22:01:40.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.650+0000 7f2cd6942700 1 Processor -- start 2021-11-21T22:01:40.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 -- start start 2021-11-21T22:01:40.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cd0062b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cd0145500 0x7f2cd0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2cd00b3450 0x7f2cd0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0156d30 con 0x7f2cb0005050 2021-11-21T22:01:40.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd01466f0 con 0x7f2cd0145500 2021-11-21T22:01:40.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0146990 con 0x7f2cd00b3450 2021-11-21T22:01:40.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cd0062b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cd0062b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51266/0 (socket says 172.21.2.116:51266) 2021-11-21T22:01:40.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 -- 172.21.2.116:0/2974463244 learned_addr learned my addr 172.21.2.116:0/2974463244 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 -- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2cd00b3450 msgr2=0x7f2cd0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2cd00b3450 0x7f2cd0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 -- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cd0145500 msgr2=0x7f2cd0156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cd0145500 0x7f2cd0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 -- 172.21.2.116:0/2974463244 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0146c30 con 0x7f2cb0005050 2021-11-21T22:01:40.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccffff700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cd0062b60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f2cc8008000 tx=0x7f2cc8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccd7fa700 1 -- 172.21.2.116:0/2974463244 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cc8004ab0 con 0x7f2cb0005050 2021-11-21T22:01:40.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccd7fa700 1 -- 172.21.2.116:0/2974463244 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8004c10 con 0x7f2cb0005050 2021-11-21T22:01:40.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccd7fa700 1 -- 172.21.2.116:0/2974463244 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cc8007170 con 0x7f2cb0005050 2021-11-21T22:01:40.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd0145bb0 con 0x7f2cb0005050 2021-11-21T22:01:40.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd0143ae0 con 0x7f2cb0005050 2021-11-21T22:01:40.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd0145d40 con 0x7f2cb0005050 2021-11-21T22:01:40.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccd7fa700 1 -- 172.21.2.116:0/2974463244 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2cc80072d0 con 0x7f2cb0005050 2021-11-21T22:01:40.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccd7fa700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2cb4053640 0x7f2cb4055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccd7fa700 1 -- 172.21.2.116:0/2974463244 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2cc806bbe0 con 0x7f2cb0005050 2021-11-21T22:01:40.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.654+0000 7f2ccf7fe700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2cb4053640 0x7f2cb4055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.658+0000 7f2ccf7fe700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2cb4053640 0x7f2cb4055b00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f2cc0007000 tx=0x7f2cc0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.658+0000 7f2ccd7fa700 1 -- 172.21.2.116:0/2974463244 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2cc8043070 con 0x7f2cb0005050 2021-11-21T22:01:40.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.738+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2cd00573d0 con 0x7f2cb0005050 2021-11-21T22:01:40.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.738+0000 7f2ccd7fa700 1 -- 172.21.2.116:0/2974463244 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v106) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2cc8046690 con 0x7f2cb0005050 2021-11-21T22:01:40.744 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:40.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.738+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2cb4053640 msgr2=0x7f2cb4055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.738+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2cb4053640 0x7f2cb4055b00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f2cc0007000 tx=0x7f2cc0008040).stop 2021-11-21T22:01:40.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.738+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 msgr2=0x7f2cd0062b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.738+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cd0062b60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f2cc8008000 tx=0x7f2cc8009040).stop 2021-11-21T22:01:40.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 shutdown_connections 2021-11-21T22:01:40.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2cd00b3450 0x7f2cd0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cd0145500 0x7f2cd0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2cb4053640 0x7f2cb4055b00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 --2- 172.21.2.116:0/2974463244 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cb0005050 0x7f2cd0062b60 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 >> 172.21.2.116:0/2974463244 conn(0x7f2cd005bd20 msgr2=0x7f2cd013d600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 shutdown_connections 2021-11-21T22:01:40.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.742+0000 7f2cd6942700 1 -- 172.21.2.116:0/2974463244 wait complete. 2021-11-21T22:01:40.750 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T22:01:40.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 Processor -- start 2021-11-21T22:01:40.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- start start 2021-11-21T22:01:40.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35b40a86b0 0x7f35b414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b40a6e80 con 0x7f35b40a6aa0 2021-11-21T22:01:40.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b414a7b0 con 0x7f35b4062410 2021-11-21T22:01:40.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b40a8bc0 con 0x7f35b40a86b0 2021-11-21T22:01:40.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b1d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35b40a86b0 0x7f35b414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b1d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35b40a86b0 0x7f35b414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51274/0 (socket says 172.21.2.116:51274) 2021-11-21T22:01:40.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b1d9b700 1 -- 172.21.2.116:0/2711815811 learned_addr learned my addr 172.21.2.116:0/2711815811 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b259c700 1 --1- 172.21.2.116:0/2711815811 >> v1:172.21.2.116:6789/0 conn(0x7f35b40a6aa0 0x7f35b41470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35000/0 (socket says 172.21.2.116:35000) 2021-11-21T22:01:40.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b1d9b700 1 -- 172.21.2.116:0/2711815811 >> v1:172.21.2.117:6789/0 conn(0x7f35b4062410 legacy=0x7f35b41443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:40.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b1d9b700 1 -- 172.21.2.116:0/2711815811 >> v1:172.21.2.116:6789/0 conn(0x7f35b40a6aa0 legacy=0x7f35b41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b1d9b700 1 -- 172.21.2.116:0/2711815811 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b414aa10 con 0x7f35b40a86b0 2021-11-21T22:01:40.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b1d9b700 1 --2- 172.21.2.116:0/2711815811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b40a86b0 0x7f35b414b150 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f35b4061b50 tx=0x7f35a4006eb0).ready entity=mon.0 client_cookie=51e331a7ee16ad28 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b159a700 1 -- 172.21.2.116:0/2711815811 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35a400d610 con 0x7f35b40a86b0 2021-11-21T22:01:40.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b159a700 1 -- 172.21.2.116:0/2711815811 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4000e90 con 0x7f35b40a86b0 2021-11-21T22:01:40.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b159a700 1 -- 172.21.2.116:0/2711815811 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35a400d960 con 0x7f35b40a86b0 2021-11-21T22:01:40.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- 172.21.2.116:0/2711815811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b40a86b0 msgr2=0x7f35b414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 --2- 172.21.2.116:0/2711815811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b40a86b0 0x7f35b414b150 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f35b4061b50 tx=0x7f35a4006eb0).stop 2021-11-21T22:01:40.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- 172.21.2.116:0/2711815811 shutdown_connections 2021-11-21T22:01:40.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 --2- 172.21.2.116:0/2711815811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b40a86b0 0x7f35b414b150 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- 172.21.2.116:0/2711815811 >> 172.21.2.116:0/2711815811 conn(0x7f35b405bd20 msgr2=0x7f35b405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- 172.21.2.116:0/2711815811 shutdown_connections 2021-11-21T22:01:40.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- 172.21.2.116:0/2711815811 wait complete. 2021-11-21T22:01:40.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 Processor -- start 2021-11-21T22:01:40.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- start start 2021-11-21T22:01:40.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b4062410 0x7f35b4147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35b40a6aa0 0x7f35b4064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35b40a86b0 0x7f35b415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b41567f0 con 0x7f35b4062410 2021-11-21T22:01:40.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40aaca0 con 0x7f35b40a86b0 2021-11-21T22:01:40.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b89f2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40aa4c0 con 0x7f35b40a6aa0 2021-11-21T22:01:40.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b4062410 0x7f35b4147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b4062410 0x7f35b4147440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51280/0 (socket says 172.21.2.116:51280) 2021-11-21T22:01:40.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.798+0000 7f35b259c700 1 -- 172.21.2.116:0/3003920550 learned_addr learned my addr 172.21.2.116:0/3003920550 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b259c700 1 -- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35b40a6aa0 msgr2=0x7f35b4064ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b259c700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35b40a6aa0 0x7f35b4064ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b259c700 1 -- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35b40a86b0 msgr2=0x7f35b415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b259c700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35b40a86b0 0x7f35b415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b259c700 1 -- 172.21.2.116:0/3003920550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a400b1e0 con 0x7f35b4062410 2021-11-21T22:01:40.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b259c700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b4062410 0x7f35b4147440 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f35b4061ab0 tx=0x7f359c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35ab7fe700 1 -- 172.21.2.116:0/3003920550 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359c00ce30 con 0x7f35b4062410 2021-11-21T22:01:40.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35ab7fe700 1 -- 172.21.2.116:0/3003920550 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c0072e0 con 0x7f35b4062410 2021-11-21T22:01:40.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35ab7fe700 1 -- 172.21.2.116:0/3003920550 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359c004710 con 0x7f35b4062410 2021-11-21T22:01:40.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b40aa760 con 0x7f35b4062410 2021-11-21T22:01:40.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b40aa1b0 con 0x7f35b4062410 2021-11-21T22:01:40.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b4159dd0 con 0x7f35b4062410 2021-11-21T22:01:40.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35ab7fe700 1 -- 172.21.2.116:0/3003920550 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f359c004870 con 0x7f35b4062410 2021-11-21T22:01:40.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35ab7fe700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3598053580 0x7f3598055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35ab7fe700 1 -- 172.21.2.116:0/3003920550 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f359c06aff0 con 0x7f35b4062410 2021-11-21T22:01:40.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b1d9b700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3598053580 0x7f3598055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.802+0000 7f35b1d9b700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3598053580 0x7f3598055a40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f35b4061b50 tx=0x7f35a4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.806+0000 7f35ab7fe700 1 -- 172.21.2.116:0/3003920550 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f359c042370 con 0x7f35b4062410 2021-11-21T22:01:40.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f35b40573d0 con 0x7f35b4062410 2021-11-21T22:01:40.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35ab7fe700 1 -- 172.21.2.116:0/3003920550 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) v1 ==== 84+0+143 (secure 0 0 0) 0x7f359c045990 con 0x7f35b4062410 2021-11-21T22:01:40.894 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:40.894 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T22:01:40.894 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.894 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T22:01:40.895 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T22:01:40.895 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:40.895 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:40.895 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T22:01:40.896 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T22:01:40.896 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:40.896 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T22:01:40.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3598053580 msgr2=0x7f3598055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3598053580 0x7f3598055a40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f35b4061b50 tx=0x7f35a4015000).stop 2021-11-21T22:01:40.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b4062410 msgr2=0x7f35b4147440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b4062410 0x7f35b4147440 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f35b4061ab0 tx=0x7f359c007bd0).stop 2021-11-21T22:01:40.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 shutdown_connections 2021-11-21T22:01:40.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35b40a86b0 0x7f35b415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35b40a6aa0 0x7f35b4064ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3598053580 0x7f3598055a40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 --2- 172.21.2.116:0/3003920550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35b4062410 0x7f35b4147440 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 >> 172.21.2.116:0/3003920550 conn(0x7f35b405bd20 msgr2=0x7f35b41441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 shutdown_connections 2021-11-21T22:01:40.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.890+0000 7f35b89f2700 1 -- 172.21.2.116:0/3003920550 wait complete. 2021-11-21T22:01:40.901 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T22:01:40.902 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:01:40.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87af216700 1 Processor -- start 2021-11-21T22:01:40.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87af216700 1 -- start start 2021-11-21T22:01:40.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87af216700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 0x7f87a81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87af216700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a8147e80 con 0x7f87a8144810 2021-11-21T22:01:40.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87af216700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a80af5a0 con 0x7f87a80ac600 2021-11-21T22:01:40.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87af216700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80b1490 con 0x7f87a80abce0 2021-11-21T22:01:40.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87acfb2700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f87a8144810 0x7f87a80b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35012/0 (socket says 172.21.2.116:35012) 2021-11-21T22:01:40.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87acfb2700 1 -- 172.21.2.116:0/803773410 learned_addr learned my addr 172.21.2.116:0/803773410 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:40.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043420148 0 0) 0x7f87a8147e80 con 0x7f87a8144810 2021-11-21T22:01:40.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8788003260 con 0x7f87a8144810 2021-11-21T22:01:40.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a7fff700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 0x7f87a81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 854043914 0 0) 0x7f8788003260 con 0x7f87a8144810 2021-11-21T22:01:40.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 msgr2=0x7f87a81478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 0x7f87a81478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 >> v1:172.21.2.117:6789/0 conn(0x7f87a80ac600 legacy=0x7f87a80acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80b1fd0 con 0x7f87a8144810 2021-11-21T22:01:40.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8798001660 con 0x7f87a8144810 2021-11-21T22:01:40.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 >> v1:172.21.2.116:6789/0 conn(0x7f87a8144810 legacy=0x7f87a80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8788003780 0x7f8788003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f8788004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 0x7f8788004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80b1490 con 0x7f8788004500 2021-11-21T22:01:40.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878800b860 con 0x7f8788003780 2021-11-21T22:01:40.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878800ba20 con 0x7f87a80abce0 2021-11-21T22:01:40.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.950+0000 7f87a7fff700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f8788004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87ad7b3700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 0x7f8788004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a7fff700 1 -- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 msgr2=0x7f8788004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a7fff700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 0x7f8788004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a7fff700 1 -- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8788003780 msgr2=0x7f8788003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:40.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a7fff700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8788003780 0x7f8788003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:40.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a7fff700 1 -- 172.21.2.116:0/803773410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788003260 con 0x7f8788004500 2021-11-21T22:01:40.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87ad7b3700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a80abce0 0x7f8788004f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:40.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a7fff700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f8788004980 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f879000a000 tx=0x7f8790008040).ready entity=mon.0 client_cookie=14443a1345711a94 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8790013670 con 0x7f8788004500 2021-11-21T22:01:40.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87900137d0 con 0x7f8788004500 2021-11-21T22:01:40.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a77fe700 1 -- 172.21.2.116:0/803773410 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8790013b30 con 0x7f8788004500 2021-11-21T22:01:40.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/803773410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 msgr2=0x7f8788004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 --2- 172.21.2.116:0/803773410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f8788004980 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f879000a000 tx=0x7f8790008040).stop 2021-11-21T22:01:40.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87ad7b3700 1 -- 172.21.2.116:0/803773410 reap_dead start 2021-11-21T22:01:40.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/803773410 shutdown_connections 2021-11-21T22:01:40.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/803773410 >> 172.21.2.116:0/803773410 conn(0x7f87a805bd20 msgr2=0x7f87a805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:40.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/803773410 shutdown_connections 2021-11-21T22:01:40.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/803773410 wait complete. 2021-11-21T22:01:40.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 Processor -- start 2021-11-21T22:01:40.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- start start 2021-11-21T22:01:40.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f87a813cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a8146350 0x7f87a80b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87a8144d20 0x7f87a8145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80b4160 con 0x7f8788004500 2021-11-21T22:01:40.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8143420 con 0x7f87a8144d20 2021-11-21T22:01:40.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81436c0 con 0x7f87a8146350 2021-11-21T22:01:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f87a813cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f87a813cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51300/0 (socket says 172.21.2.116:51300) 2021-11-21T22:01:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 -- 172.21.2.116:0/4034686537 learned_addr learned my addr 172.21.2.116:0/4034686537 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87ad7b3700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87a8144d20 0x7f87a8145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a7fff700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a8146350 0x7f87a80b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 -- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a8146350 msgr2=0x7f87a80b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a8146350 0x7f87a80b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 -- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87a8144d20 msgr2=0x7f87a8145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:40.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87a8144d20 0x7f87a8145320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 -- 172.21.2.116:0/4034686537 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80b37d0 con 0x7f8788004500 2021-11-21T22:01:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87acfb2700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f87a813cfc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f8798001370 tx=0x7f87a8146e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a5ffb700 1 -- 172.21.2.116:0/4034686537 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8798007170 con 0x7f8788004500 2021-11-21T22:01:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a5ffb700 1 -- 172.21.2.116:0/4034686537 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87980072d0 con 0x7f8788004500 2021-11-21T22:01:40.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a8145810 con 0x7f8788004500 2021-11-21T22:01:40.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a80af760 con 0x7f8788004500 2021-11-21T22:01:40.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a5ffb700 1 -- 172.21.2.116:0/4034686537 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8798004920 con 0x7f8788004500 2021-11-21T22:01:40.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a8143880 con 0x7f8788004500 2021-11-21T22:01:40.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.954+0000 7f87a5ffb700 1 -- 172.21.2.116:0/4034686537 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f879801b070 con 0x7f8788004500 2021-11-21T22:01:40.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.958+0000 7f87a5ffb700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f878c053610 0x7f878c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:40.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.958+0000 7f87a5ffb700 1 -- 172.21.2.116:0/4034686537 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f879806afc0 con 0x7f8788004500 2021-11-21T22:01:40.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.958+0000 7f87a7fff700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f878c053610 0x7f878c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:40.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.958+0000 7f87a7fff700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f878c053610 0x7f878c055ad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f8790019000 tx=0x7f8790017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:40.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:40.958+0000 7f87a5ffb700 1 -- 172.21.2.116:0/4034686537 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8798042350 con 0x7f8788004500 2021-11-21T22:01:41.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:41.042+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f87a80573d0 con 0x7f8788004500 2021-11-21T22:01:42.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87a5ffb700 1 -- 172.21.2.116:0/4034686537 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v107) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8798045970 con 0x7f8788004500 2021-11-21T22:01:42.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f878c053610 msgr2=0x7f878c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f878c053610 0x7f878c055ad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f8790019000 tx=0x7f8790017000).stop 2021-11-21T22:01:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 msgr2=0x7f87a813cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f87a813cfc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f8798001370 tx=0x7f87a8146e80).stop 2021-11-21T22:01:42.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 shutdown_connections 2021-11-21T22:01:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f87a8144d20 0x7f87a8145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87a8146350 0x7f87a80b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f878c053610 0x7f878c055ad0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:42.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 --2- 172.21.2.116:0/4034686537 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8788004500 0x7f87a813cfc0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 >> 172.21.2.116:0/4034686537 conn(0x7f87a805bd20 msgr2=0x7f87a81400b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 shutdown_connections 2021-11-21T22:01:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.022+0000 7f87af216700 1 -- 172.21.2.116:0/4034686537 wait complete. 2021-11-21T22:01:42.030 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:01:42.034 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:01:42.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f74868700 1 Processor -- start 2021-11-21T22:01:42.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f74868700 1 -- start start 2021-11-21T22:01:42.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f74868700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0f700abce0 0x7f0f701478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:42.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f74868700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0f700ac600 0x7f0f700aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:42.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f74868700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f70147e80 con 0x7f0f70144810 2021-11-21T22:01:42.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f74868700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f700b1490 con 0x7f0f700abce0 2021-11-21T22:01:42.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f74868700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f700b1fd0 con 0x7f0f700ac600 2021-11-21T22:01:42.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f6e59c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0f70144810 0x7f0f700b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35032/0 (socket says 172.21.2.116:35032) 2021-11-21T22:01:42.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f6e59c700 1 -- 172.21.2.116:0/2870790095 learned_addr learned my addr 172.21.2.116:0/2870790095 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:42.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.082+0000 7f0f6dd9b700 1 --2- 172.21.2.116:0/2870790095 >> v2:172.21.2.116:3300/0 conn(0x7f0f700abce0 0x7f0f701478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:42.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6d59a700 1 -- 172.21.2.116:0/2870790095 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801158875 0 0) 0x7f0f70147e80 con 0x7f0f70144810 2021-11-21T22:01:42.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6d59a700 1 -- 172.21.2.116:0/2870790095 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f50003260 con 0x7f0f70144810 2021-11-21T22:01:42.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6dd9b700 1 -- 172.21.2.116:0/2870790095 >> v2:172.21.2.117:3300/0 conn(0x7f0f700ac600 msgr2=0x7f0f700aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:42.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6dd9b700 1 --2- 172.21.2.116:0/2870790095 >> v2:172.21.2.117:3300/0 conn(0x7f0f700ac600 0x7f0f700aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:42.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6dd9b700 1 -- 172.21.2.116:0/2870790095 >> v1:172.21.2.116:6789/0 conn(0x7f0f70144810 legacy=0x7f0f700b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:42.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6dd9b700 1 -- 172.21.2.116:0/2870790095 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700af5a0 con 0x7f0f700abce0 2021-11-21T22:01:42.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6dd9b700 1 --2- 172.21.2.116:0/2870790095 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f701478c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f0f60002f70 tx=0x7f0f60006df0).ready entity=mon.0 client_cookie=d9c1ec1eade58dd1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:42.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6d59a700 1 -- 172.21.2.116:0/2870790095 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f6000d570 con 0x7f0f700abce0 2021-11-21T22:01:42.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6d59a700 1 -- 172.21.2.116:0/2870790095 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f60000e90 con 0x7f0f700abce0 2021-11-21T22:01:42.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6d59a700 1 -- 172.21.2.116:0/2870790095 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f6000d8c0 con 0x7f0f700abce0 2021-11-21T22:01:42.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- 172.21.2.116:0/2870790095 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 msgr2=0x7f0f701478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:42.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 --2- 172.21.2.116:0/2870790095 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f701478c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f0f60002f70 tx=0x7f0f60006df0).stop 2021-11-21T22:01:42.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- 172.21.2.116:0/2870790095 shutdown_connections 2021-11-21T22:01:42.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 --2- 172.21.2.116:0/2870790095 >> v2:172.21.2.117:3300/0 conn(0x7f0f700ac600 0x7f0f700aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:42.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 --2- 172.21.2.116:0/2870790095 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f701478c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:42.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- 172.21.2.116:0/2870790095 >> 172.21.2.116:0/2870790095 conn(0x7f0f7005bd20 msgr2=0x7f0f7005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:42.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- 172.21.2.116:0/2870790095 shutdown_connections 2021-11-21T22:01:42.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- 172.21.2.116:0/2870790095 wait complete. 2021-11-21T22:01:42.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 Processor -- start 2021-11-21T22:01:42.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- start start 2021-11-21T22:01:42.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f70156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:42.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f700ac600 0x7f0f70143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:42.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f70144810 0x7f0f70066e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:42.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701541e0 con 0x7f0f700abce0 2021-11-21T22:01:42.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700b43e0 con 0x7f0f700ac600 2021-11-21T22:01:42.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700b3c00 con 0x7f0f70144810 2021-11-21T22:01:42.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f70156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:42.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f70156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51316/0 (socket says 172.21.2.116:51316) 2021-11-21T22:01:42.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 -- 172.21.2.116:0/3001211039 learned_addr learned my addr 172.21.2.116:0/3001211039 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:42.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 -- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f70144810 msgr2=0x7f0f70066e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:42.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f70144810 0x7f0f70066e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:42.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 -- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f700ac600 msgr2=0x7f0f70143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:42.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f700ac600 0x7f0f70143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:42.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 -- 172.21.2.116:0/3001211039 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6000b180 con 0x7f0f700abce0 2021-11-21T22:01:42.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f6e59c700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f70156f40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f0f701455f0 tx=0x7f0f58007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:42.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f677fe700 1 -- 172.21.2.116:0/3001211039 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f580070d0 con 0x7f0f700abce0 2021-11-21T22:01:42.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f677fe700 1 -- 172.21.2.116:0/3001211039 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f58007230 con 0x7f0f700abce0 2021-11-21T22:01:42.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f700b3ea0 con 0x7f0f700abce0 2021-11-21T22:01:42.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f70144d20 con 0x7f0f700abce0 2021-11-21T22:01:42.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.086+0000 7f0f677fe700 1 -- 172.21.2.116:0/3001211039 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f58005610 con 0x7f0f700abce0 2021-11-21T22:01:42.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.090+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f700af060 con 0x7f0f700abce0 2021-11-21T22:01:42.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.090+0000 7f0f677fe700 1 -- 172.21.2.116:0/3001211039 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0f58010070 con 0x7f0f700abce0 2021-11-21T22:01:42.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.090+0000 7f0f677fe700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f54053600 0x7f0f54055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:42.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.090+0000 7f0f677fe700 1 -- 172.21.2.116:0/3001211039 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f0f5806af40 con 0x7f0f700abce0 2021-11-21T22:01:42.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.090+0000 7f0f6dd9b700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f54053600 0x7f0f54055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:42.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.090+0000 7f0f6dd9b700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f54053600 0x7f0f54055ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f0f60002f70 tx=0x7f0f60014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:42.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.090+0000 7f0f677fe700 1 -- 172.21.2.116:0/3001211039 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f58042160 con 0x7f0f700abce0 2021-11-21T22:01:42.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:42.174+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f0f700573d0 con 0x7f0f700abce0 2021-11-21T22:01:43.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f677fe700 1 -- 172.21.2.116:0/3001211039 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v108) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0f58045780 con 0x7f0f700abce0 2021-11-21T22:01:43.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f54053600 msgr2=0x7f0f54055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f54053600 0x7f0f54055ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f0f60002f70 tx=0x7f0f60014000).stop 2021-11-21T22:01:43.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 msgr2=0x7f0f70156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f70156f40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f0f701455f0 tx=0x7f0f58007aa0).stop 2021-11-21T22:01:43.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 shutdown_connections 2021-11-21T22:01:43.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f70144810 0x7f0f70066e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:43.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f700ac600 0x7f0f70143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:43.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f54053600 0x7f0f54055ac0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 --2- 172.21.2.116:0/3001211039 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f700abce0 0x7f0f70156f40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.034+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 >> 172.21.2.116:0/3001211039 conn(0x7f0f7005bd20 msgr2=0x7f0f701406c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.038+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 shutdown_connections 2021-11-21T22:01:43.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.038+0000 7f0f74868700 1 -- 172.21.2.116:0/3001211039 wait complete. 2021-11-21T22:01:43.048 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:01:43.050 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:01:43.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 Processor -- start 2021-11-21T22:01:43.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- start start 2021-11-21T22:01:43.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa0ec0af620 0x7fa0ec0af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec060e80 con 0x7fa0ec0b1a60 2021-11-21T22:01:43.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec0b2870 con 0x7fa0ec0b2490 2021-11-21T22:01:43.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ec0ad600 con 0x7fa0ec0af620 2021-11-21T22:01:43.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0ea7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa0ec0af620 0x7fa0ec0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0ea7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa0ec0af620 0x7fa0ec0af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51326/0 (socket says 172.21.2.116:51326) 2021-11-21T22:01:43.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0ea7fc700 1 -- 172.21.2.116:0/3898586313 learned_addr learned my addr 172.21.2.116:0/3898586313 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:43.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0ea7fc700 1 -- 172.21.2.116:0/3898586313 >> v1:172.21.2.117:6789/0 conn(0x7fa0ec0b2490 legacy=0x7fa0ec0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0ea7fc700 1 -- 172.21.2.116:0/3898586313 >> v1:172.21.2.116:6789/0 conn(0x7fa0ec0b1a60 legacy=0x7fa0ec14ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0ea7fc700 1 -- 172.21.2.116:0/3898586313 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec060940 con 0x7fa0ec0af620 2021-11-21T22:01:43.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0ea7fc700 1 --2- 172.21.2.116:0/3898586313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec0af050 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fa0ec0b6b80 tx=0x7fa0e0006eb0).ready entity=mon.0 client_cookie=6da5acc4794e3704 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0e9ffb700 1 -- 172.21.2.116:0/3898586313 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0e000d610 con 0x7fa0ec0af620 2021-11-21T22:01:43.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0e9ffb700 1 -- 172.21.2.116:0/3898586313 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e0000e90 con 0x7fa0ec0af620 2021-11-21T22:01:43.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0e9ffb700 1 -- 172.21.2.116:0/3898586313 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0e000d960 con 0x7fa0ec0af620 2021-11-21T22:01:43.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- 172.21.2.116:0/3898586313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 msgr2=0x7fa0ec0af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3898586313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec0af050 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fa0ec0b6b80 tx=0x7fa0e0006eb0).stop 2021-11-21T22:01:43.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- 172.21.2.116:0/3898586313 shutdown_connections 2021-11-21T22:01:43.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3898586313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec0af050 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- 172.21.2.116:0/3898586313 >> 172.21.2.116:0/3898586313 conn(0x7fa0ec0b6000 msgr2=0x7fa0ec0b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- 172.21.2.116:0/3898586313 shutdown_connections 2021-11-21T22:01:43.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- 172.21.2.116:0/3898586313 wait complete. 2021-11-21T22:01:43.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 Processor -- start 2021-11-21T22:01:43.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.102+0000 7fa0f1513700 1 -- start start 2021-11-21T22:01:43.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0ec0b1a60 0x7fa0ec159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0ec0b2490 0x7fa0ec159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec15a4d0 con 0x7fa0ec0af620 2021-11-21T22:01:43.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec1541a0 con 0x7fa0ec0b2490 2021-11-21T22:01:43.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0ace50 con 0x7fa0ec0b1a60 2021-11-21T22:01:43.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec14ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51330/0 (socket says 172.21.2.116:51330) 2021-11-21T22:01:43.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 -- 172.21.2.116:0/3877135320 learned_addr learned my addr 172.21.2.116:0/3877135320 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:43.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 -- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0ec0b1a60 msgr2=0x7fa0ec159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0ea7fc700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0ec0b1a60 0x7fa0ec159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eb7fe700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0ec0b2490 0x7fa0ec159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0ec0b1a60 0x7fa0ec159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 -- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0ec0b2490 msgr2=0x7fa0ec159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0ec0b2490 0x7fa0ec159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 -- 172.21.2.116:0/3877135320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e000b1e0 con 0x7fa0ec0af620 2021-11-21T22:01:43.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0ea7fc700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0ec0b1a60 0x7fa0ec159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0eaffd700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec14ad80 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fa0e40032b0 tx=0x7fa0e40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0dbfff700 1 -- 172.21.2.116:0/3877135320 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0e400d4a0 con 0x7fa0ec0af620 2021-11-21T22:01:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0dbfff700 1 -- 172.21.2.116:0/3877135320 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e400d600 con 0x7fa0ec0af620 2021-11-21T22:01:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec0641a0 con 0x7fa0ec0af620 2021-11-21T22:01:43.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0dbfff700 1 -- 172.21.2.116:0/3877135320 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0e400d960 con 0x7fa0ec0af620 2021-11-21T22:01:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec0646e0 con 0x7fa0ec0af620 2021-11-21T22:01:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec0573d0 con 0x7fa0ec0af620 2021-11-21T22:01:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0dbfff700 1 -- 172.21.2.116:0/3877135320 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa0e4018030 con 0x7fa0ec0af620 2021-11-21T22:01:43.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0dbfff700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c8053660 0x7fa0c8055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0dbfff700 1 -- 172.21.2.116:0/3877135320 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fa0e406b430 con 0x7fa0ec0af620 2021-11-21T22:01:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0ea7fc700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c8053660 0x7fa0c8055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.106+0000 7fa0ea7fc700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c8053660 0x7fa0c8055b20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fa0e0017000 tx=0x7fa0e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.110+0000 7fa0dbfff700 1 -- 172.21.2.116:0/3877135320 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa0e4042490 con 0x7fa0ec0af620 2021-11-21T22:01:43.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.222+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa0ec0648e0 con 0x7fa0ec0af620 2021-11-21T22:01:43.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.226+0000 7fa0dbfff700 1 -- 172.21.2.116:0/3877135320 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v48) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa0e4045ab0 con 0x7fa0ec0af620 2021-11-21T22:01:43.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c8053660 msgr2=0x7fa0c8055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c8053660 0x7fa0c8055b20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fa0e0017000 tx=0x7fa0e0015000).stop 2021-11-21T22:01:43.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 msgr2=0x7fa0ec14ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec14ad80 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fa0e40032b0 tx=0x7fa0e40091a0).stop 2021-11-21T22:01:43.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 shutdown_connections 2021-11-21T22:01:43.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0ec0b2490 0x7fa0ec159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0ec0b1a60 0x7fa0ec159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c8053660 0x7fa0c8055b20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 --2- 172.21.2.116:0/3877135320 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0ec0af620 0x7fa0ec14ad80 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 >> 172.21.2.116:0/3877135320 conn(0x7fa0ec0b6000 msgr2=0x7fa0ec148de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 shutdown_connections 2021-11-21T22:01:43.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.230+0000 7fa0f1513700 1 -- 172.21.2.116:0/3877135320 wait complete. 2021-11-21T22:01:43.236 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 19 and data pool 20 2021-11-21T22:01:43.239 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:43.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 Processor -- start 2021-11-21T22:01:43.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- start start 2021-11-21T22:01:43.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 0x7f9e90144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e900a6e80 con 0x7f9e900a86b0 2021-11-21T22:01:43.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9014a7b0 con 0x7f9e900a6aa0 2021-11-21T22:01:43.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e900a8bc0 con 0x7f9e90062410 2021-11-21T22:01:43.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8e7fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9e900a86b0 0x7f9e901443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35074/0 (socket says 172.21.2.116:35074) 2021-11-21T22:01:43.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8e7fc700 1 -- 172.21.2.116:0/2132563046 learned_addr learned my addr 172.21.2.116:0/2132563046 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:43.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024801416 0 0) 0x7f9e900a6e80 con 0x7f9e900a86b0 2021-11-21T22:01:43.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e70003260 con 0x7f9e900a86b0 2021-11-21T22:01:43.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 0x7f9e90144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3308703747 0 0) 0x7f9e70003260 con 0x7f9e900a86b0 2021-11-21T22:01:43.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 msgr2=0x7f9e90144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 0x7f9e90144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 >> v1:172.21.2.117:6789/0 conn(0x7f9e900a6aa0 legacy=0x7f9e901470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9014aa10 con 0x7f9e900a86b0 2021-11-21T22:01:43.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9e80001680 con 0x7f9e900a86b0 2021-11-21T22:01:43.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 >> v1:172.21.2.116:6789/0 conn(0x7f9e900a86b0 legacy=0x7f9e901443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 0x7f9e70003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e70003cb0 0x7f9e70004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e700049f0 0x7f9e70004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e900a8bc0 con 0x7f9e700049f0 2021-11-21T22:01:43.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7000b800 con 0x7f9e70003cb0 2021-11-21T22:01:43.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7000b9c0 con 0x7f9e90062410 2021-11-21T22:01:43.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e700049f0 0x7f9e70004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8e7fc700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e70003cb0 0x7f9e70004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8effd700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 0x7f9e70003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 -- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 msgr2=0x7f9e70003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e90062410 0x7f9e70003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 -- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e70003cb0 msgr2=0x7f9e70004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e70003cb0 0x7f9e70004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 -- 172.21.2.116:0/2132563046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9014aa10 con 0x7f9e700049f0 2021-11-21T22:01:43.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e700049f0 0x7f9e70004e70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f9e8800c000 tx=0x7f9e880091d0).ready entity=mon.0 client_cookie=1833dc0808960cca server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e8800e7e0 con 0x7f9e700049f0 2021-11-21T22:01:43.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8800e940 con 0x7f9e700049f0 2021-11-21T22:01:43.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8dffb700 1 -- 172.21.2.116:0/2132563046 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e8800eca0 con 0x7f9e700049f0 2021-11-21T22:01:43.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- 172.21.2.116:0/2132563046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e700049f0 msgr2=0x7f9e70004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 --2- 172.21.2.116:0/2132563046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e700049f0 0x7f9e70004e70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f9e8800c000 tx=0x7f9e880091d0).stop 2021-11-21T22:01:43.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e8f7fe700 1 -- 172.21.2.116:0/2132563046 reap_dead start 2021-11-21T22:01:43.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- 172.21.2.116:0/2132563046 shutdown_connections 2021-11-21T22:01:43.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- 172.21.2.116:0/2132563046 >> 172.21.2.116:0/2132563046 conn(0x7f9e9005bd20 msgr2=0x7f9e9005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- 172.21.2.116:0/2132563046 shutdown_connections 2021-11-21T22:01:43.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.286+0000 7f9e953f4700 1 -- 172.21.2.116:0/2132563046 wait complete. 2021-11-21T22:01:43.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 Processor -- start 2021-11-21T22:01:43.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 -- start start 2021-11-21T22:01:43.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e700049f0 0x7f9e900a9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e90064100 0x7f9e900655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e900a97f0 0x7f9e90065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015a4e0 con 0x7f9e90064100 2021-11-21T22:01:43.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e90159fd0 con 0x7f9e700049f0 2021-11-21T22:01:43.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015a240 con 0x7f9e900a97f0 2021-11-21T22:01:43.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e90064100 0x7f9e900655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e90064100 0x7f9e900655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51362/0 (socket says 172.21.2.116:51362) 2021-11-21T22:01:43.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 -- 172.21.2.116:0/1923236879 learned_addr learned my addr 172.21.2.116:0/1923236879 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:43.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8f7fe700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e900a97f0 0x7f9e90065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8effd700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e700049f0 0x7f9e900a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 -- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e900a97f0 msgr2=0x7f9e90065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e900a97f0 0x7f9e90065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 -- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e700049f0 msgr2=0x7f9e900a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e700049f0 0x7f9e900a9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 -- 172.21.2.116:0/1923236879 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e900649d0 con 0x7f9e90064100 2021-11-21T22:01:43.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8effd700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e700049f0 0x7f9e900a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:43.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8e7fc700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e90064100 0x7f9e900655c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f9e8000d000 tx=0x7f9e80001de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e87fff700 1 -- 172.21.2.116:0/1923236879 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e80011780 con 0x7f9e90064100 2021-11-21T22:01:43.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e87fff700 1 -- 172.21.2.116:0/1923236879 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e800118e0 con 0x7f9e90064100 2021-11-21T22:01:43.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e90065fa0 con 0x7f9e90064100 2021-11-21T22:01:43.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e87fff700 1 -- 172.21.2.116:0/1923236879 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e80011c40 con 0x7f9e90064100 2021-11-21T22:01:43.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e900b1550 con 0x7f9e90064100 2021-11-21T22:01:43.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e90064b60 con 0x7f9e90064100 2021-11-21T22:01:43.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e87fff700 1 -- 172.21.2.116:0/1923236879 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9e8001b030 con 0x7f9e90064100 2021-11-21T22:01:43.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e87fff700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e74053610 0x7f9e74055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e87fff700 1 -- 172.21.2.116:0/1923236879 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9e8006b570 con 0x7f9e90064100 2021-11-21T22:01:43.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.290+0000 7f9e8effd700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e74053610 0x7f9e74055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.294+0000 7f9e8effd700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e74053610 0x7f9e74055ad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f9e78009000 tx=0x7f9e78009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.294+0000 7f9e87fff700 1 -- 172.21.2.116:0/1923236879 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e800426d0 con 0x7f9e90064100 2021-11-21T22:01:43.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e9005db70 con 0x7f9e90064100 2021-11-21T22:01:43.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e87fff700 1 -- 172.21.2.116:0/1923236879 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f9e80019030 con 0x7f9e90064100 2021-11-21T22:01:43.383 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:43.384 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":110,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:43.226137+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:41.057032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:42.188874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:43.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e74053610 msgr2=0x7f9e74055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e74053610 0x7f9e74055ad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f9e78009000 tx=0x7f9e78009040).stop 2021-11-21T22:01:43.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e90064100 msgr2=0x7f9e900655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e90064100 0x7f9e900655c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f9e8000d000 tx=0x7f9e80001de0).stop 2021-11-21T22:01:43.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 shutdown_connections 2021-11-21T22:01:43.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e900a97f0 0x7f9e90065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e74053610 0x7f9e74055ad0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e90064100 0x7f9e900655c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 --2- 172.21.2.116:0/1923236879 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e700049f0 0x7f9e900a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 >> 172.21.2.116:0/1923236879 conn(0x7f9e9005bd20 msgr2=0x7f9e900b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.378+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 shutdown_connections 2021-11-21T22:01:43.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.382+0000 7f9e953f4700 1 -- 172.21.2.116:0/1923236879 wait complete. 2021-11-21T22:01:43.391 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:43.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 Processor -- start 2021-11-21T22:01:43.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- start start 2021-11-21T22:01:43.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9a140a86b0 0x7f9a1414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 0x7f9a141443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a140a6e80 con 0x7f9a140a6aa0 2021-11-21T22:01:43.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a140a8bc0 con 0x7f9a140a86b0 2021-11-21T22:01:43.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1414a7b0 con 0x7f9a14062410 2021-11-21T22:01:43.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a137fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9a140a6aa0 0x7f9a141470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35092/0 (socket says 172.21.2.116:35092) 2021-11-21T22:01:43.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a137fe700 1 -- 172.21.2.116:0/1158210480 learned_addr learned my addr 172.21.2.116:0/1158210480 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:43.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264489947 0 0) 0x7f9a140a6e80 con 0x7f9a140a6aa0 2021-11-21T22:01:43.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f4003260 con 0x7f9a140a6aa0 2021-11-21T22:01:43.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805637534 0 0) 0x7f99f4003260 con 0x7f9a140a6aa0 2021-11-21T22:01:43.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 msgr2=0x7f9a141443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a13fff700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 0x7f9a141443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 0x7f9a141443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/1158210480 >> v2:172.21.2.117:3300/0 conn(0x7f9a140a86b0 0x7f9a1414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 >> v2:172.21.2.117:3300/0 conn(0x7f9a140a86b0 msgr2=0x7f9a1414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 --2- 172.21.2.116:0/1158210480 >> v2:172.21.2.117:3300/0 conn(0x7f9a140a86b0 0x7f9a1414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1414aa10 con 0x7f9a140a6aa0 2021-11-21T22:01:43.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9a04001680 con 0x7f9a140a6aa0 2021-11-21T22:01:43.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 >> v1:172.21.2.116:6789/0 conn(0x7f9a140a6aa0 legacy=0x7f9a141470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 0x7f99f40037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99f4003e90 0x7f99f4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99f4004b10 0x7f99f4004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140a8bc0 con 0x7f99f4003e90 2021-11-21T22:01:43.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1414a7b0 con 0x7f99f4004b10 2021-11-21T22:01:43.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f400ba60 con 0x7f9a14062410 2021-11-21T22:01:43.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99f4003e90 0x7f99f4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a137fe700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 0x7f99f40037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 -- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 msgr2=0x7f99f40037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a14062410 0x7f99f40037d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 -- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99f4004b10 msgr2=0x7f99f4004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f99f4004b10 0x7f99f4004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:43.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 -- 172.21.2.116:0/1158210480 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1414aa10 con 0x7f99f4003e90 2021-11-21T22:01:43.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99f4003e90 0x7f99f4004500 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f99fc007000 tx=0x7f99fc008040).ready entity=mon.0 client_cookie=b97ab6c4e9026f64 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99fc012470 con 0x7f99f4003e90 2021-11-21T22:01:43.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00c030 con 0x7f99f4003e90 2021-11-21T22:01:43.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a127fc700 1 -- 172.21.2.116:0/1158210480 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99fc0127c0 con 0x7f99f4003e90 2021-11-21T22:01:43.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- 172.21.2.116:0/1158210480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99f4003e90 msgr2=0x7f99f4004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 --2- 172.21.2.116:0/1158210480 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f99f4003e90 0x7f99f4004500 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f99fc007000 tx=0x7f99fc008040).stop 2021-11-21T22:01:43.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a13fff700 1 -- 172.21.2.116:0/1158210480 reap_dead start 2021-11-21T22:01:43.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- 172.21.2.116:0/1158210480 shutdown_connections 2021-11-21T22:01:43.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- 172.21.2.116:0/1158210480 >> 172.21.2.116:0/1158210480 conn(0x7f9a1405bd20 msgr2=0x7f9a1405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- 172.21.2.116:0/1158210480 shutdown_connections 2021-11-21T22:01:43.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.478+0000 7f9a19b60700 1 -- 172.21.2.116:0/1158210480 wait complete. 2021-11-21T22:01:43.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 Processor -- start 2021-11-21T22:01:43.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 -- start start 2021-11-21T22:01:43.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a140b3ed0 0x7f9a14066f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a141563d0 0x7f9a14156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a14153530 0x7f9a141539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140b42b0 con 0x7f9a140b3ed0 2021-11-21T22:01:43.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1415a880 con 0x7f9a14153530 2021-11-21T22:01:43.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141599a0 con 0x7f9a141563d0 2021-11-21T22:01:43.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a140b3ed0 0x7f9a14066f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a140b3ed0 0x7f9a14066f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51380/0 (socket says 172.21.2.116:51380) 2021-11-21T22:01:43.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 -- 172.21.2.116:0/712083205 learned_addr learned my addr 172.21.2.116:0/712083205 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:43.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a141563d0 0x7f9a14156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 -- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a141563d0 msgr2=0x7f9a14156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a141563d0 0x7f9a14156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 -- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a14153530 msgr2=0x7f9a141539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a13fff700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a14153530 0x7f9a141539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a14153530 0x7f9a141539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 -- 172.21.2.116:0/712083205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a14159c00 con 0x7f9a140b3ed0 2021-11-21T22:01:43.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a141563d0 0x7f9a14156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:43.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a137fe700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a140b3ed0 0x7f9a14066f10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f9a04008000 tx=0x7f9a04009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a10ff9700 1 -- 172.21.2.116:0/712083205 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a04007170 con 0x7f9a140b3ed0 2021-11-21T22:01:43.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a10ff9700 1 -- 172.21.2.116:0/712083205 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040072d0 con 0x7f9a140b3ed0 2021-11-21T22:01:43.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a14159ea0 con 0x7f9a140b3ed0 2021-11-21T22:01:43.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a10ff9700 1 -- 172.21.2.116:0/712083205 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a040049e0 con 0x7f9a140b3ed0 2021-11-21T22:01:43.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1415a3e0 con 0x7f9a140b3ed0 2021-11-21T22:01:43.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1414abf0 con 0x7f9a140b3ed0 2021-11-21T22:01:43.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a10ff9700 1 -- 172.21.2.116:0/712083205 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9a04004b40 con 0x7f9a140b3ed0 2021-11-21T22:01:43.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a10ff9700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99f8053580 0x7f99f8055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a10ff9700 1 -- 172.21.2.116:0/712083205 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9a0406bed0 con 0x7f9a140b3ed0 2021-11-21T22:01:43.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.482+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99f8053580 0x7f99f8055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.486+0000 7f9a12ffd700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99f8053580 0x7f99f8055a40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f99fc013000 tx=0x7f99fc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.486+0000 7f9a10ff9700 1 -- 172.21.2.116:0/712083205 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a04043030 con 0x7f9a140b3ed0 2021-11-21T22:01:43.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.566+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a1405db70 con 0x7f9a140b3ed0 2021-11-21T22:01:43.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a10ff9700 1 -- 172.21.2.116:0/712083205 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f9a04046650 con 0x7f9a140b3ed0 2021-11-21T22:01:43.573 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:43.573 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":110,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:43.226137+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:41.057032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:42.188874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:43.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99f8053580 msgr2=0x7f99f8055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99f8053580 0x7f99f8055a40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f99fc013000 tx=0x7f99fc008040).stop 2021-11-21T22:01:43.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a140b3ed0 msgr2=0x7f9a14066f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a140b3ed0 0x7f9a14066f10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f9a04008000 tx=0x7f9a04009040).stop 2021-11-21T22:01:43.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 shutdown_connections 2021-11-21T22:01:43.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9a14153530 0x7f9a141539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9a141563d0 0x7f9a14156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f99f8053580 0x7f99f8055a40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 --2- 172.21.2.116:0/712083205 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9a140b3ed0 0x7f9a14066f10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 >> 172.21.2.116:0/712083205 conn(0x7f9a1405bd20 msgr2=0x7f9a141441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 shutdown_connections 2021-11-21T22:01:43.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.570+0000 7f9a19b60700 1 -- 172.21.2.116:0/712083205 wait complete. 2021-11-21T22:01:43.581 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:01:43.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae7a264700 1 Processor -- start 2021-11-21T22:01:43.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae7a264700 1 -- start start 2021-11-21T22:01:43.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae7a264700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae741478c0 0x7fae740acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae7a264700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae74147e80 con 0x7fae740abce0 2021-11-21T22:01:43.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae7a264700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae740af5a0 con 0x7fae74144810 2021-11-21T22:01:43.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae7a264700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae740b1490 con 0x7fae741478c0 2021-11-21T22:01:43.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae737fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fae740abce0 0x7fae740ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35112/0 (socket says 172.21.2.116:35112) 2021-11-21T22:01:43.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae737fe700 1 -- 172.21.2.116:0/1478196843 learned_addr learned my addr 172.21.2.116:0/1478196843 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:43.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535017859 0 0) 0x7fae74147e80 con 0x7fae740abce0 2021-11-21T22:01:43.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae54003260 con 0x7fae740abce0 2021-11-21T22:01:43.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae78801700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae741478c0 0x7fae740acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4172438396 0 0) 0x7fae54003260 con 0x7fae740abce0 2021-11-21T22:01:43.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae741478c0 msgr2=0x7fae740acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae741478c0 0x7fae740acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:43.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 >> v1:172.21.2.117:6789/0 conn(0x7fae74144810 legacy=0x7fae740b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae740b1fd0 con 0x7fae740abce0 2021-11-21T22:01:43.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fae64001680 con 0x7fae740abce0 2021-11-21T22:01:43.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 >> v1:172.21.2.116:6789/0 conn(0x7fae740abce0 legacy=0x7fae740ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae54003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae741478c0 0x7fae54004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fae54004b10 0x7fae54004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740b1490 con 0x7fae54003800 2021-11-21T22:01:43.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5400b910 con 0x7fae54004b10 2021-11-21T22:01:43.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5400bad0 con 0x7fae741478c0 2021-11-21T22:01:43.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae73fff700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae54003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae73fff700 1 -- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae741478c0 msgr2=0x7fae54004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae73fff700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae741478c0 0x7fae54004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:43.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae73fff700 1 -- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fae54004b10 msgr2=0x7fae54004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:43.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae73fff700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fae54004b10 0x7fae54004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:43.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae73fff700 1 -- 172.21.2.116:0/1478196843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54003260 con 0x7fae54003800 2021-11-21T22:01:43.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.630+0000 7fae73fff700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae54003c00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7fae5c001fa0 tx=0x7fae5c007a00).ready entity=mon.0 client_cookie=109d698f5a8544ad server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae5c007070 con 0x7fae54003800 2021-11-21T22:01:43.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c0071d0 con 0x7fae54003800 2021-11-21T22:01:43.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae72ffd700 1 -- 172.21.2.116:0/1478196843 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae5c006510 con 0x7fae54003800 2021-11-21T22:01:43.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/1478196843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 msgr2=0x7fae54003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 --2- 172.21.2.116:0/1478196843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae54003c00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7fae5c001fa0 tx=0x7fae5c007a00).stop 2021-11-21T22:01:43.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae78801700 1 -- 172.21.2.116:0/1478196843 reap_dead start 2021-11-21T22:01:43.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/1478196843 shutdown_connections 2021-11-21T22:01:43.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/1478196843 >> 172.21.2.116:0/1478196843 conn(0x7fae7405bd20 msgr2=0x7fae7405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:43.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/1478196843 shutdown_connections 2021-11-21T22:01:43.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/1478196843 wait complete. 2021-11-21T22:01:43.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 Processor -- start 2021-11-21T22:01:43.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- start start 2021-11-21T22:01:43.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae740b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fae741563a0 0x7fae74159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae740b4220 0x7fae7415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740b4c00 con 0x7fae54003800 2021-11-21T22:01:43.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740b46c0 con 0x7fae741563a0 2021-11-21T22:01:43.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae740b4960 con 0x7fae740b4220 2021-11-21T22:01:43.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae740b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae740b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51400/0 (socket says 172.21.2.116:51400) 2021-11-21T22:01:43.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 -- 172.21.2.116:0/572610892 learned_addr learned my addr 172.21.2.116:0/572610892 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:43.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 -- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae740b4220 msgr2=0x7fae7415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae740b4220 0x7fae7415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:43.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 -- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fae741563a0 msgr2=0x7fae74159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:43.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fae741563a0 0x7fae74159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:43.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 -- 172.21.2.116:0/572610892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae74159a80 con 0x7fae54003800 2021-11-21T22:01:43.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae73fff700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae740b5030 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fae5c001fa0 tx=0x7fae5c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae717fa700 1 -- 172.21.2.116:0/572610892 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae5c014620 con 0x7fae54003800 2021-11-21T22:01:43.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae717fa700 1 -- 172.21.2.116:0/572610892 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c000e90 con 0x7fae54003800 2021-11-21T22:01:43.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae740b5520 con 0x7fae54003800 2021-11-21T22:01:43.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae717fa700 1 -- 172.21.2.116:0/572610892 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae5c014970 con 0x7fae54003800 2021-11-21T22:01:43.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae740b3c10 con 0x7fae54003800 2021-11-21T22:01:43.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.634+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae740af060 con 0x7fae54003800 2021-11-21T22:01:43.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.638+0000 7fae717fa700 1 -- 172.21.2.116:0/572610892 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fae5c01d070 con 0x7fae54003800 2021-11-21T22:01:43.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.638+0000 7fae717fa700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fae58053660 0x7fae58055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:43.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.638+0000 7fae717fa700 1 -- 172.21.2.116:0/572610892 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fae5c06cac0 con 0x7fae54003800 2021-11-21T22:01:43.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.638+0000 7fae737fe700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fae58053660 0x7fae58055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:43.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.638+0000 7fae737fe700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fae58053660 0x7fae58055b20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fae740b4e00 tx=0x7fae64001590).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:43.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.638+0000 7fae717fa700 1 -- 172.21.2.116:0/572610892 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fae5c043c20 con 0x7fae54003800 2021-11-21T22:01:43.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:43.746+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fae74067cc0 con 0x7fae54003800 2021-11-21T22:01:44.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.238+0000 7fae717fa700 1 -- 172.21.2.116:0/572610892 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae5c047240 con 0x7fae54003800 2021-11-21T22:01:44.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fae58053660 msgr2=0x7fae58055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fae58053660 0x7fae58055b20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fae740b4e00 tx=0x7fae64001590).stop 2021-11-21T22:01:44.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 msgr2=0x7fae740b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae740b5030 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fae5c001fa0 tx=0x7fae5c015000).stop 2021-11-21T22:01:44.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 shutdown_connections 2021-11-21T22:01:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fae740b4220 0x7fae7415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fae741563a0 0x7fae74159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fae58053660 0x7fae58055b20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 --2- 172.21.2.116:0/572610892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fae54003800 0x7fae740b5030 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 >> 172.21.2.116:0/572610892 conn(0x7fae7405bd20 msgr2=0x7fae741484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 shutdown_connections 2021-11-21T22:01:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.242+0000 7fae7a264700 1 -- 172.21.2.116:0/572610892 wait complete. 2021-11-21T22:01:44.251 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:44.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 Processor -- start 2021-11-21T22:01:44.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- start start 2021-11-21T22:01:44.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f71900a6aa0 0x7f719014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 0x7f71901443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71900a6e80 con 0x7f71900a86b0 2021-11-21T22:01:44.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71900a8bc0 con 0x7f71900a6aa0 2021-11-21T22:01:44.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719014a7b0 con 0x7f7190062410 2021-11-21T22:01:44.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f71900a6aa0 0x7f719014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f71900a6aa0 0x7f719014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51408/0 (socket says 172.21.2.116:51408) 2021-11-21T22:01:44.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 -- 172.21.2.116:0/2564991873 learned_addr learned my addr 172.21.2.116:0/2564991873 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:44.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718dd9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f71900a86b0 0x7f71901470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35136/0 (socket says 172.21.2.116:35136) 2021-11-21T22:01:44.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 -- 172.21.2.116:0/2564991873 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 msgr2=0x7f71901443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:44.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 --2- 172.21.2.116:0/2564991873 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 0x7f71901443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 -- 172.21.2.116:0/2564991873 >> v1:172.21.2.116:6789/0 conn(0x7f71900a86b0 legacy=0x7f71901470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 -- 172.21.2.116:0/2564991873 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719014aa10 con 0x7f71900a6aa0 2021-11-21T22:01:44.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718e59c700 1 --2- 172.21.2.116:0/2564991873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719014b150 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f7180002f70 tx=0x7f7180006eb0).ready entity=mon.0 client_cookie=9d68f17033e4c907 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718d59a700 1 -- 172.21.2.116:0/2564991873 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f718000d5c0 con 0x7f71900a6aa0 2021-11-21T22:01:44.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718d59a700 1 -- 172.21.2.116:0/2564991873 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7180000e90 con 0x7f71900a6aa0 2021-11-21T22:01:44.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f718d59a700 1 -- 172.21.2.116:0/2564991873 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f718000d950 con 0x7f71900a6aa0 2021-11-21T22:01:44.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- 172.21.2.116:0/2564991873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 msgr2=0x7f719014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 --2- 172.21.2.116:0/2564991873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719014b150 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f7180002f70 tx=0x7f7180006eb0).stop 2021-11-21T22:01:44.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- 172.21.2.116:0/2564991873 shutdown_connections 2021-11-21T22:01:44.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 --2- 172.21.2.116:0/2564991873 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 0x7f71901443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 --2- 172.21.2.116:0/2564991873 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719014b150 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- 172.21.2.116:0/2564991873 >> 172.21.2.116:0/2564991873 conn(0x7f719005bd20 msgr2=0x7f719005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- 172.21.2.116:0/2564991873 shutdown_connections 2021-11-21T22:01:44.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.314+0000 7f719497c700 1 -- 172.21.2.116:0/2564991873 wait complete. 2021-11-21T22:01:44.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 Processor -- start 2021-11-21T22:01:44.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 -- start start 2021-11-21T22:01:44.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 0x7f719013c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719013d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f71900a86b0 0x7f719013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901455b0 con 0x7f71900a6aa0 2021-11-21T22:01:44.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190147320 con 0x7f71900a86b0 2021-11-21T22:01:44.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7190141400 con 0x7f7190062410 2021-11-21T22:01:44.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719013d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719013d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51416/0 (socket says 172.21.2.116:51416) 2021-11-21T22:01:44.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 -- 172.21.2.116:0/2773962434 learned_addr learned my addr 172.21.2.116:0/2773962434 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:44.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 -- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 msgr2=0x7f719013c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 0x7f719013c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 -- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f71900a86b0 msgr2=0x7f719013d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f71900a86b0 0x7f719013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 -- 172.21.2.116:0/2773962434 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718000b1e0 con 0x7f71900a6aa0 2021-11-21T22:01:44.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f718dd9b700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719013d3c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f7190061ab0 tx=0x7f7178007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f71877fe700 1 -- 172.21.2.116:0/2773962434 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f717800ce30 con 0x7f71900a6aa0 2021-11-21T22:01:44.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f71877fe700 1 -- 172.21.2.116:0/2773962434 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178007170 con 0x7f71900a6aa0 2021-11-21T22:01:44.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7190146de0 con 0x7f71900a6aa0 2021-11-21T22:01:44.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f71877fe700 1 -- 172.21.2.116:0/2773962434 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7178004730 con 0x7f71900a6aa0 2021-11-21T22:01:44.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719013e7e0 con 0x7f71900a6aa0 2021-11-21T22:01:44.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71900573d0 con 0x7f71900a6aa0 2021-11-21T22:01:44.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f71877fe700 1 -- 172.21.2.116:0/2773962434 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7178004890 con 0x7f71900a6aa0 2021-11-21T22:01:44.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f71877fe700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7174053690 0x7f7174055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.318+0000 7f71877fe700 1 -- 172.21.2.116:0/2773962434 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f717806b2c0 con 0x7f71900a6aa0 2021-11-21T22:01:44.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.322+0000 7f718e59c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7174053690 0x7f7174055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.322+0000 7f718e59c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7174053690 0x7f7174055b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7180002f70 tx=0x7f7180015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.322+0000 7f71877fe700 1 -- 172.21.2.116:0/2773962434 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7178042310 con 0x7f71900a6aa0 2021-11-21T22:01:44.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.430+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f719005db70 con 0x7f71900a6aa0 2021-11-21T22:01:44.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.430+0000 7f71877fe700 1 -- 172.21.2.116:0/2773962434 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f7178045930 con 0x7f71900a6aa0 2021-11-21T22:01:44.437 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:44.437 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":49,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:43.226243+0000","modified":"2021-11-21T22:01:44.230974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1808795000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6833","nonce":1808795000},{"type":"v1","addr":"172.21.2.116:6835","nonce":1808795000}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-11-21T22:01:44.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7174053690 msgr2=0x7f7174055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7174053690 0x7f7174055b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7180002f70 tx=0x7f7180015000).stop 2021-11-21T22:01:44.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 msgr2=0x7f719013d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719013d3c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f7190061ab0 tx=0x7f7178007c10).stop 2021-11-21T22:01:44.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 shutdown_connections 2021-11-21T22:01:44.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f71900a86b0 0x7f719013d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7174053690 0x7f7174055b50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f71900a6aa0 0x7f719013d3c0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 --2- 172.21.2.116:0/2773962434 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7190062410 0x7f719013c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 >> 172.21.2.116:0/2773962434 conn(0x7f719005bd20 msgr2=0x7f7190149c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 shutdown_connections 2021-11-21T22:01:44.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.434+0000 7f719497c700 1 -- 172.21.2.116:0/2773962434 wait complete. 2021-11-21T22:01:44.441 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 49 2021-11-21T22:01:44.444 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:44.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 Processor -- start 2021-11-21T22:01:44.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 -- start start 2021-11-21T22:01:44.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd740b57d0 0x7fcd74066d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcd74146970 0x7fcd74143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcd74143640 0x7fcd740996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd740b1260 con 0x7fcd74146970 2021-11-21T22:01:44.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd74067670 con 0x7fcd74143640 2021-11-21T22:01:44.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd79418700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740b5bb0 con 0x7fcd740b57d0 2021-11-21T22:01:44.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcd74146970 0x7fcd74143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcd74146970 0x7fcd74143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51422/0 (socket says 172.21.2.116:51422) 2021-11-21T22:01:44.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 -- 172.21.2.116:0/1097346848 learned_addr learned my addr 172.21.2.116:0/1097346848 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:44.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 -- 172.21.2.116:0/1097346848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd740b57d0 msgr2=0x7fcd74066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:44.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 --2- 172.21.2.116:0/1097346848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd740b57d0 0x7fcd74066d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 -- 172.21.2.116:0/1097346848 >> v2:172.21.2.117:3300/0 conn(0x7fcd74143640 msgr2=0x7fcd740996b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:44.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 --2- 172.21.2.116:0/1097346848 >> v2:172.21.2.117:3300/0 conn(0x7fcd74143640 0x7fcd740996b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 -- 172.21.2.116:0/1097346848 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740a89a0 con 0x7fcd74146970 2021-11-21T22:01:44.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.490+0000 7fcd737fe700 1 --2- 172.21.2.116:0/1097346848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd74146970 0x7fcd74143150 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fcd5c002f70 tx=0x7fcd5c006eb0).ready entity=mon.0 client_cookie=4b89df9a6348063e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd72ffd700 1 -- 172.21.2.116:0/1097346848 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5c00d570 con 0x7fcd74146970 2021-11-21T22:01:44.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd72ffd700 1 -- 172.21.2.116:0/1097346848 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c000e90 con 0x7fcd74146970 2021-11-21T22:01:44.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd72ffd700 1 -- 172.21.2.116:0/1097346848 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5c00d900 con 0x7fcd74146970 2021-11-21T22:01:44.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/1097346848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd74146970 msgr2=0x7fcd74143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 --2- 172.21.2.116:0/1097346848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd74146970 0x7fcd74143150 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fcd5c002f70 tx=0x7fcd5c006eb0).stop 2021-11-21T22:01:44.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/1097346848 shutdown_connections 2021-11-21T22:01:44.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 --2- 172.21.2.116:0/1097346848 >> v2:172.21.2.117:3300/0 conn(0x7fcd74143640 0x7fcd740996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 --2- 172.21.2.116:0/1097346848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd74146970 0x7fcd74143150 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 --2- 172.21.2.116:0/1097346848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd740b57d0 0x7fcd74066d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/1097346848 >> 172.21.2.116:0/1097346848 conn(0x7fcd7405c7f0 msgr2=0x7fcd740b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/1097346848 shutdown_connections 2021-11-21T22:01:44.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/1097346848 wait complete. 2021-11-21T22:01:44.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 Processor -- start 2021-11-21T22:01:44.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- start start 2021-11-21T22:01:44.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd740b57d0 0x7fcd7413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd74143640 0x7fcd74152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd74146970 0x7fcd741552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74140170 con 0x7fcd740b57d0 2021-11-21T22:01:44.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74151e90 con 0x7fcd74143640 2021-11-21T22:01:44.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74152130 con 0x7fcd74146970 2021-11-21T22:01:44.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd740b57d0 0x7fcd7413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd740b57d0 0x7fcd7413f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51428/0 (socket says 172.21.2.116:51428) 2021-11-21T22:01:44.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 -- 172.21.2.116:0/186167366 learned_addr learned my addr 172.21.2.116:0/186167366 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:44.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 -- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd74146970 msgr2=0x7fcd741552d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:44.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd74146970 0x7fcd741552d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 -- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd74143640 msgr2=0x7fcd74152870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:44.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd74143640 0x7fcd74152870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 -- 172.21.2.116:0/186167366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c00b1e0 con 0x7fcd740b57d0 2021-11-21T22:01:44.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd73fff700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd740b57d0 0x7fcd7413f6b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7fcd64002fb0 tx=0x7fcd64006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd717fa700 1 -- 172.21.2.116:0/186167366 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd64013460 con 0x7fcd740b57d0 2021-11-21T22:01:44.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd717fa700 1 -- 172.21.2.116:0/186167366 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640135c0 con 0x7fcd740b57d0 2021-11-21T22:01:44.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd741523d0 con 0x7fcd740b57d0 2021-11-21T22:01:44.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7413fe40 con 0x7fcd740b57d0 2021-11-21T22:01:44.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd717fa700 1 -- 172.21.2.116:0/186167366 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd64013920 con 0x7fcd740b57d0 2021-11-21T22:01:44.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd74152560 con 0x7fcd740b57d0 2021-11-21T22:01:44.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd717fa700 1 -- 172.21.2.116:0/186167366 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcd6400f070 con 0x7fcd740b57d0 2021-11-21T22:01:44.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd717fa700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd58053540 0x7fcd58055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.494+0000 7fcd717fa700 1 -- 172.21.2.116:0/186167366 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcd6406b500 con 0x7fcd740b57d0 2021-11-21T22:01:44.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.498+0000 7fcd737fe700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd58053540 0x7fcd58055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.498+0000 7fcd737fe700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd58053540 0x7fcd58055a00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fcd5c002f70 tx=0x7fcd5c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.498+0000 7fcd717fa700 1 -- 172.21.2.116:0/186167366 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd64042550 con 0x7fcd740b57d0 2021-11-21T22:01:44.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd74067860 con 0x7fcd740b57d0 2021-11-21T22:01:44.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd717fa700 1 -- 172.21.2.116:0/186167366 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16052 (secure 0 0 0) 0x7fcd64045b70 con 0x7fcd740b57d0 2021-11-21T22:01:44.586 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:44.586 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":111,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:44.231112+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:41.057032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:42.188874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:44.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd58053540 msgr2=0x7fcd58055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd58053540 0x7fcd58055a00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fcd5c002f70 tx=0x7fcd5c015000).stop 2021-11-21T22:01:44.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd740b57d0 msgr2=0x7fcd7413f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd740b57d0 0x7fcd7413f6b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7fcd64002fb0 tx=0x7fcd64006df0).stop 2021-11-21T22:01:44.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 shutdown_connections 2021-11-21T22:01:44.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcd74146970 0x7fcd741552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcd74143640 0x7fcd74152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcd58053540 0x7fcd58055a00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 --2- 172.21.2.116:0/186167366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcd740b57d0 0x7fcd7413f6b0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 >> 172.21.2.116:0/186167366 conn(0x7fcd7405c7f0 msgr2=0x7fcd74099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 shutdown_connections 2021-11-21T22:01:44.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.582+0000 7fcd79418700 1 -- 172.21.2.116:0/186167366 wait complete. 2021-11-21T22:01:44.594 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:01:44.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.642+0000 7f59f521d700 1 Processor -- start 2021-11-21T22:01:44.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- start start 2021-11-21T22:01:44.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f59f00a8670 0x7f59f0147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59f0143e60 0x7f59f01442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f00b1940 con 0x7f59f00b11c0 2021-11-21T22:01:44.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f00b15a0 con 0x7f59f00a8670 2021-11-21T22:01:44.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00b1b70 con 0x7f59f0143e60 2021-11-21T22:01:44.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f59f00b11c0 0x7f59f0147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35158/0 (socket says 172.21.2.116:35158) 2021-11-21T22:01:44.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 -- 172.21.2.116:0/2441291938 learned_addr learned my addr 172.21.2.116:0/2441291938 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:44.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262255377 0 0) 0x7f59f00b1940 con 0x7f59f00b11c0 2021-11-21T22:01:44.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d0003260 con 0x7f59f00b11c0 2021-11-21T22:01:44.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59ef59e700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59f0143e60 0x7f59f01442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59ee59c700 1 --2- 172.21.2.116:0/2441291938 >> v2:172.21.2.117:3300/0 conn(0x7f59f00a8670 0x7f59f0147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 696021519 0 0) 0x7f59d0003260 con 0x7f59f00b11c0 2021-11-21T22:01:44.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59f0143e60 msgr2=0x7f59f01442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:44.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59f0143e60 0x7f59f01442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 >> v2:172.21.2.117:3300/0 conn(0x7f59f00a8670 msgr2=0x7f59f0147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:44.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 --2- 172.21.2.116:0/2441291938 >> v2:172.21.2.117:3300/0 conn(0x7f59f00a8670 0x7f59f0147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00b1e10 con 0x7f59f00b11c0 2021-11-21T22:01:44.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f59d8001660 con 0x7f59f00b11c0 2021-11-21T22:01:44.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 >> v1:172.21.2.116:6789/0 conn(0x7f59f00b11c0 legacy=0x7f59f0147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59d00037d0 0x7f59d0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59d0004500 0x7f59d0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59f0143e60 0x7f59d0004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00b15a0 con 0x7f59d00037d0 2021-11-21T22:01:44.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00b1b70 con 0x7f59d0004500 2021-11-21T22:01:44.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d000b8d0 con 0x7f59f0143e60 2021-11-21T22:01:44.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59d00037d0 0x7f59d0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 -- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59f0143e60 msgr2=0x7f59d0004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:44.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59f0143e60 0x7f59d0004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 -- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59d0004500 msgr2=0x7f59d0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:44.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59d0004500 0x7f59d0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 -- 172.21.2.116:0/2441291938 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0003260 con 0x7f59d00037d0 2021-11-21T22:01:44.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59eed9d700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59d00037d0 0x7f59d0003bf0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f59d8001370 tx=0x7f59f00b2350).ready entity=mon.0 client_cookie=d4ece92347cbf9a9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59d80070d0 con 0x7f59d00037d0 2021-11-21T22:01:44.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8007230 con 0x7f59d00037d0 2021-11-21T22:01:44.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59edd9b700 1 -- 172.21.2.116:0/2441291938 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59d800c8f0 con 0x7f59d00037d0 2021-11-21T22:01:44.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- 172.21.2.116:0/2441291938 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59d00037d0 msgr2=0x7f59d0003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 --2- 172.21.2.116:0/2441291938 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59d00037d0 0x7f59d0003bf0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f59d8001370 tx=0x7f59f00b2350).stop 2021-11-21T22:01:44.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59ef59e700 1 -- 172.21.2.116:0/2441291938 reap_dead start 2021-11-21T22:01:44.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- 172.21.2.116:0/2441291938 shutdown_connections 2021-11-21T22:01:44.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- 172.21.2.116:0/2441291938 >> 172.21.2.116:0/2441291938 conn(0x7f59f005bd20 msgr2=0x7f59f005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- 172.21.2.116:0/2441291938 shutdown_connections 2021-11-21T22:01:44.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.646+0000 7f59f521d700 1 -- 172.21.2.116:0/2441291938 wait complete. 2021-11-21T22:01:44.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 Processor -- start 2021-11-21T22:01:44.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 -- start start 2021-11-21T22:01:44.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59d00037d0 0x7f59f0149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59f0065450 0x7f59f00b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59f00b5110 0x7f59f0065ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f01560e0 con 0x7f59f0065450 2021-11-21T22:01:44.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f01562d0 con 0x7f59f00b5110 2021-11-21T22:01:44.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0156570 con 0x7f59d00037d0 2021-11-21T22:01:44.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59f0065450 0x7f59f00b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59f0065450 0x7f59f00b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51448/0 (socket says 172.21.2.116:51448) 2021-11-21T22:01:44.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 -- 172.21.2.116:0/2304182394 learned_addr learned my addr 172.21.2.116:0/2304182394 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:44.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 -- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59d00037d0 msgr2=0x7f59f0149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59d00037d0 0x7f59f0149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 -- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59f00b5110 msgr2=0x7f59f0065ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59f00b5110 0x7f59f0065ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 -- 172.21.2.116:0/2304182394 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0156810 con 0x7f59f0065450 2021-11-21T22:01:44.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59ee59c700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59f0065450 0x7f59f00b4c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f59e000a000 tx=0x7f59e0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59e7fff700 1 -- 172.21.2.116:0/2304182394 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59e0016070 con 0x7f59f0065450 2021-11-21T22:01:44.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59e7fff700 1 -- 172.21.2.116:0/2304182394 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e0000e90 con 0x7f59f0065450 2021-11-21T22:01:44.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f0156ab0 con 0x7f59f0065450 2021-11-21T22:01:44.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59e7fff700 1 -- 172.21.2.116:0/2304182394 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59e0011a40 con 0x7f59f0065450 2021-11-21T22:01:44.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f00b58d0 con 0x7f59f0065450 2021-11-21T22:01:44.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f00b20b0 con 0x7f59f0065450 2021-11-21T22:01:44.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59e7fff700 1 -- 172.21.2.116:0/2304182394 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f59e0011ba0 con 0x7f59f0065450 2021-11-21T22:01:44.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59e7fff700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f59d40536a0 0x7f59d4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59e7fff700 1 -- 172.21.2.116:0/2304182394 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f59e006b600 con 0x7f59f0065450 2021-11-21T22:01:44.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.650+0000 7f59eed9d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f59d40536a0 0x7f59d4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.654+0000 7f59eed9d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f59d40536a0 0x7f59d4055b60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f59d8001370 tx=0x7f59d8013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.654+0000 7f59e7fff700 1 -- 172.21.2.116:0/2304182394 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59e0042760 con 0x7f59f0065450 2021-11-21T22:01:44.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.762+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f59f00573d0 con 0x7f59f0065450 2021-11-21T22:01:44.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59e7fff700 1 -- 172.21.2.116:0/2304182394 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f59e0045d80 con 0x7f59f0065450 2021-11-21T22:01:44.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f59d40536a0 msgr2=0x7f59d4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f59d40536a0 0x7f59d4055b60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f59d8001370 tx=0x7f59d8013000).stop 2021-11-21T22:01:44.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59f0065450 msgr2=0x7f59f00b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59f0065450 0x7f59f00b4c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f59e000a000 tx=0x7f59e0008040).stop 2021-11-21T22:01:44.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 shutdown_connections 2021-11-21T22:01:44.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f59f00b5110 0x7f59f0065ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f59d40536a0 0x7f59d4055b60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f59f0065450 0x7f59f00b4c20 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 --2- 172.21.2.116:0/2304182394 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f59d00037d0 0x7f59f0149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 >> 172.21.2.116:0/2304182394 conn(0x7f59f005bd20 msgr2=0x7f59f00af910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 shutdown_connections 2021-11-21T22:01:44.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.770+0000 7f59f521d700 1 -- 172.21.2.116:0/2304182394 wait complete. 2021-11-21T22:01:44.780 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:01:44.782 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:01:44.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 Processor -- start 2021-11-21T22:01:44.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- start start 2021-11-21T22:01:44.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 0x7f45d80b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45d80aed90 0x7f45d813c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d80b00d0 con 0x7f45d80af360 2021-11-21T22:01:44.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d80adba0 con 0x7f45d80aed90 2021-11-21T22:01:44.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80af7a0 con 0x7f45d80b24a0 2021-11-21T22:01:44.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45d80aed90 0x7f45d813c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45d80aed90 0x7f45d813c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51456/0 (socket says 172.21.2.116:51456) 2021-11-21T22:01:44.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 -- 172.21.2.116:0/3553416564 learned_addr learned my addr 172.21.2.116:0/3553416564 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:44.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 -- 172.21.2.116:0/3553416564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 msgr2=0x7f45d80b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:44.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 --2- 172.21.2.116:0/3553416564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 0x7f45d80b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 -- 172.21.2.116:0/3553416564 >> v1:172.21.2.117:6789/0 conn(0x7f45d80af360 legacy=0x7f45d80ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:44.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 -- 172.21.2.116:0/3553416564 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80b2880 con 0x7f45d80aed90 2021-11-21T22:01:44.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d7fff700 1 --2- 172.21.2.116:0/3553416564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80aed90 0x7f45d813c210 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f45d80630d0 tx=0x7f45d000b530).ready entity=mon.0 client_cookie=1f7287ca329841f2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d67fc700 1 -- 172.21.2.116:0/3553416564 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45d00155e0 con 0x7f45d80aed90 2021-11-21T22:01:44.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d67fc700 1 -- 172.21.2.116:0/3553416564 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0015740 con 0x7f45d80aed90 2021-11-21T22:01:44.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45d67fc700 1 -- 172.21.2.116:0/3553416564 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45d0015aa0 con 0x7f45d80aed90 2021-11-21T22:01:44.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- 172.21.2.116:0/3553416564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80aed90 msgr2=0x7f45d813c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 --2- 172.21.2.116:0/3553416564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80aed90 0x7f45d813c210 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f45d80630d0 tx=0x7f45d000b530).stop 2021-11-21T22:01:44.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- 172.21.2.116:0/3553416564 shutdown_connections 2021-11-21T22:01:44.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 --2- 172.21.2.116:0/3553416564 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80aed90 0x7f45d813c210 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 --2- 172.21.2.116:0/3553416564 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 0x7f45d80b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- 172.21.2.116:0/3553416564 >> 172.21.2.116:0/3553416564 conn(0x7f45d805fd00 msgr2=0x7f45d80600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- 172.21.2.116:0/3553416564 shutdown_connections 2021-11-21T22:01:44.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.834+0000 7f45de1f9700 1 -- 172.21.2.116:0/3553416564 wait complete. 2021-11-21T22:01:44.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 Processor -- start 2021-11-21T22:01:44.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 -- start start 2021-11-21T22:01:44.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45d80aed90 0x7f45d80663a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80af360 0x7f45d8141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 0x7f45d813a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d81562b0 con 0x7f45d80af360 2021-11-21T22:01:44.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d81145e0 con 0x7f45d80aed90 2021-11-21T22:01:44.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8114880 con 0x7f45d80b24a0 2021-11-21T22:01:44.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80af360 0x7f45d8141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80af360 0x7f45d8141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51462/0 (socket says 172.21.2.116:51462) 2021-11-21T22:01:44.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 -- 172.21.2.116:0/2386687846 learned_addr learned my addr 172.21.2.116:0/2386687846 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:44.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 -- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 msgr2=0x7f45d813a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 0x7f45d813a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 -- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45d80aed90 msgr2=0x7f45d80663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45d80aed90 0x7f45d80663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 -- 172.21.2.116:0/2386687846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d000b7d0 con 0x7f45d80af360 2021-11-21T22:01:44.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d6ffd700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80af360 0x7f45d8141d40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f45c8002f90 tx=0x7f45c8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d4ff9700 1 -- 172.21.2.116:0/2386687846 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c8013460 con 0x7f45d80af360 2021-11-21T22:01:44.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d4ff9700 1 -- 172.21.2.116:0/2386687846 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80135c0 con 0x7f45d80af360 2021-11-21T22:01:44.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d4ff9700 1 -- 172.21.2.116:0/2386687846 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c8013920 con 0x7f45d80af360 2021-11-21T22:01:44.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d8114b20 con 0x7f45d80af360 2021-11-21T22:01:44.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d8155380 con 0x7f45d80af360 2021-11-21T22:01:44.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d80573d0 con 0x7f45d80af360 2021-11-21T22:01:44.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d4ff9700 1 -- 172.21.2.116:0/2386687846 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f45c801a060 con 0x7f45d80af360 2021-11-21T22:01:44.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d4ff9700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45bc0536f0 0x7f45bc055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:44.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d4ff9700 1 -- 172.21.2.116:0/2386687846 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f45c806b400 con 0x7f45d80af360 2021-11-21T22:01:44.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d77fe700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45bc0536f0 0x7f45bc055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:44.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.838+0000 7f45d77fe700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45bc0536f0 0x7f45bc055bb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f45c0002f70 tx=0x7f45c00069f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:44.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.842+0000 7f45d4ff9700 1 -- 172.21.2.116:0/2386687846 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45c8042450 con 0x7f45d80af360 2021-11-21T22:01:44.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.950+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f45d805f2d0 con 0x7f45d80af360 2021-11-21T22:01:44.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45d4ff9700 1 -- 172.21.2.116:0/2386687846 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7f45c8045a70 con 0x7f45d80af360 2021-11-21T22:01:44.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45bc0536f0 msgr2=0x7f45bc055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45bc0536f0 0x7f45bc055bb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f45c0002f70 tx=0x7f45c00069f0).stop 2021-11-21T22:01:44.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80af360 msgr2=0x7f45d8141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:44.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80af360 0x7f45d8141d40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f45c8002f90 tx=0x7f45c8006df0).stop 2021-11-21T22:01:44.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 shutdown_connections 2021-11-21T22:01:44.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45d80b24a0 0x7f45d813a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45bc0536f0 0x7f45bc055bb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45d80af360 0x7f45d8141d40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:44.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 --2- 172.21.2.116:0/2386687846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45d80aed90 0x7f45d80663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:44.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 >> 172.21.2.116:0/2386687846 conn(0x7f45d805fd00 msgr2=0x7f45d813b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:44.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 shutdown_connections 2021-11-21T22:01:44.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:44.958+0000 7f45de1f9700 1 -- 172.21.2.116:0/2386687846 wait complete. 2021-11-21T22:01:44.967 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:01:44.969 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:45.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 Processor -- start 2021-11-21T22:01:45.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- start start 2021-11-21T22:01:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ffb300a6aa0 0x7ffb3014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ffb300a86b0 0x7ffb301470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb301475d0 0x7ffb301443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb300a8bc0 con 0x7ffb300a86b0 2021-11-21T22:01:45.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb300a6e80 con 0x7ffb300a6aa0 2021-11-21T22:01:45.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3014a7b0 con 0x7ffb301475d0 2021-11-21T22:01:45.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ffb300a86b0 0x7ffb301470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ffb300a86b0 0x7ffb301470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51470/0 (socket says 172.21.2.116:51470) 2021-11-21T22:01:45.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 -- 172.21.2.116:0/703593569 learned_addr learned my addr 172.21.2.116:0/703593569 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:45.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 -- 172.21.2.116:0/703593569 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb301475d0 msgr2=0x7ffb301443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 --2- 172.21.2.116:0/703593569 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb301475d0 0x7ffb301443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 -- 172.21.2.116:0/703593569 >> v2:172.21.2.117:3300/0 conn(0x7ffb300a6aa0 msgr2=0x7ffb3014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 --2- 172.21.2.116:0/703593569 >> v2:172.21.2.117:3300/0 conn(0x7ffb300a6aa0 0x7ffb3014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 -- 172.21.2.116:0/703593569 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3014aa10 con 0x7ffb300a86b0 2021-11-21T22:01:45.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2e7fc700 1 --2- 172.21.2.116:0/703593569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb300a86b0 0x7ffb301470e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7ffb20002f70 tx=0x7ffb20006eb0).ready entity=mon.0 client_cookie=e18116343db0c07a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2dffb700 1 -- 172.21.2.116:0/703593569 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2000d570 con 0x7ffb300a86b0 2021-11-21T22:01:45.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2dffb700 1 -- 172.21.2.116:0/703593569 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20000e90 con 0x7ffb300a86b0 2021-11-21T22:01:45.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb2dffb700 1 -- 172.21.2.116:0/703593569 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2000d900 con 0x7ffb300a86b0 2021-11-21T22:01:45.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- 172.21.2.116:0/703593569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb300a86b0 msgr2=0x7ffb301470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 --2- 172.21.2.116:0/703593569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb300a86b0 0x7ffb301470e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7ffb20002f70 tx=0x7ffb20006eb0).stop 2021-11-21T22:01:45.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- 172.21.2.116:0/703593569 shutdown_connections 2021-11-21T22:01:45.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 --2- 172.21.2.116:0/703593569 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb301475d0 0x7ffb301443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 --2- 172.21.2.116:0/703593569 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb300a86b0 0x7ffb301470e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 --2- 172.21.2.116:0/703593569 >> v2:172.21.2.117:3300/0 conn(0x7ffb300a6aa0 0x7ffb3014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- 172.21.2.116:0/703593569 >> 172.21.2.116:0/703593569 conn(0x7ffb3005bd20 msgr2=0x7ffb3005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- 172.21.2.116:0/703593569 shutdown_connections 2021-11-21T22:01:45.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- 172.21.2.116:0/703593569 wait complete. 2021-11-21T22:01:45.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 Processor -- start 2021-11-21T22:01:45.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.018+0000 7ffb356d6700 1 -- start start 2021-11-21T22:01:45.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb300a6aa0 0x7ffb300a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb300a86b0 0x7ffb30159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb301475d0 0x7ffb30159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30157190 con 0x7ffb301475d0 2021-11-21T22:01:45.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30064610 con 0x7ffb300a86b0 2021-11-21T22:01:45.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300648b0 con 0x7ffb300a6aa0 2021-11-21T22:01:45.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb301475d0 0x7ffb30159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb301475d0 0x7ffb30159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51478/0 (socket says 172.21.2.116:51478) 2021-11-21T22:01:45.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 -- 172.21.2.116:0/1857501234 learned_addr learned my addr 172.21.2.116:0/1857501234 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:45.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 -- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb300a6aa0 msgr2=0x7ffb300a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb300a6aa0 0x7ffb300a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 -- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb300a86b0 msgr2=0x7ffb30159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb300a86b0 0x7ffb30159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 -- 172.21.2.116:0/1857501234 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2000b1e0 con 0x7ffb301475d0 2021-11-21T22:01:45.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2f7fe700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb301475d0 0x7ffb30159e50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7ffb280032f0 tx=0x7ffb280091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb1ffff700 1 -- 172.21.2.116:0/1857501234 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2800f760 con 0x7ffb301475d0 2021-11-21T22:01:45.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb1ffff700 1 -- 172.21.2.116:0/1857501234 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2800f8c0 con 0x7ffb301475d0 2021-11-21T22:01:45.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb1ffff700 1 -- 172.21.2.116:0/1857501234 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2800fc20 con 0x7ffb301475d0 2021-11-21T22:01:45.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb30156d90 con 0x7ffb301475d0 2021-11-21T22:01:45.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb300a9060 con 0x7ffb301475d0 2021-11-21T22:01:45.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb30064a70 con 0x7ffb301475d0 2021-11-21T22:01:45.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb1ffff700 1 -- 172.21.2.116:0/1857501234 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ffb2801a460 con 0x7ffb301475d0 2021-11-21T22:01:45.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb1ffff700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb0c053580 0x7ffb0c055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb1ffff700 1 -- 172.21.2.116:0/1857501234 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ffb2806bc40 con 0x7ffb301475d0 2021-11-21T22:01:45.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2effd700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb0c053580 0x7ffb0c055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.022+0000 7ffb2effd700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb0c053580 0x7ffb0c055a40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7ffb24002f70 tx=0x7ffb240069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.026+0000 7ffb1ffff700 1 -- 172.21.2.116:0/1857501234 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb28042ca0 con 0x7ffb301475d0 2021-11-21T22:01:45.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.134+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb300a9060 con 0x7ffb301475d0 2021-11-21T22:01:45.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.134+0000 7ffb1ffff700 1 -- 172.21.2.116:0/1857501234 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+2490 (secure 0 0 0) 0x7ffb300a9060 con 0x7ffb301475d0 2021-11-21T22:01:45.140 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:45.141 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":49,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:43.226243+0000","modified":"2021-11-21T22:01:44.230974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1808795000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6833","nonce":1808795000},{"type":"v1","addr":"172.21.2.116:6835","nonce":1808795000}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-11-21T22:01:45.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb0c053580 msgr2=0x7ffb0c055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb0c053580 0x7ffb0c055a40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7ffb24002f70 tx=0x7ffb240069e0).stop 2021-11-21T22:01:45.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb301475d0 msgr2=0x7ffb30159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb301475d0 0x7ffb30159e50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7ffb280032f0 tx=0x7ffb280091d0).stop 2021-11-21T22:01:45.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 shutdown_connections 2021-11-21T22:01:45.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb0c053580 0x7ffb0c055a40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb301475d0 0x7ffb30159e50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb300a86b0 0x7ffb30159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 --2- 172.21.2.116:0/1857501234 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb300a6aa0 0x7ffb300a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 >> 172.21.2.116:0/1857501234 conn(0x7ffb3005bd20 msgr2=0x7ffb30148440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 shutdown_connections 2021-11-21T22:01:45.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.138+0000 7ffb356d6700 1 -- 172.21.2.116:0/1857501234 wait complete. 2021-11-21T22:01:45.145 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 49 2021-11-21T22:01:45.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:43.226243+0000', 'modified': '2021-11-21T22:01:44.230974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7356}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7356': {'gid': 7356, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/1808795000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6833', 'nonce': 1808795000}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1808795000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:45.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:45.451 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:45.516 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7356,"want_state":"up:active","state":"up:active","rank_uptime":2.2824059719999998,"mdsmap_epoch":49,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":6.4868898679999996} 2021-11-21T22:01:45.517 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7356, "mdsmap_epoch": 49, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 2.282405972, "state": "up:active", "uptime": 6.486889868, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:45.517 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:45.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.562+0000 7f1680b8b700 1 Processor -- start 2021-11-21T22:01:45.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.562+0000 7f1680b8b700 1 -- start start 2021-11-21T22:01:45.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f167c0abcb0 0x7f167c0af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167c0ac090 con 0x7f167c13d470 2021-11-21T22:01:45.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167c062590 con 0x7f167c0adb90 2021-11-21T22:01:45.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167c0b1590 con 0x7f167c0abcb0 2021-11-21T22:01:45.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f167c13d470 0x7f167c13fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35206/0 (socket says 172.21.2.116:35206) 2021-11-21T22:01:45.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 -- 172.21.2.116:0/2688109138 learned_addr learned my addr 172.21.2.116:0/2688109138 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:45.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055347867 0 0) 0x7f167c0ac090 con 0x7f167c13d470 2021-11-21T22:01:45.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f165c003260 con 0x7f167c13d470 2021-11-21T22:01:45.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 968913276 0 0) 0x7f165c003260 con 0x7f167c13d470 2021-11-21T22:01:45.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 >> v2:172.21.2.117:3300/0 conn(0x7f167c0abcb0 msgr2=0x7f167c0af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 --2- 172.21.2.116:0/2688109138 >> v2:172.21.2.117:3300/0 conn(0x7f167c0abcb0 0x7f167c0af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 >> v1:172.21.2.117:6789/0 conn(0x7f167c0adb90 legacy=0x7f167c13cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c062830 con 0x7f167c13d470 2021-11-21T22:01:45.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f1674001a00 con 0x7f167c13d470 2021-11-21T22:01:45.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 >> v1:172.21.2.116:6789/0 conn(0x7f167c13d470 legacy=0x7f167c13fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f165c003820 0x7f165c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f165c004500 0x7f165c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f165c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c00bf10 con 0x7f165c005050 2021-11-21T22:01:45.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165c00c080 con 0x7f165c003820 2021-11-21T22:01:45.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165c00c240 con 0x7f165c004500 2021-11-21T22:01:45.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f165c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 -- 172.21.2.116:0/2688109138 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f165c004500 msgr2=0x7f165c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f165c004500 0x7f165c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 -- 172.21.2.116:0/2688109138 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f165c003820 msgr2=0x7f165c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f165c003820 0x7f165c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 -- 172.21.2.116:0/2688109138 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c003260 con 0x7f165c005050 2021-11-21T22:01:45.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 -- 172.21.2.116:0/2688109138 reap_dead start 2021-11-21T22:01:45.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167ad9d700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f165c00b750 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f167c061a50 tx=0x7f167c13fa30).ready entity=mon.0 client_cookie=23341b6f571a627e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167400cc40 con 0x7f165c005050 2021-11-21T22:01:45.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167400cda0 con 0x7f165c005050 2021-11-21T22:01:45.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f167959a700 1 -- 172.21.2.116:0/2688109138 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1674004b50 con 0x7f165c005050 2021-11-21T22:01:45.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- 172.21.2.116:0/2688109138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 msgr2=0x7f165c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f165c00b750 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f167c061a50 tx=0x7f167c13fa30).stop 2021-11-21T22:01:45.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- 172.21.2.116:0/2688109138 shutdown_connections 2021-11-21T22:01:45.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 --2- 172.21.2.116:0/2688109138 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f165c00b750 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- 172.21.2.116:0/2688109138 >> 172.21.2.116:0/2688109138 conn(0x7f167c05bd20 msgr2=0x7f167c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- 172.21.2.116:0/2688109138 shutdown_connections 2021-11-21T22:01:45.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- 172.21.2.116:0/2688109138 wait complete. 2021-11-21T22:01:45.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 Processor -- start 2021-11-21T22:01:45.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.566+0000 7f1680b8b700 1 -- start start 2021-11-21T22:01:45.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f167c13c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f167c135040 0x7f167c130380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f167c130870 0x7f167c14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c136100 con 0x7f165c005050 2021-11-21T22:01:45.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0a9810 con 0x7f167c135040 2021-11-21T22:01:45.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1354a0 con 0x7f167c130870 2021-11-21T22:01:45.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f167c13c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f167c13c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51492/0 (socket says 172.21.2.116:51492) 2021-11-21T22:01:45.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 -- 172.21.2.116:0/101118713 learned_addr learned my addr 172.21.2.116:0/101118713 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:45.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 -- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f167c130870 msgr2=0x7f167c14fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f167c130870 0x7f167c14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 -- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f167c135040 msgr2=0x7f167c130380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f167c135040 0x7f167c130380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 -- 172.21.2.116:0/101118713 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c0a9a40 con 0x7f165c005050 2021-11-21T22:01:45.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f167a59c700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f167c13c240 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f166c002f70 tx=0x7f166c006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f16737fe700 1 -- 172.21.2.116:0/101118713 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166c013460 con 0x7f165c005050 2021-11-21T22:01:45.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f16737fe700 1 -- 172.21.2.116:0/101118713 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c0135c0 con 0x7f165c005050 2021-11-21T22:01:45.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c0a9ce0 con 0x7f165c005050 2021-11-21T22:01:45.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f16737fe700 1 -- 172.21.2.116:0/101118713 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166c013920 con 0x7f165c005050 2021-11-21T22:01:45.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c0aa220 con 0x7f165c005050 2021-11-21T22:01:45.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c0573d0 con 0x7f165c005050 2021-11-21T22:01:45.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f16737fe700 1 -- 172.21.2.116:0/101118713 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f166c01a060 con 0x7f165c005050 2021-11-21T22:01:45.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f16737fe700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1660053640 0x7f1660055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f16737fe700 1 -- 172.21.2.116:0/101118713 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f166c06b370 con 0x7f165c005050 2021-11-21T22:01:45.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.570+0000 7f1679d9b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1660053640 0x7f1660055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.574+0000 7f1679d9b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1660053640 0x7f1660055b00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f1664000f70 tx=0x7f16640069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.574+0000 7f16737fe700 1 -- 172.21.2.116:0/101118713 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f166c0423c0 con 0x7f165c005050 2021-11-21T22:01:45.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.654+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f167c0b3a60 con 0x7f165c005050 2021-11-21T22:01:45.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f16737fe700 1 -- 172.21.2.116:0/101118713 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f166c018020 con 0x7f165c005050 2021-11-21T22:01:45.661 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:45.661 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":111,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:44.231112+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:41.057032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:42.188874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:45.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1660053640 msgr2=0x7f1660055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1660053640 0x7f1660055b00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f1664000f70 tx=0x7f16640069b0).stop 2021-11-21T22:01:45.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 msgr2=0x7f167c13c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f167c13c240 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f166c002f70 tx=0x7f166c006e40).stop 2021-11-21T22:01:45.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 shutdown_connections 2021-11-21T22:01:45.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f167c130870 0x7f167c14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f167c135040 0x7f167c130380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1660053640 0x7f1660055b00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 --2- 172.21.2.116:0/101118713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f165c005050 0x7f167c13c240 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 >> 172.21.2.116:0/101118713 conn(0x7f167c05bd20 msgr2=0x7f167c0b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 shutdown_connections 2021-11-21T22:01:45.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.658+0000 7f1680b8b700 1 -- 172.21.2.116:0/101118713 wait complete. 2021-11-21T22:01:45.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:01:41.057032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:01:42.188874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:01:45.669 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:45.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 Processor -- start 2021-11-21T22:01:45.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 -- start start 2021-11-21T22:01:45.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe3f80abce0 0x7fe3f81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe3f80ac600 0x7fe3f80aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f8147e80 con 0x7fe3f8144810 2021-11-21T22:01:45.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80b1490 con 0x7fe3f80abce0 2021-11-21T22:01:45.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80b1fd0 con 0x7fe3f80ac600 2021-11-21T22:01:45.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe3f80abce0 0x7fe3f81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe3f80abce0 0x7fe3f81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51502/0 (socket says 172.21.2.116:51502) 2021-11-21T22:01:45.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 -- 172.21.2.116:0/1007482624 learned_addr learned my addr 172.21.2.116:0/1007482624 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:45.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 -- 172.21.2.116:0/1007482624 >> v2:172.21.2.117:3300/0 conn(0x7fe3f80ac600 msgr2=0x7fe3f80aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:45.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 --2- 172.21.2.116:0/1007482624 >> v2:172.21.2.117:3300/0 conn(0x7fe3f80ac600 0x7fe3f80aca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 -- 172.21.2.116:0/1007482624 >> v1:172.21.2.116:6789/0 conn(0x7fe3f8144810 legacy=0x7fe3f80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 -- 172.21.2.116:0/1007482624 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80af5a0 con 0x7fe3f80abce0 2021-11-21T22:01:45.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f7fff700 1 --2- 172.21.2.116:0/1007482624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80abce0 0x7fe3f81478c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fe3e0002f70 tx=0x7fe3e0006eb0).ready entity=mon.0 client_cookie=30457d01830e9bbf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f77fe700 1 -- 172.21.2.116:0/1007482624 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e000d610 con 0x7fe3f80abce0 2021-11-21T22:01:45.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f77fe700 1 -- 172.21.2.116:0/1007482624 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e0000e90 con 0x7fe3f80abce0 2021-11-21T22:01:45.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3f77fe700 1 -- 172.21.2.116:0/1007482624 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e000d9a0 con 0x7fe3f80abce0 2021-11-21T22:01:45.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 -- 172.21.2.116:0/1007482624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80abce0 msgr2=0x7fe3f81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 --2- 172.21.2.116:0/1007482624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80abce0 0x7fe3f81478c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fe3e0002f70 tx=0x7fe3e0006eb0).stop 2021-11-21T22:01:45.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.718+0000 7fe3fef65700 1 -- 172.21.2.116:0/1007482624 shutdown_connections 2021-11-21T22:01:45.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 --2- 172.21.2.116:0/1007482624 >> v2:172.21.2.117:3300/0 conn(0x7fe3f80ac600 0x7fe3f80aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 --2- 172.21.2.116:0/1007482624 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80abce0 0x7fe3f81478c0 secure :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0x7fe3e0002f70 tx=0x7fe3e0006eb0).stop 2021-11-21T22:01:45.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- 172.21.2.116:0/1007482624 >> 172.21.2.116:0/1007482624 conn(0x7fe3f805bd20 msgr2=0x7fe3f805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- 172.21.2.116:0/1007482624 shutdown_connections 2021-11-21T22:01:45.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- 172.21.2.116:0/1007482624 wait complete. 2021-11-21T22:01:45.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 Processor -- start 2021-11-21T22:01:45.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- start start 2021-11-21T22:01:45.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80ac600 0x7fe3f814c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8144810 0x7fe3f813cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe3f814ce00 0x7fe3f813d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8156860 con 0x7fe3f80ac600 2021-11-21T22:01:45.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f81577d0 con 0x7fe3f8144810 2021-11-21T22:01:45.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8141e30 con 0x7fe3f814ce00 2021-11-21T22:01:45.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80ac600 0x7fe3f814c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80ac600 0x7fe3f814c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51506/0 (socket says 172.21.2.116:51506) 2021-11-21T22:01:45.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 -- 172.21.2.116:0/959069525 learned_addr learned my addr 172.21.2.116:0/959069525 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:45.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 -- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe3f814ce00 msgr2=0x7fe3f813d420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:45.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe3f814ce00 0x7fe3f813d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 -- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8144810 msgr2=0x7fe3f813cf30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:45.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8144810 0x7fe3f813cf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 -- 172.21.2.116:0/959069525 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e000b1e0 con 0x7fe3f80ac600 2021-11-21T22:01:45.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fcd01700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80ac600 0x7fe3f814c910 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fe3f8062500 tx=0x7fe3e8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3f5ffb700 1 -- 172.21.2.116:0/959069525 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e800ce30 con 0x7fe3f80ac600 2021-11-21T22:01:45.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3f5ffb700 1 -- 172.21.2.116:0/959069525 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e800f470 con 0x7fe3f80ac600 2021-11-21T22:01:45.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3f5ffb700 1 -- 172.21.2.116:0/959069525 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e800f7d0 con 0x7fe3f80ac600 2021-11-21T22:01:45.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f80b4730 con 0x7fe3f80ac600 2021-11-21T22:01:45.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8142230 con 0x7fe3f80ac600 2021-11-21T22:01:45.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f80b48c0 con 0x7fe3f80ac600 2021-11-21T22:01:45.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3f5ffb700 1 -- 172.21.2.116:0/959069525 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe3e8019030 con 0x7fe3f80ac600 2021-11-21T22:01:45.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3f5ffb700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3dc0535b0 0x7fe3dc055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3f5ffb700 1 -- 172.21.2.116:0/959069525 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe3e806ad50 con 0x7fe3f80ac600 2021-11-21T22:01:45.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.722+0000 7fe3f7fff700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3dc0535b0 0x7fe3dc055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.726+0000 7fe3f7fff700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3dc0535b0 0x7fe3dc055a70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fe3e000a000 tx=0x7fe3e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.726+0000 7fe3f5ffb700 1 -- 172.21.2.116:0/959069525 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe3e8041eb0 con 0x7fe3f80ac600 2021-11-21T22:01:45.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.806+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe3f80af060 con 0x7fe3f80ac600 2021-11-21T22:01:45.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3f5ffb700 1 -- 172.21.2.116:0/959069525 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v111) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe3e80454d0 con 0x7fe3f80ac600 2021-11-21T22:01:45.813 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:45.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3dc0535b0 msgr2=0x7fe3dc055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3dc0535b0 0x7fe3dc055a70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fe3e000a000 tx=0x7fe3e0015000).stop 2021-11-21T22:01:45.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80ac600 msgr2=0x7fe3f814c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80ac600 0x7fe3f814c910 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fe3f8062500 tx=0x7fe3e8007c50).stop 2021-11-21T22:01:45.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 shutdown_connections 2021-11-21T22:01:45.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe3f814ce00 0x7fe3f813d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe3f8144810 0x7fe3f813cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3dc0535b0 0x7fe3dc055a70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 --2- 172.21.2.116:0/959069525 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe3f80ac600 0x7fe3f814c910 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 >> 172.21.2.116:0/959069525 conn(0x7fe3f805bd20 msgr2=0x7fe3f8140e50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 shutdown_connections 2021-11-21T22:01:45.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.810+0000 7fe3fef65700 1 -- 172.21.2.116:0/959069525 wait complete. 2021-11-21T22:01:45.820 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:45.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 Processor -- start 2021-11-21T22:01:45.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- start start 2021-11-21T22:01:45.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcee80abce0 0x7fcee81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee8147e80 con 0x7fcee8144810 2021-11-21T22:01:45.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee80af5a0 con 0x7fcee80ac600 2021-11-21T22:01:45.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80b1490 con 0x7fcee80abce0 2021-11-21T22:01:45.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee659c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fcee8144810 0x7fcee80b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35238/0 (socket says 172.21.2.116:35238) 2021-11-21T22:01:45.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee659c700 1 -- 172.21.2.116:0/3047387677 learned_addr learned my addr 172.21.2.116:0/3047387677 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:45.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564332895 0 0) 0x7fcee8147e80 con 0x7fcee8144810 2021-11-21T22:01:45.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec8003260 con 0x7fcee8144810 2021-11-21T22:01:45.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 259522816 0 0) 0x7fcec8003260 con 0x7fcee8144810 2021-11-21T22:01:45.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 >> v2:172.21.2.117:3300/0 conn(0x7fcee80abce0 msgr2=0x7fcee81478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 --2- 172.21.2.116:0/3047387677 >> v2:172.21.2.117:3300/0 conn(0x7fcee80abce0 0x7fcee81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 >> v1:172.21.2.117:6789/0 conn(0x7fcee80ac600 legacy=0x7fcee80aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee80b1fd0 con 0x7fcee8144810 2021-11-21T22:01:45.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fced8001660 con 0x7fcee8144810 2021-11-21T22:01:45.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 >> v1:172.21.2.116:6789/0 conn(0x7fcee8144810 legacy=0x7fcee80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec80037d0 0x7fcec8003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec8004500 0x7fcec8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec8005070 0x7fcec800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80b1490 con 0x7fcec8004500 2021-11-21T22:01:45.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec800bef0 con 0x7fcec8005070 2021-11-21T22:01:45.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec800c0b0 con 0x7fcec80037d0 2021-11-21T22:01:45.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee5d9b700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec8004500 0x7fcec8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee659c700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec80037d0 0x7fcec8003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee5d9b700 1 -- 172.21.2.116:0/3047387677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec80037d0 msgr2=0x7fcec8003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee5d9b700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec80037d0 0x7fcec8003bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee5d9b700 1 -- 172.21.2.116:0/3047387677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec8005070 msgr2=0x7fcec800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:45.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee5d9b700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec8005070 0x7fcec800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee5d9b700 1 -- 172.21.2.116:0/3047387677 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee80b1fd0 con 0x7fcec8004500 2021-11-21T22:01:45.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee6d9d700 1 -- 172.21.2.116:0/3047387677 reap_dead start 2021-11-21T22:01:45.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee5d9b700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec8004500 0x7fcec8004980 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7fced0002f70 tx=0x7fced0006ca0).ready entity=mon.0 client_cookie=ba7d7122192a27f5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced0012410 con 0x7fcec8004500 2021-11-21T22:01:45.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced0012570 con 0x7fcec8004500 2021-11-21T22:01:45.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fcee559a700 1 -- 172.21.2.116:0/3047387677 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced00128d0 con 0x7fcec8004500 2021-11-21T22:01:45.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- 172.21.2.116:0/3047387677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec8004500 msgr2=0x7fcec8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec8004500 0x7fcec8004980 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7fced0002f70 tx=0x7fced0006ca0).stop 2021-11-21T22:01:45.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- 172.21.2.116:0/3047387677 shutdown_connections 2021-11-21T22:01:45.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 --2- 172.21.2.116:0/3047387677 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcec8004500 0x7fcec8004980 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- 172.21.2.116:0/3047387677 >> 172.21.2.116:0/3047387677 conn(0x7fcee805bd20 msgr2=0x7fcee805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- 172.21.2.116:0/3047387677 shutdown_connections 2021-11-21T22:01:45.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.870+0000 7fceecebd700 1 -- 172.21.2.116:0/3047387677 wait complete. 2021-11-21T22:01:45.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 Processor -- start 2021-11-21T22:01:45.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 -- start start 2021-11-21T22:01:45.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec8004500 0x7fcee80633b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcee8066bc0 0x7fcee8067040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcee8159a60 0x7fcee8159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8062f80 con 0x7fcee8066bc0 2021-11-21T22:01:45.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8154140 con 0x7fcec8004500 2021-11-21T22:01:45.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8157090 con 0x7fcee8159a60 2021-11-21T22:01:45.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcee8066bc0 0x7fcee8067040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcee8066bc0 0x7fcee8067040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51528/0 (socket says 172.21.2.116:51528) 2021-11-21T22:01:45.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 -- 172.21.2.116:0/1095941575 learned_addr learned my addr 172.21.2.116:0/1095941575 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:45.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 -- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcee8159a60 msgr2=0x7fcee8159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcee8159a60 0x7fcee8159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 -- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec8004500 msgr2=0x7fcee80633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec8004500 0x7fcee80633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 -- 172.21.2.116:0/1095941575 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee8157330 con 0x7fcee8066bc0 2021-11-21T22:01:45.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee5d9b700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcee8066bc0 0x7fcee8067040 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fced0002f70 tx=0x7fced0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcedf7fe700 1 -- 172.21.2.116:0/1095941575 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced0011880 con 0x7fcee8066bc0 2021-11-21T22:01:45.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcedf7fe700 1 -- 172.21.2.116:0/1095941575 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced00119e0 con 0x7fcee8066bc0 2021-11-21T22:01:45.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcedf7fe700 1 -- 172.21.2.116:0/1095941575 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced0011d40 con 0x7fcee8066bc0 2021-11-21T22:01:45.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee81468c0 con 0x7fcee8066bc0 2021-11-21T22:01:45.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee80638a0 con 0x7fcee8066bc0 2021-11-21T22:01:45.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee80af060 con 0x7fcee8066bc0 2021-11-21T22:01:45.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcedf7fe700 1 -- 172.21.2.116:0/1095941575 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fced001a040 con 0x7fcee8066bc0 2021-11-21T22:01:45.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcedf7fe700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcecc0535b0 0x7fcecc055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:45.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcedf7fe700 1 -- 172.21.2.116:0/1095941575 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fced006c890 con 0x7fcee8066bc0 2021-11-21T22:01:45.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.874+0000 7fcee659c700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcecc0535b0 0x7fcecc055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:45.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.878+0000 7fcee659c700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcecc0535b0 0x7fcecc055a70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fced8008000 tx=0x7fced8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:45.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.878+0000 7fcedf7fe700 1 -- 172.21.2.116:0/1095941575 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fced00439f0 con 0x7fcee8066bc0 2021-11-21T22:01:45.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcee80573d0 con 0x7fcee8066bc0 2021-11-21T22:01:45.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fcedf7fe700 1 -- 172.21.2.116:0/1095941575 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7fced0047010 con 0x7fcee8066bc0 2021-11-21T22:01:45.966 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:45.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcecc0535b0 msgr2=0x7fcecc055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcecc0535b0 0x7fcecc055a70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fced8008000 tx=0x7fced8009040).stop 2021-11-21T22:01:45.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcee8066bc0 msgr2=0x7fcee8067040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:45.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcee8066bc0 0x7fcee8067040 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fced0002f70 tx=0x7fced0015000).stop 2021-11-21T22:01:45.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 shutdown_connections 2021-11-21T22:01:45.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcee8159a60 0x7fcee8159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcecc0535b0 0x7fcecc055a70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcee8066bc0 0x7fcee8067040 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:45.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 --2- 172.21.2.116:0/1095941575 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec8004500 0x7fcee80633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:45.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 >> 172.21.2.116:0/1095941575 conn(0x7fcee805bd20 msgr2=0x7fcee8148510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:45.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 shutdown_connections 2021-11-21T22:01:45.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:45.962+0000 7fceecebd700 1 -- 172.21.2.116:0/1095941575 wait complete. 2021-11-21T22:01:45.973 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:01:46.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.022+0000 7f1259b8f700 1 Processor -- start 2021-11-21T22:01:46.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.022+0000 7f1259b8f700 1 -- start start 2021-11-21T22:01:46.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.022+0000 7f1259b8f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f12540b6770 0x7f12540b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.022+0000 7f1259b8f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540ad580 0x7f12540ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.022+0000 7f1259b8f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540b0070 con 0x7f12540b1270 2021-11-21T22:01:46.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.022+0000 7f1259b8f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540b05c0 con 0x7f12540b6770 2021-11-21T22:01:46.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.022+0000 7f1259b8f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540b2230 con 0x7f12540ad580 2021-11-21T22:01:46.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540ad580 0x7f12540ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12537fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f12540b6770 0x7f12540b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540ad580 0x7f12540ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:54880/0 (socket says 172.21.2.116:54880) 2021-11-21T22:01:46.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 -- 172.21.2.116:0/2904955180 learned_addr learned my addr 172.21.2.116:0/2904955180 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:46.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 -- 172.21.2.116:0/2904955180 >> v2:172.21.2.117:3300/0 conn(0x7f12540b6770 msgr2=0x7f12540b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:46.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 --2- 172.21.2.116:0/2904955180 >> v2:172.21.2.117:3300/0 conn(0x7f12540b6770 0x7f12540b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 -- 172.21.2.116:0/2904955180 >> v1:172.21.2.117:6789/0 conn(0x7f12540b1270 legacy=0x7f12541394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 -- 172.21.2.116:0/2904955180 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12540b24d0 con 0x7f12540ad580 2021-11-21T22:01:46.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12537fe700 1 --2- 172.21.2.116:0/2904955180 >> v2:172.21.2.117:3300/0 conn(0x7f12540b6770 0x7f12540b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:01:46.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1253fff700 1 --2- 172.21.2.116:0/2904955180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540ad580 0x7f12540ad9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f124c0032f0 tx=0x7f124c009290).ready entity=mon.2 client_cookie=94c2a30e99b5f1d3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12527fc700 1 -- 172.21.2.116:0/2904955180 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f124c00b8a0 con 0x7f12540ad580 2021-11-21T22:01:46.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12527fc700 1 -- 172.21.2.116:0/2904955180 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c00ba00 con 0x7f12540ad580 2021-11-21T22:01:46.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12527fc700 1 -- 172.21.2.116:0/2904955180 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f124c00bb70 con 0x7f12540ad580 2021-11-21T22:01:46.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- 172.21.2.116:0/2904955180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540ad580 msgr2=0x7f12540ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 --2- 172.21.2.116:0/2904955180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540ad580 0x7f12540ad9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f124c0032f0 tx=0x7f124c009290).stop 2021-11-21T22:01:46.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- 172.21.2.116:0/2904955180 shutdown_connections 2021-11-21T22:01:46.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 --2- 172.21.2.116:0/2904955180 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540ad580 0x7f12540ad9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 --2- 172.21.2.116:0/2904955180 >> v2:172.21.2.117:3300/0 conn(0x7f12540b6770 0x7f12540b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- 172.21.2.116:0/2904955180 >> 172.21.2.116:0/2904955180 conn(0x7f1254062040 msgr2=0x7f12540b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- 172.21.2.116:0/2904955180 shutdown_connections 2021-11-21T22:01:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- 172.21.2.116:0/2904955180 wait complete. 2021-11-21T22:01:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 Processor -- start 2021-11-21T22:01:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- start start 2021-11-21T22:01:46.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12540ad580 0x7f12540b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540b1270 0x7f125413c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12540b6770 0x7f1254150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125413c7e0 con 0x7f12540ad580 2021-11-21T22:01:46.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125413ca70 con 0x7f12540b6770 2021-11-21T22:01:46.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f1259b8f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540657b0 con 0x7f12540b1270 2021-11-21T22:01:46.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12537fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12540ad580 0x7f12540b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12537fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12540ad580 0x7f12540b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51540/0 (socket says 172.21.2.116:51540) 2021-11-21T22:01:46.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.026+0000 7f12537fe700 1 -- 172.21.2.116:0/359824271 learned_addr learned my addr 172.21.2.116:0/359824271 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:46.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f12537fe700 1 -- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540b1270 msgr2=0x7f125413c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f12537fe700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540b1270 0x7f125413c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f12537fe700 1 -- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12540b6770 msgr2=0x7f1254150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f12537fe700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12540b6770 0x7f1254150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f12537fe700 1 -- 172.21.2.116:0/359824271 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c00b180 con 0x7f12540ad580 2021-11-21T22:01:46.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f12537fe700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12540ad580 0x7f12540b3fa0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f123c00c000 tx=0x7f123c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1250ff9700 1 -- 172.21.2.116:0/359824271 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123c00c860 con 0x7f12540ad580 2021-11-21T22:01:46.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1250ff9700 1 -- 172.21.2.116:0/359824271 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c00c9c0 con 0x7f12540ad580 2021-11-21T22:01:46.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c009530 con 0x7f12540ad580 2021-11-21T22:01:46.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125413cd10 con 0x7f12540ad580 2021-11-21T22:01:46.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1250ff9700 1 -- 172.21.2.116:0/359824271 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123c00cd20 con 0x7f12540ad580 2021-11-21T22:01:46.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1254065c50 con 0x7f12540ad580 2021-11-21T22:01:46.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1250ff9700 1 -- 172.21.2.116:0/359824271 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f123c01c460 con 0x7f12540ad580 2021-11-21T22:01:46.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1250ff9700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12380535d0 0x7f1238055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1250ff9700 1 -- 172.21.2.116:0/359824271 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f123c06c010 con 0x7f12540ad580 2021-11-21T22:01:46.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1252ffd700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12380535d0 0x7f1238055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.030+0000 7f1252ffd700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12380535d0 0x7f1238055a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1254060900 tx=0x7f12440077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.034+0000 7f1250ff9700 1 -- 172.21.2.116:0/359824271 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1254065c50 con 0x7f12540ad580 2021-11-21T22:01:46.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.114+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1254065c50 con 0x7f12540ad580 2021-11-21T22:01:46.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.114+0000 7f1250ff9700 1 -- 172.21.2.116:0/359824271 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1254065c50 con 0x7f12540ad580 2021-11-21T22:01:46.120 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:46.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12380535d0 msgr2=0x7f1238055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12380535d0 0x7f1238055a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1254060900 tx=0x7f12440077d0).stop 2021-11-21T22:01:46.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12540ad580 msgr2=0x7f12540b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12540ad580 0x7f12540b3fa0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f123c00c000 tx=0x7f123c006df0).stop 2021-11-21T22:01:46.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 shutdown_connections 2021-11-21T22:01:46.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12540b6770 0x7f1254150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12540b1270 0x7f125413c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f12380535d0 0x7f1238055a90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 --2- 172.21.2.116:0/359824271 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12540ad580 0x7f12540b3fa0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 >> 172.21.2.116:0/359824271 conn(0x7f1254062040 msgr2=0x7f12540b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:46.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 shutdown_connections 2021-11-21T22:01:46.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.118+0000 7f1259b8f700 1 -- 172.21.2.116:0/359824271 wait complete. 2021-11-21T22:01:46.128 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:01:46.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f639244b700 1 Processor -- start 2021-11-21T22:01:46.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f639244b700 1 -- start start 2021-11-21T22:01:46.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f639244b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c05f730 0x7f638c141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f639244b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f638c142370 0x7f638c142800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f639244b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c0b06c0 con 0x7f638c0b02e0 2021-11-21T22:01:46.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f639244b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c0adf20 con 0x7f638c142370 2021-11-21T22:01:46.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f639244b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0b2470 con 0x7f638c05f730 2021-11-21T22:01:46.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638b7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c05f730 0x7f638c141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f63909e8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f638c142370 0x7f638c142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638bfff700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f638c0b02e0 0x7f638c0af620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:34182/0 (socket says 172.21.2.116:34182) 2021-11-21T22:01:46.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638bfff700 1 -- 172.21.2.116:0/3979916837 learned_addr learned my addr 172.21.2.116:0/3979916837 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:46.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638b7fe700 1 -- 172.21.2.116:0/3979916837 >> v2:172.21.2.117:3300/0 conn(0x7f638c142370 msgr2=0x7f638c142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:46.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638b7fe700 1 --2- 172.21.2.116:0/3979916837 >> v2:172.21.2.117:3300/0 conn(0x7f638c142370 0x7f638c142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638b7fe700 1 -- 172.21.2.116:0/3979916837 >> v1:172.21.2.117:6789/0 conn(0x7f638c0b02e0 legacy=0x7f638c0af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638b7fe700 1 -- 172.21.2.116:0/3979916837 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c0b26d0 con 0x7f638c05f730 2021-11-21T22:01:46.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.178+0000 7f638b7fe700 1 --2- 172.21.2.116:0/3979916837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c05f730 0x7f638c141e80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f637c002f70 tx=0x7f637c006e80).ready entity=mon.2 client_cookie=56843ced2d68847f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638affd700 1 -- 172.21.2.116:0/3979916837 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637c00b7d0 con 0x7f638c05f730 2021-11-21T22:01:46.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638affd700 1 -- 172.21.2.116:0/3979916837 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c00b930 con 0x7f638c05f730 2021-11-21T22:01:46.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638affd700 1 -- 172.21.2.116:0/3979916837 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637c00bc90 con 0x7f638c05f730 2021-11-21T22:01:46.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3979916837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c05f730 msgr2=0x7f638c141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 --2- 172.21.2.116:0/3979916837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c05f730 0x7f638c141e80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f637c002f70 tx=0x7f637c006e80).stop 2021-11-21T22:01:46.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3979916837 shutdown_connections 2021-11-21T22:01:46.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 --2- 172.21.2.116:0/3979916837 >> v2:172.21.2.117:3300/0 conn(0x7f638c142370 0x7f638c142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 --2- 172.21.2.116:0/3979916837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c05f730 0x7f638c141e80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3979916837 >> 172.21.2.116:0/3979916837 conn(0x7f638c0b5ff0 msgr2=0x7f638c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:46.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3979916837 shutdown_connections 2021-11-21T22:01:46.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3979916837 wait complete. 2021-11-21T22:01:46.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 Processor -- start 2021-11-21T22:01:46.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- start start 2021-11-21T22:01:46.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f638c05f730 0x7f638c1453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f638c0b02e0 0x7f638c0abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c142370 0x7f638c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0ac470 con 0x7f638c05f730 2021-11-21T22:01:46.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c146f30 con 0x7f638c0b02e0 2021-11-21T22:01:46.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c155840 con 0x7f638c142370 2021-11-21T22:01:46.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f638c05f730 0x7f638c1453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f638c05f730 0x7f638c1453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51556/0 (socket says 172.21.2.116:51556) 2021-11-21T22:01:46.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 -- 172.21.2.116:0/3428978865 learned_addr learned my addr 172.21.2.116:0/3428978865 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:46.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 -- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c142370 msgr2=0x7f638c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c142370 0x7f638c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 -- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f638c0b02e0 msgr2=0x7f638c0abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f638c0b02e0 0x7f638c0abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 -- 172.21.2.116:0/3428978865 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c00d410 con 0x7f638c05f730 2021-11-21T22:01:46.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f638bfff700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f638c05f730 0x7f638c1453f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f638c0b2370 tx=0x7f6374007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f63897fa700 1 -- 172.21.2.116:0/3428978865 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400ce30 con 0x7f638c05f730 2021-11-21T22:01:46.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f63897fa700 1 -- 172.21.2.116:0/3428978865 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63740072e0 con 0x7f638c05f730 2021-11-21T22:01:46.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f63897fa700 1 -- 172.21.2.116:0/3428978865 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6374005800 con 0x7f638c05f730 2021-11-21T22:01:46.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c155ae0 con 0x7f638c05f730 2021-11-21T22:01:46.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c0a9d30 con 0x7f638c05f730 2021-11-21T22:01:46.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c155c70 con 0x7f638c05f730 2021-11-21T22:01:46.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f63897fa700 1 -- 172.21.2.116:0/3428978865 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6374010070 con 0x7f638c05f730 2021-11-21T22:01:46.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f63897fa700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6370053540 0x7f6370055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.182+0000 7f63897fa700 1 -- 172.21.2.116:0/3428978865 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f637406b0e0 con 0x7f638c05f730 2021-11-21T22:01:46.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.186+0000 7f638b7fe700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6370053540 0x7f6370055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.186+0000 7f638b7fe700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6370053540 0x7f6370055a00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f637c002f70 tx=0x7f638c0b2860).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.186+0000 7f63897fa700 1 -- 172.21.2.116:0/3428978865 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6374042130 con 0x7f638c05f730 2021-11-21T22:01:46.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f638c142cf0 con 0x7f638c05f730 2021-11-21T22:01:46.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f63897fa700 1 -- 172.21.2.116:0/3428978865 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6374045750 con 0x7f638c05f730 2021-11-21T22:01:46.275 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:46.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6370053540 msgr2=0x7f6370055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6370053540 0x7f6370055a00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f637c002f70 tx=0x7f638c0b2860).stop 2021-11-21T22:01:46.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f638c05f730 msgr2=0x7f638c1453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f638c05f730 0x7f638c1453f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f638c0b2370 tx=0x7f6374007c00).stop 2021-11-21T22:01:46.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 shutdown_connections 2021-11-21T22:01:46.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f638c142370 0x7f638c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f638c0b02e0 0x7f638c0abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6370053540 0x7f6370055a00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 --2- 172.21.2.116:0/3428978865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f638c05f730 0x7f638c1453f0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 >> 172.21.2.116:0/3428978865 conn(0x7f638c0b5ff0 msgr2=0x7f638c0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:46.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 shutdown_connections 2021-11-21T22:01:46.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.270+0000 7f639244b700 1 -- 172.21.2.116:0/3428978865 wait complete. 2021-11-21T22:01:46.284 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:01:46.285 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:46.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.330+0000 7f581f07f700 1 Processor -- start 2021-11-21T22:01:46.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.330+0000 7f581f07f700 1 -- start start 2021-11-21T22:01:46.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58180a6aa0 0x7f581814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58180a6e80 con 0x7f5818062410 2021-11-21T22:01:46.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581814a7b0 con 0x7f58180a86b0 2021-11-21T22:01:46.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58180a8bc0 con 0x7f58180a6aa0 2021-11-21T22:01:46.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581d61c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5818062410 0x7f58181443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35288/0 (socket says 172.21.2.116:35288) 2021-11-21T22:01:46.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581d61c700 1 -- 172.21.2.116:0/3889678239 learned_addr learned my addr 172.21.2.116:0/3889678239 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:46.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601973344 0 0) 0x7f58180a6e80 con 0x7f5818062410 2021-11-21T22:01:46.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f8003260 con 0x7f5818062410 2021-11-21T22:01:46.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3136127703 0 0) 0x7f57f8003260 con 0x7f5818062410 2021-11-21T22:01:46.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58180a6aa0 msgr2=0x7f581814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:46.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58180a6aa0 0x7f581814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 >> v1:172.21.2.117:6789/0 conn(0x7f58180a86b0 legacy=0x7f58181470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:46.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581814aa10 con 0x7f5818062410 2021-11-21T22:01:46.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f5810001a00 con 0x7f5818062410 2021-11-21T22:01:46.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 >> v1:172.21.2.116:6789/0 conn(0x7f5818062410 legacy=0x7f58181443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57f8003800 0x7f57f8003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58180a6aa0 0x7f57f8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8004bf0 0x7f57f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180a8bc0 con 0x7f57f8003800 2021-11-21T22:01:46.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80052a0 con 0x7f57f8004bf0 2021-11-21T22:01:46.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f800bc40 con 0x7f58180a6aa0 2021-11-21T22:01:46.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57f8003800 0x7f57f8003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581ce1b700 1 -- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58180a6aa0 msgr2=0x7f57f8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:46.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f58180a6aa0 0x7f57f8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581ce1b700 1 -- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8004bf0 msgr2=0x7f57f800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:46.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8004bf0 0x7f57f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:46.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581ce1b700 1 -- 172.21.2.116:0/3889678239 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8003260 con 0x7f57f8003800 2021-11-21T22:01:46.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57f8003800 0x7f57f8003c00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f5800002f70 tx=0x7f5800006ca0).ready entity=mon.0 client_cookie=59402a9322572b49 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5800012410 con 0x7f57f8003800 2021-11-21T22:01:46.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5800012570 con 0x7f57f8003800 2021-11-21T22:01:46.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f58177fe700 1 -- 172.21.2.116:0/3889678239 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58000128d0 con 0x7f57f8003800 2021-11-21T22:01:46.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- 172.21.2.116:0/3889678239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57f8003800 msgr2=0x7f57f8003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 --2- 172.21.2.116:0/3889678239 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f57f8003800 0x7f57f8003c00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f5800002f70 tx=0x7f5800006ca0).stop 2021-11-21T22:01:46.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581d61c700 1 -- 172.21.2.116:0/3889678239 reap_dead start 2021-11-21T22:01:46.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- 172.21.2.116:0/3889678239 shutdown_connections 2021-11-21T22:01:46.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- 172.21.2.116:0/3889678239 >> 172.21.2.116:0/3889678239 conn(0x7f581805bd20 msgr2=0x7f581805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:46.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- 172.21.2.116:0/3889678239 shutdown_connections 2021-11-21T22:01:46.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- 172.21.2.116:0/3889678239 wait complete. 2021-11-21T22:01:46.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 Processor -- start 2021-11-21T22:01:46.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.334+0000 7f581f07f700 1 -- start start 2021-11-21T22:01:46.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8003800 0x7f5818145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58180ab530 0x7f5818159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f581815a130 0x7f58180aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818146240 con 0x7f58180ab530 2021-11-21T22:01:46.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180a93e0 con 0x7f57f8003800 2021-11-21T22:01:46.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58180a99f0 con 0x7f581815a130 2021-11-21T22:01:46.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58180ab530 0x7f5818159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58180ab530 0x7f5818159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51576/0 (socket says 172.21.2.116:51576) 2021-11-21T22:01:46.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 -- 172.21.2.116:0/3705197855 learned_addr learned my addr 172.21.2.116:0/3705197855 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:46.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581d61c700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f581815a130 0x7f58180aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8003800 0x7f5818145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 -- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f581815a130 msgr2=0x7f58180aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f581815a130 0x7f58180aa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 -- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8003800 msgr2=0x7f5818145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8003800 0x7f5818145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 -- 172.21.2.116:0/3705197855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180a9c90 con 0x7f58180ab530 2021-11-21T22:01:46.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581d61c700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f581815a130 0x7f58180aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:46.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8003800 0x7f5818145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:46.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5817fff700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58180ab530 0x7f5818159c40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f5808000f70 tx=0x7f5808006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5815ffb700 1 -- 172.21.2.116:0/3705197855 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5808013460 con 0x7f58180ab530 2021-11-21T22:01:46.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5815ffb700 1 -- 172.21.2.116:0/3705197855 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080135c0 con 0x7f58180ab530 2021-11-21T22:01:46.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5815ffb700 1 -- 172.21.2.116:0/3705197855 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5808013920 con 0x7f58180ab530 2021-11-21T22:01:46.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5818065020 con 0x7f58180ab530 2021-11-21T22:01:46.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581815a5b0 con 0x7f58180ab530 2021-11-21T22:01:46.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58180a9550 con 0x7f58180ab530 2021-11-21T22:01:46.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5815ffb700 1 -- 172.21.2.116:0/3705197855 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f580801a060 con 0x7f58180ab530 2021-11-21T22:01:46.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5815ffb700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f57fc053600 0x7f57fc055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:46.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f5815ffb700 1 -- 172.21.2.116:0/3705197855 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f580806b370 con 0x7f58180ab530 2021-11-21T22:01:46.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.338+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f57fc053600 0x7f57fc055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:46.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.342+0000 7f581ce1b700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f57fc053600 0x7f57fc055ac0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f5800016000 tx=0x7f5800015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:46.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.342+0000 7f5815ffb700 1 -- 172.21.2.116:0/3705197855 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58180a9550 con 0x7f58180ab530 2021-11-21T22:01:46.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58180a9550 con 0x7f58180ab530 2021-11-21T22:01:46.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f5815ffb700 1 -- 172.21.2.116:0/3705197855 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f58180a9550 con 0x7f58180ab530 2021-11-21T22:01:46.454 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:46.454 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":49,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:43.226243+0000","modified":"2021-11-21T22:01:44.230974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1808795000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6833","nonce":1808795000},{"type":"v1","addr":"172.21.2.116:6835","nonce":1808795000}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-11-21T22:01:46.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f57fc053600 msgr2=0x7f57fc055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f57fc053600 0x7f57fc055ac0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f5800016000 tx=0x7f5800015000).stop 2021-11-21T22:01:46.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58180ab530 msgr2=0x7f5818159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:46.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58180ab530 0x7f5818159c40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f5808000f70 tx=0x7f5808006dc0).stop 2021-11-21T22:01:46.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 shutdown_connections 2021-11-21T22:01:46.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f581815a130 0x7f58180aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f57fc053600 0x7f57fc055ac0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f58180ab530 0x7f5818159c40 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 --2- 172.21.2.116:0/3705197855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f57f8003800 0x7f5818145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:46.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 >> 172.21.2.116:0/3705197855 conn(0x7f581805bd20 msgr2=0x7f58181441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:46.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 shutdown_connections 2021-11-21T22:01:46.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:46.450+0000 7f581f07f700 1 -- 172.21.2.116:0/3705197855 wait complete. 2021-11-21T22:01:46.459 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 49 2021-11-21T22:01:46.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:43.226243+0000', 'modified': '2021-11-21T22:01:44.230974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7356}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7356': {'gid': 7356, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/1808795000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6833', 'nonce': 1808795000}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1808795000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:46.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:46.764 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:46.829 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7356,"want_state":"up:active","state":"up:active","rank_uptime":3.5954146530000002,"mdsmap_epoch":49,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":7.7998985960000002} 2021-11-21T22:01:46.829 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7356, "mdsmap_epoch": 49, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 3.595414653, "state": "up:active", "uptime": 7.799898596, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:46.830 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:01:46.830 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:01:46.831 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:01:46.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:46.832 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:46.842 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:46.854 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:46.854 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:46.860 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:46.861 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:46.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:46.862 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:46.866 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:01:46.867 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:46.911 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:46.920 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:46.921 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:46.926 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:46.926 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:46.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:46.928 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:46.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:01:46.949+0000 7fa4c751d080 -1 init, newargv = 0x5560d3887910 newargc=9ceph-fuse[14960]: starting ceph client 2021-11-21T22:01:46.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr: 2021-11-21T22:01:46.960 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14960]: starting fuse 2021-11-21T22:01:48.032 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:01:48.045 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:48.045 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:48.096 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:48.097 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:48.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:48.098 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:01:48.104 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:01:48.104 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:01:48.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:48.105 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:01:48.105 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:01:48.106 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:01:48.106 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:01:48.107 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:01:48.107 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:48.108 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:01:48.108 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:01:48.108 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:01:48.109 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:48.109 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:01:48.110 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:01:48.110 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:01:48.110 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:01:48.111 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:48.111 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:01:48.112 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:01:48.112 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:01:48.112 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:01:48.113 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:01:48.114 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:01:48.114 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:01:48.114 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:01:48.180 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14960.asok 2021-11-21T22:01:48.182 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14960.asok 2021-11-21T22:01:48.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:48.183 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14960.asok status 2021-11-21T22:01:48.231 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:01:48.231 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:01:48.231 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:01:48.232 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:01:48.232 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:01:48.232 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:01:48.232 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:01:48.233 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14960", 2021-11-21T22:01:48.233 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:01:48.233 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:48.233 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:01:48.233 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:01:48.234 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5074, 2021-11-21T22:01:48.234 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:01:48.234 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:01:48.234 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:01:48.234 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5074 2021-11-21T22:01:48.235 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:48.235 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:01:48.235 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:01:48.235 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:01:48.236 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1667228942 2021-11-21T22:01:48.236 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:01:48.236 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:48.236 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:01:48.236 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:01:48.237 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:01:48.237 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1667228942 2021-11-21T22:01:48.237 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:01:48.237 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5074 172.21.2.119:0/1667228942", 2021-11-21T22:01:48.237 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/1667228942", 2021-11-21T22:01:48.238 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:01:48.238 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 49, 2021-11-21T22:01:48.238 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 111, 2021-11-21T22:01:48.238 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:01:48.239 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:01:48.239 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:01:48.239 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:01:48.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:48.241 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:48.241 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:48.248 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:01:48.250 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:48.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:48.251 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:48.318 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:48.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 Processor -- start 2021-11-21T22:01:48.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- start start 2021-11-21T22:01:48.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1ae00b30f0 0x7f1ae00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1ae0147a70 0x7f1ae0143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae0062330 con 0x7f1ae00b20a0 2021-11-21T22:01:48.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae00b2620 con 0x7f1ae0147a70 2021-11-21T22:01:48.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae0147330 con 0x7f1ae00b30f0 2021-11-21T22:01:48.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1ae0147a70 0x7f1ae0143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1ae0147a70 0x7f1ae0143cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51584/0 (socket says 172.21.2.116:51584) 2021-11-21T22:01:48.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 -- 172.21.2.116:0/3709687013 learned_addr learned my addr 172.21.2.116:0/3709687013 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:48.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 -- 172.21.2.116:0/3709687013 >> v2:172.21.2.117:3300/0 conn(0x7f1ae00b30f0 msgr2=0x7f1ae00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1add59a700 1 -- 172.21.2.116:0/3709687013 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238929069 0 0) 0x7f1ae0062330 con 0x7f1ae00b20a0 2021-11-21T22:01:48.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 --2- 172.21.2.116:0/3709687013 >> v2:172.21.2.117:3300/0 conn(0x7f1ae00b30f0 0x7f1ae00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 -- 172.21.2.116:0/3709687013 >> v1:172.21.2.116:6789/0 conn(0x7f1ae00b20a0 legacy=0x7f1ae005f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 -- 172.21.2.116:0/3709687013 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0147100 con 0x7f1ae0147a70 2021-11-21T22:01:48.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1aded9d700 1 --2- 172.21.2.116:0/3709687013 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0147a70 0x7f1ae0143cf0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f1ad80032f0 tx=0x7f1ad8009290).ready entity=mon.0 client_cookie=564480c2fdb26074 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1add59a700 1 -- 172.21.2.116:0/3709687013 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad800b870 con 0x7f1ae0147a70 2021-11-21T22:01:48.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1add59a700 1 -- 172.21.2.116:0/3709687013 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad800b9d0 con 0x7f1ae0147a70 2021-11-21T22:01:48.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1add59a700 1 -- 172.21.2.116:0/3709687013 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad800bb40 con 0x7f1ae0147a70 2021-11-21T22:01:48.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- 172.21.2.116:0/3709687013 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0147a70 msgr2=0x7f1ae0143cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3709687013 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0147a70 0x7f1ae0143cf0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f1ad80032f0 tx=0x7f1ad8009290).stop 2021-11-21T22:01:48.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- 172.21.2.116:0/3709687013 shutdown_connections 2021-11-21T22:01:48.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3709687013 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0147a70 0x7f1ae0143cf0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3709687013 >> v2:172.21.2.117:3300/0 conn(0x7f1ae00b30f0 0x7f1ae00b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.380 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- 172.21.2.116:0/3709687013 >> 172.21.2.116:0/3709687013 conn(0x7f1ae00b44c0 msgr2=0x7f1ae005bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- 172.21.2.116:0/3709687013 shutdown_connections 2021-11-21T22:01:48.381 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- 172.21.2.116:0/3709687013 wait complete. 2021-11-21T22:01:48.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 Processor -- start 2021-11-21T22:01:48.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.370+0000 7f1adf59e700 1 -- start start 2021-11-21T22:01:48.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae00b20a0 0x7f1ae0067710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae00b30f0 0x7f1ae0064ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0147a70 0x7f1ae00b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0067530 con 0x7f1ae00b20a0 2021-11-21T22:01:48.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0065150 con 0x7f1ae0147a70 2021-11-21T22:01:48.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae00653f0 con 0x7f1ae00b30f0 2021-11-21T22:01:48.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae00b20a0 0x7f1ae0067710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae00b20a0 0x7f1ae0067710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51588/0 (socket says 172.21.2.116:51588) 2021-11-21T22:01:48.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 -- 172.21.2.116:0/3079017936 learned_addr learned my addr 172.21.2.116:0/3079017936 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:48.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 -- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae00b30f0 msgr2=0x7f1ae0064ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae00b30f0 0x7f1ae0064ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 -- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0147a70 msgr2=0x7f1ae00b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0147a70 0x7f1ae00b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 -- 172.21.2.116:0/3079017936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad8009530 con 0x7f1ae00b20a0 2021-11-21T22:01:48.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ade59c700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae00b20a0 0x7f1ae0067710 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f1ac8002f70 tx=0x7f1ac8006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ad77fe700 1 -- 172.21.2.116:0/3079017936 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ac8013460 con 0x7f1ae00b20a0 2021-11-21T22:01:48.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ad77fe700 1 -- 172.21.2.116:0/3079017936 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac80135c0 con 0x7f1ae00b20a0 2021-11-21T22:01:48.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ad77fe700 1 -- 172.21.2.116:0/3079017936 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ac8013920 con 0x7f1ae00b20a0 2021-11-21T22:01:48.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae0159b90 con 0x7f1ae00b20a0 2021-11-21T22:01:48.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae015a030 con 0x7f1ae00b20a0 2021-11-21T22:01:48.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae00655b0 con 0x7f1ae00b20a0 2021-11-21T22:01:48.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ad77fe700 1 -- 172.21.2.116:0/3079017936 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1ac801a060 con 0x7f1ae00b20a0 2021-11-21T22:01:48.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ad77fe700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ac4053660 0x7f1ac4055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1ad77fe700 1 -- 172.21.2.116:0/3079017936 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1ac806b4d0 con 0x7f1ae00b20a0 2021-11-21T22:01:48.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1addd9b700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ac4053660 0x7f1ac4055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.374+0000 7f1addd9b700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ac4053660 0x7f1ac4055b20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1ad0000bc0 tx=0x7f1ad0007630).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.378+0000 7f1ad77fe700 1 -- 172.21.2.116:0/3079017936 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ac8042520 con 0x7f1ae00b20a0 2021-11-21T22:01:48.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.458+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ae0159d20 con 0x7f1ae00b20a0 2021-11-21T22:01:48.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.458+0000 7f1ad77fe700 1 -- 172.21.2.116:0/3079017936 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f1ac8045b40 con 0x7f1ae00b20a0 2021-11-21T22:01:48.465 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:48.465 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":111,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:44.231112+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:01:41.057032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-11-21T22:01:42.188874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:48.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ac4053660 msgr2=0x7f1ac4055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ac4053660 0x7f1ac4055b20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1ad0000bc0 tx=0x7f1ad0007630).stop 2021-11-21T22:01:48.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae00b20a0 msgr2=0x7f1ae0067710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae00b20a0 0x7f1ae0067710 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f1ac8002f70 tx=0x7f1ac8006eb0).stop 2021-11-21T22:01:48.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 shutdown_connections 2021-11-21T22:01:48.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0147a70 0x7f1ae00b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae00b30f0 0x7f1ae0064ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ac4053660 0x7f1ac4055b20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3079017936 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae00b20a0 0x7f1ae0067710 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 >> 172.21.2.116:0/3079017936 conn(0x7f1ae00b44c0 msgr2=0x7f1ae00606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 shutdown_connections 2021-11-21T22:01:48.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.462+0000 7f1adf59e700 1 -- 172.21.2.116:0/3079017936 wait complete. 2021-11-21T22:01:48.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:01:41.057032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:01:42.188874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:01:48.473 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:48.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 Processor -- start 2021-11-21T22:01:48.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- start start 2021-11-21T22:01:48.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf78144a10 0x7faf780b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780b3690 0x7faf7813a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf780b3c50 con 0x7faf780adc10 2021-11-21T22:01:48.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf780b3ef0 con 0x7faf78144a10 2021-11-21T22:01:48.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf780b4190 con 0x7faf780b3690 2021-11-21T22:01:48.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf78144a10 0x7faf780b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7faf78144a10 0x7faf780b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51596/0 (socket says 172.21.2.116:51596) 2021-11-21T22:01:48.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 -- 172.21.2.116:0/2299195945 learned_addr learned my addr 172.21.2.116:0/2299195945 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:48.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 -- 172.21.2.116:0/2299195945 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780b3690 msgr2=0x7faf7813a540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:48.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 --2- 172.21.2.116:0/2299195945 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780b3690 0x7faf7813a540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 -- 172.21.2.116:0/2299195945 >> v1:172.21.2.116:6789/0 conn(0x7faf780adc10 legacy=0x7faf78141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 -- 172.21.2.116:0/2299195945 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78140b80 con 0x7faf78144a10 2021-11-21T22:01:48.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf76ffd700 1 --2- 172.21.2.116:0/2299195945 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf78144a10 0x7faf780b2030 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c006eb0).ready entity=mon.0 client_cookie=ee9bff00cc99002e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf767fc700 1 -- 172.21.2.116:0/2299195945 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c00d5c0 con 0x7faf78144a10 2021-11-21T22:01:48.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf767fc700 1 -- 172.21.2.116:0/2299195945 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c000e90 con 0x7faf78144a10 2021-11-21T22:01:48.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf767fc700 1 -- 172.21.2.116:0/2299195945 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c00d950 con 0x7faf78144a10 2021-11-21T22:01:48.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- 172.21.2.116:0/2299195945 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf78144a10 msgr2=0x7faf780b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 --2- 172.21.2.116:0/2299195945 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf78144a10 0x7faf780b2030 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c006eb0).stop 2021-11-21T22:01:48.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- 172.21.2.116:0/2299195945 shutdown_connections 2021-11-21T22:01:48.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 --2- 172.21.2.116:0/2299195945 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780b3690 0x7faf7813a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 --2- 172.21.2.116:0/2299195945 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf78144a10 0x7faf780b2030 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- 172.21.2.116:0/2299195945 >> 172.21.2.116:0/2299195945 conn(0x7faf7805bd20 msgr2=0x7faf7805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- 172.21.2.116:0/2299195945 shutdown_connections 2021-11-21T22:01:48.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.522+0000 7faf7e20b700 1 -- 172.21.2.116:0/2299195945 wait complete. 2021-11-21T22:01:48.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 Processor -- start 2021-11-21T22:01:48.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 -- start start 2021-11-21T22:01:48.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780adc10 0x7faf780af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780b3690 0x7faf780b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf78144a10 0x7faf780afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf78133b50 con 0x7faf780b3690 2021-11-21T22:01:48.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf78133fc0 con 0x7faf78144a10 2021-11-21T22:01:48.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf780b1580 con 0x7faf780adc10 2021-11-21T22:01:48.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780b3690 0x7faf780b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780b3690 0x7faf780b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51602/0 (socket says 172.21.2.116:51602) 2021-11-21T22:01:48.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 -- 172.21.2.116:0/644305772 learned_addr learned my addr 172.21.2.116:0/644305772 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:48.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 -- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780adc10 msgr2=0x7faf780af440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:48.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780adc10 0x7faf780af440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 -- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf78144a10 msgr2=0x7faf780afd90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:48.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf78144a10 0x7faf780afd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 -- 172.21.2.116:0/644305772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c00b1e0 con 0x7faf780b3690 2021-11-21T22:01:48.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf76ffd700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780b3690 0x7faf780b5c40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf74ff9700 1 -- 172.21.2.116:0/644305772 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c0068b0 con 0x7faf780b3690 2021-11-21T22:01:48.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf74ff9700 1 -- 172.21.2.116:0/644305772 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c011ce0 con 0x7faf780b3690 2021-11-21T22:01:48.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf780b1820 con 0x7faf780b3690 2021-11-21T22:01:48.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf78062b60 con 0x7faf780b3690 2021-11-21T22:01:48.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf74ff9700 1 -- 172.21.2.116:0/644305772 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c01e650 con 0x7faf780b3690 2021-11-21T22:01:48.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf780b19b0 con 0x7faf780b3690 2021-11-21T22:01:48.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf74ff9700 1 -- 172.21.2.116:0/644305772 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7faf6c025030 con 0x7faf780b3690 2021-11-21T22:01:48.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf74ff9700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf58053660 0x7faf58055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf74ff9700 1 -- 172.21.2.116:0/644305772 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faf6c06cec0 con 0x7faf780b3690 2021-11-21T22:01:48.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.526+0000 7faf777fe700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf58053660 0x7faf58055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.530+0000 7faf777fe700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf58053660 0x7faf58055b20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7faf7805f430 tx=0x7faf68007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.530+0000 7faf74ff9700 1 -- 172.21.2.116:0/644305772 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faf6c043f10 con 0x7faf780b3690 2021-11-21T22:01:48.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.610+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7faf780573d0 con 0x7faf780b3690 2021-11-21T22:01:48.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.610+0000 7faf74ff9700 1 -- 172.21.2.116:0/644305772 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v111) v1 ==== 110+0+10 (secure 0 0 0) 0x7faf6c023020 con 0x7faf780b3690 2021-11-21T22:01:48.617 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:48.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf58053660 msgr2=0x7faf58055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf58053660 0x7faf58055b20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7faf7805f430 tx=0x7faf68007790).stop 2021-11-21T22:01:48.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780b3690 msgr2=0x7faf780b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780b3690 0x7faf780b5c40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c015000).stop 2021-11-21T22:01:48.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 shutdown_connections 2021-11-21T22:01:48.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7faf78144a10 0x7faf780afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7faf58053660 0x7faf58055b20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7faf780b3690 0x7faf780b5c40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 --2- 172.21.2.116:0/644305772 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7faf780adc10 0x7faf780af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 >> 172.21.2.116:0/644305772 conn(0x7faf7805bd20 msgr2=0x7faf7813f160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 shutdown_connections 2021-11-21T22:01:48.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.614+0000 7faf7e20b700 1 -- 172.21.2.116:0/644305772 wait complete. 2021-11-21T22:01:48.624 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:48.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.670+0000 7f9823054700 1 Processor -- start 2021-11-21T22:01:48.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- start start 2021-11-21T22:01:48.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f981c062d00 0x7f981c064c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c0b0120 0x7f981c143f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c0630e0 con 0x7f981c0af850 2021-11-21T22:01:48.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c064550 con 0x7f981c062d00 2021-11-21T22:01:48.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0b23f0 con 0x7f981c0b0120 2021-11-21T22:01:48.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f98215f1700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f981c0af850 0x7f981c13dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35334/0 (socket says 172.21.2.116:35334) 2021-11-21T22:01:48.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f98215f1700 1 -- 172.21.2.116:0/2378215726 learned_addr learned my addr 172.21.2.116:0/2378215726 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:48.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074330944 0 0) 0x7f981c0630e0 con 0x7f981c0af850 2021-11-21T22:01:48.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9808003260 con 0x7f981c0af850 2021-11-21T22:01:48.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 --2- 172.21.2.116:0/2378215726 >> v2:172.21.2.117:3300/0 conn(0x7f981c062d00 0x7f981c064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981bfff700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c0b0120 0x7f981c143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 103895231 0 0) 0x7f9808003260 con 0x7f981c0af850 2021-11-21T22:01:48.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c0b0120 msgr2=0x7f981c143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c0b0120 0x7f981c143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 >> v2:172.21.2.117:3300/0 conn(0x7f981c062d00 msgr2=0x7f981c064c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 --2- 172.21.2.116:0/2378215726 >> v2:172.21.2.117:3300/0 conn(0x7f981c062d00 0x7f981c064c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0b0ea0 con 0x7f981c0af850 2021-11-21T22:01:48.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f980c001680 con 0x7f981c0af850 2021-11-21T22:01:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 >> v1:172.21.2.116:6789/0 conn(0x7f981c0af850 legacy=0x7f981c13dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f9808003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c0b0120 0x7f9808004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98080049f0 0x7f9808004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c064550 con 0x7f98080037d0 2021-11-21T22:01:48.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0b23f0 con 0x7f98080049f0 2021-11-21T22:01:48.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980800b880 con 0x7f981c0b0120 2021-11-21T22:01:48.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f9808003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 -- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c0b0120 msgr2=0x7f9808004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c0b0120 0x7f9808004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 -- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98080049f0 msgr2=0x7f9808004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98080049f0 0x7f9808004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 -- 172.21.2.116:0/2378215726 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808003260 con 0x7f98080037d0 2021-11-21T22:01:48.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f9808003bf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f981000a000 tx=0x7f9810008040).ready entity=mon.0 client_cookie=134db62258aaa2ed server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9810013670 con 0x7f98080037d0 2021-11-21T22:01:48.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98100137d0 con 0x7f98080037d0 2021-11-21T22:01:48.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f981b7fe700 1 -- 172.21.2.116:0/2378215726 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9810013b30 con 0x7f98080037d0 2021-11-21T22:01:48.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- 172.21.2.116:0/2378215726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 msgr2=0x7f9808003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 --2- 172.21.2.116:0/2378215726 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f9808003bf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f981000a000 tx=0x7f9810008040).stop 2021-11-21T22:01:48.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f98215f1700 1 -- 172.21.2.116:0/2378215726 reap_dead start 2021-11-21T22:01:48.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- 172.21.2.116:0/2378215726 shutdown_connections 2021-11-21T22:01:48.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- 172.21.2.116:0/2378215726 >> 172.21.2.116:0/2378215726 conn(0x7f981c05bd20 msgr2=0x7f981c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- 172.21.2.116:0/2378215726 shutdown_connections 2021-11-21T22:01:48.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- 172.21.2.116:0/2378215726 wait complete. 2021-11-21T22:01:48.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 Processor -- start 2021-11-21T22:01:48.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- start start 2021-11-21T22:01:48.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f981c139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f981c0ac930 0x7f981c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c13e880 0x7f981c13c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0b34c0 con 0x7f98080037d0 2021-11-21T22:01:48.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0b48e0 con 0x7f981c0ac930 2021-11-21T22:01:48.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9823054700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0b4b80 con 0x7f981c13e880 2021-11-21T22:01:48.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f981c139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f981c139240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51622/0 (socket says 172.21.2.116:51622) 2021-11-21T22:01:48.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.674+0000 7f9820df0700 1 -- 172.21.2.116:0/2920347828 learned_addr learned my addr 172.21.2.116:0/2920347828 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:48.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f981bfff700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f981c0ac930 0x7f981c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f98215f1700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c13e880 0x7f981c13c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9820df0700 1 -- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c13e880 msgr2=0x7f981c13c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9820df0700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c13e880 0x7f981c13c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9820df0700 1 -- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f981c0ac930 msgr2=0x7f981c0add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9820df0700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f981c0ac930 0x7f981c0add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9820df0700 1 -- 172.21.2.116:0/2920347828 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0ad4b0 con 0x7f98080037d0 2021-11-21T22:01:48.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f98215f1700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c13e880 0x7f981c13c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:48.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9820df0700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f981c139240 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f981000a000 tx=0x7f9810008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9819ffb700 1 -- 172.21.2.116:0/2920347828 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9810012d00 con 0x7f98080037d0 2021-11-21T22:01:48.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9819ffb700 1 -- 172.21.2.116:0/2920347828 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810000e90 con 0x7f98080037d0 2021-11-21T22:01:48.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c0ad750 con 0x7f98080037d0 2021-11-21T22:01:48.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c0b5410 con 0x7f98080037d0 2021-11-21T22:01:48.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9819ffb700 1 -- 172.21.2.116:0/2920347828 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f981001d600 con 0x7f98080037d0 2021-11-21T22:01:48.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c0573d0 con 0x7f98080037d0 2021-11-21T22:01:48.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9819ffb700 1 -- 172.21.2.116:0/2920347828 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f981001d760 con 0x7f98080037d0 2021-11-21T22:01:48.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9819ffb700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97fc0536d0 0x7f97fc055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f9819ffb700 1 -- 172.21.2.116:0/2920347828 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f981006d310 con 0x7f98080037d0 2021-11-21T22:01:48.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f981bfff700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97fc0536d0 0x7f97fc055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.678+0000 7f981bfff700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97fc0536d0 0x7f97fc055b90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f9814008000 tx=0x7f9814008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.682+0000 7f9819ffb700 1 -- 172.21.2.116:0/2920347828 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9810044340 con 0x7f98080037d0 2021-11-21T22:01:48.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.762+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f981c13e640 con 0x7f98080037d0 2021-11-21T22:01:48.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9819ffb700 1 -- 172.21.2.116:0/2920347828 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9810012e60 con 0x7f98080037d0 2021-11-21T22:01:48.769 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:48.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97fc0536d0 msgr2=0x7f97fc055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97fc0536d0 0x7f97fc055b90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f9814008000 tx=0x7f9814008040).stop 2021-11-21T22:01:48.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 msgr2=0x7f981c139240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f981c139240 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f981000a000 tx=0x7f9810008040).stop 2021-11-21T22:01:48.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 shutdown_connections 2021-11-21T22:01:48.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f981c13e880 0x7f981c13c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f981c0ac930 0x7f981c0add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97fc0536d0 0x7f97fc055b90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 --2- 172.21.2.116:0/2920347828 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98080037d0 0x7f981c139240 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 >> 172.21.2.116:0/2920347828 conn(0x7f981c05bd20 msgr2=0x7f981c066100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 shutdown_connections 2021-11-21T22:01:48.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.766+0000 7f9823054700 1 -- 172.21.2.116:0/2920347828 wait complete. 2021-11-21T22:01:48.776 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:01:48.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 Processor -- start 2021-11-21T22:01:48.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- start start 2021-11-21T22:01:48.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7eff2c144810 0x7eff2c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff2c0abce0 0x7eff2c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c147e80 con 0x7eff2c0b1db0 2021-11-21T22:01:48.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c0b1490 con 0x7eff2c144810 2021-11-21T22:01:48.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0af060 con 0x7eff2c0abce0 2021-11-21T22:01:48.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3201e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7eff2c0b1db0 0x7eff2c0ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35356/0 (socket says 172.21.2.116:35356) 2021-11-21T22:01:48.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3201e700 1 -- 172.21.2.116:0/2251689331 learned_addr learned my addr 172.21.2.116:0/2251689331 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:48.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192558961 0 0) 0x7eff2c147e80 con 0x7eff2c0b1db0 2021-11-21T22:01:48.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff14003260 con 0x7eff2c0b1db0 2021-11-21T22:01:48.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3101c700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff2c0abce0 0x7eff2c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 --2- 172.21.2.116:0/2251689331 >> v2:172.21.2.117:3300/0 conn(0x7eff2c144810 0x7eff2c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1552778788 0 0) 0x7eff14003260 con 0x7eff2c0b1db0 2021-11-21T22:01:48.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff2c0abce0 msgr2=0x7eff2c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff2c0abce0 0x7eff2c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 >> v2:172.21.2.117:3300/0 conn(0x7eff2c144810 msgr2=0x7eff2c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 --2- 172.21.2.116:0/2251689331 >> v2:172.21.2.117:3300/0 conn(0x7eff2c144810 0x7eff2c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0af5a0 con 0x7eff2c0b1db0 2021-11-21T22:01:48.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7eff24001a00 con 0x7eff2c0b1db0 2021-11-21T22:01:48.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 >> v1:172.21.2.116:6789/0 conn(0x7eff2c0b1db0 legacy=0x7eff2c0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff140037d0 0x7eff14003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff2c0abce0 0x7eff14004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff14004b10 0x7eff14004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0b1490 con 0x7eff140037d0 2021-11-21T22:01:48.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0af060 con 0x7eff14004b10 2021-11-21T22:01:48.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400b990 con 0x7eff2c0abce0 2021-11-21T22:01:48.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 --2- 172.21.2.116:0/2251689331 >> v2:172.21.2.117:3300/0 conn(0x7eff2c144810 0x7eff2c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff140037d0 0x7eff14003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 -- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff2c0abce0 msgr2=0x7eff14004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff2c0abce0 0x7eff14004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 -- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff14004b10 msgr2=0x7eff14004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff14004b10 0x7eff14004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 -- 172.21.2.116:0/2251689331 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14003260 con 0x7eff140037d0 2021-11-21T22:01:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3181d700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff140037d0 0x7eff14003bf0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7eff28007000 tx=0x7eff28008040).ready entity=mon.0 client_cookie=c6dc8f74592cacc server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff28015070 con 0x7eff140037d0 2021-11-21T22:01:48.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff28011410 con 0x7eff140037d0 2021-11-21T22:01:48.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3081b700 1 -- 172.21.2.116:0/2251689331 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff28011770 con 0x7eff140037d0 2021-11-21T22:01:48.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- 172.21.2.116:0/2251689331 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff140037d0 msgr2=0x7eff14003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 --2- 172.21.2.116:0/2251689331 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff140037d0 0x7eff14003bf0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7eff28007000 tx=0x7eff28008040).stop 2021-11-21T22:01:48.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff3201e700 1 -- 172.21.2.116:0/2251689331 reap_dead start 2021-11-21T22:01:48.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- 172.21.2.116:0/2251689331 shutdown_connections 2021-11-21T22:01:48.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- 172.21.2.116:0/2251689331 >> 172.21.2.116:0/2251689331 conn(0x7eff2c05bd20 msgr2=0x7eff2c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- 172.21.2.116:0/2251689331 shutdown_connections 2021-11-21T22:01:48.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.826+0000 7eff33a81700 1 -- 172.21.2.116:0/2251689331 wait complete. 2021-11-21T22:01:48.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 Processor -- start 2021-11-21T22:01:48.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 -- start start 2021-11-21T22:01:48.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff140037d0 0x7eff2c143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff2c15a170 0x7eff2c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff2c14a350 0x7eff2c14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c148ba0 con 0x7eff2c15a170 2021-11-21T22:01:48.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0b45e0 con 0x7eff2c14a350 2021-11-21T22:01:48.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c149640 con 0x7eff140037d0 2021-11-21T22:01:48.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff2c15a170 0x7eff2c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff2c15a170 0x7eff2c15a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51642/0 (socket says 172.21.2.116:51642) 2021-11-21T22:01:48.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 -- 172.21.2.116:0/357680277 learned_addr learned my addr 172.21.2.116:0/357680277 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:48.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 -- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff140037d0 msgr2=0x7eff2c143980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:48.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff140037d0 0x7eff2c143980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 -- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff2c14a350 msgr2=0x7eff2c14a7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:48.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff2c14a350 0x7eff2c14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 -- 172.21.2.116:0/357680277 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c15aac0 con 0x7eff2c15a170 2021-11-21T22:01:48.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3101c700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff2c15a170 0x7eff2c15a5d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7eff1c00a000 tx=0x7eff1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff1affd700 1 -- 172.21.2.116:0/357680277 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff1c016070 con 0x7eff2c15a170 2021-11-21T22:01:48.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff1affd700 1 -- 172.21.2.116:0/357680277 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c000e90 con 0x7eff2c15a170 2021-11-21T22:01:48.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff1affd700 1 -- 172.21.2.116:0/357680277 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff1c011a40 con 0x7eff2c15a170 2021-11-21T22:01:48.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c0b4b70 con 0x7eff2c15a170 2021-11-21T22:01:48.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c0646a0 con 0x7eff2c15a170 2021-11-21T22:01:48.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c149800 con 0x7eff2c15a170 2021-11-21T22:01:48.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff1affd700 1 -- 172.21.2.116:0/357680277 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7eff1c011ba0 con 0x7eff2c15a170 2021-11-21T22:01:48.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff1affd700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff0c0535b0 0x7eff0c055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff1affd700 1 -- 172.21.2.116:0/357680277 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7eff1c06b720 con 0x7eff2c15a170 2021-11-21T22:01:48.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.830+0000 7eff3181d700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff0c0535b0 0x7eff0c055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.834+0000 7eff3181d700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff0c0535b0 0x7eff0c055a70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7eff28007000 tx=0x7eff28008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.834+0000 7eff1affd700 1 -- 172.21.2.116:0/357680277 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7eff1c042820 con 0x7eff2c15a170 2021-11-21T22:01:48.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.914+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff2c0af300 con 0x7eff2c15a170 2021-11-21T22:01:48.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.914+0000 7eff1affd700 1 -- 172.21.2.116:0/357680277 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+10 (secure 0 0 0) 0x7eff1c045e40 con 0x7eff2c15a170 2021-11-21T22:01:48.921 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:48.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff0c0535b0 msgr2=0x7eff0c055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff0c0535b0 0x7eff0c055a70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7eff28007000 tx=0x7eff28008040).stop 2021-11-21T22:01:48.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff2c15a170 msgr2=0x7eff2c15a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff2c15a170 0x7eff2c15a5d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7eff1c00a000 tx=0x7eff1c008040).stop 2021-11-21T22:01:48.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 shutdown_connections 2021-11-21T22:01:48.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7eff2c14a350 0x7eff2c14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7eff0c0535b0 0x7eff0c055a70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7eff2c15a170 0x7eff2c15a5d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 --2- 172.21.2.116:0/357680277 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7eff140037d0 0x7eff2c143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 >> 172.21.2.116:0/357680277 conn(0x7eff2c05bd20 msgr2=0x7eff2c0b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 shutdown_connections 2021-11-21T22:01:48.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.918+0000 7eff33a81700 1 -- 172.21.2.116:0/357680277 wait complete. 2021-11-21T22:01:48.928 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:01:48.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.974+0000 7fa0ea0d8700 1 Processor -- start 2021-11-21T22:01:48.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.974+0000 7fa0ea0d8700 1 -- start start 2021-11-21T22:01:48.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0ea0d8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e40ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0ea0d8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa0e40ae1c0 0x7fa0e4142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0ea0d8700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e41433b0 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0ea0d8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e4143110 con 0x7fa0e40ae1c0 2021-11-21T22:01:48.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0ea0d8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40ada80 con 0x7fa0e41426c0 2021-11-21T22:01:48.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e3fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa0e40ae1c0 0x7fa0e4142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e37fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e40ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e2ffd700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fa0e40ac2f0 0x7fa0e40af850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:34282/0 (socket says 172.21.2.116:34282) 2021-11-21T22:01:48.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e2ffd700 1 -- 172.21.2.116:0/2069522062 learned_addr learned my addr 172.21.2.116:0/2069522062 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:48.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e3fff700 1 -- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 msgr2=0x7fa0e40ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:48.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e3fff700 1 --2- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e40ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e3fff700 1 -- 172.21.2.116:0/2069522062 >> v1:172.21.2.117:6789/0 conn(0x7fa0e40ac2f0 legacy=0x7fa0e40af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e3fff700 1 -- 172.21.2.116:0/2069522062 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40add20 con 0x7fa0e40ae1c0 2021-11-21T22:01:48.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e37fe700 1 --2- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e40ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:01:48.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.978+0000 7fa0e3fff700 1 --2- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 0x7fa0e4142aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa0dc0032f0 tx=0x7fa0dc0091d0).ready entity=mon.1 client_cookie=dab286d2129bdf85 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e27fc700 1 -- 172.21.2.116:0/2069522062 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0dc00b6c0 con 0x7fa0e40ae1c0 2021-11-21T22:01:48.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e27fc700 1 -- 172.21.2.116:0/2069522062 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0dc00b820 con 0x7fa0e40ae1c0 2021-11-21T22:01:48.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e27fc700 1 -- 172.21.2.116:0/2069522062 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0dc00b990 con 0x7fa0e40ae1c0 2021-11-21T22:01:48.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 msgr2=0x7fa0e4142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 0x7fa0e4142aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa0dc0032f0 tx=0x7fa0dc0091d0).stop 2021-11-21T22:01:48.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2069522062 shutdown_connections 2021-11-21T22:01:48.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 0x7fa0e4142aa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2069522062 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e40ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:48.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2069522062 >> 172.21.2.116:0/2069522062 conn(0x7fa0e405bd20 msgr2=0x7fa0e405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:48.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2069522062 shutdown_connections 2021-11-21T22:01:48.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2069522062 wait complete. 2021-11-21T22:01:48.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 Processor -- start 2021-11-21T22:01:48.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- start start 2021-11-21T22:01:48.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0e40ac2f0 0x7fa0e4130f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 0x7fa0e41262f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e41267e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e4125b30 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e412ec20 con 0x7fa0e40ae1c0 2021-11-21T22:01:48.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e412f710 con 0x7fa0e41426c0 2021-11-21T22:01:48.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0e40ac2f0 0x7fa0e4130f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0e40ac2f0 0x7fa0e4130f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51652/0 (socket says 172.21.2.116:51652) 2021-11-21T22:01:48.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 -- 172.21.2.116:0/2486790953 learned_addr learned my addr 172.21.2.116:0/2486790953 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:48.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 -- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 msgr2=0x7fa0e41267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e41267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 -- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 msgr2=0x7fa0e41262f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:48.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 0x7fa0e41262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:48.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 -- 172.21.2.116:0/2486790953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc00b180 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e37fe700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0e40ac2f0 0x7fa0e4130f50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fa0cc00c000 tx=0x7fa0cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e0ff9700 1 -- 172.21.2.116:0/2486790953 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0cc00c860 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e0ff9700 1 -- 172.21.2.116:0/2486790953 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc00c9c0 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc009470 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0e0ff9700 1 -- 172.21.2.116:0/2486790953 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0cc00cd20 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.982+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e4125890 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.986+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e412ed90 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.986+0000 7fa0e0ff9700 1 -- 172.21.2.116:0/2486790953 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa0cc01c460 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.986+0000 7fa0e0ff9700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c80536a0 0x7fa0c8055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:48.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.986+0000 7fa0e0ff9700 1 -- 172.21.2.116:0/2486790953 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa0cc06bff0 con 0x7fa0e40ac2f0 2021-11-21T22:01:48.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.986+0000 7fa0e2ffd700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c80536a0 0x7fa0c8055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:48.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.986+0000 7fa0e2ffd700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c80536a0 0x7fa0c8055b60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fa0e40ae0a0 tx=0x7fa0d4007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:48.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:48.986+0000 7fa0e0ff9700 1 -- 172.21.2.116:0/2486790953 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa0cc043020 con 0x7fa0e40ac2f0 2021-11-21T22:01:49.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0e405db30 con 0x7fa0e40ac2f0 2021-11-21T22:01:49.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0e0ff9700 1 -- 172.21.2.116:0/2486790953 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa0cc046640 con 0x7fa0e40ac2f0 2021-11-21T22:01:49.075 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:49.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c80536a0 msgr2=0x7fa0c8055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:49.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c80536a0 0x7fa0c8055b60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fa0e40ae0a0 tx=0x7fa0d4007800).stop 2021-11-21T22:01:49.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0e40ac2f0 msgr2=0x7fa0e4130f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:49.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0e40ac2f0 0x7fa0e4130f50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fa0cc00c000 tx=0x7fa0cc006df0).stop 2021-11-21T22:01:49.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 shutdown_connections 2021-11-21T22:01:49.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa0e41426c0 0x7fa0e41267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:49.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa0e40ae1c0 0x7fa0e41262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:49.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa0c80536a0 0x7fa0c8055b60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:49.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 --2- 172.21.2.116:0/2486790953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa0e40ac2f0 0x7fa0e4130f50 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:49.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 >> 172.21.2.116:0/2486790953 conn(0x7fa0e405bd20 msgr2=0x7fa0e4134190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:49.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 shutdown_connections 2021-11-21T22:01:49.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.070+0000 7fa0ea0d8700 1 -- 172.21.2.116:0/2486790953 wait complete. 2021-11-21T22:01:49.082 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:01:49.083 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:49.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 Processor -- start 2021-11-21T22:01:49.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- start start 2021-11-21T22:01:49.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5eec062410 0x7f5eec144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:49.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec0a6e80 con 0x7f5eec0a6aa0 2021-11-21T22:01:49.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec14a7b0 con 0x7f5eec0a86b0 2021-11-21T22:01:49.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0a8bc0 con 0x7f5eec062410 2021-11-21T22:01:49.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5eeb59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5eec062410 0x7f5eec144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:49.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5eeb59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5eec062410 0x7f5eec144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51660/0 (socket says 172.21.2.116:51660) 2021-11-21T22:01:49.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5eeb59e700 1 -- 172.21.2.116:0/3696493553 learned_addr learned my addr 172.21.2.116:0/3696493553 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:49.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5eeb59e700 1 -- 172.21.2.116:0/3696493553 >> v1:172.21.2.117:6789/0 conn(0x7f5eec0a86b0 legacy=0x7f5eec1443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:49.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5eeb59e700 1 -- 172.21.2.116:0/3696493553 >> v1:172.21.2.116:6789/0 conn(0x7f5eec0a6aa0 legacy=0x7f5eec1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:49.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5eeb59e700 1 -- 172.21.2.116:0/3696493553 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec14aa10 con 0x7f5eec062410 2021-11-21T22:01:49.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5eeb59e700 1 --2- 172.21.2.116:0/3696493553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec062410 0x7f5eec144b00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f5eec061b50 tx=0x7f5ee4009290).ready entity=mon.0 client_cookie=7ea8efc28f87e189 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:49.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ee9d9b700 1 -- 172.21.2.116:0/3696493553 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee400b860 con 0x7f5eec062410 2021-11-21T22:01:49.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ee9d9b700 1 -- 172.21.2.116:0/3696493553 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400b9c0 con 0x7f5eec062410 2021-11-21T22:01:49.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ee9d9b700 1 -- 172.21.2.116:0/3696493553 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee400bb30 con 0x7f5eec062410 2021-11-21T22:01:49.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- 172.21.2.116:0/3696493553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec062410 msgr2=0x7f5eec144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:49.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/3696493553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec062410 0x7f5eec144b00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f5eec061b50 tx=0x7f5ee4009290).stop 2021-11-21T22:01:49.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- 172.21.2.116:0/3696493553 shutdown_connections 2021-11-21T22:01:49.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/3696493553 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec062410 0x7f5eec144b00 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:49.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- 172.21.2.116:0/3696493553 >> 172.21.2.116:0/3696493553 conn(0x7f5eec05bd20 msgr2=0x7f5eec05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:49.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- 172.21.2.116:0/3696493553 shutdown_connections 2021-11-21T22:01:49.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- 172.21.2.116:0/3696493553 wait complete. 2021-11-21T22:01:49.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 Processor -- start 2021-11-21T22:01:49.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.130+0000 7f5ef11f7700 1 -- start start 2021-11-21T22:01:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5eec062410 0x7f5eec159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5eec0a6aa0 0x7f5eec15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec0a86b0 0x7f5eec0649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec13e600 con 0x7f5eec0a86b0 2021-11-21T22:01:49.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0a8d20 con 0x7f5eec062410 2021-11-21T22:01:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b0df0 con 0x7f5eec0a6aa0 2021-11-21T22:01:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec0a86b0 0x7f5eec0649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec0a86b0 0x7f5eec0649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51670/0 (socket says 172.21.2.116:51670) 2021-11-21T22:01:49.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 -- 172.21.2.116:0/2604113787 learned_addr learned my addr 172.21.2.116:0/2604113787 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 -- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5eec0a6aa0 msgr2=0x7f5eec15a2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5eec0a6aa0 0x7f5eec15a2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 -- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5eec062410 msgr2=0x7f5eec159dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:49.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5eec062410 0x7f5eec159dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:49.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 -- 172.21.2.116:0/2604113787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4009530 con 0x7f5eec0a86b0 2021-11-21T22:01:49.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eeb59e700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec0a86b0 0x7f5eec0649e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f5eec063880 tx=0x7f5ee4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:49.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ee3fff700 1 -- 172.21.2.116:0/2604113787 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee401f3f0 con 0x7f5eec0a86b0 2021-11-21T22:01:49.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ee3fff700 1 -- 172.21.2.116:0/2604113787 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee401f550 con 0x7f5eec0a86b0 2021-11-21T22:01:49.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ee3fff700 1 -- 172.21.2.116:0/2604113787 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee401f8b0 con 0x7f5eec0a86b0 2021-11-21T22:01:49.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec0b0b50 con 0x7f5eec0a86b0 2021-11-21T22:01:49.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec063c00 con 0x7f5eec0a86b0 2021-11-21T22:01:49.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec15a940 con 0x7f5eec0a86b0 2021-11-21T22:01:49.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ee3fff700 1 -- 172.21.2.116:0/2604113787 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5ee4026030 con 0x7f5eec0a86b0 2021-11-21T22:01:49.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ee3fff700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ed0053660 0x7f5ed0055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:49.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5ee3fff700 1 -- 172.21.2.116:0/2604113787 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5ee406c800 con 0x7f5eec0a86b0 2021-11-21T22:01:49.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eead9d700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ed0053660 0x7f5ed0055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:49.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.134+0000 7f5eead9d700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ed0053660 0x7f5ed0055b20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f5eec061ab0 tx=0x7f5ed4007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:49.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.138+0000 7f5ee3fff700 1 -- 172.21.2.116:0/2604113787 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ee4043850 con 0x7f5eec0a86b0 2021-11-21T22:01:49.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eec0573d0 con 0x7f5eec0a86b0 2021-11-21T22:01:49.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ee3fff700 1 -- 172.21.2.116:0/2604113787 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f5ee4024090 con 0x7f5eec0a86b0 2021-11-21T22:01:49.252 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:49.252 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":49,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:43.226243+0000","modified":"2021-11-21T22:01:44.230974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1808795000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6833","nonce":1808795000},{"type":"v1","addr":"172.21.2.116:6835","nonce":1808795000}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-11-21T22:01:49.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ed0053660 msgr2=0x7f5ed0055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:49.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ed0053660 0x7f5ed0055b20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f5eec061ab0 tx=0x7f5ed4007790).stop 2021-11-21T22:01:49.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec0a86b0 msgr2=0x7f5eec0649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:49.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec0a86b0 0x7f5eec0649e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f5eec063880 tx=0x7f5ee4016000).stop 2021-11-21T22:01:49.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 shutdown_connections 2021-11-21T22:01:49.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5ed0053660 0x7f5ed0055b20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:49.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5eec0a86b0 0x7f5eec0649e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:49.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5eec0a6aa0 0x7f5eec15a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:49.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 --2- 172.21.2.116:0/2604113787 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5eec062410 0x7f5eec159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:49.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 >> 172.21.2.116:0/2604113787 conn(0x7f5eec05bd20 msgr2=0x7f5eec0b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:49.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 shutdown_connections 2021-11-21T22:01:49.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:49.246+0000 7f5ef11f7700 1 -- 172.21.2.116:0/2604113787 wait complete. 2021-11-21T22:01:49.256 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 49 2021-11-21T22:01:49.561 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:01:43.226243+0000', 'modified': '2021-11-21T22:01:44.230974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7356}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7356': {'gid': 7356, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/1808795000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6833', 'nonce': 1808795000}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1808795000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:01:49.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:01:49.563 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:01:49.627 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7356,"want_state":"up:active","state":"up:active","rank_uptime":6.3936553399999996,"mdsmap_epoch":49,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":10.598139492} 2021-11-21T22:01:49.628 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7356, "mdsmap_epoch": 49, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 6.39365534, "state": "up:active", "uptime": 10.598139492, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:01:49.628 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:01:49.629 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:01:49.629 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:01:49.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:49.630 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:49.641 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:49.653 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:49.654 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:49.659 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:49.660 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:49.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:49.662 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:49.707 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:01:49.708 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:49.710 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:01:49.719 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:01:49.720 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:01:49.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:01:49.749+0000 7f1d5882b080 -1 init, newargv = 0x55c94a3b0bb0 newargc=9 2021-11-21T22:01:49.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14434]: starting ceph client 2021-11-21T22:01:49.756 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:01:49.757 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:01:49.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:49.758 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:01:49.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14434]: starting fuse 2021-11-21T22:01:49.764 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:01:49.764 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:01:49.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:49.765 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:01:49.766 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:01:49.766 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:01:49.767 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:01:49.767 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:01:49.768 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:49.768 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:01:49.768 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:01:49.769 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:01:49.769 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:49.770 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:01:49.770 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:01:49.771 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:01:49.771 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:01:49.772 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:49.772 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:01:49.772 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:01:49.773 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:01:49.773 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:01:49.774 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:01:49.774 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:01:49.775 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:01:49.775 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:01:49.829 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14434.asok 2021-11-21T22:01:49.832 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14434.asok 2021-11-21T22:01:49.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:49.833 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14434.asok status 2021-11-21T22:01:49.881 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:01:49.882 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:01:49.882 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:01:49.882 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:01:49.883 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:01:49.883 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:01:49.883 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:01:49.883 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14434", 2021-11-21T22:01:49.884 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:01:49.884 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:49.884 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:01:49.884 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:01:49.884 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 4829, 2021-11-21T22:01:49.885 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:01:49.885 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:01:49.885 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:01:49.885 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 4829 2021-11-21T22:01:49.886 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:49.886 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:01:49.886 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:01:49.886 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:01:49.887 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 59927889 2021-11-21T22:01:49.887 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:01:49.887 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:49.887 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:01:49.888 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:01:49.888 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:01:49.888 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 59927889 2021-11-21T22:01:49.888 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:01:49.888 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.4829 172.21.2.136:0/59927889", 2021-11-21T22:01:49.889 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/59927889", 2021-11-21T22:01:49.890 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:01:49.890 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 49, 2021-11-21T22:01:49.890 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 111, 2021-11-21T22:01:49.890 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:01:49.891 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:01:49.891 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:01:49.891 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:01:49.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:49.893 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:49.893 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:49.899 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:01:49.900 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:49.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:49.901 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:49.953 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2021-11-21T22:01:50.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 Processor -- start 2021-11-21T22:01:50.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 -- start start 2021-11-21T22:01:50.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a6aa0 0x7f49ec14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f49ec0a86b0 0x7f49ec1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49ec1475d0 0x7f49ec1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ec0a8bc0 con 0x7f49ec1475d0 2021-11-21T22:01:50.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ec0a6e80 con 0x7f49ec0a86b0 2021-11-21T22:01:50.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f2dc0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec14a7b0 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49ec1475d0 0x7f49ec1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49ec1475d0 0x7f49ec1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51674/0 (socket says 172.21.2.116:51674) 2021-11-21T22:01:50.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 -- 172.21.2.116:0/2054815261 learned_addr learned my addr 172.21.2.116:0/2054815261 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:50.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49ebfff700 1 --2- 172.21.2.116:0/2054815261 >> v2:172.21.2.117:3300/0 conn(0x7f49ec0a86b0 0x7f49ec1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f0b5c700 1 --2- 172.21.2.116:0/2054815261 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a6aa0 0x7f49ec14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 -- 172.21.2.116:0/2054815261 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a6aa0 msgr2=0x7f49ec14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:50.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 --2- 172.21.2.116:0/2054815261 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a6aa0 0x7f49ec14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 -- 172.21.2.116:0/2054815261 >> v2:172.21.2.117:3300/0 conn(0x7f49ec0a86b0 msgr2=0x7f49ec1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:50.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 --2- 172.21.2.116:0/2054815261 >> v2:172.21.2.117:3300/0 conn(0x7f49ec0a86b0 0x7f49ec1470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 -- 172.21.2.116:0/2054815261 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec14aa10 con 0x7f49ec1475d0 2021-11-21T22:01:50.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.002+0000 7f49f135d700 1 --2- 172.21.2.116:0/2054815261 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec1475d0 0x7f49ec1443f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f49e40032f0 tx=0x7f49e4009290).ready entity=mon.0 client_cookie=920eb4023e5fb196 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49eb7fe700 1 -- 172.21.2.116:0/2054815261 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49e400b870 con 0x7f49ec1475d0 2021-11-21T22:01:50.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49eb7fe700 1 -- 172.21.2.116:0/2054815261 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e400b9d0 con 0x7f49ec1475d0 2021-11-21T22:01:50.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49eb7fe700 1 -- 172.21.2.116:0/2054815261 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49e400bb40 con 0x7f49ec1475d0 2021-11-21T22:01:50.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/2054815261 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec1475d0 msgr2=0x7f49ec1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/2054815261 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec1475d0 0x7f49ec1443f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f49e40032f0 tx=0x7f49e4009290).stop 2021-11-21T22:01:50.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/2054815261 shutdown_connections 2021-11-21T22:01:50.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/2054815261 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec1475d0 0x7f49ec1443f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/2054815261 >> v2:172.21.2.117:3300/0 conn(0x7f49ec0a86b0 0x7f49ec1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/2054815261 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a6aa0 0x7f49ec14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/2054815261 >> 172.21.2.116:0/2054815261 conn(0x7f49ec05bd20 msgr2=0x7f49ec05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:50.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/2054815261 shutdown_connections 2021-11-21T22:01:50.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/2054815261 wait complete. 2021-11-21T22:01:50.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 Processor -- start 2021-11-21T22:01:50.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- start start 2021-11-21T22:01:50.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec0a6aa0 0x7f49ec1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a86b0 0x7f49ec065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49ec1475d0 0x7f49ec065fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec144a20 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec14b6b0 con 0x7f49ec1475d0 2021-11-21T22:01:50.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec14a1e0 con 0x7f49ec0a86b0 2021-11-21T22:01:50.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec0a6aa0 0x7f49ec1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec0a6aa0 0x7f49ec1494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51684/0 (socket says 172.21.2.116:51684) 2021-11-21T22:01:50.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 -- 172.21.2.116:0/3877377002 learned_addr learned my addr 172.21.2.116:0/3877377002 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:50.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49ebfff700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a86b0 0x7f49ec065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f135d700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49ec1475d0 0x7f49ec065fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 -- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a86b0 msgr2=0x7f49ec065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a86b0 0x7f49ec065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 -- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49ec1475d0 msgr2=0x7f49ec065fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49ec1475d0 0x7f49ec065fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 -- 172.21.2.116:0/3877377002 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4009530 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f0b5c700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec0a6aa0 0x7f49ec1494f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f49dc00a000 tx=0x7f49dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49e9ffb700 1 -- 172.21.2.116:0/3877377002 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49dc011780 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49e9ffb700 1 -- 172.21.2.116:0/3877377002 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc0118e0 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49e9ffb700 1 -- 172.21.2.116:0/3877377002 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49dc011c40 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec146b50 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec066b30 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec14a3e0 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49e9ffb700 1 -- 172.21.2.116:0/3877377002 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f49dc01c460 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49e9ffb700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49cc053540 0x7f49cc055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.006+0000 7f49e9ffb700 1 -- 172.21.2.116:0/3877377002 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f49dc06bdc0 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.010+0000 7f49ebfff700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49cc053540 0x7f49cc055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.010+0000 7f49ebfff700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49cc053540 0x7f49cc055a00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f49e000c000 tx=0x7f49e0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.010+0000 7f49e9ffb700 1 -- 172.21.2.116:0/3877377002 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49dc042db0 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.122+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f49ec0573d0 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49e9ffb700 1 -- 172.21.2.116:0/3877377002 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1938 (secure 0 0 0) 0x7f49dc0463d0 con 0x7f49ec0a6aa0 2021-11-21T22:01:50.129 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:50.129 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:50.129 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:01:50.130 INFO:teuthology.orchestra.run.gibba016.stdout: "total_bytes": 721202184192, 2021-11-21T22:01:50.130 INFO:teuthology.orchestra.run.gibba016.stdout: "total_avail_bytes": 714957012992, 2021-11-21T22:01:50.130 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_bytes": 6244515840, 2021-11-21T22:01:50.130 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_bytes": 6245171200, 2021-11-21T22:01:50.131 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_ratio": 0.0086593907326459885, 2021-11-21T22:01:50.131 INFO:teuthology.orchestra.run.gibba016.stdout: "num_osds": 8, 2021-11-21T22:01:50.131 INFO:teuthology.orchestra.run.gibba016.stdout: "num_per_pool_osds": 0, 2021-11-21T22:01:50.132 INFO:teuthology.orchestra.run.gibba016.stdout: "num_per_pool_omap_osds": 0 2021-11-21T22:01:50.132 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.132 INFO:teuthology.orchestra.run.gibba016.stdout: "stats_by_class": { 2021-11-21T22:01:50.132 INFO:teuthology.orchestra.run.gibba016.stdout: "ssd": { 2021-11-21T22:01:50.133 INFO:teuthology.orchestra.run.gibba016.stdout: "total_bytes": 721202184192, 2021-11-21T22:01:50.133 INFO:teuthology.orchestra.run.gibba016.stdout: "total_avail_bytes": 714957012992, 2021-11-21T22:01:50.133 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_bytes": 6244515840, 2021-11-21T22:01:50.133 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_bytes": 6245171200, 2021-11-21T22:01:50.134 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_ratio": 0.0086593907326459885 2021-11-21T22:01:50.134 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:50.134 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.134 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:01:50.135 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:50.135 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "device_health_metrics", 2021-11-21T22:01:50.135 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 1, 2021-11-21T22:01:50.135 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:01:50.136 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:01:50.136 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 2, 2021-11-21T22:01:50.136 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:01:50.136 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:01:50.137 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:01:50.137 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447611392 2021-11-21T22:01:50.137 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:50.137 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.137 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:50.138 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "rbd", 2021-11-21T22:01:50.138 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 2, 2021-11-21T22:01:50.138 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:01:50.139 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:01:50.139 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 0, 2021-11-21T22:01:50.139 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:01:50.140 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:01:50.140 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:01:50.140 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447611392 2021-11-21T22:01:50.141 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:50.141 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.141 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:50.142 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "cephfs_metadata", 2021-11-21T22:01:50.142 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 19, 2021-11-21T22:01:50.142 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:01:50.142 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 2740, 2021-11-21T22:01:50.143 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 21, 2021-11-21T22:01:50.143 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 3, 2021-11-21T22:01:50.143 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 2740, 2021-11-21T22:01:50.143 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 4.03596889242408e-09, 2021-11-21T22:01:50.143 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447611392 2021-11-21T22:01:50.144 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:50.144 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.144 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:50.144 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "cephfs_data", 2021-11-21T22:01:50.145 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 20, 2021-11-21T22:01:50.145 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:01:50.145 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:01:50.145 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 0, 2021-11-21T22:01:50.145 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:01:50.146 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:01:50.146 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:01:50.146 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447611392 2021-11-21T22:01:50.146 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:50.147 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:50.147 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:50.147 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:50.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49cc053540 msgr2=0x7f49cc055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49cc053540 0x7f49cc055a00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f49e000c000 tx=0x7f49e0008040).stop 2021-11-21T22:01:50.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec0a6aa0 msgr2=0x7f49ec1494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec0a6aa0 0x7f49ec1494f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f49dc00a000 tx=0x7f49dc008040).stop 2021-11-21T22:01:50.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 shutdown_connections 2021-11-21T22:01:50.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49ec1475d0 0x7f49ec065fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49ec0a86b0 0x7f49ec065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49cc053540 0x7f49cc055a00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 --2- 172.21.2.116:0/3877377002 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49ec0a6aa0 0x7f49ec1494f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 >> 172.21.2.116:0/3877377002 conn(0x7f49ec05bd20 msgr2=0x7f49ec14c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:50.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 shutdown_connections 2021-11-21T22:01:50.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.126+0000 7f49f2dc0700 1 -- 172.21.2.116:0/3877377002 wait complete. 2021-11-21T22:01:50.153 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data size -f json-pretty 2021-11-21T22:01:50.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 Processor -- start 2021-11-21T22:01:50.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- start start 2021-11-21T22:01:50.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640b1270 0x7fb0640b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb0640ad580 0x7fb0640ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0640b0070 con 0x7fb0640b6770 2021-11-21T22:01:50.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0640b05c0 con 0x7fb0640ad580 2021-11-21T22:01:50.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640b2230 con 0x7fb0640b1270 2021-11-21T22:01:50.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb0640ad580 0x7fb0640ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb0640ad580 0x7fb0640ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51688/0 (socket says 172.21.2.116:51688) 2021-11-21T22:01:50.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 -- 172.21.2.116:0/3478706541 learned_addr learned my addr 172.21.2.116:0/3478706541 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:50.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 -- 172.21.2.116:0/3478706541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640b1270 msgr2=0x7fb0640b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:50.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 --2- 172.21.2.116:0/3478706541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640b1270 0x7fb0640b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 -- 172.21.2.116:0/3478706541 >> v1:172.21.2.117:6789/0 conn(0x7fb0640b6770 legacy=0x7fb0641394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:50.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 -- 172.21.2.116:0/3478706541 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640b24d0 con 0x7fb0640ad580 2021-11-21T22:01:50.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0690a3700 1 --2- 172.21.2.116:0/3478706541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640ad580 0x7fb0640ad9d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fb064060900 tx=0x7fb05c00b530).ready entity=mon.0 client_cookie=e6ceaf647b82df70 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0637fe700 1 -- 172.21.2.116:0/3478706541 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05c00be70 con 0x7fb0640ad580 2021-11-21T22:01:50.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0637fe700 1 -- 172.21.2.116:0/3478706541 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c010470 con 0x7fb0640ad580 2021-11-21T22:01:50.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb0637fe700 1 -- 172.21.2.116:0/3478706541 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05c010810 con 0x7fb0640ad580 2021-11-21T22:01:50.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- 172.21.2.116:0/3478706541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640ad580 msgr2=0x7fb0640ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 --2- 172.21.2.116:0/3478706541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640ad580 0x7fb0640ad9d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fb064060900 tx=0x7fb05c00b530).stop 2021-11-21T22:01:50.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- 172.21.2.116:0/3478706541 shutdown_connections 2021-11-21T22:01:50.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 --2- 172.21.2.116:0/3478706541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640ad580 0x7fb0640ad9d0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 --2- 172.21.2.116:0/3478706541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640b1270 0x7fb0640b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- 172.21.2.116:0/3478706541 >> 172.21.2.116:0/3478706541 conn(0x7fb064062040 msgr2=0x7fb0640b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:50.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- 172.21.2.116:0/3478706541 shutdown_connections 2021-11-21T22:01:50.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.202+0000 7fb06ab06700 1 -- 172.21.2.116:0/3478706541 wait complete. 2021-11-21T22:01:50.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 Processor -- start 2021-11-21T22:01:50.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 -- start start 2021-11-21T22:01:50.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640ad580 0x7fb06413ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0640b1270 0x7fb064151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640b6770 0x7fb06414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064064910 con 0x7fb0640b6770 2021-11-21T22:01:50.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064064b00 con 0x7fb0640b1270 2021-11-21T22:01:50.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb064064da0 con 0x7fb0640ad580 2021-11-21T22:01:50.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640b6770 0x7fb06414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640b6770 0x7fb06414dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51694/0 (socket says 172.21.2.116:51694) 2021-11-21T22:01:50.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 -- 172.21.2.116:0/2482061719 learned_addr learned my addr 172.21.2.116:0/2482061719 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:50.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 -- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640ad580 msgr2=0x7fb06413ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640ad580 0x7fb06413ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 -- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0640b1270 msgr2=0x7fb064151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0640b1270 0x7fb064151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 -- 172.21.2.116:0/2482061719 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c00b7d0 con 0x7fb0640b6770 2021-11-21T22:01:50.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0690a3700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640b6770 0x7fb06414dd70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fb064060900 tx=0x7fb06413f7c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb061ffb700 1 -- 172.21.2.116:0/2482061719 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05c01d070 con 0x7fb0640b6770 2021-11-21T22:01:50.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb061ffb700 1 -- 172.21.2.116:0/2482061719 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c0188d0 con 0x7fb0640b6770 2021-11-21T22:01:50.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb064065040 con 0x7fb0640b6770 2021-11-21T22:01:50.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06413cc10 con 0x7fb0640b6770 2021-11-21T22:01:50.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb061ffb700 1 -- 172.21.2.116:0/2482061719 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05c018c30 con 0x7fb0640b6770 2021-11-21T22:01:50.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0640573d0 con 0x7fb0640b6770 2021-11-21T22:01:50.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb061ffb700 1 -- 172.21.2.116:0/2482061719 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb05c029070 con 0x7fb0640b6770 2021-11-21T22:01:50.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb061ffb700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb048053610 0x7fb048055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb061ffb700 1 -- 172.21.2.116:0/2482061719 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb05c06d290 con 0x7fb0640b6770 2021-11-21T22:01:50.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.206+0000 7fb0688a2700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb048053610 0x7fb048055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.210+0000 7fb0688a2700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb048053610 0x7fb048055ad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fb054002f10 tx=0x7fb054006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.210+0000 7fb061ffb700 1 -- 172.21.2.116:0/2482061719 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb05c044570 con 0x7fb0640b6770 2021-11-21T22:01:50.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7fb064068320 con 0x7fb0640b6770 2021-11-21T22:01:50.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb061ffb700 1 -- 172.21.2.116:0/2482061719 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v111) v1 ==== 123+0+64 (secure 0 0 0) 0x7fb05c047a10 con 0x7fb0640b6770 2021-11-21T22:01:50.298 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:50.299 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:50.299 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": "cephfs_data", 2021-11-21T22:01:50.299 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_id": 20, 2021-11-21T22:01:50.300 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2 2021-11-21T22:01:50.300 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:50.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb048053610 msgr2=0x7fb048055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb048053610 0x7fb048055ad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fb054002f10 tx=0x7fb054006930).stop 2021-11-21T22:01:50.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640b6770 msgr2=0x7fb06414dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640b6770 0x7fb06414dd70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fb064060900 tx=0x7fb06413f7c0).stop 2021-11-21T22:01:50.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 shutdown_connections 2021-11-21T22:01:50.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb048053610 0x7fb048055ad0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb0640b6770 0x7fb06414dd70 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb0640b1270 0x7fb064151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 --2- 172.21.2.116:0/2482061719 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb0640ad580 0x7fb06413ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 >> 172.21.2.116:0/2482061719 conn(0x7fb064062040 msgr2=0x7fb064138de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:50.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 shutdown_connections 2021-11-21T22:01:50.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.294+0000 7fb06ab06700 1 -- 172.21.2.116:0/2482061719 wait complete. 2021-11-21T22:01:50.305 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-11-21T22:01:50.306 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2021-11-21T22:01:50.346 INFO:teuthology.orchestra.run.gibba019.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2021-11-21T22:01:50.347 INFO:teuthology.orchestra.run.gibba019.stdout:ceph-fuse 331489280 0 331489280 0% /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:50.348 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:01:50.349 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:50.358 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:50.371 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:50.400 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:50.414 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:01:50.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:50.415 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:50.416 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:50.452 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:01:50.453 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:50.453 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:01:50.454 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:01:50.454 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:50.468 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14960]: fuse finished with error 0 and tester_r 0 2021-11-21T22:01:50.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:50.470 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:50.470 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:50.475 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:01:50.475 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:01:50.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:01:50.565 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:50.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:50.575 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:50.575 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:50.587 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:01:50.588 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:50.589 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:01:50.589 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:01:50.589 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:50.645 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14434]: fuse finished with error 0 and tester_r 0 2021-11-21T22:01:50.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:50.647 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:50.647 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:50.652 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:01:50.652 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:01:50.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:01:50.761 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:50.813 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:01:50.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 Processor -- start 2021-11-21T22:01:50.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- start start 2021-11-21T22:01:50.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3a480a6aa0 0x7f3a4814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3a480a86b0 0x7f3a481470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a480a6e80 con 0x7f3a481475d0 2021-11-21T22:01:50.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a480a8bc0 con 0x7f3a480a6aa0 2021-11-21T22:01:50.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4814a7b0 con 0x7f3a480a86b0 2021-11-21T22:01:50.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3a480a6aa0 0x7f3a4814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3a480a6aa0 0x7f3a4814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51702/0 (socket says 172.21.2.116:51702) 2021-11-21T22:01:50.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 -- 172.21.2.116:0/2499836767 learned_addr learned my addr 172.21.2.116:0/2499836767 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:50.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 -- 172.21.2.116:0/2499836767 >> v2:172.21.2.117:3300/0 conn(0x7f3a480a86b0 msgr2=0x7f3a481470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:50.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 --2- 172.21.2.116:0/2499836767 >> v2:172.21.2.117:3300/0 conn(0x7f3a480a86b0 0x7f3a481470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 -- 172.21.2.116:0/2499836767 >> v1:172.21.2.117:6789/0 conn(0x7f3a481475d0 legacy=0x7f3a481443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:50.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 -- 172.21.2.116:0/2499836767 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4814aa10 con 0x7f3a480a6aa0 2021-11-21T22:01:50.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4e1a1700 1 --2- 172.21.2.116:0/2499836767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a480a6aa0 0x7f3a4814b150 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f3a48061ab0 tx=0x7f3a3c006eb0).ready entity=mon.0 client_cookie=b085be1476f031af server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4d19f700 1 -- 172.21.2.116:0/2499836767 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c00d5c0 con 0x7f3a480a6aa0 2021-11-21T22:01:50.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4d19f700 1 -- 172.21.2.116:0/2499836767 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c000e90 con 0x7f3a480a6aa0 2021-11-21T22:01:50.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a4d19f700 1 -- 172.21.2.116:0/2499836767 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c00d910 con 0x7f3a480a6aa0 2021-11-21T22:01:50.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- 172.21.2.116:0/2499836767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a480a6aa0 msgr2=0x7f3a4814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:50.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 --2- 172.21.2.116:0/2499836767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a480a6aa0 0x7f3a4814b150 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f3a48061ab0 tx=0x7f3a3c006eb0).stop 2021-11-21T22:01:50.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- 172.21.2.116:0/2499836767 shutdown_connections 2021-11-21T22:01:50.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 --2- 172.21.2.116:0/2499836767 >> v2:172.21.2.117:3300/0 conn(0x7f3a480a86b0 0x7f3a481470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 --2- 172.21.2.116:0/2499836767 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a480a6aa0 0x7f3a4814b150 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:50.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- 172.21.2.116:0/2499836767 >> 172.21.2.116:0/2499836767 conn(0x7f3a4805bd20 msgr2=0x7f3a4805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:50.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- 172.21.2.116:0/2499836767 shutdown_connections 2021-11-21T22:01:50.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.866+0000 7f3a50405700 1 -- 172.21.2.116:0/2499836767 wait complete. 2021-11-21T22:01:50.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 Processor -- start 2021-11-21T22:01:50.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 -- start start 2021-11-21T22:01:50.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3a480a6aa0 0x7f3a4815a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3a480a86b0 0x7f3a48149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a481475d0 0x7f3a480a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480a9510 con 0x7f3a481475d0 2021-11-21T22:01:50.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a48159a40 con 0x7f3a480a86b0 2021-11-21T22:01:50.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a48159ce0 con 0x7f3a480a6aa0 2021-11-21T22:01:50.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a481475d0 0x7f3a480a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a481475d0 0x7f3a480a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51710/0 (socket says 172.21.2.116:51710) 2021-11-21T22:01:50.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 -- 172.21.2.116:0/2690398790 learned_addr learned my addr 172.21.2.116:0/2690398790 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:50.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 -- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3a480a6aa0 msgr2=0x7f3a4815a420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:50.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3a480a6aa0 0x7f3a4815a420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 -- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3a480a86b0 msgr2=0x7f3a48149dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:50.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3a480a86b0 0x7f3a48149dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:50.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 -- 172.21.2.116:0/2690398790 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c00b1e0 con 0x7f3a481475d0 2021-11-21T22:01:50.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e9a2700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a481475d0 0x7f3a480a9020 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f3a40003270 tx=0x7f3a400090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a3b7fe700 1 -- 172.21.2.116:0/2690398790 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4000d4a0 con 0x7f3a481475d0 2021-11-21T22:01:50.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a3b7fe700 1 -- 172.21.2.116:0/2690398790 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4000d600 con 0x7f3a481475d0 2021-11-21T22:01:50.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a3b7fe700 1 -- 172.21.2.116:0/2690398790 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4000d960 con 0x7f3a481475d0 2021-11-21T22:01:50.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a48159f80 con 0x7f3a481475d0 2021-11-21T22:01:50.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a48065130 con 0x7f3a481475d0 2021-11-21T22:01:50.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4815a110 con 0x7f3a481475d0 2021-11-21T22:01:50.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a3b7fe700 1 -- 172.21.2.116:0/2690398790 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3a40018030 con 0x7f3a481475d0 2021-11-21T22:01:50.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a3b7fe700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3a28053610 0x7f3a28055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:50.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a3b7fe700 1 -- 172.21.2.116:0/2690398790 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3a4006b2c0 con 0x7f3a481475d0 2021-11-21T22:01:50.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.870+0000 7f3a4e1a1700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3a28053610 0x7f3a28055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:50.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.874+0000 7f3a4e1a1700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3a28053610 0x7f3a28055ad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f3a48061ab0 tx=0x7f3a3c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:50.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.874+0000 7f3a3b7fe700 1 -- 172.21.2.116:0/2690398790 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a400423c0 con 0x7f3a481475d0 2021-11-21T22:01:50.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3a480573d0 con 0x7f3a481475d0 2021-11-21T22:01:50.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a3b7fe700 1 -- 172.21.2.116:0/2690398790 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) v1 ==== 81+0+33659 (secure 0 0 0) 0x7f3a40028390 con 0x7f3a481475d0 2021-11-21T22:01:50.962 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:50.962 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:50.962 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 111, 2021-11-21T22:01:50.963 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:01:50.963 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:01:50.963 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:01:44.231112+0000", 2021-11-21T22:01:50.963 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:50.964 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:50.964 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:01:50.964 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:01:50.965 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:01:50.965 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:01:50.965 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:01:50.965 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:01:50.966 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:01:50.966 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:50.966 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:01:50.966 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:01:50.967 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:01:50.967 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:01:50.967 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:01:50.967 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 20, 2021-11-21T22:01:50.968 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:01:50.968 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:01:50.968 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:01:50.968 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:01:50.969 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:01:50.969 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:50.969 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:01:50.969 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:01:50.970 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:01:50.970 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:50.971 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:50.971 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:50.971 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:50.971 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:50.972 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:50.972 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:50.972 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:50.972 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:01:50.972 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:01:50.973 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:01:50.973 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:01:50.973 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:01:50.973 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:50.974 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:50.974 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:50.974 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:50.974 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:50.975 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:50.975 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:50.976 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.976 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:01:50.977 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:50.977 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:50.977 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:50.977 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:50.978 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:50.978 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:50.978 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:50.978 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:50.979 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:50.979 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:50.979 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:50.979 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:50.980 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:50.980 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:50.981 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:50.981 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:50.982 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:50.982 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:50.983 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:50.983 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:50.983 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:50.984 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:50.984 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:50.985 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:50.985 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:50.985 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:50.986 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.986 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:50.987 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:50.987 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:50.988 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:50.988 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:50.988 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:50.989 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:50.989 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:50.990 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:50.990 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:50.990 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:50.991 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:50.991 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:01:50.992 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.992 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:50.993 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:01:50.993 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:50.993 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:50.994 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:50.994 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:01:50.995 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:01:50.995 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:01:50.996 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:50.996 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:50.996 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:50.997 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:50.997 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:50.998 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:50.998 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:50.999 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:50.999 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:50.999 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:51.000 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:51.000 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:51.001 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:51.002 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:51.002 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:51.002 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:51.003 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:51.003 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:51.003 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:51.003 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:51.004 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.004 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:01:51.004 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:51.004 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:51.005 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:51.005 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:51.005 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:51.005 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:51.005 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:51.006 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:51.006 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:51.006 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:51.006 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:51.007 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:51.007 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:51.007 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:51.007 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:51.008 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:51.008 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:51.008 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:51.008 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:51.009 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:51.009 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:51.009 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:51.009 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:51.009 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:51.010 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:51.010 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:51.010 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.010 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:51.011 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:51.011 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:51.011 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:51.011 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:51.011 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:51.012 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:51.012 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:51.012 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:51.012 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:51.013 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:51.013 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:51.013 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:51.013 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:01:51.013 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.014 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.014 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.014 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 19, 2021-11-21T22:01:51.014 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:01:51.015 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:01:41.057032+0000", 2021-11-21T22:01:51.015 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:51.015 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:51.015 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:51.016 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:51.016 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:51.016 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:51.016 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:51.017 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:51.017 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:51.017 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:51.017 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:51.018 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:51.018 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:51.018 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:51.018 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:51.018 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:51.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:51.019 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:51.019 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:51.019 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:51.020 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.020 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "110", 2021-11-21T22:01:51.020 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:51.020 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:51.020 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:51.021 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:51.021 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:51.021 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:51.021 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:51.022 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:51.022 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:51.022 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:51.022 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:51.023 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:51.023 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:51.023 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:51.023 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:51.023 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:51.024 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:51.024 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:51.024 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:51.024 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:51.025 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:51.025 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:51.025 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:51.025 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:51.025 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:51.026 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:51.026 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.026 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:51.026 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:51.027 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:51.027 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:51.027 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:51.027 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:51.027 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:51.028 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:51.028 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:51.028 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:51.028 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:51.029 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:51.029 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:01:51.029 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:01:51.029 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:01:51.030 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.030 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:51.030 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:01:51.030 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:01:51.031 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.031 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.031 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.031 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.031 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 20, 2021-11-21T22:01:51.032 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:01:51.032 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:01:42.188874+0000", 2021-11-21T22:01:51.032 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:51.032 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:51.033 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:51.033 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:51.033 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:51.033 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:51.034 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:51.034 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:51.034 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:51.034 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:51.035 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:51.035 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:51.035 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:51.035 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:51.035 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:51.036 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:51.036 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:51.036 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:51.036 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:51.037 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:51.037 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.037 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "111", 2021-11-21T22:01:51.037 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:51.037 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:51.038 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:51.038 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:51.038 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:51.038 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:51.039 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:51.039 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:51.039 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:51.039 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:51.040 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:51.040 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:51.040 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:51.040 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:51.041 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:51.041 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:51.041 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:51.041 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:51.042 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:51.042 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:51.042 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:51.042 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:51.042 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:51.043 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:51.043 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:51.043 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:51.043 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.043 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:51.044 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:51.044 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:51.044 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:51.044 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:51.045 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:51.045 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:51.045 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:51.045 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:51.046 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:51.046 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:51.046 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:51.046 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:51.047 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:01:51.047 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:01:51.047 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.047 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.048 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.048 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:51.048 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:01:51.048 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.048 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:51.049 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:01:51.049 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.049 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.049 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.050 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.050 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.050 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.050 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.050 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:51.051 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.051 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.051 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.051 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.052 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.052 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.052 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:01:51.052 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.052 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.053 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.053 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.053 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:01:51.053 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.053 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.054 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.054 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.054 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.054 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.055 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.055 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.055 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:01:51.055 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.055 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.056 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.056 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.056 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:01:51.056 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.057 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.057 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.057 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.057 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.057 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.058 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.058 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.058 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:01:51.058 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.058 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.059 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.059 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.059 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:01:51.059 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.059 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.060 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.060 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.060 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.060 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.060 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.061 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.061 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:01:51.061 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.061 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.062 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.062 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.062 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:01:51.062 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:51.062 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.063 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.063 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.063 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:01:51.063 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:01:51.063 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:01:51.064 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:01:51.064 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.064 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.064 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.065 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.065 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.065 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.065 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:51.065 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:01:51.066 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.066 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.066 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.066 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.066 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:01:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:01:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:01:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:01:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:01:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:01:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.078 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:01:51.078 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.078 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.078 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.079 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.079 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:01:51.079 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:51.079 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:01:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:01:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:01:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:01:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:01:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 107, 2021-11-21T22:01:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:01:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:01:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:01:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:01:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.091 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.091 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.091 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.091 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:01:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.093 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.093 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.093 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.093 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:01:51.093 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.094 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.094 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.094 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.094 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:01:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:01:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.098 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:01:51.098 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:01:51.098 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:01:51.098 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:01:51.099 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.099 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.099 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.099 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.100 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.100 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.100 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:51.100 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:01:51.100 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.101 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.101 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.101 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.102 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.102 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.102 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.102 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:51.102 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.103 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.103 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.103 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.103 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.104 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.104 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:01:51.104 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.104 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.105 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.105 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.105 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:01:51.105 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.105 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.106 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.106 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.106 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.106 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.107 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.107 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.107 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:01:51.107 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.108 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.108 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.108 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.108 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:01:51.108 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.109 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.109 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.109 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.109 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.109 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.110 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.110 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.110 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:01:51.110 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.111 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.111 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.111 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.111 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:01:51.112 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.112 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.112 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.112 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.113 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.113 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.113 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.113 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.114 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:01:51.114 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.114 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.114 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.115 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.115 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:01:51.115 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:51.115 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.116 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.116 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.116 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:01:51.116 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:01:51.117 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:01:51.117 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:01:51.117 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.117 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.118 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.118 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.118 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.118 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.119 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:51.119 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:01:51.119 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.119 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.119 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.120 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.120 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.120 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.120 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.121 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 107, 2021-11-21T22:01:51.121 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.121 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.121 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.121 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.122 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.122 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.122 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:01:51.122 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.123 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.123 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.123 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.123 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:01:51.124 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.124 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.124 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.124 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.124 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.125 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.125 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.125 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.125 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:01:51.126 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.126 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.126 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.126 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.127 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:01:51.127 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.127 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.127 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.127 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.128 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.128 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.128 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.128 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.129 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:01:51.129 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.129 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.129 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.129 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.130 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:01:51.130 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.130 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.130 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.131 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.131 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.131 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.131 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.131 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.132 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:01:51.132 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.132 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.133 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.133 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.133 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:01:51.133 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:51.134 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.134 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.134 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.134 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:01:51.135 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:01:51.135 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:01:51.135 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:01:51.135 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.135 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.136 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.136 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.136 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.136 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.137 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:51.137 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:01:51.137 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.137 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.137 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.138 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.138 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.138 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.138 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.139 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 107, 2021-11-21T22:01:51.139 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.139 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.139 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.139 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.140 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.140 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.140 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:01:51.140 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.141 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.141 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.141 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.141 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:01:51.142 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.142 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.142 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.142 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.143 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.143 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.143 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.143 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.143 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:01:51.144 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.144 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.144 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.144 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.145 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:01:51.145 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.145 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.145 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.145 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.146 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.146 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.146 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.146 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.147 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:01:51.147 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.147 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.147 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.148 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.148 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:01:51.148 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.148 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.149 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.149 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.149 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.149 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.149 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.150 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.150 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:01:51.150 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.150 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.151 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.151 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.151 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:01:51.151 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:51.151 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.152 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.152 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.152 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:01:51.152 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:01:51.152 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:01:51.153 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:01:51.153 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.153 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.153 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.153 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.154 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.154 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.154 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:51.154 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:01:51.155 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.155 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.155 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.155 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.155 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.156 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.156 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.156 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:51.156 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.156 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.157 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.157 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.157 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.157 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.157 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:01:51.158 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.158 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.158 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.158 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.158 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:01:51.159 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.159 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.159 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.159 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.160 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.160 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.160 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.160 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.160 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:01:51.161 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.161 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.161 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.161 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.161 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:01:51.162 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.162 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.162 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.162 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.162 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.163 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.163 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.163 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.163 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:01:51.163 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.164 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.164 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.164 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.164 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:01:51.165 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.165 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.165 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.165 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.165 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.166 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.166 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.166 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.166 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:01:51.167 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.167 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.167 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.167 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.167 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:01:51.168 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:51.168 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.168 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.168 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.168 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:01:51.169 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:01:51.169 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:01:51.169 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:01:51.169 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.169 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.170 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.170 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.170 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.170 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.170 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:51.171 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:01:51.171 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:51.171 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:51.171 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:51.172 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:51.172 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:51.172 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:51.172 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:51.172 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:51.173 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:51.173 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:51.173 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:51.173 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.174 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.174 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.174 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:01:51.174 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.175 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.175 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.175 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.175 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:01:51.175 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.176 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.176 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.176 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.176 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:51.177 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.177 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.177 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.177 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:01:51.177 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.178 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.178 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.178 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.178 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:01:51.179 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.179 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.179 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.179 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.179 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:51.180 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.180 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.180 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.180 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:01:51.181 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.181 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.181 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.181 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.181 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:01:51.182 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.182 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.182 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.182 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.183 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:51.183 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:51.183 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.183 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:51.184 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:01:51.184 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.184 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.184 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.184 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:51.185 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:01:51.185 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:51.185 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.185 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.186 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.186 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:01:51.186 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:01:51.186 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:01:51.187 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:01:51.187 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:51.187 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:51.187 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:51.187 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:51.188 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.188 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:51.188 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:01:51.188 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.189 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:51.189 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.189 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.189 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.189 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.190 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.190 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:01:51.190 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.190 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.191 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.191 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:51.191 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.191 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.191 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.192 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.192 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.192 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:01:51.192 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.193 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.193 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.193 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:51.193 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.193 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.194 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.194 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.194 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.194 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:01:51.195 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.195 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.195 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.195 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:51.196 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.196 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.196 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.196 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.196 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.197 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:01:51.197 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.197 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.197 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.198 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:51.198 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.198 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.198 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.198 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.199 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.199 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:01:51.199 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.199 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.200 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.200 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:51.200 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.200 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.201 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.201 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.201 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.201 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:01:51.201 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.202 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.202 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.202 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:51.202 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.203 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.203 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.203 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.203 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.204 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:01:51.204 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.204 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.204 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:51.204 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:51.205 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:51.205 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:51.205 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:51.205 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:51.205 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:51.206 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:01:51.206 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:51.206 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.206 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:51.207 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:01:51.207 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:01:51.207 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:01:51.207 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:01:51.207 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T22:01:51.208 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:01:51.208 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:01:51.208 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:01:51.208 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:01:51.209 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:01:51.209 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:01:51.209 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:01:51.209 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:01:51.209 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:51.210 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:51.210 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:01:51.210 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:01:51.210 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:01:51.211 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:01:51.211 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:01:51.211 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3a28053610 msgr2=0x7f3a28055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3a28053610 0x7f3a28055ad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f3a48061ab0 tx=0x7f3a3c015000).stop 2021-11-21T22:01:51.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a481475d0 msgr2=0x7f3a480a9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a481475d0 0x7f3a480a9020 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f3a40003270 tx=0x7f3a400090f0).stop 2021-11-21T22:01:51.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 shutdown_connections 2021-11-21T22:01:51.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3a28053610 0x7f3a28055ad0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3a481475d0 0x7f3a480a9020 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3a480a86b0 0x7f3a48149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 --2- 172.21.2.116:0/2690398790 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3a480a6aa0 0x7f3a4815a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 >> 172.21.2.116:0/2690398790 conn(0x7f3a4805bd20 msgr2=0x7f3a481441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:51.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 shutdown_connections 2021-11-21T22:01:51.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:50.958+0000 7f3a50405700 1 -- 172.21.2.116:0/2690398790 wait complete. 2021-11-21T22:01:51.216 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:51.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5621722700 1 Processor -- start 2021-11-21T22:01:51.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5621722700 1 -- start start 2021-11-21T22:01:51.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5621722700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5621722700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f561c099650 0x7f561c099b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5621722700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c14a560 con 0x7f561c0b5630 2021-11-21T22:01:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5621722700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c144b40 con 0x7f561c099650 2021-11-21T22:01:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5621722700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c14a800 con 0x7f561c097ec0 2021-11-21T22:01:51.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f561b7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:51.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5620f21700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f561c099650 0x7f561c099b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:51.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f561b7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c14af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:55062/0 (socket says 172.21.2.116:55062) 2021-11-21T22:01:51.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f561b7fe700 1 -- 172.21.2.116:0/2265653369 learned_addr learned my addr 172.21.2.116:0/2265653369 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:51.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5620f21700 1 -- 172.21.2.116:0/2265653369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 msgr2=0x7f561c14af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5620f21700 1 --2- 172.21.2.116:0/2265653369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c14af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5620f21700 1 -- 172.21.2.116:0/2265653369 >> v1:172.21.2.117:6789/0 conn(0x7f561c0b5630 legacy=0x7f561c146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5620f21700 1 -- 172.21.2.116:0/2265653369 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c14aaa0 con 0x7f561c099650 2021-11-21T22:01:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f5620f21700 1 --2- 172.21.2.116:0/2265653369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 0x7f561c099b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f56140032f0 tx=0x7f56140091d0).ready entity=mon.1 client_cookie=52e808ff33ae7259 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:51.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f561affd700 1 -- 172.21.2.116:0/2265653369 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561400b6c0 con 0x7f561c099650 2021-11-21T22:01:51.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f561affd700 1 -- 172.21.2.116:0/2265653369 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561400b820 con 0x7f561c099650 2021-11-21T22:01:51.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.270+0000 7f561affd700 1 -- 172.21.2.116:0/2265653369 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561400b990 con 0x7f561c099650 2021-11-21T22:01:51.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/2265653369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 msgr2=0x7f561c099b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 --2- 172.21.2.116:0/2265653369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 0x7f561c099b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f56140032f0 tx=0x7f56140091d0).stop 2021-11-21T22:01:51.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/2265653369 shutdown_connections 2021-11-21T22:01:51.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 --2- 172.21.2.116:0/2265653369 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 0x7f561c099b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 --2- 172.21.2.116:0/2265653369 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c14af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/2265653369 >> 172.21.2.116:0/2265653369 conn(0x7f561c05c7f0 msgr2=0x7f561c0b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:51.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/2265653369 shutdown_connections 2021-11-21T22:01:51.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/2265653369 wait complete. 2021-11-21T22:01:51.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 Processor -- start 2021-11-21T22:01:51.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- start start 2021-11-21T22:01:51.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 0x7f561c0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f561c0b5630 0x7f561c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c159830 con 0x7f561c0b5630 2021-11-21T22:01:51.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c159ad0 con 0x7f561c099650 2021-11-21T22:01:51.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c159d70 con 0x7f561c097ec0 2021-11-21T22:01:51.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f561c0b5630 0x7f561c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:51.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f561c0b5630 0x7f561c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51720/0 (socket says 172.21.2.116:51720) 2021-11-21T22:01:51.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 -- 172.21.2.116:0/712697380 learned_addr learned my addr 172.21.2.116:0/712697380 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:51.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 -- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 msgr2=0x7f561c148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 -- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 msgr2=0x7f561c0ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 0x7f561c0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 -- 172.21.2.116:0/712697380 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614009470 con 0x7f561c0b5630 2021-11-21T22:01:51.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5620f21700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f561c0b5630 0x7f561c153540 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f56140032f0 tx=0x7f5614016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:51.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f56197fa700 1 -- 172.21.2.116:0/712697380 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561400b990 con 0x7f561c0b5630 2021-11-21T22:01:51.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c15a010 con 0x7f561c0b5630 2021-11-21T22:01:51.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f56197fa700 1 -- 172.21.2.116:0/712697380 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56140107a0 con 0x7f561c0b5630 2021-11-21T22:01:51.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f56197fa700 1 -- 172.21.2.116:0/712697380 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561400b6c0 con 0x7f561c0b5630 2021-11-21T22:01:51.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c066210 con 0x7f561c0b5630 2021-11-21T22:01:51.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c15a1a0 con 0x7f561c0b5630 2021-11-21T22:01:51.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f56197fa700 1 -- 172.21.2.116:0/712697380 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5614027020 con 0x7f561c0b5630 2021-11-21T22:01:51.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f56197fa700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55fc053640 0x7f55fc055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f56197fa700 1 -- 172.21.2.116:0/712697380 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f561406ce80 con 0x7f561c0b5630 2021-11-21T22:01:51.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.274+0000 7f561bfff700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55fc053640 0x7f55fc055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:51.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.278+0000 7f561bfff700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55fc053640 0x7f55fc055b00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f561c060e20 tx=0x7f560c007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:51.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.278+0000 7f56197fa700 1 -- 172.21.2.116:0/712697380 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5614043f40 con 0x7f561c0b5630 2021-11-21T22:01:51.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f561c147520 con 0x7f561c0b5630 2021-11-21T22:01:51.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f56197fa700 1 -- 172.21.2.116:0/712697380 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f56140159d0 con 0x7f561c0b5630 2021-11-21T22:01:51.390 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:51.391 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":49,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:43.226243+0000","modified":"2021-11-21T22:01:44.230974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7356":{"gid":7356,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1808795000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6833","nonce":1808795000},{"type":"v1","addr":"172.21.2.116:6835","nonce":1808795000}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-11-21T22:01:51.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55fc053640 msgr2=0x7f55fc055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55fc053640 0x7f55fc055b00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f561c060e20 tx=0x7f560c007810).stop 2021-11-21T22:01:51.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f561c0b5630 msgr2=0x7f561c153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f561c0b5630 0x7f561c153540 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f56140032f0 tx=0x7f5614016000).stop 2021-11-21T22:01:51.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 shutdown_connections 2021-11-21T22:01:51.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55fc053640 0x7f55fc055b00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f561c0b5630 0x7f561c153540 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f561c099650 0x7f561c0ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 --2- 172.21.2.116:0/712697380 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f561c097ec0 0x7f561c148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 >> 172.21.2.116:0/712697380 conn(0x7f561c05c7f0 msgr2=0x7f561c145090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:51.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 shutdown_connections 2021-11-21T22:01:51.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.386+0000 7f5621722700 1 -- 172.21.2.116:0/712697380 wait complete. 2021-11-21T22:01:51.398 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 49 2021-11-21T22:01:51.400 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:01:51.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 Processor -- start 2021-11-21T22:01:51.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 -- start start 2021-11-21T22:01:51.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe140144810 0x7fe1401478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1400abce0 0x7fe1400b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe1400b1db0 0x7fe1400ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe140147e80 con 0x7fe140144810 2021-11-21T22:01:51.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1400af5a0 con 0x7fe1400b1db0 2021-11-21T22:01:51.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe147265700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1400b1490 con 0x7fe1400abce0 2021-11-21T22:01:51.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe145001700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe140144810 0x7fe1401478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:51.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe145001700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe140144810 0x7fe1401478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51730/0 (socket says 172.21.2.116:51730) 2021-11-21T22:01:51.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.446+0000 7fe145001700 1 -- 172.21.2.116:0/2998724177 learned_addr learned my addr 172.21.2.116:0/2998724177 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:51.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe145001700 1 -- 172.21.2.116:0/2998724177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1400abce0 msgr2=0x7fe1400b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:51.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe145001700 1 --2- 172.21.2.116:0/2998724177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1400abce0 0x7fe1400b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe145001700 1 -- 172.21.2.116:0/2998724177 >> v2:172.21.2.117:3300/0 conn(0x7fe1400b1db0 msgr2=0x7fe1400ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:51.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe145001700 1 --2- 172.21.2.116:0/2998724177 >> v2:172.21.2.117:3300/0 conn(0x7fe1400b1db0 0x7fe1400ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe145001700 1 -- 172.21.2.116:0/2998724177 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1400af060 con 0x7fe140144810 2021-11-21T22:01:51.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe145001700 1 --2- 172.21.2.116:0/2998724177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe1401478c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7fe134002f70 tx=0x7fe134006eb0).ready entity=mon.0 client_cookie=45f4ae536df06cfe server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:51.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe133fff700 1 -- 172.21.2.116:0/2998724177 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13400d570 con 0x7fe140144810 2021-11-21T22:01:51.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe133fff700 1 -- 172.21.2.116:0/2998724177 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe134000e90 con 0x7fe140144810 2021-11-21T22:01:51.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe133fff700 1 -- 172.21.2.116:0/2998724177 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13400d900 con 0x7fe140144810 2021-11-21T22:01:51.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- 172.21.2.116:0/2998724177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 msgr2=0x7fe1401478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 --2- 172.21.2.116:0/2998724177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe1401478c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7fe134002f70 tx=0x7fe134006eb0).stop 2021-11-21T22:01:51.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- 172.21.2.116:0/2998724177 shutdown_connections 2021-11-21T22:01:51.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 --2- 172.21.2.116:0/2998724177 >> v2:172.21.2.117:3300/0 conn(0x7fe1400b1db0 0x7fe1400ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 --2- 172.21.2.116:0/2998724177 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1400abce0 0x7fe1400b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 --2- 172.21.2.116:0/2998724177 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe1401478c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:51.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- 172.21.2.116:0/2998724177 >> 172.21.2.116:0/2998724177 conn(0x7fe14005bd20 msgr2=0x7fe14005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:51.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- 172.21.2.116:0/2998724177 shutdown_connections 2021-11-21T22:01:51.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- 172.21.2.116:0/2998724177 wait complete. 2021-11-21T22:01:51.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 Processor -- start 2021-11-21T22:01:51.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- start start 2021-11-21T22:01:51.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1400abce0 0x7fe14013f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe14013a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe14013a6f0 0x7fe140159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1401541c0 con 0x7fe140144810 2021-11-21T22:01:51.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14013c5a0 con 0x7fe1400abce0 2021-11-21T22:01:51.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14013ba80 con 0x7fe14013a6f0 2021-11-21T22:01:51.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe14013a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:51.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe14013a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51734/0 (socket says 172.21.2.116:51734) 2021-11-21T22:01:51.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 -- 172.21.2.116:0/1743756925 learned_addr learned my addr 172.21.2.116:0/1743756925 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:51.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 -- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe14013a6f0 msgr2=0x7fe140159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe14013a6f0 0x7fe140159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 -- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1400abce0 msgr2=0x7fe14013f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:51.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1400abce0 0x7fe14013f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:51.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 -- 172.21.2.116:0/1743756925 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13400b1e0 con 0x7fe140144810 2021-11-21T22:01:51.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe144800700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe14013a200 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7fe13c002fb0 tx=0x7fe13c006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:51.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe1327fc700 1 -- 172.21.2.116:0/1743756925 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13c013460 con 0x7fe140144810 2021-11-21T22:01:51.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe1327fc700 1 -- 172.21.2.116:0/1743756925 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c0135c0 con 0x7fe140144810 2021-11-21T22:01:51.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe1327fc700 1 -- 172.21.2.116:0/1743756925 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13c013920 con 0x7fe140144810 2021-11-21T22:01:51.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14013c7d0 con 0x7fe140144810 2021-11-21T22:01:51.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.450+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14013bf80 con 0x7fe140144810 2021-11-21T22:01:51.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.454+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1400af300 con 0x7fe140144810 2021-11-21T22:01:51.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.454+0000 7fe1327fc700 1 -- 172.21.2.116:0/1743756925 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe13c00f070 con 0x7fe140144810 2021-11-21T22:01:51.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.454+0000 7fe1327fc700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1200536a0 0x7fe120055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:51.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.454+0000 7fe1327fc700 1 -- 172.21.2.116:0/1743756925 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe13c06b160 con 0x7fe140144810 2021-11-21T22:01:51.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.454+0000 7fe145001700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1200536a0 0x7fe120055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:51.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.454+0000 7fe145001700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1200536a0 0x7fe120055b60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fe134002f70 tx=0x7fe134015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:51.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.454+0000 7fe1327fc700 1 -- 172.21.2.116:0/1743756925 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe13c0421b0 con 0x7fe140144810 2021-11-21T22:01:51.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:51.562+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe1400573d0 con 0x7fe140144810 2021-11-21T22:01:52.086 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:01:52.082+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:01:52.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.094+0000 7fe1327fc700 1 -- 172.21.2.116:0/1743756925 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe13c018030 con 0x7fe140144810 2021-11-21T22:01:52.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1200536a0 msgr2=0x7fe120055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1200536a0 0x7fe120055b60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fe134002f70 tx=0x7fe134015000).stop 2021-11-21T22:01:52.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 msgr2=0x7fe14013a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe14013a200 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7fe13c002fb0 tx=0x7fe13c006de0).stop 2021-11-21T22:01:52.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 shutdown_connections 2021-11-21T22:01:52.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe14013a6f0 0x7fe140159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1200536a0 0x7fe120055b60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe140144810 0x7fe14013a200 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 --2- 172.21.2.116:0/1743756925 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1400abce0 0x7fe14013f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 >> 172.21.2.116:0/1743756925 conn(0x7fe14005bd20 msgr2=0x7fe14005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:52.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 shutdown_connections 2021-11-21T22:01:52.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.098+0000 7fe147265700 1 -- 172.21.2.116:0/1743756925 wait complete. 2021-11-21T22:01:52.105 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:01:52.108 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:52.113 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:01:52.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.154+0000 7f4c14505700 1 Processor -- start 2021-11-21T22:01:52.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.154+0000 7f4c14505700 1 -- start start 2021-11-21T22:01:52.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 0x7f4c0c13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c144970 con 0x7f4c0c05cef0 2021-11-21T22:01:52.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c0aa0e0 con 0x7f4c0c05c840 2021-11-21T22:01:52.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c0b1200 con 0x7f4c0c0a9d00 2021-11-21T22:01:52.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f4c0c05cef0 0x7f4c0c14b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35480/0 (socket says 172.21.2.116:35480) 2021-11-21T22:01:52.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 -- 172.21.2.116:0/1604344635 learned_addr learned my addr 172.21.2.116:0/1604344635 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:52.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821886411 0 0) 0x7f4c0c144970 con 0x7f4c0c05cef0 2021-11-21T22:01:52.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf4003260 con 0x7f4c0c05cef0 2021-11-21T22:01:52.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c122a1700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 0x7f4c0c13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3681232073 0 0) 0x7f4bf4003260 con 0x7f4c0c05cef0 2021-11-21T22:01:52.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 msgr2=0x7f4c0c13dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:52.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 0x7f4c0c13dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 >> v1:172.21.2.117:6789/0 conn(0x7f4c0c05c840 legacy=0x7f4c0c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:52.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c0abb80 con 0x7f4c0c05cef0 2021-11-21T22:01:52.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f4c040019e0 con 0x7f4c0c05cef0 2021-11-21T22:01:52.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 >> v1:172.21.2.116:6789/0 conn(0x7f4c0c05cef0 legacy=0x7f4c0c14b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 0x7f4bf40037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bf4003e00 0x7f4bf4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bf4004b10 0x7f4bf4004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf400b930 con 0x7f4bf4003e00 2021-11-21T22:01:52.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf400baa0 con 0x7f4bf4004b10 2021-11-21T22:01:52.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf400bc60 con 0x7f4c0c0a9d00 2021-11-21T22:01:52.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c11aa0700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bf4003e00 0x7f4bf4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c122a1700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 0x7f4bf40037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bf4004b10 0x7f4bf4004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c11aa0700 1 -- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 msgr2=0x7f4bf40037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:52.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c11aa0700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4c0c0a9d00 0x7f4bf40037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c11aa0700 1 -- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bf4004b10 msgr2=0x7f4bf4004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:52.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c11aa0700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bf4004b10 0x7f4bf4004f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c11aa0700 1 -- 172.21.2.116:0/1604344635 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4003260 con 0x7f4bf4003e00 2021-11-21T22:01:52.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4bf4004b10 0x7f4bf4004f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:52.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c11aa0700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bf4003e00 0x7f4bf4004500 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f4bfc001fa0 tx=0x7f4bfc007a00).ready entity=mon.0 client_cookie=f75d8f04613e74ac server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:52.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bfc007070 con 0x7f4bf4003e00 2021-11-21T22:01:52.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc0071d0 con 0x7f4bf4003e00 2021-11-21T22:01:52.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c1129f700 1 -- 172.21.2.116:0/1604344635 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bfc006510 con 0x7f4bf4003e00 2021-11-21T22:01:52.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- 172.21.2.116:0/1604344635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bf4003e00 msgr2=0x7f4bf4004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 --2- 172.21.2.116:0/1604344635 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4bf4003e00 0x7f4bf4004500 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f4bfc001fa0 tx=0x7f4bfc007a00).stop 2021-11-21T22:01:52.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 -- 172.21.2.116:0/1604344635 reap_dead start 2021-11-21T22:01:52.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- 172.21.2.116:0/1604344635 shutdown_connections 2021-11-21T22:01:52.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- 172.21.2.116:0/1604344635 >> 172.21.2.116:0/1604344635 conn(0x7f4c0c05bd20 msgr2=0x7f4c0c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:52.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- 172.21.2.116:0/1604344635 shutdown_connections 2021-11-21T22:01:52.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- 172.21.2.116:0/1604344635 wait complete. 2021-11-21T22:01:52.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 Processor -- start 2021-11-21T22:01:52.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- start start 2021-11-21T22:01:52.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bf4003e00 0x7f4c0c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4c0c145080 0x7f4c0c145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4c0c0b03e0 0x7f4c0c145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c149fd0 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c145f20 con 0x7f4c0c145080 2021-11-21T22:01:52.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c14505700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c155380 con 0x7f4bf4003e00 2021-11-21T22:01:52.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4c0c0b03e0 0x7f4c0c145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4c0c0b03e0 0x7f4c0c145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51770/0 (socket says 172.21.2.116:51770) 2021-11-21T22:01:52.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.158+0000 7f4c12aa2700 1 -- 172.21.2.116:0/3971516647 learned_addr learned my addr 172.21.2.116:0/3971516647 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:52.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c12aa2700 1 -- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bf4003e00 msgr2=0x7f4c0c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c12aa2700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bf4003e00 0x7f4c0c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c12aa2700 1 -- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4c0c145080 msgr2=0x7f4c0c145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c12aa2700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4c0c145080 0x7f4c0c145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c12aa2700 1 -- 172.21.2.116:0/3971516647 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c155620 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c12aa2700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4c0c0b03e0 0x7f4c0c145a30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f4c040016f0 tx=0x7f4c04009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:52.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4bfb7fe700 1 -- 172.21.2.116:0/3971516647 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c04004ab0 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4bfb7fe700 1 -- 172.21.2.116:0/3971516647 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c04004c10 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4bfb7fe700 1 -- 172.21.2.116:0/3971516647 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c04007030 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c1558c0 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c156000 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0c0b0f60 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4bfb7fe700 1 -- 172.21.2.116:0/3971516647 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4c0401e070 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4bfb7fe700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bec053580 0x7f4bec055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4bfb7fe700 1 -- 172.21.2.116:0/3971516647 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f4c0406b090 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c122a1700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bec053580 0x7f4bec055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.162+0000 7f4c122a1700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bec053580 0x7f4bec055a40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f4c0800c000 tx=0x7f4c08008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:52.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.166+0000 7f4bfb7fe700 1 -- 172.21.2.116:0/3971516647 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c04042090 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c0c0573d0 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4bfb7fe700 1 -- 172.21.2.116:0/3971516647 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2130 (secure 0 0 0) 0x7f4c040456b0 con 0x7f4c0c0b03e0 2021-11-21T22:01:52.278 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:52.278 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:01:43.226243+0000","modified":"2021-11-21T22:01:52.086924+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-11-21T22:01:52.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bec053580 msgr2=0x7f4bec055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bec053580 0x7f4bec055a40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f4c0800c000 tx=0x7f4c08008040).stop 2021-11-21T22:01:52.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4c0c0b03e0 msgr2=0x7f4c0c145a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4c0c0b03e0 0x7f4c0c145a30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f4c040016f0 tx=0x7f4c04009040).stop 2021-11-21T22:01:52.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 shutdown_connections 2021-11-21T22:01:52.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4bec053580 0x7f4bec055a40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4c0c0b03e0 0x7f4c0c145a30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4c0c145080 0x7f4c0c145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 --2- 172.21.2.116:0/3971516647 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4bf4003e00 0x7f4c0c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 >> 172.21.2.116:0/3971516647 conn(0x7f4c0c05bd20 msgr2=0x7f4c0c140a60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:52.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 shutdown_connections 2021-11-21T22:01:52.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.274+0000 7f4c14505700 1 -- 172.21.2.116:0/3971516647 wait complete. 2021-11-21T22:01:52.282 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 50 2021-11-21T22:01:52.286 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:01:52.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 Processor -- start 2021-11-21T22:01:52.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- start start 2021-11-21T22:01:52.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4178144810 0x7f41781478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f41780abce0 0x7f41780b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f41780b1db0 0x7f41780ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4178147e80 con 0x7f41780b1db0 2021-11-21T22:01:52.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41780af5a0 con 0x7f41780abce0 2021-11-21T22:01:52.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41780b1490 con 0x7f4178144810 2021-11-21T22:01:52.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f41780b1db0 0x7f41780ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f41780b1db0 0x7f41780ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51774/0 (socket says 172.21.2.116:51774) 2021-11-21T22:01:52.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 -- 172.21.2.116:0/1190675910 learned_addr learned my addr 172.21.2.116:0/1190675910 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:52.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 -- 172.21.2.116:0/1190675910 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4178144810 msgr2=0x7f41781478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:52.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 --2- 172.21.2.116:0/1190675910 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4178144810 0x7f41781478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 -- 172.21.2.116:0/1190675910 >> v2:172.21.2.117:3300/0 conn(0x7f41780abce0 msgr2=0x7f41780b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:52.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 --2- 172.21.2.116:0/1190675910 >> v2:172.21.2.117:3300/0 conn(0x7f41780abce0 0x7f41780b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 -- 172.21.2.116:0/1190675910 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41780af060 con 0x7f41780b1db0 2021-11-21T22:01:52.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417d8f7700 1 --2- 172.21.2.116:0/1190675910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41780b1db0 0x7f41780ac600 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f41700032f0 tx=0x7f4170009290).ready entity=mon.0 client_cookie=2afe7a59a62bd9c5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:52.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f416bfff700 1 -- 172.21.2.116:0/1190675910 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f417000b870 con 0x7f41780b1db0 2021-11-21T22:01:52.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f416bfff700 1 -- 172.21.2.116:0/1190675910 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417000b9d0 con 0x7f41780b1db0 2021-11-21T22:01:52.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f416bfff700 1 -- 172.21.2.116:0/1190675910 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f417000bb80 con 0x7f41780b1db0 2021-11-21T22:01:52.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- 172.21.2.116:0/1190675910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41780b1db0 msgr2=0x7f41780ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 --2- 172.21.2.116:0/1190675910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41780b1db0 0x7f41780ac600 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f41700032f0 tx=0x7f4170009290).stop 2021-11-21T22:01:52.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- 172.21.2.116:0/1190675910 shutdown_connections 2021-11-21T22:01:52.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 --2- 172.21.2.116:0/1190675910 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41780b1db0 0x7f41780ac600 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:52.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 --2- 172.21.2.116:0/1190675910 >> v2:172.21.2.117:3300/0 conn(0x7f41780abce0 0x7f41780b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 --2- 172.21.2.116:0/1190675910 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4178144810 0x7f41781478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- 172.21.2.116:0/1190675910 >> 172.21.2.116:0/1190675910 conn(0x7f417805bd20 msgr2=0x7f417805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:52.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- 172.21.2.116:0/1190675910 shutdown_connections 2021-11-21T22:01:52.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- 172.21.2.116:0/1190675910 wait complete. 2021-11-21T22:01:52.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 Processor -- start 2021-11-21T22:01:52.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.334+0000 7f417f35a700 1 -- start start 2021-11-21T22:01:52.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41780abce0 0x7f4178143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4178144810 0x7f4178156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4178067410 0x7f41781570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41780b3090 con 0x7f4178144810 2021-11-21T22:01:52.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41780b2b50 con 0x7f4178067410 2021-11-21T22:01:52.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41780b2df0 con 0x7f41780abce0 2021-11-21T22:01:52.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4178144810 0x7f4178156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4178144810 0x7f4178156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51780/0 (socket says 172.21.2.116:51780) 2021-11-21T22:01:52.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 -- 172.21.2.116:0/678985219 learned_addr learned my addr 172.21.2.116:0/678985219 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:52.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 -- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41780abce0 msgr2=0x7f4178143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41780abce0 0x7f4178143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 -- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4178067410 msgr2=0x7f41781570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:52.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4178067410 0x7f41781570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:52.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 -- 172.21.2.116:0/678985219 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170009530 con 0x7f4178144810 2021-11-21T22:01:52.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417c8f5700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4178144810 0x7f4178156c00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f416c002fb0 tx=0x7f416c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:52.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f416a7fc700 1 -- 172.21.2.116:0/678985219 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f416c013460 con 0x7f4178144810 2021-11-21T22:01:52.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f416a7fc700 1 -- 172.21.2.116:0/678985219 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c0135c0 con 0x7f4178144810 2021-11-21T22:01:52.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f416a7fc700 1 -- 172.21.2.116:0/678985219 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f416c013920 con 0x7f4178144810 2021-11-21T22:01:52.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4178156760 con 0x7f4178144810 2021-11-21T22:01:52.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4178144d20 con 0x7f4178144810 2021-11-21T22:01:52.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4178146d90 con 0x7f4178144810 2021-11-21T22:01:52.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f416a7fc700 1 -- 172.21.2.116:0/678985219 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f416c00f070 con 0x7f4178144810 2021-11-21T22:01:52.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f416a7fc700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4158053610 0x7f4158055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:52.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f416a7fc700 1 -- 172.21.2.116:0/678985219 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f416c06b0e0 con 0x7f4178144810 2021-11-21T22:01:52.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.338+0000 7f417d0f6700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4158053610 0x7f4158055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:52.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.342+0000 7f417d0f6700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4158053610 0x7f4158055ad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f4174000ed0 tx=0x7f4174006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:52.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.342+0000 7f416a7fc700 1 -- 172.21.2.116:0/678985219 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4178146d90 con 0x7f4178144810 2021-11-21T22:01:52.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:52.450+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4178146d90 con 0x7f4178144810 2021-11-21T22:01:53.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f416a7fc700 1 -- 172.21.2.116:0/678985219 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4178146d90 con 0x7f4178144810 2021-11-21T22:01:53.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4158053610 msgr2=0x7f4158055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:53.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4158053610 0x7f4158055ad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f4174000ed0 tx=0x7f4174006930).stop 2021-11-21T22:01:53.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4178144810 msgr2=0x7f4178156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:53.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4178144810 0x7f4178156c00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f416c002fb0 tx=0x7f416c006df0).stop 2021-11-21T22:01:53.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 shutdown_connections 2021-11-21T22:01:53.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4178067410 0x7f41781570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:53.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4158053610 0x7f4158055ad0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:53.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4178144810 0x7f4178156c00 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:53.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 --2- 172.21.2.116:0/678985219 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41780abce0 0x7f4178143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:53.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 >> 172.21.2.116:0/678985219 conn(0x7f417805bd20 msgr2=0x7f41780b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:53.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 shutdown_connections 2021-11-21T22:01:53.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.110+0000 7f417f35a700 1 -- 172.21.2.116:0/678985219 wait complete. 2021-11-21T22:01:53.121 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:01:53.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.170+0000 7f8d38dd8700 1 Processor -- start 2021-11-21T22:01:53.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- start start 2021-11-21T22:01:53.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8d340b0d00 0x7f8d340b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:53.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8d34062610 0x7f8d340b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:53.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3405cb80 con 0x7f8d341430b0 2021-11-21T22:01:53.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d340b15d0 con 0x7f8d34062610 2021-11-21T22:01:53.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d340a94c0 con 0x7f8d340b0d00 2021-11-21T22:01:53.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8d34062610 0x7f8d340b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:53.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8d34062610 0x7f8d340b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51790/0 (socket says 172.21.2.116:51790) 2021-11-21T22:01:53.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 -- 172.21.2.116:0/4074689379 learned_addr learned my addr 172.21.2.116:0/4074689379 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:53.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 -- 172.21.2.116:0/4074689379 >> v2:172.21.2.117:3300/0 conn(0x7f8d340b0d00 msgr2=0x7f8d340b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:53.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 --2- 172.21.2.116:0/4074689379 >> v2:172.21.2.117:3300/0 conn(0x7f8d340b0d00 0x7f8d340b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:53.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 -- 172.21.2.116:0/4074689379 >> v1:172.21.2.117:6789/0 conn(0x7f8d341430b0 legacy=0x7f8d34143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:53.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 -- 172.21.2.116:0/4074689379 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340af320 con 0x7f8d34062610 2021-11-21T22:01:53.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d31d9b700 1 --2- 172.21.2.116:0/4074689379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d34062610 0x7f8d340b22a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f8d340a8a80 tx=0x7f8d24006eb0).ready entity=mon.0 client_cookie=64711979e8bc38e4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d3159a700 1 -- 172.21.2.116:0/4074689379 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d2400d5c0 con 0x7f8d34062610 2021-11-21T22:01:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d3159a700 1 -- 172.21.2.116:0/4074689379 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d24000e90 con 0x7f8d34062610 2021-11-21T22:01:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d3159a700 1 -- 172.21.2.116:0/4074689379 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d2400d910 con 0x7f8d34062610 2021-11-21T22:01:53.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- 172.21.2.116:0/4074689379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d34062610 msgr2=0x7f8d340b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/4074689379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d34062610 0x7f8d340b22a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f8d340a8a80 tx=0x7f8d24006eb0).stop 2021-11-21T22:01:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- 172.21.2.116:0/4074689379 shutdown_connections 2021-11-21T22:01:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/4074689379 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d34062610 0x7f8d340b22a0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:53.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/4074689379 >> v2:172.21.2.117:3300/0 conn(0x7f8d340b0d00 0x7f8d340b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:53.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- 172.21.2.116:0/4074689379 >> 172.21.2.116:0/4074689379 conn(0x7f8d3405bd20 msgr2=0x7f8d3405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:53.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- 172.21.2.116:0/4074689379 shutdown_connections 2021-11-21T22:01:53.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- 172.21.2.116:0/4074689379 wait complete. 2021-11-21T22:01:53.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 Processor -- start 2021-11-21T22:01:53.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- start start 2021-11-21T22:01:53.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8d34062610 0x7f8d340acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:53.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8d340b0d00 0x7f8d34067190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:53.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d341430b0 0x7f8d34134260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:53.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340a8460 con 0x7f8d341430b0 2021-11-21T22:01:53.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3413c490 con 0x7f8d34062610 2021-11-21T22:01:53.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d38dd8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34065780 con 0x7f8d340b0d00 2021-11-21T22:01:53.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d341430b0 0x7f8d34134260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:53.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d341430b0 0x7f8d34134260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51794/0 (socket says 172.21.2.116:51794) 2021-11-21T22:01:53.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 -- 172.21.2.116:0/3803255410 learned_addr learned my addr 172.21.2.116:0/3803255410 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:53.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 -- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8d340b0d00 msgr2=0x7f8d34067190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:53.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8d340b0d00 0x7f8d34067190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:53.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 -- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8d34062610 msgr2=0x7f8d340acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:53.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8d34062610 0x7f8d340acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:53.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 -- 172.21.2.116:0/3803255410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2400b1e0 con 0x7f8d341430b0 2021-11-21T22:01:53.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d32d9d700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d341430b0 0x7f8d34134260 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f8d2c0012f0 tx=0x7f8d2c0013c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:53.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.174+0000 7f8d2b7fe700 1 -- 172.21.2.116:0/3803255410 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d2c00e4a0 con 0x7f8d341430b0 2021-11-21T22:01:53.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d2b7fe700 1 -- 172.21.2.116:0/3803255410 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00e600 con 0x7f8d341430b0 2021-11-21T22:01:53.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d34065b30 con 0x7f8d341430b0 2021-11-21T22:01:53.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d2b7fe700 1 -- 172.21.2.116:0/3803255410 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d2c00e960 con 0x7f8d341430b0 2021-11-21T22:01:53.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d34139e20 con 0x7f8d341430b0 2021-11-21T22:01:53.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3413ab10 con 0x7f8d341430b0 2021-11-21T22:01:53.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d2b7fe700 1 -- 172.21.2.116:0/3803255410 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8d2c010030 con 0x7f8d341430b0 2021-11-21T22:01:53.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d2b7fe700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8d180536d0 0x7f8d18055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:53.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d2b7fe700 1 -- 172.21.2.116:0/3803255410 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8d2c06bc40 con 0x7f8d341430b0 2021-11-21T22:01:53.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d3259c700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8d180536d0 0x7f8d18055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:53.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d3259c700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8d180536d0 0x7f8d18055b90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f8d1c002f70 tx=0x7f8d1c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:53.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.178+0000 7f8d2b7fe700 1 -- 172.21.2.116:0/3803255410 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d2c042d50 con 0x7f8d341430b0 2021-11-21T22:01:53.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:53.262+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d340af5c0 con 0x7f8d341430b0 2021-11-21T22:01:54.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d2b7fe700 1 -- 172.21.2.116:0/3803255410 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8d2c046370 con 0x7f8d341430b0 2021-11-21T22:01:54.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8d180536d0 msgr2=0x7f8d18055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:54.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8d180536d0 0x7f8d18055b90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f8d1c002f70 tx=0x7f8d1c006a80).stop 2021-11-21T22:01:54.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d341430b0 msgr2=0x7f8d34134260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:54.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d341430b0 0x7f8d34134260 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f8d2c0012f0 tx=0x7f8d2c0013c0).stop 2021-11-21T22:01:54.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 shutdown_connections 2021-11-21T22:01:54.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8d180536d0 0x7f8d18055b90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:54.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8d341430b0 0x7f8d34134260 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:54.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8d340b0d00 0x7f8d34067190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:54.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 --2- 172.21.2.116:0/3803255410 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8d34062610 0x7f8d340acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:54.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 >> 172.21.2.116:0/3803255410 conn(0x7f8d3405bd20 msgr2=0x7f8d340b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:54.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 shutdown_connections 2021-11-21T22:01:54.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.118+0000 7f8d38dd8700 1 -- 172.21.2.116:0/3803255410 wait complete. 2021-11-21T22:01:54.125 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:01:54.129 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:01:54.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.178+0000 7f311ce88700 1 Processor -- start 2021-11-21T22:01:54.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.178+0000 7f311ce88700 1 -- start start 2021-11-21T22:01:54.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 0x7f31180b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:54.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f31180b1660 0x7f31180b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:54.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31180b5a40 con 0x7f31180b5660 2021-11-21T22:01:54.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180aef20 con 0x7f31180b1660 2021-11-21T22:01:54.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180ae540 con 0x7f311813c620 2021-11-21T22:01:54.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f31180b1660 0x7f31180b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:54.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f31180b1660 0x7f31180b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51804/0 (socket says 172.21.2.116:51804) 2021-11-21T22:01:54.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 -- 172.21.2.116:0/1337954502 learned_addr learned my addr 172.21.2.116:0/1337954502 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:54.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311659c700 1 --1- 172.21.2.116:0/1337954502 >> v1:172.21.2.116:6789/0 conn(0x7f31180b5660 0x7f311813bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35526/0 (socket says 172.21.2.116:35526) 2021-11-21T22:01:54.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 -- 172.21.2.116:0/1337954502 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 msgr2=0x7f31180b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:54.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 --2- 172.21.2.116:0/1337954502 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 0x7f31180b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:54.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 -- 172.21.2.116:0/1337954502 >> v1:172.21.2.116:6789/0 conn(0x7f31180b5660 legacy=0x7f311813bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:54.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 -- 172.21.2.116:0/1337954502 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180ae7e0 con 0x7f31180b1660 2021-11-21T22:01:54.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3116d9d700 1 --2- 172.21.2.116:0/1337954502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b1660 0x7f31180b0c10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f31100032f0 tx=0x7f311000acc0).ready entity=mon.0 client_cookie=b8223d96bf6cdcb7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:54.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311559a700 1 -- 172.21.2.116:0/1337954502 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f311000f070 con 0x7f31180b1660 2021-11-21T22:01:54.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311559a700 1 -- 172.21.2.116:0/1337954502 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3110008de0 con 0x7f31180b1660 2021-11-21T22:01:54.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311559a700 1 -- 172.21.2.116:0/1337954502 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f311000c8c0 con 0x7f31180b1660 2021-11-21T22:01:54.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- 172.21.2.116:0/1337954502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b1660 msgr2=0x7f31180b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:54.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- 172.21.2.116:0/1337954502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b1660 0x7f31180b0c10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f31100032f0 tx=0x7f311000acc0).stop 2021-11-21T22:01:54.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- 172.21.2.116:0/1337954502 shutdown_connections 2021-11-21T22:01:54.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- 172.21.2.116:0/1337954502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b1660 0x7f31180b0c10 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:54.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- 172.21.2.116:0/1337954502 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 0x7f31180b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:54.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- 172.21.2.116:0/1337954502 >> 172.21.2.116:0/1337954502 conn(0x7f311805f730 msgr2=0x7f3118063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:54.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- 172.21.2.116:0/1337954502 shutdown_connections 2021-11-21T22:01:54.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- 172.21.2.116:0/1337954502 wait complete. 2021-11-21T22:01:54.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 Processor -- start 2021-11-21T22:01:54.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- start start 2021-11-21T22:01:54.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31180b1660 0x7f3118140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:54.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b5660 0x7f31180aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:54.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 0x7f31180aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:54.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180abef0 con 0x7f31180b5660 2021-11-21T22:01:54.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3118142e00 con 0x7f31180b1660 2021-11-21T22:01:54.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180b38e0 con 0x7f311813c620 2021-11-21T22:01:54.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b5660 0x7f31180aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:54.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b5660 0x7f31180aa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51808/0 (socket says 172.21.2.116:51808) 2021-11-21T22:01:54.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 -- 172.21.2.116:0/117670486 learned_addr learned my addr 172.21.2.116:0/117670486 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:54.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 -- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 msgr2=0x7f31180aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:54.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 0x7f31180aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:54.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 -- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31180b1660 msgr2=0x7f3118140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:54.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31180b1660 0x7f3118140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:54.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 -- 172.21.2.116:0/117670486 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110007040 con 0x7f31180b5660 2021-11-21T22:01:54.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f3115d9b700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b5660 0x7f31180aa510 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f310c002f70 tx=0x7f310c006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:54.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f31077fe700 1 -- 172.21.2.116:0/117670486 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f310c013460 con 0x7f31180b5660 2021-11-21T22:01:54.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f31077fe700 1 -- 172.21.2.116:0/117670486 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c0135c0 con 0x7f31180b5660 2021-11-21T22:01:54.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31180ab1a0 con 0x7f31180b5660 2021-11-21T22:01:54.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f31077fe700 1 -- 172.21.2.116:0/117670486 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f310c013920 con 0x7f31180b5660 2021-11-21T22:01:54.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.182+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31180ab740 con 0x7f31180b5660 2021-11-21T22:01:54.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.186+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3118142fc0 con 0x7f31180b5660 2021-11-21T22:01:54.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.186+0000 7f31077fe700 1 -- 172.21.2.116:0/117670486 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f310c01a060 con 0x7f31180b5660 2021-11-21T22:01:54.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.186+0000 7f31077fe700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30f40536d0 0x7f30f4055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:54.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.186+0000 7f31077fe700 1 -- 172.21.2.116:0/117670486 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f310c06b260 con 0x7f31180b5660 2021-11-21T22:01:54.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.186+0000 7f311659c700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30f40536d0 0x7f30f4055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:54.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.186+0000 7f311659c700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30f40536d0 0x7f30f4055b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f3118063a90 tx=0x7f31080077c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:54.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.186+0000 7f31077fe700 1 -- 172.21.2.116:0/117670486 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f310c042410 con 0x7f31180b5660 2021-11-21T22:01:54.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:54.270+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f31180aea80 con 0x7f31180b5660 2021-11-21T22:01:55.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.126+0000 7f31077fe700 1 -- 172.21.2.116:0/117670486 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) v1 ==== 173+0+0 (secure 0 0 0) 0x7f310c018020 con 0x7f31180b5660 2021-11-21T22:01:55.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30f40536d0 msgr2=0x7f30f4055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30f40536d0 0x7f30f4055b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f3118063a90 tx=0x7f31080077c0).stop 2021-11-21T22:01:55.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b5660 msgr2=0x7f31180aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b5660 0x7f31180aa510 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f310c002f70 tx=0x7f310c006e20).stop 2021-11-21T22:01:55.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 shutdown_connections 2021-11-21T22:01:55.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f311813c620 0x7f31180aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f30f40536d0 0x7f30f4055b90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f31180b5660 0x7f31180aa510 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 --2- 172.21.2.116:0/117670486 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f31180b1660 0x7f3118140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 >> 172.21.2.116:0/117670486 conn(0x7f311805f730 msgr2=0x7f311813f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:55.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 shutdown_connections 2021-11-21T22:01:55.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.130+0000 7f311ce88700 1 -- 172.21.2.116:0/117670486 wait complete. 2021-11-21T22:01:55.141 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:01:55.142 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T22:01:55.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 Processor -- start 2021-11-21T22:01:55.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- start start 2021-11-21T22:01:55.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3f641478c0 0x7f3f640acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f64147e80 con 0x7f3f64144810 2021-11-21T22:01:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f640af5a0 con 0x7f3f640abce0 2021-11-21T22:01:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640b1490 con 0x7f3f641478c0 2021-11-21T22:01:55.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f62d9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f3f64144810 0x7f3f640b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35538/0 (socket says 172.21.2.116:35538) 2021-11-21T22:01:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f62d9d700 1 -- 172.21.2.116:0/1013094433 learned_addr learned my addr 172.21.2.116:0/1013094433 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6359e700 1 --2- 172.21.2.116:0/1013094433 >> v2:172.21.2.117:3300/0 conn(0x7f3f641478c0 0x7f3f640acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500285576 0 0) 0x7f3f64147e80 con 0x7f3f64144810 2021-11-21T22:01:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f44003260 con 0x7f3f64144810 2021-11-21T22:01:55.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218531969 0 0) 0x7f3f44003260 con 0x7f3f64144810 2021-11-21T22:01:55.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 >> v2:172.21.2.117:3300/0 conn(0x7f3f641478c0 msgr2=0x7f3f640acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:55.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 --2- 172.21.2.116:0/1013094433 >> v2:172.21.2.117:3300/0 conn(0x7f3f641478c0 0x7f3f640acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 >> v1:172.21.2.117:6789/0 conn(0x7f3f640abce0 legacy=0x7f3f640ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640b1fd0 con 0x7f3f64144810 2021-11-21T22:01:55.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f3f54001660 con 0x7f3f64144810 2021-11-21T22:01:55.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 >> v1:172.21.2.116:6789/0 conn(0x7f3f64144810 legacy=0x7f3f640b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f44003750 0x7f3f44003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f44004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f44004e70 0x7f3f4400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640b1490 con 0x7f3f44004500 2021-11-21T22:01:55.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4400bd50 con 0x7f3f44004e70 2021-11-21T22:01:55.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4400bf10 con 0x7f3f44003750 2021-11-21T22:01:55.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6259c700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f44004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f62d9d700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f44003750 0x7f3f44003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6359e700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f44004e70 0x7f3f4400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6259c700 1 -- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f44003750 msgr2=0x7f3f44003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:55.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6259c700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f44003750 0x7f3f44003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6259c700 1 -- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f44004e70 msgr2=0x7f3f4400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:55.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6259c700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f44004e70 0x7f3f4400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6259c700 1 -- 172.21.2.116:0/1013094433 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640b1fd0 con 0x7f3f44004500 2021-11-21T22:01:55.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6359e700 1 -- 172.21.2.116:0/1013094433 reap_dead start 2021-11-21T22:01:55.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f62d9d700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f44003750 0x7f3f44003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:01:55.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f6259c700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f44004980 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f3f640b4e00 tx=0x7f3f4c007a00).ready entity=mon.0 client_cookie=4898234a560a6273 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:55.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f4c007d10 con 0x7f3f44004500 2021-11-21T22:01:55.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c007030 con 0x7f3f44004500 2021-11-21T22:01:55.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f61d9b700 1 -- 172.21.2.116:0/1013094433 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f4c007390 con 0x7f3f44004500 2021-11-21T22:01:55.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- 172.21.2.116:0/1013094433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 msgr2=0x7f3f44004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f44004980 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f3f640b4e00 tx=0x7f3f4c007a00).stop 2021-11-21T22:01:55.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- 172.21.2.116:0/1013094433 shutdown_connections 2021-11-21T22:01:55.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 --2- 172.21.2.116:0/1013094433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f44004980 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- 172.21.2.116:0/1013094433 >> 172.21.2.116:0/1013094433 conn(0x7f3f6405bd20 msgr2=0x7f3f6405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:55.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- 172.21.2.116:0/1013094433 shutdown_connections 2021-11-21T22:01:55.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.194+0000 7f3f69259700 1 -- 172.21.2.116:0/1013094433 wait complete. 2021-11-21T22:01:55.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 Processor -- start 2021-11-21T22:01:55.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 -- start start 2021-11-21T22:01:55.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f641457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f64145c90 0x7f3f640b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f640b3500 0x7f3f640b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6413c4b0 con 0x7f3f44004500 2021-11-21T22:01:55.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6413cbd0 con 0x7f3f640b3500 2021-11-21T22:01:55.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6413ce70 con 0x7f3f64145c90 2021-11-21T22:01:55.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f641457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f641457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51826/0 (socket says 172.21.2.116:51826) 2021-11-21T22:01:55.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 -- 172.21.2.116:0/2965855052 learned_addr learned my addr 172.21.2.116:0/2965855052 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:55.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 -- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f64145c90 msgr2=0x7f3f640b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f64145c90 0x7f3f640b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 -- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f640b3500 msgr2=0x7f3f640b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f640b3500 0x7f3f640b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 -- 172.21.2.116:0/2965855052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6413b370 con 0x7f3f44004500 2021-11-21T22:01:55.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f62d9d700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f641457a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f3f5400b000 tx=0x7f3f54001470).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:55.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f5bfff700 1 -- 172.21.2.116:0/2965855052 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f54010710 con 0x7f3f44004500 2021-11-21T22:01:55.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f5bfff700 1 -- 172.21.2.116:0/2965855052 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f54010870 con 0x7f3f44004500 2021-11-21T22:01:55.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f64146b70 con 0x7f3f44004500 2021-11-21T22:01:55.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f64159c80 con 0x7f3f44004500 2021-11-21T22:01:55.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f5bfff700 1 -- 172.21.2.116:0/2965855052 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f54010bd0 con 0x7f3f44004500 2021-11-21T22:01:55.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f5bfff700 1 -- 172.21.2.116:0/2965855052 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3f54010d30 con 0x7f3f44004500 2021-11-21T22:01:55.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c003410 con 0x7f3f44004500 2021-11-21T22:01:55.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f5bfff700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3f48053660 0x7f3f48055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f5bfff700 1 -- 172.21.2.116:0/2965855052 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3f5406bd10 con 0x7f3f44004500 2021-11-21T22:01:55.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.198+0000 7f3f6259c700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3f48053660 0x7f3f48055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.202+0000 7f3f6259c700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3f48053660 0x7f3f48055b20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f3f640b4e00 tx=0x7f3f4c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:55.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.202+0000 7f3f5bfff700 1 -- 172.21.2.116:0/2965855052 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f54043040 con 0x7f3f44004500 2021-11-21T22:01:55.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.286+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3f3c003c40 con 0x7f3f44004500 2021-11-21T22:01:55.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.290+0000 7f3f5bfff700 1 -- 172.21.2.116:0/2965855052 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3f54046660 con 0x7f3f44004500 2021-11-21T22:01:55.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3f48053660 msgr2=0x7f3f48055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3f48053660 0x7f3f48055b20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f3f640b4e00 tx=0x7f3f4c015000).stop 2021-11-21T22:01:55.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 msgr2=0x7f3f641457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f641457a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f3f5400b000 tx=0x7f3f54001470).stop 2021-11-21T22:01:55.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 shutdown_connections 2021-11-21T22:01:55.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3f640b3500 0x7f3f640b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3f64145c90 0x7f3f640b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3f48053660 0x7f3f48055b20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 --2- 172.21.2.116:0/2965855052 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3f44004500 0x7f3f641457a0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 >> 172.21.2.116:0/2965855052 conn(0x7f3f6405bd20 msgr2=0x7f3f641484b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:55.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 shutdown_connections 2021-11-21T22:01:55.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.294+0000 7f3f69259700 1 -- 172.21.2.116:0/2965855052 wait complete. 2021-11-21T22:01:55.303 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2021-11-21T22:01:55.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 Processor -- start 2021-11-21T22:01:55.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 -- start start 2021-11-21T22:01:55.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0f880a86b0 0x7f0f8814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 0x7f0f881443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f880a6e80 con 0x7f0f880a6aa0 2021-11-21T22:01:55.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f880a8bc0 con 0x7f0f880a86b0 2021-11-21T22:01:55.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8814a7b0 con 0x7f0f88062410 2021-11-21T22:01:55.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8d339700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0f880a6aa0 0x7f0f881470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35558/0 (socket says 172.21.2.116:35558) 2021-11-21T22:01:55.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8d339700 1 -- 172.21.2.116:0/2443503716 learned_addr learned my addr 172.21.2.116:0/2443503716 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8cb38700 1 --2- 172.21.2.116:0/2443503716 >> v2:172.21.2.116:3300/0 conn(0x7f0f880a86b0 0x7f0f8814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f77fff700 1 -- 172.21.2.116:0/2443503716 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424147303 0 0) 0x7f0f880a6e80 con 0x7f0f880a6aa0 2021-11-21T22:01:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f77fff700 1 -- 172.21.2.116:0/2443503716 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f70003260 con 0x7f0f880a6aa0 2021-11-21T22:01:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8cb38700 1 -- 172.21.2.116:0/2443503716 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 msgr2=0x7f0f881443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:55.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8cb38700 1 --2- 172.21.2.116:0/2443503716 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 0x7f0f881443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8cb38700 1 -- 172.21.2.116:0/2443503716 >> v1:172.21.2.116:6789/0 conn(0x7f0f880a6aa0 legacy=0x7f0f881470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8cb38700 1 -- 172.21.2.116:0/2443503716 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8814aa10 con 0x7f0f880a86b0 2021-11-21T22:01:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8cb38700 1 --2- 172.21.2.116:0/2443503716 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a86b0 0x7f0f8814b150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f0f78002f70 tx=0x7f0f78006df0).ready entity=mon.0 client_cookie=959d920de14bc70f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f77fff700 1 -- 172.21.2.116:0/2443503716 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f7800d570 con 0x7f0f880a86b0 2021-11-21T22:01:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f77fff700 1 -- 172.21.2.116:0/2443503716 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f78000e90 con 0x7f0f880a86b0 2021-11-21T22:01:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f77fff700 1 -- 172.21.2.116:0/2443503716 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f7800d8c0 con 0x7f0f880a86b0 2021-11-21T22:01:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 -- 172.21.2.116:0/2443503716 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a86b0 msgr2=0x7f0f8814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/2443503716 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a86b0 0x7f0f8814b150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f0f78002f70 tx=0x7f0f78006df0).stop 2021-11-21T22:01:55.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.350+0000 7f0f8f59d700 1 -- 172.21.2.116:0/2443503716 shutdown_connections 2021-11-21T22:01:55.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/2443503716 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 0x7f0f881443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/2443503716 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a86b0 0x7f0f8814b150 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:55.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- 172.21.2.116:0/2443503716 >> 172.21.2.116:0/2443503716 conn(0x7f0f8805bd20 msgr2=0x7f0f8805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:55.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- 172.21.2.116:0/2443503716 shutdown_connections 2021-11-21T22:01:55.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- 172.21.2.116:0/2443503716 wait complete. 2021-11-21T22:01:55.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 Processor -- start 2021-11-21T22:01:55.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- start start 2021-11-21T22:01:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 0x7f0f88147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a6aa0 0x7f0f88148c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f880a86b0 0x7f0f88159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f881464c0 con 0x7f0f880a6aa0 2021-11-21T22:01:55.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88141770 con 0x7f0f880a86b0 2021-11-21T22:01:55.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88140f90 con 0x7f0f88062410 2021-11-21T22:01:55.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a6aa0 0x7f0f88148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a6aa0 0x7f0f88148c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51844/0 (socket says 172.21.2.116:51844) 2021-11-21T22:01:55.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 -- 172.21.2.116:0/122165504 learned_addr learned my addr 172.21.2.116:0/122165504 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:55.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 -- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 msgr2=0x7f0f88147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 0x7f0f88147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 -- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f880a86b0 msgr2=0x7f0f88159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:55.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f880a86b0 0x7f0f88159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:55.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 -- 172.21.2.116:0/122165504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7800b180 con 0x7f0f880a6aa0 2021-11-21T22:01:55.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8cb38700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a6aa0 0x7f0f88148c00 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f0f78002f70 tx=0x7f0f78014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:55.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f767fc700 1 -- 172.21.2.116:0/122165504 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f78016070 con 0x7f0f880a6aa0 2021-11-21T22:01:55.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f767fc700 1 -- 172.21.2.116:0/122165504 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7800de80 con 0x7f0f880a6aa0 2021-11-21T22:01:55.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f88141230 con 0x7f0f880a6aa0 2021-11-21T22:01:55.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f767fc700 1 -- 172.21.2.116:0/122165504 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f78006b20 con 0x7f0f880a6aa0 2021-11-21T22:01:55.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f88140a10 con 0x7f0f880a6aa0 2021-11-21T22:01:55.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f880573d0 con 0x7f0f880a6aa0 2021-11-21T22:01:55.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f767fc700 1 -- 172.21.2.116:0/122165504 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0f7800f030 con 0x7f0f880a6aa0 2021-11-21T22:01:55.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f767fc700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f680536a0 0x7f0f68055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:55.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.354+0000 7f0f767fc700 1 -- 172.21.2.116:0/122165504 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0f7806c4d0 con 0x7f0f880a6aa0 2021-11-21T22:01:55.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.358+0000 7f0f8d339700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f680536a0 0x7f0f68055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:55.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.358+0000 7f0f8d339700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f680536a0 0x7f0f68055b60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f0f84001110 tx=0x7f0f840076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:55.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.358+0000 7f0f767fc700 1 -- 172.21.2.116:0/122165504 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f78043800 con 0x7f0f880a6aa0 2021-11-21T22:01:55.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:55.470+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f0f881596c0 con 0x7f0f880a6aa0 2021-11-21T22:01:56.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f767fc700 1 -- 172.21.2.116:0/122165504 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v327) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0f78023020 con 0x7f0f880a6aa0 2021-11-21T22:01:56.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f680536a0 msgr2=0x7f0f68055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:56.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f680536a0 0x7f0f68055b60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f0f84001110 tx=0x7f0f840076c0).stop 2021-11-21T22:01:56.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a6aa0 msgr2=0x7f0f88148c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:56.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a6aa0 0x7f0f88148c00 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f0f78002f70 tx=0x7f0f78014000).stop 2021-11-21T22:01:56.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 shutdown_connections 2021-11-21T22:01:56.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0f880a86b0 0x7f0f88159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:56.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f680536a0 0x7f0f68055b60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:56.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0f880a6aa0 0x7f0f88148c00 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:56.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 --2- 172.21.2.116:0/122165504 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0f88062410 0x7f0f88147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:56.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 >> 172.21.2.116:0/122165504 conn(0x7f0f8805bd20 msgr2=0x7f0f88149b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:56.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 shutdown_connections 2021-11-21T22:01:56.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.138+0000 7f0f8f59d700 1 -- 172.21.2.116:0/122165504 wait complete. 2021-11-21T22:01:56.150 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2021-11-21T22:01:56.151 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:01:56.151 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2021-11-21T22:01:56.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20d1fed700 1 Processor -- start 2021-11-21T22:01:56.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20d1fed700 1 -- start start 2021-11-21T22:01:56.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20d1fed700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f20cc0a6aa0 0x7f20cc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20d1fed700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc0a6e80 con 0x7f20cc062410 2021-11-21T22:01:56.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20d1fed700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc14a7b0 con 0x7f20cc0a86b0 2021-11-21T22:01:56.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20d1fed700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20cc0a8bc0 con 0x7f20cc0a6aa0 2021-11-21T22:01:56.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f20cc062410 0x7f20cc1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35576/0 (socket says 172.21.2.116:35576) 2021-11-21T22:01:56.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 -- 172.21.2.116:0/3811008927 learned_addr learned my addr 172.21.2.116:0/3811008927 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:56.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232137042 0 0) 0x7f20cc0a6e80 con 0x7f20cc062410 2021-11-21T22:01:56.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac003260 con 0x7f20cc062410 2021-11-21T22:01:56.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cb7fe700 1 --2- 172.21.2.116:0/3811008927 >> v2:172.21.2.117:3300/0 conn(0x7f20cc0a6aa0 0x7f20cc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:56.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047278236 0 0) 0x7f20ac003260 con 0x7f20cc062410 2021-11-21T22:01:56.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 >> v2:172.21.2.117:3300/0 conn(0x7f20cc0a6aa0 msgr2=0x7f20cc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:56.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 --2- 172.21.2.116:0/3811008927 >> v2:172.21.2.117:3300/0 conn(0x7f20cc0a6aa0 0x7f20cc14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:56.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 >> v1:172.21.2.117:6789/0 conn(0x7f20cc0a86b0 legacy=0x7f20cc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:56.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc14aa10 con 0x7f20cc062410 2021-11-21T22:01:56.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f20c4001a00 con 0x7f20cc062410 2021-11-21T22:01:56.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 >> v1:172.21.2.116:6789/0 conn(0x7f20cc062410 legacy=0x7f20cc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:56.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f20ac0037d0 0x7f20ac003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20ac004500 0x7f20ac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20ac005070 0x7f20ac00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc0a8bc0 con 0x7f20ac005070 2021-11-21T22:01:56.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00bef0 con 0x7f20ac004500 2021-11-21T22:01:56.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00c0b0 con 0x7f20ac0037d0 2021-11-21T22:01:56.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20ac005070 0x7f20ac00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:56.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 -- 172.21.2.116:0/3811008927 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f20ac0037d0 msgr2=0x7f20ac003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:56.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f20ac0037d0 0x7f20ac003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:56.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 -- 172.21.2.116:0/3811008927 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20ac004500 msgr2=0x7f20ac004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:56.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20ac004500 0x7f20ac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:56.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 -- 172.21.2.116:0/3811008927 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc14aa10 con 0x7f20ac005070 2021-11-21T22:01:56.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 -- 172.21.2.116:0/3811008927 reap_dead start 2021-11-21T22:01:56.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20cbfff700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20ac005070 0x7f20ac00b750 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f20c40013c0 tx=0x7f20c40021c0).ready entity=mon.0 client_cookie=eb3fda602d380331 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:56.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20c400cc90 con 0x7f20ac005070 2021-11-21T22:01:56.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c400cdf0 con 0x7f20ac005070 2021-11-21T22:01:56.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.202+0000 7f20ca7fc700 1 -- 172.21.2.116:0/3811008927 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20c4004b50 con 0x7f20ac005070 2021-11-21T22:01:56.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/3811008927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20ac005070 msgr2=0x7f20ac00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:56.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20ac005070 0x7f20ac00b750 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f20c40013c0 tx=0x7f20c40021c0).stop 2021-11-21T22:01:56.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/3811008927 shutdown_connections 2021-11-21T22:01:56.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 --2- 172.21.2.116:0/3811008927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20ac005070 0x7f20ac00b750 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:56.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/3811008927 >> 172.21.2.116:0/3811008927 conn(0x7f20cc05bd20 msgr2=0x7f20cc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:56.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/3811008927 shutdown_connections 2021-11-21T22:01:56.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/3811008927 wait complete. 2021-11-21T22:01:56.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 Processor -- start 2021-11-21T22:01:56.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- start start 2021-11-21T22:01:56.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f20ac005070 0x7f20cc148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20cc13c470 0x7f20cc13e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20cc0b2310 0x7f20cc156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc13c080 con 0x7f20cc13c470 2021-11-21T22:01:56.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc13cae0 con 0x7f20cc0b2310 2021-11-21T22:01:56.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc13cd80 con 0x7f20ac005070 2021-11-21T22:01:56.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20cc13c470 0x7f20cc13e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:56.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20cc13c470 0x7f20cc13e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51862/0 (socket says 172.21.2.116:51862) 2021-11-21T22:01:56.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 -- 172.21.2.116:0/132144595 learned_addr learned my addr 172.21.2.116:0/132144595 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:56.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20cbfff700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20cc0b2310 0x7f20cc156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:56.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 -- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f20ac005070 msgr2=0x7f20cc148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:56.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f20ac005070 0x7f20cc148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:56.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 -- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20cc0b2310 msgr2=0x7f20cc156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:56.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20cc0b2310 0x7f20cc156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:56.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 -- 172.21.2.116:0/132144595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc1420d0 con 0x7f20cc13c470 2021-11-21T22:01:56.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20caffd700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20cc13c470 0x7f20cc13e8a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7f20b4000f70 tx=0x7f20b4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:56.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20c8ff9700 1 -- 172.21.2.116:0/132144595 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20b4013460 con 0x7f20cc13c470 2021-11-21T22:01:56.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20c8ff9700 1 -- 172.21.2.116:0/132144595 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b40135c0 con 0x7f20cc13c470 2021-11-21T22:01:56.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20cc1566f0 con 0x7f20cc13c470 2021-11-21T22:01:56.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20c8ff9700 1 -- 172.21.2.116:0/132144595 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20b4013920 con 0x7f20cc13c470 2021-11-21T22:01:56.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc159750 con 0x7f20cc13c470 2021-11-21T22:01:56.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20cc13cf40 con 0x7f20cc13c470 2021-11-21T22:01:56.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20c8ff9700 1 -- 172.21.2.116:0/132144595 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f20b401a060 con 0x7f20cc13c470 2021-11-21T22:01:56.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20c8ff9700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f20b0053640 0x7f20b0055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:56.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.206+0000 7f20c8ff9700 1 -- 172.21.2.116:0/132144595 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f20b406af30 con 0x7f20cc13c470 2021-11-21T22:01:56.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.210+0000 7f20cb7fe700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f20b0053640 0x7f20b0055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:56.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.210+0000 7f20cb7fe700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f20b0053640 0x7f20b0055b00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f20bc007000 tx=0x7f20bc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:56.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.210+0000 7f20c8ff9700 1 -- 172.21.2.116:0/132144595 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f20b4042370 con 0x7f20cc13c470 2021-11-21T22:01:56.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:56.322+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f20cc0573d0 con 0x7f20cc13c470 2021-11-21T22:01:57.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.146+0000 7f20c8ff9700 1 -- 172.21.2.116:0/132144595 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v328) v1 ==== 125+0+0 (secure 0 0 0) 0x7f20b4018070 con 0x7f20cc13c470 2021-11-21T22:01:57.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f20b0053640 msgr2=0x7f20b0055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f20b0053640 0x7f20b0055b00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f20bc007000 tx=0x7f20bc008040).stop 2021-11-21T22:01:57.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20cc13c470 msgr2=0x7f20cc13e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20cc13c470 0x7f20cc13e8a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7f20b4000f70 tx=0x7f20b4006df0).stop 2021-11-21T22:01:57.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 shutdown_connections 2021-11-21T22:01:57.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f20cc0b2310 0x7f20cc156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f20b0053640 0x7f20b0055b00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f20cc13c470 0x7f20cc13e8a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 --2- 172.21.2.116:0/132144595 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f20ac005070 0x7f20cc148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:57.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 >> 172.21.2.116:0/132144595 conn(0x7f20cc05bd20 msgr2=0x7f20cc14c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:57.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 shutdown_connections 2021-11-21T22:01:57.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.150+0000 7f20d1fed700 1 -- 172.21.2.116:0/132144595 wait complete. 2021-11-21T22:01:57.160 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T22:01:57.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec7043700 1 Processor -- start 2021-11-21T22:01:57.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec7043700 1 -- start start 2021-11-21T22:01:57.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec7043700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcec01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec7043700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec0147e80 con 0x7fcec00ac600 2021-11-21T22:01:57.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec7043700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00af5a0 con 0x7fcec00abce0 2021-11-21T22:01:57.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec7043700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b1490 con 0x7fcec0144810 2021-11-21T22:01:57.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec55e0700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fcec00ac600 0x7fcec00aca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35596/0 (socket says 172.21.2.116:35596) 2021-11-21T22:01:57.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec55e0700 1 -- 172.21.2.116:0/2381388189 learned_addr learned my addr 172.21.2.116:0/2381388189 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:57.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec4ddf700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcec01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602583885 0 0) 0x7fcec0147e80 con 0x7fcec00ac600 2021-11-21T22:01:57.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea0003260 con 0x7fcec00ac600 2021-11-21T22:01:57.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3988587757 0 0) 0x7fcea0003260 con 0x7fcec00ac600 2021-11-21T22:01:57.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 msgr2=0x7fcec01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcec01478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 >> v1:172.21.2.117:6789/0 conn(0x7fcec00abce0 legacy=0x7fcec00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00b1fd0 con 0x7fcec00ac600 2021-11-21T22:01:57.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fceb80019e0 con 0x7fcec00ac600 2021-11-21T22:01:57.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 >> v1:172.21.2.116:6789/0 conn(0x7fcec00ac600 legacy=0x7fcec00aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcea0003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcea0003cb0 0x7fcea0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcea0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b1490 con 0x7fcea00049f0 2021-11-21T22:01:57.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea000b800 con 0x7fcea0003cb0 2021-11-21T22:01:57.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea000b9c0 con 0x7fcec0144810 2021-11-21T22:01:57.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.210+0000 7fcec55e0700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcea0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcea0003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebffff700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcea0003cb0 0x7fcea0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 -- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 msgr2=0x7fcea0003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec0144810 0x7fcea0003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 -- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcea0003cb0 msgr2=0x7fcea0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcea0003cb0 0x7fcea0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 -- 172.21.2.116:0/2381388189 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0003260 con 0x7fcea00049f0 2021-11-21T22:01:57.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebffff700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcea0003cb0 0x7fcea0004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:57.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcea0004e70 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7fceb80016f0 tx=0x7fcec0146e80).ready entity=mon.0 client_cookie=1cd0af109de8ad20 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb800cca0 con 0x7fcea00049f0 2021-11-21T22:01:57.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb800ce00 con 0x7fcea00049f0 2021-11-21T22:01:57.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebf7fe700 1 -- 172.21.2.116:0/2381388189 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb8005b80 con 0x7fcea00049f0 2021-11-21T22:01:57.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2381388189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 msgr2=0x7fcea0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 --2- 172.21.2.116:0/2381388189 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcea0004e70 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7fceb80016f0 tx=0x7fcec0146e80).stop 2021-11-21T22:01:57.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 -- 172.21.2.116:0/2381388189 reap_dead start 2021-11-21T22:01:57.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2381388189 shutdown_connections 2021-11-21T22:01:57.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2381388189 >> 172.21.2.116:0/2381388189 conn(0x7fcec005bd20 msgr2=0x7fcec005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:57.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2381388189 shutdown_connections 2021-11-21T22:01:57.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2381388189 wait complete. 2021-11-21T22:01:57.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 Processor -- start 2021-11-21T22:01:57.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- start start 2021-11-21T22:01:57.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcec01462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec00aa000 0x7fcec015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec0156460 0x7fcec015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00a7fa0 con 0x7fcea00049f0 2021-11-21T22:01:57.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00a8240 con 0x7fcec0156460 2021-11-21T22:01:57.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0159bb0 con 0x7fcec00aa000 2021-11-21T22:01:57.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcec01462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcec01462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51882/0 (socket says 172.21.2.116:51882) 2021-11-21T22:01:57.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 -- 172.21.2.116:0/2812303868 learned_addr learned my addr 172.21.2.116:0/2812303868 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:57.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebffff700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec00aa000 0x7fcec015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec55e0700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec0156460 0x7fcec015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 -- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec00aa000 msgr2=0x7fcec015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec00aa000 0x7fcec015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 -- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec0156460 msgr2=0x7fcec015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec0156460 0x7fcec015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 -- 172.21.2.116:0/2812303868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0063390 con 0x7fcea00049f0 2021-11-21T22:01:57.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec4ddf700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcec01462f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7fceb000b000 tx=0x7fceb0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebdffb700 1 -- 172.21.2.116:0/2812303868 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000e6c0 con 0x7fcea00049f0 2021-11-21T22:01:57.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebdffb700 1 -- 172.21.2.116:0/2812303868 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb000e820 con 0x7fcea00049f0 2021-11-21T22:01:57.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec00a7a20 con 0x7fcea00049f0 2021-11-21T22:01:57.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebdffb700 1 -- 172.21.2.116:0/2812303868 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000eb80 con 0x7fcea00049f0 2021-11-21T22:01:57.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec0153850 con 0x7fcea00049f0 2021-11-21T22:01:57.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec00a8400 con 0x7fcea00049f0 2021-11-21T22:01:57.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.214+0000 7fcebdffb700 1 -- 172.21.2.116:0/2812303868 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fceb000ece0 con 0x7fcea00049f0 2021-11-21T22:01:57.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.218+0000 7fcebdffb700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea40536d0 0x7fcea4055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.218+0000 7fcebdffb700 1 -- 172.21.2.116:0/2812303868 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fceb006bac0 con 0x7fcea00049f0 2021-11-21T22:01:57.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.218+0000 7fcebffff700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea40536d0 0x7fcea4055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.218+0000 7fcebffff700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea40536d0 0x7fcea4055b90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fcea800c000 tx=0x7fcea8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.218+0000 7fcebdffb700 1 -- 172.21.2.116:0/2812303868 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fceb0042f00 con 0x7fcea00049f0 2021-11-21T22:01:57.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.302+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcec00af060 con 0x7fcea00049f0 2021-11-21T22:01:57.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.306+0000 7fcebdffb700 1 -- 172.21.2.116:0/2812303868 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fceb0046520 con 0x7fcea00049f0 2021-11-21T22:01:57.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea40536d0 msgr2=0x7fcea4055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea40536d0 0x7fcea4055b90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fcea800c000 tx=0x7fcea8009040).stop 2021-11-21T22:01:57.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 msgr2=0x7fcec01462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcec01462f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7fceb000b000 tx=0x7fceb0008040).stop 2021-11-21T22:01:57.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 shutdown_connections 2021-11-21T22:01:57.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcec0156460 0x7fcec015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcec00aa000 0x7fcec015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcea40536d0 0x7fcea4055b90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 --2- 172.21.2.116:0/2812303868 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcea00049f0 0x7fcec01462f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 >> 172.21.2.116:0/2812303868 conn(0x7fcec005bd20 msgr2=0x7fcec00aca30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:57.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 shutdown_connections 2021-11-21T22:01:57.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.310+0000 7fcec7043700 1 -- 172.21.2.116:0/2812303868 wait complete. 2021-11-21T22:01:57.319 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:01:57.320 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:57.328 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:57.336 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:57.349 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:01:57.363 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:01:57.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:57.364 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:57.364 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:01:57.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:01:57.369 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:01:57.370 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:01:57.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:01:57.371 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:01:57.371 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:01:57.379 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:01:57.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:01:57.381 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:01:57.382 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:01:57.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.454+0000 7ff2665e4700 1 Processor -- start 2021-11-21T22:01:57.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.454+0000 7ff2665e4700 1 -- start start 2021-11-21T22:01:57.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.454+0000 7ff2665e4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff2600a86b0 0x7ff26014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.454+0000 7ff2665e4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2601443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.454+0000 7ff2665e4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600a6e80 con 0x7ff2600a6aa0 2021-11-21T22:01:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.454+0000 7ff2665e4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600a8bc0 con 0x7ff2600a86b0 2021-11-21T22:01:57.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.454+0000 7ff2665e4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26014a7b0 con 0x7ff260062410 2021-11-21T22:01:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25f7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff2600a86b0 0x7ff26014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff264b81700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2601443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25ffff700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7ff2600a6aa0 0x7ff2601470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:34524/0 (socket says 172.21.2.116:34524) 2021-11-21T22:01:57.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25ffff700 1 -- 172.21.2.116:0/3953017078 learned_addr learned my addr 172.21.2.116:0/3953017078 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:57.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25f7fe700 1 -- 172.21.2.116:0/3953017078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 msgr2=0x7ff2601443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25f7fe700 1 --2- 172.21.2.116:0/3953017078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2601443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25f7fe700 1 -- 172.21.2.116:0/3953017078 >> v1:172.21.2.117:6789/0 conn(0x7ff2600a6aa0 legacy=0x7ff2601470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25f7fe700 1 -- 172.21.2.116:0/3953017078 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26014aa10 con 0x7ff2600a86b0 2021-11-21T22:01:57.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25f7fe700 1 --2- 172.21.2.116:0/3953017078 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 0x7ff26014b150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7ff248001110 tx=0x7ff250006df0).ready entity=mon.1 client_cookie=4ce6631d9f521481 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25effd700 1 -- 172.21.2.116:0/3953017078 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25000d4a0 con 0x7ff2600a86b0 2021-11-21T22:01:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25effd700 1 -- 172.21.2.116:0/3953017078 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff250000e90 con 0x7ff2600a86b0 2021-11-21T22:01:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff25effd700 1 -- 172.21.2.116:0/3953017078 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25000d7f0 con 0x7ff2600a86b0 2021-11-21T22:01:57.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff2665e4700 1 -- 172.21.2.116:0/3953017078 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 msgr2=0x7ff26014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.458+0000 7ff2665e4700 1 --2- 172.21.2.116:0/3953017078 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 0x7ff26014b150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7ff248001110 tx=0x7ff250006df0).stop 2021-11-21T22:01:57.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- 172.21.2.116:0/3953017078 shutdown_connections 2021-11-21T22:01:57.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 --2- 172.21.2.116:0/3953017078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2601443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 --2- 172.21.2.116:0/3953017078 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 0x7ff26014b150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- 172.21.2.116:0/3953017078 >> 172.21.2.116:0/3953017078 conn(0x7ff26005bd20 msgr2=0x7ff26005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:57.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- 172.21.2.116:0/3953017078 shutdown_connections 2021-11-21T22:01:57.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- 172.21.2.116:0/3953017078 wait complete. 2021-11-21T22:01:57.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 Processor -- start 2021-11-21T22:01:57.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- start start 2021-11-21T22:01:57.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2600abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2600a6aa0 0x7ff2600ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 0x7ff260153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600ac800 con 0x7ff2600a6aa0 2021-11-21T22:01:57.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff260067140 con 0x7ff2600a86b0 2021-11-21T22:01:57.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff260156d00 con 0x7ff260062410 2021-11-21T22:01:57.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2600a6aa0 0x7ff2600ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2600a6aa0 0x7ff2600ac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51898/0 (socket says 172.21.2.116:51898) 2021-11-21T22:01:57.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 -- 172.21.2.116:0/1760541924 learned_addr learned my addr 172.21.2.116:0/1760541924 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:57.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25ffff700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2600abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 -- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 msgr2=0x7ff2600abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2600abe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 -- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 msgr2=0x7ff260153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 0x7ff260153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:57.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 -- 172.21.2.116:0/1760541924 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25000b180 con 0x7ff2600a6aa0 2021-11-21T22:01:57.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25f7fe700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2600a6aa0 0x7ff2600ac310 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7ff248001110 tx=0x7ff250014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25d7fa700 1 -- 172.21.2.116:0/1760541924 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff250016070 con 0x7ff2600a6aa0 2021-11-21T22:01:57.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25d7fa700 1 -- 172.21.2.116:0/1760541924 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff250013800 con 0x7ff2600a6aa0 2021-11-21T22:01:57.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25d7fa700 1 -- 172.21.2.116:0/1760541924 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff250013b60 con 0x7ff2600a6aa0 2021-11-21T22:01:57.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff260156fa0 con 0x7ff2600a6aa0 2021-11-21T22:01:57.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2600aa550 con 0x7ff2600a6aa0 2021-11-21T22:01:57.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff260157160 con 0x7ff2600a6aa0 2021-11-21T22:01:57.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25d7fa700 1 -- 172.21.2.116:0/1760541924 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff250025030 con 0x7ff2600a6aa0 2021-11-21T22:01:57.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25d7fa700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff244053640 0x7ff244055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.462+0000 7ff25d7fa700 1 -- 172.21.2.116:0/1760541924 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff25006c5c0 con 0x7ff2600a6aa0 2021-11-21T22:01:57.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.466+0000 7ff25ffff700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff244053640 0x7ff244055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.466+0000 7ff25ffff700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff244053640 0x7ff244055b00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7ff248008000 tx=0x7ff248009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.466+0000 7ff25d7fa700 1 -- 172.21.2.116:0/1760541924 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff250043a40 con 0x7ff2600a6aa0 2021-11-21T22:01:57.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.546+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff260156b30 con 0x7ff2600a6aa0 2021-11-21T22:01:57.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff25d7fa700 1 -- 172.21.2.116:0/1760541924 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v114) v1 ==== 81+0+28837 (secure 0 0 0) 0x7ff250047020 con 0x7ff2600a6aa0 2021-11-21T22:01:57.554 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:57.554 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:57.555 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 114, 2021-11-21T22:01:57.555 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:01:57.555 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:01:57.555 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:01:55.120296+0000", 2021-11-21T22:01:57.556 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:57.556 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:01:57.556 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:01:57.556 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:01:57.557 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:01:57.557 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:01:57.557 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:01:57.557 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:01:57.557 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:01:57.558 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:57.558 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:01:57.558 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:01:57.558 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:01:57.559 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:01:57.559 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:01:57.559 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 20, 2021-11-21T22:01:57.559 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:01:57.559 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:01:57.560 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:01:57.560 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:01:57.560 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:01:57.560 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.561 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:01:57.561 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:01:57.561 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:01:57.561 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:57.561 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:57.562 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:57.562 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:57.562 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:57.562 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:57.563 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:57.563 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:57.563 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:01:57.564 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:01:57.564 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:01:57.565 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:01:57.565 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:01:57.565 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:57.566 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:57.566 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:57.566 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:57.566 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:57.567 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:57.567 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:57.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.567 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:01:57.567 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:57.568 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:57.568 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:57.568 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:57.568 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:57.569 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:57.569 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:57.569 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:57.569 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:57.569 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:57.570 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:57.570 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:57.570 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:57.570 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:57.571 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:57.571 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:57.571 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:57.571 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:57.571 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:57.572 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:57.572 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:57.572 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:57.572 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:57.573 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:57.573 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:57.573 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:57.573 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.573 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:57.574 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:57.574 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:57.574 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:57.574 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:57.575 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:57.575 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:57.575 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:57.575 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:57.575 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:57.576 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:57.576 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:01:57.576 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:01:57.576 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.577 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:57.577 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:01:57.577 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.577 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.577 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.578 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:01:57.578 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:01:57.578 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:01:57.578 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:01:57.578 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:01:57.579 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:01:57.579 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:01:57.579 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:01:57.579 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:01:57.580 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:01:57.580 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:01:57.580 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:01:57.580 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:01:57.580 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:01:57.581 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:01:57.581 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:01:57.581 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:01:57.581 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:01:57.581 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:01:57.582 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:01:57.582 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:01:57.582 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:01:57.582 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:01:57.583 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.583 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:01:57.583 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:01:57.583 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:01:57.583 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:01:57.584 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:01:57.584 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:01:57.584 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:01:57.584 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:01:57.584 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:01:57.585 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:01:57.585 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:01:57.585 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:01:57.585 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:01:57.586 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:01:57.586 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:01:57.586 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:01:57.586 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:01:57.586 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:01:57.587 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:01:57.587 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:01:57.587 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:01:57.587 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:01:57.588 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:01:57.588 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:01:57.588 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:01:57.588 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:01:57.588 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:01:57.589 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.589 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:01:57.589 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:01:57.589 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:01:57.590 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:01:57.590 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:01:57.590 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:01:57.590 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:01:57.591 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:01:57.591 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:01:57.591 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:01:57.591 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:01:57.591 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:01:57.592 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:01:57.592 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:01:57.592 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.592 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.592 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:57.593 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:01:57.593 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.593 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:57.593 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:01:57.594 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.594 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.594 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.594 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.594 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.595 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.595 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.595 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:57.595 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.595 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.596 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.596 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.596 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.596 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.597 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:01:57.597 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.597 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.597 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.597 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.598 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:01:57.598 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.598 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.598 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.599 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.599 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.599 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.599 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.599 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.600 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:01:57.600 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.600 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.600 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.601 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.601 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:01:57.601 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.601 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.601 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.602 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.602 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.602 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.602 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.602 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.603 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:01:57.603 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.603 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.603 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.604 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.604 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:01:57.604 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.604 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.604 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.605 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.605 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.605 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.605 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.605 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.606 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:01:57.606 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.606 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.606 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.607 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.607 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:01:57.607 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:01:57.607 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.607 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.608 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.608 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:01:57.608 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:01:57.608 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:01:57.609 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:01:57.609 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.609 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.609 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.609 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.610 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.610 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.610 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:57.610 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:01:57.611 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.611 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.611 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.611 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.612 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.612 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.612 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.612 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:57.613 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.613 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.613 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.613 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.614 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.614 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.614 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:01:57.614 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.614 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.615 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.615 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.615 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:01:57.615 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.616 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.616 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.616 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.616 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.616 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.617 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.617 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.617 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:01:57.617 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.618 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.618 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.618 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.618 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:01:57.618 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.619 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.619 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.619 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.619 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.620 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.620 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.620 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.620 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:01:57.620 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.621 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.621 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.621 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.621 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:01:57.622 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.622 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.622 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.622 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.622 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.623 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.623 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.623 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.623 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:01:57.623 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.624 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.624 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.624 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.624 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:01:57.625 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:01:57.625 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.625 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.625 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.625 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:01:57.626 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:01:57.626 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:01:57.626 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:01:57.626 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.627 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.627 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.627 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.627 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.627 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.628 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:57.628 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:01:57.628 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.628 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.629 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.629 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.629 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.629 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.629 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.630 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 107, 2021-11-21T22:01:57.630 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.630 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.630 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.631 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.631 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.631 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.631 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:01:57.631 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.632 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.632 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.632 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.632 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:01:57.633 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.633 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.633 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.633 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.633 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.634 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.634 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.634 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.634 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:01:57.635 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.635 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.635 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.635 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.635 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:01:57.636 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.636 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.636 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.636 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.637 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.637 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.637 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.637 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.637 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:01:57.638 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.638 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.638 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.638 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.639 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:01:57.639 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.639 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.639 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.639 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.640 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.640 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.640 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.640 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.640 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:01:57.641 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.641 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.641 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.641 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.642 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:01:57.642 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:01:57.642 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.642 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.642 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.643 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:01:57.643 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:01:57.643 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:01:57.643 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:01:57.644 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.644 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.644 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.644 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.644 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.645 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.645 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:57.645 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:01:57.645 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.645 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.646 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.646 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.646 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.646 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.647 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.647 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:57.647 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.648 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.648 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.648 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.648 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.649 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.649 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:01:57.649 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.649 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.649 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.650 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.650 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:01:57.650 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.650 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.651 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.651 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.651 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.651 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.651 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.652 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.652 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:01:57.652 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.652 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.653 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.653 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.653 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:01:57.653 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.653 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.654 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.654 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.654 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.654 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.654 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.655 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.655 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:01:57.655 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.655 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.656 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.656 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.656 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:01:57.656 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.656 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.657 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.657 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.657 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.657 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.658 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.658 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.658 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:01:57.658 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.659 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.659 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.659 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.659 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:01:57.660 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:01:57.660 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.660 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.660 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.660 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:01:57.661 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:01:57.661 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:01:57.661 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:01:57.661 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.661 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.662 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.662 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.662 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.662 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.663 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:57.663 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:01:57.663 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.663 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.663 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.664 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.664 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.664 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.664 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.664 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 107, 2021-11-21T22:01:57.665 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.665 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.665 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.665 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.666 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.666 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.666 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:01:57.666 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.666 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.667 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.667 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.667 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:01:57.667 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.667 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.668 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.668 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.668 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.668 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.669 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.669 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.669 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:01:57.669 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.669 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.670 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.670 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.670 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:01:57.670 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.670 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.671 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.671 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.671 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.671 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.672 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.672 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.672 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:01:57.672 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.672 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.673 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.673 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.673 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:01:57.673 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.673 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.674 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.674 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.674 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.674 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.675 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.675 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.675 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:01:57.675 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.675 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.676 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.676 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.676 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:01:57.676 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:01:57.677 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.677 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.677 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.677 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:01:57.677 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:01:57.678 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:01:57.678 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:01:57.678 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.678 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.678 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.679 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.679 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.679 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.679 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:57.680 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:01:57.680 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.680 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.680 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.680 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.681 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.681 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.681 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.681 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 107, 2021-11-21T22:01:57.681 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.682 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.682 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.682 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.682 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.683 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.683 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:01:57.683 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.683 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.683 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.684 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.684 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:01:57.684 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.684 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.685 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.685 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.685 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.685 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.685 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.686 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.686 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:01:57.686 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.686 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.687 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.687 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.687 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:01:57.687 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.687 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.688 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.688 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.688 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.688 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.688 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.689 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.689 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:01:57.689 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.689 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.690 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.690 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.690 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:01:57.690 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.690 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.691 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.691 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.691 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.691 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.692 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.692 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.692 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:01:57.692 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.692 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.693 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.693 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.693 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:01:57.693 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:01:57.693 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.694 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.694 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.694 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:01:57.694 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:01:57.694 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:01:57.695 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:01:57.695 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.695 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.695 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.696 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.696 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.696 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.696 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:57.696 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:01:57.697 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.697 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.697 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.697 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.697 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.698 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.698 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.698 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:57.698 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.699 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.699 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.699 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.699 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.699 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.700 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:01:57.700 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.700 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.700 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.701 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.701 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:01:57.701 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.701 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.701 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.702 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.702 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.702 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.702 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.702 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.703 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:01:57.703 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.703 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.703 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.704 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.704 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:01:57.704 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.704 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.704 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.705 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.705 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.705 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.705 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.705 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.706 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:01:57.706 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.706 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.706 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.706 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.707 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:01:57.707 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.707 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.707 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.708 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.708 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.708 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.708 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.708 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.709 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:01:57.709 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.709 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.709 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.710 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.710 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:01:57.710 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:01:57.710 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.710 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.711 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.711 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:01:57.711 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:01:57.711 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:01:57.712 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:01:57.712 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.712 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.712 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.712 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.713 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.713 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.713 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:57.713 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:01:57.714 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:01:57.714 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:01:57.714 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:01:57.714 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:01:57.714 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:01:57.715 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:01:57.715 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:01:57.715 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 108, 2021-11-21T22:01:57.715 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:01:57.715 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:01:57.716 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:01:57.716 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.716 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.716 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.716 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:01:57.717 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.717 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.717 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.717 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.718 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:01:57.718 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.718 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.718 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.718 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.719 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:01:57.719 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.719 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.719 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.719 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:01:57.720 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.720 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.720 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.720 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.721 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:01:57.721 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.721 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.721 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.721 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.722 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:01:57.722 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.722 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.722 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.723 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:01:57.723 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.723 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.723 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.723 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.724 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:01:57.724 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.724 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.724 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.725 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.725 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:01:57.725 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:01:57.725 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.725 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:01:57.726 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:01:57.726 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.726 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.726 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.727 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:01:57.727 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:01:57.727 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:01:57.727 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.727 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.728 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.728 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:01:57.728 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:01:57.728 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:01:57.729 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:01:57.729 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:01:57.729 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:01:57.729 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:01:57.729 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:57.730 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.730 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:57.730 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:01:57.730 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.731 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:01:57.731 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.731 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.731 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.731 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.732 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.732 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:01:57.732 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.732 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.733 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.733 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:01:57.733 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.733 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.733 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.734 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.734 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.734 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:01:57.734 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.735 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.735 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.735 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:01:57.735 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.735 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.736 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.736 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.736 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.736 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:01:57.736 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.737 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.737 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.737 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:01:57.737 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.738 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.738 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.738 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.738 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.738 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:01:57.739 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.739 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.739 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.739 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:01:57.739 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.740 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.740 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.740 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.740 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.741 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:01:57.741 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.741 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.741 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.741 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:01:57.742 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.742 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.742 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.742 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.743 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.743 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:01:57.743 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.743 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.743 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.744 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:01:57.744 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.744 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.744 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.745 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.745 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.745 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:01:57.745 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.745 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.746 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:57.746 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:01:57.746 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:01:57.746 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:01:57.747 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:01:57.747 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:01:57.747 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:01:57.747 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:01:57.747 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:01:57.748 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.748 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:01:57.748 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:01:57.748 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:01:57.749 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:01:57.749 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:01:57.749 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:01:57.749 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/1808795000": "2021-11-22T22:01:51.567765+0000", 2021-11-21T22:01:57.749 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6833/1808795000": "2021-11-22T22:01:51.567765+0000" 2021-11-21T22:01:57.750 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.750 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:01:57.750 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:01:57.750 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:01:57.750 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:01:57.751 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:01:57.751 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:01:57.751 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:01:57.751 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:01:57.752 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:57.752 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:57.752 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:01:57.752 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:01:57.752 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:01:57.753 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:01:57.753 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:01:57.753 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:57.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff244053640 msgr2=0x7ff244055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff244053640 0x7ff244055b00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7ff248008000 tx=0x7ff248009040).stop 2021-11-21T22:01:57.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2600a6aa0 msgr2=0x7ff2600ac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2600a6aa0 0x7ff2600ac310 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7ff248001110 tx=0x7ff250014000).stop 2021-11-21T22:01:57.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 shutdown_connections 2021-11-21T22:01:57.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2600a86b0 0x7ff260153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:57.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff244053640 0x7ff244055b00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2600a6aa0 0x7ff2600ac310 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 --2- 172.21.2.116:0/1760541924 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff260062410 0x7ff2600abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 >> 172.21.2.116:0/1760541924 conn(0x7ff26005bd20 msgr2=0x7ff260148fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:57.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 shutdown_connections 2021-11-21T22:01:57.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.550+0000 7ff2665e4700 1 -- 172.21.2.116:0/1760541924 wait complete. 2021-11-21T22:01:57.758 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:57.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 Processor -- start 2021-11-21T22:01:57.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 -- start start 2021-11-21T22:01:57.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8544096bd0 0x7f8544096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 0x7f85440b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8544140b20 con 0x7f8544141730 2021-11-21T22:01:57.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85440b2340 con 0x7f8544096bd0 2021-11-21T22:01:57.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8544140d50 con 0x7f85440974a0 2021-11-21T22:01:57.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f85427fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8544141730 0x7f8544098c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35632/0 (socket says 172.21.2.116:35632) 2021-11-21T22:01:57.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f85427fc700 1 -- 172.21.2.116:0/11922145 learned_addr learned my addr 172.21.2.116:0/11922145 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:57.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085641151 0 0) 0x7f8544140b20 con 0x7f8544141730 2021-11-21T22:01:57.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8528003260 con 0x7f8544141730 2021-11-21T22:01:57.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f85437fe700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 0x7f85440b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 --2- 172.21.2.116:0/11922145 >> v2:172.21.2.117:3300/0 conn(0x7f8544096bd0 0x7f8544096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362920968 0 0) 0x7f8528003260 con 0x7f8544141730 2021-11-21T22:01:57.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 msgr2=0x7f85440b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 0x7f85440b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 >> v2:172.21.2.117:3300/0 conn(0x7f8544096bd0 msgr2=0x7f8544096fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 --2- 172.21.2.116:0/11922145 >> v2:172.21.2.117:3300/0 conn(0x7f8544096bd0 0x7f8544096fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544140ff0 con 0x7f8544141730 2021-11-21T22:01:57.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8538001660 con 0x7f8544141730 2021-11-21T22:01:57.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 >> v1:172.21.2.116:6789/0 conn(0x7f8544141730 legacy=0x7f8544098c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85280037d0 0x7f8528003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8528004500 0x7f8528004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 0x7f8528004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85440b2340 con 0x7f85280037d0 2021-11-21T22:01:57.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8544140d50 con 0x7f8528004500 2021-11-21T22:01:57.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852800fc30 con 0x7f85440974a0 2021-11-21T22:01:57.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85280037d0 0x7f8528003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f85427fc700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8528004500 0x7f8528004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f85437fe700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 0x7f8528004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 -- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 msgr2=0x7f8528004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85440974a0 0x7f8528004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 -- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8528004500 msgr2=0x7f8528004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8528004500 0x7f8528004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 -- 172.21.2.116:0/11922145 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544140ff0 con 0x7f85280037d0 2021-11-21T22:01:57.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8542ffd700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85280037d0 0x7f8528003bf0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f853400a000 tx=0x7f8534008040).ready entity=mon.0 client_cookie=66d1a70e81186767 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8534013670 con 0x7f85280037d0 2021-11-21T22:01:57.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85340137d0 con 0x7f85280037d0 2021-11-21T22:01:57.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8541ffb700 1 -- 172.21.2.116:0/11922145 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8534013b30 con 0x7f85280037d0 2021-11-21T22:01:57.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 -- 172.21.2.116:0/11922145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85280037d0 msgr2=0x7f8528003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 --2- 172.21.2.116:0/11922145 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85280037d0 0x7f8528003bf0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f853400a000 tx=0x7f8534008040).stop 2021-11-21T22:01:57.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f85437fe700 1 -- 172.21.2.116:0/11922145 reap_dead start 2021-11-21T22:01:57.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 -- 172.21.2.116:0/11922145 shutdown_connections 2021-11-21T22:01:57.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.810+0000 7f8543fff700 1 -- 172.21.2.116:0/11922145 >> 172.21.2.116:0/11922145 conn(0x7f85440b44c0 msgr2=0x7f854405bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:57.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- 172.21.2.116:0/11922145 shutdown_connections 2021-11-21T22:01:57.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- 172.21.2.116:0/11922145 wait complete. 2021-11-21T22:01:57.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 Processor -- start 2021-11-21T22:01:57.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- start start 2021-11-21T22:01:57.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85280037d0 0x7f854413e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8544063080 0x7f85440ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85440abbd0 0x7f8544151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85440b60f0 con 0x7f85440abbd0 2021-11-21T22:01:57.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85440628f0 con 0x7f85280037d0 2021-11-21T22:01:57.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8544062b90 con 0x7f8544063080 2021-11-21T22:01:57.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85440abbd0 0x7f8544151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85440abbd0 0x7f8544151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51920/0 (socket says 172.21.2.116:51920) 2021-11-21T22:01:57.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 -- 172.21.2.116:0/1412888948 learned_addr learned my addr 172.21.2.116:0/1412888948 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:57.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85427fc700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8544063080 0x7f85440ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 -- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8544063080 msgr2=0x7f85440ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8544063080 0x7f85440ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 -- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85280037d0 msgr2=0x7f854413e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85280037d0 0x7f854413e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:57.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 -- 172.21.2.116:0/1412888948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85440b5530 con 0x7f85440abbd0 2021-11-21T22:01:57.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f85437fe700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85440abbd0 0x7f8544151330 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f853c00d000 tx=0x7f853c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8533fff700 1 -- 172.21.2.116:0/1412888948 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f853c0126d0 con 0x7f85440abbd0 2021-11-21T22:01:57.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8533fff700 1 -- 172.21.2.116:0/1412888948 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c00c070 con 0x7f85440abbd0 2021-11-21T22:01:57.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8533fff700 1 -- 172.21.2.116:0/1412888948 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f853c012a60 con 0x7f85440abbd0 2021-11-21T22:01:57.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85440b2110 con 0x7f85440abbd0 2021-11-21T22:01:57.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85440635d0 con 0x7f85440abbd0 2021-11-21T22:01:57.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85440ac3a0 con 0x7f85440abbd0 2021-11-21T22:01:57.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8533fff700 1 -- 172.21.2.116:0/1412888948 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f853c012040 con 0x7f85440abbd0 2021-11-21T22:01:57.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8533fff700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85200536a0 0x7f8520055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8533fff700 1 -- 172.21.2.116:0/1412888948 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f853c06b5e0 con 0x7f85440abbd0 2021-11-21T22:01:57.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.814+0000 7f8542ffd700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85200536a0 0x7f8520055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.818+0000 7f8542ffd700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85200536a0 0x7f8520055b60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f853400a000 tx=0x7f8534008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.818+0000 7f8533fff700 1 -- 172.21.2.116:0/1412888948 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f853c042a20 con 0x7f85440abbd0 2021-11-21T22:01:57.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.926+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85440573d0 con 0x7f85440abbd0 2021-11-21T22:01:57.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.926+0000 7f8533fff700 1 -- 172.21.2.116:0/1412888948 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f853c046040 con 0x7f85440abbd0 2021-11-21T22:01:57.933 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:57.933 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47},{"gid":7590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/618837985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6833","nonce":618837985},{"type":"v1","addr":"172.21.2.116:6835","nonce":618837985}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":52}],"filesystems":[]} 2021-11-21T22:01:57.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85200536a0 msgr2=0x7f8520055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85200536a0 0x7f8520055b60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f853400a000 tx=0x7f8534008040).stop 2021-11-21T22:01:57.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85440abbd0 msgr2=0x7f8544151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85440abbd0 0x7f8544151330 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f853c00d000 tx=0x7f853c008040).stop 2021-11-21T22:01:57.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 shutdown_connections 2021-11-21T22:01:57.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f85200536a0 0x7f8520055b60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85440abbd0 0x7f8544151330 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8544063080 0x7f85440ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 --2- 172.21.2.116:0/1412888948 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85280037d0 0x7f854413e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:57.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 >> 172.21.2.116:0/1412888948 conn(0x7f85440b44c0 msgr2=0x7f854405c260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:57.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 shutdown_connections 2021-11-21T22:01:57.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.930+0000 7f8543fff700 1 -- 172.21.2.116:0/1412888948 wait complete. 2021-11-21T22:01:57.938 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 52 2021-11-21T22:01:57.940 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:01:57.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e7fff700 1 Processor -- start 2021-11-21T22:01:57.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e7fff700 1 -- start start 2021-11-21T22:01:57.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e80b5630 0x7f56e814af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e7fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f56e8097ec0 0x7f56e8146e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:57.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e7fff700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e814a560 con 0x7f56e8147300 2021-11-21T22:01:57.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e7fff700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e8144b40 con 0x7f56e8097ec0 2021-11-21T22:01:57.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e7fff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e814a800 con 0x7f56e80b5630 2021-11-21T22:01:57.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e67fc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f56e8097ec0 0x7f56e8146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e6ffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e80b5630 0x7f56e814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:57.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e6ffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e80b5630 0x7f56e814af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:55268/0 (socket says 172.21.2.116:55268) 2021-11-21T22:01:57.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e67fc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f56e8097ec0 0x7f56e8146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:53094/0 (socket says 172.21.2.116:53094) 2021-11-21T22:01:57.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.986+0000 7f56e6ffd700 1 -- 172.21.2.116:0/613404348 learned_addr learned my addr 172.21.2.116:0/613404348 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:57.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e6ffd700 1 -- 172.21.2.116:0/613404348 >> v2:172.21.2.117:3300/0 conn(0x7f56e8097ec0 msgr2=0x7f56e8146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:57.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e6ffd700 1 --2- 172.21.2.116:0/613404348 >> v2:172.21.2.117:3300/0 conn(0x7f56e8097ec0 0x7f56e8146e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e6ffd700 1 -- 172.21.2.116:0/613404348 >> v1:172.21.2.117:6789/0 conn(0x7f56e8147300 legacy=0x7f56e8099650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e6ffd700 1 -- 172.21.2.116:0/613404348 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e814aaa0 con 0x7f56e80b5630 2021-11-21T22:01:57.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e6ffd700 1 --2- 172.21.2.116:0/613404348 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e80b5630 0x7f56e814af40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f56d8002f70 tx=0x7f56d8006eb0).ready entity=mon.2 client_cookie=e5f6646a8e9f444e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:57.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e5ffb700 1 -- 172.21.2.116:0/613404348 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56d800d570 con 0x7f56e80b5630 2021-11-21T22:01:57.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e5ffb700 1 -- 172.21.2.116:0/613404348 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d8000e90 con 0x7f56e80b5630 2021-11-21T22:01:57.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e5ffb700 1 -- 172.21.2.116:0/613404348 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56d800d900 con 0x7f56e80b5630 2021-11-21T22:01:57.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- 172.21.2.116:0/613404348 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e80b5630 msgr2=0x7f56e814af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:57.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 --2- 172.21.2.116:0/613404348 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e80b5630 0x7f56e814af40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f56d8002f70 tx=0x7f56d8006eb0).stop 2021-11-21T22:01:57.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- 172.21.2.116:0/613404348 shutdown_connections 2021-11-21T22:01:57.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 --2- 172.21.2.116:0/613404348 >> v2:172.21.2.117:3300/0 conn(0x7f56e8097ec0 0x7f56e8146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 --2- 172.21.2.116:0/613404348 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e80b5630 0x7f56e814af40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:57.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- 172.21.2.116:0/613404348 >> 172.21.2.116:0/613404348 conn(0x7f56e805c7f0 msgr2=0x7f56e80b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:58.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- 172.21.2.116:0/613404348 shutdown_connections 2021-11-21T22:01:58.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- 172.21.2.116:0/613404348 wait complete. 2021-11-21T22:01:58.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 Processor -- start 2021-11-21T22:01:58.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- start start 2021-11-21T22:01:58.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f56e8097ec0 0x7f56e8159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f56e80b5630 0x7f56e80b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e8147300 0x7f56e80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80ac6a0 con 0x7f56e80b5630 2021-11-21T22:01:58.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80adc80 con 0x7f56e8097ec0 2021-11-21T22:01:58.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80adf20 con 0x7f56e8147300 2021-11-21T22:01:58.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f56e80b5630 0x7f56e80b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:58.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f56e80b5630 0x7f56e80b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51932/0 (socket says 172.21.2.116:51932) 2021-11-21T22:01:58.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 -- 172.21.2.116:0/3210839479 learned_addr learned my addr 172.21.2.116:0/3210839479 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:58.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 -- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e8147300 msgr2=0x7f56e80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e8147300 0x7f56e80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:58.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 -- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f56e8097ec0 msgr2=0x7f56e8159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f56e8097ec0 0x7f56e8159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:58.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 -- 172.21.2.116:0/3210839479 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d800b1e0 con 0x7f56e80b5630 2021-11-21T22:01:58.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e67fc700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f56e80b5630 0x7f56e80b2c60 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f56dc00c000 tx=0x7f56dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:58.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56d7fff700 1 -- 172.21.2.116:0/3210839479 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc010590 con 0x7f56e80b5630 2021-11-21T22:01:58.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56d7fff700 1 -- 172.21.2.116:0/3210839479 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc0106f0 con 0x7f56e80b5630 2021-11-21T22:01:58.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e80ac3d0 con 0x7f56e80b5630 2021-11-21T22:01:58.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.990+0000 7f56d7fff700 1 -- 172.21.2.116:0/3210839479 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc010a50 con 0x7f56e80b5630 2021-11-21T22:01:58.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e80ac060 con 0x7f56e80b5630 2021-11-21T22:01:58.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e80573d0 con 0x7f56e80b5630 2021-11-21T22:01:58.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56d7fff700 1 -- 172.21.2.116:0/3210839479 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f56dc010bb0 con 0x7f56e80b5630 2021-11-21T22:01:58.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56d7fff700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f56c4053610 0x7f56c4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56d7fff700 1 -- 172.21.2.116:0/3210839479 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f56dc06b7a0 con 0x7f56e80b5630 2021-11-21T22:01:58.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56e6ffd700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f56c4053610 0x7f56c4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:58.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56e6ffd700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f56c4053610 0x7f56c4055ad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f56d8002f70 tx=0x7f56d8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:58.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:57.994+0000 7f56d7fff700 1 -- 172.21.2.116:0/3210839479 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f56dc042b80 con 0x7f56e80b5630 2021-11-21T22:01:58.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.102+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56e80ae120 con 0x7f56e80b5630 2021-11-21T22:01:58.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.102+0000 7f56d7fff700 1 -- 172.21.2.116:0/3210839479 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f56dc0461a0 con 0x7f56e80b5630 2021-11-21T22:01:58.110 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:58.110 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/3091400410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3091400410},{"type":"v1","addr":"172.21.2.117:6835","nonce":3091400410}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6834/444609645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":444609645},{"type":"v1","addr":"172.21.2.116:6834","nonce":444609645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47},{"gid":7590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/618837985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6833","nonce":618837985},{"type":"v1","addr":"172.21.2.116:6835","nonce":618837985}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":52}],"filesystems":[]} 2021-11-21T22:01:58.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f56c4053610 msgr2=0x7f56c4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f56c4053610 0x7f56c4055ad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f56d8002f70 tx=0x7f56d8015000).stop 2021-11-21T22:01:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f56e80b5630 msgr2=0x7f56e80b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f56e80b5630 0x7f56e80b2c60 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f56dc00c000 tx=0x7f56dc006df0).stop 2021-11-21T22:01:58.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 shutdown_connections 2021-11-21T22:01:58.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f56e8147300 0x7f56e80b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:58.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f56c4053610 0x7f56c4055ad0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:58.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f56e80b5630 0x7f56e80b2c60 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:58.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 --2- 172.21.2.116:0/3210839479 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f56e8097ec0 0x7f56e8159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 >> 172.21.2.116:0/3210839479 conn(0x7f56e805c7f0 msgr2=0x7f56e80afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 shutdown_connections 2021-11-21T22:01:58.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.106+0000 7f56e7fff700 1 -- 172.21.2.116:0/3210839479 wait complete. 2021-11-21T22:01:58.117 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 52 2021-11-21T22:01:58.119 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T22:01:58.120 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T22:01:58.120 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:01:58.121 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:58.121 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T22:01:58.122 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T22:01:58.122 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:01:58.123 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:58.123 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T22:01:58.124 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T22:01:58.125 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:01:58.125 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:01:58.132 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:01:58.133 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T22:01:58.135 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:01:58.136 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T22:01:58.138 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:01:58.138 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T22:01:58.140 INFO:tasks.ceph.mds.b:Started 2021-11-21T22:01:58.141 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:58.174 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T22:01:58.177 INFO:tasks.ceph.mds.a:Started 2021-11-21T22:01:58.178 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:58.179 INFO:tasks.ceph.mds.c:Started 2021-11-21T22:01:58.180 DEBUG:teuthology.parallel:result is None 2021-11-21T22:01:58.180 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T22:01:58.215 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:01:58.224 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:01:58.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 Processor -- start 2021-11-21T22:01:58.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- start start 2021-11-21T22:01:58.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 0x7fdce00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdce0144a10 0x7fdce0141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce00b3c50 con 0x7fdce0141510 2021-11-21T22:01:58.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce00b3ef0 con 0x7fdce0144a10 2021-11-21T22:01:58.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00b4190 con 0x7fdce00adc10 2021-11-21T22:01:58.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdffff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 0x7fdce00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:58.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdce0144a10 0x7fdce0141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:58.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdce0144a10 0x7fdce0141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51962/0 (socket says 172.21.2.116:51962) 2021-11-21T22:01:58.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/967193227 learned_addr learned my addr 172.21.2.116:0/967193227 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:58.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/967193227 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 msgr2=0x7fdce00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:58.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- 172.21.2.116:0/967193227 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 0x7fdce00b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:58.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/967193227 >> v1:172.21.2.117:6789/0 conn(0x7fdce0141510 legacy=0x7fdce013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/967193227 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce0140b80 con 0x7fdce0144a10 2021-11-21T22:01:58.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- 172.21.2.116:0/967193227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0144a10 0x7fdce0141020 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7fdcc8002f70 tx=0x7fdcc8006eb0).ready entity=mon.0 client_cookie=445a2b69e648d5b4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:58.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdeffd700 1 -- 172.21.2.116:0/967193227 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc800d570 con 0x7fdce0144a10 2021-11-21T22:01:58.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdeffd700 1 -- 172.21.2.116:0/967193227 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc8000e90 con 0x7fdce0144a10 2021-11-21T22:01:58.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdeffd700 1 -- 172.21.2.116:0/967193227 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc800d900 con 0x7fdce0144a10 2021-11-21T22:01:58.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- 172.21.2.116:0/967193227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0144a10 msgr2=0x7fdce0141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- 172.21.2.116:0/967193227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0144a10 0x7fdce0141020 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7fdcc8002f70 tx=0x7fdcc8006eb0).stop 2021-11-21T22:01:58.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- 172.21.2.116:0/967193227 shutdown_connections 2021-11-21T22:01:58.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- 172.21.2.116:0/967193227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0144a10 0x7fdce0141020 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:58.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- 172.21.2.116:0/967193227 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 0x7fdce00b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:58.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- 172.21.2.116:0/967193227 >> 172.21.2.116:0/967193227 conn(0x7fdce005bd20 msgr2=0x7fdce005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:58.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- 172.21.2.116:0/967193227 shutdown_connections 2021-11-21T22:01:58.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- 172.21.2.116:0/967193227 wait complete. 2021-11-21T22:01:58.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 Processor -- start 2021-11-21T22:01:58.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- start start 2021-11-21T22:01:58.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 0x7fdce014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0141510 0x7fdce00b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdce0144a10 0x7fdce00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00b4cb0 con 0x7fdce0141510 2021-11-21T22:01:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce0143a30 con 0x7fdce0144a10 2021-11-21T22:01:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce66cc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00b62f0 con 0x7fdce00adc10 2021-11-21T22:01:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0141510 0x7fdce00b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0141510 0x7fdce00b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51970/0 (socket says 172.21.2.116:51970) 2021-11-21T22:01:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/239961998 learned_addr learned my addr 172.21.2.116:0/239961998 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce4c69700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdce0144a10 0x7fdce00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 msgr2=0x7fdce014fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 0x7fdce014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdce0144a10 msgr2=0x7fdce00b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdce0144a10 0x7fdce00b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 -- 172.21.2.116:0/239961998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc800b1e0 con 0x7fdce0141510 2021-11-21T22:01:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdce4c69700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdce0144a10 0x7fdce00b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdf7fe700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0141510 0x7fdce00b1320 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fdcc8002f70 tx=0x7fdcc8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.242+0000 7fdcdd7fa700 1 -- 172.21.2.116:0/239961998 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc80068b0 con 0x7fdce0141510 2021-11-21T22:01:58.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdd7fa700 1 -- 172.21.2.116:0/239961998 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc800de20 con 0x7fdce0141510 2021-11-21T22:01:58.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce00b6590 con 0x7fdce0141510 2021-11-21T22:01:58.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdd7fa700 1 -- 172.21.2.116:0/239961998 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc801e460 con 0x7fdce0141510 2021-11-21T22:01:58.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce00b5fe0 con 0x7fdce0141510 2021-11-21T22:01:58.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce00b5110 con 0x7fdce0141510 2021-11-21T22:01:58.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdd7fa700 1 -- 172.21.2.116:0/239961998 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdcc8025030 con 0x7fdce0141510 2021-11-21T22:01:58.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdd7fa700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdcc4053580 0x7fdcc4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:58.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdd7fa700 1 -- 172.21.2.116:0/239961998 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdcc806c910 con 0x7fdce0141510 2021-11-21T22:01:58.261 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdffff700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdcc4053580 0x7fdcc4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:58.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdffff700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdcc4053580 0x7fdcc4055a40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fdcd000c000 tx=0x7fdcd0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:58.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.246+0000 7fdcdd7fa700 1 -- 172.21.2.116:0/239961998 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdce00b5110 con 0x7fdce0141510 2021-11-21T22:01:58.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2021-11-21T22:01:58.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:58.342+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fdce00b5110 con 0x7fdce0141510 2021-11-21T22:01:59.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.246+0000 7fdcdd7fa700 1 -- 172.21.2.116:0/239961998 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdce00b5110 con 0x7fdce0141510 2021-11-21T22:01:59.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdcc4053580 msgr2=0x7fdcc4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdcc4053580 0x7fdcc4055a40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fdcd000c000 tx=0x7fdcd0006df0).stop 2021-11-21T22:01:59.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0141510 msgr2=0x7fdce00b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0141510 0x7fdce00b1320 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fdcc8002f70 tx=0x7fdcc8015000).stop 2021-11-21T22:01:59.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 shutdown_connections 2021-11-21T22:01:59.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdce0144a10 0x7fdce00b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdcc4053580 0x7fdcc4055a40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdce0141510 0x7fdce00b1320 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 --2- 172.21.2.116:0/239961998 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdce00adc10 0x7fdce014fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 >> 172.21.2.116:0/239961998 conn(0x7fdce005bd20 msgr2=0x7fdce013dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 shutdown_connections 2021-11-21T22:01:59.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.250+0000 7fdce66cc700 1 -- 172.21.2.116:0/239961998 wait complete. 2021-11-21T22:01:59.258 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T22:01:59.268 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T22:01:59.268 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T22:01:59.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2bbb22700 1 Processor -- start 2021-11-21T22:01:59.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2bbb22700 1 -- start start 2021-11-21T22:01:59.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2bbb22700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa2b41443a0 0x7fa2b4144810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2bbb22700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b40b29b0 con 0x7fa2b40b3320 2021-11-21T22:01:59.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2bbb22700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b4067c30 con 0x7fa2b4067850 2021-11-21T22:01:59.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2bbb22700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b40ab720 con 0x7fa2b41443a0 2021-11-21T22:01:59.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2ba0bf700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa2b41443a0 0x7fa2b4144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2ba0bf700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa2b41443a0 0x7fa2b4144810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51978/0 (socket says 172.21.2.116:51978) 2021-11-21T22:01:59.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2ba0bf700 1 -- 172.21.2.116:0/1866140336 learned_addr learned my addr 172.21.2.116:0/1866140336 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:59.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.314+0000 7fa2b90bd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa2b40b3320 0x7fa2b40ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35700/0 (socket says 172.21.2.116:35700) 2021-11-21T22:01:59.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2ba0bf700 1 -- 172.21.2.116:0/1866140336 >> v1:172.21.2.117:6789/0 conn(0x7fa2b4067850 legacy=0x7fa2b40b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2ba0bf700 1 -- 172.21.2.116:0/1866140336 >> v1:172.21.2.116:6789/0 conn(0x7fa2b40b3320 legacy=0x7fa2b40ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2ba0bf700 1 -- 172.21.2.116:0/1866140336 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40b3780 con 0x7fa2b41443a0 2021-11-21T22:01:59.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2ba0bf700 1 --2- 172.21.2.116:0/1866140336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b41443a0 0x7fa2b4144810 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7fa2b4064400 tx=0x7fa2ac009290).ready entity=mon.0 client_cookie=b8145c2fa111ffe3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b88bc700 1 -- 172.21.2.116:0/1866140336 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2ac00b830 con 0x7fa2b41443a0 2021-11-21T22:01:59.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b88bc700 1 -- 172.21.2.116:0/1866140336 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac00b990 con 0x7fa2b41443a0 2021-11-21T22:01:59.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b88bc700 1 -- 172.21.2.116:0/1866140336 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2ac00bb00 con 0x7fa2b41443a0 2021-11-21T22:01:59.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/1866140336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b41443a0 msgr2=0x7fa2b4144810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/1866140336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b41443a0 0x7fa2b4144810 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7fa2b4064400 tx=0x7fa2ac009290).stop 2021-11-21T22:01:59.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/1866140336 shutdown_connections 2021-11-21T22:01:59.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/1866140336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b41443a0 0x7fa2b4144810 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/1866140336 >> 172.21.2.116:0/1866140336 conn(0x7fa2b4064d60 msgr2=0x7fa2b4065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/1866140336 shutdown_connections 2021-11-21T22:01:59.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/1866140336 wait complete. 2021-11-21T22:01:59.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 Processor -- start 2021-11-21T22:01:59.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- start start 2021-11-21T22:01:59.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b4067850 0x7fa2b40a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2b40b3320 0x7fa2b4145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2b41443a0 0x7fa2b40afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40af7b0 con 0x7fa2b4067850 2021-11-21T22:01:59.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40af9e0 con 0x7fa2b41443a0 2021-11-21T22:01:59.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40a2d70 con 0x7fa2b40b3320 2021-11-21T22:01:59.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b4067850 0x7fa2b40a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b4067850 0x7fa2b40a34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:51982/0 (socket says 172.21.2.116:51982) 2021-11-21T22:01:59.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 -- 172.21.2.116:0/715577892 learned_addr learned my addr 172.21.2.116:0/715577892 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:59.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 -- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2b40b3320 msgr2=0x7fa2b4145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2b40b3320 0x7fa2b4145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 -- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2b41443a0 msgr2=0x7fa2b40afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2b41443a0 0x7fa2b40afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 -- 172.21.2.116:0/715577892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac009530 con 0x7fa2b4067850 2021-11-21T22:01:59.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2b98be700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b4067850 0x7fa2b40a34b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fa2b0002f90 tx=0x7fa2b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2a6ffd700 1 -- 172.21.2.116:0/715577892 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b0013460 con 0x7fa2b4067850 2021-11-21T22:01:59.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2a6ffd700 1 -- 172.21.2.116:0/715577892 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b00135c0 con 0x7fa2b4067850 2021-11-21T22:01:59.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b40a3010 con 0x7fa2b4067850 2021-11-21T22:01:59.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2a6ffd700 1 -- 172.21.2.116:0/715577892 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b0013920 con 0x7fa2b4067850 2021-11-21T22:01:59.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b40b0130 con 0x7fa2b4067850 2021-11-21T22:01:59.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b40a31a0 con 0x7fa2b4067850 2021-11-21T22:01:59.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.318+0000 7fa2a6ffd700 1 -- 172.21.2.116:0/715577892 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa2b000f070 con 0x7fa2b4067850 2021-11-21T22:01:59.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.322+0000 7fa2a6ffd700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2940536a0 0x7fa294055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.322+0000 7fa2a6ffd700 1 -- 172.21.2.116:0/715577892 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa2b006adf0 con 0x7fa2b4067850 2021-11-21T22:01:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.322+0000 7fa2b90bd700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2940536a0 0x7fa294055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.322+0000 7fa2b90bd700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2940536a0 0x7fa294055b60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa2b40b3200 tx=0x7fa2a80077c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.322+0000 7fa2a6ffd700 1 -- 172.21.2.116:0/715577892 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2b0068e90 con 0x7fa2b4067850 2021-11-21T22:01:59.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.434+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa2b40b7910 con 0x7fa2b4067850 2021-11-21T22:01:59.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.434+0000 7fa2a6ffd700 1 -- 172.21.2.116:0/715577892 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fa2b0047070 con 0x7fa2b4067850 2021-11-21T22:01:59.441 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:59.441 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:01:59.441 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T22:01:59.442 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.442 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T22:01:59.442 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T22:01:59.442 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.443 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:59.443 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.443 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.443 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.444 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.444 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.444 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.445 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T22:01:59.445 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T22:01:59.445 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.445 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:59.446 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.446 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.446 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.446 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.447 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.447 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.447 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T22:01:59.447 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T22:01:59.448 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.448 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:59.448 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.448 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.449 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.449 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.449 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.450 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T22:01:59.450 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T22:01:59.450 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.451 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.451 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.451 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.451 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.452 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.452 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T22:01:59.452 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T22:01:59.452 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.453 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.453 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.453 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.453 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.454 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.454 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.454 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T22:01:59.454 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T22:01:59.455 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.455 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.455 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.455 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.456 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.456 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.456 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.456 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T22:01:59.457 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T22:01:59.457 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.457 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.457 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.458 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.458 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.458 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.458 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.459 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T22:01:59.459 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T22:01:59.459 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.460 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.460 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.460 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.460 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.461 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.461 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.461 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T22:01:59.461 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T22:01:59.462 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.462 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.462 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.462 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.463 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.463 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.463 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.464 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T22:01:59.464 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T22:01:59.464 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.464 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.465 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.465 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.465 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.466 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.466 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.466 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T22:01:59.467 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T22:01:59.467 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.467 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.468 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.468 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.468 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.468 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.469 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.469 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T22:01:59.469 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T22:01:59.469 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.470 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:59.470 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:59.470 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:59.470 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.471 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.471 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.471 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T22:01:59.471 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T22:01:59.472 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.472 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:01:59.472 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:01:59.473 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:01:59.473 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.473 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.473 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.473 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T22:01:59.474 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T22:01:59.474 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.474 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:59.474 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:01:59.475 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:01:59.475 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.475 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.476 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.476 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T22:01:59.476 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T22:01:59.477 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.477 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:59.477 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:59.477 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.478 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.478 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:01:59.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:01:59.478 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T22:01:59.479 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T22:01:59.479 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:01:59.479 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:01:59.479 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:01:59.480 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:01:59.480 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.480 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:01:59.480 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:01:59.480 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:01:59.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2940536a0 msgr2=0x7fa294055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2940536a0 0x7fa294055b60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa2b40b3200 tx=0x7fa2a80077c0).stop 2021-11-21T22:01:59.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b4067850 msgr2=0x7fa2b40a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b4067850 0x7fa2b40a34b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fa2b0002f90 tx=0x7fa2b0006df0).stop 2021-11-21T22:01:59.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 shutdown_connections 2021-11-21T22:01:59.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa2b41443a0 0x7fa2b40afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa2b40b3320 0x7fa2b4145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa2940536a0 0x7fa294055b60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 --2- 172.21.2.116:0/715577892 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa2b4067850 0x7fa2b40a34b0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 >> 172.21.2.116:0/715577892 conn(0x7fa2b4064d60 msgr2=0x7fa2b40b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 shutdown_connections 2021-11-21T22:01:59.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.438+0000 7fa2bbb22700 1 -- 172.21.2.116:0/715577892 wait complete. 2021-11-21T22:01:59.486 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:01:59.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 Processor -- start 2021-11-21T22:01:59.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- start start 2021-11-21T22:01:59.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 0x7fc240144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2400a6e80 con 0x7fc2400a6aa0 2021-11-21T22:01:59.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24014a7b0 con 0x7fc2400a86b0 2021-11-21T22:01:59.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400a8bc0 con 0x7fc240062410 2021-11-21T22:01:59.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2464e6700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc2400a6aa0 0x7fc2401470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35712/0 (socket says 172.21.2.116:35712) 2021-11-21T22:01:59.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2464e6700 1 -- 172.21.2.116:0/3003887589 learned_addr learned my addr 172.21.2.116:0/3003887589 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:59.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767898180 0 0) 0x7fc2400a6e80 con 0x7fc2400a6aa0 2021-11-21T22:01:59.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc228003260 con 0x7fc2400a6aa0 2021-11-21T22:01:59.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2843666071 0 0) 0x7fc228003260 con 0x7fc2400a6aa0 2021-11-21T22:01:59.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 msgr2=0x7fc240144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 0x7fc240144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 >> v1:172.21.2.117:6789/0 conn(0x7fc2400a86b0 legacy=0x7fc2401443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24014aa10 con 0x7fc2400a6aa0 2021-11-21T22:01:59.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fc234001680 con 0x7fc2400a6aa0 2021-11-21T22:01:59.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 >> v1:172.21.2.116:6789/0 conn(0x7fc2400a6aa0 legacy=0x7fc2401470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280037d0 0x7fc228003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 0x7fc228004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228004bf0 0x7fc22800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400a8bc0 con 0x7fc228004bf0 2021-11-21T22:01:59.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280052b0 con 0x7fc2280037d0 2021-11-21T22:01:59.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22800bc40 con 0x7fc240062410 2021-11-21T22:01:59.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228004bf0 0x7fc22800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2464e6700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280037d0 0x7fc228003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc245ce5700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 0x7fc228004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 -- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 msgr2=0x7fc228004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 0x7fc228004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 -- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280037d0 msgr2=0x7fc228003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc2280037d0 0x7fc228003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 -- 172.21.2.116:0/3003887589 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228003260 con 0x7fc228004bf0 2021-11-21T22:01:59.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc245ce5700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc240062410 0x7fc228004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:01:59.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228004bf0 0x7fc22800b750 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fc2380032f0 tx=0x7fc238006f90).ready entity=mon.0 client_cookie=3ca225407fef87a3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc23800b460 con 0x7fc228004bf0 2021-11-21T22:01:59.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800b5c0 con 0x7fc228004bf0 2021-11-21T22:01:59.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc2454e4700 1 -- 172.21.2.116:0/3003887589 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc23800b920 con 0x7fc228004bf0 2021-11-21T22:01:59.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- 172.21.2.116:0/3003887589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228004bf0 msgr2=0x7fc22800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 --2- 172.21.2.116:0/3003887589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc228004bf0 0x7fc22800b750 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fc2380032f0 tx=0x7fc238006f90).stop 2021-11-21T22:01:59.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc246ce7700 1 -- 172.21.2.116:0/3003887589 reap_dead start 2021-11-21T22:01:59.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- 172.21.2.116:0/3003887589 shutdown_connections 2021-11-21T22:01:59.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- 172.21.2.116:0/3003887589 >> 172.21.2.116:0/3003887589 conn(0x7fc24005bd20 msgr2=0x7fc24005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- 172.21.2.116:0/3003887589 shutdown_connections 2021-11-21T22:01:59.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.534+0000 7fc24874a700 1 -- 172.21.2.116:0/3003887589 wait complete. 2021-11-21T22:01:59.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 Processor -- start 2021-11-21T22:01:59.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 -- start start 2021-11-21T22:01:59.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc228004bf0 0x7fc24013c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc24013f290 0x7fc240159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc24015a020 0x7fc24013d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24013aa60 con 0x7fc24013f290 2021-11-21T22:01:59.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24013eb90 con 0x7fc228004bf0 2021-11-21T22:01:59.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24013ee30 con 0x7fc24015a020 2021-11-21T22:01:59.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc24013f290 0x7fc240159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc24013f290 0x7fc240159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52000/0 (socket says 172.21.2.116:52000) 2021-11-21T22:01:59.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 -- 172.21.2.116:0/2603497520 learned_addr learned my addr 172.21.2.116:0/2603497520 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:59.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 -- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc24015a020 msgr2=0x7fc24013d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc24015a020 0x7fc24013d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 -- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc228004bf0 msgr2=0x7fc24013c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc228004bf0 0x7fc24013c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 -- 172.21.2.116:0/2603497520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24013a5c0 con 0x7fc24013f290 2021-11-21T22:01:59.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc245ce5700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc24013f290 0x7fc240159b30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fc23c008000 tx=0x7fc23c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc2337fe700 1 -- 172.21.2.116:0/2603497520 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc23c007e60 con 0x7fc24013f290 2021-11-21T22:01:59.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc2337fe700 1 -- 172.21.2.116:0/2603497520 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c00f440 con 0x7fc24013f290 2021-11-21T22:01:59.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc2337fe700 1 -- 172.21.2.116:0/2603497520 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc23c00f7a0 con 0x7fc24013f290 2021-11-21T22:01:59.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24013cb40 con 0x7fc24013f290 2021-11-21T22:01:59.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24015a4b0 con 0x7fc24013f290 2021-11-21T22:01:59.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24013a750 con 0x7fc24013f290 2021-11-21T22:01:59.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc2337fe700 1 -- 172.21.2.116:0/2603497520 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc23c00f900 con 0x7fc24013f290 2021-11-21T22:01:59.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc2337fe700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc220053690 0x7fc220055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc2337fe700 1 -- 172.21.2.116:0/2603497520 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc23c06b720 con 0x7fc24013f290 2021-11-21T22:01:59.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.538+0000 7fc2464e6700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc220053690 0x7fc220055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.542+0000 7fc2464e6700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc220053690 0x7fc220055b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fc23400b000 tx=0x7fc240062060).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.542+0000 7fc2337fe700 1 -- 172.21.2.116:0/2603497520 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc23c042bb0 con 0x7fc24013f290 2021-11-21T22:01:59.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2400573d0 con 0x7fc24013f290 2021-11-21T22:01:59.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc2337fe700 1 -- 172.21.2.116:0/2603497520 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fc23c0461d0 con 0x7fc24013f290 2021-11-21T22:01:59.631 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:01:59.632 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":117,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:01:59.240503+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:01:59.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc220053690 msgr2=0x7fc220055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc220053690 0x7fc220055b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fc23400b000 tx=0x7fc240062060).stop 2021-11-21T22:01:59.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc24013f290 msgr2=0x7fc240159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc24013f290 0x7fc240159b30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fc23c008000 tx=0x7fc23c009040).stop 2021-11-21T22:01:59.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 shutdown_connections 2021-11-21T22:01:59.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc24015a020 0x7fc24013d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc220053690 0x7fc220055b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc24013f290 0x7fc240159b30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 --2- 172.21.2.116:0/2603497520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc228004bf0 0x7fc24013c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 >> 172.21.2.116:0/2603497520 conn(0x7fc24005bd20 msgr2=0x7fc240148c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 shutdown_connections 2021-11-21T22:01:59.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.626+0000 7fc24874a700 1 -- 172.21.2.116:0/2603497520 wait complete. 2021-11-21T22:01:59.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T22:01:59.639 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:01:59.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.686+0000 7f357892c700 1 Processor -- start 2021-11-21T22:01:59.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.686+0000 7f357892c700 1 -- start start 2021-11-21T22:01:59.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.686+0000 7f357892c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f35740adc10 0x7f35740b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.686+0000 7f357892c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740b3c50 con 0x7f3574144a10 2021-11-21T22:01:59.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.686+0000 7f357892c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740b3ef0 con 0x7f35740b3690 2021-11-21T22:01:59.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.686+0000 7f357892c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740b4190 con 0x7f35740adc10 2021-11-21T22:01:59.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f3574144a10 0x7f3574141020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35732/0 (socket says 172.21.2.116:35732) 2021-11-21T22:01:59.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 -- 172.21.2.116:0/2343363732 learned_addr learned my addr 172.21.2.116:0/2343363732 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:01:59.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112590886 0 0) 0x7f35740b3c50 con 0x7f3574144a10 2021-11-21T22:01:59.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3554003260 con 0x7f3574144a10 2021-11-21T22:01:59.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370974351 0 0) 0x7f3554003260 con 0x7f3574144a10 2021-11-21T22:01:59.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 >> v2:172.21.2.117:3300/0 conn(0x7f35740adc10 msgr2=0x7f35740b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 --2- 172.21.2.116:0/2343363732 >> v2:172.21.2.117:3300/0 conn(0x7f35740adc10 0x7f35740b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 >> v1:172.21.2.117:6789/0 conn(0x7f35740b3690 legacy=0x7f357413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574140b80 con 0x7f3574144a10 2021-11-21T22:01:59.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f3564001660 con 0x7f3574144a10 2021-11-21T22:01:59.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 >> v1:172.21.2.116:6789/0 conn(0x7f3574144a10 legacy=0x7f3574141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3554003820 0x7f3554003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3554004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3554005050 0x7f355400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740b4190 con 0x7f3554004500 2021-11-21T22:01:59.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355400bf40 con 0x7f3554003820 2021-11-21T22:01:59.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355400c100 con 0x7f3554005050 2021-11-21T22:01:59.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3554004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 -- 172.21.2.116:0/2343363732 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3554005050 msgr2=0x7f355400b750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:01:59.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3554005050 0x7f355400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 -- 172.21.2.116:0/2343363732 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3554003820 msgr2=0x7f3554003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3554003820 0x7f3554003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 -- 172.21.2.116:0/2343363732 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554003260 con 0x7f3554004500 2021-11-21T22:01:59.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3572d9d700 1 -- 172.21.2.116:0/2343363732 reap_dead start 2021-11-21T22:01:59.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f3571d9b700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3554004980 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f35740b7000 tx=0x7f3574064980).ready entity=mon.0 client_cookie=9c8fd6e2dc8cb2c1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35640070d0 con 0x7f3554004500 2021-11-21T22:01:59.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3564007230 con 0x7f3554004500 2021-11-21T22:01:59.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357159a700 1 -- 172.21.2.116:0/2343363732 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f356400c8c0 con 0x7f3554004500 2021-11-21T22:01:59.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- 172.21.2.116:0/2343363732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 msgr2=0x7f3554004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3554004980 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f35740b7000 tx=0x7f3574064980).stop 2021-11-21T22:01:59.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- 172.21.2.116:0/2343363732 shutdown_connections 2021-11-21T22:01:59.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 --2- 172.21.2.116:0/2343363732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3554004980 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- 172.21.2.116:0/2343363732 >> 172.21.2.116:0/2343363732 conn(0x7f357405bd20 msgr2=0x7f357405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- 172.21.2.116:0/2343363732 shutdown_connections 2021-11-21T22:01:59.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- 172.21.2.116:0/2343363732 wait complete. 2021-11-21T22:01:59.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 Processor -- start 2021-11-21T22:01:59.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- start start 2021-11-21T22:01:59.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3574111770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35740b6a10 0x7f35741109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3574110ee0 0x7f357413c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740b0690 con 0x7f3554004500 2021-11-21T22:01:59.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740b13e0 con 0x7f3574110ee0 2021-11-21T22:01:59.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740b1140 con 0x7f35740b6a10 2021-11-21T22:01:59.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3574111770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3574111770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52018/0 (socket says 172.21.2.116:52018) 2021-11-21T22:01:59.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 -- 172.21.2.116:0/1187359736 learned_addr learned my addr 172.21.2.116:0/1187359736 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:59.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 -- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35740b6a10 msgr2=0x7f35741109f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:59.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35740b6a10 0x7f35741109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 -- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3574110ee0 msgr2=0x7f357413c870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:59.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3574110ee0 0x7f357413c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 -- 172.21.2.116:0/1187359736 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740b08f0 con 0x7f3554004500 2021-11-21T22:01:59.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357259c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3574111770 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f355c002f70 tx=0x7f355c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f356b7fe700 1 -- 172.21.2.116:0/1187359736 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f355c010960 con 0x7f3554004500 2021-11-21T22:01:59.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f356b7fe700 1 -- 172.21.2.116:0/1187359736 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c010ac0 con 0x7f3554004500 2021-11-21T22:01:59.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357413d7f0 con 0x7f3554004500 2021-11-21T22:01:59.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357414d7f0 con 0x7f3554004500 2021-11-21T22:01:59.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.690+0000 7f356b7fe700 1 -- 172.21.2.116:0/1187359736 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f355c010e20 con 0x7f3554004500 2021-11-21T22:01:59.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.694+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740b6e30 con 0x7f3554004500 2021-11-21T22:01:59.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.694+0000 7f356b7fe700 1 -- 172.21.2.116:0/1187359736 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f355c01a420 con 0x7f3554004500 2021-11-21T22:01:59.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.694+0000 7f356b7fe700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3558053690 0x7f3558055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.694+0000 7f356b7fe700 1 -- 172.21.2.116:0/1187359736 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f355c015020 con 0x7f3554004500 2021-11-21T22:01:59.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.694+0000 7f3571d9b700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3558053690 0x7f3558055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.694+0000 7f3571d9b700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3558053690 0x7f3558055b50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f35740b7000 tx=0x7f3564013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.694+0000 7f356b7fe700 1 -- 172.21.2.116:0/1187359736 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f355c046aa0 con 0x7f3554004500 2021-11-21T22:01:59.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f35740b0a80 con 0x7f3554004500 2021-11-21T22:01:59.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f356b7fe700 1 -- 172.21.2.116:0/1187359736 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v117) v1 ==== 110+0+10 (secure 0 0 0) 0x7f355c04a0c0 con 0x7f3554004500 2021-11-21T22:01:59.783 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:01:59.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3558053690 msgr2=0x7f3558055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3558053690 0x7f3558055b50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f35740b7000 tx=0x7f3564013000).stop 2021-11-21T22:01:59.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 msgr2=0x7f3574111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3574111770 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f355c002f70 tx=0x7f355c006ec0).stop 2021-11-21T22:01:59.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 shutdown_connections 2021-11-21T22:01:59.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3574110ee0 0x7f357413c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35740b6a10 0x7f35741109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3558053690 0x7f3558055b50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 --2- 172.21.2.116:0/1187359736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3554004500 0x7f3574111770 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 >> 172.21.2.116:0/1187359736 conn(0x7f357405bd20 msgr2=0x7f3574143c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 shutdown_connections 2021-11-21T22:01:59.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.778+0000 7f357892c700 1 -- 172.21.2.116:0/1187359736 wait complete. 2021-11-21T22:01:59.789 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:01:59.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 Processor -- start 2021-11-21T22:01:59.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- start start 2021-11-21T22:01:59.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8bd8144810 0x7f8bd81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd80ac600 0x7f8bd80aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd8147e80 con 0x7f8bd80abce0 2021-11-21T22:01:59.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd80b1490 con 0x7f8bd8144810 2021-11-21T22:01:59.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80b1fd0 con 0x7f8bd80ac600 2021-11-21T22:01:59.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8bd8144810 0x7f8bd81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8bd8144810 0x7f8bd81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52026/0 (socket says 172.21.2.116:52026) 2021-11-21T22:01:59.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/824827584 learned_addr learned my addr 172.21.2.116:0/824827584 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:59.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/824827584 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd80ac600 msgr2=0x7f8bd80aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:01:59.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 --2- 172.21.2.116:0/824827584 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd80ac600 0x7f8bd80aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/824827584 >> v1:172.21.2.116:6789/0 conn(0x7f8bd80abce0 legacy=0x7f8bd80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/824827584 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80af5a0 con 0x7f8bd8144810 2021-11-21T22:01:59.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdcc2e700 1 --2- 172.21.2.116:0/824827584 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd8144810 0x7f8bd81478c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f8bc0002f70 tx=0x7f8bc0006eb0).ready entity=mon.0 client_cookie=f469f0685ca8c48d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bd77fe700 1 -- 172.21.2.116:0/824827584 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bc000d5c0 con 0x7f8bd8144810 2021-11-21T22:01:59.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bd77fe700 1 -- 172.21.2.116:0/824827584 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0000e90 con 0x7f8bd8144810 2021-11-21T22:01:59.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bd77fe700 1 -- 172.21.2.116:0/824827584 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bc000d950 con 0x7f8bd8144810 2021-11-21T22:01:59.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- 172.21.2.116:0/824827584 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd8144810 msgr2=0x7f8bd81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 --2- 172.21.2.116:0/824827584 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd8144810 0x7f8bd81478c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f8bc0002f70 tx=0x7f8bc0006eb0).stop 2021-11-21T22:01:59.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- 172.21.2.116:0/824827584 shutdown_connections 2021-11-21T22:01:59.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 --2- 172.21.2.116:0/824827584 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd80ac600 0x7f8bd80aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 --2- 172.21.2.116:0/824827584 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd8144810 0x7f8bd81478c0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- 172.21.2.116:0/824827584 >> 172.21.2.116:0/824827584 conn(0x7f8bd805bd20 msgr2=0x7f8bd805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- 172.21.2.116:0/824827584 shutdown_connections 2021-11-21T22:01:59.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.838+0000 7f8bdee92700 1 -- 172.21.2.116:0/824827584 wait complete. 2021-11-21T22:01:59.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 Processor -- start 2021-11-21T22:01:59.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 -- start start 2021-11-21T22:01:59.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd80abce0 0x7f8bd8143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8bd80ac600 0x7f8bd8159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd8144810 0x7f8bd815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8145360 con 0x7f8bd80abce0 2021-11-21T22:01:59.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8156570 con 0x7f8bd80ac600 2021-11-21T22:01:59.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8149820 con 0x7f8bd8144810 2021-11-21T22:01:59.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd80abce0 0x7f8bd8143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd80abce0 0x7f8bd8143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52034/0 (socket says 172.21.2.116:52034) 2021-11-21T22:01:59.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/3929978615 learned_addr learned my addr 172.21.2.116:0/3929978615 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:01:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd8144810 msgr2=0x7f8bd815a3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd8144810 0x7f8bd815a3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8bd80ac600 msgr2=0x7f8bd8159ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:01:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8bd80ac600 0x7f8bd8159ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 -- 172.21.2.116:0/3929978615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc000b1e0 con 0x7f8bd80abce0 2021-11-21T22:01:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdcc2e700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd80abce0 0x7f8bd8143670 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f8bc0002f70 tx=0x7f8bc0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bd5ffb700 1 -- 172.21.2.116:0/3929978615 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bc00118a0 con 0x7f8bd80abce0 2021-11-21T22:01:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bd5ffb700 1 -- 172.21.2.116:0/3929978615 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0011a00 con 0x7f8bd80abce0 2021-11-21T22:01:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bd5ffb700 1 -- 172.21.2.116:0/3929978615 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bc0011d60 con 0x7f8bd80abce0 2021-11-21T22:01:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd8149ac0 con 0x7f8bd80abce0 2021-11-21T22:01:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd81494a0 con 0x7f8bd80abce0 2021-11-21T22:01:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd8149f60 con 0x7f8bd80abce0 2021-11-21T22:01:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bd5ffb700 1 -- 172.21.2.116:0/3929978615 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8bc0025030 con 0x7f8bd80abce0 2021-11-21T22:01:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bd5ffb700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8bbc053610 0x7f8bbc055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bd5ffb700 1 -- 172.21.2.116:0/3929978615 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8bc006c960 con 0x7f8bd80abce0 2021-11-21T22:01:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.842+0000 7f8bd7fff700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8bbc053610 0x7f8bbc055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:01:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.846+0000 7f8bd7fff700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8bbc053610 0x7f8bbc055ad0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f8bd80b4e00 tx=0x7f8bc80077b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:01:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.846+0000 7f8bd5ffb700 1 -- 172.21.2.116:0/3929978615 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8bc0043df0 con 0x7f8bd80abce0 2021-11-21T22:01:59.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8bd8159bd0 con 0x7f8bd80abce0 2021-11-21T22:01:59.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bd5ffb700 1 -- 172.21.2.116:0/3929978615 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8bc0023020 con 0x7f8bd80abce0 2021-11-21T22:01:59.932 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:01:59.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8bbc053610 msgr2=0x7f8bbc055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8bbc053610 0x7f8bbc055ad0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f8bd80b4e00 tx=0x7f8bc80077b0).stop 2021-11-21T22:01:59.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd80abce0 msgr2=0x7f8bd8143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:01:59.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd80abce0 0x7f8bd8143670 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f8bc0002f70 tx=0x7f8bc0015000).stop 2021-11-21T22:01:59.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 shutdown_connections 2021-11-21T22:01:59.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8bd8144810 0x7f8bd815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8bd80ac600 0x7f8bd8159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:01:59.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8bbc053610 0x7f8bbc055ad0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 --2- 172.21.2.116:0/3929978615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8bd80abce0 0x7f8bd8143670 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:01:59.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 >> 172.21.2.116:0/3929978615 conn(0x7f8bd805bd20 msgr2=0x7f8bd80ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:01:59.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 shutdown_connections 2021-11-21T22:01:59.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.926+0000 7f8bdee92700 1 -- 172.21.2.116:0/3929978615 wait complete. 2021-11-21T22:01:59.939 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T22:01:59.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 Processor -- start 2021-11-21T22:01:59.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- start start 2021-11-21T22:01:59.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f09580a6aa0 0x7f095814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0958062410 0x7f09581443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:01:59.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09580a6e80 con 0x7f09580a86b0 2021-11-21T22:01:59.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580a8bc0 con 0x7f0958062410 2021-11-21T22:02:00.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095814a7b0 con 0x7f09580a6aa0 2021-11-21T22:02:00.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0958062410 0x7f09581443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:00.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0958062410 0x7f09581443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52044/0 (socket says 172.21.2.116:52044) 2021-11-21T22:02:00.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 -- 172.21.2.116:0/2426093541 learned_addr learned my addr 172.21.2.116:0/2426093541 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:00.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 -- 172.21.2.116:0/2426093541 >> v2:172.21.2.117:3300/0 conn(0x7f09580a6aa0 msgr2=0x7f095814b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:00.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 --2- 172.21.2.116:0/2426093541 >> v2:172.21.2.117:3300/0 conn(0x7f09580a6aa0 0x7f095814b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 -- 172.21.2.116:0/2426093541 >> v1:172.21.2.117:6789/0 conn(0x7f09580a86b0 legacy=0x7f09581470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:00.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 -- 172.21.2.116:0/2426093541 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095814aa10 con 0x7f0958062410 2021-11-21T22:02:00.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f0956d9d700 1 --2- 172.21.2.116:0/2426093541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09581443f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f0958061ab0 tx=0x7f0950009290).ready entity=mon.0 client_cookie=1b7869478eae62ff server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:00.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095559a700 1 -- 172.21.2.116:0/2426093541 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f095000b830 con 0x7f0958062410 2021-11-21T22:02:00.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095559a700 1 -- 172.21.2.116:0/2426093541 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095000b990 con 0x7f0958062410 2021-11-21T22:02:00.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095559a700 1 -- 172.21.2.116:0/2426093541 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f095000bb40 con 0x7f0958062410 2021-11-21T22:02:00.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- 172.21.2.116:0/2426093541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 msgr2=0x7f09581443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:00.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 --2- 172.21.2.116:0/2426093541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09581443f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f0958061ab0 tx=0x7f0950009290).stop 2021-11-21T22:02:00.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- 172.21.2.116:0/2426093541 shutdown_connections 2021-11-21T22:02:00.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 --2- 172.21.2.116:0/2426093541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09581443f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:00.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 --2- 172.21.2.116:0/2426093541 >> v2:172.21.2.117:3300/0 conn(0x7f09580a6aa0 0x7f095814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- 172.21.2.116:0/2426093541 >> 172.21.2.116:0/2426093541 conn(0x7f095805bd20 msgr2=0x7f095805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:00.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- 172.21.2.116:0/2426093541 shutdown_connections 2021-11-21T22:02:00.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- 172.21.2.116:0/2426093541 wait complete. 2021-11-21T22:02:00.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 Processor -- start 2021-11-21T22:02:00.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.986+0000 7f095cbb2700 1 -- start start 2021-11-21T22:02:00.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09580ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09580a6aa0 0x7f0958063a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09580a86b0 0x7f0958064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958140f60 con 0x7f0958062410 2021-11-21T22:02:00.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958159ba0 con 0x7f09580a6aa0 2021-11-21T22:02:00.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0958064830 con 0x7f09580a86b0 2021-11-21T22:02:00.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09580ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:00.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09580ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52046/0 (socket says 172.21.2.116:52046) 2021-11-21T22:02:00.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 -- 172.21.2.116:0/834327940 learned_addr learned my addr 172.21.2.116:0/834327940 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:00.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 -- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09580a86b0 msgr2=0x7f0958064ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:00.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09580a86b0 0x7f0958064ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 -- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09580a6aa0 msgr2=0x7f0958063a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:00.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09580a6aa0 0x7f0958063a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 -- 172.21.2.116:0/834327940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950009530 con 0x7f0958062410 2021-11-21T22:02:00.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095659c700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09580ab070 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f09580ab9d0 tx=0x7f0948006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:00.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f094f7fe700 1 -- 172.21.2.116:0/834327940 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0948013460 con 0x7f0958062410 2021-11-21T22:02:00.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f094f7fe700 1 -- 172.21.2.116:0/834327940 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09480135c0 con 0x7f0958062410 2021-11-21T22:02:00.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0958147e80 con 0x7f0958062410 2021-11-21T22:02:00.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0958147940 con 0x7f0958062410 2021-11-21T22:02:00.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f094f7fe700 1 -- 172.21.2.116:0/834327940 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0948013920 con 0x7f0958062410 2021-11-21T22:02:00.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09580573d0 con 0x7f0958062410 2021-11-21T22:02:00.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f094f7fe700 1 -- 172.21.2.116:0/834327940 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f094800f070 con 0x7f0958062410 2021-11-21T22:02:00.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f094f7fe700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f093c053570 0x7f093c055a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f094f7fe700 1 -- 172.21.2.116:0/834327940 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f094806b1c0 con 0x7f0958062410 2021-11-21T22:02:00.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.990+0000 7f0955d9b700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f093c053570 0x7f093c055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:00.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.994+0000 7f0955d9b700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f093c053570 0x7f093c055a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f0940002ef0 tx=0x7f0940006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:00.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:01:59.994+0000 7f094f7fe700 1 -- 172.21.2.116:0/834327940 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0948042540 con 0x7f0958062410 2021-11-21T22:02:00.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0958142ba0 con 0x7f0958062410 2021-11-21T22:02:00.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f094f7fe700 1 -- 172.21.2.116:0/834327940 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0948045b60 con 0x7f0958062410 2021-11-21T22:02:00.082 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:00.082 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T22:02:00.082 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:00.083 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T22:02:00.083 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T22:02:00.083 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:00.083 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:00.084 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T22:02:00.084 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T22:02:00.084 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:00.084 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T22:02:00.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f093c053570 msgr2=0x7f093c055a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:00.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f093c053570 0x7f093c055a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f0940002ef0 tx=0x7f0940006890).stop 2021-11-21T22:02:00.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 msgr2=0x7f09580ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:00.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09580ab070 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f09580ab9d0 tx=0x7f0948006df0).stop 2021-11-21T22:02:00.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 shutdown_connections 2021-11-21T22:02:00.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09580a86b0 0x7f0958064ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09580a6aa0 0x7f0958063a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f093c053570 0x7f093c055a30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:00.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 --2- 172.21.2.116:0/834327940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0958062410 0x7f09580ab070 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:00.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 >> 172.21.2.116:0/834327940 conn(0x7f095805bd20 msgr2=0x7f09580b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:00.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 shutdown_connections 2021-11-21T22:02:00.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.078+0000 7f095cbb2700 1 -- 172.21.2.116:0/834327940 wait complete. 2021-11-21T22:02:00.089 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T22:02:00.090 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:02:00.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb67bdfe700 1 Processor -- start 2021-11-21T22:02:00.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb67bdfe700 1 -- start start 2021-11-21T22:02:00.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb67bdfe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6741478c0 0x7fb6740acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb67bdfe700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb674147e80 con 0x7fb674144810 2021-11-21T22:02:00.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb67bdfe700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6740af5a0 con 0x7fb6740abce0 2021-11-21T22:02:00.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb67bdfe700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740b1490 con 0x7fb6741478c0 2021-11-21T22:02:00.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679b9a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb674144810 0x7fb6740b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35778/0 (socket says 172.21.2.116:35778) 2021-11-21T22:02:00.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679b9a700 1 -- 172.21.2.116:0/2942587091 learned_addr learned my addr 172.21.2.116:0/2942587091 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:00.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512732479 0 0) 0x7fb674147e80 con 0x7fb674144810 2021-11-21T22:02:00.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb65c003260 con 0x7fb674144810 2021-11-21T22:02:00.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320699466 0 0) 0x7fb65c003260 con 0x7fb674144810 2021-11-21T22:02:00.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6741478c0 msgr2=0x7fb6740acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:00.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6741478c0 0x7fb6740acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 >> v1:172.21.2.117:6789/0 conn(0x7fb6740abce0 legacy=0x7fb6740ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:00.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740b1fd0 con 0x7fb674144810 2021-11-21T22:02:00.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb670001680 con 0x7fb674144810 2021-11-21T22:02:00.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 >> v1:172.21.2.116:6789/0 conn(0x7fb674144810 legacy=0x7fb6740b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:00.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6741478c0 0x7fb65c003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb65c003ef0 0x7fb65c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb65c004bf0 0x7fb65c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740b1490 con 0x7fb65c003ef0 2021-11-21T22:02:00.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0052f0 con 0x7fb65c004bf0 2021-11-21T22:02:00.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c00bc40 con 0x7fb6741478c0 2021-11-21T22:02:00.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679399700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb65c003ef0 0x7fb65c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:00.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679399700 1 -- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6741478c0 msgr2=0x7fb65c003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:00.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679399700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6741478c0 0x7fb65c003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679399700 1 -- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb65c004bf0 msgr2=0x7fb65c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:00.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679399700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb65c004bf0 0x7fb65c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679399700 1 -- 172.21.2.116:0/2942587091 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c003260 con 0x7fb65c003ef0 2021-11-21T22:02:00.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb679399700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb65c003ef0 0x7fb65c004500 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fb664000f70 tx=0x7fb6640079e0).ready entity=mon.0 client_cookie=4f6355e547ac9cdf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:00.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb664007cf0 con 0x7fb65c003ef0 2021-11-21T22:02:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb664001d50 con 0x7fb65c003ef0 2021-11-21T22:02:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.138+0000 7fb678b98700 1 -- 172.21.2.116:0/2942587091 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb664014600 con 0x7fb65c003ef0 2021-11-21T22:02:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/2942587091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb65c003ef0 msgr2=0x7fb65c004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 --2- 172.21.2.116:0/2942587091 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb65c003ef0 0x7fb65c004500 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fb664000f70 tx=0x7fb6640079e0).stop 2021-11-21T22:02:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 -- 172.21.2.116:0/2942587091 reap_dead start 2021-11-21T22:02:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/2942587091 shutdown_connections 2021-11-21T22:02:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/2942587091 >> 172.21.2.116:0/2942587091 conn(0x7fb67405bd20 msgr2=0x7fb67405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/2942587091 shutdown_connections 2021-11-21T22:02:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/2942587091 wait complete. 2021-11-21T22:02:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 Processor -- start 2021-11-21T22:02:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- start start 2021-11-21T22:02:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb65c003ef0 0x7fb674146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6740a6760 0x7fb67415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6740ab460 0x7fb67415a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67415ae50 con 0x7fb6740ab460 2021-11-21T22:02:00.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb674147500 con 0x7fb6740a6760 2021-11-21T22:02:00.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb674064960 con 0x7fb65c003ef0 2021-11-21T22:02:00.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6740ab460 0x7fb67415a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:00.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6740ab460 0x7fb67415a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52070/0 (socket says 172.21.2.116:52070) 2021-11-21T22:02:00.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 -- 172.21.2.116:0/1535847227 learned_addr learned my addr 172.21.2.116:0/1535847227 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:00.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 -- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb65c003ef0 msgr2=0x7fb674146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:00.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb65c003ef0 0x7fb674146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 -- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6740a6760 msgr2=0x7fb67415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:00.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6740a6760 0x7fb67415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:00.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 -- 172.21.2.116:0/1535847227 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674064c00 con 0x7fb6740ab460 2021-11-21T22:02:00.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67a39b700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6740ab460 0x7fb67415a960 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fb670001110 tx=0x7fb66c00a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:00.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb662ffd700 1 -- 172.21.2.116:0/1535847227 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb66c00cea0 con 0x7fb6740ab460 2021-11-21T22:02:00.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb662ffd700 1 -- 172.21.2.116:0/1535847227 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c005c30 con 0x7fb6740ab460 2021-11-21T22:02:00.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb674159fd0 con 0x7fb6740ab460 2021-11-21T22:02:00.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb662ffd700 1 -- 172.21.2.116:0/1535847227 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb66c007070 con 0x7fb6740ab460 2021-11-21T22:02:00.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6741471f0 con 0x7fb6740ab460 2021-11-21T22:02:00.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67415a160 con 0x7fb6740ab460 2021-11-21T22:02:00.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb662ffd700 1 -- 172.21.2.116:0/1535847227 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb66c01b060 con 0x7fb6740ab460 2021-11-21T22:02:00.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb662ffd700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb654053610 0x7fb654055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:00.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb662ffd700 1 -- 172.21.2.116:0/1535847227 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb66c06ad50 con 0x7fb6740ab460 2021-11-21T22:02:00.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.142+0000 7fb679b9a700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb654053610 0x7fb654055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:00.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.146+0000 7fb679b9a700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb654053610 0x7fb654055ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fb6740b4e00 tx=0x7fb670001660).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:00.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.146+0000 7fb662ffd700 1 -- 172.21.2.116:0/1535847227 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb66c0421e0 con 0x7fb6740ab460 2021-11-21T22:02:00.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:00.230+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb6740573d0 con 0x7fb6740ab460 2021-11-21T22:02:01.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.186+0000 7fb662ffd700 1 -- 172.21.2.116:0/1535847227 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v118) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb66c019070 con 0x7fb6740ab460 2021-11-21T22:02:01.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb654053610 msgr2=0x7fb654055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:01.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb654053610 0x7fb654055ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fb6740b4e00 tx=0x7fb670001660).stop 2021-11-21T22:02:01.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6740ab460 msgr2=0x7fb67415a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:01.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6740ab460 0x7fb67415a960 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fb670001110 tx=0x7fb66c00a060).stop 2021-11-21T22:02:01.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 shutdown_connections 2021-11-21T22:02:01.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb654053610 0x7fb654055ad0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:01.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6740ab460 0x7fb67415a960 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6740a6760 0x7fb67415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 --2- 172.21.2.116:0/1535847227 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb65c003ef0 0x7fb674146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 >> 172.21.2.116:0/1535847227 conn(0x7fb67405bd20 msgr2=0x7fb674149070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 shutdown_connections 2021-11-21T22:02:01.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.190+0000 7fb67bdfe700 1 -- 172.21.2.116:0/1535847227 wait complete. 2021-11-21T22:02:01.197 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:02:01.201 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:02:01.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522efa8700 1 Processor -- start 2021-11-21T22:02:01.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522efa8700 1 -- start start 2021-11-21T22:02:01.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522efa8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f52280abce0 0x7f52281478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522efa8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 0x7f52280aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522efa8700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5228147e80 con 0x7f5228144810 2021-11-21T22:02:01.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522efa8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52280b1490 con 0x7f52280abce0 2021-11-21T22:02:01.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522efa8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52280b1fd0 con 0x7f52280ac600 2021-11-21T22:02:01.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522cd44700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5228144810 0x7f52280b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35800/0 (socket says 172.21.2.116:35800) 2021-11-21T22:02:01.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f522cd44700 1 -- 172.21.2.116:0/851467147 learned_addr learned my addr 172.21.2.116:0/851467147 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:01.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.250+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156777369 0 0) 0x7f5228147e80 con 0x7f5228144810 2021-11-21T22:02:01.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5208003260 con 0x7f5228144810 2021-11-21T22:02:01.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 0x7f52280aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:01.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f5227fff700 1 --2- 172.21.2.116:0/851467147 >> v2:172.21.2.117:3300/0 conn(0x7f52280abce0 0x7f52281478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:01.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343530909 0 0) 0x7f5208003260 con 0x7f5228144810 2021-11-21T22:02:01.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 msgr2=0x7f52280aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:01.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 0x7f52280aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:01.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 >> v2:172.21.2.117:3300/0 conn(0x7f52280abce0 msgr2=0x7f52281478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:01.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 --2- 172.21.2.116:0/851467147 >> v2:172.21.2.117:3300/0 conn(0x7f52280abce0 0x7f52281478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:01.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280af5a0 con 0x7f5228144810 2021-11-21T22:02:01.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f5218001660 con 0x7f5228144810 2021-11-21T22:02:01.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 >> v1:172.21.2.116:6789/0 conn(0x7f5228144810 legacy=0x7f52280b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:01.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 0x7f5208003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5208003d30 0x7f5208004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5208004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52280b1490 con 0x7f52080049f0 2021-11-21T22:02:01.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52280b1fd0 con 0x7f5208003d30 2021-11-21T22:02:01.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520800b800 con 0x7f52280ac600 2021-11-21T22:02:01.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5208004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:01.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 0x7f5208003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:01.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 -- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 msgr2=0x7f5208003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:01.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 0x7f5208003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:01.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 -- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5208003d30 msgr2=0x7f5208004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:01.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5208003d30 0x7f5208004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:01.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 -- 172.21.2.116:0/851467147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208003260 con 0x7f52080049f0 2021-11-21T22:02:01.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f52280ac600 0x7f5208003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:01.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5208004e70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f522000a000 tx=0x7f5220008040).ready entity=mon.0 client_cookie=b48c890c9ac3c980 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:01.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f522000c070 con 0x7f52080049f0 2021-11-21T22:02:01.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522000e620 con 0x7f52080049f0 2021-11-21T22:02:01.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f52277fe700 1 -- 172.21.2.116:0/851467147 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f522000e980 con 0x7f52080049f0 2021-11-21T22:02:01.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- 172.21.2.116:0/851467147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 msgr2=0x7f5208004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:01.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 --2- 172.21.2.116:0/851467147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5208004e70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f522000a000 tx=0x7f5220008040).stop 2021-11-21T22:02:01.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 -- 172.21.2.116:0/851467147 reap_dead start 2021-11-21T22:02:01.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- 172.21.2.116:0/851467147 shutdown_connections 2021-11-21T22:02:01.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- 172.21.2.116:0/851467147 >> 172.21.2.116:0/851467147 conn(0x7f522805bd20 msgr2=0x7f522805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:01.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- 172.21.2.116:0/851467147 shutdown_connections 2021-11-21T22:02:01.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- 172.21.2.116:0/851467147 wait complete. 2021-11-21T22:02:01.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 Processor -- start 2021-11-21T22:02:01.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- start start 2021-11-21T22:02:01.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5228148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5228159990 0x7f5228159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f522813c0e0 0x7f522813c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522815a300 con 0x7f52080049f0 2021-11-21T22:02:01.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52281457c0 con 0x7f522813c0e0 2021-11-21T22:02:01.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52280b2c30 con 0x7f5228159990 2021-11-21T22:02:01.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5228148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:01.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5228148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52086/0 (socket says 172.21.2.116:52086) 2021-11-21T22:02:01.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 -- 172.21.2.116:0/261228832 learned_addr learned my addr 172.21.2.116:0/261228832 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:01.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f522813c0e0 0x7f522813c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:01.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 -- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5228159990 msgr2=0x7f5228159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:01.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5228159990 0x7f5228159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:01.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 -- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f522813c0e0 msgr2=0x7f522813c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:01.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f522813c0e0 0x7f522813c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:01.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 -- 172.21.2.116:0/261228832 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228153f70 con 0x7f52080049f0 2021-11-21T22:02:01.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522d545700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f522813c0e0 0x7f522813c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:01.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522cd44700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5228148740 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f5218008000 tx=0x7f5218009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:01.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f5225ffb700 1 -- 172.21.2.116:0/261228832 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5218007170 con 0x7f52080049f0 2021-11-21T22:02:01.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f5225ffb700 1 -- 172.21.2.116:0/261228832 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52180072d0 con 0x7f52080049f0 2021-11-21T22:02:01.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f5225ffb700 1 -- 172.21.2.116:0/261228832 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5218004920 con 0x7f52080049f0 2021-11-21T22:02:01.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5228149510 con 0x7f52080049f0 2021-11-21T22:02:01.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.254+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5228146b00 con 0x7f52080049f0 2021-11-21T22:02:01.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.258+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52281459c0 con 0x7f52080049f0 2021-11-21T22:02:01.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.258+0000 7f5225ffb700 1 -- 172.21.2.116:0/261228832 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5218004a80 con 0x7f52080049f0 2021-11-21T22:02:01.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.258+0000 7f5225ffb700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0535b0 0x7f520c055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:01.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.258+0000 7f5225ffb700 1 -- 172.21.2.116:0/261228832 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f521806b5e0 con 0x7f52080049f0 2021-11-21T22:02:01.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.258+0000 7f5227fff700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0535b0 0x7f520c055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:01.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.258+0000 7f5227fff700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0535b0 0x7f520c055a70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f521000a000 tx=0x7f5210008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:01.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.258+0000 7f5225ffb700 1 -- 172.21.2.116:0/261228832 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52180427a0 con 0x7f52080049f0 2021-11-21T22:02:01.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:01.342+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f52280b2df0 con 0x7f52080049f0 2021-11-21T22:02:02.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.202+0000 7f5225ffb700 1 -- 172.21.2.116:0/261228832 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v119) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5218045dc0 con 0x7f52080049f0 2021-11-21T22:02:02.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0535b0 msgr2=0x7f520c055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0535b0 0x7f520c055a70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f521000a000 tx=0x7f5210008040).stop 2021-11-21T22:02:02.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 msgr2=0x7f5228148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5228148740 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f5218008000 tx=0x7f5218009040).stop 2021-11-21T22:02:02.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 shutdown_connections 2021-11-21T22:02:02.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f522813c0e0 0x7f522813c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5228159990 0x7f5228159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f520c0535b0 0x7f520c055a70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 --2- 172.21.2.116:0/261228832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f52080049f0 0x7f5228148740 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 >> 172.21.2.116:0/261228832 conn(0x7f522805bd20 msgr2=0x7f5228146eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 shutdown_connections 2021-11-21T22:02:02.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.206+0000 7f522efa8700 1 -- 172.21.2.116:0/261228832 wait complete. 2021-11-21T22:02:02.212 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:02:02.215 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:02:02.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84aaaf700 1 Processor -- start 2021-11-21T22:02:02.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84aaaf700 1 -- start start 2021-11-21T22:02:02.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84aaaf700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd8440affc0 0x7fd84405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84aaaf700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd844143f70 con 0x7fd8440ad780 2021-11-21T22:02:02.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84aaaf700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8440adb60 con 0x7fd8440af7e0 2021-11-21T22:02:02.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84aaaf700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd844143160 con 0x7fd8440affc0 2021-11-21T22:02:02.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd8440ad780 0x7fd844143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35818/0 (socket says 172.21.2.116:35818) 2021-11-21T22:02:02.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 -- 172.21.2.116:0/2747418 learned_addr learned my addr 172.21.2.116:0/2747418 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:02.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040822696 0 0) 0x7fd844143f70 con 0x7fd8440ad780 2021-11-21T22:02:02.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd824003260 con 0x7fd8440ad780 2021-11-21T22:02:02.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd843fff700 1 --2- 172.21.2.116:0/2747418 >> v2:172.21.2.117:3300/0 conn(0x7fd8440affc0 0x7fd84405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 457733405 0 0) 0x7fd824003260 con 0x7fd8440ad780 2021-11-21T22:02:02.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 >> v2:172.21.2.117:3300/0 conn(0x7fd8440affc0 msgr2=0x7fd84405f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:02.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 --2- 172.21.2.116:0/2747418 >> v2:172.21.2.117:3300/0 conn(0x7fd8440affc0 0x7fd84405f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 >> v1:172.21.2.117:6789/0 conn(0x7fd8440af7e0 legacy=0x7fd84413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:02.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440b0ce0 con 0x7fd8440ad780 2021-11-21T22:02:02.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd834001660 con 0x7fd8440ad780 2021-11-21T22:02:02.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 >> v1:172.21.2.116:6789/0 conn(0x7fd8440ad780 legacy=0x7fd844143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd824003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd824004500 0x7fd824004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd824004ec0 0x7fd82400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd844143160 con 0x7fd824003750 2021-11-21T22:02:02.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82400bd80 con 0x7fd824004ec0 2021-11-21T22:02:02.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82400bf40 con 0x7fd824004500 2021-11-21T22:02:02.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd824003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 -- 172.21.2.116:0/2747418 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd824004500 msgr2=0x7fd824004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:02.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd824004500 0x7fd824004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 -- 172.21.2.116:0/2747418 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd824004ec0 msgr2=0x7fd82400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:02.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd824004ec0 0x7fd82400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 -- 172.21.2.116:0/2747418 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824003260 con 0x7fd824003750 2021-11-21T22:02:02.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84904c700 1 -- 172.21.2.116:0/2747418 reap_dead start 2021-11-21T22:02:02.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd84884b700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd824003b50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7fd8440685b0 tx=0x7fd834001e30).ready entity=mon.0 client_cookie=bb96ca6ee3301eb2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8340071a0 con 0x7fd824003750 2021-11-21T22:02:02.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834007300 con 0x7fd824003750 2021-11-21T22:02:02.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.266+0000 7fd8437fe700 1 -- 172.21.2.116:0/2747418 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd83400c8f0 con 0x7fd824003750 2021-11-21T22:02:02.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/2747418 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 msgr2=0x7fd824003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd824003b50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7fd8440685b0 tx=0x7fd834001e30).stop 2021-11-21T22:02:02.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/2747418 shutdown_connections 2021-11-21T22:02:02.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/2747418 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd824003b50 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/2747418 >> 172.21.2.116:0/2747418 conn(0x7fd8440b6000 msgr2=0x7fd8440b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/2747418 shutdown_connections 2021-11-21T22:02:02.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/2747418 wait complete. 2021-11-21T22:02:02.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 Processor -- start 2021-11-21T22:02:02.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- start start 2021-11-21T22:02:02.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd8440602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8440aa240 0x7fd8440aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd844066b90 0x7fd844067040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84415a7e0 con 0x7fd824003750 2021-11-21T22:02:02.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b5810 con 0x7fd844066b90 2021-11-21T22:02:02.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b5a40 con 0x7fd8440aa240 2021-11-21T22:02:02.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd8440602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd8440602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52108/0 (socket says 172.21.2.116:52108) 2021-11-21T22:02:02.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 -- 172.21.2.116:0/229869262 learned_addr learned my addr 172.21.2.116:0/229869262 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:02.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd843fff700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8440aa240 0x7fd8440aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84904c700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd844066b90 0x7fd844067040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 -- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8440aa240 msgr2=0x7fd8440aa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8440aa240 0x7fd8440aa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 -- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd844066b90 msgr2=0x7fd844067040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd844066b90 0x7fd844067040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 -- 172.21.2.116:0/229869262 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440b53a0 con 0x7fd824003750 2021-11-21T22:02:02.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84884b700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd8440602e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fd8440685b0 tx=0x7fd834015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd841ffb700 1 -- 172.21.2.116:0/229869262 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd834004920 con 0x7fd824003750 2021-11-21T22:02:02.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd841ffb700 1 -- 172.21.2.116:0/229869262 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834004a80 con 0x7fd824003750 2021-11-21T22:02:02.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8440acbd0 con 0x7fd824003750 2021-11-21T22:02:02.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd841ffb700 1 -- 172.21.2.116:0/229869262 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd834004de0 con 0x7fd824003750 2021-11-21T22:02:02.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8440a9680 con 0x7fd824003750 2021-11-21T22:02:02.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd841ffb700 1 -- 172.21.2.116:0/229869262 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd834027070 con 0x7fd824003750 2021-11-21T22:02:02.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8440acd60 con 0x7fd824003750 2021-11-21T22:02:02.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd841ffb700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8280536a0 0x7fd828055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.270+0000 7fd841ffb700 1 -- 172.21.2.116:0/229869262 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fd83406c560 con 0x7fd824003750 2021-11-21T22:02:02.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.274+0000 7fd843fff700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8280536a0 0x7fd828055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.274+0000 7fd843fff700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8280536a0 0x7fd828055b60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fd82c00b000 tx=0x7fd82c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.274+0000 7fd841ffb700 1 -- 172.21.2.116:0/229869262 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd834043730 con 0x7fd824003750 2021-11-21T22:02:02.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.382+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd8440b0a40 con 0x7fd824003750 2021-11-21T22:02:02.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd841ffb700 1 -- 172.21.2.116:0/229869262 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v54) v1 ==== 189+0+0 (secure 0 0 0) 0x7fd834046d50 con 0x7fd824003750 2021-11-21T22:02:02.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8280536a0 msgr2=0x7fd828055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8280536a0 0x7fd828055b60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fd82c00b000 tx=0x7fd82c008040).stop 2021-11-21T22:02:02.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 msgr2=0x7fd8440602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd8440602e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fd8440685b0 tx=0x7fd834015000).stop 2021-11-21T22:02:02.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 shutdown_connections 2021-11-21T22:02:02.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd844066b90 0x7fd844067040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8440aa240 0x7fd8440aa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8280536a0 0x7fd828055b60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 --2- 172.21.2.116:0/229869262 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd824003750 0x7fd8440602e0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 >> 172.21.2.116:0/229869262 conn(0x7fd8440b6000 msgr2=0x7fd844140e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 shutdown_connections 2021-11-21T22:02:02.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.390+0000 7fd84aaaf700 1 -- 172.21.2.116:0/229869262 wait complete. 2021-11-21T22:02:02.399 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 21 and data pool 22 2021-11-21T22:02:02.403 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:02.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 Processor -- start 2021-11-21T22:02:02.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- start start 2021-11-21T22:02:02.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1dc0ad780 0x7fc1dc05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc1dc0affc0 0x7fc1dc143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1dc0adb60 con 0x7fc1dc0af7e0 2021-11-21T22:02:02.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc143160 con 0x7fc1dc0ad780 2021-11-21T22:02:02.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc0b0ce0 con 0x7fc1dc0affc0 2021-11-21T22:02:02.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1daffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1dc0ad780 0x7fc1dc05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1db7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc1dc0af7e0 0x7fc1dc0b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35852/0 (socket says 172.21.2.116:35852) 2021-11-21T22:02:02.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1db7fe700 1 -- 172.21.2.116:0/236724400 learned_addr learned my addr 172.21.2.116:0/236724400 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:02.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1daffd700 1 -- 172.21.2.116:0/236724400 >> v2:172.21.2.117:3300/0 conn(0x7fc1dc0affc0 msgr2=0x7fc1dc143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:02.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1da7fc700 1 --2- 172.21.2.116:0/236724400 >> v2:172.21.2.117:3300/0 conn(0x7fc1dc0affc0 0x7fc1dc143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1d9ffb700 1 -- 172.21.2.116:0/236724400 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346351008 0 0) 0x7fc1dc0adb60 con 0x7fc1dc0af7e0 2021-11-21T22:02:02.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1daffd700 1 --2- 172.21.2.116:0/236724400 >> v2:172.21.2.117:3300/0 conn(0x7fc1dc0affc0 0x7fc1dc143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1daffd700 1 -- 172.21.2.116:0/236724400 >> v1:172.21.2.116:6789/0 conn(0x7fc1dc0af7e0 legacy=0x7fc1dc0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1daffd700 1 -- 172.21.2.116:0/236724400 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc0b0a40 con 0x7fc1dc0ad780 2021-11-21T22:02:02.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1da7fc700 1 --2- 172.21.2.116:0/236724400 >> v2:172.21.2.117:3300/0 conn(0x7fc1dc0affc0 0x7fc1dc143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:02.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1daffd700 1 --2- 172.21.2.116:0/236724400 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05f730 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fc1d0002f70 tx=0x7fc1d0006e80).ready entity=mon.0 client_cookie=1c611c53e5918b0b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1d9ffb700 1 -- 172.21.2.116:0/236724400 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1d000ba40 con 0x7fc1dc0ad780 2021-11-21T22:02:02.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1d9ffb700 1 -- 172.21.2.116:0/236724400 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0000e90 con 0x7fc1dc0ad780 2021-11-21T22:02:02.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1d9ffb700 1 -- 172.21.2.116:0/236724400 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1d000bd90 con 0x7fc1dc0ad780 2021-11-21T22:02:02.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- 172.21.2.116:0/236724400 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 msgr2=0x7fc1dc05f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 --2- 172.21.2.116:0/236724400 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05f730 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fc1d0002f70 tx=0x7fc1d0006e80).stop 2021-11-21T22:02:02.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- 172.21.2.116:0/236724400 shutdown_connections 2021-11-21T22:02:02.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 --2- 172.21.2.116:0/236724400 >> v2:172.21.2.117:3300/0 conn(0x7fc1dc0affc0 0x7fc1dc143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 --2- 172.21.2.116:0/236724400 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05f730 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- 172.21.2.116:0/236724400 >> 172.21.2.116:0/236724400 conn(0x7fc1dc0b6000 msgr2=0x7fc1dc0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- 172.21.2.116:0/236724400 shutdown_connections 2021-11-21T22:02:02.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- 172.21.2.116:0/236724400 wait complete. 2021-11-21T22:02:02.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 Processor -- start 2021-11-21T22:02:02.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.450+0000 7fc1e1458700 1 -- start start 2021-11-21T22:02:02.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1dc0af7e0 0x7fc1dc0ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1dc0affc0 0x7fc1dc0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc141720 con 0x7fc1dc0ad780 2021-11-21T22:02:02.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc1421a0 con 0x7fc1dc0af7e0 2021-11-21T22:02:02.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc141950 con 0x7fc1dc0affc0 2021-11-21T22:02:02.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52134/0 (socket says 172.21.2.116:52134) 2021-11-21T22:02:02.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 -- 172.21.2.116:0/47706959 learned_addr learned my addr 172.21.2.116:0/47706959 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:02.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1da7fc700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1dc0af7e0 0x7fc1dc0ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 -- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1dc0affc0 msgr2=0x7fc1dc0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1dc0affc0 0x7fc1dc0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 -- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1dc0af7e0 msgr2=0x7fc1dc0ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1dc0af7e0 0x7fc1dc0ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 -- 172.21.2.116:0/47706959 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d000d410 con 0x7fc1dc0ad780 2021-11-21T22:02:02.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1da7fc700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1dc0af7e0 0x7fc1dc0ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:02.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1daffd700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05ff80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7fc1d0002f70 tx=0x7fc1d0006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1cbfff700 1 -- 172.21.2.116:0/47706959 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1d001f070 con 0x7fc1dc0ad780 2021-11-21T22:02:02.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1cbfff700 1 -- 172.21.2.116:0/47706959 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0018800 con 0x7fc1dc0ad780 2021-11-21T22:02:02.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1cbfff700 1 -- 172.21.2.116:0/47706959 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1d0018b60 con 0x7fc1dc0ad780 2021-11-21T22:02:02.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1dc142440 con 0x7fc1dc0ad780 2021-11-21T22:02:02.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1dc0aa770 con 0x7fc1dc0ad780 2021-11-21T22:02:02.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1dc141cd0 con 0x7fc1dc0ad780 2021-11-21T22:02:02.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1cbfff700 1 -- 172.21.2.116:0/47706959 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc1d002b070 con 0x7fc1dc0ad780 2021-11-21T22:02:02.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1cbfff700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b80534f0 0x7fc1b80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1cbfff700 1 -- 172.21.2.116:0/47706959 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc1d006f990 con 0x7fc1dc0ad780 2021-11-21T22:02:02.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1da7fc700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b80534f0 0x7fc1b80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.454+0000 7fc1da7fc700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b80534f0 0x7fc1b80559b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fc1cc008000 tx=0x7fc1cc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.458+0000 7fc1cbfff700 1 -- 172.21.2.116:0/47706959 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc1d0046af0 con 0x7fc1dc0ad780 2021-11-21T22:02:02.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1dc1433c0 con 0x7fc1dc0ad780 2021-11-21T22:02:02.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1cbfff700 1 -- 172.21.2.116:0/47706959 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fc1d004a110 con 0x7fc1dc0ad780 2021-11-21T22:02:02.546 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:02.546 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":121,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:02.389041+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:00.244205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:01.358157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:02.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b80534f0 msgr2=0x7fc1b80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b80534f0 0x7fc1b80559b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fc1cc008000 tx=0x7fc1cc008040).stop 2021-11-21T22:02:02.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 msgr2=0x7fc1dc05ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05ff80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7fc1d0002f70 tx=0x7fc1d0006e80).stop 2021-11-21T22:02:02.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 shutdown_connections 2021-11-21T22:02:02.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1dc0affc0 0x7fc1dc0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1dc0af7e0 0x7fc1dc0ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b80534f0 0x7fc1b80559b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 --2- 172.21.2.116:0/47706959 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1dc0ad780 0x7fc1dc05ff80 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 >> 172.21.2.116:0/47706959 conn(0x7fc1dc0b6000 msgr2=0x7fc1dc0b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 shutdown_connections 2021-11-21T22:02:02.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.542+0000 7fc1e1458700 1 -- 172.21.2.116:0/47706959 wait complete. 2021-11-21T22:02:02.554 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:02.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 Processor -- start 2021-11-21T22:02:02.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- start start 2021-11-21T22:02:02.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a86b0 0x7f271814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2718062410 0x7f27181443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27180a6e80 con 0x7f27180a6aa0 2021-11-21T22:02:02.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27180a8bc0 con 0x7f2718062410 2021-11-21T22:02:02.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271814a7b0 con 0x7f27180a86b0 2021-11-21T22:02:02.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f2717fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2718062410 0x7f27181443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27177fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f27180a6aa0 0x7f27181470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35866/0 (socket says 172.21.2.116:35866) 2021-11-21T22:02:02.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27177fe700 1 -- 172.21.2.116:0/2746871235 learned_addr learned my addr 172.21.2.116:0/2746871235 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:02.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f2717fff700 1 -- 172.21.2.116:0/2746871235 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a86b0 msgr2=0x7f271814b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:02.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f2717fff700 1 --2- 172.21.2.116:0/2746871235 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a86b0 0x7f271814b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f2717fff700 1 -- 172.21.2.116:0/2746871235 >> v1:172.21.2.116:6789/0 conn(0x7f27180a6aa0 legacy=0x7f27181470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f2717fff700 1 -- 172.21.2.116:0/2746871235 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271814aa10 con 0x7f2718062410 2021-11-21T22:02:02.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f2717fff700 1 --2- 172.21.2.116:0/2746871235 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27181443f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f27100032f0 tx=0x7f27100091d0).ready entity=mon.0 client_cookie=62fca60865ffc417 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27167fc700 1 -- 172.21.2.116:0/2746871235 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2710009ea0 con 0x7f2718062410 2021-11-21T22:02:02.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27167fc700 1 -- 172.21.2.116:0/2746871235 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271000b8a0 con 0x7f2718062410 2021-11-21T22:02:02.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27167fc700 1 -- 172.21.2.116:0/2746871235 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f271000ba50 con 0x7f2718062410 2021-11-21T22:02:02.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- 172.21.2.116:0/2746871235 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 msgr2=0x7f27181443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- 172.21.2.116:0/2746871235 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27181443f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f27100032f0 tx=0x7f27100091d0).stop 2021-11-21T22:02:02.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- 172.21.2.116:0/2746871235 shutdown_connections 2021-11-21T22:02:02.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- 172.21.2.116:0/2746871235 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27181443f0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- 172.21.2.116:0/2746871235 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a86b0 0x7f271814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- 172.21.2.116:0/2746871235 >> 172.21.2.116:0/2746871235 conn(0x7f271805bd20 msgr2=0x7f271805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- 172.21.2.116:0/2746871235 shutdown_connections 2021-11-21T22:02:02.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- 172.21.2.116:0/2746871235 wait complete. 2021-11-21T22:02:02.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 Processor -- start 2021-11-21T22:02:02.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- start start 2021-11-21T22:02:02.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27180ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a6aa0 0x7f2718156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27180a86b0 0x7f2718156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27180aa090 con 0x7f2718062410 2021-11-21T22:02:02.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27180639d0 con 0x7f27180a86b0 2021-11-21T22:02:02.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f271dc75700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27180a9df0 con 0x7f27180a6aa0 2021-11-21T22:02:02.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27180ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27177fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27180ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52146/0 (socket says 172.21.2.116:52146) 2021-11-21T22:02:02.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.602+0000 7f27177fe700 1 -- 172.21.2.116:0/4164319008 learned_addr learned my addr 172.21.2.116:0/4164319008 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:02.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f27177fe700 1 -- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a6aa0 msgr2=0x7f2718156660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:02.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f27177fe700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a6aa0 0x7f2718156660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f27177fe700 1 -- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27180a86b0 msgr2=0x7f2718156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f27177fe700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27180a86b0 0x7f2718156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f27177fe700 1 -- 172.21.2.116:0/4164319008 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2710009470 con 0x7f2718062410 2021-11-21T22:02:02.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f27177fe700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27180ab390 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f2718061ab0 tx=0x7f2700007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2714ff9700 1 -- 172.21.2.116:0/4164319008 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f270000ce30 con 0x7f2718062410 2021-11-21T22:02:02.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2714ff9700 1 -- 172.21.2.116:0/4164319008 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27000072e0 con 0x7f2718062410 2021-11-21T22:02:02.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2714ff9700 1 -- 172.21.2.116:0/4164319008 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27000057c0 con 0x7f2718062410 2021-11-21T22:02:02.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2718063d10 con 0x7f2718062410 2021-11-21T22:02:02.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27180a98e0 con 0x7f2718062410 2021-11-21T22:02:02.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2718063830 con 0x7f2718062410 2021-11-21T22:02:02.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2714ff9700 1 -- 172.21.2.116:0/4164319008 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2700010070 con 0x7f2718062410 2021-11-21T22:02:02.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2714ff9700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26fc053660 0x7f26fc055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2714ff9700 1 -- 172.21.2.116:0/4164319008 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f270006b0e0 con 0x7f2718062410 2021-11-21T22:02:02.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2716ffd700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26fc053660 0x7f26fc055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.606+0000 7f2716ffd700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26fc053660 0x7f26fc055b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f2708002ef0 tx=0x7f2708006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.610+0000 7f2714ff9700 1 -- 172.21.2.116:0/4164319008 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2718063830 con 0x7f2718062410 2021-11-21T22:02:02.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.690+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2718063830 con 0x7f2718062410 2021-11-21T22:02:02.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.690+0000 7f2714ff9700 1 -- 172.21.2.116:0/4164319008 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f2718063830 con 0x7f2718062410 2021-11-21T22:02:02.697 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:02.697 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":121,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:02.389041+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:00.244205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:01.358157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:02.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26fc053660 msgr2=0x7f26fc055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26fc053660 0x7f26fc055b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f2708002ef0 tx=0x7f2708006890).stop 2021-11-21T22:02:02.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 msgr2=0x7f27180ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27180ab390 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f2718061ab0 tx=0x7f2700007c30).stop 2021-11-21T22:02:02.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 shutdown_connections 2021-11-21T22:02:02.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27180a86b0 0x7f2718156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27180a6aa0 0x7f2718156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f26fc053660 0x7f26fc055b20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 --2- 172.21.2.116:0/4164319008 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2718062410 0x7f27180ab390 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 >> 172.21.2.116:0/4164319008 conn(0x7f271805bd20 msgr2=0x7f27180b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 shutdown_connections 2021-11-21T22:02:02.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.694+0000 7f271dc75700 1 -- 172.21.2.116:0/4164319008 wait complete. 2021-11-21T22:02:02.706 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:02:02.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 Processor -- start 2021-11-21T22:02:02.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- start start 2021-11-21T22:02:02.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f48ac144810 0x7f48ac1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f48ac0abce0 0x7f48ac0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac147e80 con 0x7f48ac0b1db0 2021-11-21T22:02:02.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ac0b1490 con 0x7f48ac0abce0 2021-11-21T22:02:02.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ac0af060 con 0x7f48ac144810 2021-11-21T22:02:02.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f48ac0abce0 0x7f48ac0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f48ac0abce0 0x7f48ac0b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52158/0 (socket says 172.21.2.116:52158) 2021-11-21T22:02:02.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 -- 172.21.2.116:0/3564005728 learned_addr learned my addr 172.21.2.116:0/3564005728 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:02.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 -- 172.21.2.116:0/3564005728 >> v2:172.21.2.117:3300/0 conn(0x7f48ac144810 msgr2=0x7f48ac1478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:02.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 --2- 172.21.2.116:0/3564005728 >> v2:172.21.2.117:3300/0 conn(0x7f48ac144810 0x7f48ac1478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 -- 172.21.2.116:0/3564005728 >> v1:172.21.2.116:6789/0 conn(0x7f48ac0b1db0 legacy=0x7f48ac0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 -- 172.21.2.116:0/3564005728 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac0af5a0 con 0x7f48ac0abce0 2021-11-21T22:02:02.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b1412700 1 --2- 172.21.2.116:0/3564005728 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac0abce0 0x7f48ac0b18c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f48a8002f70 tx=0x7f48a8006eb0).ready entity=mon.0 client_cookie=d5f2d7b31c965eb0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b0c11700 1 -- 172.21.2.116:0/3564005728 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a800d610 con 0x7f48ac0abce0 2021-11-21T22:02:02.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b0c11700 1 -- 172.21.2.116:0/3564005728 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a8000e90 con 0x7f48ac0abce0 2021-11-21T22:02:02.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b0c11700 1 -- 172.21.2.116:0/3564005728 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a800d9a0 con 0x7f48ac0abce0 2021-11-21T22:02:02.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- 172.21.2.116:0/3564005728 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac0abce0 msgr2=0x7f48ac0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:02.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 --2- 172.21.2.116:0/3564005728 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac0abce0 0x7f48ac0b18c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f48a8002f70 tx=0x7f48a8006eb0).stop 2021-11-21T22:02:02.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- 172.21.2.116:0/3564005728 shutdown_connections 2021-11-21T22:02:02.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 --2- 172.21.2.116:0/3564005728 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac0abce0 0x7f48ac0b18c0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:02.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 --2- 172.21.2.116:0/3564005728 >> v2:172.21.2.117:3300/0 conn(0x7f48ac144810 0x7f48ac1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- 172.21.2.116:0/3564005728 >> 172.21.2.116:0/3564005728 conn(0x7f48ac05bd20 msgr2=0x7f48ac05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:02.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- 172.21.2.116:0/3564005728 shutdown_connections 2021-11-21T22:02:02.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.754+0000 7f48b3e77700 1 -- 172.21.2.116:0/3564005728 wait complete. 2021-11-21T22:02:02.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 Processor -- start 2021-11-21T22:02:02.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 -- start start 2021-11-21T22:02:02.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48ac0abce0 0x7f48ac143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac144810 0x7f48ac146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48ac145ed0 0x7f48ac146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac144ef0 con 0x7f48ac144810 2021-11-21T22:02:02.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac1465d0 con 0x7f48ac0abce0 2021-11-21T22:02:02.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac145120 con 0x7f48ac145ed0 2021-11-21T22:02:02.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac144810 0x7f48ac146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac144810 0x7f48ac146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52162/0 (socket says 172.21.2.116:52162) 2021-11-21T22:02:02.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 -- 172.21.2.116:0/833143491 learned_addr learned my addr 172.21.2.116:0/833143491 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:02.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 -- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48ac145ed0 msgr2=0x7f48ac146ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:02.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48ac145ed0 0x7f48ac146ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 -- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48ac0abce0 msgr2=0x7f48ac143a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:02.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48ac0abce0 0x7f48ac143a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:02.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 -- 172.21.2.116:0/833143491 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a800b1e0 con 0x7f48ac144810 2021-11-21T22:02:02.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1412700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac144810 0x7f48ac146a00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f48a8002f70 tx=0x7f48a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f489affd700 1 -- 172.21.2.116:0/833143491 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a8011a00 con 0x7f48ac144810 2021-11-21T22:02:02.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f489affd700 1 -- 172.21.2.116:0/833143491 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a8011b60 con 0x7f48ac144810 2021-11-21T22:02:02.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac1453c0 con 0x7f48ac144810 2021-11-21T22:02:02.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f489affd700 1 -- 172.21.2.116:0/833143491 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a801e460 con 0x7f48ac144810 2021-11-21T22:02:02.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac0a81a0 con 0x7f48ac144810 2021-11-21T22:02:02.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac146740 con 0x7f48ac144810 2021-11-21T22:02:02.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f489affd700 1 -- 172.21.2.116:0/833143491 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f48a8025030 con 0x7f48ac144810 2021-11-21T22:02:02.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f489affd700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f488c053540 0x7f488c055a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:02.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f489affd700 1 -- 172.21.2.116:0/833143491 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f48a806ce90 con 0x7f48ac144810 2021-11-21T22:02:02.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.758+0000 7f48b1c13700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f488c053540 0x7f488c055a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:02.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.762+0000 7f48b1c13700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f488c053540 0x7f488c055a00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f489c002ef0 tx=0x7f489c006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:02.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.762+0000 7f489affd700 1 -- 172.21.2.116:0/833143491 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48ac146740 con 0x7f48ac144810 2021-11-21T22:02:02.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:02.870+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f48ac146740 con 0x7f48ac144810 2021-11-21T22:02:03.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.398+0000 7f489affd700 1 -- 172.21.2.116:0/833143491 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f48ac146740 con 0x7f48ac144810 2021-11-21T22:02:03.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.398+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f488c053540 msgr2=0x7f488c055a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.398+0000 7f48b3e77700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f488c053540 0x7f488c055a00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f489c002ef0 tx=0x7f489c006890).stop 2021-11-21T22:02:03.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.398+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac144810 msgr2=0x7f48ac146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.398+0000 7f48b3e77700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac144810 0x7f48ac146a00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f48a8002f70 tx=0x7f48a8015000).stop 2021-11-21T22:02:03.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 shutdown_connections 2021-11-21T22:02:03.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f48ac145ed0 0x7f48ac146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f488c053540 0x7f488c055a00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f48ac144810 0x7f48ac146a00 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 --2- 172.21.2.116:0/833143491 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f48ac0abce0 0x7f48ac143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 >> 172.21.2.116:0/833143491 conn(0x7f48ac05bd20 msgr2=0x7f48ac05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:03.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 shutdown_connections 2021-11-21T22:02:03.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.402+0000 7f48b3e77700 1 -- 172.21.2.116:0/833143491 wait complete. 2021-11-21T22:02:03.412 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:03.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 Processor -- start 2021-11-21T22:02:03.474 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- start start 2021-11-21T22:02:03.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fea4c062410 0x7fea4c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0a6e80 con 0x7fea4c0a86b0 2021-11-21T22:02:03.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c14a7b0 con 0x7fea4c0a6aa0 2021-11-21T22:02:03.475 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c0a8bc0 con 0x7fea4c062410 2021-11-21T22:02:03.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fea4c062410 0x7fea4c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fea4c062410 0x7fea4c144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52170/0 (socket says 172.21.2.116:52170) 2021-11-21T22:02:03.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 -- 172.21.2.116:0/2202549508 learned_addr learned my addr 172.21.2.116:0/2202549508 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:03.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 -- 172.21.2.116:0/2202549508 >> v1:172.21.2.117:6789/0 conn(0x7fea4c0a6aa0 legacy=0x7fea4c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:03.476 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 -- 172.21.2.116:0/2202549508 >> v1:172.21.2.116:6789/0 conn(0x7fea4c0a86b0 legacy=0x7fea4c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:03.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 -- 172.21.2.116:0/2202549508 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c14aa10 con 0x7fea4c062410 2021-11-21T22:02:03.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 --2- 172.21.2.116:0/2202549508 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c062410 0x7fea4c144b00 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fea4c061ab0 tx=0x7fea44009290).ready entity=mon.0 client_cookie=3c5a1bb08e3813b9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea49ffb700 1 -- 172.21.2.116:0/2202549508 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4400b7f0 con 0x7fea4c062410 2021-11-21T22:02:03.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea49ffb700 1 -- 172.21.2.116:0/2202549508 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4400b950 con 0x7fea4c062410 2021-11-21T22:02:03.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea49ffb700 1 -- 172.21.2.116:0/2202549508 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4400bb00 con 0x7fea4c062410 2021-11-21T22:02:03.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- 172.21.2.116:0/2202549508 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c062410 msgr2=0x7fea4c144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 --2- 172.21.2.116:0/2202549508 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c062410 0x7fea4c144b00 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fea4c061ab0 tx=0x7fea44009290).stop 2021-11-21T22:02:03.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- 172.21.2.116:0/2202549508 shutdown_connections 2021-11-21T22:02:03.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 --2- 172.21.2.116:0/2202549508 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c062410 0x7fea4c144b00 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- 172.21.2.116:0/2202549508 >> 172.21.2.116:0/2202549508 conn(0x7fea4c05bd20 msgr2=0x7fea4c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:03.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- 172.21.2.116:0/2202549508 shutdown_connections 2021-11-21T22:02:03.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- 172.21.2.116:0/2202549508 wait complete. 2021-11-21T22:02:03.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 Processor -- start 2021-11-21T22:02:03.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- start start 2021-11-21T22:02:03.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea4c062410 0x7fea4c066be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea4c0a6aa0 0x7fea4c1458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c0a86b0 0x7fea4c145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c157040 con 0x7fea4c0a86b0 2021-11-21T22:02:03.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c145170 con 0x7fea4c0a6aa0 2021-11-21T22:02:03.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea5186e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c145410 con 0x7fea4c062410 2021-11-21T22:02:03.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c0a86b0 0x7fea4c145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c0a86b0 0x7fea4c145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52176/0 (socket says 172.21.2.116:52176) 2021-11-21T22:02:03.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.470+0000 7fea4b7fe700 1 -- 172.21.2.116:0/1235142478 learned_addr learned my addr 172.21.2.116:0/1235142478 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:03.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4b7fe700 1 -- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea4c062410 msgr2=0x7fea4c066be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4b7fe700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea4c062410 0x7fea4c066be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4b7fe700 1 -- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea4c0a6aa0 msgr2=0x7fea4c1458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4b7fe700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea4c0a6aa0 0x7fea4c1458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4b7fe700 1 -- 172.21.2.116:0/1235142478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009530 con 0x7fea4c0a86b0 2021-11-21T22:02:03.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4b7fe700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c0a86b0 0x7fea4c145da0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fea4c061ab0 tx=0x7fea44016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea43fff700 1 -- 172.21.2.116:0/1235142478 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4400b7f0 con 0x7fea4c0a86b0 2021-11-21T22:02:03.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea43fff700 1 -- 172.21.2.116:0/1235142478 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44021630 con 0x7fea4c0a86b0 2021-11-21T22:02:03.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c156cf0 con 0x7fea4c0a86b0 2021-11-21T22:02:03.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea43fff700 1 -- 172.21.2.116:0/1235142478 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea44021990 con 0x7fea4c0a86b0 2021-11-21T22:02:03.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c146430 con 0x7fea4c0a86b0 2021-11-21T22:02:03.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c067410 con 0x7fea4c0a86b0 2021-11-21T22:02:03.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea43fff700 1 -- 172.21.2.116:0/1235142478 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fea44027020 con 0x7fea4c0a86b0 2021-11-21T22:02:03.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea43fff700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea30053610 0x7fea30055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea43fff700 1 -- 172.21.2.116:0/1235142478 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fea4406cf10 con 0x7fea4c0a86b0 2021-11-21T22:02:03.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4affd700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea30053610 0x7fea30055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea4affd700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea30053610 0x7fea30055ad0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fea34002f90 tx=0x7fea340069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.474+0000 7fea43fff700 1 -- 172.21.2.116:0/1235142478 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea44043f60 con 0x7fea4c0a86b0 2021-11-21T22:02:03.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea4c067410 con 0x7fea4c0a86b0 2021-11-21T22:02:03.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea43fff700 1 -- 172.21.2.116:0/1235142478 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fea44015070 con 0x7fea4c0a86b0 2021-11-21T22:02:03.590 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:03.590 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":55,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3682554328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3682554328},{"type":"v1","addr":"172.21.2.117:6835","nonce":3682554328}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/2101278175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2101278175},{"type":"v1","addr":"172.21.2.116:6833","nonce":2101278175}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:02.389133+0000","modified":"2021-11-21T22:02:03.393012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1577318848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1577318848},{"type":"v1","addr":"172.21.2.116:6835","nonce":1577318848}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-11-21T22:02:03.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea30053610 msgr2=0x7fea30055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea30053610 0x7fea30055ad0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fea34002f90 tx=0x7fea340069b0).stop 2021-11-21T22:02:03.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c0a86b0 msgr2=0x7fea4c145da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c0a86b0 0x7fea4c145da0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fea4c061ab0 tx=0x7fea44016000).stop 2021-11-21T22:02:03.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 shutdown_connections 2021-11-21T22:02:03.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fea30053610 0x7fea30055ad0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fea4c0a86b0 0x7fea4c145da0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fea4c0a6aa0 0x7fea4c1458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 --2- 172.21.2.116:0/1235142478 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fea4c062410 0x7fea4c066be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 >> 172.21.2.116:0/1235142478 conn(0x7fea4c05bd20 msgr2=0x7fea4c1441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:03.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 shutdown_connections 2021-11-21T22:02:03.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.586+0000 7fea5186e700 1 -- 172.21.2.116:0/1235142478 wait complete. 2021-11-21T22:02:03.594 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 55 2021-11-21T22:02:03.596 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:03.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 Processor -- start 2021-11-21T22:02:03.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 -- start start 2021-11-21T22:02:03.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3ff00a6aa0 0x7f3ff014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ff00a86b0 0x7f3ff01470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 0x7f3ff01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00a8bc0 con 0x7f3ff00a86b0 2021-11-21T22:02:03.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00a6e80 con 0x7f3ff00a6aa0 2021-11-21T22:02:03.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3ff6878700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff014a7b0 con 0x7f3ff01475d0 2021-11-21T22:02:03.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ff00a86b0 0x7f3ff01470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ff00a86b0 0x7f3ff01470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52184/0 (socket says 172.21.2.116:52184) 2021-11-21T22:02:03.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 -- 172.21.2.116:0/3190770108 learned_addr learned my addr 172.21.2.116:0/3190770108 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:03.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 -- 172.21.2.116:0/3190770108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 msgr2=0x7f3ff01443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:03.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 --2- 172.21.2.116:0/3190770108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 0x7f3ff01443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 -- 172.21.2.116:0/3190770108 >> v2:172.21.2.117:3300/0 conn(0x7f3ff00a6aa0 msgr2=0x7f3ff014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:03.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 --2- 172.21.2.116:0/3190770108 >> v2:172.21.2.117:3300/0 conn(0x7f3ff00a6aa0 0x7f3ff014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.642+0000 7f3fef7fe700 1 -- 172.21.2.116:0/3190770108 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff014aa10 con 0x7f3ff00a86b0 2021-11-21T22:02:03.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 --2- 172.21.2.116:0/3190770108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff01470e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f3fd8002f70 tx=0x7f3fd8006eb0).ready entity=mon.0 client_cookie=2f27d8ed0586af2d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3feeffd700 1 -- 172.21.2.116:0/3190770108 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd800d570 con 0x7f3ff00a86b0 2021-11-21T22:02:03.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3feeffd700 1 -- 172.21.2.116:0/3190770108 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd8000e90 con 0x7f3ff00a86b0 2021-11-21T22:02:03.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3feeffd700 1 -- 172.21.2.116:0/3190770108 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd800d900 con 0x7f3ff00a86b0 2021-11-21T22:02:03.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/3190770108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 msgr2=0x7f3ff01470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 --2- 172.21.2.116:0/3190770108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff01470e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f3fd8002f70 tx=0x7f3fd8006eb0).stop 2021-11-21T22:02:03.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/3190770108 shutdown_connections 2021-11-21T22:02:03.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 --2- 172.21.2.116:0/3190770108 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 0x7f3ff01443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 --2- 172.21.2.116:0/3190770108 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff01470e0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 --2- 172.21.2.116:0/3190770108 >> v2:172.21.2.117:3300/0 conn(0x7f3ff00a6aa0 0x7f3ff014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/3190770108 >> 172.21.2.116:0/3190770108 conn(0x7f3ff005bd20 msgr2=0x7f3ff005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:03.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/3190770108 shutdown_connections 2021-11-21T22:02:03.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/3190770108 wait complete. 2021-11-21T22:02:03.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 Processor -- start 2021-11-21T22:02:03.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- start start 2021-11-21T22:02:03.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ff00a6aa0 0x7f3ff01461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff00b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 0x7f3ff00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00ab380 con 0x7f3ff00a86b0 2021-11-21T22:02:03.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00ab5b0 con 0x7f3ff00a6aa0 2021-11-21T22:02:03.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00ab850 con 0x7f3ff01475d0 2021-11-21T22:02:03.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff00b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff00b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52188/0 (socket says 172.21.2.116:52188) 2021-11-21T22:02:03.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 -- 172.21.2.116:0/512829848 learned_addr learned my addr 172.21.2.116:0/512829848 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:03.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 -- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 msgr2=0x7f3ff00b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 0x7f3ff00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 -- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ff00a6aa0 msgr2=0x7f3ff01461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ff00a6aa0 0x7f3ff01461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 -- 172.21.2.116:0/512829848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd800b1e0 con 0x7f3ff00a86b0 2021-11-21T22:02:03.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fef7fe700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff00b39b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f3fd8002f70 tx=0x7f3fd8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fed7fa700 1 -- 172.21.2.116:0/512829848 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd8011790 con 0x7f3ff00a86b0 2021-11-21T22:02:03.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fed7fa700 1 -- 172.21.2.116:0/512829848 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd80118f0 con 0x7f3ff00a86b0 2021-11-21T22:02:03.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff00abaf0 con 0x7f3ff00a86b0 2021-11-21T22:02:03.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff01469d0 con 0x7f3ff00a86b0 2021-11-21T22:02:03.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3fed7fa700 1 -- 172.21.2.116:0/512829848 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd8011c50 con 0x7f3ff00a86b0 2021-11-21T22:02:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.646+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff0063a60 con 0x7f3ff00a86b0 2021-11-21T22:02:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.650+0000 7f3fed7fa700 1 -- 172.21.2.116:0/512829848 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3fd8025030 con 0x7f3ff00a86b0 2021-11-21T22:02:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.650+0000 7f3fed7fa700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3fd40536f0 0x7f3fd4055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.650+0000 7f3fed7fa700 1 -- 172.21.2.116:0/512829848 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3fd806cc90 con 0x7f3ff00a86b0 2021-11-21T22:02:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.650+0000 7f3feffff700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3fd40536f0 0x7f3fd4055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.650+0000 7f3feffff700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3fd40536f0 0x7f3fd4055bb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f3fe0002f70 tx=0x7f3fe00069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.650+0000 7f3fed7fa700 1 -- 172.21.2.116:0/512829848 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3fd8043ce0 con 0x7f3ff00a86b0 2021-11-21T22:02:03.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff00573d0 con 0x7f3ff00a86b0 2021-11-21T22:02:03.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3fed7fa700 1 -- 172.21.2.116:0/512829848 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f3fd8023090 con 0x7f3ff00a86b0 2021-11-21T22:02:03.738 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:03.739 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":122,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:03.394269+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:00.244205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:01.358157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:03.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3fd40536f0 msgr2=0x7f3fd4055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3fd40536f0 0x7f3fd4055bb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f3fe0002f70 tx=0x7f3fe00069e0).stop 2021-11-21T22:02:03.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 msgr2=0x7f3ff00b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff00b39b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f3fd8002f70 tx=0x7f3fd8015000).stop 2021-11-21T22:02:03.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 shutdown_connections 2021-11-21T22:02:03.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ff01475d0 0x7f3ff00b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3fd40536f0 0x7f3fd4055bb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ff00a86b0 0x7f3ff00b39b0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 --2- 172.21.2.116:0/512829848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ff00a6aa0 0x7f3ff01461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 >> 172.21.2.116:0/512829848 conn(0x7f3ff005bd20 msgr2=0x7f3ff014b730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:03.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 shutdown_connections 2021-11-21T22:02:03.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.734+0000 7f3ff6878700 1 -- 172.21.2.116:0/512829848 wait complete. 2021-11-21T22:02:03.746 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:02:03.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 Processor -- start 2021-11-21T22:02:03.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- start start 2021-11-21T22:02:03.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7efffc0affc0 0x7efffc05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7efffc0af7e0 0x7efffc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc143f70 con 0x7efffc0ad780 2021-11-21T22:02:03.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc0adb60 con 0x7efffc0af7e0 2021-11-21T22:02:03.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc143160 con 0x7efffc0affc0 2021-11-21T22:02:03.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7efffc0af7e0 0x7efffc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7efffc0af7e0 0x7efffc13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52200/0 (socket says 172.21.2.116:52200) 2021-11-21T22:02:03.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 -- 172.21.2.116:0/1565886527 learned_addr learned my addr 172.21.2.116:0/1565886527 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:03.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 -- 172.21.2.116:0/1565886527 >> v2:172.21.2.117:3300/0 conn(0x7efffc0affc0 msgr2=0x7efffc05f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:03.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 --2- 172.21.2.116:0/1565886527 >> v2:172.21.2.117:3300/0 conn(0x7efffc0affc0 0x7efffc05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 -- 172.21.2.116:0/1565886527 >> v1:172.21.2.116:6789/0 conn(0x7efffc0ad780 legacy=0x7efffc143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 -- 172.21.2.116:0/1565886527 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0b0ce0 con 0x7efffc0af7e0 2021-11-21T22:02:03.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffbfff700 1 --2- 172.21.2.116:0/1565886527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0af7e0 0x7efffc13fe70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7efff40032f0 tx=0x7efff4009290).ready entity=mon.0 client_cookie=c1d1e864f09bf1d9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffa7fc700 1 -- 172.21.2.116:0/1565886527 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff400b870 con 0x7efffc0af7e0 2021-11-21T22:02:03.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffa7fc700 1 -- 172.21.2.116:0/1565886527 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400b9d0 con 0x7efffc0af7e0 2021-11-21T22:02:03.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffa7fc700 1 -- 172.21.2.116:0/1565886527 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff400bb40 con 0x7efffc0af7e0 2021-11-21T22:02:03.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- 172.21.2.116:0/1565886527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0af7e0 msgr2=0x7efffc13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- 172.21.2.116:0/1565886527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0af7e0 0x7efffc13fe70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7efff40032f0 tx=0x7efff4009290).stop 2021-11-21T22:02:03.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- 172.21.2.116:0/1565886527 shutdown_connections 2021-11-21T22:02:03.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- 172.21.2.116:0/1565886527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0af7e0 0x7efffc13fe70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- 172.21.2.116:0/1565886527 >> v2:172.21.2.117:3300/0 conn(0x7efffc0affc0 0x7efffc05f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- 172.21.2.116:0/1565886527 >> 172.21.2.116:0/1565886527 conn(0x7efffc0b6000 msgr2=0x7efffc0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:03.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- 172.21.2.116:0/1565886527 shutdown_connections 2021-11-21T22:02:03.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- 172.21.2.116:0/1565886527 wait complete. 2021-11-21T22:02:03.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 Processor -- start 2021-11-21T22:02:03.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- start start 2021-11-21T22:02:03.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0ad780 0x7efffc135bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efffc0af7e0 0x7efffc066b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efffc0affc0 0x7efffc067010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0aa040 con 0x7efffc0ad780 2021-11-21T22:02:03.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0a9e10 con 0x7efffc0affc0 2021-11-21T22:02:03.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7f00021c5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0acee0 con 0x7efffc0af7e0 2021-11-21T22:02:03.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0ad780 0x7efffc135bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0ad780 0x7efffc135bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52202/0 (socket says 172.21.2.116:52202) 2021-11-21T22:02:03.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.798+0000 7efffb7fe700 1 -- 172.21.2.116:0/3147049981 learned_addr learned my addr 172.21.2.116:0/3147049981 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:03.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffb7fe700 1 -- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efffc0af7e0 msgr2=0x7efffc066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffb7fe700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efffc0af7e0 0x7efffc066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffb7fe700 1 -- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efffc0affc0 msgr2=0x7efffc067010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffb7fe700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efffc0affc0 0x7efffc067010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffb7fe700 1 -- 172.21.2.116:0/3147049981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4009530 con 0x7efffc0ad780 2021-11-21T22:02:03.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffb7fe700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0ad780 0x7efffc135bb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7efffc065d90 tx=0x7effec007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efff8ff9700 1 -- 172.21.2.116:0/3147049981 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec00ce20 con 0x7efffc0ad780 2021-11-21T22:02:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efff8ff9700 1 -- 172.21.2.116:0/3147049981 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec00f470 con 0x7efffc0ad780 2021-11-21T22:02:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc0648d0 con 0x7efffc0ad780 2021-11-21T22:02:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efff8ff9700 1 -- 172.21.2.116:0/3147049981 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec00f7d0 con 0x7efffc0ad780 2021-11-21T22:02:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0aca40 con 0x7efffc0ad780 2021-11-21T22:02:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc0acc00 con 0x7efffc0ad780 2021-11-21T22:02:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efff8ff9700 1 -- 172.21.2.116:0/3147049981 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7effec019030 con 0x7efffc0ad780 2021-11-21T22:02:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efff8ff9700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effe0053720 0x7effe0055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efff8ff9700 1 -- 172.21.2.116:0/3147049981 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7effec06adf0 con 0x7efffc0ad780 2021-11-21T22:02:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffaffd700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effe0053720 0x7effe0055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.802+0000 7efffaffd700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effe0053720 0x7effe0055be0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7effe4002f70 tx=0x7effe40069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.806+0000 7efff8ff9700 1 -- 172.21.2.116:0/3147049981 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7effec041f50 con 0x7efffc0ad780 2021-11-21T22:02:03.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.914+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7efffc0b0a40 con 0x7efffc0ad780 2021-11-21T22:02:03.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7efff8ff9700 1 -- 172.21.2.116:0/3147049981 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7effec045570 con 0x7efffc0ad780 2021-11-21T22:02:03.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effe0053720 msgr2=0x7effe0055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effe0053720 0x7effe0055be0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7effe4002f70 tx=0x7effe40069e0).stop 2021-11-21T22:02:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0ad780 msgr2=0x7efffc135bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0ad780 0x7efffc135bb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7efffc065d90 tx=0x7effec007c50).stop 2021-11-21T22:02:03.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 shutdown_connections 2021-11-21T22:02:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efffc0affc0 0x7efffc067010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efffc0af7e0 0x7efffc066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7effe0053720 0x7effe0055be0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 --2- 172.21.2.116:0/3147049981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efffc0ad780 0x7efffc135bb0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:03.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 >> 172.21.2.116:0/3147049981 conn(0x7efffc0b6000 msgr2=0x7efffc0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:03.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 shutdown_connections 2021-11-21T22:02:03.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.922+0000 7f00021c5700 1 -- 172.21.2.116:0/3147049981 wait complete. 2021-11-21T22:02:03.931 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:02:03.933 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:02:03.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f30a0eb1700 1 Processor -- start 2021-11-21T22:02:03.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f30a0eb1700 1 -- start start 2021-11-21T22:02:03.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f30a0eb1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 0x7f309c05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f30a0eb1700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c143f70 con 0x7f309c0affc0 2021-11-21T22:02:03.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f30a0eb1700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c0adb60 con 0x7f309c0af7e0 2021-11-21T22:02:03.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f30a0eb1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c143160 con 0x7f309c0ad780 2021-11-21T22:02:03.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f3099d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f309c0affc0 0x7f309c143860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35938/0 (socket says 172.21.2.116:35938) 2021-11-21T22:02:03.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f3099d9b700 1 -- 172.21.2.116:0/1741843491 learned_addr learned my addr 172.21.2.116:0/1741843491 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:03.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634540387 0 0) 0x7f309c143f70 con 0x7f309c0affc0 2021-11-21T22:02:03.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f307c003260 con 0x7f309c0affc0 2021-11-21T22:02:03.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1918589699 0 0) 0x7f307c003260 con 0x7f309c0affc0 2021-11-21T22:02:03.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 msgr2=0x7f309c05f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:03.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f309959a700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 0x7f309c05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:03.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.982+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 >> v1:172.21.2.117:6789/0 conn(0x7f309c0af7e0 legacy=0x7f309c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:03.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0b0ce0 con 0x7f309c0affc0 2021-11-21T22:02:03.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f3084001660 con 0x7f309c0affc0 2021-11-21T22:02:03.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 >> v1:172.21.2.116:6789/0 conn(0x7f309c0affc0 legacy=0x7f309c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:03.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 0x7f307c003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f307c003e20 0x7f307c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f307c004a60 0x7f307c004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:03.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c143160 con 0x7f307c004a60 2021-11-21T22:02:03.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c00b910 con 0x7f307c003e20 2021-11-21T22:02:03.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c00bad0 con 0x7f309c0ad780 2021-11-21T22:02:03.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f307c004a60 0x7f307c004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f307c003e20 0x7f307c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:03.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 -- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 msgr2=0x7f307c003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309a59c700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 0x7f307c003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 0x7f307c003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 -- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f307c003e20 msgr2=0x7f307c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f307c003e20 0x7f307c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 -- 172.21.2.116:0/1741843491 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0b0ce0 con 0x7f307c004a60 2021-11-21T22:02:04.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309a59c700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c0ad780 0x7f307c003780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:04.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f307c004a60 0x7f307c004ee0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f30940012f0 tx=0x7f3094007d60).ready entity=mon.0 client_cookie=cbe012ee9878e60e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30940014c0 con 0x7f307c004a60 2021-11-21T22:02:04.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3094006870 con 0x7f307c004a60 2021-11-21T22:02:04.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309959a700 1 -- 172.21.2.116:0/1741843491 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3094006bd0 con 0x7f307c004a60 2021-11-21T22:02:04.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- 172.21.2.116:0/1741843491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f307c004a60 msgr2=0x7f307c004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 --2- 172.21.2.116:0/1741843491 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f307c004a60 0x7f307c004ee0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f30940012f0 tx=0x7f3094007d60).stop 2021-11-21T22:02:04.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 -- 172.21.2.116:0/1741843491 reap_dead start 2021-11-21T22:02:04.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- 172.21.2.116:0/1741843491 shutdown_connections 2021-11-21T22:02:04.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- 172.21.2.116:0/1741843491 >> 172.21.2.116:0/1741843491 conn(0x7f309c0b6000 msgr2=0x7f309c0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- 172.21.2.116:0/1741843491 shutdown_connections 2021-11-21T22:02:04.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- 172.21.2.116:0/1741843491 wait complete. 2021-11-21T22:02:04.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 Processor -- start 2021-11-21T22:02:04.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- start start 2021-11-21T22:02:04.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c1421f0 0x7f309c1425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f309c066310 0x7f309c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f309c13fae0 0x7f309c0aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c05be10 con 0x7f309c066310 2021-11-21T22:02:04.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c15aaa0 con 0x7f309c13fae0 2021-11-21T22:02:04.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c065440 con 0x7f309c1421f0 2021-11-21T22:02:04.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f309c066310 0x7f309c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f309c066310 0x7f309c13f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52224/0 (socket says 172.21.2.116:52224) 2021-11-21T22:02:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 -- 172.21.2.116:0/4120415353 learned_addr learned my addr 172.21.2.116:0/4120415353 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f309c13fae0 0x7f309c0aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309a59c700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c1421f0 0x7f309c1425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 -- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c1421f0 msgr2=0x7f309c1425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c1421f0 0x7f309c1425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 -- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f309c13fae0 msgr2=0x7f309c0aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f309c13fae0 0x7f309c0aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 -- 172.21.2.116:0/4120415353 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0656e0 con 0x7f309c066310 2021-11-21T22:02:04.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f309ad9d700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f309c13fae0 0x7f309c0aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:02:04.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f3099d9b700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f309c066310 0x7f309c13f5f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f308400b000 tx=0x7f3084001570).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30937fe700 1 -- 172.21.2.116:0/4120415353 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3084005430 con 0x7f309c066310 2021-11-21T22:02:04.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30937fe700 1 -- 172.21.2.116:0/4120415353 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084010040 con 0x7f309c066310 2021-11-21T22:02:04.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c065980 con 0x7f309c066310 2021-11-21T22:02:04.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30937fe700 1 -- 172.21.2.116:0/4120415353 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3084004710 con 0x7f309c066310 2021-11-21T22:02:04.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.986+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c159d80 con 0x7f309c066310 2021-11-21T22:02:04.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.990+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c064ec0 con 0x7f309c066310 2021-11-21T22:02:04.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.990+0000 7f30937fe700 1 -- 172.21.2.116:0/4120415353 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f308401d070 con 0x7f309c066310 2021-11-21T22:02:04.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.990+0000 7f30937fe700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3080053520 0x7f30800559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.990+0000 7f30937fe700 1 -- 172.21.2.116:0/4120415353 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f308406b180 con 0x7f309c066310 2021-11-21T22:02:04.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.990+0000 7f309a59c700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3080053520 0x7f30800559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.990+0000 7f309a59c700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3080053520 0x7f30800559e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f308c00d000 tx=0x7f308c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:03.990+0000 7f30937fe700 1 -- 172.21.2.116:0/4120415353 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f309c064ec0 con 0x7f309c066310 2021-11-21T22:02:04.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.102+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f309c064ec0 con 0x7f309c066310 2021-11-21T22:02:04.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30937fe700 1 -- 172.21.2.116:0/4120415353 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f309c064ec0 con 0x7f309c066310 2021-11-21T22:02:04.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3080053520 msgr2=0x7f30800559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3080053520 0x7f30800559e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f308c00d000 tx=0x7f308c008040).stop 2021-11-21T22:02:04.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f309c066310 msgr2=0x7f309c13f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f309c066310 0x7f309c13f5f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f308400b000 tx=0x7f3084001570).stop 2021-11-21T22:02:04.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 shutdown_connections 2021-11-21T22:02:04.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f309c13fae0 0x7f309c0aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3080053520 0x7f30800559e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f309c066310 0x7f309c13f5f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 --2- 172.21.2.116:0/4120415353 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f309c1421f0 0x7f309c1425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.106+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 >> 172.21.2.116:0/4120415353 conn(0x7f309c0b6000 msgr2=0x7f309c143780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.110+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 shutdown_connections 2021-11-21T22:02:04.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.110+0000 7f30a0eb1700 1 -- 172.21.2.116:0/4120415353 wait complete. 2021-11-21T22:02:04.116 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:02:04.119 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:04.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 Processor -- start 2021-11-21T22:02:04.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- start start 2021-11-21T22:02:04.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a6aa0 0x7f369014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3690062410 0x7f36901443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36900a6e80 con 0x7f36900a86b0 2021-11-21T22:02:04.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36900a8bc0 con 0x7f3690062410 2021-11-21T22:02:04.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369014a7b0 con 0x7f36900a6aa0 2021-11-21T22:02:04.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3690062410 0x7f36901443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368e7fc700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f36900a86b0 0x7f36901470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35954/0 (socket says 172.21.2.116:35954) 2021-11-21T22:02:04.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368e7fc700 1 -- 172.21.2.116:0/1153674042 learned_addr learned my addr 172.21.2.116:0/1153674042 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:04.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368effd700 1 --2- 172.21.2.116:0/1153674042 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a6aa0 0x7f369014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368f7fe700 1 -- 172.21.2.116:0/1153674042 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a6aa0 msgr2=0x7f369014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368f7fe700 1 --2- 172.21.2.116:0/1153674042 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a6aa0 0x7f369014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368f7fe700 1 -- 172.21.2.116:0/1153674042 >> v1:172.21.2.116:6789/0 conn(0x7f36900a86b0 legacy=0x7f36901470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368f7fe700 1 -- 172.21.2.116:0/1153674042 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369014aa10 con 0x7f3690062410 2021-11-21T22:02:04.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368dffb700 1 -- 172.21.2.116:0/1153674042 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475954672 0 0) 0x7f36900a6e80 con 0x7f36900a86b0 2021-11-21T22:02:04.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368f7fe700 1 --2- 172.21.2.116:0/1153674042 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f36901443f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f36880032f0 tx=0x7f36880091d0).ready entity=mon.0 client_cookie=b64610060a767be4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368dffb700 1 -- 172.21.2.116:0/1153674042 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3688009ea0 con 0x7f3690062410 2021-11-21T22:02:04.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368dffb700 1 -- 172.21.2.116:0/1153674042 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368800b8a0 con 0x7f3690062410 2021-11-21T22:02:04.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f368dffb700 1 -- 172.21.2.116:0/1153674042 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368800ba10 con 0x7f3690062410 2021-11-21T22:02:04.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- 172.21.2.116:0/1153674042 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 msgr2=0x7f36901443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 --2- 172.21.2.116:0/1153674042 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f36901443f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f36880032f0 tx=0x7f36880091d0).stop 2021-11-21T22:02:04.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- 172.21.2.116:0/1153674042 shutdown_connections 2021-11-21T22:02:04.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 --2- 172.21.2.116:0/1153674042 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f36901443f0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 --2- 172.21.2.116:0/1153674042 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a6aa0 0x7f369014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- 172.21.2.116:0/1153674042 >> 172.21.2.116:0/1153674042 conn(0x7f369005bd20 msgr2=0x7f369005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- 172.21.2.116:0/1153674042 shutdown_connections 2021-11-21T22:02:04.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.166+0000 7f369527b700 1 -- 172.21.2.116:0/1153674042 wait complete. 2021-11-21T22:02:04.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 Processor -- start 2021-11-21T22:02:04.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 -- start start 2021-11-21T22:02:04.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f369013efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f36900a6aa0 0x7f3690142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a86b0 0x7f3690143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369014cd80 con 0x7f3690062410 2021-11-21T22:02:04.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369013fa10 con 0x7f36900a6aa0 2021-11-21T22:02:04.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369014cfb0 con 0x7f36900a86b0 2021-11-21T22:02:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f369013efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f369013efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52236/0 (socket says 172.21.2.116:52236) 2021-11-21T22:02:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 -- 172.21.2.116:0/1936595927 learned_addr learned my addr 172.21.2.116:0/1936595927 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 -- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a86b0 msgr2=0x7f3690143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a86b0 0x7f3690143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 -- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f36900a6aa0 msgr2=0x7f3690142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f36900a6aa0 0x7f3690142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 -- 172.21.2.116:0/1936595927 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3688009470 con 0x7f3690062410 2021-11-21T22:02:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368effd700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f369013efe0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f36780030b0 tx=0x7f3678008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f3687fff700 1 -- 172.21.2.116:0/1936595927 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3678011780 con 0x7f3690062410 2021-11-21T22:02:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f3687fff700 1 -- 172.21.2.116:0/1936595927 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36780118e0 con 0x7f3690062410 2021-11-21T22:02:04.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369014d450 con 0x7f3690062410 2021-11-21T22:02:04.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f3687fff700 1 -- 172.21.2.116:0/1936595927 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3678011c40 con 0x7f3690062410 2021-11-21T22:02:04.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369013fcb0 con 0x7f3690062410 2021-11-21T22:02:04.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369014abf0 con 0x7f3690062410 2021-11-21T22:02:04.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f3687fff700 1 -- 172.21.2.116:0/1936595927 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f367801c460 con 0x7f3690062410 2021-11-21T22:02:04.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f3687fff700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3674053720 0x7f3674055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f3687fff700 1 -- 172.21.2.116:0/1936595927 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f367806bb20 con 0x7f3690062410 2021-11-21T22:02:04.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.170+0000 7f368e7fc700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3674053720 0x7f3674055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.174+0000 7f368e7fc700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3674053720 0x7f3674055be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f3680000bc0 tx=0x7f3680007880).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.174+0000 7f3687fff700 1 -- 172.21.2.116:0/1936595927 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3678042c20 con 0x7f3690062410 2021-11-21T22:02:04.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36900573d0 con 0x7f3690062410 2021-11-21T22:02:04.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f3687fff700 1 -- 172.21.2.116:0/1936595927 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f3678046240 con 0x7f3690062410 2021-11-21T22:02:04.288 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:04.288 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":55,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3682554328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3682554328},{"type":"v1","addr":"172.21.2.117:6835","nonce":3682554328}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/2101278175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2101278175},{"type":"v1","addr":"172.21.2.116:6833","nonce":2101278175}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:02.389133+0000","modified":"2021-11-21T22:02:03.393012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1577318848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1577318848},{"type":"v1","addr":"172.21.2.116:6835","nonce":1577318848}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-11-21T22:02:04.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3674053720 msgr2=0x7f3674055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3674053720 0x7f3674055be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f3680000bc0 tx=0x7f3680007880).stop 2021-11-21T22:02:04.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 msgr2=0x7f369013efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f369013efe0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f36780030b0 tx=0x7f3678008040).stop 2021-11-21T22:02:04.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 shutdown_connections 2021-11-21T22:02:04.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f36900a86b0 0x7f3690143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f36900a6aa0 0x7f3690142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3674053720 0x7f3674055be0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 --2- 172.21.2.116:0/1936595927 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3690062410 0x7f369013efe0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 >> 172.21.2.116:0/1936595927 conn(0x7f369005bd20 msgr2=0x7f3690149bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 shutdown_connections 2021-11-21T22:02:04.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.282+0000 7f369527b700 1 -- 172.21.2.116:0/1936595927 wait complete. 2021-11-21T22:02:04.292 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 55 2021-11-21T22:02:04.458 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:04.462 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:04.466 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:04.469 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:04.597 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:02.389133+0000', 'modified': '2021-11-21T22:02:03.393012+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7674}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7674': {'gid': 7674, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/1577318848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 1577318848}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1577318848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:02:04.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:02:04.598 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-11-21T22:02:04.669 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7674,"want_state":"up:active","state":"up:active","rank_uptime":2.2717553320000001,"mdsmap_epoch":55,"osdmap_epoch":122,"osdmap_epoch_barrier":121,"uptime":6.4383634719999998} 2021-11-21T22:02:04.670 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7674, "mdsmap_epoch": 55, "osdmap_epoch": 122, "osdmap_epoch_barrier": 121, "rank_uptime": 2.271755332, "state": "up:active", "uptime": 6.438363472, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:02:04.671 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:04.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 Processor -- start 2021-11-21T22:02:04.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- start start 2021-11-21T22:02:04.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f52100a6aa0 0x7f521014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52100a6e80 con 0x7f52100a86b0 2021-11-21T22:02:04.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521014a7b0 con 0x7f5210062410 2021-11-21T22:02:04.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52100a8bc0 con 0x7f52100a6aa0 2021-11-21T22:02:04.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52157ee700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f52100a86b0 0x7f52101470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:35972/0 (socket says 172.21.2.116:35972) 2021-11-21T22:02:04.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52157ee700 1 -- 172.21.2.116:0/2324034370 learned_addr learned my addr 172.21.2.116:0/2324034370 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:04.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189609198 0 0) 0x7f52100a6e80 con 0x7f52100a86b0 2021-11-21T22:02:04.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f8003260 con 0x7f52100a86b0 2021-11-21T22:02:04.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1003029659 0 0) 0x7f51f8003260 con 0x7f52100a86b0 2021-11-21T22:02:04.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 >> v2:172.21.2.117:3300/0 conn(0x7f52100a6aa0 msgr2=0x7f521014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 --2- 172.21.2.116:0/2324034370 >> v2:172.21.2.117:3300/0 conn(0x7f52100a6aa0 0x7f521014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 >> v1:172.21.2.117:6789/0 conn(0x7f5210062410 legacy=0x7f52101443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521014aa10 con 0x7f52100a86b0 2021-11-21T22:02:04.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f520c001680 con 0x7f52100a86b0 2021-11-21T22:02:04.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 >> v1:172.21.2.116:6789/0 conn(0x7f52100a86b0 legacy=0x7f52101470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f8003820 0x7f51f8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8004500 0x7f51f8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f8005050 0x7f51f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100a8bc0 con 0x7f51f8005050 2021-11-21T22:02:04.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f800bf40 con 0x7f51f8004500 2021-11-21T22:02:04.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f800c100 con 0x7f51f8003820 2021-11-21T22:02:04.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f8005050 0x7f51f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52157ee700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8004500 0x7f51f8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 -- 172.21.2.116:0/2324034370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f8003820 msgr2=0x7f51f8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51f8003820 0x7f51f8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 -- 172.21.2.116:0/2324034370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8004500 msgr2=0x7f51f8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8004500 0x7f51f8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 -- 172.21.2.116:0/2324034370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f8003260 con 0x7f51f8005050 2021-11-21T22:02:04.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 -- 172.21.2.116:0/2324034370 reap_dead start 2021-11-21T22:02:04.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f52167f0700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f8005050 0x7f51f800b750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f52080012f0 tx=0x7f5208007d60).ready entity=mon.0 client_cookie=5f0d4b8ec7836c6f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52080014c0 con 0x7f51f8005050 2021-11-21T22:02:04.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5208006870 con 0x7f51f8005050 2021-11-21T22:02:04.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5214fed700 1 -- 172.21.2.116:0/2324034370 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5208006bd0 con 0x7f51f8005050 2021-11-21T22:02:04.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- 172.21.2.116:0/2324034370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f8005050 msgr2=0x7f51f800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f8005050 0x7f51f800b750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f52080012f0 tx=0x7f5208007d60).stop 2021-11-21T22:02:04.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- 172.21.2.116:0/2324034370 shutdown_connections 2021-11-21T22:02:04.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 --2- 172.21.2.116:0/2324034370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51f8005050 0x7f51f800b750 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- 172.21.2.116:0/2324034370 >> 172.21.2.116:0/2324034370 conn(0x7f521005bd20 msgr2=0x7f521005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- 172.21.2.116:0/2324034370 shutdown_connections 2021-11-21T22:02:04.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.718+0000 7f5218253700 1 -- 172.21.2.116:0/2324034370 wait complete. 2021-11-21T22:02:04.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 Processor -- start 2021-11-21T22:02:04.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 -- start start 2021-11-21T22:02:04.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8005050 0x7f521013c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f521014d2e0 0x7f52101562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f521015a590 0x7f5210156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5210159910 con 0x7f521014d2e0 2021-11-21T22:02:04.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5210159bb0 con 0x7f51f8005050 2021-11-21T22:02:04.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5210159e50 con 0x7f521015a590 2021-11-21T22:02:04.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f521014d2e0 0x7f52101562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f521014d2e0 0x7f52101562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52256/0 (socket says 172.21.2.116:52256) 2021-11-21T22:02:04.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 -- 172.21.2.116:0/3584934256 learned_addr learned my addr 172.21.2.116:0/3584934256 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:04.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5215fef700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8005050 0x7f521013c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 -- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f521015a590 msgr2=0x7f5210156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f521015a590 0x7f5210156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 -- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8005050 msgr2=0x7f521013c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8005050 0x7f521013c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 -- 172.21.2.116:0/3584934256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521015a0f0 con 0x7f521014d2e0 2021-11-21T22:02:04.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f52157ee700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f521014d2e0 0x7f52101562a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f520c00b000 tx=0x7f52100a8520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5202ffd700 1 -- 172.21.2.116:0/3584934256 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f520c015630 con 0x7f521014d2e0 2021-11-21T22:02:04.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5202ffd700 1 -- 172.21.2.116:0/3584934256 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c010040 con 0x7f521014d2e0 2021-11-21T22:02:04.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521014d010 con 0x7f521014d2e0 2021-11-21T22:02:04.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521013ee20 con 0x7f521014d2e0 2021-11-21T22:02:04.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5202ffd700 1 -- 172.21.2.116:0/3584934256 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f520c015980 con 0x7f521014d2e0 2021-11-21T22:02:04.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521015a280 con 0x7f521014d2e0 2021-11-21T22:02:04.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5202ffd700 1 -- 172.21.2.116:0/3584934256 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f520c015ae0 con 0x7f521014d2e0 2021-11-21T22:02:04.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5202ffd700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535d0 0x7f51f0055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5202ffd700 1 -- 172.21.2.116:0/3584934256 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f520c06b600 con 0x7f521014d2e0 2021-11-21T22:02:04.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.722+0000 7f5215fef700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535d0 0x7f51f0055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.726+0000 7f5215fef700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535d0 0x7f51f0055a90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f520400a000 tx=0x7f5204008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.726+0000 7f5202ffd700 1 -- 172.21.2.116:0/3584934256 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f520c042650 con 0x7f521014d2e0 2021-11-21T22:02:04.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.806+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52100573d0 con 0x7f521014d2e0 2021-11-21T22:02:04.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5202ffd700 1 -- 172.21.2.116:0/3584934256 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f520c045c70 con 0x7f521014d2e0 2021-11-21T22:02:04.813 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:04.813 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":122,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:03.394269+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:00.244205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:01.358157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:04.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535d0 msgr2=0x7f51f0055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535d0 0x7f51f0055a90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f520400a000 tx=0x7f5204008040).stop 2021-11-21T22:02:04.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f521014d2e0 msgr2=0x7f52101562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f521014d2e0 0x7f52101562a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f520c00b000 tx=0x7f52100a8520).stop 2021-11-21T22:02:04.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 shutdown_connections 2021-11-21T22:02:04.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f521015a590 0x7f5210156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51f00535d0 0x7f51f0055a90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f521014d2e0 0x7f52101562a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 --2- 172.21.2.116:0/3584934256 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51f8005050 0x7f521013c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 >> 172.21.2.116:0/3584934256 conn(0x7f521005bd20 msgr2=0x7f5210142da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 shutdown_connections 2021-11-21T22:02:04.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.810+0000 7f5218253700 1 -- 172.21.2.116:0/3584934256 wait complete. 2021-11-21T22:02:04.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:02:00.244205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:02:01.358157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:02:04.824 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:02:04.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 Processor -- start 2021-11-21T22:02:04.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- start start 2021-11-21T22:02:04.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ac00af440 0x7f3ac00adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac00b1d90 con 0x7f3ac00b0e30 2021-11-21T22:02:04.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac00b1fc0 con 0x7f3ac0067330 2021-11-21T22:02:04.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac0143a40 con 0x7f3ac00af440 2021-11-21T22:02:04.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac510e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ac00af440 0x7f3ac00adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac510e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ac00af440 0x7f3ac00adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52266/0 (socket says 172.21.2.116:52266) 2021-11-21T22:02:04.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac510e700 1 -- 172.21.2.116:0/590275107 learned_addr learned my addr 172.21.2.116:0/590275107 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:04.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac510e700 1 -- 172.21.2.116:0/590275107 >> v1:172.21.2.117:6789/0 conn(0x7f3ac0067330 legacy=0x7f3ac0067710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:04.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac510e700 1 -- 172.21.2.116:0/590275107 >> v1:172.21.2.116:6789/0 conn(0x7f3ac00b0e30 legacy=0x7f3ac013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac510e700 1 -- 172.21.2.116:0/590275107 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0143ce0 con 0x7f3ac00af440 2021-11-21T22:02:04.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac510e700 1 --2- 172.21.2.116:0/590275107 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac00af440 0x7f3ac00adad0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f3ac005f080 tx=0x7f3abc006eb0).ready entity=mon.0 client_cookie=a37c446d58f24364 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac490d700 1 -- 172.21.2.116:0/590275107 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3abc00d610 con 0x7f3ac00af440 2021-11-21T22:02:04.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac490d700 1 -- 172.21.2.116:0/590275107 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc000e90 con 0x7f3ac00af440 2021-11-21T22:02:04.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac490d700 1 -- 172.21.2.116:0/590275107 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3abc00d960 con 0x7f3ac00af440 2021-11-21T22:02:04.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- 172.21.2.116:0/590275107 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac00af440 msgr2=0x7f3ac00adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/590275107 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac00af440 0x7f3ac00adad0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f3ac005f080 tx=0x7f3abc006eb0).stop 2021-11-21T22:02:04.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- 172.21.2.116:0/590275107 shutdown_connections 2021-11-21T22:02:04.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/590275107 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac00af440 0x7f3ac00adad0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- 172.21.2.116:0/590275107 >> 172.21.2.116:0/590275107 conn(0x7f3ac0061900 msgr2=0x7f3ac0069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- 172.21.2.116:0/590275107 shutdown_connections 2021-11-21T22:02:04.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- 172.21.2.116:0/590275107 wait complete. 2021-11-21T22:02:04.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 Processor -- start 2021-11-21T22:02:04.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- start start 2021-11-21T22:02:04.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac0067330 0x7f3ac0066800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac00af440 0x7f3ac0064250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac00b0e30 0x7f3ac0064740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0064c30 con 0x7f3ac0067330 2021-11-21T22:02:04.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac013eb10 con 0x7f3ac00b0e30 2021-11-21T22:02:04.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac7b73700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac013edb0 con 0x7f3ac00af440 2021-11-21T22:02:04.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac590f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac0067330 0x7f3ac0066800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac590f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac0067330 0x7f3ac0066800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52270/0 (socket says 172.21.2.116:52270) 2021-11-21T22:02:04.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.874+0000 7f3ac590f700 1 -- 172.21.2.116:0/1782343152 learned_addr learned my addr 172.21.2.116:0/1782343152 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:04.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac590f700 1 -- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac00af440 msgr2=0x7f3ac0064250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac590f700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac00af440 0x7f3ac0064250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac590f700 1 -- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac00b0e30 msgr2=0x7f3ac0064740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac590f700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac00b0e30 0x7f3ac0064740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac590f700 1 -- 172.21.2.116:0/1782343152 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc00b1e0 con 0x7f3ac0067330 2021-11-21T22:02:04.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac590f700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac0067330 0x7f3ac0066800 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f3ab4002f90 tx=0x7f3ab4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ab2ffd700 1 -- 172.21.2.116:0/1782343152 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ab4013460 con 0x7f3ac0067330 2021-11-21T22:02:04.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac013f050 con 0x7f3ac0067330 2021-11-21T22:02:04.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ab2ffd700 1 -- 172.21.2.116:0/1782343152 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab40135c0 con 0x7f3ac0067330 2021-11-21T22:02:04.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ab2ffd700 1 -- 172.21.2.116:0/1782343152 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ab4013920 con 0x7f3ac0067330 2021-11-21T22:02:04.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac0064da0 con 0x7f3ac0067330 2021-11-21T22:02:04.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac013f8d0 con 0x7f3ac0067330 2021-11-21T22:02:04.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ab2ffd700 1 -- 172.21.2.116:0/1782343152 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3ab400f070 con 0x7f3ac0067330 2021-11-21T22:02:04.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ab2ffd700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa0053580 0x7f3aa0055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:04.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ab2ffd700 1 -- 172.21.2.116:0/1782343152 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3ab406b250 con 0x7f3ac0067330 2021-11-21T22:02:04.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac510e700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa0053580 0x7f3aa0055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:04.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.878+0000 7f3ac510e700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa0053580 0x7f3aa0055a40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f3ac005f080 tx=0x7f3abc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:04.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.882+0000 7f3ab2ffd700 1 -- 172.21.2.116:0/1782343152 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ac013f8d0 con 0x7f3ac0067330 2021-11-21T22:02:04.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2021-11-21T22:02:04.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3ac013f8d0 con 0x7f3ac0067330 2021-11-21T22:02:04.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ab2ffd700 1 -- 172.21.2.116:0/1782343152 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v122) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3ac013f8d0 con 0x7f3ac0067330 2021-11-21T22:02:04.966 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:02:04.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa0053580 msgr2=0x7f3aa0055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa0053580 0x7f3aa0055a40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f3ac005f080 tx=0x7f3abc015000).stop 2021-11-21T22:02:04.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac0067330 msgr2=0x7f3ac0066800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:04.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac0067330 0x7f3ac0066800 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f3ab4002f90 tx=0x7f3ab4006df0).stop 2021-11-21T22:02:04.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 shutdown_connections 2021-11-21T22:02:04.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac00b0e30 0x7f3ac0064740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac00af440 0x7f3ac0064250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:04.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa0053580 0x7f3aa0055a40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 --2- 172.21.2.116:0/1782343152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac0067330 0x7f3ac0066800 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:04.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 >> 172.21.2.116:0/1782343152 conn(0x7f3ac0061900 msgr2=0x7f3ac005bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:04.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 shutdown_connections 2021-11-21T22:02:04.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:04.962+0000 7f3ac7b73700 1 -- 172.21.2.116:0/1782343152 wait complete. 2021-11-21T22:02:04.974 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:02:05.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5894f9700 1 Processor -- start 2021-11-21T22:02:05.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5894f9700 1 -- start start 2021-11-21T22:02:05.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5894f9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb584062e60 0x7fb584142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5894f9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5840afae0 con 0x7fb5840ad470 2021-11-21T22:02:05.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5894f9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5840b0f60 con 0x7fb5841426e0 2021-11-21T22:02:05.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5894f9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb584063470 con 0x7fb584062e60 2021-11-21T22:02:05.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb5840ad470 0x7fb5840ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36006/0 (socket says 172.21.2.116:36006) 2021-11-21T22:02:05.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 -- 172.21.2.116:0/12323871 learned_addr learned my addr 172.21.2.116:0/12323871 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:05.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160992717 0 0) 0x7fb5840afae0 con 0x7fb5840ad470 2021-11-21T22:02:05.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb564003260 con 0x7fb5840ad470 2021-11-21T22:02:05.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779509660 0 0) 0x7fb564003260 con 0x7fb5840ad470 2021-11-21T22:02:05.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 >> v2:172.21.2.117:3300/0 conn(0x7fb584062e60 msgr2=0x7fb584142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:05.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 --2- 172.21.2.116:0/12323871 >> v2:172.21.2.117:3300/0 conn(0x7fb584062e60 0x7fb584142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 >> v1:172.21.2.117:6789/0 conn(0x7fb5841426e0 legacy=0x7fb5840af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:05.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584143050 con 0x7fb5840ad470 2021-11-21T22:02:05.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb57c0019e0 con 0x7fb5840ad470 2021-11-21T22:02:05.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 >> v1:172.21.2.116:6789/0 conn(0x7fb5840ad470 legacy=0x7fb5840ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb564003820 0x7fb564003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb564004500 0x7fb564004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb564005050 0x7fb564007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb584063470 con 0x7fb564005050 2021-11-21T22:02:05.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564007f20 con 0x7fb564003820 2021-11-21T22:02:05.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640080e0 con 0x7fb564004500 2021-11-21T22:02:05.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb564005050 0x7fb564007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 -- 172.21.2.116:0/12323871 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb564004500 msgr2=0x7fb564004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:05.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb564004500 0x7fb564004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 -- 172.21.2.116:0/12323871 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb564003820 msgr2=0x7fb564003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:05.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb564003820 0x7fb564003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 -- 172.21.2.116:0/12323871 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564003260 con 0x7fb564005050 2021-11-21T22:02:05.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 -- 172.21.2.116:0/12323871 reap_dead start 2021-11-21T22:02:05.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb5837fe700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb564005050 0x7fb564007730 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fb584062db0 tx=0x7fb5840b07d0).ready entity=mon.0 client_cookie=35aeda07fe1dd8ad server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb57c00cc60 con 0x7fb564005050 2021-11-21T22:02:05.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c00cdc0 con 0x7fb564005050 2021-11-21T22:02:05.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.022+0000 7fb581ffb700 1 -- 172.21.2.116:0/12323871 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb57c005b80 con 0x7fb564005050 2021-11-21T22:02:05.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/12323871 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb564005050 msgr2=0x7fb564007730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb564005050 0x7fb564007730 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fb584062db0 tx=0x7fb5840b07d0).stop 2021-11-21T22:02:05.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/12323871 shutdown_connections 2021-11-21T22:02:05.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 --2- 172.21.2.116:0/12323871 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb564005050 0x7fb564007730 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/12323871 >> 172.21.2.116:0/12323871 conn(0x7fb5840b5730 msgr2=0x7fb5840b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/12323871 shutdown_connections 2021-11-21T22:02:05.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/12323871 wait complete. 2021-11-21T22:02:05.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 Processor -- start 2021-11-21T22:02:05.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- start start 2021-11-21T22:02:05.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb564005050 0x7fb5840650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb58413f9c0 0x7fb584066d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb584067230 0x7fb58414da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb584064020 con 0x7fb584067230 2021-11-21T22:02:05.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb584064250 con 0x7fb564005050 2021-11-21T22:02:05.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840644f0 con 0x7fb58413f9c0 2021-11-21T22:02:05.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb584067230 0x7fb58414da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb584067230 0x7fb58414da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52292/0 (socket says 172.21.2.116:52292) 2021-11-21T22:02:05.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 -- 172.21.2.116:0/1377308981 learned_addr learned my addr 172.21.2.116:0/1377308981 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:05.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 -- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb58413f9c0 msgr2=0x7fb584066d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb58413f9c0 0x7fb584066d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 -- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb564005050 msgr2=0x7fb5840650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb564005050 0x7fb5840650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 -- 172.21.2.116:0/1377308981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58413f520 con 0x7fb584067230 2021-11-21T22:02:05.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5837fe700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb584067230 0x7fb58414da50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fb584062db0 tx=0x7fb57c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb57bfff700 1 -- 172.21.2.116:0/1377308981 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb57c001ec0 con 0x7fb584067230 2021-11-21T22:02:05.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb57bfff700 1 -- 172.21.2.116:0/1377308981 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c0072e0 con 0x7fb584067230 2021-11-21T22:02:05.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58413e9d0 con 0x7fb584067230 2021-11-21T22:02:05.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb57bfff700 1 -- 172.21.2.116:0/1377308981 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb57c007640 con 0x7fb584067230 2021-11-21T22:02:05.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58414e160 con 0x7fb584067230 2021-11-21T22:02:05.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58413f6b0 con 0x7fb584067230 2021-11-21T22:02:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb57bfff700 1 -- 172.21.2.116:0/1377308981 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb57c01d070 con 0x7fb584067230 2021-11-21T22:02:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb57bfff700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5680536f0 0x7fb568055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.026+0000 7fb57bfff700 1 -- 172.21.2.116:0/1377308981 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb57c06c790 con 0x7fb584067230 2021-11-21T22:02:05.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.030+0000 7fb582ffd700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5680536f0 0x7fb568055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.030+0000 7fb582ffd700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5680536f0 0x7fb568055bb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fb574000f70 tx=0x7fb5740069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.030+0000 7fb57bfff700 1 -- 172.21.2.116:0/1377308981 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb57c0437e0 con 0x7fb584067230 2021-11-21T22:02:05.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb5840b1e00 con 0x7fb584067230 2021-11-21T22:02:05.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb57bfff700 1 -- 172.21.2.116:0/1377308981 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v122) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb57c01a020 con 0x7fb584067230 2021-11-21T22:02:05.118 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:05.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5680536f0 msgr2=0x7fb568055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5680536f0 0x7fb568055bb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fb574000f70 tx=0x7fb5740069b0).stop 2021-11-21T22:02:05.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb584067230 msgr2=0x7fb58414da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb584067230 0x7fb58414da50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fb584062db0 tx=0x7fb57c015000).stop 2021-11-21T22:02:05.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 shutdown_connections 2021-11-21T22:02:05.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb5680536f0 0x7fb568055bb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb584067230 0x7fb58414da50 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb58413f9c0 0x7fb584066d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 --2- 172.21.2.116:0/1377308981 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb564005050 0x7fb5840650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 >> 172.21.2.116:0/1377308981 conn(0x7fb5840b5730 msgr2=0x7fb5840b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 shutdown_connections 2021-11-21T22:02:05.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.114+0000 7fb5894f9700 1 -- 172.21.2.116:0/1377308981 wait complete. 2021-11-21T22:02:05.125 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:02:05.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 Processor -- start 2021-11-21T22:02:05.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- start start 2021-11-21T22:02:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f92d4144810 0x7f92d41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f92d40abce0 0x7f92d40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d4147e80 con 0x7f92d40b1db0 2021-11-21T22:02:05.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d40b1490 con 0x7f92d40abce0 2021-11-21T22:02:05.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d40af060 con 0x7f92d4144810 2021-11-21T22:02:05.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f92d40abce0 0x7f92d40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f92d40abce0 0x7f92d40b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52302/0 (socket says 172.21.2.116:52302) 2021-11-21T22:02:05.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 -- 172.21.2.116:0/1594164502 learned_addr learned my addr 172.21.2.116:0/1594164502 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 -- 172.21.2.116:0/1594164502 >> v2:172.21.2.117:3300/0 conn(0x7f92d4144810 msgr2=0x7f92d41478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 --2- 172.21.2.116:0/1594164502 >> v2:172.21.2.117:3300/0 conn(0x7f92d4144810 0x7f92d41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 -- 172.21.2.116:0/1594164502 >> v1:172.21.2.117:6789/0 conn(0x7f92d40b1db0 legacy=0x7f92d40ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:05.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 -- 172.21.2.116:0/1594164502 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40af5a0 con 0x7f92d40abce0 2021-11-21T22:02:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d99e0700 1 --2- 172.21.2.116:0/1594164502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d40abce0 0x7f92d40b18c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f92d40b4e00 tx=0x7f92c4006eb0).ready entity=mon.0 client_cookie=b21244a9526d5958 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d91df700 1 -- 172.21.2.116:0/1594164502 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c400d5c0 con 0x7f92d40abce0 2021-11-21T22:02:05.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d91df700 1 -- 172.21.2.116:0/1594164502 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4000e90 con 0x7f92d40abce0 2021-11-21T22:02:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92d91df700 1 -- 172.21.2.116:0/1594164502 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c400d910 con 0x7f92d40abce0 2021-11-21T22:02:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- 172.21.2.116:0/1594164502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d40abce0 msgr2=0x7f92d40b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 --2- 172.21.2.116:0/1594164502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d40abce0 0x7f92d40b18c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f92d40b4e00 tx=0x7f92c4006eb0).stop 2021-11-21T22:02:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- 172.21.2.116:0/1594164502 shutdown_connections 2021-11-21T22:02:05.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 --2- 172.21.2.116:0/1594164502 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d40abce0 0x7f92d40b18c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 --2- 172.21.2.116:0/1594164502 >> v2:172.21.2.117:3300/0 conn(0x7f92d4144810 0x7f92d41478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- 172.21.2.116:0/1594164502 >> 172.21.2.116:0/1594164502 conn(0x7f92d405bd20 msgr2=0x7f92d405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- 172.21.2.116:0/1594164502 shutdown_connections 2021-11-21T22:02:05.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.174+0000 7f92dc445700 1 -- 172.21.2.116:0/1594164502 wait complete. 2021-11-21T22:02:05.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 Processor -- start 2021-11-21T22:02:05.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 -- start start 2021-11-21T22:02:05.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f92d40abce0 0x7f92d4143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f92d4144810 0x7f92d415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d41567d0 0x7f92d415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4145ca0 con 0x7f92d41567d0 2021-11-21T22:02:05.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4159ea0 con 0x7f92d40abce0 2021-11-21T22:02:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4145a00 con 0x7f92d4144810 2021-11-21T22:02:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d41567d0 0x7f92d415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d41567d0 0x7f92d415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52308/0 (socket says 172.21.2.116:52308) 2021-11-21T22:02:05.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 -- 172.21.2.116:0/3456464001 learned_addr learned my addr 172.21.2.116:0/3456464001 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da1e1700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f92d40abce0 0x7f92d4143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 -- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f92d4144810 msgr2=0x7f92d415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92d99e0700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f92d4144810 0x7f92d415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f92d4144810 0x7f92d415a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 -- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f92d40abce0 msgr2=0x7f92d4143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f92d40abce0 0x7f92d4143370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 -- 172.21.2.116:0/3456464001 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c400b1e0 con 0x7f92d41567d0 2021-11-21T22:02:05.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92d99e0700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f92d4144810 0x7f92d415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:05.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da9e2700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d41567d0 0x7f92d415a7f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f92cc003310 tx=0x7f92cc0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92c37fe700 1 -- 172.21.2.116:0/3456464001 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92cc001500 con 0x7f92d41567d0 2021-11-21T22:02:05.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92c37fe700 1 -- 172.21.2.116:0/3456464001 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc001660 con 0x7f92d41567d0 2021-11-21T22:02:05.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92c37fe700 1 -- 172.21.2.116:0/3456464001 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92cc0019c0 con 0x7f92d41567d0 2021-11-21T22:02:05.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d4143860 con 0x7f92d41567d0 2021-11-21T22:02:05.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d40646c0 con 0x7f92d41567d0 2021-11-21T22:02:05.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d4144f70 con 0x7f92d41567d0 2021-11-21T22:02:05.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92c37fe700 1 -- 172.21.2.116:0/3456464001 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f92cc018030 con 0x7f92d41567d0 2021-11-21T22:02:05.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92c37fe700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f92b4053610 0x7f92b4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92c37fe700 1 -- 172.21.2.116:0/3456464001 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f92cc06b2c0 con 0x7f92d41567d0 2021-11-21T22:02:05.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.178+0000 7f92da1e1700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f92b4053610 0x7f92b4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.182+0000 7f92da1e1700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f92b4053610 0x7f92b4055ad0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f92d000a000 tx=0x7f92d0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.182+0000 7f92c37fe700 1 -- 172.21.2.116:0/3456464001 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92cc0424d0 con 0x7f92d41567d0 2021-11-21T22:02:05.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92d40af300 con 0x7f92d41567d0 2021-11-21T22:02:05.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92c37fe700 1 -- 172.21.2.116:0/3456464001 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+10 (secure 0 0 0) 0x7f92cc045af0 con 0x7f92d41567d0 2021-11-21T22:02:05.270 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:05.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f92b4053610 msgr2=0x7f92b4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f92b4053610 0x7f92b4055ad0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f92d000a000 tx=0x7f92d0008040).stop 2021-11-21T22:02:05.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d41567d0 msgr2=0x7f92d415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d41567d0 0x7f92d415a7f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f92cc003310 tx=0x7f92cc0091a0).stop 2021-11-21T22:02:05.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 shutdown_connections 2021-11-21T22:02:05.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f92b4053610 0x7f92b4055ad0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f92d41567d0 0x7f92d415a7f0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f92d4144810 0x7f92d415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 --2- 172.21.2.116:0/3456464001 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f92d40abce0 0x7f92d4143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 >> 172.21.2.116:0/3456464001 conn(0x7f92d405bd20 msgr2=0x7f92d405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 shutdown_connections 2021-11-21T22:02:05.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.266+0000 7f92dc445700 1 -- 172.21.2.116:0/3456464001 wait complete. 2021-11-21T22:02:05.276 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:02:05.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 Processor -- start 2021-11-21T22:02:05.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- start start 2021-11-21T22:02:05.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80140abce0 0x7f80141478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8014147e80 con 0x7f8014144810 2021-11-21T22:02:05.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140af5a0 con 0x7f80140ac600 2021-11-21T22:02:05.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80140b1490 con 0x7f80140abce0 2021-11-21T22:02:05.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f80137fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80140abce0 0x7f80141478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f80137fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f80140abce0 0x7f80141478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52316/0 (socket says 172.21.2.116:52316) 2021-11-21T22:02:05.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f80137fe700 1 -- 172.21.2.116:0/3786941764 learned_addr learned my addr 172.21.2.116:0/3786941764 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:05.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f80137fe700 1 -- 172.21.2.116:0/3786941764 >> v1:172.21.2.117:6789/0 conn(0x7f80140ac600 legacy=0x7f80140aca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:05.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f80137fe700 1 -- 172.21.2.116:0/3786941764 >> v1:172.21.2.116:6789/0 conn(0x7f8014144810 legacy=0x7f80140b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f80137fe700 1 -- 172.21.2.116:0/3786941764 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140b1fd0 con 0x7f80140abce0 2021-11-21T22:02:05.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f80137fe700 1 --2- 172.21.2.116:0/3786941764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80140abce0 0x7f80141478c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f80140b4e00 tx=0x7f7ffc006eb0).ready entity=mon.0 client_cookie=d22b208ad221f140 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f8012ffd700 1 -- 172.21.2.116:0/3786941764 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ffc00d610 con 0x7f80140abce0 2021-11-21T22:02:05.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f8012ffd700 1 -- 172.21.2.116:0/3786941764 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc000e90 con 0x7f80140abce0 2021-11-21T22:02:05.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f8012ffd700 1 -- 172.21.2.116:0/3786941764 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ffc00d960 con 0x7f80140abce0 2021-11-21T22:02:05.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- 172.21.2.116:0/3786941764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80140abce0 msgr2=0x7f80141478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 --2- 172.21.2.116:0/3786941764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80140abce0 0x7f80141478c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f80140b4e00 tx=0x7f7ffc006eb0).stop 2021-11-21T22:02:05.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- 172.21.2.116:0/3786941764 shutdown_connections 2021-11-21T22:02:05.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 --2- 172.21.2.116:0/3786941764 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f80140abce0 0x7f80141478c0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- 172.21.2.116:0/3786941764 >> 172.21.2.116:0/3786941764 conn(0x7f801405bd20 msgr2=0x7f801405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- 172.21.2.116:0/3786941764 shutdown_connections 2021-11-21T22:02:05.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 -- 172.21.2.116:0/3786941764 wait complete. 2021-11-21T22:02:05.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.326+0000 7f801a268700 1 Processor -- start 2021-11-21T22:02:05.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 -- start start 2021-11-21T22:02:05.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80140abce0 0x7f8014067720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80140ac600 0x7f8014143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8014144810 0x7f80141465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014063780 con 0x7f8014144810 2021-11-21T22:02:05.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014145c70 con 0x7f80140abce0 2021-11-21T22:02:05.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80141451c0 con 0x7f80140ac600 2021-11-21T22:02:05.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8014144810 0x7f80141465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8014144810 0x7f80141465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52322/0 (socket says 172.21.2.116:52322) 2021-11-21T22:02:05.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 -- 172.21.2.116:0/975454759 learned_addr learned my addr 172.21.2.116:0/975454759 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:05.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 -- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80140ac600 msgr2=0x7f8014143510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:05.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80140ac600 0x7f8014143510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 -- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80140abce0 msgr2=0x7f8014067720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80140abce0 0x7f8014067720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 -- 172.21.2.116:0/975454759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc00b1e0 con 0x7f8014144810 2021-11-21T22:02:05.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8018805700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8014144810 0x7f80141465c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f800c003330 tx=0x7f800c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f80117fa700 1 -- 172.21.2.116:0/975454759 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f800c001520 con 0x7f8014144810 2021-11-21T22:02:05.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f80117fa700 1 -- 172.21.2.116:0/975454759 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c001680 con 0x7f8014144810 2021-11-21T22:02:05.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f80117fa700 1 -- 172.21.2.116:0/975454759 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f800c0019e0 con 0x7f8014144810 2021-11-21T22:02:05.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8014145460 con 0x7f8014144810 2021-11-21T22:02:05.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80140629c0 con 0x7f8014144810 2021-11-21T22:02:05.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80141455f0 con 0x7f8014144810 2021-11-21T22:02:05.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f80117fa700 1 -- 172.21.2.116:0/975454759 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f800c018030 con 0x7f8014144810 2021-11-21T22:02:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f80117fa700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ff8053660 0x7f7ff8055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f80117fa700 1 -- 172.21.2.116:0/975454759 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f800c06b430 con 0x7f8014144810 2021-11-21T22:02:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.330+0000 7f8013fff700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ff8053660 0x7f7ff8055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.334+0000 7f8013fff700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ff8053660 0x7f7ff8055b20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f8014062500 tx=0x7f80040077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.334+0000 7f80117fa700 1 -- 172.21.2.116:0/975454759 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f800c042530 con 0x7f8014144810 2021-11-21T22:02:05.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80140af060 con 0x7f8014144810 2021-11-21T22:02:05.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f80117fa700 1 -- 172.21.2.116:0/975454759 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+10 (secure 0 0 0) 0x7f800c045b50 con 0x7f8014144810 2021-11-21T22:02:05.421 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:05.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ff8053660 msgr2=0x7f7ff8055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ff8053660 0x7f7ff8055b20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f8014062500 tx=0x7f80040077f0).stop 2021-11-21T22:02:05.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8014144810 msgr2=0x7f80141465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8014144810 0x7f80141465c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f800c003330 tx=0x7f800c0091a0).stop 2021-11-21T22:02:05.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 shutdown_connections 2021-11-21T22:02:05.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ff8053660 0x7f7ff8055b20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8014144810 0x7f80141465c0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f80140ac600 0x7f8014143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 --2- 172.21.2.116:0/975454759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f80140abce0 0x7f8014067720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 >> 172.21.2.116:0/975454759 conn(0x7f801405bd20 msgr2=0x7f801405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 shutdown_connections 2021-11-21T22:02:05.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.418+0000 7f801a268700 1 -- 172.21.2.116:0/975454759 wait complete. 2021-11-21T22:02:05.429 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:02:05.429 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:05.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 Processor -- start 2021-11-21T22:02:05.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- start start 2021-11-21T22:02:05.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9ad80ad850 0x7f9ad80afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9ad80b09d0 0x7f9ad80af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad805c780 con 0x7f9ad80af6b0 2021-11-21T22:02:05.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80adc30 con 0x7f9ad80b09d0 2021-11-21T22:02:05.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80b1ed0 con 0x7f9ad80ad850 2021-11-21T22:02:05.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9ad80b09d0 0x7f9ad80af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9ad80b09d0 0x7f9ad80af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52326/0 (socket says 172.21.2.116:52326) 2021-11-21T22:02:05.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 -- 172.21.2.116:0/245861358 learned_addr learned my addr 172.21.2.116:0/245861358 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:05.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 -- 172.21.2.116:0/245861358 >> v2:172.21.2.117:3300/0 conn(0x7f9ad80ad850 msgr2=0x7f9ad80afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:05.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 --2- 172.21.2.116:0/245861358 >> v2:172.21.2.117:3300/0 conn(0x7f9ad80ad850 0x7f9ad80afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 -- 172.21.2.116:0/245861358 >> v1:172.21.2.116:6789/0 conn(0x7f9ad80af6b0 legacy=0x7f9ad805f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 -- 172.21.2.116:0/245861358 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80b2170 con 0x7f9ad80b09d0 2021-11-21T22:02:05.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad77fe700 1 --2- 172.21.2.116:0/245861358 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad80af1c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f9ac0002f70 tx=0x7f9ac0006eb0).ready entity=mon.0 client_cookie=d6ba6349f2b2fe23 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad6ffd700 1 -- 172.21.2.116:0/245861358 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac000d610 con 0x7f9ad80b09d0 2021-11-21T22:02:05.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad6ffd700 1 -- 172.21.2.116:0/245861358 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0000e90 con 0x7f9ad80b09d0 2021-11-21T22:02:05.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad6ffd700 1 -- 172.21.2.116:0/245861358 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac000d9a0 con 0x7f9ad80b09d0 2021-11-21T22:02:05.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- 172.21.2.116:0/245861358 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 msgr2=0x7f9ad80af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/245861358 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad80af1c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f9ac0002f70 tx=0x7f9ac0006eb0).stop 2021-11-21T22:02:05.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- 172.21.2.116:0/245861358 shutdown_connections 2021-11-21T22:02:05.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/245861358 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad80af1c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/245861358 >> v2:172.21.2.117:3300/0 conn(0x7f9ad80ad850 0x7f9ad80afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- 172.21.2.116:0/245861358 >> 172.21.2.116:0/245861358 conn(0x7f9ad80b6030 msgr2=0x7f9ad80b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- 172.21.2.116:0/245861358 shutdown_connections 2021-11-21T22:02:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- 172.21.2.116:0/245861358 wait complete. 2021-11-21T22:02:05.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 Processor -- start 2021-11-21T22:02:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- start start 2021-11-21T22:02:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9ad80ad850 0x7f9ad8148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9ad80af6b0 0x7f9ad8146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad8146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad81571d0 con 0x7f9ad80b09d0 2021-11-21T22:02:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8064100 con 0x7f9ad80af6b0 2021-11-21T22:02:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ade7f5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80643a0 con 0x7f9ad80ad850 2021-11-21T22:02:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad8146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad8146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52332/0 (socket says 172.21.2.116:52332) 2021-11-21T22:02:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 -- 172.21.2.116:0/2299622532 learned_addr learned my addr 172.21.2.116:0/2299622532 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 -- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9ad80ad850 msgr2=0x7f9ad8148470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9ad80ad850 0x7f9ad8148470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 -- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9ad80af6b0 msgr2=0x7f9ad8146690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:05.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9ad80af6b0 0x7f9ad8146690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 -- 172.21.2.116:0/2299622532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac000b1e0 con 0x7f9ad80b09d0 2021-11-21T22:02:05.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9adcd92700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad8146b80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f9ad8063280 tx=0x7f9ad0007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.478+0000 7f9ad57fa700 1 -- 172.21.2.116:0/2299622532 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad000e4a0 con 0x7f9ad80b09d0 2021-11-21T22:02:05.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad57fa700 1 -- 172.21.2.116:0/2299622532 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad000e600 con 0x7f9ad80b09d0 2021-11-21T22:02:05.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8064640 con 0x7f9ad80b09d0 2021-11-21T22:02:05.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad8066620 con 0x7f9ad80b09d0 2021-11-21T22:02:05.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad57fa700 1 -- 172.21.2.116:0/2299622532 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad000e960 con 0x7f9ad80b09d0 2021-11-21T22:02:05.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad80b2410 con 0x7f9ad80b09d0 2021-11-21T22:02:05.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad57fa700 1 -- 172.21.2.116:0/2299622532 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9ad0019030 con 0x7f9ad80b09d0 2021-11-21T22:02:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad57fa700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9abc0536a0 0x7f9abc055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad57fa700 1 -- 172.21.2.116:0/2299622532 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9ad006af10 con 0x7f9ad80b09d0 2021-11-21T22:02:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad7fff700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9abc0536a0 0x7f9abc055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad7fff700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9abc0536a0 0x7f9abc055b60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f9ac8002f70 tx=0x7f9ac8006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:05.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.482+0000 7f9ad57fa700 1 -- 172.21.2.116:0/2299622532 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ad0042070 con 0x7f9ad80b09d0 2021-11-21T22:02:05.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.594+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ad805c780 con 0x7f9ad80b09d0 2021-11-21T22:02:05.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.594+0000 7f9ad57fa700 1 -- 172.21.2.116:0/2299622532 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f9ad805c780 con 0x7f9ad80b09d0 2021-11-21T22:02:05.600 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:05.600 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":55,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3682554328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3682554328},{"type":"v1","addr":"172.21.2.117:6835","nonce":3682554328}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/2101278175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2101278175},{"type":"v1","addr":"172.21.2.116:6833","nonce":2101278175}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:02.389133+0000","modified":"2021-11-21T22:02:03.393012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1577318848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1577318848},{"type":"v1","addr":"172.21.2.116:6835","nonce":1577318848}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-11-21T22:02:05.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.594+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9abc0536a0 msgr2=0x7f9abc055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.594+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9abc0536a0 0x7f9abc055b60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f9ac8002f70 tx=0x7f9ac8006a00).stop 2021-11-21T22:02:05.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.594+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 msgr2=0x7f9ad8146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:05.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.594+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad8146b80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f9ad8063280 tx=0x7f9ad0007f10).stop 2021-11-21T22:02:05.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 shutdown_connections 2021-11-21T22:02:05.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9abc0536a0 0x7f9abc055b60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9ad80b09d0 0x7f9ad8146b80 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:05.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9ad80af6b0 0x7f9ad8146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 --2- 172.21.2.116:0/2299622532 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9ad80ad850 0x7f9ad8148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:05.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 >> 172.21.2.116:0/2299622532 conn(0x7f9ad80b6030 msgr2=0x7f9ad80b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:05.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 shutdown_connections 2021-11-21T22:02:05.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:05.598+0000 7f9ade7f5700 1 -- 172.21.2.116:0/2299622532 wait complete. 2021-11-21T22:02:05.603 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 55 2021-11-21T22:02:05.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:02.389133+0000', 'modified': '2021-11-21T22:02:03.393012+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7674}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7674': {'gid': 7674, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/1577318848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 1577318848}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1577318848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:02:05.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:02:05.911 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-11-21T22:02:05.971 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7674,"want_state":"up:active","state":"up:active","rank_uptime":3.5743189549999999,"mdsmap_epoch":55,"osdmap_epoch":122,"osdmap_epoch_barrier":121,"uptime":7.7409267679999996} 2021-11-21T22:02:05.972 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7674, "mdsmap_epoch": 55, "osdmap_epoch": 122, "osdmap_epoch_barrier": 121, "rank_uptime": 3.574318955, "state": "up:active", "uptime": 7.740926768, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:02:05.972 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:02:05.973 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:02:05.973 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:02:05.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:05.975 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:05.986 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:02:06.040 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:06.041 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:06.048 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:06.049 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:06.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:06.051 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:02:06.056 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:02:06.056 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:06.101 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:02:06.111 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:06.112 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:06.117 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:06.118 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:06.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:06.119 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:02:06.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:02:06.137+0000 7f4ff0289080 -1 init, newargv = 0x563c59151cd0 newargc=9 2021-11-21T22:02:06.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15048]: starting ceph client 2021-11-21T22:02:06.151 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15048]: starting fuse 2021-11-21T22:02:07.245 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:02:07.257 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:07.257 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:07.308 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:07.309 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:07.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:07.310 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:02:07.315 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:02:07.316 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:02:07.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:07.317 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:02:07.317 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:02:07.318 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:02:07.318 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:02:07.319 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:02:07.319 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:07.320 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:02:07.320 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:02:07.321 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:02:07.321 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:07.321 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:02:07.324 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:02:07.325 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:02:07.325 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:02:07.325 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:07.326 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:02:07.326 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:02:07.327 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:02:07.327 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:02:07.327 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:02:07.328 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:07.328 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:02:07.329 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:02:07.392 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15048.asok 2021-11-21T22:02:07.395 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15048.asok 2021-11-21T22:02:07.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:07.396 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15048.asok status 2021-11-21T22:02:07.443 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:02:07.443 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:02:07.444 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:02:07.444 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:02:07.444 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:02:07.444 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:02:07.445 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:02:07.445 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15048", 2021-11-21T22:02:07.445 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:02:07.445 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:07.445 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:02:07.446 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:02:07.446 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 7848, 2021-11-21T22:02:07.446 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:02:07.446 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:02:07.447 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:02:07.447 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 7848 2021-11-21T22:02:07.447 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:07.447 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:02:07.447 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:02:07.448 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:02:07.448 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1984994386 2021-11-21T22:02:07.448 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:02:07.448 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:07.449 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:02:07.449 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:02:07.449 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:02:07.449 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1984994386 2021-11-21T22:02:07.449 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:07.450 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.7848 172.21.2.119:0/1984994386", 2021-11-21T22:02:07.450 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/1984994386", 2021-11-21T22:02:07.450 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:02:07.450 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 55, 2021-11-21T22:02:07.451 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 122, 2021-11-21T22:02:07.451 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:02:07.451 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:02:07.451 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:02:07.451 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:02:07.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:07.453 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:07.454 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:07.459 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:02:07.459 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:07.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:07.460 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:07.469 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:07.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d5d603700 1 Processor -- start 2021-11-21T22:02:07.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d5d603700 1 -- start start 2021-11-21T22:02:07.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d5d603700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6d580a86b0 0x7f6d5814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d5d603700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6d58062410 0x7f6d581443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d5d603700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d580a6e80 con 0x7f6d580a6aa0 2021-11-21T22:02:07.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d5d603700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d580a8bc0 con 0x7f6d58062410 2021-11-21T22:02:07.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d5d603700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5814a7b0 con 0x7f6d580a86b0 2021-11-21T22:02:07.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d577fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6d58062410 0x7f6d581443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d577fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6d58062410 0x7f6d581443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52344/0 (socket says 172.21.2.116:52344) 2021-11-21T22:02:07.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d577fe700 1 -- 172.21.2.116:0/1553442851 learned_addr learned my addr 172.21.2.116:0/1553442851 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:07.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d567fc700 1 --2- 172.21.2.116:0/1553442851 >> v2:172.21.2.117:3300/0 conn(0x7f6d580a86b0 0x7f6d5814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d577fe700 1 -- 172.21.2.116:0/1553442851 >> v2:172.21.2.117:3300/0 conn(0x7f6d580a86b0 msgr2=0x7f6d5814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:07.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.518+0000 7f6d577fe700 1 --2- 172.21.2.116:0/1553442851 >> v2:172.21.2.117:3300/0 conn(0x7f6d580a86b0 0x7f6d5814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d577fe700 1 -- 172.21.2.116:0/1553442851 >> v1:172.21.2.117:6789/0 conn(0x7f6d580a6aa0 legacy=0x7f6d581470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:07.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d577fe700 1 -- 172.21.2.116:0/1553442851 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5814aa10 con 0x7f6d58062410 2021-11-21T22:02:07.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d577fe700 1 --2- 172.21.2.116:0/1553442851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d581443f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f6d58061ab0 tx=0x7f6d50009290).ready entity=mon.0 client_cookie=793259f8cc9a318a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d55ffb700 1 -- 172.21.2.116:0/1553442851 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d5000b830 con 0x7f6d58062410 2021-11-21T22:02:07.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d55ffb700 1 -- 172.21.2.116:0/1553442851 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5000b990 con 0x7f6d58062410 2021-11-21T22:02:07.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d55ffb700 1 -- 172.21.2.116:0/1553442851 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d5000bb00 con 0x7f6d58062410 2021-11-21T22:02:07.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- 172.21.2.116:0/1553442851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 msgr2=0x7f6d581443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 --2- 172.21.2.116:0/1553442851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d581443f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f6d58061ab0 tx=0x7f6d50009290).stop 2021-11-21T22:02:07.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- 172.21.2.116:0/1553442851 shutdown_connections 2021-11-21T22:02:07.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 --2- 172.21.2.116:0/1553442851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d581443f0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 --2- 172.21.2.116:0/1553442851 >> v2:172.21.2.117:3300/0 conn(0x7f6d580a86b0 0x7f6d5814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- 172.21.2.116:0/1553442851 >> 172.21.2.116:0/1553442851 conn(0x7f6d5805bd20 msgr2=0x7f6d5805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:07.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- 172.21.2.116:0/1553442851 shutdown_connections 2021-11-21T22:02:07.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- 172.21.2.116:0/1553442851 wait complete. 2021-11-21T22:02:07.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 Processor -- start 2021-11-21T22:02:07.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- start start 2021-11-21T22:02:07.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d58066d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d580a6aa0 0x7f6d581564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d580a86b0 0x7f6d58159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58065f20 con 0x7f6d58062410 2021-11-21T22:02:07.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580676a0 con 0x7f6d580a6aa0 2021-11-21T22:02:07.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580aace0 con 0x7f6d580a86b0 2021-11-21T22:02:07.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d58066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d58066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52346/0 (socket says 172.21.2.116:52346) 2021-11-21T22:02:07.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 -- 172.21.2.116:0/3023214732 learned_addr learned my addr 172.21.2.116:0/3023214732 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:07.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d567fc700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d580a6aa0 0x7f6d581564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d577fe700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d580a86b0 0x7f6d58159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 -- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d580a86b0 msgr2=0x7f6d58159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d580a86b0 0x7f6d58159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 -- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d580a6aa0 msgr2=0x7f6d581564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d580a6aa0 0x7f6d581564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 -- 172.21.2.116:0/3023214732 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50009530 con 0x7f6d58062410 2021-11-21T22:02:07.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d56ffd700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d58066d80 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f6d48002f90 tx=0x7f6d48006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d4ffff700 1 -- 172.21.2.116:0/3023214732 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d48009e20 con 0x7f6d58062410 2021-11-21T22:02:07.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d4ffff700 1 -- 172.21.2.116:0/3023214732 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d48013460 con 0x7f6d58062410 2021-11-21T22:02:07.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d4ffff700 1 -- 172.21.2.116:0/3023214732 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d480137c0 con 0x7f6d58062410 2021-11-21T22:02:07.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d58064170 con 0x7f6d58062410 2021-11-21T22:02:07.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.522+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d580aa6a0 con 0x7f6d58062410 2021-11-21T22:02:07.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.526+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d58064610 con 0x7f6d58062410 2021-11-21T22:02:07.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.526+0000 7f6d4ffff700 1 -- 172.21.2.116:0/3023214732 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6d4800f070 con 0x7f6d58062410 2021-11-21T22:02:07.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.526+0000 7f6d4ffff700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d3c0534f0 0x7f6d3c0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.526+0000 7f6d4ffff700 1 -- 172.21.2.116:0/3023214732 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d4806b1a0 con 0x7f6d58062410 2021-11-21T22:02:07.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.526+0000 7f6d567fc700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d3c0534f0 0x7f6d3c0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.526+0000 7f6d567fc700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d3c0534f0 0x7f6d3c0559b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f6d4000c000 tx=0x7f6d40008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.526+0000 7f6d4ffff700 1 -- 172.21.2.116:0/3023214732 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d48042170 con 0x7f6d58062410 2021-11-21T22:02:07.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.610+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d5805db70 con 0x7f6d58062410 2021-11-21T22:02:07.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.610+0000 7f6d4ffff700 1 -- 172.21.2.116:0/3023214732 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f6d48045790 con 0x7f6d58062410 2021-11-21T22:02:07.616 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:07.616 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":122,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:03.394269+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:00.244205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:01.358157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:07.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d3c0534f0 msgr2=0x7f6d3c0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d3c0534f0 0x7f6d3c0559b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f6d4000c000 tx=0x7f6d40008040).stop 2021-11-21T22:02:07.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 msgr2=0x7f6d58066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d58066d80 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f6d48002f90 tx=0x7f6d48006df0).stop 2021-11-21T22:02:07.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 shutdown_connections 2021-11-21T22:02:07.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d580a86b0 0x7f6d58159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d580a6aa0 0x7f6d581564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d3c0534f0 0x7f6d3c0559b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 --2- 172.21.2.116:0/3023214732 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d58062410 0x7f6d58066d80 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 >> 172.21.2.116:0/3023214732 conn(0x7f6d5805bd20 msgr2=0x7f6d580b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:07.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 shutdown_connections 2021-11-21T22:02:07.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.614+0000 7f6d5d603700 1 -- 172.21.2.116:0/3023214732 wait complete. 2021-11-21T22:02:07.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:02:00.244205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:02:01.358157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:02:07.624 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:02:07.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1cb291700 1 Processor -- start 2021-11-21T22:02:07.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1cb291700 1 -- start start 2021-11-21T22:02:07.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1cb291700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c40adc10 0x7fa1c40b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1cb291700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa1c4144a10 0x7fa1c4141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1cb291700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c40b3c50 con 0x7fa1c4141510 2021-11-21T22:02:07.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1cb291700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c40b3ef0 con 0x7fa1c4144a10 2021-11-21T22:02:07.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1cb291700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40b4190 con 0x7fa1c40adc10 2021-11-21T22:02:07.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c902d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c40adc10 0x7fa1c40b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c882c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa1c4144a10 0x7fa1c4141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c982e700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fa1c4141510 0x7fa1c413a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:34988/0 (socket says 172.21.2.116:34988) 2021-11-21T22:02:07.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c982e700 1 -- 172.21.2.116:0/3527843636 learned_addr learned my addr 172.21.2.116:0/3527843636 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:07.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c882c700 1 -- 172.21.2.116:0/3527843636 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c40adc10 msgr2=0x7fa1c40b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:07.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c882c700 1 --2- 172.21.2.116:0/3527843636 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c40adc10 0x7fa1c40b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c882c700 1 -- 172.21.2.116:0/3527843636 >> v1:172.21.2.117:6789/0 conn(0x7fa1c4141510 legacy=0x7fa1c413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c882c700 1 -- 172.21.2.116:0/3527843636 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4140b80 con 0x7fa1c4144a10 2021-11-21T22:02:07.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.674+0000 7fa1c882c700 1 --2- 172.21.2.116:0/3527843636 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4144a10 0x7fa1c4141020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fa1b4002f70 tx=0x7fa1b4006df0).ready entity=mon.1 client_cookie=918db5e5970d65af server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1b3fff700 1 -- 172.21.2.116:0/3527843636 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b400d570 con 0x7fa1c4144a10 2021-11-21T22:02:07.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1b3fff700 1 -- 172.21.2.116:0/3527843636 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b4000e90 con 0x7fa1c4144a10 2021-11-21T22:02:07.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1b3fff700 1 -- 172.21.2.116:0/3527843636 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b400d8c0 con 0x7fa1c4144a10 2021-11-21T22:02:07.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- 172.21.2.116:0/3527843636 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4144a10 msgr2=0x7fa1c4141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 --2- 172.21.2.116:0/3527843636 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4144a10 0x7fa1c4141020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fa1b4002f70 tx=0x7fa1b4006df0).stop 2021-11-21T22:02:07.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- 172.21.2.116:0/3527843636 shutdown_connections 2021-11-21T22:02:07.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 --2- 172.21.2.116:0/3527843636 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4144a10 0x7fa1c4141020 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 --2- 172.21.2.116:0/3527843636 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c40adc10 0x7fa1c40b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- 172.21.2.116:0/3527843636 >> 172.21.2.116:0/3527843636 conn(0x7fa1c405bd20 msgr2=0x7fa1c405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:07.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- 172.21.2.116:0/3527843636 shutdown_connections 2021-11-21T22:02:07.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- 172.21.2.116:0/3527843636 wait complete. 2021-11-21T22:02:07.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 Processor -- start 2021-11-21T22:02:07.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- start start 2021-11-21T22:02:07.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1c40adc10 0x7fa1c40ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4141510 0x7fa1c4156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c4144a10 0x7fa1c4156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c41571e0 con 0x7fa1c40adc10 2021-11-21T22:02:07.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c413cfe0 con 0x7fa1c4141510 2021-11-21T22:02:07.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c413d280 con 0x7fa1c4144a10 2021-11-21T22:02:07.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1c40adc10 0x7fa1c40ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1c40adc10 0x7fa1c40ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52358/0 (socket says 172.21.2.116:52358) 2021-11-21T22:02:07.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 -- 172.21.2.116:0/405090467 learned_addr learned my addr 172.21.2.116:0/405090467 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:07.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 -- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c4144a10 msgr2=0x7fa1c4156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c982e700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c4144a10 0x7fa1c4156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c882c700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4141510 0x7fa1c4156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c4144a10 0x7fa1c4156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 -- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4141510 msgr2=0x7fa1c4156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4141510 0x7fa1c4156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 -- 172.21.2.116:0/405090467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b400b180 con 0x7fa1c40adc10 2021-11-21T22:02:07.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1c902d700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1c40adc10 0x7fa1c40ae5c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fa1c000c000 tx=0x7fa1c0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1b27fc700 1 -- 172.21.2.116:0/405090467 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c0010780 con 0x7fa1c40adc10 2021-11-21T22:02:07.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1b27fc700 1 -- 172.21.2.116:0/405090467 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c00108e0 con 0x7fa1c40adc10 2021-11-21T22:02:07.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c413d520 con 0x7fa1c40adc10 2021-11-21T22:02:07.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c40aeab0 con 0x7fa1c40adc10 2021-11-21T22:02:07.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.678+0000 7fa1b27fc700 1 -- 172.21.2.116:0/405090467 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c0010c40 con 0x7fa1c40adc10 2021-11-21T22:02:07.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.682+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c40573d0 con 0x7fa1c40adc10 2021-11-21T22:02:07.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.682+0000 7fa1b27fc700 1 -- 172.21.2.116:0/405090467 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa1c001c460 con 0x7fa1c40adc10 2021-11-21T22:02:07.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.682+0000 7fa1b27fc700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1a40534f0 0x7fa1a40559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.682+0000 7fa1b27fc700 1 -- 172.21.2.116:0/405090467 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa1c006bfe0 con 0x7fa1c40adc10 2021-11-21T22:02:07.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.682+0000 7fa1c882c700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1a40534f0 0x7fa1a40559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.682+0000 7fa1c882c700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1a40534f0 0x7fa1a40559b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fa1b4015000 tx=0x7fa1b4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.682+0000 7fa1b27fc700 1 -- 172.21.2.116:0/405090467 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa1c0042fd0 con 0x7fa1c40adc10 2021-11-21T22:02:07.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa1c40b26a0 con 0x7fa1c40adc10 2021-11-21T22:02:07.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1b27fc700 1 -- 172.21.2.116:0/405090467 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v122) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa1c00465f0 con 0x7fa1c40adc10 2021-11-21T22:02:07.770 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:02:07.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1a40534f0 msgr2=0x7fa1a40559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1a40534f0 0x7fa1a40559b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fa1b4015000 tx=0x7fa1b4014000).stop 2021-11-21T22:02:07.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1c40adc10 msgr2=0x7fa1c40ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1c40adc10 0x7fa1c40ae5c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fa1c000c000 tx=0x7fa1c0006df0).stop 2021-11-21T22:02:07.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 shutdown_connections 2021-11-21T22:02:07.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa1c4144a10 0x7fa1c4156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa1c4141510 0x7fa1c4156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa1a40534f0 0x7fa1a40559b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 --2- 172.21.2.116:0/405090467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa1c40adc10 0x7fa1c40ae5c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 >> 172.21.2.116:0/405090467 conn(0x7fa1c405bd20 msgr2=0x7fa1c40b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:07.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 shutdown_connections 2021-11-21T22:02:07.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.766+0000 7fa1cb291700 1 -- 172.21.2.116:0/405090467 wait complete. 2021-11-21T22:02:07.777 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:02:07.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 Processor -- start 2021-11-21T22:02:07.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- start start 2021-11-21T22:02:07.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe0f40b6240 0x7fe0f405ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe0f40b5620 0x7fe0f412ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f4063000 con 0x7fe0f4062c20 2021-11-21T22:02:07.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f4142dd0 con 0x7fe0f40b5620 2021-11-21T22:02:07.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f4143070 con 0x7fe0f40b6240 2021-11-21T22:02:07.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9c77700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe0f40b5620 0x7fe0f412ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9476700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe0f4062c20 0x7fe0f40b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36090/0 (socket says 172.21.2.116:36090) 2021-11-21T22:02:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9476700 1 -- 172.21.2.116:0/3955653030 learned_addr learned my addr 172.21.2.116:0/3955653030 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9c77700 1 -- 172.21.2.116:0/3955653030 >> v2:172.21.2.117:3300/0 conn(0x7fe0f40b6240 msgr2=0x7fe0f405ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9c77700 1 --2- 172.21.2.116:0/3955653030 >> v2:172.21.2.117:3300/0 conn(0x7fe0f40b6240 0x7fe0f405ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9c77700 1 -- 172.21.2.116:0/3955653030 >> v1:172.21.2.116:6789/0 conn(0x7fe0f4062c20 legacy=0x7fe0f40b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9c77700 1 -- 172.21.2.116:0/3955653030 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4143310 con 0x7fe0f40b5620 2021-11-21T22:02:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0f9c77700 1 --2- 172.21.2.116:0/3955653030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b5620 0x7fe0f412ba00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fe0ec0032f0 tx=0x7fe0ec0091d0).ready entity=mon.0 client_cookie=ac6fd374918a7b3e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0e3fff700 1 -- 172.21.2.116:0/3955653030 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec009ea0 con 0x7fe0f40b5620 2021-11-21T22:02:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0e3fff700 1 -- 172.21.2.116:0/3955653030 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec00b870 con 0x7fe0f40b5620 2021-11-21T22:02:07.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0e3fff700 1 -- 172.21.2.116:0/3955653030 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec00b9e0 con 0x7fe0f40b5620 2021-11-21T22:02:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- 172.21.2.116:0/3955653030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b5620 msgr2=0x7fe0f412ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/3955653030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b5620 0x7fe0f412ba00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fe0ec0032f0 tx=0x7fe0ec0091d0).stop 2021-11-21T22:02:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- 172.21.2.116:0/3955653030 shutdown_connections 2021-11-21T22:02:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/3955653030 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b5620 0x7fe0f412ba00 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/3955653030 >> v2:172.21.2.117:3300/0 conn(0x7fe0f40b6240 0x7fe0f405ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- 172.21.2.116:0/3955653030 >> 172.21.2.116:0/3955653030 conn(0x7fe0f40ac9b0 msgr2=0x7fe0f4069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:07.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- 172.21.2.116:0/3955653030 shutdown_connections 2021-11-21T22:02:07.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- 172.21.2.116:0/3955653030 wait complete. 2021-11-21T22:02:07.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 Processor -- start 2021-11-21T22:02:07.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.826+0000 7fe0fb6da700 1 -- start start 2021-11-21T22:02:07.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0f4062c20 0x7fe0f40b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe0f40b5620 0x7fe0f4146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b6240 0x7fe0f4146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4145d10 con 0x7fe0f40b6240 2021-11-21T22:02:07.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4144c70 con 0x7fe0f40b5620 2021-11-21T22:02:07.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4144f10 con 0x7fe0f4062c20 2021-11-21T22:02:07.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b6240 0x7fe0f4146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b6240 0x7fe0f4146b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52376/0 (socket says 172.21.2.116:52376) 2021-11-21T22:02:07.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 -- 172.21.2.116:0/2079515644 learned_addr learned my addr 172.21.2.116:0/2079515644 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:07.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 -- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0f4062c20 msgr2=0x7fe0f40b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0f4062c20 0x7fe0f40b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 -- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe0f40b5620 msgr2=0x7fe0f4146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe0f40b5620 0x7fe0f4146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 -- 172.21.2.116:0/2079515644 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec009470 con 0x7fe0f40b6240 2021-11-21T22:02:07.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9c77700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b6240 0x7fe0f4146b10 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fe0ec0032f0 tx=0x7fe0ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0e27fc700 1 -- 172.21.2.116:0/2079515644 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec009ea0 con 0x7fe0f40b6240 2021-11-21T22:02:07.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0e27fc700 1 -- 172.21.2.116:0/2079515644 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec0107a0 con 0x7fe0f40b6240 2021-11-21T22:02:07.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f41451b0 con 0x7fe0f40b6240 2021-11-21T22:02:07.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0e27fc700 1 -- 172.21.2.116:0/2079515644 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec010b00 con 0x7fe0f40b6240 2021-11-21T22:02:07.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f4147000 con 0x7fe0f40b6240 2021-11-21T22:02:07.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f41435b0 con 0x7fe0f40b6240 2021-11-21T22:02:07.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0e27fc700 1 -- 172.21.2.116:0/2079515644 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe0ec00f070 con 0x7fe0f40b6240 2021-11-21T22:02:07.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0e27fc700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0d4053610 0x7fe0d4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0e27fc700 1 -- 172.21.2.116:0/2079515644 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe0ec06ce90 con 0x7fe0f40b6240 2021-11-21T22:02:07.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9476700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0d4053610 0x7fe0d4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.830+0000 7fe0f9476700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0d4053610 0x7fe0d4055ad0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fe0f4143930 tx=0x7fe0e40077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.834+0000 7fe0e27fc700 1 -- 172.21.2.116:0/2079515644 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe0ec043f20 con 0x7fe0f40b6240 2021-11-21T22:02:07.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe0f40573d0 con 0x7fe0f40b6240 2021-11-21T22:02:07.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0e27fc700 1 -- 172.21.2.116:0/2079515644 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v122) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe0ec015020 con 0x7fe0f40b6240 2021-11-21T22:02:07.920 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:07.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0d4053610 msgr2=0x7fe0d4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0d4053610 0x7fe0d4055ad0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fe0f4143930 tx=0x7fe0e40077f0).stop 2021-11-21T22:02:07.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b6240 msgr2=0x7fe0f4146b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b6240 0x7fe0f4146b10 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fe0ec0032f0 tx=0x7fe0ec016000).stop 2021-11-21T22:02:07.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 shutdown_connections 2021-11-21T22:02:07.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0d4053610 0x7fe0d4055ad0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0f40b6240 0x7fe0f4146b10 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe0f40b5620 0x7fe0f4146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 --2- 172.21.2.116:0/2079515644 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0f4062c20 0x7fe0f40b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 >> 172.21.2.116:0/2079515644 conn(0x7fe0f40ac9b0 msgr2=0x7fe0f412adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:07.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.914+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 shutdown_connections 2021-11-21T22:02:07.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.918+0000 7fe0fb6da700 1 -- 172.21.2.116:0/2079515644 wait complete. 2021-11-21T22:02:07.927 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:02:07.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 Processor -- start 2021-11-21T22:02:07.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 -- start start 2021-11-21T22:02:07.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f25c8144810 0x7f25c81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 0x7f25c80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f25c80b1db0 0x7f25c80ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c8147e80 con 0x7f25c8144810 2021-11-21T22:02:07.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c80af5a0 con 0x7f25c80b1db0 2021-11-21T22:02:07.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25cf01e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80b1490 con 0x7f25c80abce0 2021-11-21T22:02:07.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f25c8144810 0x7f25c81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f25c8144810 0x7f25c81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52380/0 (socket says 172.21.2.116:52380) 2021-11-21T22:02:07.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 -- 172.21.2.116:0/167565827 learned_addr learned my addr 172.21.2.116:0/167565827 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:07.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 -- 172.21.2.116:0/167565827 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 msgr2=0x7f25c80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:07.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 --2- 172.21.2.116:0/167565827 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 0x7f25c80b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 -- 172.21.2.116:0/167565827 >> v2:172.21.2.117:3300/0 conn(0x7f25c80b1db0 msgr2=0x7f25c80ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:07.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 --2- 172.21.2.116:0/167565827 >> v2:172.21.2.117:3300/0 conn(0x7f25c80b1db0 0x7f25c80ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 -- 172.21.2.116:0/167565827 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80af060 con 0x7f25c8144810 2021-11-21T22:02:07.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25ccdba700 1 --2- 172.21.2.116:0/167565827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8144810 0x7f25c81478c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f25bc002f70 tx=0x7f25bc006eb0).ready entity=mon.0 client_cookie=583a5aae5d46e30a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25c77fe700 1 -- 172.21.2.116:0/167565827 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25bc00d570 con 0x7f25c8144810 2021-11-21T22:02:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25c77fe700 1 -- 172.21.2.116:0/167565827 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc000e90 con 0x7f25c8144810 2021-11-21T22:02:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.974+0000 7f25c77fe700 1 -- 172.21.2.116:0/167565827 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25bc00d900 con 0x7f25c8144810 2021-11-21T22:02:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/167565827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8144810 msgr2=0x7f25c81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 --2- 172.21.2.116:0/167565827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8144810 0x7f25c81478c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f25bc002f70 tx=0x7f25bc006eb0).stop 2021-11-21T22:02:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/167565827 shutdown_connections 2021-11-21T22:02:07.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 --2- 172.21.2.116:0/167565827 >> v2:172.21.2.117:3300/0 conn(0x7f25c80b1db0 0x7f25c80ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 --2- 172.21.2.116:0/167565827 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 0x7f25c80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 --2- 172.21.2.116:0/167565827 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8144810 0x7f25c81478c0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:07.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/167565827 >> 172.21.2.116:0/167565827 conn(0x7f25c805bd20 msgr2=0x7f25c805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:07.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/167565827 shutdown_connections 2021-11-21T22:02:07.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/167565827 wait complete. 2021-11-21T22:02:07.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 Processor -- start 2021-11-21T22:02:07.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- start start 2021-11-21T22:02:07.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 0x7f25c8143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25c8144810 0x7f25c8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8064320 0x7f25c8156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8063240 con 0x7f25c8064320 2021-11-21T22:02:07.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c815a590 con 0x7f25c8144810 2021-11-21T22:02:07.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c815a830 con 0x7f25c80abce0 2021-11-21T22:02:07.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8064320 0x7f25c8156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8064320 0x7f25c8156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52384/0 (socket says 172.21.2.116:52384) 2021-11-21T22:02:07.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 -- 172.21.2.116:0/2019859436 learned_addr learned my addr 172.21.2.116:0/2019859436 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:07.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 -- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 msgr2=0x7f25c8143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 0x7f25c8143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 -- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25c8144810 msgr2=0x7f25c8156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:07.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25c8144810 0x7f25c8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:07.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 -- 172.21.2.116:0/2019859436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc00b1e0 con 0x7f25c8064320 2021-11-21T22:02:07.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cd5bb700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8064320 0x7f25c8156b20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f25c0003330 tx=0x7f25c0009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25c5ffb700 1 -- 172.21.2.116:0/2019859436 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25c0001520 con 0x7f25c8064320 2021-11-21T22:02:07.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25c5ffb700 1 -- 172.21.2.116:0/2019859436 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c0001680 con 0x7f25c8064320 2021-11-21T22:02:07.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c8063be0 con 0x7f25c8064320 2021-11-21T22:02:07.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25c5ffb700 1 -- 172.21.2.116:0/2019859436 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25c00019e0 con 0x7f25c8064320 2021-11-21T22:02:07.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c80b3c00 con 0x7f25c8064320 2021-11-21T22:02:07.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c80af300 con 0x7f25c8064320 2021-11-21T22:02:07.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25c5ffb700 1 -- 172.21.2.116:0/2019859436 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f25c0018030 con 0x7f25c8064320 2021-11-21T22:02:07.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25c5ffb700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25a80536a0 0x7f25a8055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:07.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.978+0000 7f25c5ffb700 1 -- 172.21.2.116:0/2019859436 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f25c006b2f0 con 0x7f25c8064320 2021-11-21T22:02:07.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.982+0000 7f25ccdba700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25a80536a0 0x7f25a8055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:07.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.982+0000 7f25ccdba700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25a80536a0 0x7f25a8055b60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f25bc002f70 tx=0x7f25bc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:07.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:07.982+0000 7f25c5ffb700 1 -- 172.21.2.116:0/2019859436 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f25c0042500 con 0x7f25c8064320 2021-11-21T22:02:08.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25c80573d0 con 0x7f25c8064320 2021-11-21T22:02:08.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25c5ffb700 1 -- 172.21.2.116:0/2019859436 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+10 (secure 0 0 0) 0x7f25c0016020 con 0x7f25c8064320 2021-11-21T22:02:08.070 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:08.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25a80536a0 msgr2=0x7f25a8055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25a80536a0 0x7f25a8055b60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f25bc002f70 tx=0x7f25bc015000).stop 2021-11-21T22:02:08.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8064320 msgr2=0x7f25c8156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8064320 0x7f25c8156b20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f25c0003330 tx=0x7f25c0009120).stop 2021-11-21T22:02:08.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 shutdown_connections 2021-11-21T22:02:08.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f25a80536a0 0x7f25a8055b60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25c8064320 0x7f25c8156b20 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25c8144810 0x7f25c8156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 --2- 172.21.2.116:0/2019859436 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25c80abce0 0x7f25c8143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 >> 172.21.2.116:0/2019859436 conn(0x7f25c805bd20 msgr2=0x7f25c80af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:08.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 shutdown_connections 2021-11-21T22:02:08.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.066+0000 7f25cf01e700 1 -- 172.21.2.116:0/2019859436 wait complete. 2021-11-21T22:02:08.076 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:02:08.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 Processor -- start 2021-11-21T22:02:08.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- start start 2021-11-21T22:02:08.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6d90144a10 0x7f6d900b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6d900b3690 0x7f6d9013a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d900b3c50 con 0x7f6d900adc10 2021-11-21T22:02:08.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d900b3ef0 con 0x7f6d90144a10 2021-11-21T22:02:08.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d900b4190 con 0x7f6d900b3690 2021-11-21T22:02:08.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8ffff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f6d900adc10 0x7f6d90141020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36116/0 (socket says 172.21.2.116:36116) 2021-11-21T22:02:08.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8ffff700 1 -- 172.21.2.116:0/3110486568 learned_addr learned my addr 172.21.2.116:0/3110486568 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:08.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8f7fe700 1 --2- 172.21.2.116:0/3110486568 >> v2:172.21.2.116:3300/0 conn(0x7f6d90144a10 0x7f6d900b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:08.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8effd700 1 -- 172.21.2.116:0/3110486568 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189948209 0 0) 0x7f6d900b3c50 con 0x7f6d900adc10 2021-11-21T22:02:08.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8effd700 1 -- 172.21.2.116:0/3110486568 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d70003260 con 0x7f6d900adc10 2021-11-21T22:02:08.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8f7fe700 1 -- 172.21.2.116:0/3110486568 >> v2:172.21.2.117:3300/0 conn(0x7f6d900b3690 msgr2=0x7f6d9013a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:08.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8f7fe700 1 --2- 172.21.2.116:0/3110486568 >> v2:172.21.2.117:3300/0 conn(0x7f6d900b3690 0x7f6d9013a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8f7fe700 1 -- 172.21.2.116:0/3110486568 >> v1:172.21.2.116:6789/0 conn(0x7f6d900adc10 legacy=0x7f6d90141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8f7fe700 1 -- 172.21.2.116:0/3110486568 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d90140b80 con 0x7f6d90144a10 2021-11-21T22:02:08.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8f7fe700 1 --2- 172.21.2.116:0/3110486568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d90144a10 0x7f6d900b2030 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d78006df0).ready entity=mon.0 client_cookie=b644f1c177a6ae96 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:08.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8effd700 1 -- 172.21.2.116:0/3110486568 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7800d570 con 0x7f6d90144a10 2021-11-21T22:02:08.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8effd700 1 -- 172.21.2.116:0/3110486568 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78000e90 con 0x7f6d90144a10 2021-11-21T22:02:08.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d8effd700 1 -- 172.21.2.116:0/3110486568 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7800d8c0 con 0x7f6d90144a10 2021-11-21T22:02:08.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- 172.21.2.116:0/3110486568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d90144a10 msgr2=0x7f6d900b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 --2- 172.21.2.116:0/3110486568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d90144a10 0x7f6d900b2030 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d78006df0).stop 2021-11-21T22:02:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- 172.21.2.116:0/3110486568 shutdown_connections 2021-11-21T22:02:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 --2- 172.21.2.116:0/3110486568 >> v2:172.21.2.117:3300/0 conn(0x7f6d900b3690 0x7f6d9013a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 --2- 172.21.2.116:0/3110486568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d90144a10 0x7f6d900b2030 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- 172.21.2.116:0/3110486568 >> 172.21.2.116:0/3110486568 conn(0x7f6d9005bd20 msgr2=0x7f6d9005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:08.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- 172.21.2.116:0/3110486568 shutdown_connections 2021-11-21T22:02:08.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- 172.21.2.116:0/3110486568 wait complete. 2021-11-21T22:02:08.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 Processor -- start 2021-11-21T22:02:08.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.126+0000 7f6d96302700 1 -- start start 2021-11-21T22:02:08.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d900adc10 0x7f6d900b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d900b3690 0x7f6d9013c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d90144a10 0x7f6d9013cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9013fb50 con 0x7f6d900b3690 2021-11-21T22:02:08.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90113930 con 0x7f6d900adc10 2021-11-21T22:02:08.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900b1600 con 0x7f6d90144a10 2021-11-21T22:02:08.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d900b3690 0x7f6d9013c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:08.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d900b3690 0x7f6d9013c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52400/0 (socket says 172.21.2.116:52400) 2021-11-21T22:02:08.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 -- 172.21.2.116:0/231339567 learned_addr learned my addr 172.21.2.116:0/231339567 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:08.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 -- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d90144a10 msgr2=0x7f6d9013cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d90144a10 0x7f6d9013cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 -- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d900adc10 msgr2=0x7f6d900b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d900adc10 0x7f6d900b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 -- 172.21.2.116:0/231339567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7800b180 con 0x7f6d900b3690 2021-11-21T22:02:08.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8f7fe700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d900b3690 0x7f6d9013c8e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d78014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:08.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8d7fa700 1 -- 172.21.2.116:0/231339567 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d78016070 con 0x7f6d900b3690 2021-11-21T22:02:08.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8d7fa700 1 -- 172.21.2.116:0/231339567 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78022070 con 0x7f6d900b3690 2021-11-21T22:02:08.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9013fd80 con 0x7f6d900b3690 2021-11-21T22:02:08.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d900b18a0 con 0x7f6d900b3690 2021-11-21T22:02:08.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8d7fa700 1 -- 172.21.2.116:0/231339567 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7801eb20 con 0x7f6d900b3690 2021-11-21T22:02:08.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d900573d0 con 0x7f6d900b3690 2021-11-21T22:02:08.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8d7fa700 1 -- 172.21.2.116:0/231339567 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6d7802e070 con 0x7f6d900b3690 2021-11-21T22:02:08.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8d7fa700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d74053660 0x7f6d74055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8d7fa700 1 -- 172.21.2.116:0/231339567 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d78070ad0 con 0x7f6d900b3690 2021-11-21T22:02:08.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8ffff700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d74053660 0x7f6d74055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:08.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.130+0000 7f6d8ffff700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d74053660 0x7f6d74055b20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f6d80001110 tx=0x7f6d800076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:08.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.134+0000 7f6d8d7fa700 1 -- 172.21.2.116:0/231339567 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d78047b20 con 0x7f6d900b3690 2021-11-21T22:02:08.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d9005c620 con 0x7f6d900b3690 2021-11-21T22:02:08.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d8d7fa700 1 -- 172.21.2.116:0/231339567 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6d7804b140 con 0x7f6d900b3690 2021-11-21T22:02:08.223 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:08.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d74053660 msgr2=0x7f6d74055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d74053660 0x7f6d74055b20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f6d80001110 tx=0x7f6d800076c0).stop 2021-11-21T22:02:08.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d900b3690 msgr2=0x7f6d9013c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d900b3690 0x7f6d9013c8e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d78014000).stop 2021-11-21T22:02:08.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 shutdown_connections 2021-11-21T22:02:08.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d90144a10 0x7f6d9013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d74053660 0x7f6d74055b20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d900b3690 0x7f6d9013c8e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 --2- 172.21.2.116:0/231339567 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d900adc10 0x7f6d900b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 >> 172.21.2.116:0/231339567 conn(0x7f6d9005bd20 msgr2=0x7f6d90143c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:08.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 shutdown_connections 2021-11-21T22:02:08.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.218+0000 7f6d96302700 1 -- 172.21.2.116:0/231339567 wait complete. 2021-11-21T22:02:08.230 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:02:08.231 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:08.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 Processor -- start 2021-11-21T22:02:08.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- start start 2021-11-21T22:02:08.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9e8c0a6aa0 0x7f9e8c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a86b0 0x7f9e8c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9e8c1475d0 0x7f9e8c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8c0a8bc0 con 0x7f9e8c0a6aa0 2021-11-21T22:02:08.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8c0a6e80 con 0x7f9e8c1475d0 2021-11-21T22:02:08.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c14a7b0 con 0x7f9e8c0a86b0 2021-11-21T22:02:08.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9e8c0a6aa0 0x7f9e8c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:08.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9e8c0a6aa0 0x7f9e8c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52406/0 (socket says 172.21.2.116:52406) 2021-11-21T22:02:08.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 -- 172.21.2.116:0/2980007756 learned_addr learned my addr 172.21.2.116:0/2980007756 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:08.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 -- 172.21.2.116:0/2980007756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a86b0 msgr2=0x7f9e8c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:08.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 --2- 172.21.2.116:0/2980007756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a86b0 0x7f9e8c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 -- 172.21.2.116:0/2980007756 >> v2:172.21.2.117:3300/0 conn(0x7f9e8c1475d0 msgr2=0x7f9e8c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:08.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 --2- 172.21.2.116:0/2980007756 >> v2:172.21.2.117:3300/0 conn(0x7f9e8c1475d0 0x7f9e8c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 -- 172.21.2.116:0/2980007756 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c14aa10 con 0x7f9e8c0a6aa0 2021-11-21T22:02:08.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e90df9700 1 --2- 172.21.2.116:0/2980007756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c0a6aa0 0x7f9e8c14b150 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f9e80002f70 tx=0x7f9e80006eb0).ready entity=mon.0 client_cookie=ab15c0dd95fdd535 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:08.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e8b7fe700 1 -- 172.21.2.116:0/2980007756 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e8000d570 con 0x7f9e8c0a6aa0 2021-11-21T22:02:08.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e8b7fe700 1 -- 172.21.2.116:0/2980007756 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e80000e90 con 0x7f9e8c0a6aa0 2021-11-21T22:02:08.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e8b7fe700 1 -- 172.21.2.116:0/2980007756 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e8000d900 con 0x7f9e8c0a6aa0 2021-11-21T22:02:08.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- 172.21.2.116:0/2980007756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c0a6aa0 msgr2=0x7f9e8c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 --2- 172.21.2.116:0/2980007756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c0a6aa0 0x7f9e8c14b150 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f9e80002f70 tx=0x7f9e80006eb0).stop 2021-11-21T22:02:08.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- 172.21.2.116:0/2980007756 shutdown_connections 2021-11-21T22:02:08.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 --2- 172.21.2.116:0/2980007756 >> v2:172.21.2.117:3300/0 conn(0x7f9e8c1475d0 0x7f9e8c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 --2- 172.21.2.116:0/2980007756 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a86b0 0x7f9e8c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 --2- 172.21.2.116:0/2980007756 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c0a6aa0 0x7f9e8c14b150 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- 172.21.2.116:0/2980007756 >> 172.21.2.116:0/2980007756 conn(0x7f9e8c05bd20 msgr2=0x7f9e8c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:08.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- 172.21.2.116:0/2980007756 shutdown_connections 2021-11-21T22:02:08.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.278+0000 7f9e9305d700 1 -- 172.21.2.116:0/2980007756 wait complete. 2021-11-21T22:02:08.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 Processor -- start 2021-11-21T22:02:08.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 -- start start 2021-11-21T22:02:08.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a6aa0 0x7f9e8c14ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e8c0a86b0 0x7f9e8c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c1475d0 0x7f9e8c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0b3b30 con 0x7f9e8c1475d0 2021-11-21T22:02:08.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c065640 con 0x7f9e8c0a86b0 2021-11-21T22:02:08.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0b3890 con 0x7f9e8c0a6aa0 2021-11-21T22:02:08.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c1475d0 0x7f9e8c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:08.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c1475d0 0x7f9e8c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52414/0 (socket says 172.21.2.116:52414) 2021-11-21T22:02:08.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 -- 172.21.2.116:0/225200851 learned_addr learned my addr 172.21.2.116:0/225200851 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:08.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 -- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a6aa0 msgr2=0x7f9e8c14ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a6aa0 0x7f9e8c14ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 -- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e8c0a86b0 msgr2=0x7f9e8c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e8c0a86b0 0x7f9e8c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 -- 172.21.2.116:0/225200851 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8000b1e0 con 0x7f9e8c1475d0 2021-11-21T22:02:08.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e915fa700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c1475d0 0x7f9e8c159ce0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f9e840032f0 tx=0x7f9e84009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:08.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e89ffb700 1 -- 172.21.2.116:0/225200851 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e8400f510 con 0x7f9e8c1475d0 2021-11-21T22:02:08.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e89ffb700 1 -- 172.21.2.116:0/225200851 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8400f670 con 0x7f9e8c1475d0 2021-11-21T22:02:08.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c0658a0 con 0x7f9e8c1475d0 2021-11-21T22:02:08.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e89ffb700 1 -- 172.21.2.116:0/225200851 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e8400f7e0 con 0x7f9e8c1475d0 2021-11-21T22:02:08.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c153dd0 con 0x7f9e8c1475d0 2021-11-21T22:02:08.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c0b3540 con 0x7f9e8c1475d0 2021-11-21T22:02:08.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e89ffb700 1 -- 172.21.2.116:0/225200851 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9e8400f070 con 0x7f9e8c1475d0 2021-11-21T22:02:08.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e89ffb700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e6c0536f0 0x7f9e6c055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:08.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e89ffb700 1 -- 172.21.2.116:0/225200851 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9e8406b5d0 con 0x7f9e8c1475d0 2021-11-21T22:02:08.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.282+0000 7f9e90df9700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e6c0536f0 0x7f9e6c055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:08.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.286+0000 7f9e90df9700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e6c0536f0 0x7f9e6c055bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9e80002f70 tx=0x7f9e80015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:08.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.286+0000 7f9e89ffb700 1 -- 172.21.2.116:0/225200851 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e84042620 con 0x7f9e8c1475d0 2021-11-21T22:02:08.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.394+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e8c065a30 con 0x7f9e8c1475d0 2021-11-21T22:02:08.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e89ffb700 1 -- 172.21.2.116:0/225200851 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f9e84045c40 con 0x7f9e8c1475d0 2021-11-21T22:02:08.401 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:08.401 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":55,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/3682554328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":3682554328},{"type":"v1","addr":"172.21.2.117:6835","nonce":3682554328}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6833/2101278175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2101278175},{"type":"v1","addr":"172.21.2.116:6833","nonce":2101278175}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:02.389133+0000","modified":"2021-11-21T22:02:03.393012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/1577318848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1577318848},{"type":"v1","addr":"172.21.2.116:6835","nonce":1577318848}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-11-21T22:02:08.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e6c0536f0 msgr2=0x7f9e6c055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e6c0536f0 0x7f9e6c055bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9e80002f70 tx=0x7f9e80015000).stop 2021-11-21T22:02:08.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c1475d0 msgr2=0x7f9e8c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:08.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c1475d0 0x7f9e8c159ce0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f9e840032f0 tx=0x7f9e84009120).stop 2021-11-21T22:02:08.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 shutdown_connections 2021-11-21T22:02:08.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9e6c0536f0 0x7f9e6c055bb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9e8c1475d0 0x7f9e8c159ce0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:08.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9e8c0a86b0 0x7f9e8c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 --2- 172.21.2.116:0/225200851 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9e8c0a6aa0 0x7f9e8c14ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:08.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 >> 172.21.2.116:0/225200851 conn(0x7f9e8c05bd20 msgr2=0x7f9e8c1490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:08.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 shutdown_connections 2021-11-21T22:02:08.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:08.398+0000 7f9e9305d700 1 -- 172.21.2.116:0/225200851 wait complete. 2021-11-21T22:02:08.405 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 55 2021-11-21T22:02:08.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:02.389133+0000', 'modified': '2021-11-21T22:02:03.393012+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7674}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7674': {'gid': 7674, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/1577318848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 1577318848}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1577318848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:02:08.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:02:08.713 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-11-21T22:02:08.781 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":7674,"want_state":"up:active","state":"up:active","rank_uptime":6.3839196969999996,"mdsmap_epoch":55,"osdmap_epoch":122,"osdmap_epoch_barrier":121,"uptime":10.550525898} 2021-11-21T22:02:08.781 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 7674, "mdsmap_epoch": 55, "osdmap_epoch": 122, "osdmap_epoch_barrier": 121, "rank_uptime": 6.383919697, "state": "up:active", "uptime": 10.550525898, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:02:08.782 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:02:08.782 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:02:08.783 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:02:08.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:08.784 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:08.795 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:02:08.847 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:08.848 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:08.854 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:08.855 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:08.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:08.856 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:02:08.904 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:02:08.904 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:08.911 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:02:08.921 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:08.922 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:08.927 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:08.927 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:08.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:08.929 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:02:08.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14516]: starting ceph client 2021-11-21T22:02:08.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:02:08.949+0000 7f2583e01080 -1 init, newargv = 0x55e76ea3ac30 newargc=9 2021-11-21T22:02:08.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14516]: starting fuse 2021-11-21T22:02:10.055 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:02:10.064 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:10.065 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:10.115 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:10.116 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:10.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.118 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:02:10.124 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:02:10.124 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:02:10.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.125 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:02:10.126 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:02:10.126 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:02:10.127 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:02:10.127 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:02:10.127 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:10.128 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:02:10.128 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:02:10.129 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:02:10.129 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:10.129 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:02:10.130 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:02:10.130 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:02:10.131 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:02:10.131 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:10.132 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:02:10.132 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:02:10.133 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:02:10.133 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:02:10.133 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:02:10.134 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:10.134 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:02:10.135 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:02:10.159 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14516.asok 2021-11-21T22:02:10.162 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14516.asok 2021-11-21T22:02:10.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.163 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14516.asok status 2021-11-21T22:02:10.253 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:02:10.253 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:02:10.254 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:02:10.254 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:02:10.254 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:02:10.254 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:02:10.255 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:02:10.255 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14516", 2021-11-21T22:02:10.255 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:02:10.255 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:10.255 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:02:10.256 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:02:10.256 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 5185, 2021-11-21T22:02:10.256 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:02:10.256 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:02:10.257 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:02:10.257 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 5185 2021-11-21T22:02:10.258 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:10.258 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:02:10.258 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:02:10.258 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:02:10.259 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1173774302 2021-11-21T22:02:10.259 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:02:10.259 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:10.259 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:02:10.260 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:02:10.260 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:02:10.260 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1173774302 2021-11-21T22:02:10.260 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:10.261 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.5185 172.21.2.136:0/1173774302", 2021-11-21T22:02:10.261 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/1173774302", 2021-11-21T22:02:10.261 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:02:10.261 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 55, 2021-11-21T22:02:10.262 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 122, 2021-11-21T22:02:10.262 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:02:10.262 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:02:10.262 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:02:10.262 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:02:10.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.264 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:10.265 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:10.271 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:10.271 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:10.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.272 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:10.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.281 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:10.282 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:10.293 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:02:10.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:10.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.296 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:10.296 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:10.307 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:02:10.308 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:10.309 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:02:10.309 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:02:10.310 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:10.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15048]: fuse finished with error 0 and tester_r 0 2021-11-21T22:02:10.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.326 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:10.326 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:10.389 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:02:10.390 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:02:10.391 INFO:teuthology.orchestra.run:waiting for 900 2021-11-21T22:02:10.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:02:10.392 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:10.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.400 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:10.400 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:10.412 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:10.413 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:10.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.414 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:10.415 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:10.426 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:10.427 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:10.428 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:02:10.428 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:02:10.428 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:10.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14516]: fuse finished with error 0 and tester_r 0 2021-11-21T22:02:10.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:10.451 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:10.452 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:10.504 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:02:10.505 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:02:10.505 INFO:teuthology.orchestra.run:waiting for 900 2021-11-21T22:02:10.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:02:10.561 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:10.571 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:02:10.571 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:02:10.572 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:02:10.572 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:02:10.572 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:02:10.573 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:02:10.584 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:02:10.585 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:10.585 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:02:10.586 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:10.593 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:02:10.594 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:10.594 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2021-11-21T22:02:10.596 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2021-11-21T22:02:10.596 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2021-11-21T22:02:10.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 Processor -- start 2021-11-21T22:02:10.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- start start 2021-11-21T22:02:10.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc813b90700 1 Processor -- start 2021-11-21T22:02:10.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc813b90700 1 -- start start 2021-11-21T22:02:10.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 Processor -- start 2021-11-21T22:02:10.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- start start 2021-11-21T22:02:10.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ed81466b0 0x7f0ed80b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc813b90700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc80c0b3d00 0x7fc80c0b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc813b90700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc80c09bbe0 0x7fc80c09c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed8146a90 con 0x7f0ed805cfe0 2021-11-21T22:02:10.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80b66d0 con 0x7f0ed80b6250 2021-11-21T22:02:10.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc813b90700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80c0b5300 con 0x7fc80c0b9a70 2021-11-21T22:02:10.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc813b90700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80c09b340 con 0x7fc80c0b3d00 2021-11-21T22:02:10.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed805bd20 con 0x7f0ed81466b0 2021-11-21T22:02:10.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc813b90700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0b43d0 con 0x7fc80c09bbe0 2021-11-21T22:02:10.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc738098e80 0x7fc73809b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 0x7fc7380b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7380acad0 con 0x7fc7381463d0 2021-11-21T22:02:10.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc738143d10 con 0x7fc738098e80 2021-11-21T22:02:10.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73809ba00 con 0x7fc7380b4ce0 2021-11-21T22:02:10.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed6d9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0ed805cfe0 0x7f0ed812ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36152/0 (socket says 172.21.2.116:36152) 2021-11-21T22:02:10.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed659c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0ed81466b0 0x7f0ed80b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed6d9d700 1 -- 172.21.2.116:0/1553748507 learned_addr learned my addr 172.21.2.116:0/1553748507 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:10.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc737fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc7381463d0 0x7fc7380b45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36156/0 (socket says 172.21.2.116:36156) 2021-11-21T22:02:10.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc737fff700 1 -- 172.21.2.116:0/1230095506 learned_addr learned my addr 172.21.2.116:0/1230095506 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:10.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc80c0b9a70 0x7fc80c09ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36144/0 (socket says 172.21.2.116:36144) 2021-11-21T22:02:10.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 -- 172.21.2.116:0/1275391721 learned_addr learned my addr 172.21.2.116:0/1275391721 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:10.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc7377fe700 1 --2- 172.21.2.116:0/1230095506 >> v2:172.21.2.116:3300/0 conn(0x7fc738098e80 0x7fc73809b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed659c700 1 -- 172.21.2.116:0/1553748507 >> v1:172.21.2.117:6789/0 conn(0x7f0ed80b6250 legacy=0x7f0ed805c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:10.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed659c700 1 -- 172.21.2.116:0/1553748507 >> v1:172.21.2.116:6789/0 conn(0x7f0ed805cfe0 legacy=0x7f0ed812ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed659c700 1 -- 172.21.2.116:0/1553748507 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed805bfc0 con 0x7f0ed81466b0 2021-11-21T22:02:10.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323796633 0 0) 0x7fc80c0b5300 con 0x7fc80c0b9a70 2021-11-21T22:02:10.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f0003260 con 0x7fc80c0b9a70 2021-11-21T22:02:10.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc736ffd700 1 -- 172.21.2.116:0/1230095506 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103454461 0 0) 0x7fc7380acad0 con 0x7fc7381463d0 2021-11-21T22:02:10.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc7377fe700 1 -- 172.21.2.116:0/1230095506 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 msgr2=0x7fc7380b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:10.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc7377fe700 1 --2- 172.21.2.116:0/1230095506 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 0x7fc7380b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc7377fe700 1 -- 172.21.2.116:0/1230095506 >> v1:172.21.2.116:6789/0 conn(0x7fc7381463d0 legacy=0x7fc7380b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc7377fe700 1 -- 172.21.2.116:0/1230095506 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7380acfa0 con 0x7fc738098e80 2021-11-21T22:02:10.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed659c700 1 --2- 172.21.2.116:0/1553748507 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed81466b0 0x7f0ed80b4370 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f0ed80b6b10 tx=0x7f0ec8006df0).ready entity=mon.0 client_cookie=55ba3d1b151bd956 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc7377fe700 1 --2- 172.21.2.116:0/1230095506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc738098e80 0x7fc73809b510 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7fc7300032f0 tx=0x7fc7300091d0).ready entity=mon.0 client_cookie=36a566fdef847ac6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81112b700 1 --2- 172.21.2.116:0/1275391721 >> v2:172.21.2.117:3300/0 conn(0x7fc80c0b3d00 0x7fc80c0b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3326110646 0 0) 0x7fc7f0003260 con 0x7fc80c0b9a70 2021-11-21T22:02:10.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc80c09bbe0 msgr2=0x7fc80c09c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:10.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc80c09bbe0 0x7fc80c09c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 >> v2:172.21.2.117:3300/0 conn(0x7fc80c0b3d00 msgr2=0x7fc80c0b9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:10.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 --2- 172.21.2.116:0/1275391721 >> v2:172.21.2.117:3300/0 conn(0x7fc80c0b3d00 0x7fc80c0b9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:10.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c0b5560 con 0x7fc80c0b9a70 2021-11-21T22:02:10.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fc800001680 con 0x7fc80c0b9a70 2021-11-21T22:02:10.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed559a700 1 -- 172.21.2.116:0/1553748507 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800d5c0 con 0x7f0ed81466b0 2021-11-21T22:02:10.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed559a700 1 -- 172.21.2.116:0/1553748507 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8000e90 con 0x7f0ed81466b0 2021-11-21T22:02:10.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 >> v1:172.21.2.116:6789/0 conn(0x7fc80c0b9a70 legacy=0x7fc80c09ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0ed559a700 1 -- 172.21.2.116:0/1553748507 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800d910 con 0x7f0ed81466b0 2021-11-21T22:02:10.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0003750 0x7fc7f0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc736ffd700 1 -- 172.21.2.116:0/1230095506 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc730009ea0 con 0x7fc738098e80 2021-11-21T22:02:10.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7f0004500 0x7fc7f0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc736ffd700 1 -- 172.21.2.116:0/1230095506 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000b8a0 con 0x7fc738098e80 2021-11-21T22:02:10.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc736ffd700 1 -- 172.21.2.116:0/1230095506 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc73000ba10 con 0x7fc738098e80 2021-11-21T22:02:10.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc80c09bbe0 0x7fc7f0004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c09b340 con 0x7fc7f0003750 2021-11-21T22:02:10.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0b43d0 con 0x7fc7f0004500 2021-11-21T22:02:10.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f000b9c0 con 0x7fc80c09bbe0 2021-11-21T22:02:10.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0003750 0x7fc7f0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 -- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc80c09bbe0 msgr2=0x7fc7f0004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:10.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc80c09bbe0 0x7fc7f0004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 -- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7f0004500 msgr2=0x7fc7f0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:10.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc7f0004500 0x7fc7f0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 -- 172.21.2.116:0/1275391721 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f0003260 con 0x7fc7f0003750 2021-11-21T22:02:10.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81192c700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0003750 0x7fc7f0003b70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7fc800001390 tx=0x7fc80c0b5ab0).ready entity=mon.0 client_cookie=7ade7735db50dde3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- 172.21.2.116:0/1553748507 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed81466b0 msgr2=0x7f0ed80b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/1553748507 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed81466b0 0x7f0ed80b4370 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f0ed80b6b10 tx=0x7f0ec8006df0).stop 2021-11-21T22:02:10.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- 172.21.2.116:0/1230095506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc738098e80 msgr2=0x7fc73809b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 --2- 172.21.2.116:0/1230095506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc738098e80 0x7fc73809b510 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7fc7300032f0 tx=0x7fc7300091d0).stop 2021-11-21T22:02:10.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- 172.21.2.116:0/1553748507 shutdown_connections 2021-11-21T22:02:10.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/1553748507 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed81466b0 0x7f0ed80b4370 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:10.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- 172.21.2.116:0/1553748507 >> 172.21.2.116:0/1553748507 conn(0x7f0ed80629c0 msgr2=0x7f0ed8062dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:10.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc800007170 con 0x7fc7f0003750 2021-11-21T22:02:10.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- 172.21.2.116:0/1230095506 shutdown_connections 2021-11-21T22:02:10.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 --2- 172.21.2.116:0/1230095506 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 0x7fc7380b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 --2- 172.21.2.116:0/1230095506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc738098e80 0x7fc73809b510 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:10.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- 172.21.2.116:0/1230095506 >> 172.21.2.116:0/1230095506 conn(0x7fc7380b5df0 msgr2=0x7fc7380b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:10.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- 172.21.2.116:0/1553748507 shutdown_connections 2021-11-21T22:02:10.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000072d0 con 0x7fc7f0003750 2021-11-21T22:02:10.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc81092a700 1 -- 172.21.2.116:0/1275391721 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc80000c8c0 con 0x7fc7f0003750 2021-11-21T22:02:10.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7f0edc9f0700 1 -- 172.21.2.116:0/1553748507 wait complete. 2021-11-21T22:02:10.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- 172.21.2.116:0/1230095506 shutdown_connections 2021-11-21T22:02:10.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.646+0000 7fc73e90e700 1 -- 172.21.2.116:0/1230095506 wait complete. 2021-11-21T22:02:10.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/1275391721 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0003750 msgr2=0x7fc7f0003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 --2- 172.21.2.116:0/1275391721 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7f0003750 0x7fc7f0003b70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7fc800001390 tx=0x7fc80c0b5ab0).stop 2021-11-21T22:02:10.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81212d700 1 -- 172.21.2.116:0/1275391721 reap_dead start 2021-11-21T22:02:10.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/1275391721 shutdown_connections 2021-11-21T22:02:10.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/1275391721 >> 172.21.2.116:0/1275391721 conn(0x7fc80c05c790 msgr2=0x7fc80c0b9360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:10.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/1275391721 shutdown_connections 2021-11-21T22:02:10.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/1275391721 wait complete. 2021-11-21T22:02:10.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 Processor -- start 2021-11-21T22:02:10.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 Processor -- start 2021-11-21T22:02:10.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 -- start start 2021-11-21T22:02:10.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 -- start start 2021-11-21T22:02:10.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 Processor -- start 2021-11-21T22:02:10.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- start start 2021-11-21T22:02:10.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc738098e80 0x7fc7381443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ed805cfe0 0x7f0ed8141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 0x7fc7380af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7381463d0 0x7fc7381556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed80b6250 0x7f0ed8152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738151e50 con 0x7fc7381463d0 2021-11-21T22:02:10.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73814fb00 con 0x7fc738098e80 2021-11-21T22:02:10.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380af0e0 con 0x7fc7380b4ce0 2021-11-21T22:02:10.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ed81466b0 0x7f0ed814f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8143560 con 0x7f0ed80b6250 2021-11-21T22:02:10.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8141ba0 con 0x7f0ed805cfe0 2021-11-21T22:02:10.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81432c0 con 0x7f0ed81466b0 2021-11-21T22:02:10.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed80b6250 0x7f0ed8152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7381463d0 0x7fc7381556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7381463d0 0x7fc7381556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52450/0 (socket says 172.21.2.116:52450) 2021-11-21T22:02:10.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 -- 172.21.2.116:0/3453498381 learned_addr learned my addr 172.21.2.116:0/3453498381 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:10.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed80b6250 0x7f0ed8152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52452/0 (socket says 172.21.2.116:52452) 2021-11-21T22:02:10.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 -- 172.21.2.116:0/3716384093 learned_addr learned my addr 172.21.2.116:0/3716384093 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:10.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 -- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 msgr2=0x7fc7380af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 0x7fc7380af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 -- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc738098e80 msgr2=0x7fc7381443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc738098e80 0x7fc7381443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 -- 172.21.2.116:0/3453498381 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730009470 con 0x7fc7381463d0 2021-11-21T22:02:10.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed659c700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ed805cfe0 0x7f0ed8141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f0003750 0x7fc80c0b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc80c093dd0 0x7fc80c0af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 -- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ed81466b0 msgr2=0x7f0ed814f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ed81466b0 0x7f0ed814f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 -- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ed805cfe0 msgr2=0x7f0ed8141450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ed805cfe0 0x7f0ed8141450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:10.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc80c099000 0x7fc80c0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 -- 172.21.2.116:0/3716384093 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec800b180 con 0x7f0ed80b6250 2021-11-21T22:02:10.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c090d50 con 0x7fc80c093dd0 2021-11-21T22:02:10.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c094270 con 0x7fc80c099000 2021-11-21T22:02:10.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c094510 con 0x7fc7f0003750 2021-11-21T22:02:10.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73ceab700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7381463d0 0x7fc7381556f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7fc7380af270 tx=0x7fc728006da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc80c093dd0 0x7fc80c0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc80c093dd0 0x7fc80c0af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52456/0 (socket says 172.21.2.116:52456) 2021-11-21T22:02:10.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed5d9b700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed80b6250 0x7f0ed8152280 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f0ecc002f90 tx=0x7f0ecc006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 -- 172.21.2.116:0/3195132942 learned_addr learned my addr 172.21.2.116:0/3195132942 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:10.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7357fa700 1 -- 172.21.2.116:0/3453498381 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc728012410 con 0x7fc7381463d0 2021-11-21T22:02:10.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7357fa700 1 -- 172.21.2.116:0/3453498381 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728012570 con 0x7fc7381463d0 2021-11-21T22:02:10.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7380af820 con 0x7fc7381463d0 2021-11-21T22:02:10.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7357fa700 1 -- 172.21.2.116:0/3453498381 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7280128d0 con 0x7fc7381463d0 2021-11-21T22:02:10.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7380aee40 con 0x7fc7381463d0 2021-11-21T22:02:10.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ec77fe700 1 -- 172.21.2.116:0/3716384093 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ecc013460 con 0x7f0ed80b6250 2021-11-21T22:02:10.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ec77fe700 1 -- 172.21.2.116:0/3716384093 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ecc0135c0 con 0x7f0ed80b6250 2021-11-21T22:02:10.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed80b9c20 con 0x7f0ed80b6250 2021-11-21T22:02:10.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ec77fe700 1 -- 172.21.2.116:0/3716384093 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ecc013920 con 0x7f0ed80b6250 2021-11-21T22:02:10.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed8143880 con 0x7f0ed80b6250 2021-11-21T22:02:10.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 -- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f0003750 msgr2=0x7fc80c0b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f0003750 0x7fc80c0b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 -- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc80c099000 msgr2=0x7fc80c0afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:10.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc80c099000 0x7fc80c0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:10.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 -- 172.21.2.116:0/3195132942 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c0947b0 con 0x7fc80c093dd0 2021-11-21T22:02:10.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81112b700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc80c093dd0 0x7fc80c0af730 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7fc808007000 tx=0x7fc808008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7feffd700 1 -- 172.21.2.116:0/3195132942 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc80800f4e0 con 0x7fc80c093dd0 2021-11-21T22:02:10.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c0af290 con 0x7fc80c093dd0 2021-11-21T22:02:10.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c092830 con 0x7fc80c093dd0 2021-11-21T22:02:10.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7feffd700 1 -- 172.21.2.116:0/3195132942 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808000e90 con 0x7fc80c093dd0 2021-11-21T22:02:10.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7feffd700 1 -- 172.21.2.116:0/3195132942 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc80800f640 con 0x7fc80c093dd0 2021-11-21T22:02:10.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7380acd00 con 0x7fc7381463d0 2021-11-21T22:02:10.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7357fa700 1 -- 172.21.2.116:0/3453498381 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc72800f030 con 0x7fc7381463d0 2021-11-21T22:02:10.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed8143100 con 0x7f0ed80b6250 2021-11-21T22:02:10.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e8003410 con 0x7fc80c093dd0 2021-11-21T22:02:10.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ec77fe700 1 -- 172.21.2.116:0/3716384093 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0ecc01a060 con 0x7f0ed80b6250 2021-11-21T22:02:10.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7feffd700 1 -- 172.21.2.116:0/3195132942 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc80800f7a0 con 0x7fc80c093dd0 2021-11-21T22:02:10.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7357fa700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc718053610 0x7fc718055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7357fa700 1 -- 172.21.2.116:0/3453498381 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc72806b740 con 0x7fc7381463d0 2021-11-21T22:02:10.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ec77fe700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0eb40536a0 0x7f0eb4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.725 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ec77fe700 1 -- 172.21.2.116:0/3716384093 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0ecc06b300 con 0x7f0ed80b6250 2021-11-21T22:02:10.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7feffd700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7f4053610 0x7fc7f4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:10.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc7feffd700 1 -- 172.21.2.116:0/3195132942 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc80806b3b0 con 0x7fc80c093dd0 2021-11-21T22:02:10.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc737fff700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc718053610 0x7fc718055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7f0ed659c700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0eb40536a0 0x7f0eb4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.650+0000 7fc81192c700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7f4053610 0x7fc7f4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:10.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.654+0000 7fc81192c700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7f4053610 0x7fc7f4055ad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fc800001390 tx=0x7fc800015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.654+0000 7fc737fff700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc718053610 0x7fc718055ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fc72c000ed0 tx=0x7fc72c0076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.654+0000 7f0ed659c700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0eb40536a0 0x7f0eb4055b60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f0ec8015000 tx=0x7f0ec8014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:10.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.654+0000 7fc7357fa700 1 -- 172.21.2.116:0/3453498381 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc728042800 con 0x7fc7381463d0 2021-11-21T22:02:10.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.658+0000 7f0ec77fe700 1 -- 172.21.2.116:0/3716384093 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ecc042460 con 0x7f0ed80b6250 2021-11-21T22:02:10.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.662+0000 7fc7feffd700 1 -- 172.21.2.116:0/3195132942 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc808042470 con 0x7fc80c093dd0 2021-11-21T22:02:10.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.770+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7fc7380573d0 con 0x7fc7381463d0 2021-11-21T22:02:10.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.774+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f0ed80573d0 con 0x7f0ed80b6250 2021-11-21T22:02:10.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:10.778+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fc7e80036c0 con 0x7fc80c093dd0 2021-11-21T22:02:11.439 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:02:11.434+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:02:11.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc7357fa700 1 -- 172.21.2.116:0/3453498381 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7674 v56) v1 ==== 95+0+0 (secure 0 0 0) 0x7fc728045e20 con 0x7fc7381463d0 2021-11-21T22:02:11.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0ec77fe700 1 -- 172.21.2.116:0/3716384093 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7665 v56) v1 ==== 95+0+0 (secure 0 0 0) 0x7f0ecc018020 con 0x7f0ed80b6250 2021-11-21T22:02:11.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc7feffd700 1 -- 172.21.2.116:0/3195132942 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 4868 v56) v1 ==== 95+0+0 (secure 0 0 0) 0x7fc808045a90 con 0x7fc80c093dd0 2021-11-21T22:02:11.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc718053610 msgr2=0x7fc718055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc718053610 0x7fc718055ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fc72c000ed0 tx=0x7fc72c0076c0).stop 2021-11-21T22:02:11.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7381463d0 msgr2=0x7fc7381556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7381463d0 0x7fc7381556f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7fc7380af270 tx=0x7fc728006da0).stop 2021-11-21T22:02:11.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 shutdown_connections 2021-11-21T22:02:11.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc718053610 0x7fc718055ad0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc7381463d0 0x7fc7381556f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7380b4ce0 0x7fc7380af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:11.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 --2- 172.21.2.116:0/3453498381 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc738098e80 0x7fc7381443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:11.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 >> 172.21.2.116:0/3453498381 conn(0x7fc7380b5df0 msgr2=0x7fc7381434b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:11.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 shutdown_connections 2021-11-21T22:02:11.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc73e90e700 1 -- 172.21.2.116:0/3453498381 wait complete. 2021-11-21T22:02:11.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0eb40536a0 msgr2=0x7f0eb4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7f4053610 msgr2=0x7fc7f4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0eb40536a0 0x7f0eb4055b60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f0ec8015000 tx=0x7f0ec8014000).stop 2021-11-21T22:02:11.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed80b6250 msgr2=0x7f0ed8152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7f4053610 0x7fc7f4055ad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fc800001390 tx=0x7fc800015000).stop 2021-11-21T22:02:11.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc80c093dd0 msgr2=0x7fc80c0af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed80b6250 0x7f0ed8152280 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f0ecc002f90 tx=0x7f0ecc006de0).stop 2021-11-21T22:02:11.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc80c093dd0 0x7fc80c0af730 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7fc808007000 tx=0x7fc808008040).stop 2021-11-21T22:02:11.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 shutdown_connections 2021-11-21T22:02:11.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0ed81466b0 0x7f0ed814f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:11.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0eb40536a0 0x7f0eb4055b60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0ed80b6250 0x7f0ed8152280 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 --2- 172.21.2.116:0/3716384093 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0ed805cfe0 0x7f0ed8141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 >> 172.21.2.116:0/3716384093 conn(0x7f0ed80629c0 msgr2=0x7f0ed8128ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:11.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 shutdown_connections 2021-11-21T22:02:11.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc80c099000 0x7fc80c0afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:11.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc7f4053610 0x7fc7f4055ad0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc80c093dd0 0x7fc80c0af730 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 --2- 172.21.2.116:0/3195132942 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc7f0003750 0x7fc80c0b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:11.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 >> 172.21.2.116:0/3195132942 conn(0x7fc80c05c790 msgr2=0x7fc80c05f730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:11.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 shutdown_connections 2021-11-21T22:02:11.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7f0edc9f0700 1 -- 172.21.2.116:0/3716384093 wait complete. 2021-11-21T22:02:11.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 shutdown_connections 2021-11-21T22:02:11.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.450+0000 7fc813b90700 1 -- 172.21.2.116:0/3195132942 wait complete. 2021-11-21T22:02:11.466 INFO:teuthology.orchestra.run.gibba016.stderr:failed mds gid 7674 2021-11-21T22:02:11.467 INFO:teuthology.orchestra.run.gibba016.stderr:failed mds gid 7665 2021-11-21T22:02:11.467 INFO:teuthology.orchestra.run.gibba016.stderr:failed mds gid 4868 2021-11-21T22:02:11.468 DEBUG:teuthology.parallel:result is '' 2021-11-21T22:02:11.470 DEBUG:teuthology.parallel:result is '' 2021-11-21T22:02:11.470 DEBUG:teuthology.parallel:result is '' 2021-11-21T22:02:11.471 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:02:11.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27bdc7f700 1 Processor -- start 2021-11-21T22:02:11.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27bdc7f700 1 -- start start 2021-11-21T22:02:11.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27bdc7f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 0x7f27b80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27bdc7f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b8147e80 con 0x7f27b80abce0 2021-11-21T22:02:11.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27bdc7f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b80af5a0 con 0x7f27b8144810 2021-11-21T22:02:11.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27bdc7f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80b1490 con 0x7f27b81478c0 2021-11-21T22:02:11.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b6ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f27b80abce0 0x7f27b80ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36196/0 (socket says 172.21.2.116:36196) 2021-11-21T22:02:11.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b6ffd700 1 -- 172.21.2.116:0/4039506150 learned_addr learned my addr 172.21.2.116:0/4039506150 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:11.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158087414 0 0) 0x7f27b8147e80 con 0x7f27b80abce0 2021-11-21T22:02:11.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2798003260 con 0x7f27b80abce0 2021-11-21T22:02:11.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 0x7f27b80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916670172 0 0) 0x7f2798003260 con 0x7f27b80abce0 2021-11-21T22:02:11.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 msgr2=0x7f27b80acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:11.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 0x7f27b80acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 >> v1:172.21.2.117:6789/0 conn(0x7f27b8144810 legacy=0x7f27b80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b80b1fd0 con 0x7f27b80abce0 2021-11-21T22:02:11.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f27a8001680 con 0x7f27b80abce0 2021-11-21T22:02:11.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 >> v1:172.21.2.116:6789/0 conn(0x7f27b80abce0 legacy=0x7f27b80ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 0x7f2798003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2798003d00 0x7f2798004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27980049f0 0x7f2798004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80b1490 con 0x7f27980049f0 2021-11-21T22:02:11.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279800b830 con 0x7f2798003d00 2021-11-21T22:02:11.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279800b9f0 con 0x7f27b81478c0 2021-11-21T22:02:11.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27980049f0 0x7f2798004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b77fe700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 0x7f2798003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b6ffd700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2798003d00 0x7f2798004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 -- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 msgr2=0x7f2798003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:11.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27b81478c0 0x7f2798003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 -- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2798003d00 msgr2=0x7f2798004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:11.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2798003d00 0x7f2798004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 -- 172.21.2.116:0/4039506150 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798003260 con 0x7f27980049f0 2021-11-21T22:02:11.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b7fff700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27980049f0 0x7f2798004e70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f27b000c000 tx=0x7f27b00091d0).ready entity=mon.0 client_cookie=52ca3a1d6a33824 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:11.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27b000e7e0 con 0x7f27980049f0 2021-11-21T22:02:11.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b000e940 con 0x7f27980049f0 2021-11-21T22:02:11.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.526+0000 7f27b67fc700 1 -- 172.21.2.116:0/4039506150 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27b000eca0 con 0x7f27980049f0 2021-11-21T22:02:11.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/4039506150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27980049f0 msgr2=0x7f2798004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 --2- 172.21.2.116:0/4039506150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27980049f0 0x7f2798004e70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f27b000c000 tx=0x7f27b00091d0).stop 2021-11-21T22:02:11.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b7fff700 1 -- 172.21.2.116:0/4039506150 reap_dead start 2021-11-21T22:02:11.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/4039506150 shutdown_connections 2021-11-21T22:02:11.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/4039506150 >> 172.21.2.116:0/4039506150 conn(0x7f27b805bd20 msgr2=0x7f27b805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:11.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/4039506150 shutdown_connections 2021-11-21T22:02:11.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/4039506150 wait complete. 2021-11-21T22:02:11.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 Processor -- start 2021-11-21T22:02:11.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- start start 2021-11-21T22:02:11.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27980049f0 0x7f27b8067170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27b8156830 0x7f27b8156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27b8157180 0x7f27b8159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b814c090 con 0x7f27b8156830 2021-11-21T22:02:11.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b814d940 con 0x7f27b8157180 2021-11-21T22:02:11.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b814dba0 con 0x7f27980049f0 2021-11-21T22:02:11.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27b8156830 0x7f27b8156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27b8156830 0x7f27b8156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52480/0 (socket says 172.21.2.116:52480) 2021-11-21T22:02:11.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 -- 172.21.2.116:0/1530623811 learned_addr learned my addr 172.21.2.116:0/1530623811 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:11.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b7fff700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27b8157180 0x7f27b8159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b77fe700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27980049f0 0x7f27b8067170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 -- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27980049f0 msgr2=0x7f27b8067170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27980049f0 0x7f27b8067170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 -- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27b8157180 msgr2=0x7f27b8159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:11.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27b8157180 0x7f27b8159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:11.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 -- 172.21.2.116:0/1530623811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b814c960 con 0x7f27b8156830 2021-11-21T22:02:11.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b7fff700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27b8157180 0x7f27b8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:02:11.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b6ffd700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27b8156830 0x7f27b8156c90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f27a800b000 tx=0x7f27a8001770).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:11.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b4ff9700 1 -- 172.21.2.116:0/1530623811 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a8015630 con 0x7f27b8156830 2021-11-21T22:02:11.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b4ff9700 1 -- 172.21.2.116:0/1530623811 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a8010040 con 0x7f27b8156830 2021-11-21T22:02:11.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b814cc00 con 0x7f27b8156830 2021-11-21T22:02:11.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b4ff9700 1 -- 172.21.2.116:0/1530623811 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a8015980 con 0x7f27b8156830 2021-11-21T22:02:11.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b8064630 con 0x7f27b8156830 2021-11-21T22:02:11.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b80af060 con 0x7f27b8156830 2021-11-21T22:02:11.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.530+0000 7f27b4ff9700 1 -- 172.21.2.116:0/1530623811 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f27a80103c0 con 0x7f27b8156830 2021-11-21T22:02:11.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.534+0000 7f27b4ff9700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f279c0536d0 0x7f279c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:11.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.534+0000 7f27b4ff9700 1 -- 172.21.2.116:0/1530623811 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f27a806b3a0 con 0x7f27b8156830 2021-11-21T22:02:11.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.534+0000 7f27b77fe700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f279c0536d0 0x7f279c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:11.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.534+0000 7f27b77fe700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f279c0536d0 0x7f279c055b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f27a000d000 tx=0x7f27a0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:11.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.534+0000 7f27b4ff9700 1 -- 172.21.2.116:0/1530623811 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f27a8042450 con 0x7f27b8156830 2021-11-21T22:02:11.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:11.642+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f27b80573d0 con 0x7f27b8156830 2021-11-21T22:02:12.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.454+0000 7f27b4ff9700 1 -- 172.21.2.116:0/1530623811 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f27a8045a70 con 0x7f27b8156830 2021-11-21T22:02:12.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f279c0536d0 msgr2=0x7f279c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:12.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f279c0536d0 0x7f279c055b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f27a000d000 tx=0x7f27a0009040).stop 2021-11-21T22:02:12.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27b8156830 msgr2=0x7f27b8156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:12.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27b8156830 0x7f27b8156c90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f27a800b000 tx=0x7f27a8001770).stop 2021-11-21T22:02:12.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 shutdown_connections 2021-11-21T22:02:12.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f27b8157180 0x7f27b8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f279c0536d0 0x7f279c055b90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f27b8156830 0x7f27b8156c90 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 --2- 172.21.2.116:0/1530623811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f27980049f0 0x7f27b8067170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 >> 172.21.2.116:0/1530623811 conn(0x7f27b805bd20 msgr2=0x7f27b81422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:12.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 shutdown_connections 2021-11-21T22:02:12.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.458+0000 7f27bdc7f700 1 -- 172.21.2.116:0/1530623811 wait complete. 2021-11-21T22:02:12.468 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:02:12.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.514+0000 7fe48d6fa700 1 Processor -- start 2021-11-21T22:02:12.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.514+0000 7fe48d6fa700 1 -- start start 2021-11-21T22:02:12.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 0x7fe488147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe4880a8670 0x7fe488147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4880b1940 con 0x7fe488147c10 2021-11-21T22:02:12.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4880b15a0 con 0x7fe4880a8670 2021-11-21T22:02:12.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880b1b70 con 0x7fe4880b11c0 2021-11-21T22:02:12.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe4877fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe488147c10 0x7fe48814a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36216/0 (socket says 172.21.2.116:36216) 2021-11-21T22:02:12.525 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe4877fe700 1 -- 172.21.2.116:0/1349294103 learned_addr learned my addr 172.21.2.116:0/1349294103 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:12.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624383493 0 0) 0x7fe4880b1940 con 0x7fe488147c10 2021-11-21T22:02:12.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe468003260 con 0x7fe488147c10 2021-11-21T22:02:12.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe4867fc700 1 --2- 172.21.2.116:0/1349294103 >> v2:172.21.2.117:3300/0 conn(0x7fe4880a8670 0x7fe488147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:12.526 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 0x7fe488147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:12.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438649004 0 0) 0x7fe468003260 con 0x7fe488147c10 2021-11-21T22:02:12.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 msgr2=0x7fe488147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:12.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 0x7fe488147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 >> v2:172.21.2.117:3300/0 conn(0x7fe4880a8670 msgr2=0x7fe488147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:12.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 --2- 172.21.2.116:0/1349294103 >> v2:172.21.2.117:3300/0 conn(0x7fe4880a8670 0x7fe488147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880b1e10 con 0x7fe488147c10 2021-11-21T22:02:12.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe4800019e0 con 0x7fe488147c10 2021-11-21T22:02:12.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 >> v1:172.21.2.116:6789/0 conn(0x7fe488147c10 legacy=0x7fe48814a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:12.528 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4680037d0 0x7fe468003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 0x7fe468004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4680049f0 0x7fe468004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880b15a0 con 0x7fe4680037d0 2021-11-21T22:02:12.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880b1b70 con 0x7fe4680049f0 2021-11-21T22:02:12.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46800b750 con 0x7fe4880b11c0 2021-11-21T22:02:12.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4680037d0 0x7fe468003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:12.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe4877fe700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4680049f0 0x7fe468004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:12.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe4867fc700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 0x7fe468004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:12.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 -- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 msgr2=0x7fe468004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:12.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 0x7fe468004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 -- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4680049f0 msgr2=0x7fe468004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:12.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4680049f0 0x7fe468004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:12.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 -- 172.21.2.116:0/1349294103 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468003260 con 0x7fe4680037d0 2021-11-21T22:02:12.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe4867fc700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4880b11c0 0x7fe468004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:12.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe486ffd700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4680037d0 0x7fe468003bf0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fe47000c000 tx=0x7fe470006df0).ready entity=mon.0 client_cookie=e65d7ee16ef7d01e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:12.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47000e830 con 0x7fe4680037d0 2021-11-21T22:02:12.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47000e990 con 0x7fe4680037d0 2021-11-21T22:02:12.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe485ffb700 1 -- 172.21.2.116:0/1349294103 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47000ecf0 con 0x7fe4680037d0 2021-11-21T22:02:12.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1349294103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4680037d0 msgr2=0x7fe468003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:12.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 --2- 172.21.2.116:0/1349294103 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4680037d0 0x7fe468003bf0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fe47000c000 tx=0x7fe470006df0).stop 2021-11-21T22:02:12.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe4877fe700 1 -- 172.21.2.116:0/1349294103 reap_dead start 2021-11-21T22:02:12.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1349294103 shutdown_connections 2021-11-21T22:02:12.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1349294103 >> 172.21.2.116:0/1349294103 conn(0x7fe48805bd20 msgr2=0x7fe48805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:12.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1349294103 shutdown_connections 2021-11-21T22:02:12.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1349294103 wait complete. 2021-11-21T22:02:12.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 Processor -- start 2021-11-21T22:02:12.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.518+0000 7fe48d6fa700 1 -- start start 2021-11-21T22:02:12.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4680037d0 0x7fe488145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe488159d10 0x7fe48815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe488143cb0 0x7fe488144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48813f8a0 con 0x7fe488143cb0 2021-11-21T22:02:12.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488140cf0 con 0x7fe488159d10 2021-11-21T22:02:12.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe488140290 con 0x7fe4680037d0 2021-11-21T22:02:12.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe488143cb0 0x7fe488144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:12.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe488143cb0 0x7fe488144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52504/0 (socket says 172.21.2.116:52504) 2021-11-21T22:02:12.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 -- 172.21.2.116:0/1979899837 learned_addr learned my addr 172.21.2.116:0/1979899837 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:12.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 -- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4680037d0 msgr2=0x7fe488145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:12.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4680037d0 0x7fe488145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:12.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 -- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe488159d10 msgr2=0x7fe48815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:12.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe488159d10 0x7fe48815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:12.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 -- 172.21.2.116:0/1979899837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe488140850 con 0x7fe488143cb0 2021-11-21T22:02:12.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe4877fe700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe488143cb0 0x7fe488144160 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7fe480001210 tx=0x7fe4880b1fa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:12.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe47ffff700 1 -- 172.21.2.116:0/1979899837 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe480010710 con 0x7fe488143cb0 2021-11-21T22:02:12.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe47ffff700 1 -- 172.21.2.116:0/1979899837 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480010870 con 0x7fe488143cb0 2021-11-21T22:02:12.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4881542f0 con 0x7fe488143cb0 2021-11-21T22:02:12.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe47ffff700 1 -- 172.21.2.116:0/1979899837 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe480010bd0 con 0x7fe488143cb0 2021-11-21T22:02:12.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48815a680 con 0x7fe488143cb0 2021-11-21T22:02:12.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe488140490 con 0x7fe488143cb0 2021-11-21T22:02:12.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe47ffff700 1 -- 172.21.2.116:0/1979899837 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe480010d30 con 0x7fe488143cb0 2021-11-21T22:02:12.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe47ffff700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe46c0535b0 0x7fe46c055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:12.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe47ffff700 1 -- 172.21.2.116:0/1979899837 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fe48006b990 con 0x7fe488143cb0 2021-11-21T22:02:12.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe486ffd700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe46c0535b0 0x7fe46c055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:12.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe486ffd700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe46c0535b0 0x7fe46c055a70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe47000c000 tx=0x7fe470019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:12.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.522+0000 7fe47ffff700 1 -- 172.21.2.116:0/1979899837 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe480042930 con 0x7fe488143cb0 2021-11-21T22:02:12.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:12.606+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe4881409e0 con 0x7fe488143cb0 2021-11-21T22:02:13.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe47ffff700 1 -- 172.21.2.116:0/1979899837 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 185+0+0 (secure 0 0 0) 0x7fe480045f50 con 0x7fe488143cb0 2021-11-21T22:02:13.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe46c0535b0 msgr2=0x7fe46c055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:13.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe46c0535b0 0x7fe46c055a70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe47000c000 tx=0x7fe470019000).stop 2021-11-21T22:02:13.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe488143cb0 msgr2=0x7fe488144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:13.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe488143cb0 0x7fe488144160 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7fe480001210 tx=0x7fe4880b1fa0).stop 2021-11-21T22:02:13.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 shutdown_connections 2021-11-21T22:02:13.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe46c0535b0 0x7fe46c055a70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:13.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe488143cb0 0x7fe488144160 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:13.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe488159d10 0x7fe48815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:13.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 --2- 172.21.2.116:0/1979899837 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4680037d0 0x7fe488145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:13.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 >> 172.21.2.116:0/1979899837 conn(0x7fe48805bd20 msgr2=0x7fe48805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:13.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 shutdown_connections 2021-11-21T22:02:13.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.462+0000 7fe48d6fa700 1 -- 172.21.2.116:0/1979899837 wait complete. 2021-11-21T22:02:13.470 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:02:13.474 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:02:13.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7deae9c700 1 Processor -- start 2021-11-21T22:02:13.527 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7deae9c700 1 -- start start 2021-11-21T22:02:13.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7deae9c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7de40adc10 0x7f7de40b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:13.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7deae9c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7de4144a10 0x7f7de4141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:13.529 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7deae9c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de40b3c50 con 0x7f7de4141510 2021-11-21T22:02:13.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7deae9c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40b3ef0 con 0x7f7de4144a10 2021-11-21T22:02:13.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7deae9c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40b4190 con 0x7f7de40adc10 2021-11-21T22:02:13.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7de4144a10 0x7f7de4141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:13.530 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de9439700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7de4141510 0x7f7de413a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36232/0 (socket says 172.21.2.116:36232) 2021-11-21T22:02:13.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de9439700 1 -- 172.21.2.116:0/3606764832 learned_addr learned my addr 172.21.2.116:0/3606764832 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:13.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de3fff700 1 -- 172.21.2.116:0/3606764832 >> v2:172.21.2.117:3300/0 conn(0x7f7de40adc10 msgr2=0x7f7de40b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:13.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de3fff700 1 --2- 172.21.2.116:0/3606764832 >> v2:172.21.2.117:3300/0 conn(0x7f7de40adc10 0x7f7de40b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:13.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de3fff700 1 -- 172.21.2.116:0/3606764832 >> v1:172.21.2.116:6789/0 conn(0x7f7de4141510 legacy=0x7f7de413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:13.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de37fe700 1 -- 172.21.2.116:0/3606764832 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224369785 0 0) 0x7f7de40b3c50 con 0x7f7de4141510 2021-11-21T22:02:13.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.522+0000 7f7de3fff700 1 -- 172.21.2.116:0/3606764832 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4140b80 con 0x7f7de4144a10 2021-11-21T22:02:13.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 --2- 172.21.2.116:0/3606764832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4144a10 0x7f7de4141020 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f7dd8002f70 tx=0x7f7dd8006df0).ready entity=mon.0 client_cookie=66bcd9d0560e17c0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:13.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de37fe700 1 -- 172.21.2.116:0/3606764832 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd800d5c0 con 0x7f7de4144a10 2021-11-21T22:02:13.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de37fe700 1 -- 172.21.2.116:0/3606764832 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd8000e90 con 0x7f7de4144a10 2021-11-21T22:02:13.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de37fe700 1 -- 172.21.2.116:0/3606764832 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd800d910 con 0x7f7de4144a10 2021-11-21T22:02:13.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/3606764832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4144a10 msgr2=0x7f7de4141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:13.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 --2- 172.21.2.116:0/3606764832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4144a10 0x7f7de4141020 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f7dd8002f70 tx=0x7f7dd8006df0).stop 2021-11-21T22:02:13.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/3606764832 shutdown_connections 2021-11-21T22:02:13.533 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 --2- 172.21.2.116:0/3606764832 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4144a10 0x7f7de4141020 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:13.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 --2- 172.21.2.116:0/3606764832 >> v2:172.21.2.117:3300/0 conn(0x7f7de40adc10 0x7f7de40b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:13.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/3606764832 >> 172.21.2.116:0/3606764832 conn(0x7f7de405bd20 msgr2=0x7f7de405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:13.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/3606764832 shutdown_connections 2021-11-21T22:02:13.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/3606764832 wait complete. 2021-11-21T22:02:13.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 Processor -- start 2021-11-21T22:02:13.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- start start 2021-11-21T22:02:13.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7de40adc10 0x7f7de4131220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:13.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4141510 0x7f7de4142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:13.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7de4144a10 0x7f7de4061ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:13.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de41435e0 con 0x7f7de4141510 2021-11-21T22:02:13.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4130df0 con 0x7f7de4144a10 2021-11-21T22:02:13.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4143810 con 0x7f7de40adc10 2021-11-21T22:02:13.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4141510 0x7f7de4142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:13.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4141510 0x7f7de4142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52518/0 (socket says 172.21.2.116:52518) 2021-11-21T22:02:13.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 -- 172.21.2.116:0/878186998 learned_addr learned my addr 172.21.2.116:0/878186998 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:13.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 -- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7de40adc10 msgr2=0x7f7de4131220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:13.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7de40adc10 0x7f7de4131220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:13.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 -- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7de4144a10 msgr2=0x7f7de4061ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:13.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7de4144a10 0x7f7de4061ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:13.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 -- 172.21.2.116:0/878186998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd800b180 con 0x7f7de4141510 2021-11-21T22:02:13.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de3fff700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4141510 0x7f7de4142e00 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f7dd8002f70 tx=0x7f7dd8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:13.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de1ffb700 1 -- 172.21.2.116:0/878186998 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd801e400 con 0x7f7de4141510 2021-11-21T22:02:13.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de1ffb700 1 -- 172.21.2.116:0/878186998 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd801e560 con 0x7f7de4141510 2021-11-21T22:02:13.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de40b5fd0 con 0x7f7de4141510 2021-11-21T22:02:13.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de1ffb700 1 -- 172.21.2.116:0/878186998 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd801e8c0 con 0x7f7de4141510 2021-11-21T22:02:13.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de40b5ce0 con 0x7f7de4141510 2021-11-21T22:02:13.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de40573d0 con 0x7f7de4141510 2021-11-21T22:02:13.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de1ffb700 1 -- 172.21.2.116:0/878186998 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7dd801c070 con 0x7f7de4141510 2021-11-21T22:02:13.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de1ffb700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7dc40536d0 0x7f7dc4055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:13.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de1ffb700 1 -- 172.21.2.116:0/878186998 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f7dd806c3e0 con 0x7f7de4141510 2021-11-21T22:02:13.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.526+0000 7f7de8c38700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7dc40536d0 0x7f7dc4055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:13.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.530+0000 7f7de8c38700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7dc40536d0 0x7f7dc4055b90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f7dd4002f70 tx=0x7f7dd4006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:13.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.530+0000 7f7de1ffb700 1 -- 172.21.2.116:0/878186998 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7dd80436a0 con 0x7f7de4141510 2021-11-21T22:02:13.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:13.614+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f7de40b6da0 con 0x7f7de4141510 2021-11-21T22:02:14.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.474+0000 7f7de1ffb700 1 -- 172.21.2.116:0/878186998 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v127) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7dd8046cc0 con 0x7f7de4141510 2021-11-21T22:02:14.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.474+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7dc40536d0 msgr2=0x7f7dc4055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:14.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.474+0000 7f7deae9c700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7dc40536d0 0x7f7dc4055b90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f7dd4002f70 tx=0x7f7dd4006a80).stop 2021-11-21T22:02:14.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.474+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4141510 msgr2=0x7f7de4142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:14.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.474+0000 7f7deae9c700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4141510 0x7f7de4142e00 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f7dd8002f70 tx=0x7f7dd8014000).stop 2021-11-21T22:02:14.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 shutdown_connections 2021-11-21T22:02:14.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7de4144a10 0x7f7de4061ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:14.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7dc40536d0 0x7f7dc4055b90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7de4141510 0x7f7de4142e00 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 --2- 172.21.2.116:0/878186998 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7de40adc10 0x7f7de4131220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:14.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 >> 172.21.2.116:0/878186998 conn(0x7f7de405bd20 msgr2=0x7f7de40670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:14.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 shutdown_connections 2021-11-21T22:02:14.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:14.478+0000 7f7deae9c700 1 -- 172.21.2.116:0/878186998 wait complete. 2021-11-21T22:02:14.486 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:02:14.488 DEBUG:teuthology.orchestra.run.gibba019:> sudo rados -p cephfs_metadata put key /etc/fstab 2021-11-21T22:02:14.512 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 -- 172.21.2.119:0/1898294054 wait complete. 2021-11-21T22:02:14.513 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 Processor -- start 2021-11-21T22:02:14.513 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 -- start start 2021-11-21T22:02:14.514 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b7b150 0x564999b70e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.514 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b71300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.514 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b806b0 0x564999b717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.514 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x564999a78210 con 0x564999b7b150 2021-11-21T22:02:14.515 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564999a68460 con 0x564999b7dda0 2021-11-21T22:02:14.515 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b40e8b80 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564999b79cf0 con 0x564999b806b0 2021-11-21T22:02:14.519 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b2526700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b7b150 0x564999b70e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.519 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b2526700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b7b150 0x564999b70e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.119:50700/0 (socket says 172.21.2.119:50700) 2021-11-21T22:02:14.519 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b2526700 1 -- 172.21.2.119:0/4037253604 learned_addr learned my addr 172.21.2.119:0/4037253604 (peer_addr_for_me v2:172.21.2.119:0/0) 2021-11-21T22:02:14.519 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b2d27700 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b806b0 0x564999b717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.520 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.509+0000 7f88b1d25700 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b71300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.520 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 -- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b806b0 msgr2=0x564999b717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:14.520 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b806b0 0x564999b717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.520 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 -- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 msgr2=0x564999b71300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:14.521 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b71300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.521 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 -- 172.21.2.119:0/4037253604 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a8005cd0 con 0x564999b7b150 2021-11-21T22:02:14.521 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b7b150 0x564999b70e10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x564999b76350 tx=0x7f88a4001870).ready entity=mon.0 client_cookie=7f5a55ec8a16e863 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:14.521 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a3fff700 1 -- 172.21.2.119:0/4037253604 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a400bc60 con 0x564999b7b150 2021-11-21T22:02:14.522 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a3fff700 1 -- 172.21.2.119:0/4037253604 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4008070 con 0x564999b7b150 2021-11-21T22:02:14.522 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a3fff700 1 -- 172.21.2.119:0/4037253604 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a4010460 con 0x564999b7b150 2021-11-21T22:02:14.522 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- 172.21.2.119:0/4037253604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b7b150 msgr2=0x564999b70e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:14.522 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b7b150 0x564999b70e10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x564999b76350 tx=0x7f88a4001870).stop 2021-11-21T22:02:14.522 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- 172.21.2.119:0/4037253604 shutdown_connections 2021-11-21T22:02:14.523 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b806b0 0x564999b717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.523 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b71300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.523 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/4037253604 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b7b150 0x564999b70e10 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.523 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- 172.21.2.119:0/4037253604 >> 172.21.2.119:0/4037253604 conn(0x564999b70a30 msgr2=0x564999b7cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:14.524 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- 172.21.2.119:0/4037253604 shutdown_connections 2021-11-21T22:02:14.524 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- 172.21.2.119:0/4037253604 wait complete. 2021-11-21T22:02:14.524 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 Processor -- start 2021-11-21T22:02:14.524 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- start start 2021-11-21T22:02:14.525 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b7b150 0x564999b77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.525 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b781a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.525 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b806b0 0x564999b78690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.525 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x564999b77b70 con 0x564999b806b0 2021-11-21T22:02:14.526 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564999b79930 con 0x564999b7dda0 2021-11-21T22:02:14.526 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564999b99160 con 0x564999b7b150 2021-11-21T22:02:14.526 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b7b150 0x564999b77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.526 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b1d25700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.527 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b7b150 0x564999b77cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.119:37264/0 (socket says 172.21.2.119:37264) 2021-11-21T22:02:14.527 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 -- 172.21.2.119:0/2456278775 learned_addr learned my addr 172.21.2.119:0/2456278775 (peer_addr_for_me v2:172.21.2.119:0/0) 2021-11-21T22:02:14.527 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2d27700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b806b0 0x564999b78690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.527 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 -- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 msgr2=0x564999b781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:14.527 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.528 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 -- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b806b0 msgr2=0x564999b78690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:14.528 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b806b0 0x564999b78690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:14.528 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 -- 172.21.2.119:0/2456278775 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a400a040 con 0x564999b7b150 2021-11-21T22:02:14.528 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2d27700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x564999b806b0 0x564999b78690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:02:14.529 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b1d25700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x564999b7dda0 0x564999b781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:02:14.529 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b2526700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b7b150 0x564999b77cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x564999b76350 tx=0x7f88a4001870).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:14.529 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a37fe700 1 -- 172.21.2.119:0/2456278775 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a40109b0 con 0x564999b7b150 2021-11-21T22:02:14.529 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a37fe700 1 -- 172.21.2.119:0/2456278775 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4010b10 con 0x564999b7b150 2021-11-21T22:02:14.530 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564999a60fe0 con 0x564999b7b150 2021-11-21T22:02:14.530 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a37fe700 1 -- 172.21.2.119:0/2456278775 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a4010e70 con 0x564999b7b150 2021-11-21T22:02:14.530 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564999b99510 con 0x564999b7b150 2021-11-21T22:02:14.530 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a37fe700 1 -- 172.21.2.119:0/2456278775 <== mon.2 v2:172.21.2.117:3301/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f88a402c070 con 0x564999b7b150 2021-11-21T22:02:14.531 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.513+0000 7f88a37fe700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8894052190 0x7f8894054650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.531 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.517+0000 7f88a37fe700 1 -- 172.21.2.119:0/2456278775 <== mon.2 v2:172.21.2.117:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f88a406bf40 con 0x564999b7b150 2021-11-21T22:02:14.531 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.517+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x564999b9a320 0x564999b9c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:14.531 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.517+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 --> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x564999b9cca0 con 0x564999b9a320 2021-11-21T22:02:14.532 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.517+0000 7f88b1d25700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8894052190 0x7f8894054650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.532 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.517+0000 7f88b2d27700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x564999b9a320 0x564999b9c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:14.532 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.517+0000 7f88b1d25700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8894052190 0x7f8894054650 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f88a8019000 tx=0x7f88a8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:14.532 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:14.517+0000 7f88b2d27700 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x564999b9a320 0x564999b9c710 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:16.233 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.233+0000 7f88a37fe700 1 -- 172.21.2.119:0/2456278775 <== osd.0 v2:172.21.2.116:6800/14707 1 ==== osd_map(128..128 src has 1..128) v4 ==== 320+0+0 (crc 0 0 0) 0x7f88ac012330 con 0x564999b9a320 2021-11-21T22:02:16.237 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.233+0000 7f88b2d27700 1 -- 172.21.2.119:0/2456278775 <== osd.0 v2:172.21.2.116:6800/14707 2 ==== osd_op_reply(1 key [writefull 0~617] v128'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f88ac0124f0 con 0x564999b9a320 2021-11-21T22:02:16.237 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.233+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x564999b9a320 msgr2=0x564999b9c710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.237 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.233+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x564999b9a320 0x564999b9c710 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.238 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8894052190 msgr2=0x7f8894054650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.239 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8894052190 0x7f8894054650 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f88a8019000 tx=0x7f88a8015000).stop 2021-11-21T22:02:16.239 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b7b150 msgr2=0x564999b77cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.239 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 --2- 172.21.2.119:0/2456278775 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x564999b7b150 0x564999b77cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x564999b76350 tx=0x7f88a4001870).stop 2021-11-21T22:02:16.239 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b2d27700 1 -- 172.21.2.119:0/2456278775 reap_dead start 2021-11-21T22:02:16.240 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 shutdown_connections 2021-11-21T22:02:16.240 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 >> 172.21.2.119:0/2456278775 conn(0x564999b70a30 msgr2=0x564999ae9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:16.240 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 shutdown_connections 2021-11-21T22:02:16.240 INFO:teuthology.orchestra.run.gibba019.stderr:2021-11-21T22:02:16.237+0000 7f88b40e8b80 1 -- 172.21.2.119:0/2456278775 wait complete. 2021-11-21T22:02:16.246 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2021-11-21T22:02:16.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f43649f6700 1 Processor -- start 2021-11-21T22:02:16.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f43649f6700 1 -- start start 2021-11-21T22:02:16.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f43649f6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600b66c0 0x7f43600b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f43649f6700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600b6aa0 con 0x7f43600b5440 2021-11-21T22:02:16.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f43649f6700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600a9420 con 0x7f43600b09c0 2021-11-21T22:02:16.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f43649f6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43600a96c0 con 0x7f43600b66c0 2021-11-21T22:02:16.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f43600b5440 0x7f436005bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36250/0 (socket says 172.21.2.116:36250) 2021-11-21T22:02:16.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 -- 172.21.2.116:0/312111070 learned_addr learned my addr 172.21.2.116:0/312111070 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:16.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064757893 0 0) 0x7f43600b6aa0 con 0x7f43600b5440 2021-11-21T22:02:16.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4340003260 con 0x7f43600b5440 2021-11-21T22:02:16.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435ed9d700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600b66c0 0x7f43600b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:16.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3822217767 0 0) 0x7f4340003260 con 0x7f43600b5440 2021-11-21T22:02:16.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600b66c0 msgr2=0x7f43600b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:16.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600b66c0 0x7f43600b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 >> v1:172.21.2.117:6789/0 conn(0x7f43600b09c0 legacy=0x7f4360143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:16.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600b4a20 con 0x7f43600b5440 2021-11-21T22:02:16.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f4348001680 con 0x7f43600b5440 2021-11-21T22:02:16.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 >> v1:172.21.2.116:6789/0 conn(0x7f43600b5440 legacy=0x7f436005bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600b66c0 0x7f4340003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4340003e10 0x7f4340004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340004b10 0x7f4340004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600a96c0 con 0x7f4340003e10 2021-11-21T22:02:16.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434000b960 con 0x7f4340004b10 2021-11-21T22:02:16.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434000bb20 con 0x7f43600b66c0 2021-11-21T22:02:16.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4340003e10 0x7f4340004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:16.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 -- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600b66c0 msgr2=0x7f4340003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:16.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600b66c0 0x7f4340003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:16.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 -- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340004b10 msgr2=0x7f4340004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:16.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340004b10 0x7f4340004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:16.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 -- 172.21.2.116:0/312111070 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340003260 con 0x7f4340003e10 2021-11-21T22:02:16.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435659c700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4340003e10 0x7f4340004500 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f4360060620 tx=0x7f43600b3680).ready entity=mon.0 client_cookie=32de2967e368e78a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:16.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43480070d0 con 0x7f4340003e10 2021-11-21T22:02:16.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348007230 con 0x7f4340003e10 2021-11-21T22:02:16.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.294+0000 7f435e59c700 1 -- 172.21.2.116:0/312111070 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f434800c8c0 con 0x7f4340003e10 2021-11-21T22:02:16.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/312111070 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4340003e10 msgr2=0x7f4340004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 --2- 172.21.2.116:0/312111070 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4340003e10 0x7f4340004500 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f4360060620 tx=0x7f43600b3680).stop 2021-11-21T22:02:16.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435f59e700 1 -- 172.21.2.116:0/312111070 reap_dead start 2021-11-21T22:02:16.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/312111070 shutdown_connections 2021-11-21T22:02:16.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/312111070 >> 172.21.2.116:0/312111070 conn(0x7f43600b9c50 msgr2=0x7f43600b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:16.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/312111070 shutdown_connections 2021-11-21T22:02:16.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/312111070 wait complete. 2021-11-21T22:02:16.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 Processor -- start 2021-11-21T22:02:16.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- start start 2021-11-21T22:02:16.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340003e10 0x7f4360143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43600ab200 0x7f43600ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600abb70 0x7f4360145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600a9a80 con 0x7f43600ab200 2021-11-21T22:02:16.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600aa0b0 con 0x7f4340003e10 2021-11-21T22:02:16.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43600a9cb0 con 0x7f43600abb70 2021-11-21T22:02:16.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43600ab200 0x7f43600ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:16.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43600ab200 0x7f43600ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52536/0 (socket says 172.21.2.116:52536) 2021-11-21T22:02:16.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 -- 172.21.2.116:0/2355308366 learned_addr learned my addr 172.21.2.116:0/2355308366 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:16.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435ed9d700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340003e10 0x7f4360143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:16.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435f59e700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600abb70 0x7f4360145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:16.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 -- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600abb70 msgr2=0x7f4360145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600abb70 0x7f4360145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 -- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340003e10 msgr2=0x7f4360143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340003e10 0x7f4360143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 -- 172.21.2.116:0/2355308366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600aa310 con 0x7f43600ab200 2021-11-21T22:02:16.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435659c700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43600ab200 0x7f43600ab680 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f4360060620 tx=0x7f4348013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:16.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435cd99700 1 -- 172.21.2.116:0/2355308366 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43480070d0 con 0x7f43600ab200 2021-11-21T22:02:16.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435cd99700 1 -- 172.21.2.116:0/2355308366 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43480019a0 con 0x7f43600ab200 2021-11-21T22:02:16.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43600aa5b0 con 0x7f43600ab200 2021-11-21T22:02:16.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435cd99700 1 -- 172.21.2.116:0/2355308366 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4348001ba0 con 0x7f43600ab200 2021-11-21T22:02:16.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4360147d20 con 0x7f43600ab200 2021-11-21T22:02:16.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43600aa740 con 0x7f43600ab200 2021-11-21T22:02:16.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435cd99700 1 -- 172.21.2.116:0/2355308366 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4348027070 con 0x7f43600ab200 2021-11-21T22:02:16.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.298+0000 7f435cd99700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4344053720 0x7f4344055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:16.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.302+0000 7f435cd99700 1 -- 172.21.2.116:0/2355308366 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f434806bfc0 con 0x7f43600ab200 2021-11-21T22:02:16.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.302+0000 7f435ed9d700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4344053720 0x7f4344055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:16.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.302+0000 7f435ed9d700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4344053720 0x7f4344055be0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f435000c000 tx=0x7f4350008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:16.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.302+0000 7f435cd99700 1 -- 172.21.2.116:0/2355308366 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43600aa740 con 0x7f43600ab200 2021-11-21T22:02:16.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f43600aa740 con 0x7f43600ab200 2021-11-21T22:02:16.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f435cd99700 1 -- 172.21.2.116:0/2355308366 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1912 (secure 0 0 0) 0x7f43600aa740 con 0x7f43600ab200 2021-11-21T22:02:16.422 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:16.423 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:16.423 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:16.423 INFO:teuthology.orchestra.run.gibba016.stdout: "total_bytes": 721202184192, 2021-11-21T22:02:16.423 INFO:teuthology.orchestra.run.gibba016.stdout: "total_avail_bytes": 714955563008, 2021-11-21T22:02:16.423 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_bytes": 6246420480, 2021-11-21T22:02:16.424 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_bytes": 6246621184, 2021-11-21T22:02:16.424 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_ratio": 0.0086614005267620087, 2021-11-21T22:02:16.424 INFO:teuthology.orchestra.run.gibba016.stdout: "num_osds": 8, 2021-11-21T22:02:16.424 INFO:teuthology.orchestra.run.gibba016.stdout: "num_per_pool_osds": 0, 2021-11-21T22:02:16.425 INFO:teuthology.orchestra.run.gibba016.stdout: "num_per_pool_omap_osds": 0 2021-11-21T22:02:16.425 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:16.425 INFO:teuthology.orchestra.run.gibba016.stdout: "stats_by_class": { 2021-11-21T22:02:16.425 INFO:teuthology.orchestra.run.gibba016.stdout: "ssd": { 2021-11-21T22:02:16.425 INFO:teuthology.orchestra.run.gibba016.stdout: "total_bytes": 721202184192, 2021-11-21T22:02:16.426 INFO:teuthology.orchestra.run.gibba016.stdout: "total_avail_bytes": 714955563008, 2021-11-21T22:02:16.426 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_bytes": 6246420480, 2021-11-21T22:02:16.426 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_bytes": 6246621184, 2021-11-21T22:02:16.426 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_ratio": 0.0086614005267620087 2021-11-21T22:02:16.427 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:16.427 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:16.427 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:02:16.427 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:16.428 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "device_health_metrics", 2021-11-21T22:02:16.428 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 1, 2021-11-21T22:02:16.428 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:16.428 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:02:16.429 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 2, 2021-11-21T22:02:16.429 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:02:16.429 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:02:16.429 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:02:16.429 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447382016 2021-11-21T22:02:16.430 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:16.430 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:16.430 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:16.430 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "rbd", 2021-11-21T22:02:16.431 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 2, 2021-11-21T22:02:16.431 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:16.431 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:02:16.431 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 0, 2021-11-21T22:02:16.432 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:02:16.432 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:02:16.432 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:02:16.432 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447382016 2021-11-21T22:02:16.432 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:16.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:16.433 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:16.433 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "cephfs_data", 2021-11-21T22:02:16.433 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 22, 2021-11-21T22:02:16.434 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:16.434 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:02:16.434 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 0, 2021-11-21T22:02:16.434 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:02:16.435 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:02:16.435 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:02:16.435 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447382016 2021-11-21T22:02:16.435 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:16.435 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:16.436 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:16.436 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "cephfs_metadata", 2021-11-21T22:02:16.436 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 23, 2021-11-21T22:02:16.436 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:16.437 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:02:16.437 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 0, 2021-11-21T22:02:16.437 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:02:16.437 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:02:16.437 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:02:16.438 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339447382016 2021-11-21T22:02:16.438 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:16.438 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:16.438 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:16.439 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:16.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4344053720 msgr2=0x7f4344055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4344053720 0x7f4344055be0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f435000c000 tx=0x7f4350008040).stop 2021-11-21T22:02:16.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43600ab200 msgr2=0x7f43600ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:16.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43600ab200 0x7f43600ab680 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f4360060620 tx=0x7f4348013000).stop 2021-11-21T22:02:16.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 shutdown_connections 2021-11-21T22:02:16.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43600abb70 0x7f4360145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4344053720 0x7f4344055be0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f43600ab200 0x7f43600ab680 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 --2- 172.21.2.116:0/2355308366 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4340003e10 0x7f4360143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:16.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 >> 172.21.2.116:0/2355308366 conn(0x7f43600b9c50 msgr2=0x7f4360142090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:16.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 shutdown_connections 2021-11-21T22:02:16.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:16.418+0000 7f43649f6700 1 -- 172.21.2.116:0/2355308366 wait complete. 2021-11-21T22:02:16.443 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2021-11-21T22:02:21.445 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2021-11-21T22:02:21.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.494+0000 7f192359e700 1 Processor -- start 2021-11-21T22:02:21.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.494+0000 7f192359e700 1 -- start start 2021-11-21T22:02:21.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.494+0000 7f192359e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f19240b5440 0x7f19240b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.494+0000 7f192359e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b09c0 0x7f1924143040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.494+0000 7f192359e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240b6aa0 con 0x7f19240b66c0 2021-11-21T22:02:21.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240a9420 con 0x7f19240b5440 2021-11-21T22:02:21.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240a96c0 con 0x7f19240b09c0 2021-11-21T22:02:21.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f19240b5440 0x7f19240b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f19240b5440 0x7f19240b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52544/0 (socket says 172.21.2.116:52544) 2021-11-21T22:02:21.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 -- 172.21.2.116:0/3550990733 learned_addr learned my addr 172.21.2.116:0/3550990733 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:21.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 -- 172.21.2.116:0/3550990733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b09c0 msgr2=0x7f1924143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 --2- 172.21.2.116:0/3550990733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b09c0 0x7f1924143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 -- 172.21.2.116:0/3550990733 >> v1:172.21.2.117:6789/0 conn(0x7f19240b66c0 legacy=0x7f192405bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 -- 172.21.2.116:0/3550990733 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240b4a20 con 0x7f19240b5440 2021-11-21T22:02:21.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1921d9b700 1 --2- 172.21.2.116:0/3550990733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b5440 0x7f19240b4ec0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f1924061fc0 tx=0x7f1918006eb0).ready entity=mon.0 client_cookie=a308e385ad01c983 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192159a700 1 -- 172.21.2.116:0/3550990733 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191800d570 con 0x7f19240b5440 2021-11-21T22:02:21.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192159a700 1 -- 172.21.2.116:0/3550990733 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918000e90 con 0x7f19240b5440 2021-11-21T22:02:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192159a700 1 -- 172.21.2.116:0/3550990733 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191800d8c0 con 0x7f19240b5440 2021-11-21T22:02:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- 172.21.2.116:0/3550990733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b5440 msgr2=0x7f19240b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 --2- 172.21.2.116:0/3550990733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b5440 0x7f19240b4ec0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f1924061fc0 tx=0x7f1918006eb0).stop 2021-11-21T22:02:21.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- 172.21.2.116:0/3550990733 shutdown_connections 2021-11-21T22:02:21.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 --2- 172.21.2.116:0/3550990733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b09c0 0x7f1924143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 --2- 172.21.2.116:0/3550990733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b5440 0x7f19240b4ec0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- 172.21.2.116:0/3550990733 >> 172.21.2.116:0/3550990733 conn(0x7f19240b9c50 msgr2=0x7f19240b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- 172.21.2.116:0/3550990733 shutdown_connections 2021-11-21T22:02:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- 172.21.2.116:0/3550990733 wait complete. 2021-11-21T22:02:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 Processor -- start 2021-11-21T22:02:21.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- start start 2021-11-21T22:02:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19240b09c0 0x7f19241465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b5440 0x7f192413e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b66c0 0x7f192413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19241472d0 con 0x7f19240b66c0 2021-11-21T22:02:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240ac400 con 0x7f19240b09c0 2021-11-21T22:02:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192413ee30 con 0x7f19240b5440 2021-11-21T22:02:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b66c0 0x7f192413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b66c0 0x7f192413e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52548/0 (socket says 172.21.2.116:52548) 2021-11-21T22:02:21.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 -- 172.21.2.116:0/1354643327 learned_addr learned my addr 172.21.2.116:0/1354643327 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:21.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 -- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b5440 msgr2=0x7f192413e450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:21.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b5440 0x7f192413e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 -- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19240b09c0 msgr2=0x7f19241465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19240b09c0 0x7f19241465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 -- 172.21.2.116:0/1354643327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191800b1e0 con 0x7f19240b66c0 2021-11-21T22:02:21.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f1922d9d700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b66c0 0x7f192413e940 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f191c0032f0 tx=0x7f191c009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f19137fe700 1 -- 172.21.2.116:0/1354643327 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191c00d580 con 0x7f19240b66c0 2021-11-21T22:02:21.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f19137fe700 1 -- 172.21.2.116:0/1354643327 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c00d6e0 con 0x7f19240b66c0 2021-11-21T22:02:21.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240afb40 con 0x7f19240b66c0 2021-11-21T22:02:21.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19240afe30 con 0x7f19240b66c0 2021-11-21T22:02:21.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.498+0000 7f19137fe700 1 -- 172.21.2.116:0/1354643327 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191c00da40 con 0x7f19240b66c0 2021-11-21T22:02:21.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.502+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19240b0160 con 0x7f19240b66c0 2021-11-21T22:02:21.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.502+0000 7f19137fe700 1 -- 172.21.2.116:0/1354643327 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f191c00dba0 con 0x7f19240b66c0 2021-11-21T22:02:21.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.502+0000 7f19137fe700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19000536f0 0x7f1900055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.502+0000 7f19137fe700 1 -- 172.21.2.116:0/1354643327 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f191c06bdc0 con 0x7f19240b66c0 2021-11-21T22:02:21.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.502+0000 7f192259c700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19000536f0 0x7f1900055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.502+0000 7f192259c700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19000536f0 0x7f1900055bb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f1914002ef0 tx=0x7f1914006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.502+0000 7f19137fe700 1 -- 172.21.2.116:0/1354643327 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f191c042e10 con 0x7f19240b66c0 2021-11-21T22:02:21.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.614+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f19240ab700 con 0x7f19240b66c0 2021-11-21T22:02:21.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.614+0000 7f19137fe700 1 -- 172.21.2.116:0/1354643327 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1937 (secure 0 0 0) 0x7f191c046430 con 0x7f19240b66c0 2021-11-21T22:02:21.620 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:21.621 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:21.621 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:21.621 INFO:teuthology.orchestra.run.gibba016.stdout: "total_bytes": 721202184192, 2021-11-21T22:02:21.621 INFO:teuthology.orchestra.run.gibba016.stdout: "total_avail_bytes": 714954035200, 2021-11-21T22:02:21.622 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_bytes": 6247329792, 2021-11-21T22:02:21.622 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_bytes": 6248148992, 2021-11-21T22:02:21.622 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_ratio": 0.0086635192856192589, 2021-11-21T22:02:21.622 INFO:teuthology.orchestra.run.gibba016.stdout: "num_osds": 8, 2021-11-21T22:02:21.622 INFO:teuthology.orchestra.run.gibba016.stdout: "num_per_pool_osds": 0, 2021-11-21T22:02:21.623 INFO:teuthology.orchestra.run.gibba016.stdout: "num_per_pool_omap_osds": 0 2021-11-21T22:02:21.623 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:21.623 INFO:teuthology.orchestra.run.gibba016.stdout: "stats_by_class": { 2021-11-21T22:02:21.623 INFO:teuthology.orchestra.run.gibba016.stdout: "ssd": { 2021-11-21T22:02:21.624 INFO:teuthology.orchestra.run.gibba016.stdout: "total_bytes": 721202184192, 2021-11-21T22:02:21.624 INFO:teuthology.orchestra.run.gibba016.stdout: "total_avail_bytes": 714954035200, 2021-11-21T22:02:21.624 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_bytes": 6247329792, 2021-11-21T22:02:21.624 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_bytes": 6248148992, 2021-11-21T22:02:21.624 INFO:teuthology.orchestra.run.gibba016.stdout: "total_used_raw_ratio": 0.0086635192856192589 2021-11-21T22:02:21.625 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:21.625 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:21.625 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:02:21.625 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:21.626 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "device_health_metrics", 2021-11-21T22:02:21.626 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 1, 2021-11-21T22:02:21.626 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:21.626 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:02:21.626 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 2, 2021-11-21T22:02:21.627 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:02:21.627 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:02:21.627 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:02:21.627 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339446661120 2021-11-21T22:02:21.628 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:21.628 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:21.628 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:21.628 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "rbd", 2021-11-21T22:02:21.628 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 2, 2021-11-21T22:02:21.629 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:21.629 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:02:21.629 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 0, 2021-11-21T22:02:21.629 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:02:21.630 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:02:21.630 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:02:21.630 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339446661120 2021-11-21T22:02:21.630 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:21.630 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:21.631 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:21.631 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "cephfs_data", 2021-11-21T22:02:21.631 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 22, 2021-11-21T22:02:21.631 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:21.632 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 0, 2021-11-21T22:02:21.632 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 0, 2021-11-21T22:02:21.632 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 0, 2021-11-21T22:02:21.632 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 0, 2021-11-21T22:02:21.632 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 0, 2021-11-21T22:02:21.633 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339446661120 2021-11-21T22:02:21.633 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:21.633 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:21.633 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:21.634 INFO:teuthology.orchestra.run.gibba016.stdout: "name": "cephfs_metadata", 2021-11-21T22:02:21.634 INFO:teuthology.orchestra.run.gibba016.stdout: "id": 23, 2021-11-21T22:02:21.634 INFO:teuthology.orchestra.run.gibba016.stdout: "stats": { 2021-11-21T22:02:21.634 INFO:teuthology.orchestra.run.gibba016.stdout: "stored": 617, 2021-11-21T22:02:21.634 INFO:teuthology.orchestra.run.gibba016.stdout: "objects": 1, 2021-11-21T22:02:21.635 INFO:teuthology.orchestra.run.gibba016.stdout: "kb_used": 1, 2021-11-21T22:02:21.635 INFO:teuthology.orchestra.run.gibba016.stdout: "bytes_used": 617, 2021-11-21T22:02:21.635 INFO:teuthology.orchestra.run.gibba016.stdout: "percent_used": 9.0883200964952948e-10, 2021-11-21T22:02:21.635 INFO:teuthology.orchestra.run.gibba016.stdout: "max_avail": 339446661120 2021-11-21T22:02:21.636 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:21.636 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:21.636 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:21.636 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:21.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19000536f0 msgr2=0x7f1900055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19000536f0 0x7f1900055bb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f1914002ef0 tx=0x7f1914006910).stop 2021-11-21T22:02:21.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b66c0 msgr2=0x7f192413e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b66c0 0x7f192413e940 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f191c0032f0 tx=0x7f191c009220).stop 2021-11-21T22:02:21.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 shutdown_connections 2021-11-21T22:02:21.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f19000536f0 0x7f1900055bb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f19240b66c0 0x7f192413e940 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f19240b5440 0x7f192413e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 --2- 172.21.2.116:0/1354643327 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f19240b09c0 0x7f19241465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 >> 172.21.2.116:0/1354643327 conn(0x7f19240b9c50 msgr2=0x7f192413f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:21.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 shutdown_connections 2021-11-21T22:02:21.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.618+0000 7f192359e700 1 -- 172.21.2.116:0/1354643327 wait complete. 2021-11-21T22:02:21.641 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2021-11-21T22:02:21.641 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2021-11-21T22:02:21.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 Processor -- start 2021-11-21T22:02:21.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- start start 2021-11-21T22:02:21.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f41ac146f50 0x7f41ac13fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f41ac05f9f0 0x7f41ac13ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ac0b2fe0 con 0x7f41ac0b2c00 2021-11-21T22:02:21.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ac0b24e0 con 0x7f41ac146f50 2021-11-21T22:02:21.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ac140b00 con 0x7f41ac05f9f0 2021-11-21T22:02:21.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b0979700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f41ac0b2c00 0x7f41ac142dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36278/0 (socket says 172.21.2.116:36278) 2021-11-21T22:02:21.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b0979700 1 -- 172.21.2.116:0/3781277020 learned_addr learned my addr 172.21.2.116:0/3781277020 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:21.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41abfff700 1 --2- 172.21.2.116:0/3781277020 >> v2:172.21.2.116:3300/0 conn(0x7f41ac146f50 0x7f41ac13fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41ab7fe700 1 -- 172.21.2.116:0/3781277020 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376639075 0 0) 0x7f41ac0b2fe0 con 0x7f41ac0b2c00 2021-11-21T22:02:21.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41ab7fe700 1 -- 172.21.2.116:0/3781277020 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f418c003260 con 0x7f41ac0b2c00 2021-11-21T22:02:21.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41abfff700 1 -- 172.21.2.116:0/3781277020 >> v2:172.21.2.117:3300/0 conn(0x7f41ac05f9f0 msgr2=0x7f41ac13ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:21.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41abfff700 1 --2- 172.21.2.116:0/3781277020 >> v2:172.21.2.117:3300/0 conn(0x7f41ac05f9f0 0x7f41ac13ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41abfff700 1 -- 172.21.2.116:0/3781277020 >> v1:172.21.2.116:6789/0 conn(0x7f41ac0b2c00 legacy=0x7f41ac142dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41abfff700 1 -- 172.21.2.116:0/3781277020 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac146310 con 0x7f41ac146f50 2021-11-21T22:02:21.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41abfff700 1 --2- 172.21.2.116:0/3781277020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac146f50 0x7f41ac13fa50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f4194002f70 tx=0x7f4194006df0).ready entity=mon.0 client_cookie=e23ebf1501eeb547 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41ab7fe700 1 -- 172.21.2.116:0/3781277020 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419400d570 con 0x7f41ac146f50 2021-11-21T22:02:21.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41ab7fe700 1 -- 172.21.2.116:0/3781277020 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194000e90 con 0x7f41ac146f50 2021-11-21T22:02:21.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41ab7fe700 1 -- 172.21.2.116:0/3781277020 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419400d8c0 con 0x7f41ac146f50 2021-11-21T22:02:21.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3781277020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac146f50 msgr2=0x7f41ac13fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 --2- 172.21.2.116:0/3781277020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac146f50 0x7f41ac13fa50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f4194002f70 tx=0x7f4194006df0).stop 2021-11-21T22:02:21.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3781277020 shutdown_connections 2021-11-21T22:02:21.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 --2- 172.21.2.116:0/3781277020 >> v2:172.21.2.117:3300/0 conn(0x7f41ac05f9f0 0x7f41ac13ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 --2- 172.21.2.116:0/3781277020 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac146f50 0x7f41ac13fa50 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3781277020 >> 172.21.2.116:0/3781277020 conn(0x7f41ac063bb0 msgr2=0x7f41ac063fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:21.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3781277020 shutdown_connections 2021-11-21T22:02:21.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3781277020 wait complete. 2021-11-21T22:02:21.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 Processor -- start 2021-11-21T22:02:21.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.690+0000 7f41b2bdd700 1 -- start start 2021-11-21T22:02:21.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41ac05f9f0 0x7f41ac1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac0b2c00 0x7f41ac13a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f41ac146f50 0x7f41ac13a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0b5d10 con 0x7f41ac0b2c00 2021-11-21T22:02:21.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac067cd0 con 0x7f41ac146f50 2021-11-21T22:02:21.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0b4d40 con 0x7f41ac05f9f0 2021-11-21T22:02:21.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac0b2c00 0x7f41ac13a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac0b2c00 0x7f41ac13a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52562/0 (socket says 172.21.2.116:52562) 2021-11-21T22:02:21.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 -- 172.21.2.116:0/3589938260 learned_addr learned my addr 172.21.2.116:0/3589938260 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:21.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b117a700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f41ac146f50 0x7f41ac13a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b0979700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41ac05f9f0 0x7f41ac1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 -- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41ac05f9f0 msgr2=0x7f41ac1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f41ac05f9f0 0x7f41ac1426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 -- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f41ac146f50 msgr2=0x7f41ac13a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f41ac146f50 0x7f41ac13a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 -- 172.21.2.116:0/3589938260 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419400b180 con 0x7f41ac0b2c00 2021-11-21T22:02:21.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41abfff700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f41ac0b2c00 0x7f41ac13a270 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f4194002f70 tx=0x7f4194014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41a9ffb700 1 -- 172.21.2.116:0/3589938260 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4194016070 con 0x7f41ac0b2c00 2021-11-21T22:02:21.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41a9ffb700 1 -- 172.21.2.116:0/3589938260 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194013a00 con 0x7f41ac0b2c00 2021-11-21T22:02:21.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41a9ffb700 1 -- 172.21.2.116:0/3589938260 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4194013d60 con 0x7f41ac0b2c00 2021-11-21T22:02:21.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3589938260 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac0b5f40 con 0x7f41ac0b2c00 2021-11-21T22:02:21.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3589938260 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ac140fa0 con 0x7f41ac0b2c00 2021-11-21T22:02:21.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3589938260 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ac143390 con 0x7f41ac0b2c00 2021-11-21T22:02:21.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41a9ffb700 1 -- 172.21.2.116:0/3589938260 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4194025030 con 0x7f41ac0b2c00 2021-11-21T22:02:21.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41a9ffb700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4190053720 0x7f4190055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41a9ffb700 1 -- 172.21.2.116:0/3589938260 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f419406cab0 con 0x7f41ac0b2c00 2021-11-21T22:02:21.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b0979700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4190053720 0x7f4190055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.694+0000 7f41b0979700 1 --2- 172.21.2.116:0/3589938260 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4190053720 0x7f4190055be0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f419c008000 tx=0x7f419c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.698+0000 7f41a9ffb700 1 -- 172.21.2.116:0/3589938260 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4194043b00 con 0x7f41ac0b2c00 2021-11-21T22:02:21.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.806+0000 7f41b2bdd700 1 -- 172.21.2.116:0/3589938260 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f41ac0573d0 con 0x7f41ac0b2c00 2021-11-21T22:02:21.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.806+0000 7f41a9ffb700 1 -- 172.21.2.116:0/3589938260 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v57) v1 ==== 209+0+0 (secure 0 0 0) 0x7f4194023020 con 0x7f41ac0b2c00 2021-11-21T22:02:21.812 INFO:teuthology.orchestra.run.gibba016.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2021-11-21T22:02:21.813 DEBUG:teuthology.orchestra.run:got remote process result: 22 2021-11-21T22:02:21.815 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:02:21.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.862+0000 7fefbfc0d700 1 Processor -- start 2021-11-21T22:02:21.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.862+0000 7fefbfc0d700 1 -- start start 2021-11-21T22:02:21.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80ad780 0x7fefb805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fefb80affc0 0x7fefb8143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fefb80af7e0 0x7fefb80b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb80adb60 con 0x7fefb80affc0 2021-11-21T22:02:21.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb8143160 con 0x7fefb80af7e0 2021-11-21T22:02:21.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80b0ce0 con 0x7fefb80ad780 2021-11-21T22:02:21.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fefb80affc0 0x7fefb8143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fefb80affc0 0x7fefb8143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52568/0 (socket says 172.21.2.116:52568) 2021-11-21T22:02:21.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/3004129568 learned_addr learned my addr 172.21.2.116:0/3004129568 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:21.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/3004129568 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80ad780 msgr2=0x7fefb805f730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:21.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- 172.21.2.116:0/3004129568 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80ad780 0x7fefb805f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/3004129568 >> v2:172.21.2.117:3300/0 conn(0x7fefb80af7e0 msgr2=0x7fefb80b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:21.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- 172.21.2.116:0/3004129568 >> v2:172.21.2.117:3300/0 conn(0x7fefb80af7e0 0x7fefb80b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/3004129568 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80b0a40 con 0x7fefb80affc0 2021-11-21T22:02:21.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- 172.21.2.116:0/3004129568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80affc0 0x7fefb8143860 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fefb4002f70 tx=0x7fefb4006f40).ready entity=mon.0 client_cookie=b5767f17974c8a9e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbc9a7700 1 -- 172.21.2.116:0/3004129568 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb400dc70 con 0x7fefb80affc0 2021-11-21T22:02:21.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbc9a7700 1 -- 172.21.2.116:0/3004129568 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb4000e90 con 0x7fefb80affc0 2021-11-21T22:02:21.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbc9a7700 1 -- 172.21.2.116:0/3004129568 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb400be90 con 0x7fefb80affc0 2021-11-21T22:02:21.878 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- 172.21.2.116:0/3004129568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80affc0 msgr2=0x7fefb8143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/3004129568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80affc0 0x7fefb8143860 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fefb4002f70 tx=0x7fefb4006f40).stop 2021-11-21T22:02:21.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- 172.21.2.116:0/3004129568 shutdown_connections 2021-11-21T22:02:21.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/3004129568 >> v2:172.21.2.117:3300/0 conn(0x7fefb80af7e0 0x7fefb80b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/3004129568 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80affc0 0x7fefb8143860 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/3004129568 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80ad780 0x7fefb805f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- 172.21.2.116:0/3004129568 >> 172.21.2.116:0/3004129568 conn(0x7fefb80b6000 msgr2=0x7fefb80b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:21.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- 172.21.2.116:0/3004129568 shutdown_connections 2021-11-21T22:02:21.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- 172.21.2.116:0/3004129568 wait complete. 2021-11-21T22:02:21.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 Processor -- start 2021-11-21T22:02:21.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- start start 2021-11-21T22:02:21.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fefb80ad780 0x7fefb805c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80af7e0 0x7fefb80ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80affc0 0x7fefb8156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb8142270 con 0x7fefb80af7e0 2021-11-21T22:02:21.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb81427f0 con 0x7fefb80ad780 2021-11-21T22:02:21.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbfc0d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80aaef0 con 0x7fefb80affc0 2021-11-21T22:02:21.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80af7e0 0x7fefb80ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80af7e0 0x7fefb80ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52578/0 (socket says 172.21.2.116:52578) 2021-11-21T22:02:21.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/4031793206 learned_addr learned my addr 172.21.2.116:0/4031793206 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:21.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80affc0 msgr2=0x7fefb8156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80affc0 0x7fefb8156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fefb80ad780 msgr2=0x7fefb805c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fefb80ad780 0x7fefb805c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.866+0000 7fefbd1a8700 1 -- 172.21.2.116:0/4031793206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb400b460 con 0x7fefb80af7e0 2021-11-21T22:02:21.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefbd1a8700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80af7e0 0x7fefb80ac490 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fefb4002f70 tx=0x7fefb4006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefaaffd700 1 -- 172.21.2.116:0/4031793206 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb401d070 con 0x7fefb80af7e0 2021-11-21T22:02:21.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefaaffd700 1 -- 172.21.2.116:0/4031793206 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb4022410 con 0x7fefb80af7e0 2021-11-21T22:02:21.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefaaffd700 1 -- 172.21.2.116:0/4031793206 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb4022770 con 0x7fefb80af7e0 2021-11-21T22:02:21.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb81424a0 con 0x7fefb80af7e0 2021-11-21T22:02:21.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb8148040 con 0x7fefb80af7e0 2021-11-21T22:02:21.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb8064aa0 con 0x7fefb80af7e0 2021-11-21T22:02:21.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefaaffd700 1 -- 172.21.2.116:0/4031793206 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fefb4029070 con 0x7fefb80af7e0 2021-11-21T22:02:21.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefaaffd700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef98053660 0x7fef98055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:21.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefaaffd700 1 -- 172.21.2.116:0/4031793206 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6071+0+0 (secure 0 0 0) 0x7fefb406ceb0 con 0x7fefb80af7e0 2021-11-21T22:02:21.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefbd9a9700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef98053660 0x7fef98055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:21.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefbd9a9700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef98053660 0x7fef98055b20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fefac002fb0 tx=0x7fefac0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:21.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.870+0000 7fefaaffd700 1 -- 172.21.2.116:0/4031793206 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fefb4043f00 con 0x7fefb80af7e0 2021-11-21T22:02:21.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.982+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fefb81433c0 con 0x7fefb80af7e0 2021-11-21T22:02:21.986 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:02:21.982+0000 7f1e505b4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:02:21.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefaaffd700 1 -- 172.21.2.116:0/4031793206 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v58) v1 ==== 189+0+0 (secure 0 0 0) 0x7fefb4047520 con 0x7fefb80af7e0 2021-11-21T22:02:21.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef98053660 msgr2=0x7fef98055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef98053660 0x7fef98055b20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fefac002fb0 tx=0x7fefac0069b0).stop 2021-11-21T22:02:21.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80af7e0 msgr2=0x7fefb80ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:21.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80af7e0 0x7fefb80ac490 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fefb4002f70 tx=0x7fefb4006f40).stop 2021-11-21T22:02:21.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 shutdown_connections 2021-11-21T22:02:21.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fefb80affc0 0x7fefb8156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef98053660 0x7fef98055b20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fefb80af7e0 0x7fefb80ac490 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:21.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 --2- 172.21.2.116:0/4031793206 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fefb80ad780 0x7fefb805c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:21.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 >> 172.21.2.116:0/4031793206 conn(0x7fefb80b6000 msgr2=0x7fefb80b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:21.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 shutdown_connections 2021-11-21T22:02:21.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:21.986+0000 7fefbfc0d700 1 -- 172.21.2.116:0/4031793206 wait complete. 2021-11-21T22:02:21.997 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 23 and data pool 22 2021-11-21T22:02:21.999 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:02:22.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.046+0000 7f2d0b244700 1 Processor -- start 2021-11-21T22:02:22.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.046+0000 7f2d0b244700 1 -- start start 2021-11-21T22:02:22.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2d040abce0 0x7f2d041478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 0x7f2d040aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d04147e80 con 0x7f2d04144810 2021-11-21T22:02:22.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d040b1490 con 0x7f2d040abce0 2021-11-21T22:02:22.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040b1fd0 con 0x7f2d040ac600 2021-11-21T22:02:22.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d08fe0700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f2d04144810 0x7f2d040b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36306/0 (socket says 172.21.2.116:36306) 2021-11-21T22:02:22.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d08fe0700 1 -- 172.21.2.116:0/943557852 learned_addr learned my addr 172.21.2.116:0/943557852 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:22.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595206035 0 0) 0x7f2d04147e80 con 0x7f2d04144810 2021-11-21T22:02:22.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf0003260 con 0x7f2d04144810 2021-11-21T22:02:22.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 11968035 0 0) 0x7f2cf0003260 con 0x7f2d04144810 2021-11-21T22:02:22.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 msgr2=0x7f2d040aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:22.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 0x7f2d040aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:22.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 >> v2:172.21.2.117:3300/0 conn(0x7f2d040abce0 msgr2=0x7f2d041478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:22.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 --2- 172.21.2.116:0/943557852 >> v2:172.21.2.117:3300/0 conn(0x7f2d040abce0 0x7f2d041478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:22.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d040af5a0 con 0x7f2d04144810 2021-11-21T22:02:22.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f2cf8001660 con 0x7f2d04144810 2021-11-21T22:02:22.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 >> v1:172.21.2.116:6789/0 conn(0x7f2d04144810 legacy=0x7f2d040b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:22.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf00037d0 0x7f2cf0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cf0004500 0x7f2cf0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 0x7f2cf000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040b1490 con 0x7f2cf0004500 2021-11-21T22:02:22.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040b1fd0 con 0x7f2cf00037d0 2021-11-21T22:02:22.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0005300 con 0x7f2d040ac600 2021-11-21T22:02:22.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cf0004500 0x7f2cf0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:22.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d097e1700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 0x7f2cf000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:22.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d08fe0700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf00037d0 0x7f2cf0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:22.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 -- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 msgr2=0x7f2cf000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:22.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 0x7f2cf000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:22.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 -- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf00037d0 msgr2=0x7f2cf0003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:22.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf00037d0 0x7f2cf0003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:22.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 -- 172.21.2.116:0/943557852 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0003260 con 0x7f2cf0004500 2021-11-21T22:02:22.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d08fe0700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf00037d0 0x7f2cf0003bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:22.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d097e1700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d040ac600 0x7f2cf000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:22.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cf0004500 0x7f2cf0004980 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f2cfc0032f0 tx=0x7f2cfc006f90).ready entity=mon.0 client_cookie=2a55fad53c362f4b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:22.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cfc00b460 con 0x7f2cf0004500 2021-11-21T22:02:22.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc00b5c0 con 0x7f2cf0004500 2021-11-21T22:02:22.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d037fe700 1 -- 172.21.2.116:0/943557852 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cfc00b730 con 0x7f2cf0004500 2021-11-21T22:02:22.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- 172.21.2.116:0/943557852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cf0004500 msgr2=0x7f2cf0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:22.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 --2- 172.21.2.116:0/943557852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2cf0004500 0x7f2cf0004980 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f2cfc0032f0 tx=0x7f2cfc006f90).stop 2021-11-21T22:02:22.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d097e1700 1 -- 172.21.2.116:0/943557852 reap_dead start 2021-11-21T22:02:22.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- 172.21.2.116:0/943557852 shutdown_connections 2021-11-21T22:02:22.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- 172.21.2.116:0/943557852 >> 172.21.2.116:0/943557852 conn(0x7f2d0405bd20 msgr2=0x7f2d0405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:22.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- 172.21.2.116:0/943557852 shutdown_connections 2021-11-21T22:02:22.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- 172.21.2.116:0/943557852 wait complete. 2021-11-21T22:02:22.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 Processor -- start 2021-11-21T22:02:22.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- start start 2021-11-21T22:02:22.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf0004500 0x7f2d041435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d040b2e90 0x7f2d04064470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d04064960 0x7f2d04144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040b3b50 con 0x7f2d040b2e90 2021-11-21T22:02:22.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04064de0 con 0x7f2cf0004500 2021-11-21T22:02:22.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d0b244700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040b4310 con 0x7f2d04064960 2021-11-21T22:02:22.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d040b2e90 0x7f2d04064470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:22.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d040b2e90 0x7f2d04064470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52594/0 (socket says 172.21.2.116:52594) 2021-11-21T22:02:22.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 -- 172.21.2.116:0/2178639678 learned_addr learned my addr 172.21.2.116:0/2178639678 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:22.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 -- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d04064960 msgr2=0x7f2d04144d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:22.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d04064960 0x7f2d04144d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:22.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 -- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf0004500 msgr2=0x7f2d041435f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:22.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.050+0000 7f2d03fff700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf0004500 0x7f2d041435f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:22.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d03fff700 1 -- 172.21.2.116:0/2178639678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d040b3d40 con 0x7f2d040b2e90 2021-11-21T22:02:22.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d03fff700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d040b2e90 0x7f2d04064470 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f2d041569e0 tx=0x7f2cfc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:22.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d01ffb700 1 -- 172.21.2.116:0/2178639678 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cfc011760 con 0x7f2d040b2e90 2021-11-21T22:02:22.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d01ffb700 1 -- 172.21.2.116:0/2178639678 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc0118c0 con 0x7f2d040b2e90 2021-11-21T22:02:22.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d040b3fe0 con 0x7f2d040b2e90 2021-11-21T22:02:22.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d040b4740 con 0x7f2d040b2e90 2021-11-21T22:02:22.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d01ffb700 1 -- 172.21.2.116:0/2178639678 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cfc011a30 con 0x7f2d040b2e90 2021-11-21T22:02:22.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d040b4450 con 0x7f2d040b2e90 2021-11-21T22:02:22.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d01ffb700 1 -- 172.21.2.116:0/2178639678 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2cfc02b070 con 0x7f2d040b2e90 2021-11-21T22:02:22.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d01ffb700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2ce40535d0 0x7f2ce4055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:22.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d01ffb700 1 -- 172.21.2.116:0/2178639678 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f2cfc06d290 con 0x7f2d040b2e90 2021-11-21T22:02:22.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d08fe0700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2ce40535d0 0x7f2ce4055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:22.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d08fe0700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2ce40535d0 0x7f2ce4055a90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f2cf8008000 tx=0x7f2cf8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:22.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.054+0000 7f2d01ffb700 1 -- 172.21.2.116:0/2178639678 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2cfc044340 con 0x7f2d040b2e90 2021-11-21T22:02:22.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.166+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2d040af060 con 0x7f2d040b2e90 2021-11-21T22:02:22.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.990+0000 7f2d01ffb700 1 -- 172.21.2.116:0/2178639678 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2cfc01f0a0 con 0x7f2d040b2e90 2021-11-21T22:02:22.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2ce40535d0 msgr2=0x7f2ce4055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:22.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2ce40535d0 0x7f2ce4055a90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f2cf8008000 tx=0x7f2cf8009040).stop 2021-11-21T22:02:22.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d040b2e90 msgr2=0x7f2d04064470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:22.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d040b2e90 0x7f2d04064470 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f2d041569e0 tx=0x7f2cfc017000).stop 2021-11-21T22:02:22.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 shutdown_connections 2021-11-21T22:02:22.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d04064960 0x7f2d04144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:22.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2ce40535d0 0x7f2ce4055a90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:22.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d040b2e90 0x7f2d04064470 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:22.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 --2- 172.21.2.116:0/2178639678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2cf0004500 0x7f2d041435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:22.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 >> 172.21.2.116:0/2178639678 conn(0x7f2d0405bd20 msgr2=0x7f2d040aca70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:22.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 shutdown_connections 2021-11-21T22:02:22.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:22.994+0000 7f2d0b244700 1 -- 172.21.2.116:0/2178639678 wait complete. 2021-11-21T22:02:23.007 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:02:23.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.054+0000 7fbfbd721700 1 Processor -- start 2021-11-21T22:02:23.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.054+0000 7fbfbd721700 1 -- start start 2021-11-21T22:02:23.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbfb80ad780 0x7fbfb805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:23.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 0x7fbfb8143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:23.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfb80af7e0 0x7fbfb80b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:23.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb80adb60 con 0x7fbfb80af7e0 2021-11-21T22:02:23.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb8143160 con 0x7fbfb80ad780 2021-11-21T22:02:23.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80b0ce0 con 0x7fbfb80affc0 2021-11-21T22:02:23.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfb80af7e0 0x7fbfb80b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:23.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbfb80af7e0 0x7fbfb80b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52602/0 (socket says 172.21.2.116:52602) 2021-11-21T22:02:23.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 -- 172.21.2.116:0/2642826653 learned_addr learned my addr 172.21.2.116:0/2642826653 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:23.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 -- 172.21.2.116:0/2642826653 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 msgr2=0x7fbfb8143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:23.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 --2- 172.21.2.116:0/2642826653 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 0x7fbfb8143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:23.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 -- 172.21.2.116:0/2642826653 >> v2:172.21.2.117:3300/0 conn(0x7fbfb80ad780 msgr2=0x7fbfb805f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:23.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 --2- 172.21.2.116:0/2642826653 >> v2:172.21.2.117:3300/0 conn(0x7fbfb80ad780 0x7fbfb805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:23.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 -- 172.21.2.116:0/2642826653 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80b0a40 con 0x7fbfb80af7e0 2021-11-21T22:02:23.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb77fe700 1 --2- 172.21.2.116:0/2642826653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb80b16c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fbfb00032f0 tx=0x7fbfb0009290).ready entity=mon.0 client_cookie=12a50cd559474afe server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:23.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb5ffb700 1 -- 172.21.2.116:0/2642826653 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb000b870 con 0x7fbfb80af7e0 2021-11-21T22:02:23.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb5ffb700 1 -- 172.21.2.116:0/2642826653 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb000b9d0 con 0x7fbfb80af7e0 2021-11-21T22:02:23.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb5ffb700 1 -- 172.21.2.116:0/2642826653 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb000bb40 con 0x7fbfb80af7e0 2021-11-21T22:02:23.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- 172.21.2.116:0/2642826653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 msgr2=0x7fbfb80b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:23.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- 172.21.2.116:0/2642826653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb80b16c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fbfb00032f0 tx=0x7fbfb0009290).stop 2021-11-21T22:02:23.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- 172.21.2.116:0/2642826653 shutdown_connections 2021-11-21T22:02:23.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- 172.21.2.116:0/2642826653 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb80b16c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:23.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- 172.21.2.116:0/2642826653 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 0x7fbfb8143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- 172.21.2.116:0/2642826653 >> v2:172.21.2.117:3300/0 conn(0x7fbfb80ad780 0x7fbfb805f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- 172.21.2.116:0/2642826653 >> 172.21.2.116:0/2642826653 conn(0x7fbfb80b6000 msgr2=0x7fbfb80b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- 172.21.2.116:0/2642826653 shutdown_connections 2021-11-21T22:02:23.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- 172.21.2.116:0/2642826653 wait complete. 2021-11-21T22:02:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 Processor -- start 2021-11-21T22:02:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- start start 2021-11-21T22:02:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfb80ad780 0x7fbfb8145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:23.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb809ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 0x7fbfb80a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb809e350 con 0x7fbfb80af7e0 2021-11-21T22:02:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80671a0 con 0x7fbfb80ad780 2021-11-21T22:02:23.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8067440 con 0x7fbfb80affc0 2021-11-21T22:02:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb809ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb809ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52606/0 (socket says 172.21.2.116:52606) 2021-11-21T22:02:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 -- 172.21.2.116:0/288784233 learned_addr learned my addr 172.21.2.116:0/288784233 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 -- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 msgr2=0x7fbfb80a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:23.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 0x7fbfb80a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 -- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfb80ad780 msgr2=0x7fbfb8145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfb80ad780 0x7fbfb8145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 -- 172.21.2.116:0/288784233 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb0009530 con 0x7fbfb80af7e0 2021-11-21T22:02:23.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfb67fc700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb809ea50 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7fbfa0002f70 tx=0x7fbfa0006db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfaffff700 1 -- 172.21.2.116:0/288784233 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfa000e470 con 0x7fbfb80af7e0 2021-11-21T22:02:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfaffff700 1 -- 172.21.2.116:0/288784233 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa000e5d0 con 0x7fbfb80af7e0 2021-11-21T22:02:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfaffff700 1 -- 172.21.2.116:0/288784233 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfa000e930 con 0x7fbfb80af7e0 2021-11-21T22:02:23.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb80a1070 con 0x7fbfb80af7e0 2021-11-21T22:02:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.058+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb809df50 con 0x7fbfb80af7e0 2021-11-21T22:02:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.062+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb81433c0 con 0x7fbfb80af7e0 2021-11-21T22:02:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.062+0000 7fbfaffff700 1 -- 172.21.2.116:0/288784233 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbfa0018030 con 0x7fbfb80af7e0 2021-11-21T22:02:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.062+0000 7fbfaffff700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf9c053660 0x7fbf9c055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:23.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.062+0000 7fbfaffff700 1 -- 172.21.2.116:0/288784233 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fbfa006adc0 con 0x7fbfb80af7e0 2021-11-21T22:02:23.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.062+0000 7fbfb6ffd700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf9c053660 0x7fbf9c055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:23.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.062+0000 7fbfb6ffd700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf9c053660 0x7fbf9c055b20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fbfa8002f70 tx=0x7fbfa8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:23.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.062+0000 7fbfaffff700 1 -- 172.21.2.116:0/288784233 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfa0041f80 con 0x7fbfb80af7e0 2021-11-21T22:02:23.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.146+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbfb80573d0 con 0x7fbfb80af7e0 2021-11-21T22:02:23.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfaffff700 1 -- 172.21.2.116:0/288784233 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 185+0+0 (secure 0 0 0) 0x7fbfa00455a0 con 0x7fbfb80af7e0 2021-11-21T22:02:23.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf9c053660 msgr2=0x7fbf9c055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:23.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf9c053660 0x7fbf9c055b20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fbfa8002f70 tx=0x7fbfa8006a80).stop 2021-11-21T22:02:23.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 msgr2=0x7fbfb809ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:24.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb809ea50 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7fbfa0002f70 tx=0x7fbfa0006db0).stop 2021-11-21T22:02:24.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 shutdown_connections 2021-11-21T22:02:24.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbfb80affc0 0x7fbfb80a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:24.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf9c053660 0x7fbf9c055b20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:24.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbfb80af7e0 0x7fbfb809ea50 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:24.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 --2- 172.21.2.116:0/288784233 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbfb80ad780 0x7fbfb8145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:24.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 >> 172.21.2.116:0/288784233 conn(0x7fbfb80b6000 msgr2=0x7fbfb80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:24.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 shutdown_connections 2021-11-21T22:02:24.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:23.994+0000 7fbfbd721700 1 -- 172.21.2.116:0/288784233 wait complete. 2021-11-21T22:02:24.002 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:02:24.008 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:02:24.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 Processor -- start 2021-11-21T22:02:24.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- start start 2021-11-21T22:02:24.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f98400b6770 0x7f98400b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:24.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f98400b1270 0x7f98401394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:24.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400b0070 con 0x7f98401399e0 2021-11-21T22:02:24.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98400b05c0 con 0x7f98400b6770 2021-11-21T22:02:24.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98400b2230 con 0x7f98400b1270 2021-11-21T22:02:24.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f98400b6770 0x7f98400b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:24.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f98400b6770 0x7f98400b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52614/0 (socket says 172.21.2.116:52614) 2021-11-21T22:02:24.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 -- 172.21.2.116:0/2554129150 learned_addr learned my addr 172.21.2.116:0/2554129150 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:24.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 -- 172.21.2.116:0/2554129150 >> v2:172.21.2.117:3300/0 conn(0x7f98400b1270 msgr2=0x7f98401394f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:24.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 --2- 172.21.2.116:0/2554129150 >> v2:172.21.2.117:3300/0 conn(0x7f98400b1270 0x7f98401394f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:24.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 -- 172.21.2.116:0/2554129150 >> v1:172.21.2.116:6789/0 conn(0x7f98401399e0 legacy=0x7f98400ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:24.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 -- 172.21.2.116:0/2554129150 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400b24d0 con 0x7f98400b6770 2021-11-21T22:02:24.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9845c7e700 1 --2- 172.21.2.116:0/2554129150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98400b6770 0x7f98400b6b50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f9834002f70 tx=0x7f9834006f40).ready entity=mon.0 client_cookie=f534e60773042253 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:24.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9844c7c700 1 -- 172.21.2.116:0/2554129150 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f983400dc70 con 0x7f98400b6770 2021-11-21T22:02:24.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9844c7c700 1 -- 172.21.2.116:0/2554129150 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9834000e90 con 0x7f98400b6770 2021-11-21T22:02:24.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9844c7c700 1 -- 172.21.2.116:0/2554129150 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f983400be90 con 0x7f98400b6770 2021-11-21T22:02:24.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- 172.21.2.116:0/2554129150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98400b6770 msgr2=0x7f98400b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:24.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- 172.21.2.116:0/2554129150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98400b6770 0x7f98400b6b50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f9834002f70 tx=0x7f9834006f40).stop 2021-11-21T22:02:24.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- 172.21.2.116:0/2554129150 shutdown_connections 2021-11-21T22:02:24.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- 172.21.2.116:0/2554129150 >> v2:172.21.2.117:3300/0 conn(0x7f98400b1270 0x7f98401394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:24.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- 172.21.2.116:0/2554129150 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98400b6770 0x7f98400b6b50 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:24.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- 172.21.2.116:0/2554129150 >> 172.21.2.116:0/2554129150 conn(0x7f9840062040 msgr2=0x7f98400b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:24.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- 172.21.2.116:0/2554129150 shutdown_connections 2021-11-21T22:02:24.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- 172.21.2.116:0/2554129150 wait complete. 2021-11-21T22:02:24.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 Processor -- start 2021-11-21T22:02:24.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- start start 2021-11-21T22:02:24.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400b1270 0x7f984013eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:24.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98400b6770 0x7f9840133740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:24.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401399e0 0x7f9840133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:24.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98400b5980 con 0x7f98401399e0 2021-11-21T22:02:24.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9840141490 con 0x7f98400b6770 2021-11-21T22:02:24.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9847ee2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98400b5bb0 con 0x7f98400b1270 2021-11-21T22:02:24.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401399e0 0x7f9840133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:24.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401399e0 0x7f9840133c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52620/0 (socket says 172.21.2.116:52620) 2021-11-21T22:02:24.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 -- 172.21.2.116:0/3870336139 learned_addr learned my addr 172.21.2.116:0/3870336139 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:24.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 -- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400b1270 msgr2=0x7f984013eef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:24.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400b1270 0x7f984013eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:24.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 -- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98400b6770 msgr2=0x7f9840133740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:24.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98400b6770 0x7f9840133740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:24.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 -- 172.21.2.116:0/3870336139 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983400b460 con 0x7f98401399e0 2021-11-21T22:02:24.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f984647f700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401399e0 0x7f9840133c30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f9840060900 tx=0x7f9838007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:24.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.058+0000 7f9832ffd700 1 -- 172.21.2.116:0/3870336139 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9838006aa0 con 0x7f98401399e0 2021-11-21T22:02:24.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9832ffd700 1 -- 172.21.2.116:0/3870336139 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9838006c00 con 0x7f98401399e0 2021-11-21T22:02:24.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9840141710 con 0x7f98401399e0 2021-11-21T22:02:24.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9840143280 con 0x7f98401399e0 2021-11-21T22:02:24.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9832ffd700 1 -- 172.21.2.116:0/3870336139 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9838007030 con 0x7f98401399e0 2021-11-21T22:02:24.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98400573d0 con 0x7f98401399e0 2021-11-21T22:02:24.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9832ffd700 1 -- 172.21.2.116:0/3870336139 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9838010030 con 0x7f98401399e0 2021-11-21T22:02:24.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9832ffd700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98200536d0 0x7f9820055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:24.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9832ffd700 1 -- 172.21.2.116:0/3870336139 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f983806b7f0 con 0x7f98401399e0 2021-11-21T22:02:24.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9845c7e700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98200536d0 0x7f9820055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:24.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9845c7e700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98200536d0 0x7f9820055b90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f9834002f70 tx=0x7f98400af4d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:24.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.062+0000 7f9832ffd700 1 -- 172.21.2.116:0/3870336139 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98380429a0 con 0x7f98401399e0 2021-11-21T22:02:24.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:24.146+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9840141a90 con 0x7f98401399e0 2021-11-21T22:02:25.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.006+0000 7f9832ffd700 1 -- 172.21.2.116:0/3870336139 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v132) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9838045fc0 con 0x7f98401399e0 2021-11-21T22:02:25.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98200536d0 msgr2=0x7f9820055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98200536d0 0x7f9820055b90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f9834002f70 tx=0x7f98400af4d0).stop 2021-11-21T22:02:25.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401399e0 msgr2=0x7f9840133c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401399e0 0x7f9840133c30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f9840060900 tx=0x7f9838007f70).stop 2021-11-21T22:02:25.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 shutdown_connections 2021-11-21T22:02:25.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f98200536d0 0x7f9820055b90 secure :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0x7f9834002f70 tx=0x7f98400af4d0).stop 2021-11-21T22:02:25.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401399e0 0x7f9840133c30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98400b6770 0x7f9840133740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 --2- 172.21.2.116:0/3870336139 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400b1270 0x7f984013eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 >> 172.21.2.116:0/3870336139 conn(0x7f9840062040 msgr2=0x7f98401365f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:25.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 shutdown_connections 2021-11-21T22:02:25.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.010+0000 7f9847ee2700 1 -- 172.21.2.116:0/3870336139 wait complete. 2021-11-21T22:02:25.017 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:02:25.027 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2021-11-21T22:02:25.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55b005a700 1 Processor -- start 2021-11-21T22:02:25.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55b005a700 1 -- start start 2021-11-21T22:02:25.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55b005a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f55a8067a40 0x7f55a80acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55b005a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55a80af160 0x7f55a81474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55b005a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a80aeb00 con 0x7f55a80b1370 2021-11-21T22:02:25.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55b005a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a80b1750 con 0x7f55a80af160 2021-11-21T22:02:25.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55b005a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a805c8a0 con 0x7f55a8067a40 2021-11-21T22:02:25.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55a80af160 0x7f55a81474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55a80af160 0x7f55a81474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52630/0 (socket says 172.21.2.116:52630) 2021-11-21T22:02:25.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 -- 172.21.2.116:0/4238332856 learned_addr learned my addr 172.21.2.116:0/4238332856 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:25.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55addf6700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f55a80b1370 0x7f55a80ae3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36352/0 (socket says 172.21.2.116:36352) 2021-11-21T22:02:25.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 -- 172.21.2.116:0/4238332856 >> v2:172.21.2.117:3300/0 conn(0x7f55a8067a40 msgr2=0x7f55a80acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:25.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 --2- 172.21.2.116:0/4238332856 >> v2:172.21.2.117:3300/0 conn(0x7f55a8067a40 0x7f55a80acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 -- 172.21.2.116:0/4238332856 >> v1:172.21.2.116:6789/0 conn(0x7f55a80b1370 legacy=0x7f55a80ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55acdf4700 1 -- 172.21.2.116:0/4238332856 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005323528 0 0) 0x7f55a80aeb00 con 0x7f55a80b1370 2021-11-21T22:02:25.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 -- 172.21.2.116:0/4238332856 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a805cb40 con 0x7f55a80af160 2021-11-21T22:02:25.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55ae5f7700 1 --2- 172.21.2.116:0/4238332856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a81474f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f55a00032f0 tx=0x7f55a0009290).ready entity=mon.0 client_cookie=85312f7dab8746f1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55acdf4700 1 -- 172.21.2.116:0/4238332856 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a000b870 con 0x7f55a80af160 2021-11-21T22:02:25.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55acdf4700 1 -- 172.21.2.116:0/4238332856 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a000b9d0 con 0x7f55a80af160 2021-11-21T22:02:25.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.078+0000 7f55acdf4700 1 -- 172.21.2.116:0/4238332856 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a000bb40 con 0x7f55a80af160 2021-11-21T22:02:25.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4238332856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 msgr2=0x7f55a81474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 --2- 172.21.2.116:0/4238332856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a81474f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f55a00032f0 tx=0x7f55a0009290).stop 2021-11-21T22:02:25.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4238332856 shutdown_connections 2021-11-21T22:02:25.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 --2- 172.21.2.116:0/4238332856 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a81474f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 --2- 172.21.2.116:0/4238332856 >> v2:172.21.2.117:3300/0 conn(0x7f55a8067a40 0x7f55a80acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4238332856 >> 172.21.2.116:0/4238332856 conn(0x7f55a80b4f10 msgr2=0x7f55a80b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:25.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4238332856 shutdown_connections 2021-11-21T22:02:25.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4238332856 wait complete. 2021-11-21T22:02:25.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 Processor -- start 2021-11-21T22:02:25.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- start start 2021-11-21T22:02:25.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a8067a40 0x7f55a8156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a80a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a80b1370 0x7f55a814c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8156070 con 0x7f55a80af160 2021-11-21T22:02:25.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8156260 con 0x7f55a8067a40 2021-11-21T22:02:25.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8156500 con 0x7f55a80b1370 2021-11-21T22:02:25.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a80a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a80a6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52636/0 (socket says 172.21.2.116:52636) 2021-11-21T22:02:25.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 -- 172.21.2.116:0/4162764161 learned_addr learned my addr 172.21.2.116:0/4162764161 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:25.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55addf6700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a8067a40 0x7f55a8156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ae5f7700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a80b1370 0x7f55a814c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 -- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a80b1370 msgr2=0x7f55a814c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a80b1370 0x7f55a814c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 -- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a8067a40 msgr2=0x7f55a8156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a8067a40 0x7f55a8156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 -- 172.21.2.116:0/4162764161 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0009530 con 0x7f55a80af160 2021-11-21T22:02:25.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ae5f7700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a80b1370 0x7f55a814c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:25.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55ad5f5700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a80a6aa0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f55a4002f70 tx=0x7f55a4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f559affd700 1 -- 172.21.2.116:0/4162764161 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a4013460 con 0x7f55a80af160 2021-11-21T22:02:25.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f559affd700 1 -- 172.21.2.116:0/4162764161 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a40135c0 con 0x7f55a80af160 2021-11-21T22:02:25.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a81567a0 con 0x7f55a80af160 2021-11-21T22:02:25.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f559affd700 1 -- 172.21.2.116:0/4162764161 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a4013920 con 0x7f55a80af160 2021-11-21T22:02:25.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8067400 con 0x7f55a80af160 2021-11-21T22:02:25.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a80573d0 con 0x7f55a80af160 2021-11-21T22:02:25.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f559affd700 1 -- 172.21.2.116:0/4162764161 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f55a401a060 con 0x7f55a80af160 2021-11-21T22:02:25.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.082+0000 7f559affd700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053640 0x7f5588055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.086+0000 7f559affd700 1 -- 172.21.2.116:0/4162764161 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f55a406b3e0 con 0x7f55a80af160 2021-11-21T22:02:25.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.086+0000 7f55addf6700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053640 0x7f5588055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.086+0000 7f55addf6700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053640 0x7f5588055b00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f559c00b000 tx=0x7f559c007160).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.086+0000 7f559affd700 1 -- 172.21.2.116:0/4162764161 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55a4042430 con 0x7f55a80af160 2021-11-21T22:02:25.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.198+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f55a8068320 con 0x7f55a80af160 2021-11-21T22:02:25.201 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:02:25.198+0000 7f1e505b4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:02:25.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f559affd700 1 -- 172.21.2.116:0/4162764161 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7f55a4045a50 con 0x7f55a80af160 2021-11-21T22:02:25.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053640 msgr2=0x7f5588055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053640 0x7f5588055b00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f559c00b000 tx=0x7f559c007160).stop 2021-11-21T22:02:25.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 msgr2=0x7f55a80a6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a80a6aa0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f55a4002f70 tx=0x7f55a4006df0).stop 2021-11-21T22:02:25.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 shutdown_connections 2021-11-21T22:02:25.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55a80b1370 0x7f55a814c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5588053640 0x7f5588055b00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55a80af160 0x7f55a80a6aa0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 --2- 172.21.2.116:0/4162764161 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55a8067a40 0x7f55a8156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 >> 172.21.2.116:0/4162764161 conn(0x7f55a80b4f10 msgr2=0x7f55a80afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:25.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 shutdown_connections 2021-11-21T22:02:25.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.202+0000 7f55b005a700 1 -- 172.21.2.116:0/4162764161 wait complete. 2021-11-21T22:02:25.211 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 24 and data pool 22 2021-11-21T22:02:25.214 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:02:25.214 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:25.221 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:25.230 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:25.243 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:25.258 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:02:25.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:25.259 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:25.260 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:25.269 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:02:25.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:25.271 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:25.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:02:25.272 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:25.321 INFO:teuthology.orchestra.run.gibba019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:02:25.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:25.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:25.323 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:25.324 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:25.333 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:02:25.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:25.335 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:25.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:02:25.339 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:25.348 INFO:teuthology.orchestra.run.gibba036.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:02:25.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:25.350 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:02:25.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 Processor -- start 2021-11-21T22:02:25.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- start start 2021-11-21T22:02:25.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 0x7fc3e814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e80a6e80 con 0x7fc3e80a86b0 2021-11-21T22:02:25.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e814a7b0 con 0x7fc3e8062410 2021-11-21T22:02:25.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80a8bc0 con 0x7fc3e80a6aa0 2021-11-21T22:02:25.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3edb0f700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc3e80a86b0 0x7fc3e81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36370/0 (socket says 172.21.2.116:36370) 2021-11-21T22:02:25.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3edb0f700 1 -- 172.21.2.116:0/394091433 learned_addr learned my addr 172.21.2.116:0/394091433 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:25.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867195212 0 0) 0x7fc3e80a6e80 con 0x7fc3e80a86b0 2021-11-21T22:02:25.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d4003260 con 0x7fc3e80a86b0 2021-11-21T22:02:25.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 0x7fc3e814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3790901979 0 0) 0x7fc3d4003260 con 0x7fc3e80a86b0 2021-11-21T22:02:25.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 msgr2=0x7fc3e814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:25.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 0x7fc3e814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 >> v1:172.21.2.117:6789/0 conn(0x7fc3e8062410 legacy=0x7fc3e81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:25.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e814aa10 con 0x7fc3e80a86b0 2021-11-21T22:02:25.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fc3dc001680 con 0x7fc3e80a86b0 2021-11-21T22:02:25.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 >> v1:172.21.2.116:6789/0 conn(0x7fc3e80a86b0 legacy=0x7fc3e81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3d4003800 0x7fc3d4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4004500 0x7fc3d4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 0x7fc3d4004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80a8bc0 con 0x7fc3d4003800 2021-11-21T22:02:25.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d400b910 con 0x7fc3d4004500 2021-11-21T22:02:25.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d400bad0 con 0x7fc3e80a6aa0 2021-11-21T22:02:25.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3d4003800 0x7fc3d4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3edb0f700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4004500 0x7fc3d4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 -- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 msgr2=0x7fc3d4004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:25.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3eeb11700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 0x7fc3d4004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 0x7fc3d4004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 -- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4004500 msgr2=0x7fc3d4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:25.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4004500 0x7fc3d4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 -- 172.21.2.116:0/394091433 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d4003260 con 0x7fc3d4003800 2021-11-21T22:02:25.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3edb0f700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4004500 0x7fc3d4004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:25.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3eeb11700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e80a6aa0 0x7fc3d4004ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:25.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ee310700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3d4003800 0x7fc3d4003c00 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fc3e000a000 tx=0x7fc3e0008040).ready entity=mon.0 client_cookie=c7818788f3cfa5bc server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3e000c070 con 0x7fc3d4003800 2021-11-21T22:02:25.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e000e620 con 0x7fc3d4003800 2021-11-21T22:02:25.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3ed30e700 1 -- 172.21.2.116:0/394091433 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3e000e980 con 0x7fc3d4003800 2021-11-21T22:02:25.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- 172.21.2.116:0/394091433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3d4003800 msgr2=0x7fc3d4003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 --2- 172.21.2.116:0/394091433 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3d4003800 0x7fc3d4003c00 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fc3e000a000 tx=0x7fc3e0008040).stop 2021-11-21T22:02:25.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3eeb11700 1 -- 172.21.2.116:0/394091433 reap_dead start 2021-11-21T22:02:25.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- 172.21.2.116:0/394091433 shutdown_connections 2021-11-21T22:02:25.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- 172.21.2.116:0/394091433 >> 172.21.2.116:0/394091433 conn(0x7fc3e805bd20 msgr2=0x7fc3e805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:25.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- 172.21.2.116:0/394091433 shutdown_connections 2021-11-21T22:02:25.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- 172.21.2.116:0/394091433 wait complete. 2021-11-21T22:02:25.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 Processor -- start 2021-11-21T22:02:25.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.402+0000 7fc3f0574700 1 -- start start 2021-11-21T22:02:25.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4003800 0x7fc3e80b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3e8064a90 0x7fc3e8141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e81415f0 0x7fc3e8159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80643e0 con 0x7fc3e8064a90 2021-11-21T22:02:25.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8064680 con 0x7fc3d4003800 2021-11-21T22:02:25.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80ac320 con 0x7fc3e81415f0 2021-11-21T22:02:25.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3e8064a90 0x7fc3e8141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3e8064a90 0x7fc3e8141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52658/0 (socket says 172.21.2.116:52658) 2021-11-21T22:02:25.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 -- 172.21.2.116:0/3692685338 learned_addr learned my addr 172.21.2.116:0/3692685338 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:25.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 -- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e81415f0 msgr2=0x7fc3e8159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e81415f0 0x7fc3e8159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 -- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4003800 msgr2=0x7fc3e80b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4003800 0x7fc3e80b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 -- 172.21.2.116:0/3692685338 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e80ac5c0 con 0x7fc3e8064a90 2021-11-21T22:02:25.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3edb0f700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3e8064a90 0x7fc3e8141100 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7fc3dc008000 tx=0x7fc3dc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3e77fe700 1 -- 172.21.2.116:0/3692685338 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3dc007170 con 0x7fc3e8064a90 2021-11-21T22:02:25.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3e77fe700 1 -- 172.21.2.116:0/3692685338 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0072d0 con 0x7fc3e8064a90 2021-11-21T22:02:25.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e80ab260 con 0x7fc3e8064a90 2021-11-21T22:02:25.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3e77fe700 1 -- 172.21.2.116:0/3692685338 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3dc004920 con 0x7fc3e8064a90 2021-11-21T22:02:25.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e8141c90 con 0x7fc3e8064a90 2021-11-21T22:02:25.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e80573d0 con 0x7fc3e8064a90 2021-11-21T22:02:25.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3e77fe700 1 -- 172.21.2.116:0/3692685338 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc3dc01b060 con 0x7fc3e8064a90 2021-11-21T22:02:25.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3e77fe700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3c80535b0 0x7fc3c8055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3e77fe700 1 -- 172.21.2.116:0/3692685338 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fc3dc06b640 con 0x7fc3e8064a90 2021-11-21T22:02:25.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3ee310700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3c80535b0 0x7fc3c8055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.406+0000 7fc3ee310700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3c80535b0 0x7fc3c8055a70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fc3e000a000 tx=0x7fc3e0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.410+0000 7fc3e77fe700 1 -- 172.21.2.116:0/3692685338 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc3dc042650 con 0x7fc3e8064a90 2021-11-21T22:02:25.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc3e805db70 con 0x7fc3e8064a90 2021-11-21T22:02:25.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3e77fe700 1 -- 172.21.2.116:0/3692685338 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v133) v1 ==== 81+0+33826 (secure 0 0 0) 0x7fc3dc019070 con 0x7fc3e8064a90 2021-11-21T22:02:25.500 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:25.500 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:25.500 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 133, 2021-11-21T22:02:25.501 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:02:25.501 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:02:25.501 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:02:25.201540+0000", 2021-11-21T22:02:25.501 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:25.502 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:25.502 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:02:25.502 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:02:25.502 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:02:25.503 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:02:25.503 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:02:25.503 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:02:25.503 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:02:25.504 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:25.504 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:02:25.504 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:02:25.504 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:02:25.505 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:02:25.505 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:02:25.505 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 24, 2021-11-21T22:02:25.505 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:02:25.506 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:02:25.506 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:02:25.506 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:02:25.506 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:02:25.507 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.507 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:02:25.507 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:02:25.508 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:02:25.508 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:25.508 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:25.508 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:25.509 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:25.509 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:25.509 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:25.509 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:25.510 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:25.511 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:02:25.511 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:02:25.512 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:02:25.512 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:02:25.512 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:02:25.513 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:25.513 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:25.514 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:25.514 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:25.515 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:25.515 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:25.515 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:25.515 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.516 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:02:25.516 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:25.516 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:25.516 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:25.517 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:25.517 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:25.517 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:25.517 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:25.518 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:25.518 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:25.518 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:25.518 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:25.519 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:25.519 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:25.519 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:25.519 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:25.519 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:25.520 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:25.520 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:25.520 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:25.520 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:25.521 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:25.521 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:25.521 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:25.521 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:25.521 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:25.522 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:25.522 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.522 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:25.522 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:25.523 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:25.523 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:25.523 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:25.523 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:25.524 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:25.524 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:25.524 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:25.524 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:25.525 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:25.525 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:02:25.525 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:02:25.525 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.525 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:25.526 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:02:25.526 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.526 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.526 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.527 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:02:25.528 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:02:25.528 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:02:25.528 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:25.528 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:25.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:25.529 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:25.529 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:25.529 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:25.529 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:25.530 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:25.530 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:25.530 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:25.531 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:25.531 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:25.531 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:25.531 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:25.532 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:25.532 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:25.532 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:25.532 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:25.532 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:25.533 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:25.533 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.533 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:02:25.533 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:25.533 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:25.534 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:25.534 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:25.534 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:25.534 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:25.535 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:25.535 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:25.536 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:25.536 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:25.536 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:25.537 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:25.537 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:25.538 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:25.538 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:25.538 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:25.539 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:25.539 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:25.540 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:25.540 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:25.541 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:25.541 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:25.541 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:25.542 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:25.542 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:25.542 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:25.543 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.543 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:25.544 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:25.544 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:25.544 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:25.545 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:25.545 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:25.545 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:25.546 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:25.546 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:25.546 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:25.546 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:25.547 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:02:25.547 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:25.548 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:02:25.548 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.548 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.548 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.549 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 22, 2021-11-21T22:02:25.549 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:02:25.549 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:02:01.358157+0000", 2021-11-21T22:02:25.550 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:25.550 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:25.550 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:25.550 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:25.551 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:25.551 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:25.551 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:25.552 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:25.552 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:25.552 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:25.552 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:25.552 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:25.553 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:25.553 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:25.553 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:25.554 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:25.554 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:25.554 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:25.554 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:25.555 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:25.555 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.555 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "133", 2021-11-21T22:02:25.555 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:25.555 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:25.556 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:25.556 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:25.556 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:25.556 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:25.557 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:25.557 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:25.557 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:25.557 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:25.557 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:25.558 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:25.558 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:25.558 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:25.558 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:25.559 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:25.559 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:25.559 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:25.560 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:25.560 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:25.560 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:25.560 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:25.561 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:25.561 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:25.562 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:25.562 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:25.562 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.562 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:25.563 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:25.563 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:25.563 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:25.564 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:25.564 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:25.564 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:25.564 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:25.565 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:25.565 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:25.565 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:25.565 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:02:25.565 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:25.566 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:02:25.566 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:02:25.566 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.566 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.567 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.567 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 24, 2021-11-21T22:02:25.568 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:02:25.568 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:02:24.161694+0000", 2021-11-21T22:02:25.568 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 32769, 2021-11-21T22:02:25.569 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool,creating", 2021-11-21T22:02:25.569 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:25.569 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:25.569 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:25.569 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:25.570 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:25.570 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:25.570 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:25.571 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:25.571 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:25.571 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:25.571 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:25.572 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:25.572 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:25.572 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:25.572 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:25.573 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:25.574 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:25.574 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:25.574 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.575 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "133", 2021-11-21T22:02:25.575 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:25.576 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:25.576 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:25.577 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:25.577 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:25.578 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:25.578 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:25.579 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:25.580 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:25.581 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:25.581 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:25.582 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:25.583 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:25.583 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:25.584 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:25.584 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:25.585 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:25.585 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:25.586 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:25.586 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:25.586 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:25.587 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:25.587 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:25.588 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:25.588 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:25.589 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:25.589 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.590 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:25.590 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:25.591 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:25.591 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:25.591 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:25.592 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:25.592 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:25.593 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:25.593 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:25.594 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:25.594 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:25.594 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:02:25.595 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:02:25.595 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:02:25.596 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:02:25.596 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.597 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:25.597 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:02:25.598 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:02:25.598 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.598 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.599 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.599 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:25.600 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:02:25.600 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.601 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:25.601 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:02:25.601 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.602 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.602 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.603 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.603 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.604 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.604 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.605 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:25.605 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.605 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.606 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.606 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.607 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.607 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.607 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:02:25.608 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.608 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.609 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.609 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.610 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:02:25.610 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.611 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.611 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.612 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.612 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.613 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.613 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.614 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.614 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:02:25.615 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.615 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.616 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.616 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.616 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:02:25.617 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.617 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.618 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.618 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.619 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.619 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.620 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.620 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.620 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:02:25.621 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.621 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.622 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.622 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.623 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:02:25.623 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.623 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.624 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.624 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.625 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.625 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.626 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.626 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.626 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:02:25.627 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.628 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.628 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.628 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.629 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:02:25.629 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:25.630 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.630 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.631 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.631 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:02:25.632 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:02:25.632 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:02:25.633 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:02:25.633 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.633 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.634 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.634 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.635 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.635 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.636 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:25.636 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:02:25.636 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.637 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.637 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.638 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.638 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.639 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.639 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.639 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:25.640 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.640 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.640 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.640 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.641 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.641 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.641 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:02:25.641 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.642 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.642 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.642 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.642 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:02:25.643 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.643 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.643 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.643 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.643 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.644 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.644 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.644 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.644 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:02:25.645 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.645 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.645 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.645 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.646 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:02:25.646 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.646 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.646 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.646 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.647 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.647 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.647 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.647 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.648 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:02:25.648 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.648 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.648 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.648 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.649 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:02:25.649 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.649 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.649 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.650 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.650 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.650 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.650 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.650 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.651 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:02:25.651 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.651 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.651 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.651 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.652 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:02:25.652 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:25.652 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.652 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.653 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.653 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:02:25.653 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:02:25.653 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:02:25.653 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:02:25.654 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.654 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.654 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.654 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.655 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.655 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.655 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:25.655 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:02:25.655 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.656 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.656 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.656 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.656 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.656 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.657 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.657 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 127, 2021-11-21T22:02:25.657 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.657 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.658 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.658 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.658 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.658 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.659 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:02:25.659 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.659 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.659 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.659 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.660 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:02:25.660 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.660 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.660 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.661 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.661 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.661 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.661 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.662 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.662 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:02:25.662 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.662 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.662 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.663 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.663 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:02:25.663 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.663 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.664 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.664 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.664 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.664 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.665 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.665 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.665 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:02:25.665 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.665 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.666 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.666 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.666 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:02:25.666 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.667 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.667 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.667 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.667 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.667 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.668 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.668 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.668 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:02:25.668 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.669 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.669 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.669 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.669 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:02:25.669 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:25.670 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.670 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.670 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.670 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:02:25.671 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:02:25.671 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:02:25.671 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:02:25.671 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.671 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.672 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.672 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.672 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.672 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.673 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:25.673 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:02:25.673 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.673 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.673 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.674 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.674 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.674 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.674 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.675 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:25.675 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.675 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.675 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.676 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.676 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.676 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.677 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:02:25.677 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.677 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.677 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.677 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.678 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:02:25.678 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.678 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.678 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.679 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.679 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.679 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.679 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.680 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.680 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:02:25.680 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.680 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.681 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.681 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.681 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:02:25.681 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.681 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.682 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.682 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.682 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.682 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.683 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.683 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.683 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:02:25.683 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.684 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.684 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.684 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.684 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:02:25.684 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.685 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.685 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.685 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.685 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.686 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.686 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.686 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.686 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:02:25.686 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.687 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.687 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.687 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.687 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:02:25.688 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:25.688 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.688 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.688 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.688 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:02:25.689 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:02:25.689 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:02:25.689 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:02:25.689 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.690 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.690 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.690 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.690 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.691 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.691 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:25.691 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:02:25.691 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.691 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.692 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.692 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.692 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.692 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.693 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.693 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 127, 2021-11-21T22:02:25.693 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.693 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.694 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.694 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.694 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.694 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.695 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:02:25.695 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.695 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.695 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.696 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.696 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:02:25.696 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.696 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.696 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.697 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.697 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.697 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.697 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.698 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.698 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:02:25.698 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.698 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.698 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.699 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.699 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:02:25.699 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.699 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.700 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.700 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.700 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.700 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.700 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.701 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.701 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:02:25.701 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.701 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.702 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.702 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.702 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:02:25.702 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.702 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.703 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.703 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.703 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.703 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.703 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.704 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.704 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:02:25.704 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.704 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.705 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.705 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.705 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:02:25.705 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:25.705 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.706 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.706 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.706 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:02:25.706 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:02:25.707 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:02:25.707 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:02:25.707 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.707 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.708 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.708 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.708 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.708 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.708 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:25.709 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:02:25.709 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.709 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.709 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.710 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.710 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.710 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.710 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.710 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:25.711 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.711 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.711 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.711 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.712 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.712 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.712 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:02:25.712 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.712 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.713 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.713 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.713 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:02:25.713 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.714 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.714 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.714 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.714 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.714 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.715 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.715 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.715 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:02:25.715 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.716 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.716 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.716 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.716 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:02:25.716 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.717 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.717 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.717 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.717 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.717 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.718 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.718 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.718 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:02:25.718 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.719 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.719 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.719 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.719 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:02:25.720 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.720 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.720 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.720 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.720 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.721 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.721 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.721 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.721 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:02:25.722 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.722 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.722 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.722 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.723 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:02:25.723 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:25.723 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.723 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.723 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.724 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:02:25.724 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:02:25.724 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:02:25.724 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:02:25.724 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.725 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.725 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.725 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.725 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.726 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.726 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:25.726 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:02:25.726 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.727 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.727 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.727 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.727 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.727 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.728 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.728 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:25.728 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.728 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.728 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.729 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.729 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.729 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.729 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:02:25.730 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.730 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.730 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.730 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.731 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:02:25.731 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.731 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.731 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.731 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.732 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.732 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.732 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.732 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.733 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:02:25.733 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.733 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.733 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.733 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.734 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:02:25.734 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.734 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.734 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.735 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.735 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.735 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.735 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.735 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.736 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:02:25.736 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.736 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.736 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.737 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.737 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:02:25.737 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.737 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.737 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.738 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.738 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.738 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.738 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.739 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.739 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:02:25.739 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.739 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.739 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.740 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.740 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:02:25.740 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:25.740 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.741 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.741 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.741 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:02:25.741 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:02:25.741 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:02:25.742 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:02:25.742 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.742 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.742 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.742 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.743 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.743 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.743 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:25.743 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:02:25.744 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:25.744 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:25.744 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:25.744 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:25.744 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:25.745 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:25.745 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:25.745 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:25.745 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:25.746 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:25.746 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:25.746 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.746 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.746 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.747 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:02:25.747 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.747 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.747 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.748 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.748 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:02:25.748 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.748 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.748 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.749 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.749 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:25.749 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.749 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.749 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.750 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:02:25.750 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.750 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.750 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.751 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.751 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:02:25.751 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.751 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.752 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.752 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.752 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:25.752 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.752 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.753 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.753 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:02:25.753 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.753 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.754 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.754 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.754 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:02:25.754 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.754 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.755 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.755 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.755 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:25.755 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:25.756 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.756 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:25.756 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:02:25.756 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.756 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.757 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.757 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:25.757 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:02:25.757 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:25.757 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.758 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.758 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.758 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:02:25.758 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:02:25.759 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:02:25.759 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:02:25.759 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:25.759 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:25.759 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:25.760 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:25.760 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.760 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:25.760 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:02:25.760 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.761 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:25.761 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.761 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.761 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.762 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.762 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.762 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:02:25.762 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.762 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.763 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.763 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:25.763 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.763 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.764 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.764 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.764 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.764 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:02:25.764 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.765 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.765 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.765 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:25.765 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.765 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.766 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.766 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.766 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.766 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:02:25.767 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.767 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.767 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.767 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:25.767 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.768 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.768 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.768 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.768 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.768 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:02:25.769 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.769 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.769 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.769 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:25.770 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.770 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.770 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.770 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.770 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.771 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:02:25.771 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.771 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.771 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.772 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:25.772 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.772 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.772 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.772 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.773 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.773 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:02:25.773 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.773 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.774 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.774 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:25.774 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.774 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.774 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.775 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.776 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.776 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:02:25.776 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.777 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.777 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:25.777 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:25.777 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:25.777 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:25.778 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:25.778 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:25.778 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:25.778 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:02:25.779 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:25.779 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.779 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:25.779 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:02:25.780 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:02:25.780 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:02:25.780 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:02:25.780 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:02:25.780 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/1577318848": "2021-11-22T22:02:10.776983+0000", 2021-11-21T22:02:25.781 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/1577318848": "2021-11-22T22:02:10.776983+0000" 2021-11-21T22:02:25.781 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.781 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:02:25.781 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:02:25.782 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:02:25.782 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:02:25.782 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:02:25.782 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:02:25.782 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:02:25.783 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:02:25.783 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:25.783 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:25.783 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:02:25.784 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:02:25.784 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:02:25.784 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:02:25.784 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:02:25.784 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:25.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3c80535b0 msgr2=0x7fc3c8055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3c80535b0 0x7fc3c8055a70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fc3e000a000 tx=0x7fc3e0008040).stop 2021-11-21T22:02:25.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3e8064a90 msgr2=0x7fc3e8141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3e8064a90 0x7fc3e8141100 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7fc3dc008000 tx=0x7fc3dc009040).stop 2021-11-21T22:02:25.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 shutdown_connections 2021-11-21T22:02:25.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc3e81415f0 0x7fc3e8159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc3c80535b0 0x7fc3c8055a70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc3e8064a90 0x7fc3e8141100 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 --2- 172.21.2.116:0/3692685338 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc3d4003800 0x7fc3e80b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 >> 172.21.2.116:0/3692685338 conn(0x7fc3e805bd20 msgr2=0x7fc3e81426b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:25.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 shutdown_connections 2021-11-21T22:02:25.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.494+0000 7fc3f0574700 1 -- 172.21.2.116:0/3692685338 wait complete. 2021-11-21T22:02:25.789 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:25.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 Processor -- start 2021-11-21T22:02:25.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- start start 2021-11-21T22:02:25.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f93a40ac690 0x7f93a40adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f93a40b2350 0x7f93a40b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a40a8650 con 0x7f93a4143200 2021-11-21T22:02:25.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a40acd30 con 0x7f93a40ac690 2021-11-21T22:02:25.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a405c920 con 0x7f93a40b2350 2021-11-21T22:02:25.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f93a40ac690 0x7f93a40adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f93a40ac690 0x7f93a40adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52662/0 (socket says 172.21.2.116:52662) 2021-11-21T22:02:25.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 -- 172.21.2.116:0/1598631423 learned_addr learned my addr 172.21.2.116:0/1598631423 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:25.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 -- 172.21.2.116:0/1598631423 >> v2:172.21.2.117:3300/0 conn(0x7f93a40b2350 msgr2=0x7f93a40b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:25.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 --2- 172.21.2.116:0/1598631423 >> v2:172.21.2.117:3300/0 conn(0x7f93a40b2350 0x7f93a40b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 -- 172.21.2.116:0/1598631423 >> v1:172.21.2.116:6789/0 conn(0x7f93a4143200 legacy=0x7f93a4143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 -- 172.21.2.116:0/1598631423 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a40aca70 con 0x7f93a40ac690 2021-11-21T22:02:25.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a37fe700 1 --2- 172.21.2.116:0/1598631423 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a40ac690 0x7f93a40adff0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f938c002f70 tx=0x7f938c006eb0).ready entity=mon.0 client_cookie=a7450ec5b0880e5b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a27fc700 1 -- 172.21.2.116:0/1598631423 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f938c00d610 con 0x7f93a40ac690 2021-11-21T22:02:25.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a27fc700 1 -- 172.21.2.116:0/1598631423 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c000e90 con 0x7f93a40ac690 2021-11-21T22:02:25.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a27fc700 1 -- 172.21.2.116:0/1598631423 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f938c00d9a0 con 0x7f93a40ac690 2021-11-21T22:02:25.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- 172.21.2.116:0/1598631423 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a40ac690 msgr2=0x7f93a40adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/1598631423 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a40ac690 0x7f93a40adff0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f938c002f70 tx=0x7f938c006eb0).stop 2021-11-21T22:02:25.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- 172.21.2.116:0/1598631423 shutdown_connections 2021-11-21T22:02:25.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/1598631423 >> v2:172.21.2.117:3300/0 conn(0x7f93a40b2350 0x7f93a40b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/1598631423 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a40ac690 0x7f93a40adff0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- 172.21.2.116:0/1598631423 >> 172.21.2.116:0/1598631423 conn(0x7f93a405bd20 msgr2=0x7f93a405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:25.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- 172.21.2.116:0/1598631423 shutdown_connections 2021-11-21T22:02:25.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 -- 172.21.2.116:0/1598631423 wait complete. 2021-11-21T22:02:25.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.846+0000 7f93a9cc3700 1 Processor -- start 2021-11-21T22:02:25.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 -- start start 2021-11-21T22:02:25.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f93a40ac690 0x7f93a40a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f93a40b2350 0x7f93a4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a4143200 0x7f93a415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40644c0 con 0x7f93a4143200 2021-11-21T22:02:25.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40ab980 con 0x7f93a40b2350 2021-11-21T22:02:25.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40abc20 con 0x7f93a40ac690 2021-11-21T22:02:25.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a4143200 0x7f93a415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a4143200 0x7f93a415a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52668/0 (socket says 172.21.2.116:52668) 2021-11-21T22:02:25.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 -- 172.21.2.116:0/4108968408 learned_addr learned my addr 172.21.2.116:0/4108968408 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:25.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 -- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f93a40ac690 msgr2=0x7f93a40a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f93a40ac690 0x7f93a40a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 -- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f93a40b2350 msgr2=0x7f93a4159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f93a40b2350 0x7f93a4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 -- 172.21.2.116:0/4108968408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c00b1e0 con 0x7f93a4143200 2021-11-21T22:02:25.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a3fff700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a4143200 0x7f93a415a100 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f93a40685b0 tx=0x7f939c0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a0ff9700 1 -- 172.21.2.116:0/4108968408 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939c006aa0 con 0x7f93a4143200 2021-11-21T22:02:25.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a40675b0 con 0x7f93a4143200 2021-11-21T22:02:25.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a0ff9700 1 -- 172.21.2.116:0/4108968408 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c006c00 con 0x7f93a4143200 2021-11-21T22:02:25.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a0ff9700 1 -- 172.21.2.116:0/4108968408 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939c007030 con 0x7f93a4143200 2021-11-21T22:02:25.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a40aacb0 con 0x7f93a4143200 2021-11-21T22:02:25.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a40a8880 con 0x7f93a4143200 2021-11-21T22:02:25.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a0ff9700 1 -- 172.21.2.116:0/4108968408 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f939c010030 con 0x7f93a4143200 2021-11-21T22:02:25.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a0ff9700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f93880536a0 0x7f9388055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:25.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a0ff9700 1 -- 172.21.2.116:0/4108968408 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f939c06b9c0 con 0x7f93a4143200 2021-11-21T22:02:25.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.850+0000 7f93a37fe700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f93880536a0 0x7f9388055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:25.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.854+0000 7f93a37fe700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f93880536a0 0x7f9388055b60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f938c002f70 tx=0x7f938c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:25.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.854+0000 7f93a0ff9700 1 -- 172.21.2.116:0/4108968408 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f939c0429c0 con 0x7f93a4143200 2021-11-21T22:02:25.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93a40573d0 con 0x7f93a4143200 2021-11-21T22:02:25.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a0ff9700 1 -- 172.21.2.116:0/4108968408 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f939c045fe0 con 0x7f93a4143200 2021-11-21T22:02:25.966 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:25.966 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":60,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:25.201640+0000","modified":"2021-11-21T22:02:25.201685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-11-21T22:02:25.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f93880536a0 msgr2=0x7f9388055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f93880536a0 0x7f9388055b60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f938c002f70 tx=0x7f938c015000).stop 2021-11-21T22:02:25.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a4143200 msgr2=0x7f93a415a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:25.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a4143200 0x7f93a415a100 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f93a40685b0 tx=0x7f939c0013a0).stop 2021-11-21T22:02:25.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 shutdown_connections 2021-11-21T22:02:25.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f93880536a0 0x7f9388055b60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f93a4143200 0x7f93a415a100 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:25.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f93a40b2350 0x7f93a4159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 --2- 172.21.2.116:0/4108968408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f93a40ac690 0x7f93a40a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:25.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 >> 172.21.2.116:0/4108968408 conn(0x7f93a405bd20 msgr2=0x7f93a40b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:25.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 shutdown_connections 2021-11-21T22:02:25.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:25.962+0000 7f93a9cc3700 1 -- 172.21.2.116:0/4108968408 wait complete. 2021-11-21T22:02:25.970 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 60 2021-11-21T22:02:25.973 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:02:26.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 Processor -- start 2021-11-21T22:02:26.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- start start 2021-11-21T22:02:26.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5b4c144810 0x7f5b4c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 0x7f5b4c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c147e80 con 0x7f5b4c0b1db0 2021-11-21T22:02:26.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c0b1490 con 0x7f5b4c144810 2021-11-21T22:02:26.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0af060 con 0x7f5b4c0abce0 2021-11-21T22:02:26.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b51130700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5b4c0b1db0 0x7f5b4c0ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36402/0 (socket says 172.21.2.116:36402) 2021-11-21T22:02:26.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b51130700 1 -- 172.21.2.116:0/2937008811 learned_addr learned my addr 172.21.2.116:0/2937008811 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:26.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 0x7f5b4c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b5092f700 1 --2- 172.21.2.116:0/2937008811 >> v2:172.21.2.117:3300/0 conn(0x7f5b4c144810 0x7f5b4c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132967544 0 0) 0x7f5b4c147e80 con 0x7f5b4c0b1db0 2021-11-21T22:02:26.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b2c003260 con 0x7f5b4c0b1db0 2021-11-21T22:02:26.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801802671 0 0) 0x7f5b2c003260 con 0x7f5b4c0b1db0 2021-11-21T22:02:26.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 msgr2=0x7f5b4c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:26.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 0x7f5b4c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:26.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 >> v2:172.21.2.117:3300/0 conn(0x7f5b4c144810 msgr2=0x7f5b4c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:26.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 --2- 172.21.2.116:0/2937008811 >> v2:172.21.2.117:3300/0 conn(0x7f5b4c144810 0x7f5b4c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:26.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0af5a0 con 0x7f5b4c0b1db0 2021-11-21T22:02:26.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f5b44001a00 con 0x7f5b4c0b1db0 2021-11-21T22:02:26.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 >> v1:172.21.2.116:6789/0 conn(0x7f5b4c0b1db0 legacy=0x7f5b4c0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:26.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 0x7f5b2c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b2c003c80 0x7f5b2c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b2c0049f0 0x7f5b2c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0b1490 con 0x7f5b2c003c80 2021-11-21T22:02:26.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0af060 con 0x7f5b2c0049f0 2021-11-21T22:02:26.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c00b750 con 0x7f5b4c0abce0 2021-11-21T22:02:26.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b2c003c80 0x7f5b2c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b51130700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b2c0049f0 0x7f5b2c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b5092f700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 0x7f5b2c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 -- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 msgr2=0x7f5b2c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:26.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c0abce0 0x7f5b2c003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:26.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 -- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b2c0049f0 msgr2=0x7f5b2c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:26.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b2c0049f0 0x7f5b2c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:26.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 -- 172.21.2.116:0/2937008811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c003260 con 0x7f5b2c003c80 2021-11-21T22:02:26.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b2c003c80 0x7f5b2c004500 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f5b2c0054d0 tx=0x7f5b3c006df0).ready entity=mon.0 client_cookie=9ddd8a317fc3b448 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:26.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b3c00e650 con 0x7f5b2c003c80 2021-11-21T22:02:26.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c00e7b0 con 0x7f5b2c003c80 2021-11-21T22:02:26.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b4b7fe700 1 -- 172.21.2.116:0/2937008811 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b3c00eb10 con 0x7f5b2c003c80 2021-11-21T22:02:26.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/2937008811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b2c003c80 msgr2=0x7f5b2c004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:26.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 --2- 172.21.2.116:0/2937008811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b2c003c80 0x7f5b2c004500 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f5b2c0054d0 tx=0x7f5b3c006df0).stop 2021-11-21T22:02:26.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b51130700 1 -- 172.21.2.116:0/2937008811 reap_dead start 2021-11-21T22:02:26.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/2937008811 shutdown_connections 2021-11-21T22:02:26.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/2937008811 >> 172.21.2.116:0/2937008811 conn(0x7f5b4c05bd20 msgr2=0x7f5b4c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:26.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/2937008811 shutdown_connections 2021-11-21T22:02:26.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/2937008811 wait complete. 2021-11-21T22:02:26.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 Processor -- start 2021-11-21T22:02:26.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.022+0000 7f5b52b93700 1 -- start start 2021-11-21T22:02:26.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b4c145ff0 0x7f5b4c1427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c149d90 0x7f5b4c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b4c14a210 0x7f5b4c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c1463d0 con 0x7f5b4c145ff0 2021-11-21T22:02:26.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0b4b60 con 0x7f5b4c14a210 2021-11-21T22:02:26.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c143a90 con 0x7f5b4c149d90 2021-11-21T22:02:26.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b4c145ff0 0x7f5b4c1427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b4c145ff0 0x7f5b4c1427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52690/0 (socket says 172.21.2.116:52690) 2021-11-21T22:02:26.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 -- 172.21.2.116:0/915118733 learned_addr learned my addr 172.21.2.116:0/915118733 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:26.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c149d90 0x7f5b4c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b51130700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b4c14a210 0x7f5b4c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 -- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c149d90 msgr2=0x7f5b4c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:26.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c149d90 0x7f5b4c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:26.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 -- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b4c14a210 msgr2=0x7f5b4c15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:26.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b4c14a210 0x7f5b4c15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:26.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 -- 172.21.2.116:0/915118733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c1437f0 con 0x7f5b4c145ff0 2021-11-21T22:02:26.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b5092f700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b4c145ff0 0x7f5b4c1427a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f5b3400b000 tx=0x7f5b34008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:26.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b49ffb700 1 -- 172.21.2.116:0/915118733 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b3400e6c0 con 0x7f5b4c145ff0 2021-11-21T22:02:26.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b49ffb700 1 -- 172.21.2.116:0/915118733 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3400e820 con 0x7f5b4c145ff0 2021-11-21T22:02:26.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c0b4660 con 0x7f5b4c145ff0 2021-11-21T22:02:26.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b49ffb700 1 -- 172.21.2.116:0/915118733 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b3400eb80 con 0x7f5b4c145ff0 2021-11-21T22:02:26.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c1469b0 con 0x7f5b4c145ff0 2021-11-21T22:02:26.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c0af300 con 0x7f5b4c145ff0 2021-11-21T22:02:26.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b49ffb700 1 -- 172.21.2.116:0/915118733 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5b34024070 con 0x7f5b4c145ff0 2021-11-21T22:02:26.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b49ffb700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b30053580 0x7f5b30055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:26.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b49ffb700 1 -- 172.21.2.116:0/915118733 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f5b3406c2c0 con 0x7f5b4c145ff0 2021-11-21T22:02:26.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b30053580 0x7f5b30055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:26.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.026+0000 7f5b4bfff700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b30053580 0x7f5b30055a40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f5b3c01b000 tx=0x7f5b3c019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:26.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.030+0000 7f5b49ffb700 1 -- 172.21.2.116:0/915118733 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b340433d0 con 0x7f5b4c145ff0 2021-11-21T22:02:26.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:26.138+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5b4c0573d0 con 0x7f5b4c145ff0 2021-11-21T22:02:27.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b49ffb700 1 -- 172.21.2.116:0/915118733 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5b340469f0 con 0x7f5b4c145ff0 2021-11-21T22:02:27.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b30053580 msgr2=0x7f5b30055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b30053580 0x7f5b30055a40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f5b3c01b000 tx=0x7f5b3c019000).stop 2021-11-21T22:02:27.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b4c145ff0 msgr2=0x7f5b4c1427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b4c145ff0 0x7f5b4c1427a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f5b3400b000 tx=0x7f5b34008040).stop 2021-11-21T22:02:27.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 shutdown_connections 2021-11-21T22:02:27.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5b4c14a210 0x7f5b4c15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5b4c149d90 0x7f5b4c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5b30053580 0x7f5b30055a40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 --2- 172.21.2.116:0/915118733 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5b4c145ff0 0x7f5b4c1427a0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 >> 172.21.2.116:0/915118733 conn(0x7f5b4c05bd20 msgr2=0x7f5b4c1400b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:27.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 shutdown_connections 2021-11-21T22:02:27.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.022+0000 7f5b52b93700 1 -- 172.21.2.116:0/915118733 wait complete. 2021-11-21T22:02:27.031 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:02:27.035 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:27.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b22ed700 1 Processor -- start 2021-11-21T22:02:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b22ed700 1 -- start start 2021-11-21T22:02:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b22ed700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f24ac0aed90 0x7f24ac0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b22ed700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac0b00d0 con 0x7f24ac0af360 2021-11-21T22:02:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b22ed700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac0b2880 con 0x7f24ac0b24a0 2021-11-21T22:02:27.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b22ed700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac0adba0 con 0x7f24ac0aed90 2021-11-21T22:02:27.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b088a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f24ac0af360 0x7f24ac13c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36422/0 (socket says 172.21.2.116:36422) 2021-11-21T22:02:27.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b088a700 1 -- 172.21.2.116:0/1816874848 learned_addr learned my addr 172.21.2.116:0/1816874848 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:27.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156988341 0 0) 0x7f24ac0b00d0 con 0x7f24ac0af360 2021-11-21T22:02:27.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f248c003260 con 0x7f24ac0af360 2021-11-21T22:02:27.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4170955589 0 0) 0x7f248c003260 con 0x7f24ac0af360 2021-11-21T22:02:27.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 >> v2:172.21.2.117:3300/0 conn(0x7f24ac0aed90 msgr2=0x7f24ac0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 --2- 172.21.2.116:0/1816874848 >> v2:172.21.2.117:3300/0 conn(0x7f24ac0aed90 0x7f24ac0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 >> v1:172.21.2.117:6789/0 conn(0x7f24ac0b24a0 legacy=0x7f24ac0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0b14a0 con 0x7f24ac0af360 2021-11-21T22:02:27.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f24a40019e0 con 0x7f24ac0af360 2021-11-21T22:02:27.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 >> v1:172.21.2.116:6789/0 conn(0x7f24ac0af360 legacy=0x7f24ac13c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f248c003820 0x7f248c003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c004500 0x7f248c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f248c005070 0x7f248c005740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0adba0 con 0x7f248c005070 2021-11-21T22:02:27.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c005f30 con 0x7f248c004500 2021-11-21T22:02:27.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0060f0 con 0x7f248c003820 2021-11-21T22:02:27.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.082+0000 7f24b088a700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f248c005070 0x7f248c005740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24abfff700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f248c003820 0x7f248c003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24ab7fe700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c004500 0x7f248c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f248c003820 msgr2=0x7f248c003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f248c003820 0x7f248c003c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c004500 msgr2=0x7f248c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c004500 0x7f248c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/1816874848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c003260 con 0x7f248c005070 2021-11-21T22:02:27.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24abfff700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f248c003820 0x7f248c003c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:27.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24ab7fe700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c004500 0x7f248c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:27.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/1816874848 reap_dead start 2021-11-21T22:02:27.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f248c005070 0x7f248c005740 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f24ac0630d0 tx=0x7f24a40020e0).ready entity=mon.0 client_cookie=bae79ea80d26a178 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:27.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24a400cc60 con 0x7f248c005070 2021-11-21T22:02:27.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a400cdc0 con 0x7f248c005070 2021-11-21T22:02:27.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24aaffd700 1 -- 172.21.2.116:0/1816874848 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24a4005b80 con 0x7f248c005070 2021-11-21T22:02:27.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/1816874848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f248c005070 msgr2=0x7f248c005740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f248c005070 0x7f248c005740 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f24ac0630d0 tx=0x7f24a40020e0).stop 2021-11-21T22:02:27.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/1816874848 shutdown_connections 2021-11-21T22:02:27.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 --2- 172.21.2.116:0/1816874848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f248c005070 0x7f248c005740 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/1816874848 >> 172.21.2.116:0/1816874848 conn(0x7f24ac05fd00 msgr2=0x7f24ac0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:27.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/1816874848 shutdown_connections 2021-11-21T22:02:27.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/1816874848 wait complete. 2021-11-21T22:02:27.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 Processor -- start 2021-11-21T22:02:27.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- start start 2021-11-21T22:02:27.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c005070 0x7f24ac0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24ac063ea0 0x7f24ac066400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24ac10e920 0x7f24ac150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac065d10 con 0x7f24ac10e920 2021-11-21T22:02:27.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0b5400 con 0x7f248c005070 2021-11-21T22:02:27.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac14e530 con 0x7f24ac063ea0 2021-11-21T22:02:27.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24ac10e920 0x7f24ac150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24ac10e920 0x7f24ac150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52708/0 (socket says 172.21.2.116:52708) 2021-11-21T22:02:27.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/2909923665 learned_addr learned my addr 172.21.2.116:0/2909923665 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:27.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24ac063ea0 msgr2=0x7f24ac066400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:27.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24ac063ea0 0x7f24ac066400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c005070 msgr2=0x7f24ac0b47f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:27.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c005070 0x7f24ac0b47f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 -- 172.21.2.116:0/2909923665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac14e7d0 con 0x7f24ac10e920 2021-11-21T22:02:27.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b088a700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24ac10e920 0x7f24ac150f60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f24ac0630d0 tx=0x7f24a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:27.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24a97fa700 1 -- 172.21.2.116:0/2909923665 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24a40020e0 con 0x7f24ac10e920 2021-11-21T22:02:27.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24a97fa700 1 -- 172.21.2.116:0/2909923665 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a40072e0 con 0x7f24ac10e920 2021-11-21T22:02:27.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac065870 con 0x7f24ac10e920 2021-11-21T22:02:27.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24a97fa700 1 -- 172.21.2.116:0/2909923665 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24a4007640 con 0x7f24ac10e920 2021-11-21T22:02:27.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac066c80 con 0x7f24ac10e920 2021-11-21T22:02:27.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac065a30 con 0x7f24ac10e920 2021-11-21T22:02:27.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24a97fa700 1 -- 172.21.2.116:0/2909923665 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f24a401d070 con 0x7f24ac10e920 2021-11-21T22:02:27.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24a97fa700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24900535b0 0x7f2490055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.086+0000 7f24a97fa700 1 -- 172.21.2.116:0/2909923665 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f24a4019030 con 0x7f24ac10e920 2021-11-21T22:02:27.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.090+0000 7f24abfff700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24900535b0 0x7f2490055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.090+0000 7f24abfff700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24900535b0 0x7f2490055a70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f2494008000 tx=0x7f2494009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:27.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.090+0000 7f24a97fa700 1 -- 172.21.2.116:0/2909923665 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24a4043650 con 0x7f24ac10e920 2021-11-21T22:02:27.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.198+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24ac14e960 con 0x7f24ac10e920 2021-11-21T22:02:27.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.198+0000 7f24a97fa700 1 -- 172.21.2.116:0/2909923665 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f24a4046c70 con 0x7f24ac10e920 2021-11-21T22:02:27.205 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:27.205 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":61,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:25.201640+0000","modified":"2021-11-21T22:02:27.013008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-11-21T22:02:27.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24900535b0 msgr2=0x7f2490055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24900535b0 0x7f2490055a70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f2494008000 tx=0x7f2494009040).stop 2021-11-21T22:02:27.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24ac10e920 msgr2=0x7f24ac150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24ac10e920 0x7f24ac150f60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f24ac0630d0 tx=0x7f24a4015000).stop 2021-11-21T22:02:27.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 shutdown_connections 2021-11-21T22:02:27.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24900535b0 0x7f2490055a70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f24ac10e920 0x7f24ac150f60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f24ac063ea0 0x7f24ac066400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 --2- 172.21.2.116:0/2909923665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f248c005070 0x7f24ac0b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 >> 172.21.2.116:0/2909923665 conn(0x7f24ac05fd00 msgr2=0x7f24ac0acac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:27.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 shutdown_connections 2021-11-21T22:02:27.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.202+0000 7f24b22ed700 1 -- 172.21.2.116:0/2909923665 wait complete. 2021-11-21T22:02:27.209 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 61 2021-11-21T22:02:27.214 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:02:27.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb2369d7700 1 Processor -- start 2021-11-21T22:02:27.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb2369d7700 1 -- start start 2021-11-21T22:02:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb2369d7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb230144810 0x7fb2301478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb2369d7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb230147e80 con 0x7fb2300abce0 2021-11-21T22:02:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb2369d7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300af5a0 con 0x7fb2300ac600 2021-11-21T22:02:27.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb2369d7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300b1490 con 0x7fb230144810 2021-11-21T22:02:27.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb22f7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb2300abce0 0x7fb2300b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36442/0 (socket says 172.21.2.116:36442) 2021-11-21T22:02:27.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb22f7fe700 1 -- 172.21.2.116:0/522658496 learned_addr learned my addr 172.21.2.116:0/522658496 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:27.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252768505 0 0) 0x7fb230147e80 con 0x7fb2300abce0 2021-11-21T22:02:27.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.262+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb210003260 con 0x7fb2300abce0 2021-11-21T22:02:27.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108824201 0 0) 0x7fb210003260 con 0x7fb2300abce0 2021-11-21T22:02:27.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb230144810 msgr2=0x7fb2301478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb230144810 0x7fb2301478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 >> v1:172.21.2.117:6789/0 conn(0x7fb2300ac600 legacy=0x7fb2300aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300b1fd0 con 0x7fb2300abce0 2021-11-21T22:02:27.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb218001660 con 0x7fb2300abce0 2021-11-21T22:02:27.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 >> v1:172.21.2.116:6789/0 conn(0x7fb2300abce0 legacy=0x7fb2300b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb230144810 0x7fb210003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb210003e20 0x7fb210004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210004b10 0x7fb210004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300b1490 con 0x7fb210003e20 2021-11-21T22:02:27.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21000b960 con 0x7fb210004b10 2021-11-21T22:02:27.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21000bb20 con 0x7fb230144810 2021-11-21T22:02:27.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb210003e20 0x7fb210004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 -- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb230144810 msgr2=0x7fb210003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb230144810 0x7fb210003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 -- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210004b10 msgr2=0x7fb210004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:27.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210004b10 0x7fb210004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:27.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 -- 172.21.2.116:0/522658496 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210003260 con 0x7fb210003e20 2021-11-21T22:02:27.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb210003e20 0x7fb210004500 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7fb2300ac4e0 tx=0x7fb230146e80).ready entity=mon.0 client_cookie=9ea84819f98c93cd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:27.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb218007270 con 0x7fb210003e20 2021-11-21T22:02:27.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180073d0 con 0x7fb210003e20 2021-11-21T22:02:27.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22effd700 1 -- 172.21.2.116:0/522658496 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb21800cae0 con 0x7fb210003e20 2021-11-21T22:02:27.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- 172.21.2.116:0/522658496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb210003e20 msgr2=0x7fb210004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 --2- 172.21.2.116:0/522658496 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb210003e20 0x7fb210004500 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7fb2300ac4e0 tx=0x7fb230146e80).stop 2021-11-21T22:02:27.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb234f74700 1 -- 172.21.2.116:0/522658496 reap_dead start 2021-11-21T22:02:27.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- 172.21.2.116:0/522658496 shutdown_connections 2021-11-21T22:02:27.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- 172.21.2.116:0/522658496 >> 172.21.2.116:0/522658496 conn(0x7fb23005bd20 msgr2=0x7fb23005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:27.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- 172.21.2.116:0/522658496 shutdown_connections 2021-11-21T22:02:27.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- 172.21.2.116:0/522658496 wait complete. 2021-11-21T22:02:27.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 Processor -- start 2021-11-21T22:02:27.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- start start 2021-11-21T22:02:27.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210003e20 0x7fb2300b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2301563a0 0x7fb23015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2301533a0 0x7fb230153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230147200 con 0x7fb2301563a0 2021-11-21T22:02:27.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230146f60 con 0x7fb210003e20 2021-11-21T22:02:27.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb230159a80 con 0x7fb2301533a0 2021-11-21T22:02:27.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2301563a0 0x7fb23015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2301563a0 0x7fb23015a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52732/0 (socket says 172.21.2.116:52732) 2021-11-21T22:02:27.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 -- 172.21.2.116:0/3772786984 learned_addr learned my addr 172.21.2.116:0/3772786984 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:27.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb234f74700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2301533a0 0x7fb230153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22ffff700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210003e20 0x7fb2300b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 -- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2301533a0 msgr2=0x7fb230153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2301533a0 0x7fb230153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 -- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210003e20 msgr2=0x7fb2300b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:27.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210003e20 0x7fb2300b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:27.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 -- 172.21.2.116:0/3772786984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230159d20 con 0x7fb2301563a0 2021-11-21T22:02:27.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22f7fe700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2301563a0 0x7fb23015a1c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7fb2300ac4e0 tx=0x7fb218015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:27.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22d7fa700 1 -- 172.21.2.116:0/3772786984 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb218001bf0 con 0x7fb2301563a0 2021-11-21T22:02:27.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22d7fa700 1 -- 172.21.2.116:0/3772786984 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218004920 con 0x7fb2301563a0 2021-11-21T22:02:27.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb22d7fa700 1 -- 172.21.2.116:0/3772786984 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb218004c80 con 0x7fb2301563a0 2021-11-21T22:02:27.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb230156820 con 0x7fb2301563a0 2021-11-21T22:02:27.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.266+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb230145f70 con 0x7fb2301563a0 2021-11-21T22:02:27.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.270+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230147160 con 0x7fb2301563a0 2021-11-21T22:02:27.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.270+0000 7fb22d7fa700 1 -- 172.21.2.116:0/3772786984 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb21801d070 con 0x7fb2301563a0 2021-11-21T22:02:27.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.270+0000 7fb22d7fa700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2140536d0 0x7fb214055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:27.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.270+0000 7fb22d7fa700 1 -- 172.21.2.116:0/3772786984 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fb21806c800 con 0x7fb2301563a0 2021-11-21T22:02:27.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.270+0000 7fb22ffff700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2140536d0 0x7fb214055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:27.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.270+0000 7fb22ffff700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2140536d0 0x7fb214055b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb220007000 tx=0x7fb220008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:27.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.270+0000 7fb22d7fa700 1 -- 172.21.2.116:0/3772786984 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb218043800 con 0x7fb2301563a0 2021-11-21T22:02:27.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:27.382+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb230159eb0 con 0x7fb2301563a0 2021-11-21T22:02:28.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb22d7fa700 1 -- 172.21.2.116:0/3772786984 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb218025070 con 0x7fb2301563a0 2021-11-21T22:02:28.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2140536d0 msgr2=0x7fb214055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:28.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2140536d0 0x7fb214055b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb220007000 tx=0x7fb220008040).stop 2021-11-21T22:02:28.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2301563a0 msgr2=0x7fb23015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:28.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2301563a0 0x7fb23015a1c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7fb2300ac4e0 tx=0x7fb218015000).stop 2021-11-21T22:02:28.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 shutdown_connections 2021-11-21T22:02:28.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2301533a0 0x7fb230153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:28.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2140536d0 0x7fb214055b90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:28.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2301563a0 0x7fb23015a1c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:28.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 --2- 172.21.2.116:0/3772786984 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb210003e20 0x7fb2300b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:28.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 >> 172.21.2.116:0/3772786984 conn(0x7fb23005bd20 msgr2=0x7fb2300aca50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:28.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 shutdown_connections 2021-11-21T22:02:28.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.038+0000 7fb2369d7700 1 -- 172.21.2.116:0/3772786984 wait complete. 2021-11-21T22:02:28.049 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:02:28.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55bb094700 1 Processor -- start 2021-11-21T22:02:28.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55bb094700 1 -- start start 2021-11-21T22:02:28.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55bb094700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55b405fbb0 0x7f55b414a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:28.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55bb094700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b40b3de0 con 0x7f55b405ced0 2021-11-21T22:02:28.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55bb094700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b40608c0 con 0x7f55b4098ef0 2021-11-21T22:02:28.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55bb094700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b40b4950 con 0x7f55b405fbb0 2021-11-21T22:02:28.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55b8e30700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f55b405ced0 0x7f55b40b4240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36460/0 (socket says 172.21.2.116:36460) 2021-11-21T22:02:28.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55b3fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55b405fbb0 0x7f55b414a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:28.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55b8e30700 1 -- 172.21.2.116:0/3065201566 learned_addr learned my addr 172.21.2.116:0/3065201566 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:28.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55b3fff700 1 -- 172.21.2.116:0/3065201566 >> v1:172.21.2.117:6789/0 conn(0x7f55b4098ef0 legacy=0x7f55b4099320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:28.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55b3fff700 1 -- 172.21.2.116:0/3065201566 >> v1:172.21.2.116:6789/0 conn(0x7f55b405ced0 legacy=0x7f55b40b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:28.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.098+0000 7f55b3fff700 1 -- 172.21.2.116:0/3065201566 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4060380 con 0x7f55b405fbb0 2021-11-21T22:02:28.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b3fff700 1 --2- 172.21.2.116:0/3065201566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b405fbb0 0x7f55b414a730 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f55b40624e0 tx=0x7f55a8006df0).ready entity=mon.0 client_cookie=546993996d0d940e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:28.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b37fe700 1 -- 172.21.2.116:0/3065201566 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a800d5c0 con 0x7f55b405fbb0 2021-11-21T22:02:28.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b37fe700 1 -- 172.21.2.116:0/3065201566 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a8000e90 con 0x7f55b405fbb0 2021-11-21T22:02:28.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b37fe700 1 -- 172.21.2.116:0/3065201566 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a800d910 con 0x7f55b405fbb0 2021-11-21T22:02:28.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- 172.21.2.116:0/3065201566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b405fbb0 msgr2=0x7f55b414a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:28.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 --2- 172.21.2.116:0/3065201566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b405fbb0 0x7f55b414a730 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f55b40624e0 tx=0x7f55a8006df0).stop 2021-11-21T22:02:28.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- 172.21.2.116:0/3065201566 shutdown_connections 2021-11-21T22:02:28.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 --2- 172.21.2.116:0/3065201566 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b405fbb0 0x7f55b414a730 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:28.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- 172.21.2.116:0/3065201566 >> 172.21.2.116:0/3065201566 conn(0x7f55b40b9da0 msgr2=0x7f55b40b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:28.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- 172.21.2.116:0/3065201566 shutdown_connections 2021-11-21T22:02:28.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- 172.21.2.116:0/3065201566 wait complete. 2021-11-21T22:02:28.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 Processor -- start 2021-11-21T22:02:28.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- start start 2021-11-21T22:02:28.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55b405ced0 0x7f55b4157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:28.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55b405fbb0 0x7f55b40af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:28.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b4098ef0 0x7f55b40b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:28.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40acbc0 con 0x7f55b4098ef0 2021-11-21T22:02:28.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b409c740 con 0x7f55b405fbb0 2021-11-21T22:02:28.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40acdf0 con 0x7f55b405ced0 2021-11-21T22:02:28.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b4098ef0 0x7f55b40b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:28.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b4098ef0 0x7f55b40b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52746/0 (socket says 172.21.2.116:52746) 2021-11-21T22:02:28.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 -- 172.21.2.116:0/1199352142 learned_addr learned my addr 172.21.2.116:0/1199352142 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:28.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 -- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55b405ced0 msgr2=0x7f55b4157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:28.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55b405ced0 0x7f55b4157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:28.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 -- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55b405fbb0 msgr2=0x7f55b40af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:28.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55b405fbb0 0x7f55b40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:28.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 -- 172.21.2.116:0/1199352142 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a800b180 con 0x7f55b4098ef0 2021-11-21T22:02:28.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b9631700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b4098ef0 0x7f55b40b2b50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f55a4002ef0 tx=0x7f55a4006cd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:28.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b1ffb700 1 -- 172.21.2.116:0/1199352142 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a4009e00 con 0x7f55b4098ef0 2021-11-21T22:02:28.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b1ffb700 1 -- 172.21.2.116:0/1199352142 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a4002320 con 0x7f55b4098ef0 2021-11-21T22:02:28.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b409ca00 con 0x7f55b4098ef0 2021-11-21T22:02:28.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b40af220 con 0x7f55b4098ef0 2021-11-21T22:02:28.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.102+0000 7f55b1ffb700 1 -- 172.21.2.116:0/1199352142 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55a4002680 con 0x7f55b4098ef0 2021-11-21T22:02:28.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.106+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b40aefc0 con 0x7f55b4098ef0 2021-11-21T22:02:28.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.106+0000 7f55b1ffb700 1 -- 172.21.2.116:0/1199352142 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f55a400f030 con 0x7f55b4098ef0 2021-11-21T22:02:28.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.106+0000 7f55b1ffb700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5594053610 0x7f5594055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:28.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.106+0000 7f55b1ffb700 1 -- 172.21.2.116:0/1199352142 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f55a406b630 con 0x7f55b4098ef0 2021-11-21T22:02:28.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.106+0000 7f55b8e30700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5594053610 0x7f5594055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:28.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.106+0000 7f55b8e30700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5594053610 0x7f5594055ad0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f55b4062140 tx=0x7f55ac007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:28.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.106+0000 7f55b1ffb700 1 -- 172.21.2.116:0/1199352142 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55a4042740 con 0x7f55b4098ef0 2021-11-21T22:02:28.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:28.190+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f55b4060620 con 0x7f55b4098ef0 2021-11-21T22:02:29.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.042+0000 7f55b1ffb700 1 -- 172.21.2.116:0/1199352142 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) v1 ==== 185+0+0 (secure 0 0 0) 0x7f55a4045d60 con 0x7f55b4098ef0 2021-11-21T22:02:29.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5594053610 msgr2=0x7f5594055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:29.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5594053610 0x7f5594055ad0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f55b4062140 tx=0x7f55ac007a20).stop 2021-11-21T22:02:29.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b4098ef0 msgr2=0x7f55b40b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:29.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b4098ef0 0x7f55b40b2b50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f55a4002ef0 tx=0x7f55a4006cd0).stop 2021-11-21T22:02:29.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 shutdown_connections 2021-11-21T22:02:29.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5594053610 0x7f5594055ad0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:29.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55b4098ef0 0x7f55b40b2b50 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:29.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55b405fbb0 0x7f55b40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:29.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 --2- 172.21.2.116:0/1199352142 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55b405ced0 0x7f55b4157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:29.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 >> 172.21.2.116:0/1199352142 conn(0x7f55b40b9da0 msgr2=0x7f55b40b7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:29.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 shutdown_connections 2021-11-21T22:02:29.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.046+0000 7f55bb094700 1 -- 172.21.2.116:0/1199352142 wait complete. 2021-11-21T22:02:29.052 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:02:29.056 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:02:29.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c860d700 1 Processor -- start 2021-11-21T22:02:29.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c860d700 1 -- start start 2021-11-21T22:02:29.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c860d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb2c00b24a0 0x7fb2c00b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c860d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c00af360 0x7fb2c0069e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c860d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c00b00d0 con 0x7fb2c00ae680 2021-11-21T22:02:29.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c860d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c00adba0 con 0x7fb2c00b24a0 2021-11-21T22:02:29.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c860d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00af7a0 con 0x7fb2c00af360 2021-11-21T22:02:29.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c6baa700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb2c00ae680 0x7fb2c00aeb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36478/0 (socket says 172.21.2.116:36478) 2021-11-21T22:02:29.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c6baa700 1 -- 172.21.2.116:0/1996492591 learned_addr learned my addr 172.21.2.116:0/1996492591 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:29.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c5ba8700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c00af360 0x7fb2c0069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:29.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/1996492591 >> v2:172.21.2.117:3300/0 conn(0x7fb2c00b24a0 0x7fb2c00b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:29.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406786377 0 0) 0x7fb2c00b00d0 con 0x7fb2c00ae680 2021-11-21T22:02:29.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a8003260 con 0x7fb2c00ae680 2021-11-21T22:02:29.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032271372 0 0) 0x7fb2a8003260 con 0x7fb2c00ae680 2021-11-21T22:02:29.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c00af360 msgr2=0x7fb2c0069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:29.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c00af360 0x7fb2c0069e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:29.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 >> v2:172.21.2.117:3300/0 conn(0x7fb2c00b24a0 msgr2=0x7fb2c00b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:29.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 --2- 172.21.2.116:0/1996492591 >> v2:172.21.2.117:3300/0 conn(0x7fb2c00b24a0 0x7fb2c00b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:29.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c00b2880 con 0x7fb2c00ae680 2021-11-21T22:02:29.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb2b80019e0 con 0x7fb2c00ae680 2021-11-21T22:02:29.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 >> v1:172.21.2.116:6789/0 conn(0x7fb2c00ae680 legacy=0x7fb2c00aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:29.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2a80037d0 0x7fb2a8003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c00af360 0x7fb2a8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80049f0 0x7fb2a8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00adba0 con 0x7fb2a80037d0 2021-11-21T22:02:29.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00af7a0 con 0x7fb2a80049f0 2021-11-21T22:02:29.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a800b800 con 0x7fb2c00af360 2021-11-21T22:02:29.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.106+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2a80037d0 0x7fb2a8003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:29.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c63a9700 1 -- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c00af360 msgr2=0x7fb2a8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:29.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c00af360 0x7fb2a8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:29.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c63a9700 1 -- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80049f0 msgr2=0x7fb2a8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:29.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80049f0 0x7fb2a8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:29.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c63a9700 1 -- 172.21.2.116:0/1996492591 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a8003260 con 0x7fb2a80037d0 2021-11-21T22:02:29.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2a80037d0 0x7fb2a8003c10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fb2bc00c000 tx=0x7fb2bc006df0).ready entity=mon.0 client_cookie=38d3be5dc79fdec3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:29.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2bc00e830 con 0x7fb2a80037d0 2021-11-21T22:02:29.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc00e990 con 0x7fb2a80037d0 2021-11-21T22:02:29.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c53a7700 1 -- 172.21.2.116:0/1996492591 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2bc00ecf0 con 0x7fb2a80037d0 2021-11-21T22:02:29.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/1996492591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2a80037d0 msgr2=0x7fb2a8003c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:29.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 --2- 172.21.2.116:0/1996492591 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2a80037d0 0x7fb2a8003c10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fb2bc00c000 tx=0x7fb2bc006df0).stop 2021-11-21T22:02:29.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 -- 172.21.2.116:0/1996492591 reap_dead start 2021-11-21T22:02:29.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/1996492591 shutdown_connections 2021-11-21T22:02:29.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/1996492591 >> 172.21.2.116:0/1996492591 conn(0x7fb2c005fd00 msgr2=0x7fb2c00600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:29.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/1996492591 shutdown_connections 2021-11-21T22:02:29.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/1996492591 wait complete. 2021-11-21T22:02:29.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 Processor -- start 2021-11-21T22:02:29.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- start start 2021-11-21T22:02:29.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80037d0 0x7fb2c00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c0142df0 0x7fb2c01416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2c01428e0 0x7fb2c0141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0066ae0 con 0x7fb2c01428e0 2021-11-21T22:02:29.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0066d80 con 0x7fb2a80037d0 2021-11-21T22:02:29.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0067020 con 0x7fb2c0142df0 2021-11-21T22:02:29.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2c01428e0 0x7fb2c0141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:29.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2c01428e0 0x7fb2c0141b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52766/0 (socket says 172.21.2.116:52766) 2021-11-21T22:02:29.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 -- 172.21.2.116:0/4185273330 learned_addr learned my addr 172.21.2.116:0/4185273330 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:29.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80037d0 0x7fb2c00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:29.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c5ba8700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c0142df0 0x7fb2c01416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:29.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 -- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c0142df0 msgr2=0x7fb2c01416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:29.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c0142df0 0x7fb2c01416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:29.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 -- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80037d0 msgr2=0x7fb2c00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:29.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80037d0 0x7fb2c00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:29.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 -- 172.21.2.116:0/4185273330 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0066820 con 0x7fb2c01428e0 2021-11-21T22:02:29.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c6baa700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2c01428e0 0x7fb2c0141b90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fb2b8001bc0 tx=0x7fb2c013eff0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:29.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2af7fe700 1 -- 172.21.2.116:0/4185273330 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2b8004ab0 con 0x7fb2c01428e0 2021-11-21T22:02:29.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c00b4d40 con 0x7fb2c01428e0 2021-11-21T22:02:29.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2af7fe700 1 -- 172.21.2.116:0/4185273330 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b8004c10 con 0x7fb2c01428e0 2021-11-21T22:02:29.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2af7fe700 1 -- 172.21.2.116:0/4185273330 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2b8007170 con 0x7fb2c01428e0 2021-11-21T22:02:29.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c0065ce0 con 0x7fb2c01428e0 2021-11-21T22:02:29.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.110+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c00671e0 con 0x7fb2c01428e0 2021-11-21T22:02:29.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.114+0000 7fb2af7fe700 1 -- 172.21.2.116:0/4185273330 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb2b801b070 con 0x7fb2c01428e0 2021-11-21T22:02:29.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.114+0000 7fb2af7fe700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2a00536d0 0x7fb2a0055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:29.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.114+0000 7fb2af7fe700 1 -- 172.21.2.116:0/4185273330 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fb2b806b110 con 0x7fb2c01428e0 2021-11-21T22:02:29.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.114+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2a00536d0 0x7fb2a0055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:29.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.114+0000 7fb2c63a9700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2a00536d0 0x7fb2a0055b90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fb2bc01c000 tx=0x7fb2bc019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:29.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.114+0000 7fb2af7fe700 1 -- 172.21.2.116:0/4185273330 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb2b80422c0 con 0x7fb2c01428e0 2021-11-21T22:02:29.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:29.198+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2c00573d0 con 0x7fb2c01428e0 2021-11-21T22:02:30.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.050+0000 7fb2af7fe700 1 -- 172.21.2.116:0/4185273330 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) v1 ==== 173+0+0 (secure 0 0 0) 0x7fb2b80458e0 con 0x7fb2c01428e0 2021-11-21T22:02:30.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2a00536d0 msgr2=0x7fb2a0055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2a00536d0 0x7fb2a0055b90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fb2bc01c000 tx=0x7fb2bc019000).stop 2021-11-21T22:02:30.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2c01428e0 msgr2=0x7fb2c0141b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2c01428e0 0x7fb2c0141b90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fb2b8001bc0 tx=0x7fb2c013eff0).stop 2021-11-21T22:02:30.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 shutdown_connections 2021-11-21T22:02:30.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb2a00536d0 0x7fb2a0055b90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb2c01428e0 0x7fb2c0141b90 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb2c0142df0 0x7fb2c01416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 --2- 172.21.2.116:0/4185273330 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb2a80037d0 0x7fb2c00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 >> 172.21.2.116:0/4185273330 conn(0x7fb2c005fd00 msgr2=0x7fb2c013d820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:30.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 shutdown_connections 2021-11-21T22:02:30.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.054+0000 7fb2c860d700 1 -- 172.21.2.116:0/4185273330 wait complete. 2021-11-21T22:02:30.061 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:02:30.068 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T22:02:30.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 Processor -- start 2021-11-21T22:02:30.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 -- start start 2021-11-21T22:02:30.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6454144810 0x7f64541478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f64540abce0 0x7f64540b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540b1db0 0x7f64540ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6454147e80 con 0x7f6454144810 2021-11-21T22:02:30.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64540af5a0 con 0x7f64540abce0 2021-11-21T22:02:30.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.114+0000 7f64589c9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64540b1490 con 0x7f64540b1db0 2021-11-21T22:02:30.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6454144810 0x7f64541478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6454144810 0x7f64541478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52770/0 (socket says 172.21.2.116:52770) 2021-11-21T22:02:30.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 -- 172.21.2.116:0/4130244530 learned_addr learned my addr 172.21.2.116:0/4130244530 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:30.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 -- 172.21.2.116:0/4130244530 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540b1db0 msgr2=0x7f64540ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:30.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 --2- 172.21.2.116:0/4130244530 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540b1db0 0x7f64540ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 -- 172.21.2.116:0/4130244530 >> v2:172.21.2.117:3300/0 conn(0x7f64540abce0 msgr2=0x7f64540b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:30.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 --2- 172.21.2.116:0/4130244530 >> v2:172.21.2.117:3300/0 conn(0x7f64540abce0 0x7f64540b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 -- 172.21.2.116:0/4130244530 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540af060 con 0x7f6454144810 2021-11-21T22:02:30.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645259c700 1 --2- 172.21.2.116:0/4130244530 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454144810 0x7f64541478c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f6444002f70 tx=0x7f6444006eb0).ready entity=mon.0 client_cookie=ec8d060e8ae0f078 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:30.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645159a700 1 -- 172.21.2.116:0/4130244530 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f644400d570 con 0x7f6454144810 2021-11-21T22:02:30.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645159a700 1 -- 172.21.2.116:0/4130244530 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444000e90 con 0x7f6454144810 2021-11-21T22:02:30.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f645159a700 1 -- 172.21.2.116:0/4130244530 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f644400d900 con 0x7f6454144810 2021-11-21T22:02:30.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- 172.21.2.116:0/4130244530 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454144810 msgr2=0x7f64541478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 --2- 172.21.2.116:0/4130244530 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454144810 0x7f64541478c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f6444002f70 tx=0x7f6444006eb0).stop 2021-11-21T22:02:30.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- 172.21.2.116:0/4130244530 shutdown_connections 2021-11-21T22:02:30.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 --2- 172.21.2.116:0/4130244530 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540b1db0 0x7f64540ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 --2- 172.21.2.116:0/4130244530 >> v2:172.21.2.117:3300/0 conn(0x7f64540abce0 0x7f64540b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 --2- 172.21.2.116:0/4130244530 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454144810 0x7f64541478c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- 172.21.2.116:0/4130244530 >> 172.21.2.116:0/4130244530 conn(0x7f645405bd20 msgr2=0x7f645405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:30.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- 172.21.2.116:0/4130244530 shutdown_connections 2021-11-21T22:02:30.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- 172.21.2.116:0/4130244530 wait complete. 2021-11-21T22:02:30.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 Processor -- start 2021-11-21T22:02:30.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- start start 2021-11-21T22:02:30.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540abce0 0x7f6454140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6454144810 0x7f6454156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454156fe0 0x7f64540b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64541564a0 con 0x7f6454156fe0 2021-11-21T22:02:30.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64540b4b20 con 0x7f6454144810 2021-11-21T22:02:30.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64540b42f0 con 0x7f64540abce0 2021-11-21T22:02:30.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454156fe0 0x7f64540b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454156fe0 0x7f64540b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52774/0 (socket says 172.21.2.116:52774) 2021-11-21T22:02:30.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 -- 172.21.2.116:0/598869548 learned_addr learned my addr 172.21.2.116:0/598869548 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:30.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 -- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540abce0 msgr2=0x7f6454140160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:30.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540abce0 0x7f6454140160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 -- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6454144810 msgr2=0x7f6454156af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:30.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6454144810 0x7f6454156af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 -- 172.21.2.116:0/598869548 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644400b1e0 con 0x7f6454156fe0 2021-11-21T22:02:30.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f6452d9d700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454156fe0 0x7f64540b30d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f644c0032f0 tx=0x7f644c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:30.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64437fe700 1 -- 172.21.2.116:0/598869548 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f644c005c30 con 0x7f6454156fe0 2021-11-21T22:02:30.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64437fe700 1 -- 172.21.2.116:0/598869548 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c005d90 con 0x7f6454156fe0 2021-11-21T22:02:30.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64437fe700 1 -- 172.21.2.116:0/598869548 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f644c006220 con 0x7f6454156fe0 2021-11-21T22:02:30.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6454154150 con 0x7f6454156fe0 2021-11-21T22:02:30.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.118+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64540b4050 con 0x7f6454156fe0 2021-11-21T22:02:30.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.122+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6454156830 con 0x7f6454156fe0 2021-11-21T22:02:30.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.122+0000 7f64437fe700 1 -- 172.21.2.116:0/598869548 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f644c00f030 con 0x7f6454156fe0 2021-11-21T22:02:30.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.122+0000 7f64437fe700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f64300536a0 0x7f6430055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.122+0000 7f64437fe700 1 -- 172.21.2.116:0/598869548 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f644c06b720 con 0x7f6454156fe0 2021-11-21T22:02:30.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.122+0000 7f645259c700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f64300536a0 0x7f6430055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.122+0000 7f645259c700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f64300536a0 0x7f6430055b60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f6444002f70 tx=0x7f6444015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:30.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.122+0000 7f64437fe700 1 -- 172.21.2.116:0/598869548 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6454156830 con 0x7f6454156fe0 2021-11-21T22:02:30.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.210+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6454156830 con 0x7f6454156fe0 2021-11-21T22:02:30.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64437fe700 1 -- 172.21.2.116:0/598869548 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6454156830 con 0x7f6454156fe0 2021-11-21T22:02:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f64300536a0 msgr2=0x7f6430055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f64300536a0 0x7f6430055b60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f6444002f70 tx=0x7f6444015000).stop 2021-11-21T22:02:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454156fe0 msgr2=0x7f64540b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454156fe0 0x7f64540b30d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f644c0032f0 tx=0x7f644c0091a0).stop 2021-11-21T22:02:30.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 shutdown_connections 2021-11-21T22:02:30.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f64300536a0 0x7f6430055b60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6454156fe0 0x7f64540b30d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6454144810 0x7f6454156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 --2- 172.21.2.116:0/598869548 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f64540abce0 0x7f6454140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 >> 172.21.2.116:0/598869548 conn(0x7f645405bd20 msgr2=0x7f6454141860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 shutdown_connections 2021-11-21T22:02:30.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.214+0000 7f64589c9700 1 -- 172.21.2.116:0/598869548 wait complete. 2021-11-21T22:02:30.227 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2021-11-21T22:02:30.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 Processor -- start 2021-11-21T22:02:30.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- start start 2021-11-21T22:02:30.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c13fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa46c0ac750 0x7fa46c1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46c13f170 con 0x7fa46c0ad340 2021-11-21T22:02:30.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46c0a8b70 con 0x7fa46c0ac750 2021-11-21T22:02:30.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c0af530 con 0x7fa46c0a8790 2021-11-21T22:02:30.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa46c0ac750 0x7fa46c1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa46c0ac750 0x7fa46c1499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52784/0 (socket says 172.21.2.116:52784) 2021-11-21T22:02:30.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 -- 172.21.2.116:0/2170217550 learned_addr learned my addr 172.21.2.116:0/2170217550 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:30.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa472099700 1 --2- 172.21.2.116:0/2170217550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c13fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa471097700 1 -- 172.21.2.116:0/2170217550 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840338473 0 0) 0x7fa46c13f170 con 0x7fa46c0ad340 2021-11-21T22:02:30.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa471097700 1 -- 172.21.2.116:0/2170217550 --> v1:172.21.2.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa454003260 con 0x7fa46c0ad340 2021-11-21T22:02:30.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 -- 172.21.2.116:0/2170217550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 msgr2=0x7fa46c13fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:30.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 --2- 172.21.2.116:0/2170217550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c13fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 -- 172.21.2.116:0/2170217550 >> v1:172.21.2.117:6789/0 conn(0x7fa46c0ad340 legacy=0x7fa46c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 -- 172.21.2.116:0/2170217550 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c062410 con 0x7fa46c0ac750 2021-11-21T22:02:30.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa47289a700 1 --2- 172.21.2.116:0/2170217550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c1499d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fa468002f70 tx=0x7fa468006f40).ready entity=mon.0 client_cookie=d776dce89b6a4235 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:30.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa471097700 1 -- 172.21.2.116:0/2170217550 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46800d9a0 con 0x7fa46c0ac750 2021-11-21T22:02:30.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa471097700 1 -- 172.21.2.116:0/2170217550 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa468000e90 con 0x7fa46c0ac750 2021-11-21T22:02:30.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa471097700 1 -- 172.21.2.116:0/2170217550 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46800dd30 con 0x7fa46c0ac750 2021-11-21T22:02:30.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- 172.21.2.116:0/2170217550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 msgr2=0x7fa46c1499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 --2- 172.21.2.116:0/2170217550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c1499d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fa468002f70 tx=0x7fa468006f40).stop 2021-11-21T22:02:30.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- 172.21.2.116:0/2170217550 shutdown_connections 2021-11-21T22:02:30.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 --2- 172.21.2.116:0/2170217550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c1499d0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 --2- 172.21.2.116:0/2170217550 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c13fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- 172.21.2.116:0/2170217550 >> 172.21.2.116:0/2170217550 conn(0x7fa46c05bd20 msgr2=0x7fa46c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:30.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.274+0000 7fa4742fd700 1 -- 172.21.2.116:0/2170217550 shutdown_connections 2021-11-21T22:02:30.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- 172.21.2.116:0/2170217550 wait complete. 2021-11-21T22:02:30.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 Processor -- start 2021-11-21T22:02:30.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- start start 2021-11-21T22:02:30.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa46c0ad340 0x7fa46c14db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c151f90 con 0x7fa46c0ac750 2021-11-21T22:02:30.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c0a9370 con 0x7fa46c0ad340 2021-11-21T22:02:30.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c063d20 con 0x7fa46c0a8790 2021-11-21T22:02:30.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52792/0 (socket says 172.21.2.116:52792) 2021-11-21T22:02:30.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 -- 172.21.2.116:0/3503344172 learned_addr learned my addr 172.21.2.116:0/3503344172 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:30.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa472099700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa47289a700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa46c0ad340 0x7fa46c14db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 -- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 msgr2=0x7fa46c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 -- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa46c0ad340 msgr2=0x7fa46c14db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:30.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa46c0ad340 0x7fa46c14db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:30.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 -- 172.21.2.116:0/3503344172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46800b460 con 0x7fa46c0ac750 2021-11-21T22:02:30.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa471898700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c151aa0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7fa464001490 tx=0x7fa464007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:30.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa45b7fe700 1 -- 172.21.2.116:0/3503344172 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa464002310 con 0x7fa46c0ac750 2021-11-21T22:02:30.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa45b7fe700 1 -- 172.21.2.116:0/3503344172 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464002470 con 0x7fa46c0ac750 2021-11-21T22:02:30.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c063fc0 con 0x7fa46c0ac750 2021-11-21T22:02:30.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa45b7fe700 1 -- 172.21.2.116:0/3503344172 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4640017a0 con 0x7fa46c0ac750 2021-11-21T22:02:30.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c064500 con 0x7fa46c0ac750 2021-11-21T22:02:30.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46c0646c0 con 0x7fa46c0ac750 2021-11-21T22:02:30.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa45b7fe700 1 -- 172.21.2.116:0/3503344172 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa464010030 con 0x7fa46c0ac750 2021-11-21T22:02:30.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa45b7fe700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa44c0534f0 0x7fa44c0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:30.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa45b7fe700 1 -- 172.21.2.116:0/3503344172 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa46406b650 con 0x7fa46c0ac750 2021-11-21T22:02:30.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa472099700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa44c0534f0 0x7fa44c0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:30.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.278+0000 7fa472099700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa44c0534f0 0x7fa44c0559b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fa45c00b000 tx=0x7fa45c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:30.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.282+0000 7fa45b7fe700 1 -- 172.21.2.116:0/3503344172 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa464042a90 con 0x7fa46c0ac750 2021-11-21T22:02:30.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:30.394+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7fa46c13f3a0 con 0x7fa46c0ac750 2021-11-21T22:02:31.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.058+0000 7fa45b7fe700 1 -- 172.21.2.116:0/3503344172 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v358) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa4640460b0 con 0x7fa46c0ac750 2021-11-21T22:02:31.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa44c0534f0 msgr2=0x7fa44c0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:31.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa44c0534f0 0x7fa44c0559b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fa45c00b000 tx=0x7fa45c006df0).stop 2021-11-21T22:02:31.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 msgr2=0x7fa46c151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:31.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c151aa0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7fa464001490 tx=0x7fa464007de0).stop 2021-11-21T22:02:31.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 shutdown_connections 2021-11-21T22:02:31.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa46c0ad340 0x7fa46c14db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:31.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa44c0534f0 0x7fa44c0559b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:31.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa46c0ac750 0x7fa46c151aa0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:31.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 --2- 172.21.2.116:0/3503344172 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa46c0a8790 0x7fa46c0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:31.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 >> 172.21.2.116:0/3503344172 conn(0x7fa46c05bd20 msgr2=0x7fa46c13da50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:31.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 shutdown_connections 2021-11-21T22:02:31.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.062+0000 7fa4742fd700 1 -- 172.21.2.116:0/3503344172 wait complete. 2021-11-21T22:02:31.072 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2021-11-21T22:02:31.073 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:02:31.074 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2021-11-21T22:02:31.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 Processor -- start 2021-11-21T22:02:31.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- start start 2021-11-21T22:02:31.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff9b40a6aa0 0x7ff9b414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:31.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff9b40a86b0 0x7ff9b41470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:31.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40a6e80 con 0x7ff9b41475d0 2021-11-21T22:02:31.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40a8bc0 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b414a7b0 con 0x7ff9b40a86b0 2021-11-21T22:02:31.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff9b40a6aa0 0x7ff9b414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:31.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff9b40a6aa0 0x7ff9b414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52796/0 (socket says 172.21.2.116:52796) 2021-11-21T22:02:31.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 -- 172.21.2.116:0/3476851284 learned_addr learned my addr 172.21.2.116:0/3476851284 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:31.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 -- 172.21.2.116:0/3476851284 >> v2:172.21.2.117:3300/0 conn(0x7ff9b40a86b0 msgr2=0x7ff9b41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:31.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 --2- 172.21.2.116:0/3476851284 >> v2:172.21.2.117:3300/0 conn(0x7ff9b40a86b0 0x7ff9b41470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:31.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 -- 172.21.2.116:0/3476851284 >> v1:172.21.2.117:6789/0 conn(0x7ff9b41475d0 legacy=0x7ff9b41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:31.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 -- 172.21.2.116:0/3476851284 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b414aa10 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b259c700 1 --2- 172.21.2.116:0/3476851284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b414b150 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7ff9b4061ab0 tx=0x7ff99c006eb0).ready entity=mon.0 client_cookie=ecb2ea26798d278f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:31.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b159a700 1 -- 172.21.2.116:0/3476851284 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c00d5c0 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b159a700 1 -- 172.21.2.116:0/3476851284 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c000e90 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b159a700 1 -- 172.21.2.116:0/3476851284 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c00d910 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/3476851284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 msgr2=0x7ff9b414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:31.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/3476851284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b414b150 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7ff9b4061ab0 tx=0x7ff99c006eb0).stop 2021-11-21T22:02:31.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/3476851284 shutdown_connections 2021-11-21T22:02:31.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/3476851284 >> v2:172.21.2.117:3300/0 conn(0x7ff9b40a86b0 0x7ff9b41470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:31.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/3476851284 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b414b150 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:31.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/3476851284 >> 172.21.2.116:0/3476851284 conn(0x7ff9b405bd20 msgr2=0x7ff9b405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:31.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/3476851284 shutdown_connections 2021-11-21T22:02:31.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.122+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/3476851284 wait complete. 2021-11-21T22:02:31.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 Processor -- start 2021-11-21T22:02:31.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 -- start start 2021-11-21T22:02:31.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:31.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40a86b0 0x7ff9b413e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:31.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b41475d0 0x7ff9b413e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:31.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40a9340 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b415a9a0 con 0x7ff9b41475d0 2021-11-21T22:02:31.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b415a470 con 0x7ff9b40a86b0 2021-11-21T22:02:31.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:31.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b40b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52802/0 (socket says 172.21.2.116:52802) 2021-11-21T22:02:31.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 -- 172.21.2.116:0/1308233682 learned_addr learned my addr 172.21.2.116:0/1308233682 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:31.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 -- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40a86b0 msgr2=0x7ff9b413e160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:31.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40a86b0 0x7ff9b413e160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:31.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 -- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b41475d0 msgr2=0x7ff9b413e650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:31.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b41475d0 0x7ff9b413e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:31.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 -- 172.21.2.116:0/1308233682 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c00b1e0 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b259c700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b40b3f50 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7ff9b415ab30 tx=0x7ff99c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:31.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9ab7fe700 1 -- 172.21.2.116:0/1308233682 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c00de70 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9ab7fe700 1 -- 172.21.2.116:0/1308233682 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c002f50 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b40a9570 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9ab7fe700 1 -- 172.21.2.116:0/1308233682 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c011e90 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b40abf90 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b40a9af0 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9ab7fe700 1 -- 172.21.2.116:0/1308233682 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff99c025030 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9ab7fe700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053640 0x7ff998055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:31.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9ab7fe700 1 -- 172.21.2.116:0/1308233682 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff99c06c810 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b1d9b700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053640 0x7ff998055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:31.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.126+0000 7ff9b1d9b700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053640 0x7ff998055b00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff9a4002f70 tx=0x7ff9a4006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:31.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.130+0000 7ff9ab7fe700 1 -- 172.21.2.116:0/1308233682 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff99c043c50 con 0x7ff9b40a6aa0 2021-11-21T22:02:31.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:31.238+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7ff9b40573d0 con 0x7ff9b40a6aa0 2021-11-21T22:02:32.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9ab7fe700 1 -- 172.21.2.116:0/1308233682 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v359) v1 ==== 131+0+0 (secure 0 0 0) 0x7ff99c023020 con 0x7ff9b40a6aa0 2021-11-21T22:02:32.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053640 msgr2=0x7ff998055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053640 0x7ff998055b00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff9a4002f70 tx=0x7ff9a4006a00).stop 2021-11-21T22:02:32.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 msgr2=0x7ff9b40b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b40b3f50 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7ff9b415ab30 tx=0x7ff99c015000).stop 2021-11-21T22:02:32.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 shutdown_connections 2021-11-21T22:02:32.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff9b41475d0 0x7ff9b413e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff9b40a86b0 0x7ff9b413e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff998053640 0x7ff998055b00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 --2- 172.21.2.116:0/1308233682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff9b40a6aa0 0x7ff9b40b3f50 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 >> 172.21.2.116:0/1308233682 conn(0x7ff9b405bd20 msgr2=0x7ff9b41441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.070+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 shutdown_connections 2021-11-21T22:02:32.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.074+0000 7ff9b8c2a700 1 -- 172.21.2.116:0/1308233682 wait complete. 2021-11-21T22:02:32.083 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T22:02:32.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2679b6d700 1 Processor -- start 2021-11-21T22:02:32.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2679b6d700 1 -- start start 2021-11-21T22:02:32.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2679b6d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f2674144810 0x7f26741478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2679b6d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f26740ac600 0x7f26740aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2679b6d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2674147e80 con 0x7f26740abce0 2021-11-21T22:02:32.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2679b6d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26740b1490 con 0x7f26740ac600 2021-11-21T22:02:32.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2679b6d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26740b1fd0 con 0x7f2674144810 2021-11-21T22:02:32.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f26740ac600 0x7f26740aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f26740ac600 0x7f26740aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52812/0 (socket says 172.21.2.116:52812) 2021-11-21T22:02:32.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 -- 172.21.2.116:0/658440137 learned_addr learned my addr 172.21.2.116:0/658440137 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:32.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 -- 172.21.2.116:0/658440137 >> v2:172.21.2.117:3300/0 conn(0x7f2674144810 msgr2=0x7f26741478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:32.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 --2- 172.21.2.116:0/658440137 >> v2:172.21.2.117:3300/0 conn(0x7f2674144810 0x7f26741478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 -- 172.21.2.116:0/658440137 >> v1:172.21.2.117:6789/0 conn(0x7f26740abce0 legacy=0x7f26740b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:32.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 -- 172.21.2.116:0/658440137 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26740af5a0 con 0x7f26740ac600 2021-11-21T22:02:32.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f2673fff700 1 --2- 172.21.2.116:0/658440137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740ac600 0x7f26740aca50 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f26740b4e00 tx=0x7f266c009290).ready entity=mon.0 client_cookie=55d6103dda31ca91 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.130+0000 7f26727fc700 1 -- 172.21.2.116:0/658440137 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266c00b830 con 0x7f26740ac600 2021-11-21T22:02:32.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26727fc700 1 -- 172.21.2.116:0/658440137 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c00b990 con 0x7f26740ac600 2021-11-21T22:02:32.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26727fc700 1 -- 172.21.2.116:0/658440137 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266c00bb40 con 0x7f26740ac600 2021-11-21T22:02:32.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/658440137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740ac600 msgr2=0x7f26740aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 --2- 172.21.2.116:0/658440137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740ac600 0x7f26740aca50 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f26740b4e00 tx=0x7f266c009290).stop 2021-11-21T22:02:32.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/658440137 shutdown_connections 2021-11-21T22:02:32.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 --2- 172.21.2.116:0/658440137 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740ac600 0x7f26740aca50 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 --2- 172.21.2.116:0/658440137 >> v2:172.21.2.117:3300/0 conn(0x7f2674144810 0x7f26741478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/658440137 >> 172.21.2.116:0/658440137 conn(0x7f267405bd20 msgr2=0x7f267405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/658440137 shutdown_connections 2021-11-21T22:02:32.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/658440137 wait complete. 2021-11-21T22:02:32.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 Processor -- start 2021-11-21T22:02:32.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- start start 2021-11-21T22:02:32.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740abce0 0x7f2674145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26740ac600 0x7f267413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2674144810 0x7f267413cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267414acf0 con 0x7f26740abce0 2021-11-21T22:02:32.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26740b3be0 con 0x7f26740ac600 2021-11-21T22:02:32.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26740b42b0 con 0x7f2674144810 2021-11-21T22:02:32.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740abce0 0x7f2674145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740abce0 0x7f2674145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52818/0 (socket says 172.21.2.116:52818) 2021-11-21T22:02:32.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 -- 172.21.2.116:0/613218917 learned_addr learned my addr 172.21.2.116:0/613218917 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:32.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 -- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2674144810 msgr2=0x7f267413cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2674144810 0x7f267413cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 -- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26740ac600 msgr2=0x7f267413c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26740ac600 0x7f267413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 -- 172.21.2.116:0/613218917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c009530 con 0x7f26740abce0 2021-11-21T22:02:32.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f26737fe700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740abce0 0x7f2674145ff0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f2674156660 tx=0x7f265c006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2670ff9700 1 -- 172.21.2.116:0/613218917 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f265c00e470 con 0x7f26740abce0 2021-11-21T22:02:32.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2670ff9700 1 -- 172.21.2.116:0/613218917 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c00e5d0 con 0x7f26740abce0 2021-11-21T22:02:32.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267414a1f0 con 0x7f26740abce0 2021-11-21T22:02:32.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2670ff9700 1 -- 172.21.2.116:0/613218917 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f265c00e930 con 0x7f26740abce0 2021-11-21T22:02:32.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26740674d0 con 0x7f26740abce0 2021-11-21T22:02:32.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26740af060 con 0x7f26740abce0 2021-11-21T22:02:32.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2670ff9700 1 -- 172.21.2.116:0/613218917 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f265c018030 con 0x7f26740abce0 2021-11-21T22:02:32.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2670ff9700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2658053580 0x7f2658055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2670ff9700 1 -- 172.21.2.116:0/613218917 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f265c06ad10 con 0x7f26740abce0 2021-11-21T22:02:32.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.134+0000 7f2672ffd700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2658053580 0x7f2658055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.138+0000 7f2672ffd700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2658053580 0x7f2658055a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f2664002f90 tx=0x7f26640069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.138+0000 7f2670ff9700 1 -- 172.21.2.116:0/613218917 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f265c042080 con 0x7f26740abce0 2021-11-21T22:02:32.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.222+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f26740573d0 con 0x7f26740abce0 2021-11-21T22:02:32.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2670ff9700 1 -- 172.21.2.116:0/613218917 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f265c0456a0 con 0x7f26740abce0 2021-11-21T22:02:32.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2658053580 msgr2=0x7f2658055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2658053580 0x7f2658055a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f2664002f90 tx=0x7f26640069b0).stop 2021-11-21T22:02:32.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740abce0 msgr2=0x7f2674145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740abce0 0x7f2674145ff0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f2674156660 tx=0x7f265c006e30).stop 2021-11-21T22:02:32.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 shutdown_connections 2021-11-21T22:02:32.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2674144810 0x7f267413cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f26740ac600 0x7f267413c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2658053580 0x7f2658055a40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 --2- 172.21.2.116:0/613218917 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f26740abce0 0x7f2674145ff0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 >> 172.21.2.116:0/613218917 conn(0x7f267405bd20 msgr2=0x7f26740b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 shutdown_connections 2021-11-21T22:02:32.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.230+0000 7f2679b6d700 1 -- 172.21.2.116:0/613218917 wait complete. 2021-11-21T22:02:32.241 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:02:32.242 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:32.250 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:32.263 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:32.276 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:32.290 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:02:32.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:32.291 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:32.291 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:32.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:32.322 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:02:32.323 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:32.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:32.324 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:32.324 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:32.333 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:02:32.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:32.335 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:32.335 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:02:32.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.386+0000 7efebe043700 1 Processor -- start 2021-11-21T22:02:32.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.386+0000 7efebe043700 1 -- start start 2021-11-21T22:02:32.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb80b3320 0x7efeb80ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7efeb81443a0 0x7efeb8144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb80b29b0 con 0x7efeb8067850 2021-11-21T22:02:32.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb8067c30 con 0x7efeb81443a0 2021-11-21T22:02:32.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80b3780 con 0x7efeb80b3320 2021-11-21T22:02:32.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7efeb81443a0 0x7efeb8144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb77fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7efeb8067850 0x7efeb80b7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36548/0 (socket says 172.21.2.116:36548) 2021-11-21T22:02:32.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb7fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7efeb81443a0 0x7efeb8144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52826/0 (socket says 172.21.2.116:52826) 2021-11-21T22:02:32.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb77fe700 1 -- 172.21.2.116:0/612154736 learned_addr learned my addr 172.21.2.116:0/612154736 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:32.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb7fff700 1 -- 172.21.2.116:0/612154736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb80b3320 msgr2=0x7efeb80ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:32.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb7fff700 1 --2- 172.21.2.116:0/612154736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb80b3320 0x7efeb80ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb7fff700 1 -- 172.21.2.116:0/612154736 >> v1:172.21.2.116:6789/0 conn(0x7efeb8067850 legacy=0x7efeb80b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb7fff700 1 -- 172.21.2.116:0/612154736 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb80b7910 con 0x7efeb81443a0 2021-11-21T22:02:32.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb7fff700 1 --2- 172.21.2.116:0/612154736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb81443a0 0x7efeb8144830 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7efeb00032f0 tx=0x7efeb00091d0).ready entity=mon.0 client_cookie=ab16c8609d220a14 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb67fc700 1 -- 172.21.2.116:0/612154736 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efeb0009ea0 con 0x7efeb81443a0 2021-11-21T22:02:32.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb67fc700 1 -- 172.21.2.116:0/612154736 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb000b8a0 con 0x7efeb81443a0 2021-11-21T22:02:32.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb67fc700 1 -- 172.21.2.116:0/612154736 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efeb000ba10 con 0x7efeb81443a0 2021-11-21T22:02:32.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- 172.21.2.116:0/612154736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb81443a0 msgr2=0x7efeb8144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- 172.21.2.116:0/612154736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb81443a0 0x7efeb8144830 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7efeb00032f0 tx=0x7efeb00091d0).stop 2021-11-21T22:02:32.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- 172.21.2.116:0/612154736 shutdown_connections 2021-11-21T22:02:32.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- 172.21.2.116:0/612154736 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb81443a0 0x7efeb8144830 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- 172.21.2.116:0/612154736 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb80b3320 0x7efeb80ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- 172.21.2.116:0/612154736 >> 172.21.2.116:0/612154736 conn(0x7efeb8064d60 msgr2=0x7efeb8065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- 172.21.2.116:0/612154736 shutdown_connections 2021-11-21T22:02:32.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- 172.21.2.116:0/612154736 wait complete. 2021-11-21T22:02:32.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 Processor -- start 2021-11-21T22:02:32.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- start start 2021-11-21T22:02:32.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb8067850 0x7efeb80b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb80b3320 0x7efeb809e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efeb81443a0 0x7efeb809e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80b15c0 con 0x7efeb80b3320 2021-11-21T22:02:32.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8143ce0 con 0x7efeb81443a0 2021-11-21T22:02:32.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb809ee70 con 0x7efeb8067850 2021-11-21T22:02:32.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb80b3320 0x7efeb809e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb80b3320 0x7efeb809e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52832/0 (socket says 172.21.2.116:52832) 2021-11-21T22:02:32.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 -- 172.21.2.116:0/3662804640 learned_addr learned my addr 172.21.2.116:0/3662804640 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:32.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 -- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb8067850 msgr2=0x7efeb80b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb8067850 0x7efeb80b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 -- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efeb81443a0 msgr2=0x7efeb809e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efeb81443a0 0x7efeb809e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 -- 172.21.2.116:0/3662804640 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0009470 con 0x7efeb80b3320 2021-11-21T22:02:32.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb6ffd700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb80b3320 0x7efeb809e490 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7efeb809efb0 tx=0x7efea8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb4ff9700 1 -- 172.21.2.116:0/3662804640 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea8013460 con 0x7efeb80b3320 2021-11-21T22:02:32.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efeb4ff9700 1 -- 172.21.2.116:0/3662804640 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea80135c0 con 0x7efeb80b3320 2021-11-21T22:02:32.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.390+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb809f510 con 0x7efeb80b3320 2021-11-21T22:02:32.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efeb4ff9700 1 -- 172.21.2.116:0/3662804640 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea8013920 con 0x7efeb80b3320 2021-11-21T22:02:32.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb809dff0 con 0x7efeb80b3320 2021-11-21T22:02:32.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb80b2be0 con 0x7efeb80b3320 2021-11-21T22:02:32.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efeb4ff9700 1 -- 172.21.2.116:0/3662804640 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7efea800f070 con 0x7efeb80b3320 2021-11-21T22:02:32.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efeb4ff9700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efe98053610 0x7efe98055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efeb4ff9700 1 -- 172.21.2.116:0/3662804640 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efea806b0e0 con 0x7efeb80b3320 2021-11-21T22:02:32.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efeb77fe700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efe98053610 0x7efe98055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efeb77fe700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efe98053610 0x7efe98055ad0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7efeb8066f90 tx=0x7efeac007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.394+0000 7efeb4ff9700 1 -- 172.21.2.116:0/3662804640 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efea8042410 con 0x7efeb80b3320 2021-11-21T22:02:32.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.478+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efeb809f370 con 0x7efeb80b3320 2021-11-21T22:02:32.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.478+0000 7efeb4ff9700 1 -- 172.21.2.116:0/3662804640 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v136) v1 ==== 81+0+28837 (secure 0 0 0) 0x7efeb809f370 con 0x7efeb80b3320 2021-11-21T22:02:32.484 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:32.484 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:32.485 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 136, 2021-11-21T22:02:32.485 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:02:32.485 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:02:32.485 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:02:30.048235+0000", 2021-11-21T22:02:32.486 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:32.486 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:32.486 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:02:32.486 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:02:32.487 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:02:32.487 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:02:32.487 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:02:32.487 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:02:32.487 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:02:32.488 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:32.488 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:02:32.488 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:02:32.488 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:02:32.489 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:02:32.489 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:02:32.489 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 24, 2021-11-21T22:02:32.489 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:02:32.490 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:02:32.490 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:02:32.490 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:02:32.491 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:02:32.491 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.491 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:02:32.492 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:02:32.492 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:02:32.492 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:32.492 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:32.493 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:32.493 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:32.493 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:32.494 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:32.494 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:32.494 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:32.494 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:02:32.494 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:02:32.495 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:02:32.495 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:02:32.495 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:02:32.495 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:32.496 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:32.496 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:32.496 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:32.496 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:32.497 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:32.497 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:32.497 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.497 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:02:32.498 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:32.498 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:32.498 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:32.498 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:32.498 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:32.499 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:32.499 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:32.499 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:32.499 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:32.500 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:32.500 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:32.500 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:32.500 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:32.501 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:32.501 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:32.501 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:32.501 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:32.501 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:32.502 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:32.502 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:32.502 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:32.502 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:32.503 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:32.503 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:32.503 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:32.503 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:32.503 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.504 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:32.504 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:32.504 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:32.504 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:32.505 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:32.505 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:32.505 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:32.505 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:32.505 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:32.506 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:32.506 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:32.506 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:02:32.506 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:02:32.507 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.507 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:32.507 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:02:32.507 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.508 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.508 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.508 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:02:32.508 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:02:32.508 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:02:32.509 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:32.509 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:32.509 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:32.509 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:32.510 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:32.510 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:32.510 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:32.510 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:32.510 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:32.511 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:32.511 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:32.511 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:32.511 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:32.512 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:32.512 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:32.512 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:32.512 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:32.512 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:32.513 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:32.513 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:32.513 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.513 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:02:32.514 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:32.514 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:32.514 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:32.514 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:32.515 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:32.515 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:32.515 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:32.515 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:32.515 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:32.516 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:32.516 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:32.516 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:32.516 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:32.517 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:32.517 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:32.517 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:32.517 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:32.518 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:32.518 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:32.518 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:32.518 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:32.518 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:32.519 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:32.519 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:32.519 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:32.519 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:32.520 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.520 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:32.520 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:32.520 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:32.520 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:32.521 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:32.521 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:32.521 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:32.521 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:32.522 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:32.522 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:32.522 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:32.522 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:02:32.523 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:32.523 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:02:32.523 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.523 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.523 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:32.524 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:02:32.524 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.524 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:32.524 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:02:32.525 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.525 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.525 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.525 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.526 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.526 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.526 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.526 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:32.526 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.527 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.527 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.528 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:02:32.528 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.528 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.528 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.529 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.529 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:02:32.529 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.529 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.530 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.530 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.530 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.530 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.530 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.531 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.531 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:02:32.531 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.531 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.532 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.532 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.532 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:02:32.532 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.532 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.533 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.533 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.533 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.533 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.534 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.534 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.534 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:02:32.534 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.535 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.535 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.535 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:02:32.535 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.536 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.536 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.536 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.536 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.537 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.537 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.537 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.537 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:02:32.537 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.538 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.538 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.538 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.538 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:02:32.539 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:32.539 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.539 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.539 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.539 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:02:32.540 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:02:32.540 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:02:32.540 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:02:32.540 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.541 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.541 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.541 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.541 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.541 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.542 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:32.542 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:02:32.542 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.542 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.543 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.543 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.543 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.543 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.544 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.544 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:32.544 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.544 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.544 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.545 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.545 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.545 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:02:32.546 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.546 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.546 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.546 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.547 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:02:32.547 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.547 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.547 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.547 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.548 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.548 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.548 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.548 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.549 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:02:32.549 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.549 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.549 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.549 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.550 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:02:32.550 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.550 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.550 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.551 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.551 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.551 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.551 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.552 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.552 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:02:32.552 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.552 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.552 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.553 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.553 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:02:32.553 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.553 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.554 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.554 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.554 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.554 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.554 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.555 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.555 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:02:32.555 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.555 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.556 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.556 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.556 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:02:32.556 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:32.557 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.557 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.557 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.557 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:02:32.557 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:02:32.558 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:02:32.558 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:02:32.558 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.558 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.559 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.559 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.559 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.559 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.560 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:32.560 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:02:32.560 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.560 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.560 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.561 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.561 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.561 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.561 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.562 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 127, 2021-11-21T22:02:32.562 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.562 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.562 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.562 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.563 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.563 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.563 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:02:32.563 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.564 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.564 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.564 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.564 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:02:32.564 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.565 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.565 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.565 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.565 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.566 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.566 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.566 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.566 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:02:32.566 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.567 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.567 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.567 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:02:32.568 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.568 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.568 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.568 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.569 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.569 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.569 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.569 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.569 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:02:32.570 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.570 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.570 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.570 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.571 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:02:32.571 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.571 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.571 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.571 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.572 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.572 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.572 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.572 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.573 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:02:32.573 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.573 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.573 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.574 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.574 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:02:32.574 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:32.574 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.575 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.575 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.575 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:02:32.575 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:02:32.575 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:02:32.576 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:02:32.576 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.576 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.576 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.577 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.577 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.577 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.577 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:32.577 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:02:32.578 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.578 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.578 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.578 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.579 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.579 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.579 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.579 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:32.579 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.580 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.580 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.580 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.580 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.581 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.581 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:02:32.581 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.581 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.581 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.582 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.582 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:02:32.582 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.582 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.583 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.583 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.583 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.583 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.583 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.584 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.584 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:02:32.584 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.584 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.585 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.585 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.585 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:02:32.585 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.586 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.586 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.586 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.586 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.587 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.587 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.587 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.587 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:02:32.587 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.588 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.588 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.588 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.588 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:02:32.589 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.589 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.589 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.589 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.589 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.590 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.590 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.590 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.590 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:02:32.591 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.591 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.591 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.591 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.592 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:02:32.592 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:32.592 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.592 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.592 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.593 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:02:32.593 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:02:32.593 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:02:32.593 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:02:32.594 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.594 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.594 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.594 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.594 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.595 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.595 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:32.595 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:02:32.595 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.596 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.596 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.596 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.596 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.596 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.597 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.597 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 127, 2021-11-21T22:02:32.597 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.597 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.598 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.598 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.598 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.598 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.599 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:02:32.599 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.599 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.599 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.599 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.600 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:02:32.600 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.600 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.600 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.601 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.601 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.601 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.601 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.601 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.602 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:02:32.602 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.602 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.602 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.603 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.603 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:02:32.603 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.603 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.603 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.604 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.604 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.604 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.604 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.605 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.605 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:02:32.605 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.605 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.605 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.606 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.606 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:02:32.606 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.606 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.607 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.607 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.607 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.607 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.608 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.608 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.608 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:02:32.608 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.608 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.609 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.609 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.609 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:02:32.609 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:32.610 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.610 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.610 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.610 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:02:32.610 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:02:32.611 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:02:32.611 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:02:32.611 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.611 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.612 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.612 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.612 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.612 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.612 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:32.613 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:02:32.613 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.613 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.613 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.614 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.614 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.614 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.614 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.614 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:32.615 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.615 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.615 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.615 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.616 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.616 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.616 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:02:32.616 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.616 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.617 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.617 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.617 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:02:32.617 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.618 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.618 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.618 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.618 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.619 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.619 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.619 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.619 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:02:32.619 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.620 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.620 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.620 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.620 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:02:32.621 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.621 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.621 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.621 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.621 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.622 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.622 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.622 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.622 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:02:32.623 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.623 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.623 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.623 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.624 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:02:32.624 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.624 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.624 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.624 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.625 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.625 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.625 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.625 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.626 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:02:32.626 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.626 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.626 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.626 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.627 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:02:32.627 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:32.627 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.627 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.628 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.628 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:02:32.628 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:02:32.628 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:02:32.628 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:02:32.629 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.629 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.629 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.629 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.630 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.630 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.630 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:32.630 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:02:32.630 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.631 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.631 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.631 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.631 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.632 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.632 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.632 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:32.632 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.632 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.633 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.633 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.633 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.633 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.634 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:02:32.634 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.634 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.634 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.634 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.635 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:02:32.635 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.635 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.635 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.636 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.636 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.636 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.636 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.636 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.637 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:02:32.637 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.637 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.637 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.638 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.638 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:02:32.638 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.638 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.639 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.639 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.639 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.639 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.639 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.640 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.640 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:02:32.640 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.640 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.641 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.641 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.641 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:02:32.641 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.641 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.642 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.642 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.642 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.642 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.643 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.643 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.643 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:02:32.643 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.643 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.644 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.644 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.644 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:02:32.644 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:32.645 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.645 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.645 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.645 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:02:32.645 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:02:32.646 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:02:32.646 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:02:32.646 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.646 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.647 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.647 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.647 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.647 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.647 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:32.648 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:02:32.648 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:32.648 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:32.648 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:32.649 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:32.649 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:32.649 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:32.649 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:32.649 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:32.650 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:32.650 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:32.650 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:32.650 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.651 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.651 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.651 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:02:32.651 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.651 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.652 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.652 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.652 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:02:32.652 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.653 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.653 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.653 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.653 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:32.654 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.654 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.654 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.654 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:02:32.654 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.655 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.655 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.655 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.655 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:02:32.656 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.656 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.656 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.656 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.657 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:32.657 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.657 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.657 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.657 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:02:32.658 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.658 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.658 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.658 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.659 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:02:32.659 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.659 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.659 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.660 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.660 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:32.660 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:32.660 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.660 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:32.661 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:02:32.661 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.661 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.661 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.662 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:32.662 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:02:32.662 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:32.662 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.662 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.663 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.663 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:02:32.663 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:02:32.663 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:02:32.664 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:02:32.664 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:32.664 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:32.664 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:32.664 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:32.665 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.665 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:32.665 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:02:32.665 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.666 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:32.666 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.666 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.666 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.667 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.667 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.667 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:02:32.667 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.667 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.668 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.668 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:32.668 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.668 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.669 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.669 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.669 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.669 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:02:32.670 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.670 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.670 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.670 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:32.670 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.671 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.671 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.671 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.671 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.672 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:02:32.672 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.672 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.672 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.673 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:32.673 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.673 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.673 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.673 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.674 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.674 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:02:32.674 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.674 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.675 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.675 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:32.675 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.675 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.675 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.676 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.676 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.676 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:02:32.676 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.677 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.677 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.677 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:32.677 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.677 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.678 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.678 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.678 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.678 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:02:32.679 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.679 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.679 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.679 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:32.680 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.680 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.680 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.680 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.680 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.681 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:02:32.681 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.681 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.681 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:32.682 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:32.682 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:32.682 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:32.682 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:32.683 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:32.683 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:32.683 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:02:32.683 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:32.683 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.684 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:32.684 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:02:32.684 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:02:32.684 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:02:32.685 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:02:32.685 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:02:32.685 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/1577318848": "2021-11-22T22:02:10.776983+0000", 2021-11-21T22:02:32.685 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/1577318848": "2021-11-22T22:02:10.776983+0000" 2021-11-21T22:02:32.685 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.686 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:02:32.686 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:02:32.686 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:02:32.686 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:02:32.687 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:02:32.687 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:02:32.687 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:02:32.687 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:02:32.687 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:32.688 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:32.688 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:02:32.688 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:02:32.688 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:02:32.689 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:02:32.689 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:02:32.689 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:32.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.478+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efe98053610 msgr2=0x7efe98055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.478+0000 7efebe043700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efe98053610 0x7efe98055ad0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7efeb8066f90 tx=0x7efeac007800).stop 2021-11-21T22:02:32.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.478+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb80b3320 msgr2=0x7efeb809e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.478+0000 7efebe043700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb80b3320 0x7efeb809e490 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7efeb809efb0 tx=0x7efea8006ec0).stop 2021-11-21T22:02:32.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 shutdown_connections 2021-11-21T22:02:32.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efeb81443a0 0x7efeb809e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efe98053610 0x7efe98055ad0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efeb80b3320 0x7efeb809e490 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 --2- 172.21.2.116:0/3662804640 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efeb8067850 0x7efeb80b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 >> 172.21.2.116:0/3662804640 conn(0x7efeb8064d60 msgr2=0x7efeb80ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 shutdown_connections 2021-11-21T22:02:32.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.482+0000 7efebe043700 1 -- 172.21.2.116:0/3662804640 wait complete. 2021-11-21T22:02:32.694 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:32.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2b49e5700 1 Processor -- start 2021-11-21T22:02:32.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2b49e5700 1 -- start start 2021-11-21T22:02:32.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2b49e5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b00a6aa0 0x7ff2b014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2b49e5700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff2b0062410 0x7ff2b01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2b49e5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b00a6e80 con 0x7ff2b00a86b0 2021-11-21T22:02:32.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2b49e5700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b00a8bc0 con 0x7ff2b0062410 2021-11-21T22:02:32.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2b49e5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b014a7b0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2ae59c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b00a6aa0 0x7ff2b014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2ae59c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b00a6aa0 0x7ff2b014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:56182/0 (socket says 172.21.2.116:56182) 2021-11-21T22:02:32.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2ae59c700 1 -- 172.21.2.116:0/2609598346 learned_addr learned my addr 172.21.2.116:0/2609598346 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:32.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2aed9d700 1 --2- 172.21.2.116:0/2609598346 >> v2:172.21.2.117:3300/0 conn(0x7ff2b0062410 0x7ff2b01443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2ae59c700 1 -- 172.21.2.116:0/2609598346 >> v2:172.21.2.117:3300/0 conn(0x7ff2b0062410 msgr2=0x7ff2b01443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:32.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2ae59c700 1 --2- 172.21.2.116:0/2609598346 >> v2:172.21.2.117:3300/0 conn(0x7ff2b0062410 0x7ff2b01443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2ae59c700 1 -- 172.21.2.116:0/2609598346 >> v1:172.21.2.117:6789/0 conn(0x7ff2b00a86b0 legacy=0x7ff2b01470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2ae59c700 1 -- 172.21.2.116:0/2609598346 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b014aa10 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.746+0000 7ff2aed9d700 1 --2- 172.21.2.116:0/2609598346 >> v2:172.21.2.117:3300/0 conn(0x7ff2b0062410 0x7ff2b01443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:02:32.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2ae59c700 1 --2- 172.21.2.116:0/2609598346 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b00a6aa0 0x7ff2b014b150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7ff2a0002f70 tx=0x7ff2a0006eb0).ready entity=mon.2 client_cookie=10743b8ebc137538 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2ad59a700 1 -- 172.21.2.116:0/2609598346 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a000d570 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2ad59a700 1 -- 172.21.2.116:0/2609598346 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a0000e90 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2ad59a700 1 -- 172.21.2.116:0/2609598346 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a000d900 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/2609598346 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b00a6aa0 msgr2=0x7ff2b014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/2609598346 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b00a6aa0 0x7ff2b014b150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7ff2a0002f70 tx=0x7ff2a0006eb0).stop 2021-11-21T22:02:32.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/2609598346 shutdown_connections 2021-11-21T22:02:32.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/2609598346 >> v2:172.21.2.117:3300/0 conn(0x7ff2b0062410 0x7ff2b01443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/2609598346 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b00a6aa0 0x7ff2b014b150 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/2609598346 >> 172.21.2.116:0/2609598346 conn(0x7ff2b005bd20 msgr2=0x7ff2b005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/2609598346 shutdown_connections 2021-11-21T22:02:32.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/2609598346 wait complete. 2021-11-21T22:02:32.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 Processor -- start 2021-11-21T22:02:32.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- start start 2021-11-21T22:02:32.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b0062410 0x7ff2b00b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2b00a6aa0 0x7ff2b00aa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2b00a86b0 0x7ff2b00aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00aaf50 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00a9230 con 0x7ff2b00a86b0 2021-11-21T22:02:32.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00a9a70 con 0x7ff2b0062410 2021-11-21T22:02:32.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2b00a6aa0 0x7ff2b00aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2b00a6aa0 0x7ff2b00aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52846/0 (socket says 172.21.2.116:52846) 2021-11-21T22:02:32.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 -- 172.21.2.116:0/1063699912 learned_addr learned my addr 172.21.2.116:0/1063699912 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:32.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 -- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b0062410 msgr2=0x7ff2b00b35c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:32.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b0062410 0x7ff2b00b35c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 -- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2b00a86b0 msgr2=0x7ff2b00aaa60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:32.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2b00a86b0 0x7ff2b00aaa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 -- 172.21.2.116:0/1063699912 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a8009470 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2add9b700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2b00a6aa0 0x7ff2b00aa570 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7ff2b0147210 tx=0x7ff2a4007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff29f7fe700 1 -- 172.21.2.116:0/1063699912 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a400ce20 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff29f7fe700 1 -- 172.21.2.116:0/1063699912 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4013450 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a000b1e0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff29f7fe700 1 -- 172.21.2.116:0/1063699912 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a40137b0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b00a9ca0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b00aa2b0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.750+0000 7ff29f7fe700 1 -- 172.21.2.116:0/1063699912 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff2a4010070 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.754+0000 7ff29f7fe700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c0535b0 0x7ff28c055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.754+0000 7ff29f7fe700 1 -- 172.21.2.116:0/1063699912 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff2a406adc0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.754+0000 7ff2ae59c700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c0535b0 0x7ff28c055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.754+0000 7ff2ae59c700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c0535b0 0x7ff28c055a70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7ff2a0002f70 tx=0x7ff2a0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.754+0000 7ff29f7fe700 1 -- 172.21.2.116:0/1063699912 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff2b00aa2b0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b00aa2b0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff29f7fe700 1 -- 172.21.2.116:0/1063699912 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+504 (secure 0 0 0) 0x7ff2b00aa2b0 con 0x7ff2b00a6aa0 2021-11-21T22:02:32.866 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:32.866 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2021-11-21T22:02:32.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c0535b0 msgr2=0x7ff28c055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c0535b0 0x7ff28c055a70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7ff2a0002f70 tx=0x7ff2a0015000).stop 2021-11-21T22:02:32.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2b00a6aa0 msgr2=0x7ff2b00aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2b00a6aa0 0x7ff2b00aa570 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7ff2b0147210 tx=0x7ff2a4007bf0).stop 2021-11-21T22:02:32.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 shutdown_connections 2021-11-21T22:02:32.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2b00a86b0 0x7ff2b00aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff28c0535b0 0x7ff28c055a70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2b00a6aa0 0x7ff2b00aa570 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 --2- 172.21.2.116:0/1063699912 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff2b0062410 0x7ff2b00b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 >> 172.21.2.116:0/1063699912 conn(0x7ff2b005bd20 msgr2=0x7ff2b01441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 shutdown_connections 2021-11-21T22:02:32.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.862+0000 7ff2b49e5700 1 -- 172.21.2.116:0/1063699912 wait complete. 2021-11-21T22:02:32.870 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 62 2021-11-21T22:02:32.872 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:32.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13ce708700 1 Processor -- start 2021-11-21T22:02:32.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13ce708700 1 -- start start 2021-11-21T22:02:32.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13ce708700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f13c80a86b0 0x7f13c814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13ce708700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f13c8062410 0x7f13c81443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13ce708700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80a6e80 con 0x7f13c80a6aa0 2021-11-21T22:02:32.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13ce708700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c80a8bc0 con 0x7f13c80a86b0 2021-11-21T22:02:32.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13ce708700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c814a7b0 con 0x7f13c8062410 2021-11-21T22:02:32.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13c77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f13c80a86b0 0x7f13c814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13c7fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f13c80a6aa0 0x7f13c81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36576/0 (socket says 172.21.2.116:36576) 2021-11-21T22:02:32.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13c7fff700 1 -- 172.21.2.116:0/4269708467 learned_addr learned my addr 172.21.2.116:0/4269708467 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:32.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.918+0000 7f13c6ffd700 1 -- 172.21.2.116:0/4269708467 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194399297 0 0) 0x7f13c80a6e80 con 0x7f13c80a6aa0 2021-11-21T22:02:32.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c6ffd700 1 -- 172.21.2.116:0/4269708467 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13a8003260 con 0x7f13c80a6aa0 2021-11-21T22:02:32.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c77fe700 1 -- 172.21.2.116:0/4269708467 >> v2:172.21.2.117:3300/0 conn(0x7f13c8062410 msgr2=0x7f13c81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:32.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c77fe700 1 --2- 172.21.2.116:0/4269708467 >> v2:172.21.2.117:3300/0 conn(0x7f13c8062410 0x7f13c81443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c77fe700 1 -- 172.21.2.116:0/4269708467 >> v1:172.21.2.116:6789/0 conn(0x7f13c80a6aa0 legacy=0x7f13c81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c77fe700 1 -- 172.21.2.116:0/4269708467 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c814aa10 con 0x7f13c80a86b0 2021-11-21T22:02:32.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c77fe700 1 --2- 172.21.2.116:0/4269708467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c814b150 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f13b0002f70 tx=0x7f13b0006df0).ready entity=mon.0 client_cookie=8188a501ddf1c758 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.934 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c6ffd700 1 -- 172.21.2.116:0/4269708467 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13b000d570 con 0x7f13c80a86b0 2021-11-21T22:02:32.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c6ffd700 1 -- 172.21.2.116:0/4269708467 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b0000e90 con 0x7f13c80a86b0 2021-11-21T22:02:32.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c6ffd700 1 -- 172.21.2.116:0/4269708467 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13b000d8c0 con 0x7f13c80a86b0 2021-11-21T22:02:32.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/4269708467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 msgr2=0x7f13c814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 --2- 172.21.2.116:0/4269708467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c814b150 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f13b0002f70 tx=0x7f13b0006df0).stop 2021-11-21T22:02:32.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/4269708467 shutdown_connections 2021-11-21T22:02:32.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 --2- 172.21.2.116:0/4269708467 >> v2:172.21.2.117:3300/0 conn(0x7f13c8062410 0x7f13c81443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 --2- 172.21.2.116:0/4269708467 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c814b150 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:32.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/4269708467 >> 172.21.2.116:0/4269708467 conn(0x7f13c805bd20 msgr2=0x7f13c805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/4269708467 shutdown_connections 2021-11-21T22:02:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/4269708467 wait complete. 2021-11-21T22:02:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 Processor -- start 2021-11-21T22:02:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- start start 2021-11-21T22:02:32.937 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f13c8062410 0x7f13c8143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f13c80a6aa0 0x7f13c81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c8156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8065670 con 0x7f13c80a86b0 2021-11-21T22:02:32.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8065b50 con 0x7f13c8062410 2021-11-21T22:02:32.938 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8156280 con 0x7f13c80a6aa0 2021-11-21T22:02:32.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c8156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c8156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52862/0 (socket says 172.21.2.116:52862) 2021-11-21T22:02:32.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 -- 172.21.2.116:0/2698957859 learned_addr learned my addr 172.21.2.116:0/2698957859 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:32.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 -- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f13c80a6aa0 msgr2=0x7f13c81566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f13c80a6aa0 0x7f13c81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 -- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f13c8062410 msgr2=0x7f13c8143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:32.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f13c8062410 0x7f13c8143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:32.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 -- 172.21.2.116:0/2698957859 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b000b180 con 0x7f13c80a86b0 2021-11-21T22:02:32.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ccca5700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c8156bd0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f13c00032f0 tx=0x7f13c0009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c57fa700 1 -- 172.21.2.116:0/2698957859 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c000d580 con 0x7f13c80a86b0 2021-11-21T22:02:32.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c57fa700 1 -- 172.21.2.116:0/2698957859 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c000d6e0 con 0x7f13c80a86b0 2021-11-21T22:02:32.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c80652e0 con 0x7f13c80a86b0 2021-11-21T22:02:32.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c815a5a0 con 0x7f13c80a86b0 2021-11-21T22:02:32.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c57fa700 1 -- 172.21.2.116:0/2698957859 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c000da40 con 0x7f13c80a86b0 2021-11-21T22:02:32.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a0003410 con 0x7f13c80a86b0 2021-11-21T22:02:32.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.922+0000 7f13c57fa700 1 -- 172.21.2.116:0/2698957859 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f13c000dba0 con 0x7f13c80a86b0 2021-11-21T22:02:32.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.926+0000 7f13c57fa700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f13ac0536f0 0x7f13ac055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:32.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.926+0000 7f13c57fa700 1 -- 172.21.2.116:0/2698957859 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f13c006b930 con 0x7f13c80a86b0 2021-11-21T22:02:32.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.926+0000 7f13c7fff700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f13ac0536f0 0x7f13ac055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:32.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.926+0000 7f13c7fff700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f13ac0536f0 0x7f13ac055bb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f13b8001110 tx=0x7f13b80076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:32.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:32.926+0000 7f13c57fa700 1 -- 172.21.2.116:0/2698957859 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13c0042d10 con 0x7f13c80a86b0 2021-11-21T22:02:33.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13a0003c40 con 0x7f13c80a86b0 2021-11-21T22:02:33.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13c57fa700 1 -- 172.21.2.116:0/2698957859 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+504 (secure 0 0 0) 0x7f13c0046330 con 0x7f13c80a86b0 2021-11-21T22:02:33.042 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:33.042 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2021-11-21T22:02:33.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f13ac0536f0 msgr2=0x7f13ac055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:33.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f13ac0536f0 0x7f13ac055bb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f13b8001110 tx=0x7f13b80076c0).stop 2021-11-21T22:02:33.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 msgr2=0x7f13c8156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:33.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c8156bd0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f13c00032f0 tx=0x7f13c0009220).stop 2021-11-21T22:02:33.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 shutdown_connections 2021-11-21T22:02:33.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f13ac0536f0 0x7f13ac055bb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:33.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f13c80a86b0 0x7f13c8156bd0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:33.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f13c80a6aa0 0x7f13c81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:33.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 --2- 172.21.2.116:0/2698957859 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f13c8062410 0x7f13c8143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:33.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 >> 172.21.2.116:0/2698957859 conn(0x7f13c805bd20 msgr2=0x7f13c8148410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:33.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 shutdown_connections 2021-11-21T22:02:33.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.038+0000 7f13ce708700 1 -- 172.21.2.116:0/2698957859 wait complete. 2021-11-21T22:02:33.046 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 62 2021-11-21T22:02:33.050 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T22:02:33.051 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T22:02:33.051 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T22:02:33.052 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T22:02:33.053 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T22:02:33.053 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T22:02:33.056 INFO:tasks.ceph.mds.a:Started 2021-11-21T22:02:33.057 INFO:tasks.ceph.mds.c:Started 2021-11-21T22:02:33.058 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:33.058 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:33.059 INFO:tasks.ceph.mds.b:Started 2021-11-21T22:02:33.059 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:33.060 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T22:02:33.089 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:02:33.090 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:02:33.096 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T22:02:33.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 Processor -- start 2021-11-21T22:02:33.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- start start 2021-11-21T22:02:33.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c05ced0 0x7fad1c0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:33.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fad1c09aa10 0x7fad1c146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:33.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c09b180 con 0x7fad1c061060 2021-11-21T22:02:33.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c09aea0 con 0x7fad1c09aa10 2021-11-21T22:02:33.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c09b3e0 con 0x7fad1c05ced0 2021-11-21T22:02:33.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad22b02700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fad1c09aa10 0x7fad1c146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:33.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad21b00700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fad1c061060 0x7fad1c09a300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36616/0 (socket says 172.21.2.116:36616) 2021-11-21T22:02:33.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad21b00700 1 -- 172.21.2.116:0/1956687218 learned_addr learned my addr 172.21.2.116:0/1956687218 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:33.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad22301700 1 --2- 172.21.2.116:0/1956687218 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c05ced0 0x7fad1c0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:33.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad22b02700 1 -- 172.21.2.116:0/1956687218 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c05ced0 msgr2=0x7fad1c0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:33.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad22b02700 1 --2- 172.21.2.116:0/1956687218 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c05ced0 0x7fad1c0acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:33.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad212ff700 1 -- 172.21.2.116:0/1956687218 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508532781 0 0) 0x7fad1c09b180 con 0x7fad1c061060 2021-11-21T22:02:33.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad22b02700 1 -- 172.21.2.116:0/1956687218 >> v1:172.21.2.116:6789/0 conn(0x7fad1c061060 legacy=0x7fad1c09a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:33.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad22b02700 1 -- 172.21.2.116:0/1956687218 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c09b680 con 0x7fad1c09aa10 2021-11-21T22:02:33.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad22b02700 1 --2- 172.21.2.116:0/1956687218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c09aa10 0x7fad1c146070 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fad1c0991e0 tx=0x7fad140091d0).ready entity=mon.0 client_cookie=7008f990fa4f1214 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:33.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad212ff700 1 -- 172.21.2.116:0/1956687218 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1400b860 con 0x7fad1c09aa10 2021-11-21T22:02:33.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad212ff700 1 -- 172.21.2.116:0/1956687218 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1400b9c0 con 0x7fad1c09aa10 2021-11-21T22:02:33.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad212ff700 1 -- 172.21.2.116:0/1956687218 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1400bb30 con 0x7fad1c09aa10 2021-11-21T22:02:33.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- 172.21.2.116:0/1956687218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c09aa10 msgr2=0x7fad1c146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:33.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 --2- 172.21.2.116:0/1956687218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c09aa10 0x7fad1c146070 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fad1c0991e0 tx=0x7fad140091d0).stop 2021-11-21T22:02:33.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- 172.21.2.116:0/1956687218 shutdown_connections 2021-11-21T22:02:33.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 --2- 172.21.2.116:0/1956687218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c09aa10 0x7fad1c146070 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:33.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 --2- 172.21.2.116:0/1956687218 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c05ced0 0x7fad1c0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:33.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- 172.21.2.116:0/1956687218 >> 172.21.2.116:0/1956687218 conn(0x7fad1c0b5de0 msgr2=0x7fad1c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:33.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- 172.21.2.116:0/1956687218 shutdown_connections 2021-11-21T22:02:33.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.110+0000 7fad24565700 1 -- 172.21.2.116:0/1956687218 wait complete. 2021-11-21T22:02:33.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 Processor -- start 2021-11-21T22:02:33.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 -- start start 2021-11-21T22:02:33.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad1c05ced0 0x7fad1c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:33.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c061060 0x7fad1c0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:33.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c09aa10 0x7fad1c141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:33.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0b1bd0 con 0x7fad1c061060 2021-11-21T22:02:33.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c152c90 con 0x7fad1c05ced0 2021-11-21T22:02:33.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c152f00 con 0x7fad1c09aa10 2021-11-21T22:02:33.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c061060 0x7fad1c0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:33.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c061060 0x7fad1c0b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52898/0 (socket says 172.21.2.116:52898) 2021-11-21T22:02:33.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 -- 172.21.2.116:0/1335851551 learned_addr learned my addr 172.21.2.116:0/1335851551 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:33.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 -- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c09aa10 msgr2=0x7fad1c141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:33.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c09aa10 0x7fad1c141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:33.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 -- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad1c05ced0 msgr2=0x7fad1c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:33.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad1c05ced0 0x7fad1c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:33.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 -- 172.21.2.116:0/1335851551 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14009470 con 0x7fad1c061060 2021-11-21T22:02:33.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad21b00700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c061060 0x7fad1c0b16e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fad1c152e20 tx=0x7fad0c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:33.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad0b7fe700 1 -- 172.21.2.116:0/1335851551 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad0c00ce40 con 0x7fad1c061060 2021-11-21T22:02:33.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad0b7fe700 1 -- 172.21.2.116:0/1335851551 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0072e0 con 0x7fad1c061060 2021-11-21T22:02:33.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad0b7fe700 1 -- 172.21.2.116:0/1335851551 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad0c005610 con 0x7fad1c061060 2021-11-21T22:02:33.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c1531a0 con 0x7fad1c061060 2021-11-21T22:02:33.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c0afa50 con 0x7fad1c061060 2021-11-21T22:02:33.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c09b920 con 0x7fad1c061060 2021-11-21T22:02:33.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad0b7fe700 1 -- 172.21.2.116:0/1335851551 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fad0c010070 con 0x7fad1c061060 2021-11-21T22:02:33.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad0b7fe700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7facfc053520 0x7facfc0559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:33.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad0b7fe700 1 -- 172.21.2.116:0/1335851551 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fad0c06ae80 con 0x7fad1c061060 2021-11-21T22:02:33.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad22301700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7facfc053520 0x7facfc0559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:33.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.114+0000 7fad22301700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7facfc053520 0x7facfc0559e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fad18007000 tx=0x7fad18008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:33.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.118+0000 7fad0b7fe700 1 -- 172.21.2.116:0/1335851551 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad0c0421b0 con 0x7fad1c061060 2021-11-21T22:02:33.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:33.210+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fad1c0573d0 con 0x7fad1c061060 2021-11-21T22:02:34.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad0b7fe700 1 -- 172.21.2.116:0/1335851551 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v137) v1 ==== 97+0+0 (secure 0 0 0) 0x7fad0c0457d0 con 0x7fad1c061060 2021-11-21T22:02:34.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7facfc053520 msgr2=0x7facfc0559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7facfc053520 0x7facfc0559e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fad18007000 tx=0x7fad18008040).stop 2021-11-21T22:02:34.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c061060 msgr2=0x7fad1c0b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c061060 0x7fad1c0b16e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fad1c152e20 tx=0x7fad0c007ac0).stop 2021-11-21T22:02:34.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 shutdown_connections 2021-11-21T22:02:34.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fad1c09aa10 0x7fad1c141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7facfc053520 0x7facfc0559e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fad1c061060 0x7fad1c0b16e0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 --2- 172.21.2.116:0/1335851551 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fad1c05ced0 0x7fad1c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 >> 172.21.2.116:0/1335851551 conn(0x7fad1c0b5de0 msgr2=0x7fad1c0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 shutdown_connections 2021-11-21T22:02:34.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.122+0000 7fad24565700 1 -- 172.21.2.116:0/1335851551 wait complete. 2021-11-21T22:02:34.130 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T22:02:34.134 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T22:02:34.134 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T22:02:34.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 Processor -- start 2021-11-21T22:02:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- start start 2021-11-21T22:02:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7f000a6aa0 0x7f7f0014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7f00062410 0x7f7f001443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f000a6e80 con 0x7f7f000a86b0 2021-11-21T22:02:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f000a8bc0 con 0x7f7f00062410 2021-11-21T22:02:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0014a7b0 con 0x7f7f000a6aa0 2021-11-21T22:02:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7f00062410 0x7f7f001443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7f00062410 0x7f7f001443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52908/0 (socket says 172.21.2.116:52908) 2021-11-21T22:02:34.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 -- 172.21.2.116:0/1568812122 learned_addr learned my addr 172.21.2.116:0/1568812122 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 -- 172.21.2.116:0/1568812122 >> v2:172.21.2.117:3300/0 conn(0x7f7f000a6aa0 msgr2=0x7f7f0014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 --2- 172.21.2.116:0/1568812122 >> v2:172.21.2.117:3300/0 conn(0x7f7f000a6aa0 0x7f7f0014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 -- 172.21.2.116:0/1568812122 >> v1:172.21.2.117:6789/0 conn(0x7f7f000a86b0 legacy=0x7f7f001470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 -- 172.21.2.116:0/1568812122 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0014aa10 con 0x7f7f00062410 2021-11-21T22:02:34.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7eff59e700 1 --2- 172.21.2.116:0/1568812122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f00062410 0x7f7f001443f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f7f00061ab0 tx=0x7f7ef8009290).ready entity=mon.0 client_cookie=5e6638991884504d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7efdd9b700 1 -- 172.21.2.116:0/1568812122 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef800b830 con 0x7f7f00062410 2021-11-21T22:02:34.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7efdd9b700 1 -- 172.21.2.116:0/1568812122 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef800b990 con 0x7f7f00062410 2021-11-21T22:02:34.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7efdd9b700 1 -- 172.21.2.116:0/1568812122 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef800bb00 con 0x7f7f00062410 2021-11-21T22:02:34.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- 172.21.2.116:0/1568812122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f00062410 msgr2=0x7f7f001443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 --2- 172.21.2.116:0/1568812122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f00062410 0x7f7f001443f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f7f00061ab0 tx=0x7f7ef8009290).stop 2021-11-21T22:02:34.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- 172.21.2.116:0/1568812122 shutdown_connections 2021-11-21T22:02:34.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 --2- 172.21.2.116:0/1568812122 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f00062410 0x7f7f001443f0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 --2- 172.21.2.116:0/1568812122 >> v2:172.21.2.117:3300/0 conn(0x7f7f000a6aa0 0x7f7f0014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- 172.21.2.116:0/1568812122 >> 172.21.2.116:0/1568812122 conn(0x7f7f0005bd20 msgr2=0x7f7f0005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- 172.21.2.116:0/1568812122 shutdown_connections 2021-11-21T22:02:34.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- 172.21.2.116:0/1568812122 wait complete. 2021-11-21T22:02:34.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 Processor -- start 2021-11-21T22:02:34.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.182+0000 7f7f05057700 1 -- start start 2021-11-21T22:02:34.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7f00062410 0x7f7f000ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f000a6aa0 0x7f7f001534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7f000a86b0 0x7f7f001539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00064a30 con 0x7f7f000a6aa0 2021-11-21T22:02:34.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000ab680 con 0x7f7f000a86b0 2021-11-21T22:02:34.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00156ea0 con 0x7f7f00062410 2021-11-21T22:02:34.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f000a6aa0 0x7f7f001534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f000a6aa0 0x7f7f001534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52918/0 (socket says 172.21.2.116:52918) 2021-11-21T22:02:34.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 -- 172.21.2.116:0/850560580 learned_addr learned my addr 172.21.2.116:0/850560580 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 -- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7f00062410 msgr2=0x7f7f000ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7f00062410 0x7f7f000ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 -- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7f000a86b0 msgr2=0x7f7f001539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7f000a86b0 0x7f7f001539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 -- 172.21.2.116:0/850560580 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef8009530 con 0x7f7f000a6aa0 2021-11-21T22:02:34.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efe59c700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f000a6aa0 0x7f7f001534f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f7ef0002ef0 tx=0x7f7ef0006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7ef7fff700 1 -- 172.21.2.116:0/850560580 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef0009ea0 con 0x7f7f000a6aa0 2021-11-21T22:02:34.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7ef7fff700 1 -- 172.21.2.116:0/850560580 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef0002320 con 0x7f7f000a6aa0 2021-11-21T22:02:34.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f00157140 con 0x7f7f000a6aa0 2021-11-21T22:02:34.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7ef7fff700 1 -- 172.21.2.116:0/850560580 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef0002680 con 0x7f7f000a6aa0 2021-11-21T22:02:34.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f00063b70 con 0x7f7f000a6aa0 2021-11-21T22:02:34.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f00064790 con 0x7f7f000a6aa0 2021-11-21T22:02:34.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7ef7fff700 1 -- 172.21.2.116:0/850560580 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7ef000f030 con 0x7f7f000a6aa0 2021-11-21T22:02:34.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7ef7fff700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ee4053580 0x7f7ee4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7ef7fff700 1 -- 172.21.2.116:0/850560580 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7ef006b2f0 con 0x7f7f000a6aa0 2021-11-21T22:02:34.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efed9d700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ee4053580 0x7f7ee4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.186+0000 7f7efed9d700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ee4053580 0x7f7ee4055a40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f7ee8002f70 tx=0x7f7ee8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.190+0000 7f7ef7fff700 1 -- 172.21.2.116:0/850560580 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f00064790 con 0x7f7f000a6aa0 2021-11-21T22:02:34.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7f00064790 con 0x7f7f000a6aa0 2021-11-21T22:02:34.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7ef7fff700 1 -- 172.21.2.116:0/850560580 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f7f00064790 con 0x7f7f000a6aa0 2021-11-21T22:02:34.306 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:34.307 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:34.307 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T22:02:34.307 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.307 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T22:02:34.308 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T22:02:34.308 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.308 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:34.308 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.308 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.309 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.309 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.309 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.309 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.310 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T22:02:34.310 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T22:02:34.310 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.310 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:34.311 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.311 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.311 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.311 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.311 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.312 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.312 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T22:02:34.312 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T22:02:34.312 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.313 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:34.313 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.313 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.313 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.313 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.314 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.314 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.314 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T22:02:34.314 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T22:02:34.315 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.315 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.315 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.315 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.315 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.316 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.316 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.316 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T22:02:34.316 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T22:02:34.317 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.317 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.317 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.318 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.318 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.318 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.318 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.319 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T22:02:34.319 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T22:02:34.319 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.319 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.320 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.320 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.320 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.320 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.321 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.321 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T22:02:34.321 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T22:02:34.321 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.322 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.322 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.322 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.322 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.323 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.323 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.323 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T22:02:34.323 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T22:02:34.323 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.324 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.324 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.324 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.324 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.325 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.325 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.325 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T22:02:34.325 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T22:02:34.326 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.326 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.326 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.326 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.327 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.327 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.327 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.327 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T22:02:34.327 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T22:02:34.328 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.328 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.328 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.328 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.329 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.329 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.329 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.329 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T22:02:34.330 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T22:02:34.330 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.330 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.330 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.330 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.331 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.331 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.331 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.332 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T22:02:34.332 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T22:02:34.333 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.333 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:34.334 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:02:34.334 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:02:34.334 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.334 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.335 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.335 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T22:02:34.335 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T22:02:34.336 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.336 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:34.336 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:02:34.336 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:02:34.337 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.337 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.337 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.337 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T22:02:34.338 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T22:02:34.338 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.338 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:02:34.338 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:34.338 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:34.339 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.339 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.339 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.339 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.340 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T22:02:34.340 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T22:02:34.340 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.340 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:02:34.341 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:02:34.341 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.341 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.341 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.341 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.342 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T22:02:34.342 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T22:02:34.342 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:34.342 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:02:34.343 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:02:34.343 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:34.343 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.343 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.343 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:34.344 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:34.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ee4053580 msgr2=0x7f7ee4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ee4053580 0x7f7ee4055a40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f7ee8002f70 tx=0x7f7ee8006a80).stop 2021-11-21T22:02:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f000a6aa0 msgr2=0x7f7f001534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f000a6aa0 0x7f7f001534f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f7ef0002ef0 tx=0x7f7ef0006d50).stop 2021-11-21T22:02:34.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 shutdown_connections 2021-11-21T22:02:34.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7f000a86b0 0x7f7f001539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ee4053580 0x7f7ee4055a40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7f000a6aa0 0x7f7f001534f0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 --2- 172.21.2.116:0/850560580 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7f00062410 0x7f7f000ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 >> 172.21.2.116:0/850560580 conn(0x7f7f0005bd20 msgr2=0x7f7f00142620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 shutdown_connections 2021-11-21T22:02:34.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.302+0000 7f7f05057700 1 -- 172.21.2.116:0/850560580 wait complete. 2021-11-21T22:02:34.349 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:34.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 Processor -- start 2021-11-21T22:02:34.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- start start 2021-11-21T22:02:34.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f79380a6aa0 0x7f793814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f79380a86b0 0x7f79381470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79381475d0 0x7f79381443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380a8bc0 con 0x7f79380a6aa0 2021-11-21T22:02:34.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380a6e80 con 0x7f79380a86b0 2021-11-21T22:02:34.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793814a7b0 con 0x7f79381475d0 2021-11-21T22:02:34.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f79380a6aa0 0x7f793814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f79380a6aa0 0x7f793814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52922/0 (socket says 172.21.2.116:52922) 2021-11-21T22:02:34.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/974891382 learned_addr learned my addr 172.21.2.116:0/974891382 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/974891382 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79381475d0 msgr2=0x7f79381443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:34.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- 172.21.2.116:0/974891382 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79381475d0 0x7f79381443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/974891382 >> v2:172.21.2.117:3300/0 conn(0x7f79380a86b0 msgr2=0x7f79381470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:34.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- 172.21.2.116:0/974891382 >> v2:172.21.2.117:3300/0 conn(0x7f79380a86b0 0x7f79381470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/974891382 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793814aa10 con 0x7f79380a6aa0 2021-11-21T22:02:34.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- 172.21.2.116:0/974891382 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f793814b150 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f7934002f70 tx=0x7f7934006eb0).ready entity=mon.0 client_cookie=bd818917109b76df server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f7927fff700 1 -- 172.21.2.116:0/974891382 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f793400d570 con 0x7f79380a6aa0 2021-11-21T22:02:34.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f7927fff700 1 -- 172.21.2.116:0/974891382 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934000e90 con 0x7f79380a6aa0 2021-11-21T22:02:34.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f7927fff700 1 -- 172.21.2.116:0/974891382 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f793400d900 con 0x7f79380a6aa0 2021-11-21T22:02:34.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- 172.21.2.116:0/974891382 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 msgr2=0x7f793814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- 172.21.2.116:0/974891382 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f793814b150 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f7934002f70 tx=0x7f7934006eb0).stop 2021-11-21T22:02:34.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- 172.21.2.116:0/974891382 shutdown_connections 2021-11-21T22:02:34.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- 172.21.2.116:0/974891382 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79381475d0 0x7f79381443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- 172.21.2.116:0/974891382 >> v2:172.21.2.117:3300/0 conn(0x7f79380a86b0 0x7f79381470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- 172.21.2.116:0/974891382 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f793814b150 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- 172.21.2.116:0/974891382 >> 172.21.2.116:0/974891382 conn(0x7f793805bd20 msgr2=0x7f793805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- 172.21.2.116:0/974891382 shutdown_connections 2021-11-21T22:02:34.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- 172.21.2.116:0/974891382 wait complete. 2021-11-21T22:02:34.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 Processor -- start 2021-11-21T22:02:34.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- start start 2021-11-21T22:02:34.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f7938064730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79380a86b0 0x7f79380a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79381475d0 0x7f79380a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938156610 con 0x7f79380a6aa0 2021-11-21T22:02:34.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79381568b0 con 0x7f79381475d0 2021-11-21T22:02:34.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793f9f8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7938156b50 con 0x7f79380a86b0 2021-11-21T22:02:34.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f7938064730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f7938064730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52930/0 (socket says 172.21.2.116:52930) 2021-11-21T22:02:34.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/3198918860 learned_addr learned my addr 172.21.2.116:0/3198918860 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79380a86b0 msgr2=0x7f79380a8d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79380a86b0 0x7f79380a8d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79381475d0 msgr2=0x7f79380a9210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:34.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79381475d0 0x7f79380a9210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.398+0000 7f793d794700 1 -- 172.21.2.116:0/3198918860 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793400b1e0 con 0x7f79380a6aa0 2021-11-21T22:02:34.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f793d794700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f7938064730 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f7934002f70 tx=0x7f7934015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f79267fc700 1 -- 172.21.2.116:0/3198918860 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7934011790 con 0x7f79380a6aa0 2021-11-21T22:02:34.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f79267fc700 1 -- 172.21.2.116:0/3198918860 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79340118f0 con 0x7f79380a6aa0 2021-11-21T22:02:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7938156300 con 0x7f79380a6aa0 2021-11-21T22:02:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f79267fc700 1 -- 172.21.2.116:0/3198918860 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7934011c50 con 0x7f79380a6aa0 2021-11-21T22:02:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7938156ff0 con 0x7f79380a6aa0 2021-11-21T22:02:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7938156d10 con 0x7f79380a6aa0 2021-11-21T22:02:34.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f79267fc700 1 -- 172.21.2.116:0/3198918860 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7934025030 con 0x7f79380a6aa0 2021-11-21T22:02:34.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f79267fc700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7918053640 0x7f7918055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f79267fc700 1 -- 172.21.2.116:0/3198918860 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f793406c7d0 con 0x7f79380a6aa0 2021-11-21T22:02:34.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f793cf93700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7918053640 0x7f7918055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.402+0000 7f793cf93700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7918053640 0x7f7918055b00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f7928002fb0 tx=0x7f7928006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.406+0000 7f79267fc700 1 -- 172.21.2.116:0/3198918860 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7934043c60 con 0x7f79380a6aa0 2021-11-21T22:02:34.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79380573d0 con 0x7f79380a6aa0 2021-11-21T22:02:34.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f79267fc700 1 -- 172.21.2.116:0/3198918860 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f7934023020 con 0x7f79380a6aa0 2021-11-21T22:02:34.494 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:34.494 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":137,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:34.114243+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:34.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7918053640 msgr2=0x7f7918055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7918053640 0x7f7918055b00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f7928002fb0 tx=0x7f7928006930).stop 2021-11-21T22:02:34.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 msgr2=0x7f7938064730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f7938064730 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f7934002f70 tx=0x7f7934015000).stop 2021-11-21T22:02:34.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 shutdown_connections 2021-11-21T22:02:34.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f79381475d0 0x7f79380a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f79380a86b0 0x7f79380a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7918053640 0x7f7918055b00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 --2- 172.21.2.116:0/3198918860 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f79380a6aa0 0x7f7938064730 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 >> 172.21.2.116:0/3198918860 conn(0x7f793805bd20 msgr2=0x7f793813f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 shutdown_connections 2021-11-21T22:02:34.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.490+0000 7f793f9f8700 1 -- 172.21.2.116:0/3198918860 wait complete. 2021-11-21T22:02:34.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T22:02:34.502 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:02:34.526 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:34.528 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:34.532 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:34.535 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:02:34.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f83666e0700 1 Processor -- start 2021-11-21T22:02:34.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f83666e0700 1 -- start start 2021-11-21T22:02:34.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f83666e0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83600b1270 0x7f83600b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f83666e0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83600b0070 con 0x7f83600ad580 2021-11-21T22:02:34.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f83666e0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83600b05c0 con 0x7f83600b6770 2021-11-21T22:02:34.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f83666e0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83600b2230 con 0x7f83600b1270 2021-11-21T22:02:34.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f8364c7d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f83600ad580 0x7f83600ad9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36664/0 (socket says 172.21.2.116:36664) 2021-11-21T22:02:34.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f8364c7d700 1 -- 172.21.2.116:0/2804375495 learned_addr learned my addr 172.21.2.116:0/2804375495 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:34.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835f7fe700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83600b1270 0x7f83600b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245612254 0 0) 0x7f83600b0070 con 0x7f83600ad580 2021-11-21T22:02:34.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8340003260 con 0x7f83600ad580 2021-11-21T22:02:34.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749090156 0 0) 0x7f8340003260 con 0x7f83600ad580 2021-11-21T22:02:34.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83600b1270 msgr2=0x7f83600b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83600b1270 0x7f83600b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 >> v1:172.21.2.117:6789/0 conn(0x7f83600b6770 legacy=0x7f83601394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83600b24d0 con 0x7f83600ad580 2021-11-21T22:02:34.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.550+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f83580019e0 con 0x7f83600ad580 2021-11-21T22:02:34.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 >> v1:172.21.2.116:6789/0 conn(0x7f83600ad580 legacy=0x7f83600ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f8340003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340004500 0x7f8340004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83600b1270 0x7f8340004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600b2230 con 0x7f8340003780 2021-11-21T22:02:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834000d990 con 0x7f8340004500 2021-11-21T22:02:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834000db50 con 0x7f83600b1270 2021-11-21T22:02:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f8340003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835f7fe700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340004500 0x7f8340004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 -- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83600b1270 msgr2=0x7f8340004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f83600b1270 0x7f8340004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 -- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340004500 msgr2=0x7f8340004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340004500 0x7f8340004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 -- 172.21.2.116:0/2804375495 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340003260 con 0x7f8340003780 2021-11-21T22:02:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835f7fe700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8340004500 0x7f8340004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f8340003ba0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f83600609a0 tx=0x7f8348007a00).ready entity=mon.0 client_cookie=f2c384b0396f7e5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8348007d10 con 0x7f8340003780 2021-11-21T22:02:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348007030 con 0x7f8340003780 2021-11-21T22:02:34.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835effd700 1 -- 172.21.2.116:0/2804375495 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8348007390 con 0x7f8340003780 2021-11-21T22:02:34.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/2804375495 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 msgr2=0x7f8340003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 --2- 172.21.2.116:0/2804375495 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f8340003ba0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f83600609a0 tx=0x7f8348007a00).stop 2021-11-21T22:02:34.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f8364c7d700 1 -- 172.21.2.116:0/2804375495 reap_dead start 2021-11-21T22:02:34.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/2804375495 shutdown_connections 2021-11-21T22:02:34.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/2804375495 >> 172.21.2.116:0/2804375495 conn(0x7f8360062040 msgr2=0x7f83600b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/2804375495 shutdown_connections 2021-11-21T22:02:34.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/2804375495 wait complete. 2021-11-21T22:02:34.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 Processor -- start 2021-11-21T22:02:34.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- start start 2021-11-21T22:02:34.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f836013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836013c8d0 0x7f8360151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836014d9f0 0x7f8360066d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836013d0c0 con 0x7f8340003780 2021-11-21T22:02:34.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83601510d0 con 0x7f836014d9f0 2021-11-21T22:02:34.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8360151370 con 0x7f836013c8d0 2021-11-21T22:02:34.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f836013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f836013e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52948/0 (socket says 172.21.2.116:52948) 2021-11-21T22:02:34.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 -- 172.21.2.116:0/4078667541 learned_addr learned my addr 172.21.2.116:0/4078667541 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835f7fe700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836013c8d0 0x7f8360151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f8364c7d700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836014d9f0 0x7f8360066d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 -- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836013c8d0 msgr2=0x7f8360151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.574 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836013c8d0 0x7f8360151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 -- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836014d9f0 msgr2=0x7f8360066d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836014d9f0 0x7f8360066d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 -- 172.21.2.116:0/4078667541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360151610 con 0x7f8340003780 2021-11-21T22:02:34.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835ffff700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f836013e6f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f83600609a0 tx=0x7f8348015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835d7fa700 1 -- 172.21.2.116:0/4078667541 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8348007d10 con 0x7f8340003780 2021-11-21T22:02:34.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835d7fa700 1 -- 172.21.2.116:0/4078667541 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83480127b0 con 0x7f8340003780 2021-11-21T22:02:34.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8360064750 con 0x7f8340003780 2021-11-21T22:02:34.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835d7fa700 1 -- 172.21.2.116:0/4078667541 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8348012b10 con 0x7f8340003780 2021-11-21T22:02:34.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836013cd40 con 0x7f8340003780 2021-11-21T22:02:34.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83601517a0 con 0x7f8340003780 2021-11-21T22:02:34.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835d7fa700 1 -- 172.21.2.116:0/4078667541 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f834801d070 con 0x7f8340003780 2021-11-21T22:02:34.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.554+0000 7f835d7fa700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8344057a20 0x7f8344059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.558+0000 7f835d7fa700 1 -- 172.21.2.116:0/4078667541 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8348001e40 con 0x7f8340003780 2021-11-21T22:02:34.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.558+0000 7f835f7fe700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8344057a20 0x7f8344059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.558+0000 7f835f7fe700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8344057a20 0x7f8344059ee0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f835000b000 tx=0x7f8350008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.558+0000 7f835d7fa700 1 -- 172.21.2.116:0/4078667541 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8348043940 con 0x7f8340003780 2021-11-21T22:02:34.645 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f83600573d0 con 0x7f8340003780 2021-11-21T22:02:34.646 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f835d7fa700 1 -- 172.21.2.116:0/4078667541 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8348046f60 con 0x7f8340003780 2021-11-21T22:02:34.646 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:02:34.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8344057a20 msgr2=0x7f8344059ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8344057a20 0x7f8344059ee0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f835000b000 tx=0x7f8350008040).stop 2021-11-21T22:02:34.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 msgr2=0x7f836013e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f836013e6f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f83600609a0 tx=0x7f8348015000).stop 2021-11-21T22:02:34.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 shutdown_connections 2021-11-21T22:02:34.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f836014d9f0 0x7f8360066d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f836013c8d0 0x7f8360151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8344057a20 0x7f8344059ee0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 --2- 172.21.2.116:0/4078667541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8340003780 0x7f836013e6f0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 >> 172.21.2.116:0/4078667541 conn(0x7f8360062040 msgr2=0x7f8360140230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 shutdown_connections 2021-11-21T22:02:34.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.642+0000 7f83666e0700 1 -- 172.21.2.116:0/4078667541 wait complete. 2021-11-21T22:02:34.653 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:02:34.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 Processor -- start 2021-11-21T22:02:34.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- start start 2021-11-21T22:02:34.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff1800abce0 0x7ff1801478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800ac600 0x7ff1800aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff180147e80 con 0x7ff180144810 2021-11-21T22:02:34.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800b1490 con 0x7ff1800abce0 2021-11-21T22:02:34.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800b1fd0 con 0x7ff1800ac600 2021-11-21T22:02:34.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff1800abce0 0x7ff1801478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff1800abce0 0x7ff1801478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52958/0 (socket says 172.21.2.116:52958) 2021-11-21T22:02:34.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3039484586 learned_addr learned my addr 172.21.2.116:0/3039484586 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3039484586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800ac600 msgr2=0x7ff1800aca90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:34.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 --2- 172.21.2.116:0/3039484586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800ac600 0x7ff1800aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3039484586 >> v1:172.21.2.117:6789/0 conn(0x7ff180144810 legacy=0x7ff1800b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3039484586 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800af5a0 con 0x7ff1800abce0 2021-11-21T22:02:34.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17e7fc700 1 --2- 172.21.2.116:0/3039484586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800abce0 0x7ff1801478c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7ff180062500 tx=0x7ff174006eb0).ready entity=mon.0 client_cookie=b65585787808068d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17dffb700 1 -- 172.21.2.116:0/3039484586 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff17400d570 con 0x7ff1800abce0 2021-11-21T22:02:34.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17dffb700 1 -- 172.21.2.116:0/3039484586 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174000e90 con 0x7ff1800abce0 2021-11-21T22:02:34.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff17dffb700 1 -- 172.21.2.116:0/3039484586 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff17400d8c0 con 0x7ff1800abce0 2021-11-21T22:02:34.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- 172.21.2.116:0/3039484586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800abce0 msgr2=0x7ff1801478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 --2- 172.21.2.116:0/3039484586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800abce0 0x7ff1801478c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7ff180062500 tx=0x7ff174006eb0).stop 2021-11-21T22:02:34.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- 172.21.2.116:0/3039484586 shutdown_connections 2021-11-21T22:02:34.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 --2- 172.21.2.116:0/3039484586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800ac600 0x7ff1800aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 --2- 172.21.2.116:0/3039484586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800abce0 0x7ff1801478c0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- 172.21.2.116:0/3039484586 >> 172.21.2.116:0/3039484586 conn(0x7ff18005bd20 msgr2=0x7ff18005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- 172.21.2.116:0/3039484586 shutdown_connections 2021-11-21T22:02:34.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.702+0000 7ff185658700 1 -- 172.21.2.116:0/3039484586 wait complete. 2021-11-21T22:02:34.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 Processor -- start 2021-11-21T22:02:34.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 -- start start 2021-11-21T22:02:34.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800abce0 0x7ff1800afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800ac600 0x7ff180063e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff180144810 0x7ff180064310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff180146f90 con 0x7ff1800ac600 2021-11-21T22:02:34.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff180146290 con 0x7ff180144810 2021-11-21T22:02:34.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff180145040 con 0x7ff1800abce0 2021-11-21T22:02:34.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800ac600 0x7ff180063e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800ac600 0x7ff180063e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52964/0 (socket says 172.21.2.116:52964) 2021-11-21T22:02:34.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3960181724 learned_addr learned my addr 172.21.2.116:0/3960181724 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800abce0 msgr2=0x7ff1800afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800abce0 0x7ff1800afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff180144810 msgr2=0x7ff180064310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff180144810 0x7ff180064310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 -- 172.21.2.116:0/3960181724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17400b1e0 con 0x7ff1800ac600 2021-11-21T22:02:34.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17e7fc700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800ac600 0x7ff180063e20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7ff180062500 tx=0x7ff174015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff16ffff700 1 -- 172.21.2.116:0/3960181724 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff17400d8f0 con 0x7ff1800ac600 2021-11-21T22:02:34.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff16ffff700 1 -- 172.21.2.116:0/3960181724 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174011ce0 con 0x7ff1800ac600 2021-11-21T22:02:34.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1801452e0 con 0x7ff1800ac600 2021-11-21T22:02:34.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff180145580 con 0x7ff1800ac600 2021-11-21T22:02:34.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff16ffff700 1 -- 172.21.2.116:0/3960181724 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff17401e650 con 0x7ff1800ac600 2021-11-21T22:02:34.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff180159fb0 con 0x7ff1800ac600 2021-11-21T22:02:34.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff16ffff700 1 -- 172.21.2.116:0/3960181724 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff174025030 con 0x7ff1800ac600 2021-11-21T22:02:34.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff16ffff700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff15c053610 0x7ff15c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff16ffff700 1 -- 172.21.2.116:0/3960181724 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff17406ca20 con 0x7ff1800ac600 2021-11-21T22:02:34.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.706+0000 7ff17effd700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff15c053610 0x7ff15c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.710+0000 7ff17effd700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff15c053610 0x7ff15c055ad0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7ff170002f10 tx=0x7ff170006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.710+0000 7ff16ffff700 1 -- 172.21.2.116:0/3960181724 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff174043eb0 con 0x7ff1800ac600 2021-11-21T22:02:34.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.790+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1800af060 con 0x7ff1800ac600 2021-11-21T22:02:34.796 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.790+0000 7ff16ffff700 1 -- 172.21.2.116:0/3960181724 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff174023020 con 0x7ff1800ac600 2021-11-21T22:02:34.797 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:34.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff15c053610 msgr2=0x7ff15c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff15c053610 0x7ff15c055ad0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7ff170002f10 tx=0x7ff170006930).stop 2021-11-21T22:02:34.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800ac600 msgr2=0x7ff180063e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800ac600 0x7ff180063e20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7ff180062500 tx=0x7ff174015000).stop 2021-11-21T22:02:34.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 shutdown_connections 2021-11-21T22:02:34.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff180144810 0x7ff180064310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff15c053610 0x7ff15c055ad0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.799 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff1800ac600 0x7ff180063e20 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 --2- 172.21.2.116:0/3960181724 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff1800abce0 0x7ff1800afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 >> 172.21.2.116:0/3960181724 conn(0x7ff18005bd20 msgr2=0x7ff180148580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 shutdown_connections 2021-11-21T22:02:34.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.794+0000 7ff185658700 1 -- 172.21.2.116:0/3960181724 wait complete. 2021-11-21T22:02:34.804 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T22:02:34.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94f892700 1 Processor -- start 2021-11-21T22:02:34.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94f892700 1 -- start start 2021-11-21T22:02:34.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94f892700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb9480a6aa0 0x7fb94814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94f892700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb948062410 0x7fb9481443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94f892700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9480a6e80 con 0x7fb9480a86b0 2021-11-21T22:02:34.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94f892700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9480a8bc0 con 0x7fb9480a6aa0 2021-11-21T22:02:34.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94f892700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814a7b0 con 0x7fb948062410 2021-11-21T22:02:34.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94ce2d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb9480a86b0 0x7fb9481470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36694/0 (socket says 172.21.2.116:36694) 2021-11-21T22:02:34.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94ce2d700 1 -- 172.21.2.116:0/1600786267 learned_addr learned my addr 172.21.2.116:0/1600786267 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:34.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238925143 0 0) 0x7fb9480a6e80 con 0x7fb9480a86b0 2021-11-21T22:02:34.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb930003260 con 0x7fb9480a86b0 2021-11-21T22:02:34.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94d62e700 1 --2- 172.21.2.116:0/1600786267 >> v2:172.21.2.117:3300/0 conn(0x7fb9480a6aa0 0x7fb94814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 678260372 0 0) 0x7fb930003260 con 0x7fb9480a86b0 2021-11-21T22:02:34.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb948062410 msgr2=0x7fb9481443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb948062410 0x7fb9481443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 >> v2:172.21.2.117:3300/0 conn(0x7fb9480a6aa0 msgr2=0x7fb94814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 --2- 172.21.2.116:0/1600786267 >> v2:172.21.2.117:3300/0 conn(0x7fb9480a6aa0 0x7fb94814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94814aa10 con 0x7fb9480a86b0 2021-11-21T22:02:34.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb93c001680 con 0x7fb9480a86b0 2021-11-21T22:02:34.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 >> v1:172.21.2.116:6789/0 conn(0x7fb9480a86b0 legacy=0x7fb9481470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb9300037d0 0x7fb930003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb948062410 0x7fb930004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb930004bf0 0x7fb93000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480a8bc0 con 0x7fb930004bf0 2021-11-21T22:02:34.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814a7b0 con 0x7fb9300037d0 2021-11-21T22:02:34.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb930005330 con 0x7fb948062410 2021-11-21T22:02:34.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94de2f700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb930004bf0 0x7fb93000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94de2f700 1 -- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb948062410 msgr2=0x7fb930004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94de2f700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb948062410 0x7fb930004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:34.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94de2f700 1 -- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb9300037d0 msgr2=0x7fb930003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:34.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94d62e700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb9300037d0 0x7fb930003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94de2f700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb9300037d0 0x7fb930003bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.850+0000 7fb94de2f700 1 -- 172.21.2.116:0/1600786267 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930003260 con 0x7fb930004bf0 2021-11-21T22:02:34.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb930004bf0 0x7fb93000b750 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fb9400032f0 tx=0x7fb940006f90).ready entity=mon.0 client_cookie=f6d48bee1d0f0c1e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb94000b460 con 0x7fb930004bf0 2021-11-21T22:02:34.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94000b5c0 con 0x7fb930004bf0 2021-11-21T22:02:34.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb93bfff700 1 -- 172.21.2.116:0/1600786267 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb94000b730 con 0x7fb930004bf0 2021-11-21T22:02:34.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- 172.21.2.116:0/1600786267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb930004bf0 msgr2=0x7fb93000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 --2- 172.21.2.116:0/1600786267 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb930004bf0 0x7fb93000b750 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fb9400032f0 tx=0x7fb940006f90).stop 2021-11-21T22:02:34.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 -- 172.21.2.116:0/1600786267 reap_dead start 2021-11-21T22:02:34.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- 172.21.2.116:0/1600786267 shutdown_connections 2021-11-21T22:02:34.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- 172.21.2.116:0/1600786267 >> 172.21.2.116:0/1600786267 conn(0x7fb94805bd20 msgr2=0x7fb94805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- 172.21.2.116:0/1600786267 shutdown_connections 2021-11-21T22:02:34.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- 172.21.2.116:0/1600786267 wait complete. 2021-11-21T22:02:34.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 Processor -- start 2021-11-21T22:02:34.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- start start 2021-11-21T22:02:34.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb930004bf0 0x7fb9481453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb948065fb0 0x7fb948066430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb948142110 0x7fb948142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb948065800 con 0x7fb948142110 2021-11-21T22:02:34.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb948065a20 con 0x7fb948065fb0 2021-11-21T22:02:34.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb948064bd0 con 0x7fb930004bf0 2021-11-21T22:02:34.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb948142110 0x7fb948142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb948142110 0x7fb948142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52982/0 (socket says 172.21.2.116:52982) 2021-11-21T22:02:34.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 -- 172.21.2.116:0/3487861063 learned_addr learned my addr 172.21.2.116:0/3487861063 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:34.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94d62e700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb930004bf0 0x7fb9481453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94ce2d700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb948065fb0 0x7fb948066430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 -- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb930004bf0 msgr2=0x7fb9481453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb930004bf0 0x7fb9481453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 -- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb948065fb0 msgr2=0x7fb948066430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb948065fb0 0x7fb948066430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 -- 172.21.2.116:0/3487861063 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480a9b20 con 0x7fb948142110 2021-11-21T22:02:34.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94de2f700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb948142110 0x7fb948142590 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fb948060090 tx=0x7fb940017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb93a7fc700 1 -- 172.21.2.116:0/3487861063 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb940011a70 con 0x7fb948142110 2021-11-21T22:02:34.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb93a7fc700 1 -- 172.21.2.116:0/3487861063 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb940011bd0 con 0x7fb948142110 2021-11-21T22:02:34.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb93a7fc700 1 -- 172.21.2.116:0/3487861063 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb94001b440 con 0x7fb948142110 2021-11-21T22:02:34.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb948065070 con 0x7fb948142110 2021-11-21T22:02:34.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.854+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb948145d20 con 0x7fb948142110 2021-11-21T22:02:34.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.858+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb948142c20 con 0x7fb948142110 2021-11-21T22:02:34.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.858+0000 7fb93a7fc700 1 -- 172.21.2.116:0/3487861063 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb94000f070 con 0x7fb948142110 2021-11-21T22:02:34.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.858+0000 7fb93a7fc700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb928053660 0x7fb928055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:34.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.858+0000 7fb93a7fc700 1 -- 172.21.2.116:0/3487861063 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb94006cdb0 con 0x7fb948142110 2021-11-21T22:02:34.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.858+0000 7fb94d62e700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb928053660 0x7fb928055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:34.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.858+0000 7fb94d62e700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb928053660 0x7fb928055b20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fb94400c000 tx=0x7fb944008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:34.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.858+0000 7fb93a7fc700 1 -- 172.21.2.116:0/3487861063 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb948142c20 con 0x7fb948142110 2021-11-21T22:02:34.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.942+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb948142c20 con 0x7fb948142110 2021-11-21T22:02:34.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.942+0000 7fb93a7fc700 1 -- 172.21.2.116:0/3487861063 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb948142c20 con 0x7fb948142110 2021-11-21T22:02:34.948 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:34.948 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T22:02:34.949 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.949 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T22:02:34.949 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T22:02:34.949 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:34.950 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:34.950 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T22:02:34.950 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T22:02:34.950 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:34.950 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T22:02:34.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb928053660 msgr2=0x7fb928055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb928053660 0x7fb928055b20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fb94400c000 tx=0x7fb944008040).stop 2021-11-21T22:02:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb948142110 msgr2=0x7fb948142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb948142110 0x7fb948142590 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fb948060090 tx=0x7fb940017000).stop 2021-11-21T22:02:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 shutdown_connections 2021-11-21T22:02:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb928053660 0x7fb928055b20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb948142110 0x7fb948142590 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb948065fb0 0x7fb948066430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 --2- 172.21.2.116:0/3487861063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb930004bf0 0x7fb9481453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:34.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 >> 172.21.2.116:0/3487861063 conn(0x7fb94805bd20 msgr2=0x7fb948149090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:34.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 shutdown_connections 2021-11-21T22:02:34.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:34.946+0000 7fb94f892700 1 -- 172.21.2.116:0/3487861063 wait complete. 2021-11-21T22:02:34.955 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T22:02:34.955 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:02:35.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.002+0000 7f1a2e611700 1 Processor -- start 2021-11-21T22:02:35.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.002+0000 7f1a2e611700 1 -- start start 2021-11-21T22:02:35.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1a280b6240 0x7f1a2805ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a28063000 con 0x7f1a28062c20 2021-11-21T22:02:35.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a28142dd0 con 0x7f1a280b5620 2021-11-21T22:02:35.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a28143070 con 0x7f1a280b6240 2021-11-21T22:02:35.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1a280b6240 0x7f1a2805ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a27fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f1a28062c20 0x7f1a280b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36710/0 (socket says 172.21.2.116:36710) 2021-11-21T22:02:35.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a27fff700 1 -- 172.21.2.116:0/3225912022 learned_addr learned my addr 172.21.2.116:0/3225912022 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:35.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- 172.21.2.116:0/3225912022 >> v2:172.21.2.116:3300/0 conn(0x7f1a280b6240 0x7f1a2805ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52988/0 (socket says 172.21.2.116:52988) 2021-11-21T22:02:35.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 -- 172.21.2.116:0/3225912022 >> v1:172.21.2.117:6789/0 conn(0x7f1a280b5620 legacy=0x7f1a2812ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:35.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 -- 172.21.2.116:0/3225912022 >> v1:172.21.2.116:6789/0 conn(0x7f1a28062c20 legacy=0x7f1a280b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 -- 172.21.2.116:0/3225912022 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28143310 con 0x7f1a280b6240 2021-11-21T22:02:35.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- 172.21.2.116:0/3225912022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b6240 0x7f1a2805ce60 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f1a28063430 tx=0x7f1a1c006df0).ready entity=mon.0 client_cookie=34fb11754a82e61c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:35.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a26ffd700 1 -- 172.21.2.116:0/3225912022 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1c00d5c0 con 0x7f1a280b6240 2021-11-21T22:02:35.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a26ffd700 1 -- 172.21.2.116:0/3225912022 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c000e90 con 0x7f1a280b6240 2021-11-21T22:02:35.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a26ffd700 1 -- 172.21.2.116:0/3225912022 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1c00d910 con 0x7f1a280b6240 2021-11-21T22:02:35.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- 172.21.2.116:0/3225912022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b6240 msgr2=0x7f1a2805ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 --2- 172.21.2.116:0/3225912022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b6240 0x7f1a2805ce60 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f1a28063430 tx=0x7f1a1c006df0).stop 2021-11-21T22:02:35.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- 172.21.2.116:0/3225912022 shutdown_connections 2021-11-21T22:02:35.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 --2- 172.21.2.116:0/3225912022 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b6240 0x7f1a2805ce60 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:35.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- 172.21.2.116:0/3225912022 >> 172.21.2.116:0/3225912022 conn(0x7f1a280ac9b0 msgr2=0x7f1a28069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:35.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- 172.21.2.116:0/3225912022 shutdown_connections 2021-11-21T22:02:35.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- 172.21.2.116:0/3225912022 wait complete. 2021-11-21T22:02:35.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 Processor -- start 2021-11-21T22:02:35.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- start start 2021-11-21T22:02:35.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a28062c20 0x7f1a28149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b5620 0x7f1a280af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a280b6240 0x7f1a280afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28144050 con 0x7f1a280b5620 2021-11-21T22:02:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281442f0 con 0x7f1a280b6240 2021-11-21T22:02:35.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28144850 con 0x7f1a28062c20 2021-11-21T22:02:35.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b5620 0x7f1a280af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b5620 0x7f1a280af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:52994/0 (socket says 172.21.2.116:52994) 2021-11-21T22:02:35.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 -- 172.21.2.116:0/1002668408 learned_addr learned my addr 172.21.2.116:0/1002668408 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:35.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 -- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a28062c20 msgr2=0x7f1a28149fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:35.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a28062c20 0x7f1a28149fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:35.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 -- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a280b6240 msgr2=0x7f1a280afed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:35.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a280b6240 0x7f1a280afed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 -- 172.21.2.116:0/1002668408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c00b180 con 0x7f1a280b5620 2021-11-21T22:02:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a277fe700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b5620 0x7f1a280af9e0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f1a28063430 tx=0x7f1a1c012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a257fa700 1 -- 172.21.2.116:0/1002668408 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1c016070 con 0x7f1a280b5620 2021-11-21T22:02:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a257fa700 1 -- 172.21.2.116:0/1002668408 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c002f50 con 0x7f1a280b5620 2021-11-21T22:02:35.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a28144af0 con 0x7f1a280b5620 2021-11-21T22:02:35.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a257fa700 1 -- 172.21.2.116:0/1002668408 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1c01e450 con 0x7f1a280b5620 2021-11-21T22:02:35.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.006+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a281555f0 con 0x7f1a280b5620 2021-11-21T22:02:35.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.010+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a281435b0 con 0x7f1a280b5620 2021-11-21T22:02:35.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.010+0000 7f1a257fa700 1 -- 172.21.2.116:0/1002668408 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1a1c025030 con 0x7f1a280b5620 2021-11-21T22:02:35.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.010+0000 7f1a257fa700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a080535b0 0x7f1a08055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.010+0000 7f1a257fa700 1 -- 172.21.2.116:0/1002668408 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1a1c06c710 con 0x7f1a280b5620 2021-11-21T22:02:35.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.010+0000 7f1a27fff700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a080535b0 0x7f1a08055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.010+0000 7f1a27fff700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a080535b0 0x7f1a08055a70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f1a28143930 tx=0x7f1a180077b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:35.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.010+0000 7f1a257fa700 1 -- 172.21.2.116:0/1002668408 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a1c043a90 con 0x7f1a280b5620 2021-11-21T22:02:35.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.094+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f1a280573d0 con 0x7f1a280b5620 2021-11-21T22:02:35.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a257fa700 1 -- 172.21.2.116:0/1002668408 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v138) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1a1c0230a0 con 0x7f1a280b5620 2021-11-21T22:02:35.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a080535b0 msgr2=0x7f1a08055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a080535b0 0x7f1a08055a70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f1a28143930 tx=0x7f1a180077b0).stop 2021-11-21T22:02:35.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b5620 msgr2=0x7f1a280af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b5620 0x7f1a280af9e0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f1a28063430 tx=0x7f1a1c012000).stop 2021-11-21T22:02:35.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 shutdown_connections 2021-11-21T22:02:35.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1a280b6240 0x7f1a280afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:35.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1a080535b0 0x7f1a08055a70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:35.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1a280b5620 0x7f1a280af9e0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:35.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 --2- 172.21.2.116:0/1002668408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1a28062c20 0x7f1a28149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:35.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 >> 172.21.2.116:0/1002668408 conn(0x7f1a280ac9b0 msgr2=0x7f1a28142320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:35.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 shutdown_connections 2021-11-21T22:02:35.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.134+0000 7f1a2e611700 1 -- 172.21.2.116:0/1002668408 wait complete. 2021-11-21T22:02:35.145 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:02:35.147 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:02:35.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23ec586700 1 Processor -- start 2021-11-21T22:02:35.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23ec586700 1 -- start start 2021-11-21T22:02:35.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23ec586700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f23e41478c0 0x7f23e40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23ec586700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e4147e80 con 0x7f23e40abce0 2021-11-21T22:02:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23ec586700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e40af5a0 con 0x7f23e4144810 2021-11-21T22:02:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23ec586700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e40b1490 con 0x7f23e41478c0 2021-11-21T22:02:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23e9b21700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f23e40abce0 0x7f23e40ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36730/0 (socket says 172.21.2.116:36730) 2021-11-21T22:02:35.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23e9b21700 1 -- 172.21.2.116:0/3129889807 learned_addr learned my addr 172.21.2.116:0/3129889807 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.194+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939650972 0 0) 0x7f23e4147e80 con 0x7f23e40abce0 2021-11-21T22:02:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23cc003260 con 0x7f23e40abce0 2021-11-21T22:02:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- 172.21.2.116:0/3129889807 >> v2:172.21.2.117:3300/0 conn(0x7f23e41478c0 0x7f23e40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 831999268 0 0) 0x7f23cc003260 con 0x7f23e40abce0 2021-11-21T22:02:35.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 >> v2:172.21.2.117:3300/0 conn(0x7f23e41478c0 msgr2=0x7f23e40acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:35.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 --2- 172.21.2.116:0/3129889807 >> v2:172.21.2.117:3300/0 conn(0x7f23e41478c0 0x7f23e40acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:35.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 >> v1:172.21.2.117:6789/0 conn(0x7f23e4144810 legacy=0x7f23e40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e40b1fd0 con 0x7f23e40abce0 2021-11-21T22:02:35.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f23e0001660 con 0x7f23e40abce0 2021-11-21T22:02:35.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 >> v1:172.21.2.116:6789/0 conn(0x7f23e40abce0 legacy=0x7f23e40ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc003750 0x7f23cc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23cc004500 0x7f23cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23cc004e70 0x7f23cc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e40b1490 con 0x7f23cc004e70 2021-11-21T22:02:35.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc00bd20 con 0x7f23cc003750 2021-11-21T22:02:35.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc00bee0 con 0x7f23cc004500 2021-11-21T22:02:35.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23cc004e70 0x7f23cc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ea322700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc003750 0x7f23cc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9b21700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23cc004500 0x7f23cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23cc004500 msgr2=0x7f23cc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:35.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23cc004500 0x7f23cc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:35.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc003750 msgr2=0x7f23cc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:35.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc003750 0x7f23cc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:35.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/3129889807 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc003260 con 0x7f23cc004e70 2021-11-21T22:02:35.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/3129889807 reap_dead start 2021-11-21T22:02:35.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23cc004e70 0x7f23cc00b750 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f23dc00c000 tx=0x7f23dc0091d0).ready entity=mon.0 client_cookie=65b70bae91e55c85 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:35.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23dc00e7e0 con 0x7f23cc004e70 2021-11-21T22:02:35.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc00e940 con 0x7f23cc004e70 2021-11-21T22:02:35.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23e9320700 1 -- 172.21.2.116:0/3129889807 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23dc00eca0 con 0x7f23cc004e70 2021-11-21T22:02:35.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- 172.21.2.116:0/3129889807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23cc004e70 msgr2=0x7f23cc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23cc004e70 0x7f23cc00b750 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f23dc00c000 tx=0x7f23dc0091d0).stop 2021-11-21T22:02:35.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- 172.21.2.116:0/3129889807 shutdown_connections 2021-11-21T22:02:35.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 --2- 172.21.2.116:0/3129889807 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23cc004e70 0x7f23cc00b750 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:35.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- 172.21.2.116:0/3129889807 >> 172.21.2.116:0/3129889807 conn(0x7f23e405bd20 msgr2=0x7f23e405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:35.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- 172.21.2.116:0/3129889807 shutdown_connections 2021-11-21T22:02:35.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- 172.21.2.116:0/3129889807 wait complete. 2021-11-21T22:02:35.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 Processor -- start 2021-11-21T22:02:35.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- start start 2021-11-21T22:02:35.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc004e70 0x7f23e4143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23e4139e00 0x7f23e415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23e412cd60 0x7f23e4138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e40b2ab0 con 0x7f23e412cd60 2021-11-21T22:02:35.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4159ab0 con 0x7f23cc004e70 2021-11-21T22:02:35.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23ec586700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e415a910 con 0x7f23e4139e00 2021-11-21T22:02:35.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23e412cd60 0x7f23e4138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23e412cd60 0x7f23e4138f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53014/0 (socket says 172.21.2.116:53014) 2021-11-21T22:02:35.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/4055284383 learned_addr learned my addr 172.21.2.116:0/4055284383 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:35.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23e4139e00 msgr2=0x7f23e415a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23e4139e00 0x7f23e415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:35.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc004e70 msgr2=0x7f23e4143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:35.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc004e70 0x7f23e4143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:35.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.198+0000 7f23eab23700 1 -- 172.21.2.116:0/4055284383 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e40b2ce0 con 0x7f23e412cd60 2021-11-21T22:02:35.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23eab23700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23e412cd60 0x7f23e4138f50 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f23e40b2fb0 tx=0x7f23dc019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:35.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23d77fe700 1 -- 172.21.2.116:0/4055284383 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23dc009600 con 0x7f23e412cd60 2021-11-21T22:02:35.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23d77fe700 1 -- 172.21.2.116:0/4055284383 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc01de80 con 0x7f23e412cd60 2021-11-21T22:02:35.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23d77fe700 1 -- 172.21.2.116:0/4055284383 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23dc01b640 con 0x7f23e412cd60 2021-11-21T22:02:35.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e4159ce0 con 0x7f23e412cd60 2021-11-21T22:02:35.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e412d1e0 con 0x7f23e412cd60 2021-11-21T22:02:35.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e40573d0 con 0x7f23e412cd60 2021-11-21T22:02:35.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23d77fe700 1 -- 172.21.2.116:0/4055284383 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f23dc02e070 con 0x7f23e412cd60 2021-11-21T22:02:35.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23d77fe700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f23c4053640 0x7f23c4055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:35.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23d77fe700 1 -- 172.21.2.116:0/4055284383 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f23dc06d6b0 con 0x7f23e412cd60 2021-11-21T22:02:35.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23ea322700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f23c4053640 0x7f23c4055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:35.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.202+0000 7f23ea322700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f23c4053640 0x7f23c4055b00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f23d800b000 tx=0x7f23d8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:35.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.206+0000 7f23d77fe700 1 -- 172.21.2.116:0/4055284383 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f23dc044980 con 0x7f23e412cd60 2021-11-21T22:02:35.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:35.290+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f23e40b2ab0 con 0x7f23e412cd60 2021-11-21T22:02:36.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.146+0000 7f23d77fe700 1 -- 172.21.2.116:0/4055284383 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v139) v1 ==== 125+0+0 (secure 0 0 0) 0x7f23dc047fa0 con 0x7f23e412cd60 2021-11-21T22:02:36.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f23c4053640 msgr2=0x7f23c4055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f23c4053640 0x7f23c4055b00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f23d800b000 tx=0x7f23d8009040).stop 2021-11-21T22:02:36.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23e412cd60 msgr2=0x7f23e4138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23e412cd60 0x7f23e4138f50 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f23e40b2fb0 tx=0x7f23dc019000).stop 2021-11-21T22:02:36.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 shutdown_connections 2021-11-21T22:02:36.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f23c4053640 0x7f23c4055b00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23e412cd60 0x7f23e4138f50 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23e4139e00 0x7f23e415a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 --2- 172.21.2.116:0/4055284383 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23cc004e70 0x7f23e4143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 >> 172.21.2.116:0/4055284383 conn(0x7f23e405bd20 msgr2=0x7f23e40ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 shutdown_connections 2021-11-21T22:02:36.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.150+0000 7f23ec586700 1 -- 172.21.2.116:0/4055284383 wait complete. 2021-11-21T22:02:36.157 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:02:36.161 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:02:36.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa362f700 1 Processor -- start 2021-11-21T22:02:36.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa362f700 1 -- start start 2021-11-21T22:02:36.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa362f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fef9c0b11c0 0x7fef9c147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa362f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fef9c143e60 0x7fef9c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa362f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c0b1940 con 0x7fef9c0a8670 2021-11-21T22:02:36.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa362f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c0b15a0 con 0x7fef9c143e60 2021-11-21T22:02:36.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa362f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c0b1b70 con 0x7fef9c0b11c0 2021-11-21T22:02:36.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa1bcc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fef9c143e60 0x7fef9c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa1bcc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fef9c143e60 0x7fef9c1442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53026/0 (socket says 172.21.2.116:53026) 2021-11-21T22:02:36.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa1bcc700 1 -- 172.21.2.116:0/3758200960 learned_addr learned my addr 172.21.2.116:0/3758200960 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:36.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa1bcc700 1 -- 172.21.2.116:0/3758200960 >> v2:172.21.2.117:3300/0 conn(0x7fef9c0b11c0 msgr2=0x7fef9c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:36.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa1bcc700 1 --2- 172.21.2.116:0/3758200960 >> v2:172.21.2.117:3300/0 conn(0x7fef9c0b11c0 0x7fef9c147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa1bcc700 1 -- 172.21.2.116:0/3758200960 >> v1:172.21.2.117:6789/0 conn(0x7fef9c0a8670 legacy=0x7fef9c147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:36.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.210+0000 7fefa1bcc700 1 -- 172.21.2.116:0/3758200960 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c0b1e10 con 0x7fef9c143e60 2021-11-21T22:02:36.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 --2- 172.21.2.116:0/3758200960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1442d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7fef9c147180 tx=0x7fef94009290).ready entity=mon.0 client_cookie=d30275e3c10d28d7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fef8bfff700 1 -- 172.21.2.116:0/3758200960 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef9400b830 con 0x7fef9c143e60 2021-11-21T22:02:36.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fef8bfff700 1 -- 172.21.2.116:0/3758200960 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9400b990 con 0x7fef9c143e60 2021-11-21T22:02:36.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fef8bfff700 1 -- 172.21.2.116:0/3758200960 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef9400bb40 con 0x7fef9c143e60 2021-11-21T22:02:36.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/3758200960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 msgr2=0x7fef9c1442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 --2- 172.21.2.116:0/3758200960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1442d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7fef9c147180 tx=0x7fef94009290).stop 2021-11-21T22:02:36.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/3758200960 shutdown_connections 2021-11-21T22:02:36.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 --2- 172.21.2.116:0/3758200960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1442d0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 --2- 172.21.2.116:0/3758200960 >> v2:172.21.2.117:3300/0 conn(0x7fef9c0b11c0 0x7fef9c147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/3758200960 >> 172.21.2.116:0/3758200960 conn(0x7fef9c05bd20 msgr2=0x7fef9c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/3758200960 shutdown_connections 2021-11-21T22:02:36.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/3758200960 wait complete. 2021-11-21T22:02:36.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 Processor -- start 2021-11-21T22:02:36.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- start start 2021-11-21T22:02:36.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fef9c0a8670 0x7fef9c066960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fef9c0b11c0 0x7fef9c141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c065e00 con 0x7fef9c143e60 2021-11-21T22:02:36.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0b5970 con 0x7fef9c0a8670 2021-11-21T22:02:36.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0b5ba0 con 0x7fef9c0b11c0 2021-11-21T22:02:36.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53032/0 (socket says 172.21.2.116:53032) 2021-11-21T22:02:36.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 -- 172.21.2.116:0/4268482848 learned_addr learned my addr 172.21.2.116:0/4268482848 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:36.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 -- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fef9c0b11c0 msgr2=0x7fef9c141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fef9c0b11c0 0x7fef9c141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 -- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fef9c0a8670 msgr2=0x7fef9c066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fef9c0a8670 0x7fef9c066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 -- 172.21.2.116:0/4268482848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94009530 con 0x7fef9c143e60 2021-11-21T22:02:36.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa1bcc700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1415f0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fef9c066570 tx=0x7fef94016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fef8a7fc700 1 -- 172.21.2.116:0/4268482848 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef9400b830 con 0x7fef9c143e60 2021-11-21T22:02:36.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fef8a7fc700 1 -- 172.21.2.116:0/4268482848 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9401f600 con 0x7fef9c143e60 2021-11-21T22:02:36.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fef8a7fc700 1 -- 172.21.2.116:0/4268482848 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef9401f960 con 0x7fef9c143e60 2021-11-21T22:02:36.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c156ed0 con 0x7fef9c143e60 2021-11-21T22:02:36.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c157410 con 0x7fef9c143e60 2021-11-21T22:02:36.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0639c0 con 0x7fef9c143e60 2021-11-21T22:02:36.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.214+0000 7fef8a7fc700 1 -- 172.21.2.116:0/4268482848 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fef9400f070 con 0x7fef9c143e60 2021-11-21T22:02:36.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.218+0000 7fef8a7fc700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef7c0536a0 0x7fef7c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.218+0000 7fef8a7fc700 1 -- 172.21.2.116:0/4268482848 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fef9406c770 con 0x7fef9c143e60 2021-11-21T22:02:36.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.218+0000 7fefa13cb700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef7c0536a0 0x7fef7c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.218+0000 7fefa13cb700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef7c0536a0 0x7fef7c055b60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fef98002f70 tx=0x7fef98006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.218+0000 7fef8a7fc700 1 -- 172.21.2.116:0/4268482848 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef9c0639c0 con 0x7fef9c143e60 2021-11-21T22:02:36.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.330+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fef9c0639c0 con 0x7fef9c143e60 2021-11-21T22:02:36.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fef8a7fc700 1 -- 172.21.2.116:0/4268482848 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v64) v1 ==== 189+0+0 (secure 0 0 0) 0x7fef9c0639c0 con 0x7fef9c143e60 2021-11-21T22:02:36.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef7c0536a0 msgr2=0x7fef7c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef7c0536a0 0x7fef7c055b60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fef98002f70 tx=0x7fef98006a80).stop 2021-11-21T22:02:36.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 msgr2=0x7fef9c1415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1415f0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fef9c066570 tx=0x7fef94016000).stop 2021-11-21T22:02:36.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 shutdown_connections 2021-11-21T22:02:36.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fef7c0536a0 0x7fef7c055b60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fef9c143e60 0x7fef9c1415f0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fef9c0b11c0 0x7fef9c141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 --2- 172.21.2.116:0/4268482848 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fef9c0a8670 0x7fef9c066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 >> 172.21.2.116:0/4268482848 conn(0x7fef9c05bd20 msgr2=0x7fef9c14c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 shutdown_connections 2021-11-21T22:02:36.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.334+0000 7fefa362f700 1 -- 172.21.2.116:0/4268482848 wait complete. 2021-11-21T22:02:36.344 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 25 and data pool 26 2021-11-21T22:02:36.347 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:36.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.390+0000 7f4d2531d700 1 Processor -- start 2021-11-21T22:02:36.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- start start 2021-11-21T22:02:36.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4d200b57d0 0x7f4d20141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f4d200ab3c0 0x7f4d200ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d200b5bb0 con 0x7f4d20060e60 2021-11-21T22:02:36.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2005f7d0 con 0x7f4d200b57d0 2021-11-21T22:02:36.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2005fa30 con 0x7f4d200ab3c0 2021-11-21T22:02:36.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4d200b57d0 0x7f4d20141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4d200b57d0 0x7f4d20141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53046/0 (socket says 172.21.2.116:53046) 2021-11-21T22:02:36.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/4255098242 learned_addr learned my addr 172.21.2.116:0/4255098242 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:36.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/4255098242 >> v2:172.21.2.117:3300/0 conn(0x7f4d200ab3c0 msgr2=0x7f4d200ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:36.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- 172.21.2.116:0/4255098242 >> v2:172.21.2.117:3300/0 conn(0x7f4d200ab3c0 0x7f4d200ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/4255098242 >> v1:172.21.2.117:6789/0 conn(0x7f4d20060e60 legacy=0x7f4d200af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:36.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/4255098242 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2005fcd0 con 0x7f4d200b57d0 2021-11-21T22:02:36.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- 172.21.2.116:0/4255098242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d200b57d0 0x7f4d20141250 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f4d20060c20 tx=0x7f4d10006eb0).ready entity=mon.0 client_cookie=5db9faeb432a7150 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1effd700 1 -- 172.21.2.116:0/4255098242 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d1000d5c0 con 0x7f4d200b57d0 2021-11-21T22:02:36.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1effd700 1 -- 172.21.2.116:0/4255098242 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10000e90 con 0x7f4d200b57d0 2021-11-21T22:02:36.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1effd700 1 -- 172.21.2.116:0/4255098242 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d1000d910 con 0x7f4d200b57d0 2021-11-21T22:02:36.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- 172.21.2.116:0/4255098242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d200b57d0 msgr2=0x7f4d20141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- 172.21.2.116:0/4255098242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d200b57d0 0x7f4d20141250 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f4d20060c20 tx=0x7f4d10006eb0).stop 2021-11-21T22:02:36.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- 172.21.2.116:0/4255098242 shutdown_connections 2021-11-21T22:02:36.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- 172.21.2.116:0/4255098242 >> v2:172.21.2.117:3300/0 conn(0x7f4d200ab3c0 0x7f4d200ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- 172.21.2.116:0/4255098242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d200b57d0 0x7f4d20141250 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- 172.21.2.116:0/4255098242 >> 172.21.2.116:0/4255098242 conn(0x7f4d2005c7f0 msgr2=0x7f4d200b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- 172.21.2.116:0/4255098242 shutdown_connections 2021-11-21T22:02:36.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- 172.21.2.116:0/4255098242 wait complete. 2021-11-21T22:02:36.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 Processor -- start 2021-11-21T22:02:36.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- start start 2021-11-21T22:02:36.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d20060e60 0x7f4d200628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d200ab3c0 0x7f4d20064a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d200b57d0 0x7f4d20090540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20066f60 con 0x7f4d20060e60 2021-11-21T22:02:36.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200673f0 con 0x7f4d200ab3c0 2021-11-21T22:02:36.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d2531d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200652a0 con 0x7f4d200b57d0 2021-11-21T22:02:36.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d20060e60 0x7f4d200628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d20060e60 0x7f4d200628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53052/0 (socket says 172.21.2.116:53052) 2021-11-21T22:02:36.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/1523459634 learned_addr learned my addr 172.21.2.116:0/1523459634 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:36.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d200b57d0 msgr2=0x7f4d20090540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d200b57d0 0x7f4d20090540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d200ab3c0 msgr2=0x7f4d20064a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d200ab3c0 0x7f4d20064a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 -- 172.21.2.116:0/1523459634 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1000b1e0 con 0x7f4d20060e60 2021-11-21T22:02:36.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1ffff700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d20060e60 0x7f4d200628a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f4d20064230 tx=0x7f4d10015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.394+0000 7f4d1d7fa700 1 -- 172.21.2.116:0/1523459634 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d10002f70 con 0x7f4d20060e60 2021-11-21T22:02:36.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1d7fa700 1 -- 172.21.2.116:0/1523459634 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1000d940 con 0x7f4d20060e60 2021-11-21T22:02:36.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d20067150 con 0x7f4d20060e60 2021-11-21T22:02:36.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1d7fa700 1 -- 172.21.2.116:0/1523459634 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d10011e90 con 0x7f4d20060e60 2021-11-21T22:02:36.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d20065730 con 0x7f4d20060e60 2021-11-21T22:02:36.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2005ff70 con 0x7f4d20060e60 2021-11-21T22:02:36.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1d7fa700 1 -- 172.21.2.116:0/1523459634 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4d1001a040 con 0x7f4d20060e60 2021-11-21T22:02:36.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1d7fa700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d040536a0 0x7f4d04055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1d7fa700 1 -- 172.21.2.116:0/1523459634 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4d1006c980 con 0x7f4d20060e60 2021-11-21T22:02:36.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1f7fe700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d040536a0 0x7f4d04055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1f7fe700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d040536a0 0x7f4d04055b60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f4d08002ef0 tx=0x7f4d08006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.398+0000 7f4d1d7fa700 1 -- 172.21.2.116:0/1523459634 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d10043a40 con 0x7f4d20060e60 2021-11-21T22:02:36.487 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.482+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d200573d0 con 0x7f4d20060e60 2021-11-21T22:02:36.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.482+0000 7f4d1d7fa700 1 -- 172.21.2.116:0/1523459634 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f4d10023070 con 0x7f4d20060e60 2021-11-21T22:02:36.489 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:36.489 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":140,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:36.333726+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:35.110650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:35.304135+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:36.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d040536a0 msgr2=0x7f4d04055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d040536a0 0x7f4d04055b60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f4d08002ef0 tx=0x7f4d08006910).stop 2021-11-21T22:02:36.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d20060e60 msgr2=0x7f4d200628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d20060e60 0x7f4d200628a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f4d20064230 tx=0x7f4d10015000).stop 2021-11-21T22:02:36.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 shutdown_connections 2021-11-21T22:02:36.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4d200b57d0 0x7f4d20090540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4d200ab3c0 0x7f4d20064a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4d040536a0 0x7f4d04055b60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 --2- 172.21.2.116:0/1523459634 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4d20060e60 0x7f4d200628a0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 >> 172.21.2.116:0/1523459634 conn(0x7f4d2005c7f0 msgr2=0x7f4d200b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 shutdown_connections 2021-11-21T22:02:36.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.486+0000 7f4d2531d700 1 -- 172.21.2.116:0/1523459634 wait complete. 2021-11-21T22:02:36.496 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:36.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f157a2a6700 1 Processor -- start 2021-11-21T22:02:36.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f157a2a6700 1 -- start start 2021-11-21T22:02:36.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f157a2a6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f15740a6aa0 0x7f157414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f157a2a6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a86b0 0x7f15741470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f157a2a6700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15740a6e80 con 0x7f15741475d0 2021-11-21T22:02:36.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f157a2a6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15740a8bc0 con 0x7f15740a6aa0 2021-11-21T22:02:36.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f157a2a6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157414a7b0 con 0x7f15740a86b0 2021-11-21T22:02:36.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f1573fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f15740a6aa0 0x7f157414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f1578843700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f15741475d0 0x7f15741443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36788/0 (socket says 172.21.2.116:36788) 2021-11-21T22:02:36.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.542+0000 7f1578843700 1 -- 172.21.2.116:0/3389152272 learned_addr learned my addr 172.21.2.116:0/3389152272 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:36.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1573fff700 1 -- 172.21.2.116:0/3389152272 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a86b0 msgr2=0x7f15741470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:36.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1573fff700 1 --2- 172.21.2.116:0/3389152272 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a86b0 0x7f15741470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1573fff700 1 -- 172.21.2.116:0/3389152272 >> v1:172.21.2.116:6789/0 conn(0x7f15741475d0 legacy=0x7f15741443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1573fff700 1 -- 172.21.2.116:0/3389152272 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157414aa10 con 0x7f15740a6aa0 2021-11-21T22:02:36.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1573fff700 1 --2- 172.21.2.116:0/3389152272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15740a6aa0 0x7f157414b150 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f1564002f70 tx=0x7f1564006df0).ready entity=mon.0 client_cookie=b642f8f0f4c39ac server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1572ffd700 1 -- 172.21.2.116:0/3389152272 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f156400d570 con 0x7f15740a6aa0 2021-11-21T22:02:36.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1572ffd700 1 -- 172.21.2.116:0/3389152272 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1564000e90 con 0x7f15740a6aa0 2021-11-21T22:02:36.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1572ffd700 1 -- 172.21.2.116:0/3389152272 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f156400d900 con 0x7f15740a6aa0 2021-11-21T22:02:36.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/3389152272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15740a6aa0 msgr2=0x7f157414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 --2- 172.21.2.116:0/3389152272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15740a6aa0 0x7f157414b150 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f1564002f70 tx=0x7f1564006df0).stop 2021-11-21T22:02:36.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/3389152272 shutdown_connections 2021-11-21T22:02:36.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 --2- 172.21.2.116:0/3389152272 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a86b0 0x7f15741470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 --2- 172.21.2.116:0/3389152272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15740a6aa0 0x7f157414b150 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/3389152272 >> 172.21.2.116:0/3389152272 conn(0x7f157405bd20 msgr2=0x7f157405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/3389152272 shutdown_connections 2021-11-21T22:02:36.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/3389152272 wait complete. 2021-11-21T22:02:36.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 Processor -- start 2021-11-21T22:02:36.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- start start 2021-11-21T22:02:36.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a6aa0 0x7f15741450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15740a86b0 0x7f1574065700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15741475d0 0x7f1574065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1574156e80 con 0x7f15741475d0 2021-11-21T22:02:36.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15740667b0 con 0x7f15740a86b0 2021-11-21T22:02:36.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1574064eb0 con 0x7f15740a6aa0 2021-11-21T22:02:36.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15741475d0 0x7f1574065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15741475d0 0x7f1574065bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53068/0 (socket says 172.21.2.116:53068) 2021-11-21T22:02:36.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 -- 172.21.2.116:0/398944322 learned_addr learned my addr 172.21.2.116:0/398944322 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:36.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 -- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a6aa0 msgr2=0x7f15741450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a6aa0 0x7f15741450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 -- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15740a86b0 msgr2=0x7f1574065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15740a86b0 0x7f1574065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 -- 172.21.2.116:0/398944322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156400b180 con 0x7f15741475d0 2021-11-21T22:02:36.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f1578843700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15741475d0 0x7f1574065bf0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f1574061ab0 tx=0x7f156c0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f15717fa700 1 -- 172.21.2.116:0/398944322 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f156c005ba0 con 0x7f15741475d0 2021-11-21T22:02:36.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f15717fa700 1 -- 172.21.2.116:0/398944322 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c005d00 con 0x7f15741475d0 2021-11-21T22:02:36.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1574157060 con 0x7f15741475d0 2021-11-21T22:02:36.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f15717fa700 1 -- 172.21.2.116:0/398944322 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f156c0072c0 con 0x7f15741475d0 2021-11-21T22:02:36.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15740660e0 con 0x7f15741475d0 2021-11-21T22:02:36.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15740650b0 con 0x7f15741475d0 2021-11-21T22:02:36.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.546+0000 7f15717fa700 1 -- 172.21.2.116:0/398944322 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f156c01a060 con 0x7f15741475d0 2021-11-21T22:02:36.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.550+0000 7f15717fa700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1558053640 0x7f1558055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.550+0000 7f15717fa700 1 -- 172.21.2.116:0/398944322 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f156c06b350 con 0x7f15741475d0 2021-11-21T22:02:36.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.550+0000 7f1573fff700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1558053640 0x7f1558055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.550+0000 7f1573fff700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1558053640 0x7f1558055b00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f1564002f70 tx=0x7f1564015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.550+0000 7f15717fa700 1 -- 172.21.2.116:0/398944322 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f156c0424b0 con 0x7f15741475d0 2021-11-21T22:02:36.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15741571f0 con 0x7f15741475d0 2021-11-21T22:02:36.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f15717fa700 1 -- 172.21.2.116:0/398944322 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f156c018070 con 0x7f15741475d0 2021-11-21T22:02:36.640 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:36.640 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":140,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:36.333726+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:35.110650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:35.304135+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:36.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1558053640 msgr2=0x7f1558055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1558053640 0x7f1558055b00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f1564002f70 tx=0x7f1564015000).stop 2021-11-21T22:02:36.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15741475d0 msgr2=0x7f1574065bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15741475d0 0x7f1574065bf0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f1574061ab0 tx=0x7f156c0013a0).stop 2021-11-21T22:02:36.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 shutdown_connections 2021-11-21T22:02:36.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1558053640 0x7f1558055b00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f15741475d0 0x7f1574065bf0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15740a86b0 0x7f1574065700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 --2- 172.21.2.116:0/398944322 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15740a6aa0 0x7f15741450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 >> 172.21.2.116:0/398944322 conn(0x7f157405bd20 msgr2=0x7f157413f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 shutdown_connections 2021-11-21T22:02:36.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.634+0000 7f157a2a6700 1 -- 172.21.2.116:0/398944322 wait complete. 2021-11-21T22:02:36.647 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:02:36.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.694+0000 7f3ad06c6700 1 Processor -- start 2021-11-21T22:02:36.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.694+0000 7f3ad06c6700 1 -- start start 2021-11-21T22:02:36.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ac8144810 0x7f3ac81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac80abce0 0x7f3ac80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f3ac80b1db0 0x7f3ac80ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac8147e80 con 0x7f3ac8144810 2021-11-21T22:02:36.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac80af5a0 con 0x7f3ac80b1db0 2021-11-21T22:02:36.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80b1490 con 0x7f3ac80abce0 2021-11-21T22:02:36.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ac8144810 0x7f3ac81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3ac8144810 0x7f3ac81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53074/0 (socket says 172.21.2.116:53074) 2021-11-21T22:02:36.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/1406168890 learned_addr learned my addr 172.21.2.116:0/1406168890 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:36.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/1406168890 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac80abce0 msgr2=0x7f3ac80b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:36.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- 172.21.2.116:0/1406168890 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac80abce0 0x7f3ac80b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/1406168890 >> v2:172.21.2.117:3300/0 conn(0x7f3ac80b1db0 msgr2=0x7f3ac80ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:36.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- 172.21.2.116:0/1406168890 >> v2:172.21.2.117:3300/0 conn(0x7f3ac80b1db0 0x7f3ac80ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/1406168890 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80af060 con 0x7f3ac8144810 2021-11-21T22:02:36.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- 172.21.2.116:0/1406168890 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac8144810 0x7f3ac81478c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f3abc002f70 tx=0x7f3abc006eb0).ready entity=mon.0 client_cookie=b8884a261559ab75 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3acd460700 1 -- 172.21.2.116:0/1406168890 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3abc00d570 con 0x7f3ac8144810 2021-11-21T22:02:36.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3acd460700 1 -- 172.21.2.116:0/1406168890 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc000e90 con 0x7f3ac8144810 2021-11-21T22:02:36.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3acd460700 1 -- 172.21.2.116:0/1406168890 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3abc00d900 con 0x7f3ac8144810 2021-11-21T22:02:36.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- 172.21.2.116:0/1406168890 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac8144810 msgr2=0x7f3ac81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:36.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/1406168890 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac8144810 0x7f3ac81478c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f3abc002f70 tx=0x7f3abc006eb0).stop 2021-11-21T22:02:36.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- 172.21.2.116:0/1406168890 shutdown_connections 2021-11-21T22:02:36.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/1406168890 >> v2:172.21.2.117:3300/0 conn(0x7f3ac80b1db0 0x7f3ac80ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/1406168890 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac80abce0 0x7f3ac80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/1406168890 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac8144810 0x7f3ac81478c0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:36.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- 172.21.2.116:0/1406168890 >> 172.21.2.116:0/1406168890 conn(0x7f3ac805bd20 msgr2=0x7f3ac805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:36.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- 172.21.2.116:0/1406168890 shutdown_connections 2021-11-21T22:02:36.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- 172.21.2.116:0/1406168890 wait complete. 2021-11-21T22:02:36.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 Processor -- start 2021-11-21T22:02:36.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- start start 2021-11-21T22:02:36.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac80abce0 0x7f3ac8143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac8144810 0x7f3ac8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac8064320 0x7f3ac8156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8063240 con 0x7f3ac80abce0 2021-11-21T22:02:36.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac815a590 con 0x7f3ac8064320 2021-11-21T22:02:36.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac815a830 con 0x7f3ac8144810 2021-11-21T22:02:36.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac80abce0 0x7f3ac8143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac80abce0 0x7f3ac8143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53078/0 (socket says 172.21.2.116:53078) 2021-11-21T22:02:36.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/3603201809 learned_addr learned my addr 172.21.2.116:0/3603201809 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:36.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac8144810 msgr2=0x7f3ac8156630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:36.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac8144810 0x7f3ac8156630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac8064320 msgr2=0x7f3ac8156b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:36.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac8064320 0x7f3ac8156b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:36.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 -- 172.21.2.116:0/3603201809 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc00b1e0 con 0x7f3ac80abce0 2021-11-21T22:02:36.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ace462700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac80abce0 0x7f3ac8143510 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f3abc002f70 tx=0x7f3abc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3abb7fe700 1 -- 172.21.2.116:0/3603201809 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3abc00de40 con 0x7f3ac80abce0 2021-11-21T22:02:36.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3abb7fe700 1 -- 172.21.2.116:0/3603201809 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc011ce0 con 0x7f3ac80abce0 2021-11-21T22:02:36.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac8063be0 con 0x7f3ac80abce0 2021-11-21T22:02:36.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3abb7fe700 1 -- 172.21.2.116:0/3603201809 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3abc01e650 con 0x7f3ac80abce0 2021-11-21T22:02:36.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.698+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac80b3c00 con 0x7f3ac80abce0 2021-11-21T22:02:36.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.702+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac80af300 con 0x7f3ac80abce0 2021-11-21T22:02:36.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.702+0000 7f3abb7fe700 1 -- 172.21.2.116:0/3603201809 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3abc025030 con 0x7f3ac80abce0 2021-11-21T22:02:36.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.702+0000 7f3abb7fe700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa80536f0 0x7f3aa8055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:36.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.702+0000 7f3abb7fe700 1 -- 172.21.2.116:0/3603201809 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3abc06cee0 con 0x7f3ac80abce0 2021-11-21T22:02:36.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.702+0000 7f3acdc61700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa80536f0 0x7f3aa8055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:36.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.702+0000 7f3acdc61700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa80536f0 0x7f3aa8055bb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f3ac4002fb0 tx=0x7f3ac4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:36.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.702+0000 7f3abb7fe700 1 -- 172.21.2.116:0/3603201809 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3abc043f30 con 0x7f3ac80abce0 2021-11-21T22:02:36.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:36.810+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3ac80573d0 con 0x7f3ac80abce0 2021-11-21T22:02:37.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3abb7fe700 1 -- 172.21.2.116:0/3603201809 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3abc023020 con 0x7f3ac80abce0 2021-11-21T22:02:37.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa80536f0 msgr2=0x7f3aa8055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa80536f0 0x7f3aa8055bb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f3ac4002fb0 tx=0x7f3ac4006930).stop 2021-11-21T22:02:37.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac80abce0 msgr2=0x7f3ac8143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac80abce0 0x7f3ac8143510 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f3abc002f70 tx=0x7f3abc015000).stop 2021-11-21T22:02:37.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 shutdown_connections 2021-11-21T22:02:37.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3ac8064320 0x7f3ac8156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3ac8144810 0x7f3ac8156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3aa80536f0 0x7f3aa8055bb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 --2- 172.21.2.116:0/3603201809 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3ac80abce0 0x7f3ac8143510 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 >> 172.21.2.116:0/3603201809 conn(0x7f3ac805bd20 msgr2=0x7f3ac80af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 shutdown_connections 2021-11-21T22:02:37.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.350+0000 7f3ad06c6700 1 -- 172.21.2.116:0/3603201809 wait complete. 2021-11-21T22:02:37.361 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:37.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 Processor -- start 2021-11-21T22:02:37.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- start start 2021-11-21T22:02:37.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd1640a86b0 0x7fd16414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd164062410 0x7fd1641443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1640a6e80 con 0x7fd1640a6aa0 2021-11-21T22:02:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1640a8bc0 con 0x7fd164062410 2021-11-21T22:02:37.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16414a7b0 con 0x7fd1640a86b0 2021-11-21T22:02:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd16359e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd164062410 0x7fd1641443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd162d9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd1640a6aa0 0x7fd1641470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36810/0 (socket says 172.21.2.116:36810) 2021-11-21T22:02:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd162d9d700 1 -- 172.21.2.116:0/2097876852 learned_addr learned my addr 172.21.2.116:0/2097876852 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:37.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd16359e700 1 -- 172.21.2.116:0/2097876852 >> v2:172.21.2.117:3300/0 conn(0x7fd1640a86b0 msgr2=0x7fd16414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:37.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd16359e700 1 --2- 172.21.2.116:0/2097876852 >> v2:172.21.2.117:3300/0 conn(0x7fd1640a86b0 0x7fd16414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd16359e700 1 -- 172.21.2.116:0/2097876852 >> v1:172.21.2.116:6789/0 conn(0x7fd1640a6aa0 legacy=0x7fd1641470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd16359e700 1 -- 172.21.2.116:0/2097876852 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16414aa10 con 0x7fd164062410 2021-11-21T22:02:37.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd16359e700 1 --2- 172.21.2.116:0/2097876852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd164062410 0x7fd1641443f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fd15c0032f0 tx=0x7fd15c0091d0).ready entity=mon.0 client_cookie=c54001a7c55c8bbc server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd161d9b700 1 -- 172.21.2.116:0/2097876852 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd15c009ea0 con 0x7fd164062410 2021-11-21T22:02:37.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd161d9b700 1 -- 172.21.2.116:0/2097876852 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c00b870 con 0x7fd164062410 2021-11-21T22:02:37.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd161d9b700 1 -- 172.21.2.116:0/2097876852 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd15c00b9e0 con 0x7fd164062410 2021-11-21T22:02:37.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- 172.21.2.116:0/2097876852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd164062410 msgr2=0x7fd1641443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 --2- 172.21.2.116:0/2097876852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd164062410 0x7fd1641443f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fd15c0032f0 tx=0x7fd15c0091d0).stop 2021-11-21T22:02:37.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- 172.21.2.116:0/2097876852 shutdown_connections 2021-11-21T22:02:37.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 --2- 172.21.2.116:0/2097876852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd164062410 0x7fd1641443f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 --2- 172.21.2.116:0/2097876852 >> v2:172.21.2.117:3300/0 conn(0x7fd1640a86b0 0x7fd16414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- 172.21.2.116:0/2097876852 >> 172.21.2.116:0/2097876852 conn(0x7fd16405bd20 msgr2=0x7fd16405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- 172.21.2.116:0/2097876852 shutdown_connections 2021-11-21T22:02:37.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.418+0000 7fd169096700 1 -- 172.21.2.116:0/2097876852 wait complete. 2021-11-21T22:02:37.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 Processor -- start 2021-11-21T22:02:37.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 -- start start 2021-11-21T22:02:37.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd164062410 0x7fd1641464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1640a6aa0 0x7fd164156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd1640a86b0 0x7fd1640659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd164064d10 con 0x7fd1640a6aa0 2021-11-21T22:02:37.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1640638c0 con 0x7fd1640a86b0 2021-11-21T22:02:37.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd164064f40 con 0x7fd164062410 2021-11-21T22:02:37.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1640a6aa0 0x7fd164156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1640a6aa0 0x7fd164156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53094/0 (socket says 172.21.2.116:53094) 2021-11-21T22:02:37.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 -- 172.21.2.116:0/2988706223 learned_addr learned my addr 172.21.2.116:0/2988706223 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:37.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 -- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd164062410 msgr2=0x7fd1641464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd164062410 0x7fd1641464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 -- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd1640a86b0 msgr2=0x7fd1640659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd1640a86b0 0x7fd1640659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 -- 172.21.2.116:0/2988706223 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c009470 con 0x7fd1640a6aa0 2021-11-21T22:02:37.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd16259c700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1640a6aa0 0x7fd164156d70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7fd158002f70 tx=0x7fd158006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd153fff700 1 -- 172.21.2.116:0/2988706223 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd158013460 con 0x7fd1640a6aa0 2021-11-21T22:02:37.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd153fff700 1 -- 172.21.2.116:0/2988706223 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1580135c0 con 0x7fd1640a6aa0 2021-11-21T22:02:37.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd153fff700 1 -- 172.21.2.116:0/2988706223 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd158013920 con 0x7fd1640a6aa0 2021-11-21T22:02:37.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1640651e0 con 0x7fd1640a6aa0 2021-11-21T22:02:37.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1640abb20 con 0x7fd1640a6aa0 2021-11-21T22:02:37.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1640abdd0 con 0x7fd1640a6aa0 2021-11-21T22:02:37.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd153fff700 1 -- 172.21.2.116:0/2988706223 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd15801a060 con 0x7fd1640a6aa0 2021-11-21T22:02:37.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd153fff700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1400536f0 0x7fd140055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd153fff700 1 -- 172.21.2.116:0/2988706223 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd15806b490 con 0x7fd1640a6aa0 2021-11-21T22:02:37.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.422+0000 7fd162d9d700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1400536f0 0x7fd140055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.426+0000 7fd162d9d700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1400536f0 0x7fd140055bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fd164061ab0 tx=0x7fd1540077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.426+0000 7fd153fff700 1 -- 172.21.2.116:0/2988706223 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd158042480 con 0x7fd1640a6aa0 2021-11-21T22:02:37.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1640573d0 con 0x7fd1640a6aa0 2021-11-21T22:02:37.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd153fff700 1 -- 172.21.2.116:0/2988706223 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fd158018090 con 0x7fd1640a6aa0 2021-11-21T22:02:37.540 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:37.540 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:36.333834+0000","modified":"2021-11-21T22:02:37.338971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/1832322648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1832322648},{"type":"v1","addr":"172.21.2.116:6835","nonce":1832322648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-11-21T22:02:37.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1400536f0 msgr2=0x7fd140055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1400536f0 0x7fd140055bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fd164061ab0 tx=0x7fd1540077f0).stop 2021-11-21T22:02:37.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1640a6aa0 msgr2=0x7fd164156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1640a6aa0 0x7fd164156d70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7fd158002f70 tx=0x7fd158006e20).stop 2021-11-21T22:02:37.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 shutdown_connections 2021-11-21T22:02:37.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd1640a86b0 0x7fd1640659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1400536f0 0x7fd140055bb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1640a6aa0 0x7fd164156d70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 --2- 172.21.2.116:0/2988706223 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd164062410 0x7fd1641464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 >> 172.21.2.116:0/2988706223 conn(0x7fd16405bd20 msgr2=0x7fd1641441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 shutdown_connections 2021-11-21T22:02:37.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.534+0000 7fd169096700 1 -- 172.21.2.116:0/2988706223 wait complete. 2021-11-21T22:02:37.544 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 65 2021-11-21T22:02:37.547 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:37.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 Processor -- start 2021-11-21T22:02:37.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- start start 2021-11-21T22:02:37.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b40a86b0 0x7f09b414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f09b4062410 0x7f09b41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b40a6e80 con 0x7f09b40a6aa0 2021-11-21T22:02:37.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b40a8bc0 con 0x7f09b4062410 2021-11-21T22:02:37.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b414a7b0 con 0x7f09b40a86b0 2021-11-21T22:02:37.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f09b4062410 0x7f09b41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f09b4062410 0x7f09b41443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53102/0 (socket says 172.21.2.116:53102) 2021-11-21T22:02:37.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 -- 172.21.2.116:0/313972125 learned_addr learned my addr 172.21.2.116:0/313972125 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:37.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 -- 172.21.2.116:0/313972125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b40a86b0 msgr2=0x7f09b414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:37.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 --2- 172.21.2.116:0/313972125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b40a86b0 0x7f09b414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 -- 172.21.2.116:0/313972125 >> v1:172.21.2.117:6789/0 conn(0x7f09b40a6aa0 legacy=0x7f09b41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:37.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 -- 172.21.2.116:0/313972125 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b414aa10 con 0x7f09b4062410 2021-11-21T22:02:37.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09ba5bc700 1 --2- 172.21.2.116:0/313972125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b4062410 0x7f09b41443f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f09b4061ab0 tx=0x7f09ac009290).ready entity=mon.0 client_cookie=8221b7865eaaefc2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09b8db9700 1 -- 172.21.2.116:0/313972125 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09ac00b830 con 0x7f09b4062410 2021-11-21T22:02:37.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09b8db9700 1 -- 172.21.2.116:0/313972125 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac00b990 con 0x7f09b4062410 2021-11-21T22:02:37.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09b8db9700 1 -- 172.21.2.116:0/313972125 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09ac00bb00 con 0x7f09b4062410 2021-11-21T22:02:37.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- 172.21.2.116:0/313972125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b4062410 msgr2=0x7f09b41443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 --2- 172.21.2.116:0/313972125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b4062410 0x7f09b41443f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f09b4061ab0 tx=0x7f09ac009290).stop 2021-11-21T22:02:37.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- 172.21.2.116:0/313972125 shutdown_connections 2021-11-21T22:02:37.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 --2- 172.21.2.116:0/313972125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b4062410 0x7f09b41443f0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 --2- 172.21.2.116:0/313972125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b40a86b0 0x7f09b414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- 172.21.2.116:0/313972125 >> 172.21.2.116:0/313972125 conn(0x7f09b405bd20 msgr2=0x7f09b405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- 172.21.2.116:0/313972125 shutdown_connections 2021-11-21T22:02:37.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.594+0000 7f09bc01f700 1 -- 172.21.2.116:0/313972125 wait complete. 2021-11-21T22:02:37.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 Processor -- start 2021-11-21T22:02:37.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 -- start start 2021-11-21T22:02:37.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b4062410 0x7f09b40ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b40a6aa0 0x7f09b4156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09b40a86b0 0x7f09b4156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4156f70 con 0x7f09b40a6aa0 2021-11-21T22:02:37.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40aa0c0 con 0x7f09b40a86b0 2021-11-21T22:02:37.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40639d0 con 0x7f09b4062410 2021-11-21T22:02:37.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b40a6aa0 0x7f09b4156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b40a6aa0 0x7f09b4156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53106/0 (socket says 172.21.2.116:53106) 2021-11-21T22:02:37.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 -- 172.21.2.116:0/2345932046 learned_addr learned my addr 172.21.2.116:0/2345932046 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:37.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 -- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b4062410 msgr2=0x7f09b40ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b4062410 0x7f09b40ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 -- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09b40a86b0 msgr2=0x7f09b4156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09b40a86b0 0x7f09b4156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 -- 172.21.2.116:0/2345932046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac009530 con 0x7f09b40a6aa0 2021-11-21T22:02:37.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09b95ba700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b40a6aa0 0x7f09b4156590 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f09a4002f70 tx=0x7f09a4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09a2ffd700 1 -- 172.21.2.116:0/2345932046 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a4013460 con 0x7f09b40a6aa0 2021-11-21T22:02:37.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09a2ffd700 1 -- 172.21.2.116:0/2345932046 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a40135c0 con 0x7f09b40a6aa0 2021-11-21T22:02:37.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b40a9e20 con 0x7f09b40a6aa0 2021-11-21T22:02:37.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09a2ffd700 1 -- 172.21.2.116:0/2345932046 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a4013920 con 0x7f09b40a6aa0 2021-11-21T22:02:37.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b40a9910 con 0x7f09b40a6aa0 2021-11-21T22:02:37.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b40573d0 con 0x7f09b40a6aa0 2021-11-21T22:02:37.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09a2ffd700 1 -- 172.21.2.116:0/2345932046 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f09a400f070 con 0x7f09b40a6aa0 2021-11-21T22:02:37.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09a2ffd700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0994053610 0x7f0994055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.598+0000 7f09a2ffd700 1 -- 172.21.2.116:0/2345932046 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f09a406b120 con 0x7f09b40a6aa0 2021-11-21T22:02:37.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.602+0000 7f09b9dbb700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0994053610 0x7f0994055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.602+0000 7f09b9dbb700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0994053610 0x7f0994055ad0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f09b0002f90 tx=0x7f09b00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.602+0000 7f09a2ffd700 1 -- 172.21.2.116:0/2345932046 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09a4068e90 con 0x7f09b40a6aa0 2021-11-21T22:02:37.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09b405db70 con 0x7f09b40a6aa0 2021-11-21T22:02:37.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09a2ffd700 1 -- 172.21.2.116:0/2345932046 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f09a404aed0 con 0x7f09b40a6aa0 2021-11-21T22:02:37.692 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:37.692 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":141,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:37.337392+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:35.110650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:35.304135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:37.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0994053610 msgr2=0x7f0994055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0994053610 0x7f0994055ad0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f09b0002f90 tx=0x7f09b00069b0).stop 2021-11-21T22:02:37.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b40a6aa0 msgr2=0x7f09b4156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b40a6aa0 0x7f09b4156590 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f09a4002f70 tx=0x7f09a4006df0).stop 2021-11-21T22:02:37.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 shutdown_connections 2021-11-21T22:02:37.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f09b40a86b0 0x7f09b4156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0994053610 0x7f0994055ad0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f09b40a6aa0 0x7f09b4156590 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 --2- 172.21.2.116:0/2345932046 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f09b4062410 0x7f09b40ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 >> 172.21.2.116:0/2345932046 conn(0x7f09b405bd20 msgr2=0x7f09b40b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 shutdown_connections 2021-11-21T22:02:37.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.686+0000 7f09bc01f700 1 -- 172.21.2.116:0/2345932046 wait complete. 2021-11-21T22:02:37.699 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:02:37.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 Processor -- start 2021-11-21T22:02:37.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- start start 2021-11-21T22:02:37.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7540b11c0 0x7fd754147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 0x7fd754147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd754147c10 0x7fd75414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7540b1940 con 0x7fd7540b11c0 2021-11-21T22:02:37.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7540b15a0 con 0x7fd754147c10 2021-11-21T22:02:37.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7540b1b70 con 0x7fd7540a8670 2021-11-21T22:02:37.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7540b11c0 0x7fd754147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd7540b11c0 0x7fd754147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53112/0 (socket says 172.21.2.116:53112) 2021-11-21T22:02:37.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 -- 172.21.2.116:0/2626293371 learned_addr learned my addr 172.21.2.116:0/2626293371 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:37.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 -- 172.21.2.116:0/2626293371 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 msgr2=0x7fd754147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:37.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 --2- 172.21.2.116:0/2626293371 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 0x7fd754147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 -- 172.21.2.116:0/2626293371 >> v2:172.21.2.117:3300/0 conn(0x7fd754147c10 msgr2=0x7fd75414a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:37.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 --2- 172.21.2.116:0/2626293371 >> v2:172.21.2.117:3300/0 conn(0x7fd754147c10 0x7fd75414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 -- 172.21.2.116:0/2626293371 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7540b1e10 con 0x7fd7540b11c0 2021-11-21T22:02:37.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd752ffd700 1 --2- 172.21.2.116:0/2626293371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754147230 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fd744002f70 tx=0x7fd744006eb0).ready entity=mon.0 client_cookie=b501d62843216811 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.759 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd751ffb700 1 -- 172.21.2.116:0/2626293371 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd74400d570 con 0x7fd7540b11c0 2021-11-21T22:02:37.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd751ffb700 1 -- 172.21.2.116:0/2626293371 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd744000e90 con 0x7fd7540b11c0 2021-11-21T22:02:37.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd751ffb700 1 -- 172.21.2.116:0/2626293371 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd74400d900 con 0x7fd7540b11c0 2021-11-21T22:02:37.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- 172.21.2.116:0/2626293371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 msgr2=0x7fd754147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 --2- 172.21.2.116:0/2626293371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754147230 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fd744002f70 tx=0x7fd744006eb0).stop 2021-11-21T22:02:37.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- 172.21.2.116:0/2626293371 shutdown_connections 2021-11-21T22:02:37.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 --2- 172.21.2.116:0/2626293371 >> v2:172.21.2.117:3300/0 conn(0x7fd754147c10 0x7fd75414a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 --2- 172.21.2.116:0/2626293371 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 0x7fd754147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 --2- 172.21.2.116:0/2626293371 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754147230 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- 172.21.2.116:0/2626293371 >> 172.21.2.116:0/2626293371 conn(0x7fd75405bd20 msgr2=0x7fd75405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- 172.21.2.116:0/2626293371 shutdown_connections 2021-11-21T22:02:37.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.750+0000 7fd75927b700 1 -- 172.21.2.116:0/2626293371 wait complete. 2021-11-21T22:02:37.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 Processor -- start 2021-11-21T22:02:37.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 -- start start 2021-11-21T22:02:37.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 0x7fd7541497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd754147c10 0x7fd75415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd754144a60 con 0x7fd7540b11c0 2021-11-21T22:02:37.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd754144230 con 0x7fd754147c10 2021-11-21T22:02:37.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75413f950 con 0x7fd7540a8670 2021-11-21T22:02:37.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53118/0 (socket says 172.21.2.116:53118) 2021-11-21T22:02:37.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 -- 172.21.2.116:0/2517264349 learned_addr learned my addr 172.21.2.116:0/2517264349 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:37.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 -- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 msgr2=0x7fd7541497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 0x7fd7541497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 -- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd754147c10 msgr2=0x7fd75415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd754147c10 0x7fd75415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 -- 172.21.2.116:0/2517264349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74400b1e0 con 0x7fd7540b11c0 2021-11-21T22:02:37.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd7527fc700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754159b30 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fd7541423e0 tx=0x7fd73c006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd74bfff700 1 -- 172.21.2.116:0/2517264349 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c013460 con 0x7fd7540b11c0 2021-11-21T22:02:37.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75413f6b0 con 0x7fd7540b11c0 2021-11-21T22:02:37.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd74bfff700 1 -- 172.21.2.116:0/2517264349 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c0135c0 con 0x7fd7540b11c0 2021-11-21T22:02:37.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd74bfff700 1 -- 172.21.2.116:0/2517264349 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c013920 con 0x7fd7540b11c0 2021-11-21T22:02:37.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75413fde0 con 0x7fd7540b11c0 2021-11-21T22:02:37.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7541598d0 con 0x7fd7540b11c0 2021-11-21T22:02:37.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd74bfff700 1 -- 172.21.2.116:0/2517264349 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd73c01a060 con 0x7fd7540b11c0 2021-11-21T22:02:37.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd74bfff700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7380536f0 0x7fd738055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd74bfff700 1 -- 172.21.2.116:0/2517264349 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd73c06b430 con 0x7fd7540b11c0 2021-11-21T22:02:37.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.754+0000 7fd752ffd700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7380536f0 0x7fd738055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.758+0000 7fd752ffd700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7380536f0 0x7fd738055bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fd744002f70 tx=0x7fd744015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.758+0000 7fd74bfff700 1 -- 172.21.2.116:0/2517264349 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd73c042420 con 0x7fd7540b11c0 2021-11-21T22:02:37.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.866+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd7540b20b0 con 0x7fd7540b11c0 2021-11-21T22:02:37.877 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd74bfff700 1 -- 172.21.2.116:0/2517264349 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd73c0180e0 con 0x7fd7540b11c0 2021-11-21T22:02:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7380536f0 msgr2=0x7fd738055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7380536f0 0x7fd738055bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fd744002f70 tx=0x7fd744015000).stop 2021-11-21T22:02:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 msgr2=0x7fd754159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.879 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754159b30 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fd7541423e0 tx=0x7fd73c006e20).stop 2021-11-21T22:02:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 shutdown_connections 2021-11-21T22:02:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd754147c10 0x7fd75415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7380536f0 0x7fd738055bb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.880 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7540b11c0 0x7fd754159b30 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 --2- 172.21.2.116:0/2517264349 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7540a8670 0x7fd7541497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 >> 172.21.2.116:0/2517264349 conn(0x7fd75405bd20 msgr2=0x7fd754145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 shutdown_connections 2021-11-21T22:02:37.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.874+0000 7fd75927b700 1 -- 172.21.2.116:0/2517264349 wait complete. 2021-11-21T22:02:37.882 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:02:37.885 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:02:37.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909c7e4700 1 Processor -- start 2021-11-21T22:02:37.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909c7e4700 1 -- start start 2021-11-21T22:02:37.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909c7e4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f90940b11c0 0x7f9094147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909c7e4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9094143e60 0x7f90941442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909c7e4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90940b1940 con 0x7f90940a8670 2021-11-21T22:02:37.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909c7e4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90940b15a0 con 0x7f9094143e60 2021-11-21T22:02:37.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909c7e4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90940b1b70 con 0x7f90940b11c0 2021-11-21T22:02:37.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9094143e60 0x7f90941442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9094143e60 0x7f90941442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53124/0 (socket says 172.21.2.116:53124) 2021-11-21T22:02:37.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 -- 172.21.2.116:0/982697220 learned_addr learned my addr 172.21.2.116:0/982697220 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:37.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 -- 172.21.2.116:0/982697220 >> v2:172.21.2.117:3300/0 conn(0x7f90940b11c0 msgr2=0x7f9094147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:37.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 --2- 172.21.2.116:0/982697220 >> v2:172.21.2.117:3300/0 conn(0x7f90940b11c0 0x7f9094147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 -- 172.21.2.116:0/982697220 >> v1:172.21.2.117:6789/0 conn(0x7f90940a8670 legacy=0x7f9094147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:37.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 -- 172.21.2.116:0/982697220 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940b1e10 con 0x7f9094143e60 2021-11-21T22:02:37.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.934+0000 7f909ad81700 1 --2- 172.21.2.116:0/982697220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9094143e60 0x7f90941442d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f9094147180 tx=0x7f908c009290).ready entity=mon.0 client_cookie=34efb493f6b2a861 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909957e700 1 -- 172.21.2.116:0/982697220 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f908c00b830 con 0x7f9094143e60 2021-11-21T22:02:37.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909957e700 1 -- 172.21.2.116:0/982697220 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c00b990 con 0x7f9094143e60 2021-11-21T22:02:37.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909957e700 1 -- 172.21.2.116:0/982697220 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f908c00bb40 con 0x7f9094143e60 2021-11-21T22:02:37.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/982697220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9094143e60 msgr2=0x7f90941442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 --2- 172.21.2.116:0/982697220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9094143e60 0x7f90941442d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f9094147180 tx=0x7f908c009290).stop 2021-11-21T22:02:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/982697220 shutdown_connections 2021-11-21T22:02:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 --2- 172.21.2.116:0/982697220 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9094143e60 0x7f90941442d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 --2- 172.21.2.116:0/982697220 >> v2:172.21.2.117:3300/0 conn(0x7f90940b11c0 0x7f9094147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/982697220 >> 172.21.2.116:0/982697220 conn(0x7f909405bd20 msgr2=0x7f909405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:37.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/982697220 shutdown_connections 2021-11-21T22:02:37.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/982697220 wait complete. 2021-11-21T22:02:37.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 Processor -- start 2021-11-21T22:02:37.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- start start 2021-11-21T22:02:37.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90940a8670 0x7f90941497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90940b11c0 0x7f90941411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9094143e60 0x7f90941416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094065980 con 0x7f90940a8670 2021-11-21T22:02:37.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940638f0 con 0x7f90940b11c0 2021-11-21T22:02:37.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940b5780 con 0x7f9094143e60 2021-11-21T22:02:37.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90940a8670 0x7f90941497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90940a8670 0x7f90941497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53134/0 (socket says 172.21.2.116:53134) 2021-11-21T22:02:37.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 -- 172.21.2.116:0/1584446834 learned_addr learned my addr 172.21.2.116:0/1584446834 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:37.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 -- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9094143e60 msgr2=0x7f90941416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9094143e60 0x7f90941416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 -- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90940b11c0 msgr2=0x7f90941411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:37.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90940b11c0 0x7f90941411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:37.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 -- 172.21.2.116:0/1584446834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c009530 con 0x7f90940a8670 2021-11-21T22:02:37.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909a580700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90940a8670 0x7f90941497a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f90940b5910 tx=0x7f9090006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f90837fe700 1 -- 172.21.2.116:0/1584446834 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9090013460 con 0x7f90940a8670 2021-11-21T22:02:37.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f90837fe700 1 -- 172.21.2.116:0/1584446834 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90900135c0 con 0x7f90940a8670 2021-11-21T22:02:37.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f90837fe700 1 -- 172.21.2.116:0/1584446834 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9090013920 con 0x7f90940a8670 2021-11-21T22:02:37.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094063ff0 con 0x7f90940a8670 2021-11-21T22:02:37.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90940b54e0 con 0x7f90940a8670 2021-11-21T22:02:37.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.938+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9094063cf0 con 0x7f90940a8670 2021-11-21T22:02:37.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.942+0000 7f90837fe700 1 -- 172.21.2.116:0/1584446834 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f909001a060 con 0x7f90940a8670 2021-11-21T22:02:37.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.942+0000 7f90837fe700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f90740536f0 0x7f9074055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:37.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.942+0000 7f90837fe700 1 -- 172.21.2.116:0/1584446834 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f909006b4e0 con 0x7f90940a8670 2021-11-21T22:02:37.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.942+0000 7f9099d7f700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f90740536f0 0x7f9074055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:37.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.942+0000 7f9099d7f700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f90740536f0 0x7f9074055bb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f9084002f90 tx=0x7f90840069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:37.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:37.942+0000 7f90837fe700 1 -- 172.21.2.116:0/1584446834 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9090042530 con 0x7f90940a8670 2021-11-21T22:02:38.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.054+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f90940573d0 con 0x7f90940a8670 2021-11-21T22:02:38.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f90837fe700 1 -- 172.21.2.116:0/1584446834 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9090018020 con 0x7f90940a8670 2021-11-21T22:02:38.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f90740536f0 msgr2=0x7f9074055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f90740536f0 0x7f9074055bb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f9084002f90 tx=0x7f90840069b0).stop 2021-11-21T22:02:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90940a8670 msgr2=0x7f90941497a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90940a8670 0x7f90941497a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f90940b5910 tx=0x7f9090006ec0).stop 2021-11-21T22:02:38.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 shutdown_connections 2021-11-21T22:02:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9094143e60 0x7f90941416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f90940b11c0 0x7f90941411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f90740536f0 0x7f9074055bb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 --2- 172.21.2.116:0/1584446834 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f90940a8670 0x7f90941497a0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.058+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 >> 172.21.2.116:0/1584446834 conn(0x7f909405bd20 msgr2=0x7f909414c860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.062+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 shutdown_connections 2021-11-21T22:02:38.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.062+0000 7f909c7e4700 1 -- 172.21.2.116:0/1584446834 wait complete. 2021-11-21T22:02:38.068 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:02:38.071 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:38.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaaa175700 1 Processor -- start 2021-11-21T22:02:38.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaaa175700 1 -- start start 2021-11-21T22:02:38.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaaa175700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcaa40aed90 0x7fcaa40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaaa175700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40af360 0x7fcaa413c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaaa175700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40b00d0 con 0x7fcaa40b24a0 2021-11-21T22:02:38.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaaa175700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40adba0 con 0x7fcaa40aed90 2021-11-21T22:02:38.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaaa175700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40b2880 con 0x7fcaa40af360 2021-11-21T22:02:38.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa3fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40af360 0x7fcaa413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa2ffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcaa40aed90 0x7fcaa40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa37fe700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fcaa40b24a0 0x7fcaa40ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:35772/0 (socket says 172.21.2.116:35772) 2021-11-21T22:02:38.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa37fe700 1 -- 172.21.2.116:0/928862920 learned_addr learned my addr 172.21.2.116:0/928862920 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:38.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa2ffd700 1 -- 172.21.2.116:0/928862920 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40af360 msgr2=0x7fcaa413c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:38.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa2ffd700 1 --2- 172.21.2.116:0/928862920 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40af360 0x7fcaa413c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa2ffd700 1 -- 172.21.2.116:0/928862920 >> v1:172.21.2.117:6789/0 conn(0x7fcaa40b24a0 legacy=0x7fcaa40ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa2ffd700 1 -- 172.21.2.116:0/928862920 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40b14a0 con 0x7fcaa40aed90 2021-11-21T22:02:38.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.118+0000 7fcaa2ffd700 1 --2- 172.21.2.116:0/928862920 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40aed90 0x7fcaa40b1a40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fca98002f70 tx=0x7fca98006e80).ready entity=mon.1 client_cookie=67d72a951efe40d4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa27fc700 1 -- 172.21.2.116:0/928862920 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9800f730 con 0x7fcaa40aed90 2021-11-21T22:02:38.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa27fc700 1 -- 172.21.2.116:0/928862920 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca98000e90 con 0x7fcaa40aed90 2021-11-21T22:02:38.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa27fc700 1 -- 172.21.2.116:0/928862920 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9800fa80 con 0x7fcaa40aed90 2021-11-21T22:02:38.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/928862920 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40aed90 msgr2=0x7fcaa40b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 --2- 172.21.2.116:0/928862920 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40aed90 0x7fcaa40b1a40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fca98002f70 tx=0x7fca98006e80).stop 2021-11-21T22:02:38.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/928862920 shutdown_connections 2021-11-21T22:02:38.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 --2- 172.21.2.116:0/928862920 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40af360 0x7fcaa413c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 --2- 172.21.2.116:0/928862920 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40aed90 0x7fcaa40b1a40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/928862920 >> 172.21.2.116:0/928862920 conn(0x7fcaa405fd00 msgr2=0x7fcaa40600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/928862920 shutdown_connections 2021-11-21T22:02:38.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/928862920 wait complete. 2021-11-21T22:02:38.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 Processor -- start 2021-11-21T22:02:38.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- start start 2021-11-21T22:02:38.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40aed90 0x7fcaa4065660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40af360 0x7fcaa4156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaa40b24a0 0x7fcaa4156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41106c0 con 0x7fcaa40b24a0 2021-11-21T22:02:38.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa4066290 con 0x7fcaa40af360 2021-11-21T22:02:38.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa410fee0 con 0x7fcaa40aed90 2021-11-21T22:02:38.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaa40b24a0 0x7fcaa4156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaa40b24a0 0x7fcaa4156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53148/0 (socket says 172.21.2.116:53148) 2021-11-21T22:02:38.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 -- 172.21.2.116:0/351312783 learned_addr learned my addr 172.21.2.116:0/351312783 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:38.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa37fe700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40aed90 0x7fcaa4065660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa2ffd700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40af360 0x7fcaa4156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 -- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40aed90 msgr2=0x7fcaa4065660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40aed90 0x7fcaa4065660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 -- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40af360 msgr2=0x7fcaa4156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40af360 0x7fcaa4156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 -- 172.21.2.116:0/351312783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9800c310 con 0x7fcaa40b24a0 2021-11-21T22:02:38.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa3fff700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaa40b24a0 0x7fcaa4156b00 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7fca9c006e00 tx=0x7fca9c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa0ff9700 1 -- 172.21.2.116:0/351312783 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9c00c640 con 0x7fcaa40b24a0 2021-11-21T22:02:38.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa0ff9700 1 -- 172.21.2.116:0/351312783 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c00c7a0 con 0x7fcaa40b24a0 2021-11-21T22:02:38.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa4110180 con 0x7fcaa40b24a0 2021-11-21T22:02:38.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa0ff9700 1 -- 172.21.2.116:0/351312783 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9c00cb00 con 0x7fcaa40b24a0 2021-11-21T22:02:38.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa413acc0 con 0x7fcaa40b24a0 2021-11-21T22:02:38.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa4157190 con 0x7fcaa40b24a0 2021-11-21T22:02:38.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.122+0000 7fcaa0ff9700 1 -- 172.21.2.116:0/351312783 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fca9c00cc60 con 0x7fcaa40b24a0 2021-11-21T22:02:38.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.126+0000 7fcaa0ff9700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca84053610 0x7fca84055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.126+0000 7fcaa0ff9700 1 -- 172.21.2.116:0/351312783 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fca9c06bb40 con 0x7fcaa40b24a0 2021-11-21T22:02:38.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.126+0000 7fcaa37fe700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca84053610 0x7fca84055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.126+0000 7fcaa37fe700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca84053610 0x7fca84055ad0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fca9400d000 tx=0x7fca94000bc0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.126+0000 7fcaa0ff9700 1 -- 172.21.2.116:0/351312783 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca9c042ca0 con 0x7fcaa40b24a0 2021-11-21T22:02:38.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.234+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaa4156320 con 0x7fcaa40b24a0 2021-11-21T22:02:38.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.234+0000 7fcaa0ff9700 1 -- 172.21.2.116:0/351312783 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fca9c0462c0 con 0x7fcaa40b24a0 2021-11-21T22:02:38.241 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:38.241 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:36.333834+0000","modified":"2021-11-21T22:02:37.338971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/1832322648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1832322648},{"type":"v1","addr":"172.21.2.116:6835","nonce":1832322648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-11-21T22:02:38.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca84053610 msgr2=0x7fca84055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca84053610 0x7fca84055ad0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fca9400d000 tx=0x7fca94000bc0).stop 2021-11-21T22:02:38.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaa40b24a0 msgr2=0x7fcaa4156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaa40b24a0 0x7fcaa4156b00 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7fca9c006e00 tx=0x7fca9c0091d0).stop 2021-11-21T22:02:38.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 shutdown_connections 2021-11-21T22:02:38.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca84053610 0x7fca84055ad0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcaa40b24a0 0x7fcaa4156b00 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcaa40af360 0x7fcaa4156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 --2- 172.21.2.116:0/351312783 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcaa40aed90 0x7fcaa4065660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 >> 172.21.2.116:0/351312783 conn(0x7fcaa405fd00 msgr2=0x7fcaa413eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 shutdown_connections 2021-11-21T22:02:38.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.238+0000 7fcaaa175700 1 -- 172.21.2.116:0/351312783 wait complete. 2021-11-21T22:02:38.245 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 65 2021-11-21T22:02:38.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:36.333834+0000', 'modified': '2021-11-21T22:02:37.338971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8109': {'gid': 8109, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.116:6835/1832322648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 1832322648}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1832322648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:02:38.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:02:38.552 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-11-21T22:02:38.615 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8109,"want_state":"up:active","state":"up:active","rank_uptime":2.274318659,"mdsmap_epoch":65,"osdmap_epoch":141,"osdmap_epoch_barrier":141,"uptime":5.5203055770000002} 2021-11-21T22:02:38.616 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8109, "mdsmap_epoch": 65, "osdmap_epoch": 141, "osdmap_epoch_barrier": 141, "rank_uptime": 2.274318659, "state": "up:active", "uptime": 5.520305577, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:02:38.617 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:38.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5aed762700 1 Processor -- start 2021-11-21T22:02:38.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5aed762700 1 -- start start 2021-11-21T22:02:38.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5aed762700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5ae80a86b0 0x7f5ae814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5aed762700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae80a6e80 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5aed762700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae814a7b0 con 0x7f5ae8062410 2021-11-21T22:02:38.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5aed762700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae80a8bc0 con 0x7f5ae80a86b0 2021-11-21T22:02:38.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae67fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5ae80a86b0 0x7f5ae814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae67fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f5ae80a86b0 0x7f5ae814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53154/0 (socket says 172.21.2.116:53154) 2021-11-21T22:02:38.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae67fc700 1 -- 172.21.2.116:0/2401138787 learned_addr learned my addr 172.21.2.116:0/2401138787 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:38.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae6ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5ae80a6aa0 0x7f5ae81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36876/0 (socket says 172.21.2.116:36876) 2021-11-21T22:02:38.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae67fc700 1 -- 172.21.2.116:0/2401138787 >> v1:172.21.2.117:6789/0 conn(0x7f5ae8062410 legacy=0x7f5ae81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:38.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae67fc700 1 -- 172.21.2.116:0/2401138787 >> v1:172.21.2.116:6789/0 conn(0x7f5ae80a6aa0 legacy=0x7f5ae81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae67fc700 1 -- 172.21.2.116:0/2401138787 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae814aa10 con 0x7f5ae80a86b0 2021-11-21T22:02:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae67fc700 1 --2- 172.21.2.116:0/2401138787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a86b0 0x7f5ae814b150 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f5ae8061b50 tx=0x7f5ad8006eb0).ready entity=mon.0 client_cookie=727c416e2d67f179 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae5ffb700 1 -- 172.21.2.116:0/2401138787 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ad800d610 con 0x7f5ae80a86b0 2021-11-21T22:02:38.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae5ffb700 1 -- 172.21.2.116:0/2401138787 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad8000e90 con 0x7f5ae80a86b0 2021-11-21T22:02:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.662+0000 7f5ae5ffb700 1 -- 172.21.2.116:0/2401138787 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ad800d960 con 0x7f5ae80a86b0 2021-11-21T22:02:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/2401138787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a86b0 msgr2=0x7f5ae814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 --2- 172.21.2.116:0/2401138787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a86b0 0x7f5ae814b150 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f5ae8061b50 tx=0x7f5ad8006eb0).stop 2021-11-21T22:02:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/2401138787 shutdown_connections 2021-11-21T22:02:38.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 --2- 172.21.2.116:0/2401138787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a86b0 0x7f5ae814b150 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/2401138787 >> 172.21.2.116:0/2401138787 conn(0x7f5ae805bd20 msgr2=0x7f5ae805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/2401138787 shutdown_connections 2021-11-21T22:02:38.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/2401138787 wait complete. 2021-11-21T22:02:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 Processor -- start 2021-11-21T22:02:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- start start 2021-11-21T22:02:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ae8062410 0x7f5ae8147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a6aa0 0x7f5ae8064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ae80a86b0 0x7f5ae815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81567f0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80aaca0 con 0x7f5ae80a86b0 2021-11-21T22:02:38.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80aa4c0 con 0x7f5ae8062410 2021-11-21T22:02:38.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a6aa0 0x7f5ae8064ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a6aa0 0x7f5ae8064ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53160/0 (socket says 172.21.2.116:53160) 2021-11-21T22:02:38.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 -- 172.21.2.116:0/3214544668 learned_addr learned my addr 172.21.2.116:0/3214544668 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:38.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 -- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ae8062410 msgr2=0x7f5ae8147440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:38.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ae8062410 0x7f5ae8147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 -- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ae80a86b0 msgr2=0x7f5ae815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ae80a86b0 0x7f5ae815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 -- 172.21.2.116:0/3214544668 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad800b1e0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5ae67fc700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a6aa0 0x7f5ae8064ff0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f5ae8061b50 tx=0x7f5ad8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5adffff700 1 -- 172.21.2.116:0/3214544668 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ad8011930 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5adffff700 1 -- 172.21.2.116:0/3214544668 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad8011a90 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5adffff700 1 -- 172.21.2.116:0/3214544668 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ad8011df0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae80aa760 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae80aa1b0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8159dd0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5adffff700 1 -- 172.21.2.116:0/3214544668 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5ad8025030 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5adffff700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5acc0535d0 0x7f5acc055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.666+0000 7f5adffff700 1 -- 172.21.2.116:0/3214544668 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5ad806cd70 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.670+0000 7f5ae6ffd700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5acc0535d0 0x7f5acc055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.670+0000 7f5ae6ffd700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5acc0535d0 0x7f5acc055a90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f5ae8061ab0 tx=0x7f5ad00077c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.670+0000 7f5adffff700 1 -- 172.21.2.116:0/3214544668 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ae8159dd0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae8159dd0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.758 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5adffff700 1 -- 172.21.2.116:0/3214544668 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f5ae8159dd0 con 0x7f5ae80a6aa0 2021-11-21T22:02:38.759 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:38.759 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":141,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:37.337392+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:35.110650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:35.304135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:38.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5acc0535d0 msgr2=0x7f5acc055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5acc0535d0 0x7f5acc055a90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f5ae8061ab0 tx=0x7f5ad00077c0).stop 2021-11-21T22:02:38.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a6aa0 msgr2=0x7f5ae8064ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a6aa0 0x7f5ae8064ff0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f5ae8061b50 tx=0x7f5ad8015000).stop 2021-11-21T22:02:38.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 shutdown_connections 2021-11-21T22:02:38.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5ae80a86b0 0x7f5ae815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5acc0535d0 0x7f5acc055a90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5ae80a6aa0 0x7f5ae8064ff0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 --2- 172.21.2.116:0/3214544668 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5ae8062410 0x7f5ae8147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 >> 172.21.2.116:0/3214544668 conn(0x7f5ae805bd20 msgr2=0x7f5ae81441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 shutdown_connections 2021-11-21T22:02:38.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.754+0000 7f5aed762700 1 -- 172.21.2.116:0/3214544668 wait complete. 2021-11-21T22:02:38.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:02:35.110650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:02:35.304135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:02:38.767 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:02:38.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe41a8e4700 1 Processor -- start 2021-11-21T22:02:38.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe41a8e4700 1 -- start start 2021-11-21T22:02:38.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe41a8e4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 0x7fe4141478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe41a8e4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe4140ac600 0x7fe4140acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe41a8e4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe414147e80 con 0x7fe414144810 2021-11-21T22:02:38.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe41a8e4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4140b1490 con 0x7fe4140ac600 2021-11-21T22:02:38.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe41a8e4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140b1fd0 con 0x7fe4140abce0 2021-11-21T22:02:38.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe418e81700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe4140ac600 0x7fe4140acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe418e81700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe4140ac600 0x7fe4140acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53166/0 (socket says 172.21.2.116:53166) 2021-11-21T22:02:38.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.814+0000 7fe418e81700 1 -- 172.21.2.116:0/2786393439 learned_addr learned my addr 172.21.2.116:0/2786393439 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:38.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 -- 172.21.2.116:0/2786393439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 msgr2=0x7fe4141478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 --2- 172.21.2.116:0/2786393439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 0x7fe4141478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 -- 172.21.2.116:0/2786393439 >> v1:172.21.2.117:6789/0 conn(0x7fe414144810 legacy=0x7fe4140b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 -- 172.21.2.116:0/2786393439 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140af5a0 con 0x7fe4140ac600 2021-11-21T22:02:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 --2- 172.21.2.116:0/2786393439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4140ac600 0x7fe4140acab0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7fe4140b2a30 tx=0x7fe40c009290).ready entity=mon.0 client_cookie=60b42f7c2863d948 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe412ffd700 1 -- 172.21.2.116:0/2786393439 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40c00b860 con 0x7fe4140ac600 2021-11-21T22:02:38.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe412ffd700 1 -- 172.21.2.116:0/2786393439 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c00b9c0 con 0x7fe4140ac600 2021-11-21T22:02:38.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe412ffd700 1 -- 172.21.2.116:0/2786393439 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40c00bb70 con 0x7fe4140ac600 2021-11-21T22:02:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/2786393439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4140ac600 msgr2=0x7fe4140acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/2786393439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4140ac600 0x7fe4140acab0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7fe4140b2a30 tx=0x7fe40c009290).stop 2021-11-21T22:02:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/2786393439 shutdown_connections 2021-11-21T22:02:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/2786393439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe4140ac600 0x7fe4140acab0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/2786393439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 0x7fe4141478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/2786393439 >> 172.21.2.116:0/2786393439 conn(0x7fe41405bd20 msgr2=0x7fe41405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/2786393439 shutdown_connections 2021-11-21T22:02:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/2786393439 wait complete. 2021-11-21T22:02:38.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 Processor -- start 2021-11-21T22:02:38.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- start start 2021-11-21T22:02:38.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 0x7fe414143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4140ac600 0x7fe414145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe414144810 0x7fe41415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414157250 con 0x7fe414144810 2021-11-21T22:02:38.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141574f0 con 0x7fe4140ac600 2021-11-21T22:02:38.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe414159c40 con 0x7fe4140abce0 2021-11-21T22:02:38.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe414144810 0x7fe41415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe414144810 0x7fe41415a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53176/0 (socket says 172.21.2.116:53176) 2021-11-21T22:02:38.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 -- 172.21.2.116:0/4136189061 learned_addr learned my addr 172.21.2.116:0/4136189061 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 -- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 msgr2=0x7fe414143510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:38.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 0x7fe414143510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 -- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4140ac600 msgr2=0x7fe414145e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:38.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4140ac600 0x7fe414145e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 -- 172.21.2.116:0/4136189061 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c009530 con 0x7fe414144810 2021-11-21T22:02:38.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe418e81700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe414144810 0x7fe41415a380 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fe4140b2a30 tx=0x7fe40c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe4117fa700 1 -- 172.21.2.116:0/4136189061 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40c021620 con 0x7fe414144810 2021-11-21T22:02:38.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe4117fa700 1 -- 172.21.2.116:0/4136189061 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c021780 con 0x7fe414144810 2021-11-21T22:02:38.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe414159ee0 con 0x7fe414144810 2021-11-21T22:02:38.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe414143c20 con 0x7fe414144810 2021-11-21T22:02:38.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe4117fa700 1 -- 172.21.2.116:0/4136189061 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40c021ae0 con 0x7fe414144810 2021-11-21T22:02:38.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41415a070 con 0x7fe414144810 2021-11-21T22:02:38.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.818+0000 7fe4117fa700 1 -- 172.21.2.116:0/4136189061 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe40c00f070 con 0x7fe414144810 2021-11-21T22:02:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.822+0000 7fe4117fa700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3f8053580 0x7fe3f8055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.822+0000 7fe4117fa700 1 -- 172.21.2.116:0/4136189061 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe40c06cf90 con 0x7fe414144810 2021-11-21T22:02:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.822+0000 7fe413fff700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3f8053580 0x7fe3f8055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.822+0000 7fe413fff700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3f8053580 0x7fe3f8055a40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fe404000ed0 tx=0x7fe4040068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.822+0000 7fe4117fa700 1 -- 172.21.2.116:0/4136189061 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe40c044170 con 0x7fe414144810 2021-11-21T22:02:38.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe4140573d0 con 0x7fe414144810 2021-11-21T22:02:38.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe4117fa700 1 -- 172.21.2.116:0/4136189061 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe40c015030 con 0x7fe414144810 2021-11-21T22:02:38.911 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:02:38.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3f8053580 msgr2=0x7fe3f8055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3f8053580 0x7fe3f8055a40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fe404000ed0 tx=0x7fe4040068b0).stop 2021-11-21T22:02:38.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe414144810 msgr2=0x7fe41415a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe414144810 0x7fe41415a380 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fe4140b2a30 tx=0x7fe40c016000).stop 2021-11-21T22:02:38.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 shutdown_connections 2021-11-21T22:02:38.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe3f8053580 0x7fe3f8055a40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe414144810 0x7fe41415a380 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe4140ac600 0x7fe414145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 --2- 172.21.2.116:0/4136189061 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe4140abce0 0x7fe414143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 >> 172.21.2.116:0/4136189061 conn(0x7fe41405bd20 msgr2=0x7fe414149460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 shutdown_connections 2021-11-21T22:02:38.915 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.906+0000 7fe41a8e4700 1 -- 172.21.2.116:0/4136189061 wait complete. 2021-11-21T22:02:38.918 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:02:38.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 Processor -- start 2021-11-21T22:02:38.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- start start 2021-11-21T22:02:38.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 0x7f6f141478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6f140abce0 0x7f6f140b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6f140b1db0 0x7f6f140ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f14147e80 con 0x7f6f140abce0 2021-11-21T22:02:38.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f140af5a0 con 0x7f6f140b1db0 2021-11-21T22:02:38.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140b1490 con 0x7f6f14144810 2021-11-21T22:02:38.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6f140abce0 0x7f6f140b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6f140abce0 0x7f6f140b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53180/0 (socket says 172.21.2.116:53180) 2021-11-21T22:02:38.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 -- 172.21.2.116:0/2946549128 learned_addr learned my addr 172.21.2.116:0/2946549128 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:38.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 -- 172.21.2.116:0/2946549128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 msgr2=0x7f6f141478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:38.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 --2- 172.21.2.116:0/2946549128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 0x7f6f141478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 -- 172.21.2.116:0/2946549128 >> v2:172.21.2.117:3300/0 conn(0x7f6f140b1db0 msgr2=0x7f6f140ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:38.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 --2- 172.21.2.116:0/2946549128 >> v2:172.21.2.117:3300/0 conn(0x7f6f140b1db0 0x7f6f140ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 -- 172.21.2.116:0/2946549128 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140af060 con 0x7f6f140abce0 2021-11-21T22:02:38.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f188f0700 1 --2- 172.21.2.116:0/2946549128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f140abce0 0x7f6f140b18c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f6f10002f70 tx=0x7f6f10006eb0).ready entity=mon.0 client_cookie=4b3fa46a5189186c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f07fff700 1 -- 172.21.2.116:0/2946549128 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f1000d570 con 0x7f6f140abce0 2021-11-21T22:02:38.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f07fff700 1 -- 172.21.2.116:0/2946549128 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f10000e90 con 0x7f6f140abce0 2021-11-21T22:02:38.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f07fff700 1 -- 172.21.2.116:0/2946549128 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f1000d900 con 0x7f6f140abce0 2021-11-21T22:02:38.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- 172.21.2.116:0/2946549128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f140abce0 msgr2=0x7f6f140b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 --2- 172.21.2.116:0/2946549128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f140abce0 0x7f6f140b18c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f6f10002f70 tx=0x7f6f10006eb0).stop 2021-11-21T22:02:38.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- 172.21.2.116:0/2946549128 shutdown_connections 2021-11-21T22:02:38.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 --2- 172.21.2.116:0/2946549128 >> v2:172.21.2.117:3300/0 conn(0x7f6f140b1db0 0x7f6f140ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 --2- 172.21.2.116:0/2946549128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f140abce0 0x7f6f140b18c0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:38.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 --2- 172.21.2.116:0/2946549128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 0x7f6f141478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- 172.21.2.116:0/2946549128 >> 172.21.2.116:0/2946549128 conn(0x7f6f1405bd20 msgr2=0x7f6f1405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:38.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- 172.21.2.116:0/2946549128 shutdown_connections 2021-11-21T22:02:38.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.966+0000 7f6f1b355700 1 -- 172.21.2.116:0/2946549128 wait complete. 2021-11-21T22:02:38.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 Processor -- start 2021-11-21T22:02:38.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 -- start start 2021-11-21T22:02:38.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f140abce0 0x7f6f140b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 0x7f6f1413c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f14144d20 0x7f6f1413c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14145db0 con 0x7f6f14144d20 2021-11-21T22:02:38.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14146280 con 0x7f6f140abce0 2021-11-21T22:02:38.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14145230 con 0x7f6f14144810 2021-11-21T22:02:38.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f14144d20 0x7f6f1413c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f14144d20 0x7f6f1413c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53188/0 (socket says 172.21.2.116:53188) 2021-11-21T22:02:38.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 -- 172.21.2.116:0/786833678 learned_addr learned my addr 172.21.2.116:0/786833678 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:38.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 -- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 msgr2=0x7f6f1413c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 0x7f6f1413c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 -- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f140abce0 msgr2=0x7f6f140b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:38.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f140abce0 0x7f6f140b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:38.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 -- 172.21.2.116:0/786833678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1000b1e0 con 0x7f6f14144d20 2021-11-21T22:02:38.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f198f2700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f14144d20 0x7f6f1413c770 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f6f0c0032f0 tx=0x7f6f0c009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f067fc700 1 -- 172.21.2.116:0/786833678 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f0c00d6f0 con 0x7f6f14144d20 2021-11-21T22:02:38.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f067fc700 1 -- 172.21.2.116:0/786833678 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c00d850 con 0x7f6f14144d20 2021-11-21T22:02:38.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f14145fe0 con 0x7f6f14144d20 2021-11-21T22:02:38.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f067fc700 1 -- 172.21.2.116:0/786833678 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f0c00dbf0 con 0x7f6f14144d20 2021-11-21T22:02:38.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1413cc60 con 0x7f6f14144d20 2021-11-21T22:02:38.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f140af300 con 0x7f6f14144d20 2021-11-21T22:02:38.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f067fc700 1 -- 172.21.2.116:0/786833678 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6f0c018030 con 0x7f6f14144d20 2021-11-21T22:02:38.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f067fc700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ef4053610 0x7f6ef4055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:38.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.970+0000 7f6f067fc700 1 -- 172.21.2.116:0/786833678 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6f0c06b700 con 0x7f6f14144d20 2021-11-21T22:02:38.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.974+0000 7f6f190f1700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ef4053610 0x7f6ef4055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:38.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.974+0000 7f6f190f1700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ef4053610 0x7f6ef4055ad0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f6f08002f70 tx=0x7f6f08006960).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:38.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:38.974+0000 7f6f067fc700 1 -- 172.21.2.116:0/786833678 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6f0c041e90 con 0x7f6f14144d20 2021-11-21T22:02:39.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6f140ad630 con 0x7f6f14144d20 2021-11-21T22:02:39.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f067fc700 1 -- 172.21.2.116:0/786833678 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6f0c042870 con 0x7f6f14144d20 2021-11-21T22:02:39.063 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ef4053610 msgr2=0x7f6ef4055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ef4053610 0x7f6ef4055ad0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f6f08002f70 tx=0x7f6f08006960).stop 2021-11-21T22:02:39.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f14144d20 msgr2=0x7f6f1413c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f14144d20 0x7f6f1413c770 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f6f0c0032f0 tx=0x7f6f0c009220).stop 2021-11-21T22:02:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 shutdown_connections 2021-11-21T22:02:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6ef4053610 0x7f6ef4055ad0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f14144d20 0x7f6f1413c770 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f14144810 0x7f6f1413c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 --2- 172.21.2.116:0/786833678 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f140abce0 0x7f6f140b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 >> 172.21.2.116:0/786833678 conn(0x7f6f1405bd20 msgr2=0x7f6f14141690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:39.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 shutdown_connections 2021-11-21T22:02:39.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.058+0000 7f6f1b355700 1 -- 172.21.2.116:0/786833678 wait complete. 2021-11-21T22:02:39.069 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:02:39.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff28e8d3700 1 Processor -- start 2021-11-21T22:02:39.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff28e8d3700 1 -- start start 2021-11-21T22:02:39.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff28e8d3700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2881426e0 0x7ff288142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff28e8d3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff288062e60 0x7ff2880af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff28e8d3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2880b0f60 con 0x7ff2880ad470 2021-11-21T22:02:39.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff28e8d3700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880ae030 con 0x7ff2881426e0 2021-11-21T22:02:39.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff28e8d3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff288063470 con 0x7ff288062e60 2021-11-21T22:02:39.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2881426e0 0x7ff288142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff2881426e0 0x7ff288142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53192/0 (socket says 172.21.2.116:53192) 2021-11-21T22:02:39.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 -- 172.21.2.116:0/2953492740 learned_addr learned my addr 172.21.2.116:0/2953492740 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:39.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 -- 172.21.2.116:0/2953492740 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff288062e60 msgr2=0x7ff2880af3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:39.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 --2- 172.21.2.116:0/2953492740 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff288062e60 0x7ff2880af3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 -- 172.21.2.116:0/2953492740 >> v1:172.21.2.117:6789/0 conn(0x7ff2880ad470 legacy=0x7ff2880ad920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:39.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 -- 172.21.2.116:0/2953492740 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288143050 con 0x7ff2881426e0 2021-11-21T22:02:39.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff287fff700 1 --2- 172.21.2.116:0/2953492740 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2881426e0 0x7ff288142ac0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7ff2881432f0 tx=0x7ff278006e40).ready entity=mon.0 client_cookie=6aa9ea2ac7f32570 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff286ffd700 1 -- 172.21.2.116:0/2953492740 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff27800bcb0 con 0x7ff2881426e0 2021-11-21T22:02:39.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff286ffd700 1 -- 172.21.2.116:0/2953492740 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278000e90 con 0x7ff2881426e0 2021-11-21T22:02:39.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.118+0000 7ff286ffd700 1 -- 172.21.2.116:0/2953492740 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2780153f0 con 0x7ff2881426e0 2021-11-21T22:02:39.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/2953492740 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2881426e0 msgr2=0x7ff288142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/2953492740 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2881426e0 0x7ff288142ac0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7ff2881432f0 tx=0x7ff278006e40).stop 2021-11-21T22:02:39.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/2953492740 shutdown_connections 2021-11-21T22:02:39.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/2953492740 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff288062e60 0x7ff2880af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/2953492740 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2881426e0 0x7ff288142ac0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/2953492740 >> 172.21.2.116:0/2953492740 conn(0x7ff2880b5730 msgr2=0x7ff2880b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:39.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/2953492740 shutdown_connections 2021-11-21T22:02:39.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/2953492740 wait complete. 2021-11-21T22:02:39.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 Processor -- start 2021-11-21T22:02:39.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- start start 2021-11-21T22:02:39.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2880ad470 0x7ff28813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2881426e0 0x7ff28805cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28813e9f0 0x7ff288156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28813f3c0 con 0x7ff2880ad470 2021-11-21T22:02:39.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28813e260 con 0x7ff2881426e0 2021-11-21T22:02:39.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28813e500 con 0x7ff28813e9f0 2021-11-21T22:02:39.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2880ad470 0x7ff28813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2880ad470 0x7ff28813fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53200/0 (socket says 172.21.2.116:53200) 2021-11-21T22:02:39.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 -- 172.21.2.116:0/684416674 learned_addr learned my addr 172.21.2.116:0/684416674 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:39.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 -- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28813e9f0 msgr2=0x7ff288156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28813e9f0 0x7ff288156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 -- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2881426e0 msgr2=0x7ff28805cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2881426e0 0x7ff28805cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 -- 172.21.2.116:0/684416674 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27800d4d0 con 0x7ff2880ad470 2021-11-21T22:02:39.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff287fff700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2880ad470 0x7ff28813fab0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7ff2881432f0 tx=0x7ff278018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff2857fa700 1 -- 172.21.2.116:0/684416674 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff278009030 con 0x7ff2880ad470 2021-11-21T22:02:39.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff2857fa700 1 -- 172.21.2.116:0/684416674 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800a040 con 0x7ff2880ad470 2021-11-21T22:02:39.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff2857fa700 1 -- 172.21.2.116:0/684416674 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff278015e20 con 0x7ff2880ad470 2021-11-21T22:02:39.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28813f610 con 0x7ff2880ad470 2021-11-21T22:02:39.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28813f050 con 0x7ff2880ad470 2021-11-21T22:02:39.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2880b1e00 con 0x7ff2880ad470 2021-11-21T22:02:39.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff2857fa700 1 -- 172.21.2.116:0/684416674 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff278024030 con 0x7ff2880ad470 2021-11-21T22:02:39.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff2857fa700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff268053610 0x7ff268055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.122+0000 7ff2857fa700 1 -- 172.21.2.116:0/684416674 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff27806c840 con 0x7ff2880ad470 2021-11-21T22:02:39.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.126+0000 7ff2877fe700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff268053610 0x7ff268055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.126+0000 7ff2877fe700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff268053610 0x7ff268055ad0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7ff27c002fb0 tx=0x7ff27c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.126+0000 7ff2857fa700 1 -- 172.21.2.116:0/684416674 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff278043890 con 0x7ff2880ad470 2021-11-21T22:02:39.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.206+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2880573d0 con 0x7ff2880ad470 2021-11-21T22:02:39.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff2857fa700 1 -- 172.21.2.116:0/684416674 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff278046eb0 con 0x7ff2880ad470 2021-11-21T22:02:39.214 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:39.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff268053610 msgr2=0x7ff268055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff268053610 0x7ff268055ad0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7ff27c002fb0 tx=0x7ff27c0069b0).stop 2021-11-21T22:02:39.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2880ad470 msgr2=0x7ff28813fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2880ad470 0x7ff28813fab0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7ff2881432f0 tx=0x7ff278018000).stop 2021-11-21T22:02:39.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 shutdown_connections 2021-11-21T22:02:39.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff28813e9f0 0x7ff288156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff2881426e0 0x7ff28805cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff268053610 0x7ff268055ad0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 --2- 172.21.2.116:0/684416674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff2880ad470 0x7ff28813fab0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 >> 172.21.2.116:0/684416674 conn(0x7ff2880b5730 msgr2=0x7ff2880b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:39.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 shutdown_connections 2021-11-21T22:02:39.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.210+0000 7ff28e8d3700 1 -- 172.21.2.116:0/684416674 wait complete. 2021-11-21T22:02:39.221 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:02:39.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 Processor -- start 2021-11-21T22:02:39.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- start start 2021-11-21T22:02:39.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c1426e0 0x7f3d5c142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3d5c069e70 0x7f3d5c0ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c0afae0 con 0x7f3d5c062e60 2021-11-21T22:02:39.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0b0f60 con 0x7f3d5c069e70 2021-11-21T22:02:39.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c063470 con 0x7f3d5c1426e0 2021-11-21T22:02:39.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3d5c069e70 0x7f3d5c0ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f3d5c069e70 0x7f3d5c0ad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53208/0 (socket says 172.21.2.116:53208) 2021-11-21T22:02:39.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 -- 172.21.2.116:0/3743270484 learned_addr learned my addr 172.21.2.116:0/3743270484 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:39.276 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 --1- 172.21.2.116:0/3743270484 >> v1:172.21.2.116:6789/0 conn(0x7f3d5c062e60 0x7f3d5c0af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36932/0 (socket says 172.21.2.116:36932) 2021-11-21T22:02:39.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 -- 172.21.2.116:0/3743270484 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c1426e0 msgr2=0x7f3d5c142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:39.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 --2- 172.21.2.116:0/3743270484 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c1426e0 0x7f3d5c142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 -- 172.21.2.116:0/3743270484 >> v1:172.21.2.116:6789/0 conn(0x7f3d5c062e60 legacy=0x7f3d5c0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 -- 172.21.2.116:0/3743270484 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c143050 con 0x7f3d5c069e70 2021-11-21T22:02:39.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6295c700 1 --2- 172.21.2.116:0/3743270484 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c069e70 0x7f3d5c0ad470 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f3d540032f0 tx=0x7f3d54009290).ready entity=mon.0 client_cookie=1913929243a9b139 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d61159700 1 -- 172.21.2.116:0/3743270484 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d5400b8a0 con 0x7f3d5c069e70 2021-11-21T22:02:39.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d61159700 1 -- 172.21.2.116:0/3743270484 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5400ba00 con 0x7f3d5c069e70 2021-11-21T22:02:39.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d61159700 1 -- 172.21.2.116:0/3743270484 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d5400bbb0 con 0x7f3d5c069e70 2021-11-21T22:02:39.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- 172.21.2.116:0/3743270484 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c069e70 msgr2=0x7f3d5c0ad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- 172.21.2.116:0/3743270484 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c069e70 0x7f3d5c0ad470 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f3d540032f0 tx=0x7f3d54009290).stop 2021-11-21T22:02:39.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- 172.21.2.116:0/3743270484 shutdown_connections 2021-11-21T22:02:39.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- 172.21.2.116:0/3743270484 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c069e70 0x7f3d5c0ad470 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- 172.21.2.116:0/3743270484 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c1426e0 0x7f3d5c142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- 172.21.2.116:0/3743270484 >> 172.21.2.116:0/3743270484 conn(0x7f3d5c0b5730 msgr2=0x7f3d5c0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:39.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- 172.21.2.116:0/3743270484 shutdown_connections 2021-11-21T22:02:39.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- 172.21.2.116:0/3743270484 wait complete. 2021-11-21T22:02:39.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 Processor -- start 2021-11-21T22:02:39.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- start start 2021-11-21T22:02:39.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c069e70 0x7f3d5c13fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c1426e0 0x7f3d5c067190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c11d190 0x7f3d5c11bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c13ff70 con 0x7f3d5c1426e0 2021-11-21T22:02:39.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c13e080 con 0x7f3d5c069e70 2021-11-21T22:02:39.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d643bf700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c13ab80 con 0x7f3d5c11d190 2021-11-21T22:02:39.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c1426e0 0x7f3d5c067190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c1426e0 0x7f3d5c067190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53212/0 (socket says 172.21.2.116:53212) 2021-11-21T22:02:39.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 -- 172.21.2.116:0/1748737658 learned_addr learned my addr 172.21.2.116:0/1748737658 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:39.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 -- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c11d190 msgr2=0x7f3d5c11bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c11d190 0x7f3d5c11bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 -- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c069e70 msgr2=0x7f3d5c13fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c069e70 0x7f3d5c13fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.270+0000 7f3d6195a700 1 -- 172.21.2.116:0/1748737658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54009530 con 0x7f3d5c1426e0 2021-11-21T22:02:39.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d6195a700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c1426e0 0x7f3d5c067190 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f3d5c13e630 tx=0x7f3d58007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d4b7fe700 1 -- 172.21.2.116:0/1748737658 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d5800ce20 con 0x7f3d5c1426e0 2021-11-21T22:02:39.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d4b7fe700 1 -- 172.21.2.116:0/1748737658 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d580072e0 con 0x7f3d5c1426e0 2021-11-21T22:02:39.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c13b8c0 con 0x7f3d5c1426e0 2021-11-21T22:02:39.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c13e2b0 con 0x7f3d5c1426e0 2021-11-21T22:02:39.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d4b7fe700 1 -- 172.21.2.116:0/1748737658 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d58007450 con 0x7f3d5c1426e0 2021-11-21T22:02:39.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c0b1e00 con 0x7f3d5c1426e0 2021-11-21T22:02:39.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d4b7fe700 1 -- 172.21.2.116:0/1748737658 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3d58010070 con 0x7f3d5c1426e0 2021-11-21T22:02:39.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d4b7fe700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d3c053720 0x7f3d3c055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d4b7fe700 1 -- 172.21.2.116:0/1748737658 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3d5806b210 con 0x7f3d5c1426e0 2021-11-21T22:02:39.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d6215b700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d3c053720 0x7f3d3c055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d6215b700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d3c053720 0x7f3d3c055be0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f3d4c002ef0 tx=0x7f3d4c006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.290 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.274+0000 7f3d4b7fe700 1 -- 172.21.2.116:0/1748737658 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3d58042200 con 0x7f3d5c1426e0 2021-11-21T22:02:39.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.358+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d5c0573d0 con 0x7f3d5c1426e0 2021-11-21T22:02:39.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.358+0000 7f3d4b7fe700 1 -- 172.21.2.116:0/1748737658 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3d58045820 con 0x7f3d5c1426e0 2021-11-21T22:02:39.364 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:39.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d3c053720 msgr2=0x7f3d3c055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d3c053720 0x7f3d3c055be0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f3d4c002ef0 tx=0x7f3d4c006910).stop 2021-11-21T22:02:39.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c1426e0 msgr2=0x7f3d5c067190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c1426e0 0x7f3d5c067190 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f3d5c13e630 tx=0x7f3d58007c40).stop 2021-11-21T22:02:39.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 shutdown_connections 2021-11-21T22:02:39.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f3d5c11d190 0x7f3d5c11bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3d3c053720 0x7f3d3c055be0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f3d5c1426e0 0x7f3d5c067190 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 --2- 172.21.2.116:0/1748737658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f3d5c069e70 0x7f3d5c13fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 >> 172.21.2.116:0/1748737658 conn(0x7f3d5c0b5730 msgr2=0x7f3d5c0af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:39.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 shutdown_connections 2021-11-21T22:02:39.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.362+0000 7f3d643bf700 1 -- 172.21.2.116:0/1748737658 wait complete. 2021-11-21T22:02:39.371 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:02:39.372 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:39.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 Processor -- start 2021-11-21T22:02:39.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 -- start start 2021-11-21T22:02:39.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7d540b24a0 0x7f7d540b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7d540af360 0x7f7d540ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540aeb70 0x7f7d5413c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d540adba0 con 0x7f7d540af360 2021-11-21T22:02:39.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d540af7a0 con 0x7f7d540b24a0 2021-11-21T22:02:39.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d599fb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b00d0 con 0x7f7d540aeb70 2021-11-21T22:02:39.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7d540af360 0x7f7d540ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7d540af360 0x7f7d540ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53224/0 (socket says 172.21.2.116:53224) 2021-11-21T22:02:39.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 -- 172.21.2.116:0/868166276 learned_addr learned my addr 172.21.2.116:0/868166276 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:39.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 -- 172.21.2.116:0/868166276 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540aeb70 msgr2=0x7f7d5413c210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:39.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 --2- 172.21.2.116:0/868166276 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540aeb70 0x7f7d5413c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 -- 172.21.2.116:0/868166276 >> v2:172.21.2.117:3300/0 conn(0x7f7d540b24a0 msgr2=0x7f7d540b1a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:39.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 --2- 172.21.2.116:0/868166276 >> v2:172.21.2.117:3300/0 conn(0x7f7d540b24a0 0x7f7d540b1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 -- 172.21.2.116:0/868166276 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540b2880 con 0x7f7d540af360 2021-11-21T22:02:39.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d527fc700 1 --2- 172.21.2.116:0/868166276 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540af360 0x7f7d540ae680 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f7d44002f70 tx=0x7f7d44006f40).ready entity=mon.0 client_cookie=f7192e084fd9778e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d51ffb700 1 -- 172.21.2.116:0/868166276 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d4400c900 con 0x7f7d540af360 2021-11-21T22:02:39.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d51ffb700 1 -- 172.21.2.116:0/868166276 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d44000e90 con 0x7f7d540af360 2021-11-21T22:02:39.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.418+0000 7f7d51ffb700 1 -- 172.21.2.116:0/868166276 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d4400cc90 con 0x7f7d540af360 2021-11-21T22:02:39.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/868166276 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540af360 msgr2=0x7f7d540ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 --2- 172.21.2.116:0/868166276 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540af360 0x7f7d540ae680 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f7d44002f70 tx=0x7f7d44006f40).stop 2021-11-21T22:02:39.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/868166276 shutdown_connections 2021-11-21T22:02:39.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 --2- 172.21.2.116:0/868166276 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540aeb70 0x7f7d5413c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 --2- 172.21.2.116:0/868166276 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540af360 0x7f7d540ae680 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 --2- 172.21.2.116:0/868166276 >> v2:172.21.2.117:3300/0 conn(0x7f7d540b24a0 0x7f7d540b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/868166276 >> 172.21.2.116:0/868166276 conn(0x7f7d5405fd00 msgr2=0x7f7d540600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:39.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/868166276 shutdown_connections 2021-11-21T22:02:39.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/868166276 wait complete. 2021-11-21T22:02:39.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 Processor -- start 2021-11-21T22:02:39.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- start start 2021-11-21T22:02:39.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540aeb70 0x7f7d54064040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540af360 0x7f7d54133230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7d540b24a0 0x7f7d54133720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d541531a0 con 0x7f7d540aeb70 2021-11-21T22:02:39.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54152520 con 0x7f7d540b24a0 2021-11-21T22:02:39.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d541527c0 con 0x7f7d540af360 2021-11-21T22:02:39.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540aeb70 0x7f7d54064040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540aeb70 0x7f7d54064040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53228/0 (socket says 172.21.2.116:53228) 2021-11-21T22:02:39.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 -- 172.21.2.116:0/3757151864 learned_addr learned my addr 172.21.2.116:0/3757151864 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:39.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 -- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540af360 msgr2=0x7f7d54133230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540af360 0x7f7d54133230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 -- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7d540b24a0 msgr2=0x7f7d54133720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7d540b24a0 0x7f7d54133720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 -- 172.21.2.116:0/3757151864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4400a360 con 0x7f7d540aeb70 2021-11-21T22:02:39.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d52ffd700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540aeb70 0x7f7d54064040 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f7d48000ed0 tx=0x7f7d48006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d43fff700 1 -- 172.21.2.116:0/3757151864 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d48012410 con 0x7f7d540aeb70 2021-11-21T22:02:39.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d43fff700 1 -- 172.21.2.116:0/3757151864 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d48012570 con 0x7f7d540aeb70 2021-11-21T22:02:39.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d54152a60 con 0x7f7d540aeb70 2021-11-21T22:02:39.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d43fff700 1 -- 172.21.2.116:0/3757151864 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d480128d0 con 0x7f7d540aeb70 2021-11-21T22:02:39.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d540645b0 con 0x7f7d540aeb70 2021-11-21T22:02:39.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d540573d0 con 0x7f7d540aeb70 2021-11-21T22:02:39.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d43fff700 1 -- 172.21.2.116:0/3757151864 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7d4800f030 con 0x7f7d540aeb70 2021-11-21T22:02:39.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d43fff700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7d300536f0 0x7f7d30055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:39.443 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d43fff700 1 -- 172.21.2.116:0/3757151864 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7d4806b550 con 0x7f7d540aeb70 2021-11-21T22:02:39.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.422+0000 7f7d527fc700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7d300536f0 0x7f7d30055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:39.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.426+0000 7f7d527fc700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7d300536f0 0x7f7d30055bb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f7d44002f70 tx=0x7f7d5413eff0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:39.444 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.426+0000 7f7d43fff700 1 -- 172.21.2.116:0/3757151864 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d480426b0 con 0x7f7d540aeb70 2021-11-21T22:02:39.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5405f2d0 con 0x7f7d540aeb70 2021-11-21T22:02:39.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d43fff700 1 -- 172.21.2.116:0/3757151864 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f7d48045cd0 con 0x7f7d540aeb70 2021-11-21T22:02:39.539 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:39.540 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:36.333834+0000","modified":"2021-11-21T22:02:37.338971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/1832322648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1832322648},{"type":"v1","addr":"172.21.2.116:6835","nonce":1832322648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-11-21T22:02:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7d300536f0 msgr2=0x7f7d30055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7d300536f0 0x7f7d30055bb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f7d44002f70 tx=0x7f7d5413eff0).stop 2021-11-21T22:02:39.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540aeb70 msgr2=0x7f7d54064040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:39.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540aeb70 0x7f7d54064040 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f7d48000ed0 tx=0x7f7d48006d70).stop 2021-11-21T22:02:39.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 shutdown_connections 2021-11-21T22:02:39.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7d540b24a0 0x7f7d54133720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7d540af360 0x7f7d54133230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7d300536f0 0x7f7d30055bb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 --2- 172.21.2.116:0/3757151864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7d540aeb70 0x7f7d54064040 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:39.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 >> 172.21.2.116:0/3757151864 conn(0x7f7d5405fd00 msgr2=0x7f7d5413c730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:39.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 shutdown_connections 2021-11-21T22:02:39.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:39.534+0000 7f7d599fb700 1 -- 172.21.2.116:0/3757151864 wait complete. 2021-11-21T22:02:39.544 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 65 2021-11-21T22:02:39.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:36.333834+0000', 'modified': '2021-11-21T22:02:37.338971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8109': {'gid': 8109, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.116:6835/1832322648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 1832322648}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1832322648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:02:39.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:02:39.850 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-11-21T22:02:39.915 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8109,"want_state":"up:active","state":"up:active","rank_uptime":3.5729211630000002,"mdsmap_epoch":65,"osdmap_epoch":141,"osdmap_epoch_barrier":141,"uptime":6.8189077429999996} 2021-11-21T22:02:39.915 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8109, "mdsmap_epoch": 65, "osdmap_epoch": 141, "osdmap_epoch_barrier": 141, "rank_uptime": 3.572921163, "state": "up:active", "uptime": 6.818907743, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:02:39.916 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:02:39.916 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:02:39.917 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:02:39.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:39.918 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:39.928 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:02:39.980 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:39.981 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:39.987 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:39.987 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:39.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:39.989 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:02:39.993 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:02:39.993 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:40.040 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:02:40.049 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:40.050 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:40.055 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:40.056 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:40.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:40.057 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:02:40.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:02:40.077+0000 7fba15055080 -1 init, newargv = 0x5639dee93bc0 newargc=9 2021-11-21T22:02:40.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15164]: starting ceph client 2021-11-21T22:02:40.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15164]: starting fuse 2021-11-21T22:02:41.183 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:02:41.196 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:41.196 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:41.245 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:41.245 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:41.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:41.247 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:02:41.253 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:02:41.253 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:02:41.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:41.254 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:02:41.254 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:02:41.255 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:02:41.255 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:02:41.256 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:02:41.256 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:41.256 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:02:41.257 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:02:41.257 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:02:41.258 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:41.258 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:02:41.259 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:02:41.259 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:02:41.260 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:02:41.260 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:41.261 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:02:41.261 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:02:41.261 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:02:41.262 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:02:41.262 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:02:41.263 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:41.263 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:02:41.264 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:02:41.294 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15164.asok 2021-11-21T22:02:41.296 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15164.asok 2021-11-21T22:02:41.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:41.298 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15164.asok status 2021-11-21T22:02:41.401 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:02:41.401 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:02:41.401 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:02:41.402 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:02:41.402 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:02:41.402 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:02:41.402 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:02:41.403 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15164", 2021-11-21T22:02:41.403 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:02:41.403 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:41.403 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:02:41.403 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:02:41.404 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5290, 2021-11-21T22:02:41.404 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:02:41.404 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:02:41.404 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:02:41.405 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5290 2021-11-21T22:02:41.405 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:41.405 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:02:41.405 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:02:41.406 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:02:41.406 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 4201519846 2021-11-21T22:02:41.406 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:02:41.406 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:41.406 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:02:41.407 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:02:41.407 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:02:41.408 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 4201519846 2021-11-21T22:02:41.408 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:02:41.408 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5290 172.21.2.119:0/4201519846", 2021-11-21T22:02:41.408 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/4201519846", 2021-11-21T22:02:41.408 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:02:41.409 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 65, 2021-11-21T22:02:41.409 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 141, 2021-11-21T22:02:41.409 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:02:41.409 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:02:41.410 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:02:41.410 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:02:41.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:41.412 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:41.413 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:41.418 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:02:41.419 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:41.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:41.420 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:41.429 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:41.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc293e2700 1 Processor -- start 2021-11-21T22:02:41.486 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc293e2700 1 -- start start 2021-11-21T22:02:41.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc293e2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbc240a86b0 0x7fbc2414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc293e2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240a6e80 con 0x7fbc240a6aa0 2021-11-21T22:02:41.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc293e2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2414a7b0 con 0x7fbc24062410 2021-11-21T22:02:41.490 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc293e2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc240a8bc0 con 0x7fbc240a86b0 2021-11-21T22:02:41.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc22ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbc240a6aa0 0x7fbc241470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:36958/0 (socket says 172.21.2.116:36958) 2021-11-21T22:02:41.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc22ffd700 1 -- 172.21.2.116:0/3565584940 learned_addr learned my addr 172.21.2.116:0/3565584940 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:41.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261598295 0 0) 0x7fbc240a6e80 con 0x7fbc240a6aa0 2021-11-21T22:02:41.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc08003260 con 0x7fbc240a6aa0 2021-11-21T22:02:41.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc227fc700 1 --2- 172.21.2.116:0/3565584940 >> v2:172.21.2.117:3300/0 conn(0x7fbc240a86b0 0x7fbc2414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1278506361 0 0) 0x7fbc08003260 con 0x7fbc240a6aa0 2021-11-21T22:02:41.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 >> v2:172.21.2.117:3300/0 conn(0x7fbc240a86b0 msgr2=0x7fbc2414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:41.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 --2- 172.21.2.116:0/3565584940 >> v2:172.21.2.117:3300/0 conn(0x7fbc240a86b0 0x7fbc2414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 >> v1:172.21.2.117:6789/0 conn(0x7fbc24062410 legacy=0x7fbc241443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:41.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2414aa10 con 0x7fbc240a6aa0 2021-11-21T22:02:41.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fbc14001680 con 0x7fbc240a6aa0 2021-11-21T22:02:41.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 >> v1:172.21.2.116:6789/0 conn(0x7fbc240a6aa0 legacy=0x7fbc241470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08003820 0x7fbc08003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc08004500 0x7fbc08004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc08004f70 0x7fbc0800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240a8bc0 con 0x7fbc08004f70 2021-11-21T22:02:41.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0800be50 con 0x7fbc08003820 2021-11-21T22:02:41.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0800c010 con 0x7fbc08004500 2021-11-21T22:02:41.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc08004f70 0x7fbc0800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc227fc700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc08004500 0x7fbc08004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 -- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc08004500 msgr2=0x7fbc08004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:41.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc08004500 0x7fbc08004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 -- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08003820 msgr2=0x7fbc08003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:41.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc22ffd700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08003820 0x7fbc08003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08003820 0x7fbc08003c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 -- 172.21.2.116:0/3565584940 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08003260 con 0x7fbc08004f70 2021-11-21T22:02:41.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 -- 172.21.2.116:0/3565584940 reap_dead start 2021-11-21T22:02:41.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc237fe700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc08004f70 0x7fbc0800b750 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fbc1c002320 tx=0x7fbc1c007d80).ready entity=mon.0 client_cookie=3707c6b6e2efd1a5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.482+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c002510 con 0x7fbc08004f70 2021-11-21T22:02:41.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c006890 con 0x7fbc08004f70 2021-11-21T22:02:41.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc21ffb700 1 -- 172.21.2.116:0/3565584940 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c006bf0 con 0x7fbc08004f70 2021-11-21T22:02:41.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/3565584940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc08004f70 msgr2=0x7fbc0800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc08004f70 0x7fbc0800b750 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fbc1c002320 tx=0x7fbc1c007d80).stop 2021-11-21T22:02:41.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/3565584940 shutdown_connections 2021-11-21T22:02:41.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 --2- 172.21.2.116:0/3565584940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc08004f70 0x7fbc0800b750 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/3565584940 >> 172.21.2.116:0/3565584940 conn(0x7fbc2405bd20 msgr2=0x7fbc2405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:41.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/3565584940 shutdown_connections 2021-11-21T22:02:41.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/3565584940 wait complete. 2021-11-21T22:02:41.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 Processor -- start 2021-11-21T22:02:41.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- start start 2021-11-21T22:02:41.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08004f70 0x7fbc24147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc24065bb0 0x7fbc240ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc24066d50 0x7fbc24067200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240676f0 con 0x7fbc24066d50 2021-11-21T22:02:41.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc24066050 con 0x7fbc08004f70 2021-11-21T22:02:41.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240662f0 con 0x7fbc24065bb0 2021-11-21T22:02:41.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc24066d50 0x7fbc24067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc24066d50 0x7fbc24067200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53250/0 (socket says 172.21.2.116:53250) 2021-11-21T22:02:41.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 -- 172.21.2.116:0/2294325120 learned_addr learned my addr 172.21.2.116:0/2294325120 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:41.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 -- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc24065bb0 msgr2=0x7fbc240ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc24065bb0 0x7fbc240ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 -- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08004f70 msgr2=0x7fbc24147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08004f70 0x7fbc24147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 -- 172.21.2.116:0/2294325120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24066590 con 0x7fbc24066d50 2021-11-21T22:02:41.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc237fe700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc24066d50 0x7fbc24067200 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fbc1c002320 tx=0x7fbc1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc13fff700 1 -- 172.21.2.116:0/2294325120 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c002510 con 0x7fbc24066d50 2021-11-21T22:02:41.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc13fff700 1 -- 172.21.2.116:0/2294325120 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c01e670 con 0x7fbc24066d50 2021-11-21T22:02:41.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc24066890 con 0x7fbc24066d50 2021-11-21T22:02:41.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc13fff700 1 -- 172.21.2.116:0/2294325120 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c01e9d0 con 0x7fbc24066d50 2021-11-21T22:02:41.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc24159530 con 0x7fbc24066d50 2021-11-21T22:02:41.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.486+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc240573d0 con 0x7fbc24066d50 2021-11-21T22:02:41.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.490+0000 7fbc13fff700 1 -- 172.21.2.116:0/2294325120 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbc1c01b040 con 0x7fbc24066d50 2021-11-21T22:02:41.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.490+0000 7fbc13fff700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbc000535d0 0x7fbc00055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.490+0000 7fbc13fff700 1 -- 172.21.2.116:0/2294325120 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbc1c06ccd0 con 0x7fbc24066d50 2021-11-21T22:02:41.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.490+0000 7fbc22ffd700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbc000535d0 0x7fbc00055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.490+0000 7fbc22ffd700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbc000535d0 0x7fbc00055a90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fbc14008000 tx=0x7fbc14009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.490+0000 7fbc13fff700 1 -- 172.21.2.116:0/2294325120 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbc1c043cc0 con 0x7fbc24066d50 2021-11-21T22:02:41.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.574+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc24066a60 con 0x7fbc24066d50 2021-11-21T22:02:41.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.574+0000 7fbc13fff700 1 -- 172.21.2.116:0/2294325120 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16052 (secure 0 0 0) 0x7fbc1c0472e0 con 0x7fbc24066d50 2021-11-21T22:02:41.580 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:41.580 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":141,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:37.337392+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:35.110650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:35.304135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:41.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbc000535d0 msgr2=0x7fbc00055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbc000535d0 0x7fbc00055a90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fbc14008000 tx=0x7fbc14009040).stop 2021-11-21T22:02:41.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc24066d50 msgr2=0x7fbc24067200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc24066d50 0x7fbc24067200 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fbc1c002320 tx=0x7fbc1c016000).stop 2021-11-21T22:02:41.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 shutdown_connections 2021-11-21T22:02:41.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbc000535d0 0x7fbc00055a90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbc24066d50 0x7fbc24067200 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbc24065bb0 0x7fbc240ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 --2- 172.21.2.116:0/2294325120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbc08004f70 0x7fbc24147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 >> 172.21.2.116:0/2294325120 conn(0x7fbc2405bd20 msgr2=0x7fbc24148f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:41.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 shutdown_connections 2021-11-21T22:02:41.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.578+0000 7fbc293e2700 1 -- 172.21.2.116:0/2294325120 wait complete. 2021-11-21T22:02:41.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:02:35.110650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:02:35.304135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:02:41.588 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:02:41.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 Processor -- start 2021-11-21T22:02:41.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- start start 2021-11-21T22:02:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0800144810 0x7f08001478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f08000ac600 0x7f08000aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0800147e80 con 0x7f08000abce0 2021-11-21T22:02:41.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000b1490 con 0x7f08000ac600 2021-11-21T22:02:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000b1fd0 con 0x7f0800144810 2021-11-21T22:02:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f08000ac600 0x7f08000aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f08000ac600 0x7f08000aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53256/0 (socket says 172.21.2.116:53256) 2021-11-21T22:02:41.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 -- 172.21.2.116:0/2215713940 learned_addr learned my addr 172.21.2.116:0/2215713940 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:41.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 -- 172.21.2.116:0/2215713940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0800144810 msgr2=0x7f08001478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:41.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 --2- 172.21.2.116:0/2215713940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0800144810 0x7f08001478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 -- 172.21.2.116:0/2215713940 >> v1:172.21.2.116:6789/0 conn(0x7f08000abce0 legacy=0x7f08000b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 -- 172.21.2.116:0/2215713940 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000af5a0 con 0x7f08000ac600 2021-11-21T22:02:41.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fed9d700 1 --2- 172.21.2.116:0/2215713940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08000ac600 0x7f08000aca90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f07f80032f0 tx=0x7f07f8009290).ready entity=mon.0 client_cookie=5f3ae9e12cb767f7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fd59a700 1 -- 172.21.2.116:0/2215713940 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f800b8a0 con 0x7f08000ac600 2021-11-21T22:02:41.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fd59a700 1 -- 172.21.2.116:0/2215713940 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f800ba00 con 0x7f08000ac600 2021-11-21T22:02:41.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f07fd59a700 1 -- 172.21.2.116:0/2215713940 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f800bb70 con 0x7f08000ac600 2021-11-21T22:02:41.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- 172.21.2.116:0/2215713940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08000ac600 msgr2=0x7f08000aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 --2- 172.21.2.116:0/2215713940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08000ac600 0x7f08000aca90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f07f80032f0 tx=0x7f07f8009290).stop 2021-11-21T22:02:41.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- 172.21.2.116:0/2215713940 shutdown_connections 2021-11-21T22:02:41.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 --2- 172.21.2.116:0/2215713940 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f08000ac600 0x7f08000aca90 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 --2- 172.21.2.116:0/2215713940 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0800144810 0x7f08001478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- 172.21.2.116:0/2215713940 >> 172.21.2.116:0/2215713940 conn(0x7f080005bd20 msgr2=0x7f080005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:41.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- 172.21.2.116:0/2215713940 shutdown_connections 2021-11-21T22:02:41.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.646+0000 7f0804af7700 1 -- 172.21.2.116:0/2215713940 wait complete. 2021-11-21T22:02:41.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 Processor -- start 2021-11-21T22:02:41.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 -- start start 2021-11-21T22:02:41.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08000abce0 0x7f0800145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08000ac600 0x7f0800159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0800144810 0x7f0800159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001574f0 con 0x7f0800144810 2021-11-21T22:02:41.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800157250 con 0x7f08000abce0 2021-11-21T22:02:41.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000639f0 con 0x7f08000ac600 2021-11-21T22:02:41.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0800144810 0x7f0800159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0800144810 0x7f0800159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53262/0 (socket says 172.21.2.116:53262) 2021-11-21T22:02:41.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 -- 172.21.2.116:0/4229841214 learned_addr learned my addr 172.21.2.116:0/4229841214 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:41.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 -- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08000ac600 msgr2=0x7f0800159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08000ac600 0x7f0800159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 -- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08000abce0 msgr2=0x7f0800145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08000abce0 0x7f0800145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 -- 172.21.2.116:0/4229841214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009530 con 0x7f0800144810 2021-11-21T22:02:41.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fed9d700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0800144810 0x7f0800159ff0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f07f80032f0 tx=0x7f07f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07f77fe700 1 -- 172.21.2.116:0/4229841214 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f800bde0 con 0x7f0800144810 2021-11-21T22:02:41.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07f77fe700 1 -- 172.21.2.116:0/4229841214 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8020430 con 0x7f0800144810 2021-11-21T22:02:41.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0800063c90 con 0x7f0800144810 2021-11-21T22:02:41.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0800143660 con 0x7f0800144810 2021-11-21T22:02:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07f77fe700 1 -- 172.21.2.116:0/4229841214 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f8020790 con 0x7f0800144810 2021-11-21T22:02:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0800063e20 con 0x7f0800144810 2021-11-21T22:02:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07f77fe700 1 -- 172.21.2.116:0/4229841214 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f07f800f070 con 0x7f0800144810 2021-11-21T22:02:41.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07f77fe700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07e40535d0 0x7f07e4055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07f77fe700 1 -- 172.21.2.116:0/4229841214 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f07f806cd70 con 0x7f0800144810 2021-11-21T22:02:41.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.650+0000 7f07fe59c700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07e40535d0 0x7f07e4055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.654+0000 7f07fe59c700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07e40535d0 0x7f07e4055a90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f07f0002f70 tx=0x7f07f00069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.654+0000 7f07f77fe700 1 -- 172.21.2.116:0/4229841214 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07f8043dc0 con 0x7f0800144810 2021-11-21T22:02:41.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.734+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f08000af060 con 0x7f0800144810 2021-11-21T22:02:41.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.734+0000 7f07f77fe700 1 -- 172.21.2.116:0/4229841214 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7f07f8015030 con 0x7f0800144810 2021-11-21T22:02:41.740 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:02:41.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07e40535d0 msgr2=0x7f07e4055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07e40535d0 0x7f07e4055a90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f07f0002f70 tx=0x7f07f00069e0).stop 2021-11-21T22:02:41.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0800144810 msgr2=0x7f0800159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0800144810 0x7f0800159ff0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f07f80032f0 tx=0x7f07f8016000).stop 2021-11-21T22:02:41.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 shutdown_connections 2021-11-21T22:02:41.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f07e40535d0 0x7f07e4055a90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0800144810 0x7f0800159ff0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f08000ac600 0x7f0800159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 --2- 172.21.2.116:0/4229841214 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f08000abce0 0x7f0800145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 >> 172.21.2.116:0/4229841214 conn(0x7f080005bd20 msgr2=0x7f0800147020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:41.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 shutdown_connections 2021-11-21T22:02:41.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.738+0000 7f0804af7700 1 -- 172.21.2.116:0/4229841214 wait complete. 2021-11-21T22:02:41.749 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:02:41.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 Processor -- start 2021-11-21T22:02:41.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- start start 2021-11-21T22:02:41.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d38144810 0x7f6d381478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6d380ac600 0x7f6d380aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d38147e80 con 0x7f6d380abce0 2021-11-21T22:02:41.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380b1490 con 0x7f6d380ac600 2021-11-21T22:02:41.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b1fd0 con 0x7f6d38144810 2021-11-21T22:02:41.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6d380ac600 0x7f6d380aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6d380ac600 0x7f6d380aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53268/0 (socket says 172.21.2.116:53268) 2021-11-21T22:02:41.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 -- 172.21.2.116:0/1535637170 learned_addr learned my addr 172.21.2.116:0/1535637170 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:41.808 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 -- 172.21.2.116:0/1535637170 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d38144810 msgr2=0x7f6d381478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:41.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 --2- 172.21.2.116:0/1535637170 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d38144810 0x7f6d381478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 -- 172.21.2.116:0/1535637170 >> v1:172.21.2.116:6789/0 conn(0x7f6d380abce0 legacy=0x7f6d380b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 -- 172.21.2.116:0/1535637170 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380af5a0 con 0x7f6d380ac600 2021-11-21T22:02:41.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d377fe700 1 --2- 172.21.2.116:0/1535637170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d380aca90 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f6d300032f0 tx=0x7f6d30009290).ready entity=mon.0 client_cookie=91c3d4735854e4f3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d35ffb700 1 -- 172.21.2.116:0/1535637170 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d3000b8a0 con 0x7f6d380ac600 2021-11-21T22:02:41.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d35ffb700 1 -- 172.21.2.116:0/1535637170 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3000ba00 con 0x7f6d380ac600 2021-11-21T22:02:41.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d35ffb700 1 -- 172.21.2.116:0/1535637170 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d3000bbb0 con 0x7f6d380ac600 2021-11-21T22:02:41.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1535637170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 msgr2=0x7f6d380aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1535637170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d380aca90 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f6d300032f0 tx=0x7f6d30009290).stop 2021-11-21T22:02:41.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1535637170 shutdown_connections 2021-11-21T22:02:41.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1535637170 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d380aca90 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1535637170 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d38144810 0x7f6d381478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1535637170 >> 172.21.2.116:0/1535637170 conn(0x7f6d3805bd20 msgr2=0x7f6d3805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:41.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1535637170 shutdown_connections 2021-11-21T22:02:41.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.798+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1535637170 wait complete. 2021-11-21T22:02:41.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 Processor -- start 2021-11-21T22:02:41.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 -- start start 2021-11-21T22:02:41.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d380abce0 0x7f6d38144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d38143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d38144810 0x7f6d38156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38063250 con 0x7f6d380ac600 2021-11-21T22:02:41.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38143c70 con 0x7f6d38144810 2021-11-21T22:02:41.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381564f0 con 0x7f6d380abce0 2021-11-21T22:02:41.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d38143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d38143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53274/0 (socket says 172.21.2.116:53274) 2021-11-21T22:02:41.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 -- 172.21.2.116:0/1423367609 learned_addr learned my addr 172.21.2.116:0/1423367609 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:41.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 -- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d380abce0 msgr2=0x7f6d38144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d380abce0 0x7f6d38144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 -- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d38144810 msgr2=0x7f6d38156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d38144810 0x7f6d38156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 -- 172.21.2.116:0/1423367609 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d30009530 con 0x7f6d380ac600 2021-11-21T22:02:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d367fc700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d38143510 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f6d380b2a30 tx=0x7f6d20007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d2ffff700 1 -- 172.21.2.116:0/1423367609 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d2000ce30 con 0x7f6d380ac600 2021-11-21T22:02:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d2ffff700 1 -- 172.21.2.116:0/1423367609 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2000f470 con 0x7f6d380ac600 2021-11-21T22:02:41.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d2ffff700 1 -- 172.21.2.116:0/1423367609 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d2000f7d0 con 0x7f6d380ac600 2021-11-21T22:02:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3815a770 con 0x7f6d380ac600 2021-11-21T22:02:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3815a480 con 0x7f6d380ac600 2021-11-21T22:02:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d380af060 con 0x7f6d380ac600 2021-11-21T22:02:41.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d2ffff700 1 -- 172.21.2.116:0/1423367609 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6d20019030 con 0x7f6d380ac600 2021-11-21T22:02:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d2ffff700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d1c053610 0x7f6d1c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d2ffff700 1 -- 172.21.2.116:0/1423367609 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d2006aef0 con 0x7f6d380ac600 2021-11-21T22:02:41.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.802+0000 7f6d36ffd700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d1c053610 0x7f6d1c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.806+0000 7f6d36ffd700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d1c053610 0x7f6d1c055ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f6d28002ef0 tx=0x7f6d28006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.806+0000 7f6d2ffff700 1 -- 172.21.2.116:0/1423367609 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d20041ee0 con 0x7f6d380ac600 2021-11-21T22:02:41.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d380573d0 con 0x7f6d380ac600 2021-11-21T22:02:41.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d2ffff700 1 -- 172.21.2.116:0/1423367609 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d20045500 con 0x7f6d380ac600 2021-11-21T22:02:41.894 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:41.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d1c053610 msgr2=0x7f6d1c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d1c053610 0x7f6d1c055ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f6d28002ef0 tx=0x7f6d28006910).stop 2021-11-21T22:02:41.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 msgr2=0x7f6d38143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d38143510 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f6d380b2a30 tx=0x7f6d20007c50).stop 2021-11-21T22:02:41.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 shutdown_connections 2021-11-21T22:02:41.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6d38144810 0x7f6d38156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6d1c053610 0x7f6d1c055ad0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6d380ac600 0x7f6d38143510 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 --2- 172.21.2.116:0/1423367609 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6d380abce0 0x7f6d38144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 >> 172.21.2.116:0/1423367609 conn(0x7f6d3805bd20 msgr2=0x7f6d381494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:41.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 shutdown_connections 2021-11-21T22:02:41.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.890+0000 7f6d3d35b700 1 -- 172.21.2.116:0/1423367609 wait complete. 2021-11-21T22:02:41.900 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:02:41.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 Processor -- start 2021-11-21T22:02:41.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- start start 2021-11-21T22:02:41.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa7481478c0 0x7fa7480acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa748147e80 con 0x7fa748144810 2021-11-21T22:02:41.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7480af5a0 con 0x7fa7480abce0 2021-11-21T22:02:41.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7480b1490 con 0x7fa7481478c0 2021-11-21T22:02:41.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa746d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa7481478c0 0x7fa7480acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa746d9d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa7481478c0 0x7fa7480acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53282/0 (socket says 172.21.2.116:53282) 2021-11-21T22:02:41.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa746d9d700 1 -- 172.21.2.116:0/169418506 learned_addr learned my addr 172.21.2.116:0/169418506 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:41.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa746d9d700 1 -- 172.21.2.116:0/169418506 >> v1:172.21.2.117:6789/0 conn(0x7fa7480abce0 legacy=0x7fa7480ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:41.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa746d9d700 1 -- 172.21.2.116:0/169418506 >> v1:172.21.2.116:6789/0 conn(0x7fa748144810 legacy=0x7fa7480b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa746d9d700 1 -- 172.21.2.116:0/169418506 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480b1fd0 con 0x7fa7481478c0 2021-11-21T22:02:41.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa746d9d700 1 --2- 172.21.2.116:0/169418506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa7480acd10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fa7480b4e00 tx=0x7fa740009290).ready entity=mon.0 client_cookie=f511f91209aa02b8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74559a700 1 -- 172.21.2.116:0/169418506 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa74000b860 con 0x7fa7481478c0 2021-11-21T22:02:41.958 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74559a700 1 -- 172.21.2.116:0/169418506 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74000b9c0 con 0x7fa7481478c0 2021-11-21T22:02:41.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74559a700 1 -- 172.21.2.116:0/169418506 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa74000bb30 con 0x7fa7481478c0 2021-11-21T22:02:41.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- 172.21.2.116:0/169418506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 msgr2=0x7fa7480acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/169418506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa7480acd10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fa7480b4e00 tx=0x7fa740009290).stop 2021-11-21T22:02:41.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- 172.21.2.116:0/169418506 shutdown_connections 2021-11-21T22:02:41.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/169418506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa7480acd10 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:41.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- 172.21.2.116:0/169418506 >> 172.21.2.116:0/169418506 conn(0x7fa74805bd20 msgr2=0x7fa74805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:41.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- 172.21.2.116:0/169418506 shutdown_connections 2021-11-21T22:02:41.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- 172.21.2.116:0/169418506 wait complete. 2021-11-21T22:02:41.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 Processor -- start 2021-11-21T22:02:41.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- start start 2021-11-21T22:02:41.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7480abce0 0x7fa748144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa748144810 0x7fa74813f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa748143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748146cf0 con 0x7fa7481478c0 2021-11-21T22:02:41.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74815aaf0 con 0x7fa7480abce0 2021-11-21T22:02:41.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.950+0000 7fa74ce9a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa748159c00 con 0x7fa748144810 2021-11-21T22:02:41.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa748143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa748143360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53286/0 (socket says 172.21.2.116:53286) 2021-11-21T22:02:41.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 -- 172.21.2.116:0/4271478085 learned_addr learned my addr 172.21.2.116:0/4271478085 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:41.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 -- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa748144810 msgr2=0x7fa74813f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa748144810 0x7fa74813f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 -- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7480abce0 msgr2=0x7fa748144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:41.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7480abce0 0x7fa748144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:41.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 -- 172.21.2.116:0/4271478085 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa740009530 con 0x7fa7481478c0 2021-11-21T22:02:41.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa746d9d700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa748143360 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fa7480b4e00 tx=0x7fa740016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa7377fe700 1 -- 172.21.2.116:0/4271478085 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa74000b860 con 0x7fa7481478c0 2021-11-21T22:02:41.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa7377fe700 1 -- 172.21.2.116:0/4271478085 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74001f5e0 con 0x7fa7481478c0 2021-11-21T22:02:41.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa7377fe700 1 -- 172.21.2.116:0/4271478085 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa74001f940 con 0x7fa7481478c0 2021-11-21T22:02:41.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7480b32a0 con 0x7fa7481478c0 2021-11-21T22:02:41.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7480b37e0 con 0x7fa7481478c0 2021-11-21T22:02:41.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa748143070 con 0x7fa7481478c0 2021-11-21T22:02:41.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa7377fe700 1 -- 172.21.2.116:0/4271478085 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa740026030 con 0x7fa7481478c0 2021-11-21T22:02:41.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa7377fe700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa724053600 0x7fa724055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:41.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa7377fe700 1 -- 172.21.2.116:0/4271478085 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa74006c7f0 con 0x7fa7481478c0 2021-11-21T22:02:41.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa74659c700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa724053600 0x7fa724055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:41.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.954+0000 7fa74659c700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa724053600 0x7fa724055ac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fa748062500 tx=0x7fa738007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:41.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:41.958+0000 7fa7377fe700 1 -- 172.21.2.116:0/4271478085 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa740043840 con 0x7fa7481478c0 2021-11-21T22:02:42.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.038+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7480573d0 con 0x7fa7481478c0 2021-11-21T22:02:42.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa7377fe700 1 -- 172.21.2.116:0/4271478085 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa740046e60 con 0x7fa7481478c0 2021-11-21T22:02:42.063 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:42.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa724053600 msgr2=0x7fa724055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa724053600 0x7fa724055ac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fa748062500 tx=0x7fa738007810).stop 2021-11-21T22:02:42.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 msgr2=0x7fa748143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa748143360 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fa7480b4e00 tx=0x7fa740016000).stop 2021-11-21T22:02:42.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 shutdown_connections 2021-11-21T22:02:42.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa724053600 0x7fa724055ac0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa7481478c0 0x7fa748143360 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa748144810 0x7fa74813f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 --2- 172.21.2.116:0/4271478085 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa7480abce0 0x7fa748144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 >> 172.21.2.116:0/4271478085 conn(0x7fa74805bd20 msgr2=0x7fa7481483e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:42.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 shutdown_connections 2021-11-21T22:02:42.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.042+0000 7fa74ce9a700 1 -- 172.21.2.116:0/4271478085 wait complete. 2021-11-21T22:02:42.068 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:02:42.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 Processor -- start 2021-11-21T22:02:42.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- start start 2021-11-21T22:02:42.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc540b6b50 0x7efc540adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc540ae180 con 0x7efc540b1270 2021-11-21T22:02:42.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc540b0070 con 0x7efc540b6770 2021-11-21T22:02:42.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc540b05c0 con 0x7efc540b6b50 2021-11-21T22:02:42.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc51d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7efc540b1270 0x7efc540ad580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37022/0 (socket says 172.21.2.116:37022) 2021-11-21T22:02:42.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc51d9b700 1 -- 172.21.2.116:0/3692131352 learned_addr learned my addr 172.21.2.116:0/3692131352 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:42.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349309004 0 0) 0x7efc540ae180 con 0x7efc540b1270 2021-11-21T22:02:42.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc38003260 con 0x7efc540b1270 2021-11-21T22:02:42.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2828273347 0 0) 0x7efc38003260 con 0x7efc540b1270 2021-11-21T22:02:42.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc540b6b50 msgr2=0x7efc540adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:42.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc540b6b50 0x7efc540adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 >> v1:172.21.2.117:6789/0 conn(0x7efc540b6770 legacy=0x7efc541394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:42.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540b2230 con 0x7efc540b1270 2021-11-21T22:02:42.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7efc44001680 con 0x7efc540b1270 2021-11-21T22:02:42.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 >> v1:172.21.2.116:6789/0 conn(0x7efc540b1270 legacy=0x7efc540ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efc38003800 0x7efc38003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc540b6b50 0x7efc38004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc38004bf0 0x7efc3800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540b05c0 con 0x7efc38004bf0 2021-11-21T22:02:42.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380052d0 con 0x7efc38003800 2021-11-21T22:02:42.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3800bc40 con 0x7efc540b6b50 2021-11-21T22:02:42.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc38004bf0 0x7efc3800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:42.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 -- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc540b6b50 msgr2=0x7efc38004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:42.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc540b6b50 0x7efc38004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 -- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efc38003800 msgr2=0x7efc38003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:42.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efc38003800 0x7efc38003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 -- 172.21.2.116:0/3692131352 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc38003260 con 0x7efc38004bf0 2021-11-21T22:02:42.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc38004bf0 0x7efc3800b750 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7efc4c0032f0 tx=0x7efc4c006f90).ready entity=mon.0 client_cookie=4f223915631bd87c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:42.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4c00f690 con 0x7efc38004bf0 2021-11-21T22:02:42.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00f7f0 con 0x7efc38004bf0 2021-11-21T22:02:42.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc5159a700 1 -- 172.21.2.116:0/3692131352 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4c00fb50 con 0x7efc38004bf0 2021-11-21T22:02:42.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- 172.21.2.116:0/3692131352 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc38004bf0 msgr2=0x7efc3800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 --2- 172.21.2.116:0/3692131352 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc38004bf0 0x7efc3800b750 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7efc4c0032f0 tx=0x7efc4c006f90).stop 2021-11-21T22:02:42.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 -- 172.21.2.116:0/3692131352 reap_dead start 2021-11-21T22:02:42.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- 172.21.2.116:0/3692131352 shutdown_connections 2021-11-21T22:02:42.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- 172.21.2.116:0/3692131352 >> 172.21.2.116:0/3692131352 conn(0x7efc54062040 msgr2=0x7efc540b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:42.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- 172.21.2.116:0/3692131352 shutdown_connections 2021-11-21T22:02:42.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- 172.21.2.116:0/3692131352 wait complete. 2021-11-21T22:02:42.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 Processor -- start 2021-11-21T22:02:42.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- start start 2021-11-21T22:02:42.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efc38004bf0 0x7efc54067960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc5414da20 0x7efc54151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc5413ac70 0x7efc540b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54065820 con 0x7efc5413ac70 2021-11-21T22:02:42.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54151170 con 0x7efc38004bf0 2021-11-21T22:02:42.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc58d20700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc54065580 con 0x7efc5414da20 2021-11-21T22:02:42.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc5413ac70 0x7efc540b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:42.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc5413ac70 0x7efc540b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53308/0 (socket says 172.21.2.116:53308) 2021-11-21T22:02:42.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.118+0000 7efc52d9d700 1 -- 172.21.2.116:0/1127657370 learned_addr learned my addr 172.21.2.116:0/1127657370 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:42.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc52d9d700 1 -- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc5414da20 msgr2=0x7efc54151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc52d9d700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc5414da20 0x7efc54151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc52d9d700 1 -- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efc38004bf0 msgr2=0x7efc54067960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc52d9d700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efc38004bf0 0x7efc54067960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc52d9d700 1 -- 172.21.2.116:0/1127657370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc541513d0 con 0x7efc5413ac70 2021-11-21T22:02:42.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc52d9d700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc5413ac70 0x7efc540b5100 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7efc540af490 tx=0x7efc4c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:42.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc437fe700 1 -- 172.21.2.116:0/1127657370 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4c01f070 con 0x7efc5413ac70 2021-11-21T22:02:42.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc437fe700 1 -- 172.21.2.116:0/1127657370 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c010cd0 con 0x7efc5413ac70 2021-11-21T22:02:42.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5414dea0 con 0x7efc5413ac70 2021-11-21T22:02:42.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc437fe700 1 -- 172.21.2.116:0/1127657370 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4c019640 con 0x7efc5413ac70 2021-11-21T22:02:42.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc54151d60 con 0x7efc5413ac70 2021-11-21T22:02:42.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc54150f80 con 0x7efc5413ac70 2021-11-21T22:02:42.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc437fe700 1 -- 172.21.2.116:0/1127657370 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7efc4c02b070 con 0x7efc5413ac70 2021-11-21T22:02:42.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc437fe700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efc300536d0 0x7efc30055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc437fe700 1 -- 172.21.2.116:0/1127657370 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7efc4c06ed20 con 0x7efc5413ac70 2021-11-21T22:02:42.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc5259c700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efc300536d0 0x7efc30055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:42.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc5259c700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efc300536d0 0x7efc30055b90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7efc48000f70 tx=0x7efc480069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:42.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.122+0000 7efc437fe700 1 -- 172.21.2.116:0/1127657370 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efc4c045e80 con 0x7efc5413ac70 2021-11-21T22:02:42.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.206+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc54151560 con 0x7efc5413ac70 2021-11-21T22:02:42.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc437fe700 1 -- 172.21.2.116:0/1127657370 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+10 (secure 0 0 0) 0x7efc4c0494a0 con 0x7efc5413ac70 2021-11-21T22:02:42.213 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:42.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efc300536d0 msgr2=0x7efc30055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efc300536d0 0x7efc30055b90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7efc48000f70 tx=0x7efc480069b0).stop 2021-11-21T22:02:42.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc5413ac70 msgr2=0x7efc540b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc5413ac70 0x7efc540b5100 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7efc540af490 tx=0x7efc4c019000).stop 2021-11-21T22:02:42.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 shutdown_connections 2021-11-21T22:02:42.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efc300536d0 0x7efc30055b90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efc5413ac70 0x7efc540b5100 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efc5414da20 0x7efc54151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 --2- 172.21.2.116:0/1127657370 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efc38004bf0 0x7efc54067960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 >> 172.21.2.116:0/1127657370 conn(0x7efc54062040 msgr2=0x7efc5413bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:42.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 shutdown_connections 2021-11-21T22:02:42.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.210+0000 7efc58d20700 1 -- 172.21.2.116:0/1127657370 wait complete. 2021-11-21T22:02:42.220 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:02:42.221 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:42.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.266+0000 7f7e4263a700 1 Processor -- start 2021-11-21T22:02:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.266+0000 7f7e4263a700 1 -- start start 2021-11-21T22:02:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.266+0000 7f7e4263a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7e3c0a9d00 0x7f7e3c13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.266+0000 7f7e4263a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 0x7f7e3c13e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.266+0000 7f7e4263a700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c144970 con 0x7f7e3c05cef0 2021-11-21T22:02:42.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.266+0000 7f7e4263a700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c13e750 con 0x7f7e3c0a9d00 2021-11-21T22:02:42.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.266+0000 7f7e4263a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0b1200 con 0x7f7e3c05c840 2021-11-21T22:02:42.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3bfff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7e3c0a9d00 0x7f7e3c13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:42.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3b7fe700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 0x7f7e3c13e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:42.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e40bd7700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f7e3c05cef0 0x7f7e3c0b17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:35952/0 (socket says 172.21.2.116:35952) 2021-11-21T22:02:42.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e40bd7700 1 -- 172.21.2.116:0/1055052498 learned_addr learned my addr 172.21.2.116:0/1055052498 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:42.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3bfff700 1 -- 172.21.2.116:0/1055052498 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 msgr2=0x7f7e3c13e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:42.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3bfff700 1 --2- 172.21.2.116:0/1055052498 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 0x7f7e3c13e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3bfff700 1 -- 172.21.2.116:0/1055052498 >> v1:172.21.2.117:6789/0 conn(0x7f7e3c05cef0 legacy=0x7f7e3c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3bfff700 1 -- 172.21.2.116:0/1055052498 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0aa0e0 con 0x7f7e3c0a9d00 2021-11-21T22:02:42.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3bfff700 1 --2- 172.21.2.116:0/1055052498 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 0x7f7e3c13dd70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f7e30002f70 tx=0x7f7e30006df0).ready entity=mon.1 client_cookie=c2766d575032781b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:42.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3affd700 1 -- 172.21.2.116:0/1055052498 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e3000d570 con 0x7f7e3c0a9d00 2021-11-21T22:02:42.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3affd700 1 -- 172.21.2.116:0/1055052498 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e30000e90 con 0x7f7e3c0a9d00 2021-11-21T22:02:42.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e3affd700 1 -- 172.21.2.116:0/1055052498 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e3000d900 con 0x7f7e3c0a9d00 2021-11-21T22:02:42.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 -- 172.21.2.116:0/1055052498 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 msgr2=0x7f7e3c13dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 --2- 172.21.2.116:0/1055052498 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 0x7f7e3c13dd70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f7e30002f70 tx=0x7f7e30006df0).stop 2021-11-21T22:02:42.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 -- 172.21.2.116:0/1055052498 shutdown_connections 2021-11-21T22:02:42.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 --2- 172.21.2.116:0/1055052498 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 0x7f7e3c13e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 --2- 172.21.2.116:0/1055052498 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 0x7f7e3c13dd70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 -- 172.21.2.116:0/1055052498 >> 172.21.2.116:0/1055052498 conn(0x7f7e3c05bd20 msgr2=0x7f7e3c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:42.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 -- 172.21.2.116:0/1055052498 shutdown_connections 2021-11-21T22:02:42.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 -- 172.21.2.116:0/1055052498 wait complete. 2021-11-21T22:02:42.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 Processor -- start 2021-11-21T22:02:42.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.270+0000 7f7e4263a700 1 -- start start 2021-11-21T22:02:42.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 0x7f7e3c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e3c05cef0 0x7f7e3c065700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 0x7f7e3c065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0ae6b0 con 0x7f7e3c05cef0 2021-11-21T22:02:42.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0673d0 con 0x7f7e3c0a9d00 2021-11-21T22:02:42.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0ac580 con 0x7f7e3c05c840 2021-11-21T22:02:42.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e3c05cef0 0x7f7e3c065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:42.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e3c05cef0 0x7f7e3c065700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53322/0 (socket says 172.21.2.116:53322) 2021-11-21T22:02:42.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 -- 172.21.2.116:0/2422171360 learned_addr learned my addr 172.21.2.116:0/2422171360 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:42.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 -- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 msgr2=0x7f7e3c0afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 0x7f7e3c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 -- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 msgr2=0x7f7e3c065bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 0x7f7e3c065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 -- 172.21.2.116:0/2422171360 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3000b180 con 0x7f7e3c05cef0 2021-11-21T22:02:42.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3b7fe700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e3c05cef0 0x7f7e3c065700 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f7e3c0ae7f0 tx=0x7f7e340091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:42.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e397fa700 1 -- 172.21.2.116:0/2422171360 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e3400c850 con 0x7f7e3c05cef0 2021-11-21T22:02:42.285 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e397fa700 1 -- 172.21.2.116:0/2422171360 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3400c9b0 con 0x7f7e3c05cef0 2021-11-21T22:02:42.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c0ac320 con 0x7f7e3c05cef0 2021-11-21T22:02:42.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c067600 con 0x7f7e3c05cef0 2021-11-21T22:02:42.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e397fa700 1 -- 172.21.2.116:0/2422171360 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e3400cd10 con 0x7f7e3c05cef0 2021-11-21T22:02:42.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c0abb80 con 0x7f7e3c05cef0 2021-11-21T22:02:42.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e397fa700 1 -- 172.21.2.116:0/2422171360 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7e34022070 con 0x7f7e3c05cef0 2021-11-21T22:02:42.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e397fa700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e1c0536a0 0x7f7e1c055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:42.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e397fa700 1 -- 172.21.2.116:0/2422171360 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7e3406c010 con 0x7f7e3c05cef0 2021-11-21T22:02:42.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.274+0000 7f7e3bfff700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e1c0536a0 0x7f7e1c055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:42.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.278+0000 7f7e3bfff700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e1c0536a0 0x7f7e1c055b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f7e30002f70 tx=0x7f7e30014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:42.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.278+0000 7f7e397fa700 1 -- 172.21.2.116:0/2422171360 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7e34043170 con 0x7f7e3c05cef0 2021-11-21T22:02:42.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e3c0573d0 con 0x7f7e3c05cef0 2021-11-21T22:02:42.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e397fa700 1 -- 172.21.2.116:0/2422171360 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f7e34046790 con 0x7f7e3c05cef0 2021-11-21T22:02:42.392 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:42.392 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:36.333834+0000","modified":"2021-11-21T22:02:37.338971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/1832322648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1832322648},{"type":"v1","addr":"172.21.2.116:6835","nonce":1832322648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-11-21T22:02:42.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e1c0536a0 msgr2=0x7f7e1c055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e1c0536a0 0x7f7e1c055b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f7e30002f70 tx=0x7f7e30014000).stop 2021-11-21T22:02:42.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e3c05cef0 msgr2=0x7f7e3c065700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:42.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e3c05cef0 0x7f7e3c065700 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f7e3c0ae7f0 tx=0x7f7e340091d0).stop 2021-11-21T22:02:42.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 shutdown_connections 2021-11-21T22:02:42.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7e3c0a9d00 0x7f7e3c065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7e1c0536a0 0x7f7e1c055b60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7e3c05cef0 0x7f7e3c065700 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:42.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 --2- 172.21.2.116:0/2422171360 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7e3c05c840 0x7f7e3c0afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:42.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 >> 172.21.2.116:0/2422171360 conn(0x7f7e3c05bd20 msgr2=0x7f7e3c1408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:42.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 shutdown_connections 2021-11-21T22:02:42.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:42.386+0000 7f7e4263a700 1 -- 172.21.2.116:0/2422171360 wait complete. 2021-11-21T22:02:42.395 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 65 2021-11-21T22:02:42.702 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:36.333834+0000', 'modified': '2021-11-21T22:02:37.338971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8109': {'gid': 8109, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.116:6835/1832322648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 1832322648}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1832322648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:02:42.702 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:02:42.703 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-11-21T22:02:42.767 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8109,"want_state":"up:active","state":"up:active","rank_uptime":6.4260139870000001,"mdsmap_epoch":65,"osdmap_epoch":141,"osdmap_epoch_barrier":141,"uptime":9.6719994109999998} 2021-11-21T22:02:42.768 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8109, "mdsmap_epoch": 65, "osdmap_epoch": 141, "osdmap_epoch_barrier": 141, "rank_uptime": 6.426013987, "state": "up:active", "uptime": 9.671999411, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:02:42.769 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:02:42.769 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:02:42.769 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:02:42.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:42.770 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:42.782 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:02:42.794 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:42.795 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:42.801 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:42.801 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:42.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:42.803 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:02:42.852 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:02:42.853 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:42.856 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:02:42.865 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:42.866 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:42.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:02:42.894+0000 7fd4f9d95080 -1 init, newargv = 0x55919ac164b0 newargc=9 2021-11-21T22:02:42.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14605]: starting ceph client 2021-11-21T22:02:42.899 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:42.901 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:42.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:42.903 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:02:42.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14605]: starting fuse 2021-11-21T22:02:42.909 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:02:42.909 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:02:42.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:42.910 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:02:42.911 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:02:42.911 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:02:42.912 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:02:42.912 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:02:42.912 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:42.913 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:02:42.913 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:02:42.914 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:02:42.914 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:42.914 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:02:42.915 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:02:42.916 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:02:42.917 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:02:42.918 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:42.918 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:02:42.919 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:02:42.920 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:02:42.921 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:02:42.921 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:02:42.922 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:42.923 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:02:42.924 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:02:43.005 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14605.asok 2021-11-21T22:02:43.009 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14605.asok 2021-11-21T22:02:43.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:43.010 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14605.asok status 2021-11-21T22:02:43.073 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:02:43.074 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:02:43.074 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:02:43.074 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:02:43.074 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:02:43.075 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:02:43.075 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:02:43.075 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14605", 2021-11-21T22:02:43.075 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:02:43.075 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:43.076 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:02:43.076 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:02:43.076 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 5302, 2021-11-21T22:02:43.077 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:02:43.077 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:02:43.077 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:02:43.077 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 5302 2021-11-21T22:02:43.078 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:43.078 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:02:43.078 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:02:43.078 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:02:43.079 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 3166042311 2021-11-21T22:02:43.079 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:02:43.079 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:43.079 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:02:43.079 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:02:43.080 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:02:43.080 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 3166042311 2021-11-21T22:02:43.080 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:43.080 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.5302 172.21.2.136:0/3166042311", 2021-11-21T22:02:43.080 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/3166042311", 2021-11-21T22:02:43.081 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:02:43.081 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 65, 2021-11-21T22:02:43.082 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 141, 2021-11-21T22:02:43.082 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:02:43.083 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:02:43.083 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:02:43.083 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:02:43.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:43.085 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:43.086 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:43.091 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:43.091 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:43.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:43.092 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:43.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:43.152 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:43.152 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:43.159 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:43.159 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:43.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:43.160 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:43.161 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:43.209 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:43.210 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:43.211 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:02:43.211 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:02:43.211 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:43.223 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14605]: fuse finished with error 0 and tester_r 0 2021-11-21T22:02:43.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:43.226 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:43.226 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:43.236 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:02:43.237 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:02:43.238 INFO:teuthology.orchestra.run:waiting for 900 2021-11-21T22:02:43.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:02:43.302 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:43.312 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:43.317 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:43.320 DEBUG:teuthology.orchestra.run.gibba019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:43.324 DEBUG:teuthology.orchestra.run.gibba036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:43.332 INFO:teuthology.orchestra.run.gibba016.stdout:[global] 2021-11-21T22:02:43.333 INFO:teuthology.orchestra.run.gibba016.stdout: chdir = "" 2021-11-21T22:02:43.333 INFO:teuthology.orchestra.run.gibba016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:43.333 INFO:teuthology.orchestra.run.gibba016.stdout: auth supported = cephx 2021-11-21T22:02:43.333 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.333 INFO:teuthology.orchestra.run.gibba016.stdout: filestore xattr use omap = true 2021-11-21T22:02:43.334 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.334 INFO:teuthology.orchestra.run.gibba016.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:43.334 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.334 INFO:teuthology.orchestra.run.gibba016.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:43.335 INFO:teuthology.orchestra.run.gibba016.stdout: auth debug = true 2021-11-21T22:02:43.335 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.335 INFO:teuthology.orchestra.run.gibba016.stdout: ms die on old message = true 2021-11-21T22:02:43.335 INFO:teuthology.orchestra.run.gibba016.stdout: ms die on bug = true 2021-11-21T22:02:43.336 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.336 INFO:teuthology.orchestra.run.gibba016.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:43.336 INFO:teuthology.orchestra.run.gibba016.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:43.336 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.337 INFO:teuthology.orchestra.run.gibba016.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:43.337 INFO:teuthology.orchestra.run.gibba016.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:43.337 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.337 INFO:teuthology.orchestra.run.gibba016.stdout: osd pool default size = 2 2021-11-21T22:02:43.338 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.338 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:43.338 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd allow pg remap = true 2021-11-21T22:02:43.339 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:43.339 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:43.339 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:43.339 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:43.340 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on too few osds = false 2021-11-21T22:02:43.340 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:43.340 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:43.340 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.340 INFO:teuthology.orchestra.run.gibba016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:43.341 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.341 INFO:teuthology.orchestra.run.gibba016.stdout: osd default data pool replay window = 5 2021-11-21T22:02:43.341 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.341 INFO:teuthology.orchestra.run.gibba016.stdout: mon allow pool delete = true 2021-11-21T22:02:43.342 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.342 INFO:teuthology.orchestra.run.gibba016.stdout: mon cluster log file level = debug 2021-11-21T22:02:43.342 INFO:teuthology.orchestra.run.gibba016.stdout: debug asserts on shutdown = true 2021-11-21T22:02:43.342 INFO:teuthology.orchestra.run.gibba016.stdout: mon health detail to clog = false 2021-11-21T22:02:43.342 INFO:teuthology.orchestra.run.gibba016.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:43.343 INFO:teuthology.orchestra.run.gibba016.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:43.343 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.343 INFO:teuthology.orchestra.run.gibba016.stdout:[osd] 2021-11-21T22:02:43.343 INFO:teuthology.orchestra.run.gibba016.stdout: osd journal size = 100 2021-11-21T22:02:43.344 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.344 INFO:teuthology.orchestra.run.gibba016.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:43.344 INFO:teuthology.orchestra.run.gibba016.stdout: osd scrub max interval = 600 2021-11-21T22:02:43.344 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.344 INFO:teuthology.orchestra.run.gibba016.stdout: osd recover clone overlap = true 2021-11-21T22:02:43.345 INFO:teuthology.orchestra.run.gibba016.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:43.345 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.345 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug shutdown = true 2021-11-21T22:02:43.345 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug op order = true 2021-11-21T22:02:43.346 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:43.346 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.346 INFO:teuthology.orchestra.run.gibba016.stdout: osd open classes on start = true 2021-11-21T22:02:43.346 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug pg log writeout = true 2021-11-21T22:02:43.346 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.347 INFO:teuthology.orchestra.run.gibba016.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:43.347 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.347 INFO:teuthology.orchestra.run.gibba016.stdout: osd map max advance = 10 2021-11-21T22:02:43.347 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.348 INFO:teuthology.orchestra.run.gibba016.stdout: journal zero on create = true 2021-11-21T22:02:43.348 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.348 INFO:teuthology.orchestra.run.gibba016.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:43.348 INFO:teuthology.orchestra.run.gibba016.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:43.348 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.349 INFO:teuthology.orchestra.run.gibba016.stdout: bdev debug aio = true 2021-11-21T22:02:43.349 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug misdirected ops = true 2021-11-21T22:02:43.349 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:43.350 INFO:teuthology.orchestra.run.gibba016.stdout: debug osd = 20 2021-11-21T22:02:43.350 INFO:teuthology.orchestra.run.gibba016.stdout: osd objectstore = filestore 2021-11-21T22:02:43.350 INFO:teuthology.orchestra.run.gibba016.stdout: osd op complaint time = 180 2021-11-21T22:02:43.350 INFO:teuthology.orchestra.run.gibba016.stdout: osd sloppy crc = True 2021-11-21T22:02:43.350 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.351 INFO:teuthology.orchestra.run.gibba016.stdout:[mgr] 2021-11-21T22:02:43.351 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:43.351 INFO:teuthology.orchestra.run.gibba016.stdout: debug mgr = 20 2021-11-21T22:02:43.351 INFO:teuthology.orchestra.run.gibba016.stdout: debug mon = 20 2021-11-21T22:02:43.352 INFO:teuthology.orchestra.run.gibba016.stdout: debug auth = 20 2021-11-21T22:02:43.352 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.352 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.352 INFO:teuthology.orchestra.run.gibba016.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:43.353 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.353 INFO:teuthology.orchestra.run.gibba016.stdout:[mon] 2021-11-21T22:02:43.353 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:43.353 INFO:teuthology.orchestra.run.gibba016.stdout: debug mon = 20 2021-11-21T22:02:43.353 INFO:teuthology.orchestra.run.gibba016.stdout: debug paxos = 20 2021-11-21T22:02:43.354 INFO:teuthology.orchestra.run.gibba016.stdout: debug auth = 20 2021-11-21T22:02:43.354 INFO:teuthology.orchestra.run.gibba016.stdout: mon data avail warn = 5 2021-11-21T22:02:43.354 INFO:teuthology.orchestra.run.gibba016.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:43.354 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.355 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:43.355 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd prime pg temp = true 2021-11-21T22:02:43.355 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.355 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.356 INFO:teuthology.orchestra.run.gibba016.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:43.356 INFO:teuthology.orchestra.run.gibba016.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:43.356 INFO:teuthology.orchestra.run.gibba016.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:43.356 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.357 INFO:teuthology.orchestra.run.gibba016.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:43.357 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:43.357 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:43.357 INFO:teuthology.orchestra.run.gibba016.stdout: mon op complaint time = 120 2021-11-21T22:02:43.358 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.358 INFO:teuthology.orchestra.run.gibba016.stdout:[client] 2021-11-21T22:02:43.358 INFO:teuthology.orchestra.run.gibba016.stdout: rgw cache enabled = true 2021-11-21T22:02:43.358 INFO:teuthology.orchestra.run.gibba016.stdout: rgw enable ops log = true 2021-11-21T22:02:43.359 INFO:teuthology.orchestra.run.gibba016.stdout: rgw enable usage log = true 2021-11-21T22:02:43.359 INFO:teuthology.orchestra.run.gibba016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:43.359 INFO:teuthology.orchestra.run.gibba016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:43.359 INFO:teuthology.orchestra.run.gibba016.stdout: client mount timeout = 600 2021-11-21T22:02:43.359 INFO:teuthology.orchestra.run.gibba016.stdout: debug client = 20 2021-11-21T22:02:43.360 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:43.360 INFO:teuthology.orchestra.run.gibba016.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.360 INFO:teuthology.orchestra.run.gibba016.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.360 INFO:teuthology.orchestra.run.gibba016.stdout: client debug getattr caps = true 2021-11-21T22:02:43.361 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.a] 2021-11-21T22:02:43.361 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.b] 2021-11-21T22:02:43.361 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.c] 2021-11-21T22:02:43.361 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.a] 2021-11-21T22:02:43.361 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.c] 2021-11-21T22:02:43.362 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.b] 2021-11-21T22:02:43.362 INFO:teuthology.orchestra.run.gibba016.stdout:[mds] 2021-11-21T22:02:43.362 INFO:teuthology.orchestra.run.gibba016.stdout: debug mds = 20 2021-11-21T22:02:43.362 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:43.363 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal frag = True 2021-11-21T22:02:43.363 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:43.363 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal merge size = 5 2021-11-21T22:02:43.363 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal split bits = 3 2021-11-21T22:02:43.363 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal split size = 100 2021-11-21T22:02:43.364 INFO:teuthology.orchestra.run.gibba016.stdout: mds debug frag = True 2021-11-21T22:02:43.364 INFO:teuthology.orchestra.run.gibba016.stdout: mds debug scatterstat = True 2021-11-21T22:02:43.364 INFO:teuthology.orchestra.run.gibba016.stdout: mds op complaint time = 180 2021-11-21T22:02:43.364 INFO:teuthology.orchestra.run.gibba016.stdout: mds verify scatter = True 2021-11-21T22:02:43.365 INFO:teuthology.orchestra.run.gibba016.stdout: osd op complaint time = 180 2021-11-21T22:02:43.365 INFO:teuthology.orchestra.run.gibba016.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.365 INFO:teuthology.orchestra.run.gibba016.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.368 INFO:teuthology.orchestra.run.gibba036.stdout:[global] 2021-11-21T22:02:43.368 INFO:teuthology.orchestra.run.gibba036.stdout: chdir = "" 2021-11-21T22:02:43.369 INFO:teuthology.orchestra.run.gibba036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:43.369 INFO:teuthology.orchestra.run.gibba036.stdout: auth supported = cephx 2021-11-21T22:02:43.369 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.369 INFO:teuthology.orchestra.run.gibba036.stdout: filestore xattr use omap = true 2021-11-21T22:02:43.370 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.370 INFO:teuthology.orchestra.run.gibba036.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:43.370 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.370 INFO:teuthology.orchestra.run.gibba036.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:43.370 INFO:teuthology.orchestra.run.gibba036.stdout: auth debug = true 2021-11-21T22:02:43.371 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.371 INFO:teuthology.orchestra.run.gibba036.stdout: ms die on old message = true 2021-11-21T22:02:43.371 INFO:teuthology.orchestra.run.gibba036.stdout: ms die on bug = true 2021-11-21T22:02:43.371 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.371 INFO:teuthology.orchestra.run.gibba036.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:43.372 INFO:teuthology.orchestra.run.gibba036.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:43.372 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.372 INFO:teuthology.orchestra.run.gibba036.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:43.372 INFO:teuthology.orchestra.run.gibba036.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:43.372 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.373 INFO:teuthology.orchestra.run.gibba036.stdout: osd pool default size = 2 2021-11-21T22:02:43.373 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.373 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:43.373 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd allow pg remap = true 2021-11-21T22:02:43.374 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:43.374 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:43.374 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:43.374 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:43.374 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on too few osds = false 2021-11-21T22:02:43.375 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:43.375 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:43.375 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.375 INFO:teuthology.orchestra.run.gibba036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:43.375 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.376 INFO:teuthology.orchestra.run.gibba036.stdout: osd default data pool replay window = 5 2021-11-21T22:02:43.376 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.376 INFO:teuthology.orchestra.run.gibba036.stdout: mon allow pool delete = true 2021-11-21T22:02:43.376 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.376 INFO:teuthology.orchestra.run.gibba036.stdout: mon cluster log file level = debug 2021-11-21T22:02:43.377 INFO:teuthology.orchestra.run.gibba036.stdout: debug asserts on shutdown = true 2021-11-21T22:02:43.377 INFO:teuthology.orchestra.run.gibba036.stdout: mon health detail to clog = false 2021-11-21T22:02:43.377 INFO:teuthology.orchestra.run.gibba036.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:43.377 INFO:teuthology.orchestra.run.gibba036.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:43.377 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.378 INFO:teuthology.orchestra.run.gibba036.stdout:[osd] 2021-11-21T22:02:43.378 INFO:teuthology.orchestra.run.gibba036.stdout: osd journal size = 100 2021-11-21T22:02:43.378 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.378 INFO:teuthology.orchestra.run.gibba036.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:43.379 INFO:teuthology.orchestra.run.gibba036.stdout: osd scrub max interval = 600 2021-11-21T22:02:43.379 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.380 INFO:teuthology.orchestra.run.gibba036.stdout: osd recover clone overlap = true 2021-11-21T22:02:43.380 INFO:teuthology.orchestra.run.gibba036.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:43.380 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.380 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug shutdown = true 2021-11-21T22:02:43.381 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug op order = true 2021-11-21T22:02:43.381 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:43.381 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.381 INFO:teuthology.orchestra.run.gibba036.stdout: osd open classes on start = true 2021-11-21T22:02:43.382 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug pg log writeout = true 2021-11-21T22:02:43.382 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.382 INFO:teuthology.orchestra.run.gibba036.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:43.382 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.383 INFO:teuthology.orchestra.run.gibba036.stdout: osd map max advance = 10 2021-11-21T22:02:43.383 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.383 INFO:teuthology.orchestra.run.gibba036.stdout: journal zero on create = true 2021-11-21T22:02:43.383 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.383 INFO:teuthology.orchestra.run.gibba036.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:43.384 INFO:teuthology.orchestra.run.gibba036.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:43.384 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.384 INFO:teuthology.orchestra.run.gibba036.stdout: bdev debug aio = true 2021-11-21T22:02:43.384 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug misdirected ops = true 2021-11-21T22:02:43.385 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:43.385 INFO:teuthology.orchestra.run.gibba036.stdout: debug osd = 20 2021-11-21T22:02:43.385 INFO:teuthology.orchestra.run.gibba036.stdout: osd objectstore = filestore 2021-11-21T22:02:43.385 INFO:teuthology.orchestra.run.gibba036.stdout: osd op complaint time = 180 2021-11-21T22:02:43.385 INFO:teuthology.orchestra.run.gibba036.stdout: osd sloppy crc = True 2021-11-21T22:02:43.386 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.386 INFO:teuthology.orchestra.run.gibba036.stdout:[mgr] 2021-11-21T22:02:43.386 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:43.386 INFO:teuthology.orchestra.run.gibba036.stdout: debug mgr = 20 2021-11-21T22:02:43.386 INFO:teuthology.orchestra.run.gibba036.stdout: debug mon = 20 2021-11-21T22:02:43.387 INFO:teuthology.orchestra.run.gibba036.stdout: debug auth = 20 2021-11-21T22:02:43.387 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.387 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.387 INFO:teuthology.orchestra.run.gibba036.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:43.388 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.388 INFO:teuthology.orchestra.run.gibba036.stdout:[mon] 2021-11-21T22:02:43.388 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:43.388 INFO:teuthology.orchestra.run.gibba036.stdout: debug mon = 20 2021-11-21T22:02:43.388 INFO:teuthology.orchestra.run.gibba036.stdout: debug paxos = 20 2021-11-21T22:02:43.389 INFO:teuthology.orchestra.run.gibba036.stdout: debug auth = 20 2021-11-21T22:02:43.389 INFO:teuthology.orchestra.run.gibba036.stdout: mon data avail warn = 5 2021-11-21T22:02:43.389 INFO:teuthology.orchestra.run.gibba036.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:43.389 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.389 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:43.390 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd prime pg temp = true 2021-11-21T22:02:43.390 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.390 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.390 INFO:teuthology.orchestra.run.gibba036.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:43.390 INFO:teuthology.orchestra.run.gibba036.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:43.391 INFO:teuthology.orchestra.run.gibba036.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:43.391 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.391 INFO:teuthology.orchestra.run.gibba036.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:43.391 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:43.392 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:43.392 INFO:teuthology.orchestra.run.gibba036.stdout: mon op complaint time = 120 2021-11-21T22:02:43.392 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:43.392 INFO:teuthology.orchestra.run.gibba036.stdout:[client] 2021-11-21T22:02:43.392 INFO:teuthology.orchestra.run.gibba036.stdout: rgw cache enabled = true 2021-11-21T22:02:43.393 INFO:teuthology.orchestra.run.gibba036.stdout: rgw enable ops log = true 2021-11-21T22:02:43.393 INFO:teuthology.orchestra.run.gibba036.stdout: rgw enable usage log = true 2021-11-21T22:02:43.393 INFO:teuthology.orchestra.run.gibba036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:43.393 INFO:teuthology.orchestra.run.gibba036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:43.393 INFO:teuthology.orchestra.run.gibba036.stdout: client mount timeout = 600 2021-11-21T22:02:43.394 INFO:teuthology.orchestra.run.gibba036.stdout: debug client = 20 2021-11-21T22:02:43.394 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:43.394 INFO:teuthology.orchestra.run.gibba036.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.394 INFO:teuthology.orchestra.run.gibba036.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.394 INFO:teuthology.orchestra.run.gibba036.stdout: client debug getattr caps = true 2021-11-21T22:02:43.395 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.a] 2021-11-21T22:02:43.395 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.b] 2021-11-21T22:02:43.395 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.c] 2021-11-21T22:02:43.395 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.a] 2021-11-21T22:02:43.395 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.c] 2021-11-21T22:02:43.396 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.b] 2021-11-21T22:02:43.396 INFO:teuthology.orchestra.run.gibba036.stdout:[mds] 2021-11-21T22:02:43.396 INFO:teuthology.orchestra.run.gibba036.stdout: debug mds = 20 2021-11-21T22:02:43.396 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:43.397 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal frag = True 2021-11-21T22:02:43.397 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:43.397 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal merge size = 5 2021-11-21T22:02:43.397 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal split bits = 3 2021-11-21T22:02:43.397 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal split size = 100 2021-11-21T22:02:43.398 INFO:teuthology.orchestra.run.gibba036.stdout: mds debug frag = True 2021-11-21T22:02:43.398 INFO:teuthology.orchestra.run.gibba036.stdout: mds debug scatterstat = True 2021-11-21T22:02:43.398 INFO:teuthology.orchestra.run.gibba036.stdout: mds op complaint time = 180 2021-11-21T22:02:43.398 INFO:teuthology.orchestra.run.gibba036.stdout: mds verify scatter = True 2021-11-21T22:02:43.399 INFO:teuthology.orchestra.run.gibba036.stdout: osd op complaint time = 180 2021-11-21T22:02:43.399 INFO:teuthology.orchestra.run.gibba036.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.399 INFO:teuthology.orchestra.run.gibba036.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.399 INFO:teuthology.orchestra.run.gibba019.stdout:[global] 2021-11-21T22:02:43.400 INFO:teuthology.orchestra.run.gibba019.stdout: chdir = "" 2021-11-21T22:02:43.400 INFO:teuthology.orchestra.run.gibba019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:43.400 INFO:teuthology.orchestra.run.gibba019.stdout: auth supported = cephx 2021-11-21T22:02:43.400 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.400 INFO:teuthology.orchestra.run.gibba019.stdout: filestore xattr use omap = true 2021-11-21T22:02:43.401 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.401 INFO:teuthology.orchestra.run.gibba019.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:43.401 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.401 INFO:teuthology.orchestra.run.gibba019.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:43.401 INFO:teuthology.orchestra.run.gibba019.stdout: auth debug = true 2021-11-21T22:02:43.402 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.402 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on old message = true 2021-11-21T22:02:43.402 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on bug = true 2021-11-21T22:02:43.402 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.402 INFO:teuthology.orchestra.run.gibba019.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:43.403 INFO:teuthology.orchestra.run.gibba019.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:43.403 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.403 INFO:teuthology.orchestra.run.gibba019.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:43.403 INFO:teuthology.orchestra.run.gibba019.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:43.404 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.404 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default size = 2 2021-11-21T22:02:43.404 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.404 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:43.404 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow pg remap = true 2021-11-21T22:02:43.405 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:43.405 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:43.405 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:43.405 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:43.405 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on too few osds = false 2021-11-21T22:02:43.406 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:43.406 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:43.406 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.406 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:43.407 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.407 INFO:teuthology.orchestra.run.gibba019.stdout: osd default data pool replay window = 5 2021-11-21T22:02:43.407 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.407 INFO:teuthology.orchestra.run.gibba019.stdout: mon allow pool delete = true 2021-11-21T22:02:43.407 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.408 INFO:teuthology.orchestra.run.gibba019.stdout: mon cluster log file level = debug 2021-11-21T22:02:43.408 INFO:teuthology.orchestra.run.gibba019.stdout: debug asserts on shutdown = true 2021-11-21T22:02:43.408 INFO:teuthology.orchestra.run.gibba019.stdout: mon health detail to clog = false 2021-11-21T22:02:43.408 INFO:teuthology.orchestra.run.gibba019.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:43.409 INFO:teuthology.orchestra.run.gibba019.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:43.409 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.409 INFO:teuthology.orchestra.run.gibba019.stdout:[osd] 2021-11-21T22:02:43.409 INFO:teuthology.orchestra.run.gibba019.stdout: osd journal size = 100 2021-11-21T22:02:43.410 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.410 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:43.410 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub max interval = 600 2021-11-21T22:02:43.410 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.411 INFO:teuthology.orchestra.run.gibba019.stdout: osd recover clone overlap = true 2021-11-21T22:02:43.411 INFO:teuthology.orchestra.run.gibba019.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:43.411 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.411 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug shutdown = true 2021-11-21T22:02:43.411 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug op order = true 2021-11-21T22:02:43.412 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:43.412 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.412 INFO:teuthology.orchestra.run.gibba019.stdout: osd open classes on start = true 2021-11-21T22:02:43.412 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug pg log writeout = true 2021-11-21T22:02:43.412 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.413 INFO:teuthology.orchestra.run.gibba019.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:43.413 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.413 INFO:teuthology.orchestra.run.gibba019.stdout: osd map max advance = 10 2021-11-21T22:02:43.413 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.413 INFO:teuthology.orchestra.run.gibba019.stdout: journal zero on create = true 2021-11-21T22:02:43.414 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.414 INFO:teuthology.orchestra.run.gibba019.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:43.414 INFO:teuthology.orchestra.run.gibba019.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:43.414 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.414 INFO:teuthology.orchestra.run.gibba019.stdout: bdev debug aio = true 2021-11-21T22:02:43.415 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug misdirected ops = true 2021-11-21T22:02:43.415 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:43.415 INFO:teuthology.orchestra.run.gibba019.stdout: debug osd = 20 2021-11-21T22:02:43.415 INFO:teuthology.orchestra.run.gibba019.stdout: osd objectstore = filestore 2021-11-21T22:02:43.416 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-11-21T22:02:43.416 INFO:teuthology.orchestra.run.gibba019.stdout: osd sloppy crc = True 2021-11-21T22:02:43.416 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.416 INFO:teuthology.orchestra.run.gibba019.stdout:[mgr] 2021-11-21T22:02:43.417 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:43.417 INFO:teuthology.orchestra.run.gibba019.stdout: debug mgr = 20 2021-11-21T22:02:43.417 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-11-21T22:02:43.417 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-11-21T22:02:43.417 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.418 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.418 INFO:teuthology.orchestra.run.gibba019.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:43.418 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.418 INFO:teuthology.orchestra.run.gibba019.stdout:[mon] 2021-11-21T22:02:43.419 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:43.419 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-11-21T22:02:43.419 INFO:teuthology.orchestra.run.gibba019.stdout: debug paxos = 20 2021-11-21T22:02:43.419 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-11-21T22:02:43.419 INFO:teuthology.orchestra.run.gibba019.stdout: mon data avail warn = 5 2021-11-21T22:02:43.420 INFO:teuthology.orchestra.run.gibba019.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:43.420 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.420 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:43.420 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd prime pg temp = true 2021-11-21T22:02:43.420 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.421 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.421 INFO:teuthology.orchestra.run.gibba019.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:43.421 INFO:teuthology.orchestra.run.gibba019.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:43.421 INFO:teuthology.orchestra.run.gibba019.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:43.422 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.422 INFO:teuthology.orchestra.run.gibba019.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:43.422 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:43.422 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:43.422 INFO:teuthology.orchestra.run.gibba019.stdout: mon op complaint time = 120 2021-11-21T22:02:43.423 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:43.423 INFO:teuthology.orchestra.run.gibba019.stdout:[client] 2021-11-21T22:02:43.423 INFO:teuthology.orchestra.run.gibba019.stdout: rgw cache enabled = true 2021-11-21T22:02:43.423 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable ops log = true 2021-11-21T22:02:43.424 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable usage log = true 2021-11-21T22:02:43.424 INFO:teuthology.orchestra.run.gibba019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:43.424 INFO:teuthology.orchestra.run.gibba019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:43.424 INFO:teuthology.orchestra.run.gibba019.stdout: client mount timeout = 600 2021-11-21T22:02:43.424 INFO:teuthology.orchestra.run.gibba019.stdout: debug client = 20 2021-11-21T22:02:43.425 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:43.425 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.425 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.425 INFO:teuthology.orchestra.run.gibba019.stdout: client debug getattr caps = true 2021-11-21T22:02:43.426 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.a] 2021-11-21T22:02:43.426 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.b] 2021-11-21T22:02:43.426 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.c] 2021-11-21T22:02:43.426 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.a] 2021-11-21T22:02:43.427 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.c] 2021-11-21T22:02:43.427 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.b] 2021-11-21T22:02:43.427 INFO:teuthology.orchestra.run.gibba019.stdout:[mds] 2021-11-21T22:02:43.427 INFO:teuthology.orchestra.run.gibba019.stdout: debug mds = 20 2021-11-21T22:02:43.428 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:43.428 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal frag = True 2021-11-21T22:02:43.428 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:43.428 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal merge size = 5 2021-11-21T22:02:43.428 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split bits = 3 2021-11-21T22:02:43.429 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split size = 100 2021-11-21T22:02:43.429 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug frag = True 2021-11-21T22:02:43.429 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug scatterstat = True 2021-11-21T22:02:43.429 INFO:teuthology.orchestra.run.gibba019.stdout: mds op complaint time = 180 2021-11-21T22:02:43.430 INFO:teuthology.orchestra.run.gibba019.stdout: mds verify scatter = True 2021-11-21T22:02:43.430 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-11-21T22:02:43.430 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.430 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.431 INFO:teuthology.orchestra.run.gibba017.stdout:[global] 2021-11-21T22:02:43.431 INFO:teuthology.orchestra.run.gibba017.stdout: chdir = "" 2021-11-21T22:02:43.431 INFO:teuthology.orchestra.run.gibba017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:43.431 INFO:teuthology.orchestra.run.gibba017.stdout: auth supported = cephx 2021-11-21T22:02:43.432 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.432 INFO:teuthology.orchestra.run.gibba017.stdout: filestore xattr use omap = true 2021-11-21T22:02:43.432 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.432 INFO:teuthology.orchestra.run.gibba017.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:43.433 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.433 INFO:teuthology.orchestra.run.gibba017.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:43.433 INFO:teuthology.orchestra.run.gibba017.stdout: auth debug = true 2021-11-21T22:02:43.433 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.433 INFO:teuthology.orchestra.run.gibba017.stdout: ms die on old message = true 2021-11-21T22:02:43.434 INFO:teuthology.orchestra.run.gibba017.stdout: ms die on bug = true 2021-11-21T22:02:43.434 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.434 INFO:teuthology.orchestra.run.gibba017.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:43.435 INFO:teuthology.orchestra.run.gibba017.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:43.435 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.435 INFO:teuthology.orchestra.run.gibba017.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:43.435 INFO:teuthology.orchestra.run.gibba017.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:43.436 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.436 INFO:teuthology.orchestra.run.gibba017.stdout: osd pool default size = 2 2021-11-21T22:02:43.436 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.436 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:43.437 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd allow pg remap = true 2021-11-21T22:02:43.437 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:43.437 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:43.437 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:43.437 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:43.438 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on too few osds = false 2021-11-21T22:02:43.438 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:43.438 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:43.438 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.438 INFO:teuthology.orchestra.run.gibba017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:43.439 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.439 INFO:teuthology.orchestra.run.gibba017.stdout: osd default data pool replay window = 5 2021-11-21T22:02:43.439 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.439 INFO:teuthology.orchestra.run.gibba017.stdout: mon allow pool delete = true 2021-11-21T22:02:43.440 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.440 INFO:teuthology.orchestra.run.gibba017.stdout: mon cluster log file level = debug 2021-11-21T22:02:43.440 INFO:teuthology.orchestra.run.gibba017.stdout: debug asserts on shutdown = true 2021-11-21T22:02:43.440 INFO:teuthology.orchestra.run.gibba017.stdout: mon health detail to clog = false 2021-11-21T22:02:43.440 INFO:teuthology.orchestra.run.gibba017.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:43.441 INFO:teuthology.orchestra.run.gibba017.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:43.441 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.441 INFO:teuthology.orchestra.run.gibba017.stdout:[osd] 2021-11-21T22:02:43.441 INFO:teuthology.orchestra.run.gibba017.stdout: osd journal size = 100 2021-11-21T22:02:43.442 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.442 INFO:teuthology.orchestra.run.gibba017.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:43.442 INFO:teuthology.orchestra.run.gibba017.stdout: osd scrub max interval = 600 2021-11-21T22:02:43.442 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.442 INFO:teuthology.orchestra.run.gibba017.stdout: osd recover clone overlap = true 2021-11-21T22:02:43.443 INFO:teuthology.orchestra.run.gibba017.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:43.443 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.443 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug shutdown = true 2021-11-21T22:02:43.443 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug op order = true 2021-11-21T22:02:43.443 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:43.444 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.444 INFO:teuthology.orchestra.run.gibba017.stdout: osd open classes on start = true 2021-11-21T22:02:43.444 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug pg log writeout = true 2021-11-21T22:02:43.444 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.445 INFO:teuthology.orchestra.run.gibba017.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:43.445 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.445 INFO:teuthology.orchestra.run.gibba017.stdout: osd map max advance = 10 2021-11-21T22:02:43.446 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.446 INFO:teuthology.orchestra.run.gibba017.stdout: journal zero on create = true 2021-11-21T22:02:43.446 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.447 INFO:teuthology.orchestra.run.gibba017.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:43.447 INFO:teuthology.orchestra.run.gibba017.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:43.447 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.447 INFO:teuthology.orchestra.run.gibba017.stdout: bdev debug aio = true 2021-11-21T22:02:43.448 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug misdirected ops = true 2021-11-21T22:02:43.448 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:43.448 INFO:teuthology.orchestra.run.gibba017.stdout: debug osd = 20 2021-11-21T22:02:43.448 INFO:teuthology.orchestra.run.gibba017.stdout: osd objectstore = filestore 2021-11-21T22:02:43.448 INFO:teuthology.orchestra.run.gibba017.stdout: osd op complaint time = 180 2021-11-21T22:02:43.449 INFO:teuthology.orchestra.run.gibba017.stdout: osd sloppy crc = True 2021-11-21T22:02:43.449 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.449 INFO:teuthology.orchestra.run.gibba017.stdout:[mgr] 2021-11-21T22:02:43.449 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:43.450 INFO:teuthology.orchestra.run.gibba017.stdout: debug mgr = 20 2021-11-21T22:02:43.450 INFO:teuthology.orchestra.run.gibba017.stdout: debug mon = 20 2021-11-21T22:02:43.451 INFO:teuthology.orchestra.run.gibba017.stdout: debug auth = 20 2021-11-21T22:02:43.451 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.451 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.451 INFO:teuthology.orchestra.run.gibba017.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:43.451 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.452 INFO:teuthology.orchestra.run.gibba017.stdout:[mon] 2021-11-21T22:02:43.452 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:43.452 INFO:teuthology.orchestra.run.gibba017.stdout: debug mon = 20 2021-11-21T22:02:43.452 INFO:teuthology.orchestra.run.gibba017.stdout: debug paxos = 20 2021-11-21T22:02:43.453 INFO:teuthology.orchestra.run.gibba017.stdout: debug auth = 20 2021-11-21T22:02:43.453 INFO:teuthology.orchestra.run.gibba017.stdout: mon data avail warn = 5 2021-11-21T22:02:43.453 INFO:teuthology.orchestra.run.gibba017.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:43.453 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:43.454 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:43.454 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd prime pg temp = true 2021-11-21T22:02:43.454 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:43.454 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.454 INFO:teuthology.orchestra.run.gibba017.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:43.455 INFO:teuthology.orchestra.run.gibba017.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:43.455 INFO:teuthology.orchestra.run.gibba017.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:43.455 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.455 INFO:teuthology.orchestra.run.gibba017.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:43.455 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:43.456 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:43.456 INFO:teuthology.orchestra.run.gibba017.stdout: mon op complaint time = 120 2021-11-21T22:02:43.456 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:43.456 INFO:teuthology.orchestra.run.gibba017.stdout:[client] 2021-11-21T22:02:43.457 INFO:teuthology.orchestra.run.gibba017.stdout: rgw cache enabled = true 2021-11-21T22:02:43.457 INFO:teuthology.orchestra.run.gibba017.stdout: rgw enable ops log = true 2021-11-21T22:02:43.457 INFO:teuthology.orchestra.run.gibba017.stdout: rgw enable usage log = true 2021-11-21T22:02:43.457 INFO:teuthology.orchestra.run.gibba017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:43.457 INFO:teuthology.orchestra.run.gibba017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:43.458 INFO:teuthology.orchestra.run.gibba017.stdout: client mount timeout = 600 2021-11-21T22:02:43.458 INFO:teuthology.orchestra.run.gibba017.stdout: debug client = 20 2021-11-21T22:02:43.458 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:43.458 INFO:teuthology.orchestra.run.gibba017.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.458 INFO:teuthology.orchestra.run.gibba017.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.459 INFO:teuthology.orchestra.run.gibba017.stdout: client debug getattr caps = true 2021-11-21T22:02:43.459 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.a] 2021-11-21T22:02:43.459 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.b] 2021-11-21T22:02:43.459 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.c] 2021-11-21T22:02:43.459 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.a] 2021-11-21T22:02:43.460 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.c] 2021-11-21T22:02:43.460 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.b] 2021-11-21T22:02:43.460 INFO:teuthology.orchestra.run.gibba017.stdout:[mds] 2021-11-21T22:02:43.460 INFO:teuthology.orchestra.run.gibba017.stdout: debug mds = 20 2021-11-21T22:02:43.460 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:43.461 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal frag = True 2021-11-21T22:02:43.461 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:43.461 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal merge size = 5 2021-11-21T22:02:43.461 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal split bits = 3 2021-11-21T22:02:43.462 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal split size = 100 2021-11-21T22:02:43.462 INFO:teuthology.orchestra.run.gibba017.stdout: mds debug frag = True 2021-11-21T22:02:43.462 INFO:teuthology.orchestra.run.gibba017.stdout: mds debug scatterstat = True 2021-11-21T22:02:43.462 INFO:teuthology.orchestra.run.gibba017.stdout: mds op complaint time = 180 2021-11-21T22:02:43.463 INFO:teuthology.orchestra.run.gibba017.stdout: mds verify scatter = True 2021-11-21T22:02:43.463 INFO:teuthology.orchestra.run.gibba017.stdout: osd op complaint time = 180 2021-11-21T22:02:43.463 INFO:teuthology.orchestra.run.gibba017.stdout: rados mon op timeout = 15m 2021-11-21T22:02:43.463 INFO:teuthology.orchestra.run.gibba017.stdout: rados osd op timeout = 15m 2021-11-21T22:02:43.464 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:43.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.530+0000 7f44c9cef700 1 Processor -- start 2021-11-21T22:02:43.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.530+0000 7f44c9cef700 1 -- start start 2021-11-21T22:02:43.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.530+0000 7f44c9cef700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f44c4065b00 0x7f44c4148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c9cef700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c9cef700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c409d080 con 0x7f44c405bf60 2021-11-21T22:02:43.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c9cef700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c40b1f40 con 0x7f44c4065b00 2021-11-21T22:02:43.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c9cef700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40ae420 con 0x7f44c4148fd0 2021-11-21T22:02:43.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c2ffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f44c4065b00 0x7f44c4148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c3fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c2ffd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f44c4065b00 0x7f44c4148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:54498/0 (socket says 172.21.2.116:54498) 2021-11-21T22:02:43.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c3fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c414b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:56676/0 (socket says 172.21.2.116:56676) 2021-11-21T22:02:43.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c2ffd700 1 -- 172.21.2.116:0/674602856 learned_addr learned my addr 172.21.2.116:0/674602856 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:43.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c37fe700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f44c405bf60 0x7f44c40b33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:35962/0 (socket says 172.21.2.116:35962) 2021-11-21T22:02:43.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c27fc700 1 -- 172.21.2.116:0/674602856 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794053881 0 0) 0x7f44c409d080 con 0x7f44c405bf60 2021-11-21T22:02:43.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c27fc700 1 -- 172.21.2.116:0/674602856 --> v1:172.21.2.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b0003260 con 0x7f44c405bf60 2021-11-21T22:02:43.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c3fff700 1 -- 172.21.2.116:0/674602856 >> v2:172.21.2.117:3300/0 conn(0x7f44c4065b00 msgr2=0x7f44c4148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:43.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c3fff700 1 --2- 172.21.2.116:0/674602856 >> v2:172.21.2.117:3300/0 conn(0x7f44c4065b00 0x7f44c4148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c3fff700 1 -- 172.21.2.116:0/674602856 >> v1:172.21.2.117:6789/0 conn(0x7f44c405bf60 legacy=0x7f44c40b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c3fff700 1 -- 172.21.2.116:0/674602856 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40b2ad0 con 0x7f44c4148fd0 2021-11-21T22:02:43.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c3fff700 1 --2- 172.21.2.116:0/674602856 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c414b3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f44b4001110 tx=0x7f44bc0091d0).ready entity=mon.2 client_cookie=f19d10beb6fc1c89 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c27fc700 1 -- 172.21.2.116:0/674602856 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44bc00b5a0 con 0x7f44c4148fd0 2021-11-21T22:02:43.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c27fc700 1 -- 172.21.2.116:0/674602856 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc00b700 con 0x7f44c4148fd0 2021-11-21T22:02:43.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.534+0000 7f44c27fc700 1 -- 172.21.2.116:0/674602856 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44bc00b8b0 con 0x7f44c4148fd0 2021-11-21T22:02:43.546 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/674602856 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 msgr2=0x7f44c414b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 --2- 172.21.2.116:0/674602856 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c414b3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f44b4001110 tx=0x7f44bc0091d0).stop 2021-11-21T22:02:43.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/674602856 shutdown_connections 2021-11-21T22:02:43.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 --2- 172.21.2.116:0/674602856 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c414b3c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 --2- 172.21.2.116:0/674602856 >> v2:172.21.2.117:3300/0 conn(0x7f44c4065b00 0x7f44c4148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.547 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/674602856 >> 172.21.2.116:0/674602856 conn(0x7f44c40b7290 msgr2=0x7f44c40b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:43.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/674602856 shutdown_connections 2021-11-21T22:02:43.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/674602856 wait complete. 2021-11-21T22:02:43.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 Processor -- start 2021-11-21T22:02:43.548 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- start start 2021-11-21T22:02:43.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44c405bf60 0x7f44c409baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44c4065b00 0x7f44c40ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c40aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.549 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40ae020 con 0x7f44c405bf60 2021-11-21T22:02:43.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40ae740 con 0x7f44c4065b00 2021-11-21T22:02:43.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40add80 con 0x7f44c4148fd0 2021-11-21T22:02:43.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44c405bf60 0x7f44c409baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44c405bf60 0x7f44c409baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53334/0 (socket says 172.21.2.116:53334) 2021-11-21T22:02:43.550 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 -- 172.21.2.116:0/877702699 learned_addr learned my addr 172.21.2.116:0/877702699 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:43.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c2ffd700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44c4065b00 0x7f44c40ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 -- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 msgr2=0x7f44c40aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c40aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.551 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 -- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44c4065b00 msgr2=0x7f44c40ac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44c4065b00 0x7f44c40ac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 -- 172.21.2.116:0/877702699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc009470 con 0x7f44c405bf60 2021-11-21T22:02:43.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c2ffd700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44c4065b00 0x7f44c40ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:43.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c37fe700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44c405bf60 0x7f44c409baa0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f44b4000ed0 tx=0x7f44b4007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c0ff9700 1 -- 172.21.2.116:0/877702699 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44b40072e0 con 0x7f44c405bf60 2021-11-21T22:02:43.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c0ff9700 1 -- 172.21.2.116:0/877702699 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4005550 con 0x7f44c405bf60 2021-11-21T22:02:43.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c40ae9a0 con 0x7f44c405bf60 2021-11-21T22:02:43.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c40af330 con 0x7f44c405bf60 2021-11-21T22:02:43.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c0ff9700 1 -- 172.21.2.116:0/877702699 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44b40058b0 con 0x7f44c405bf60 2021-11-21T22:02:43.553 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c40aeb30 con 0x7f44c405bf60 2021-11-21T22:02:43.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c0ff9700 1 -- 172.21.2.116:0/877702699 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f44b401a060 con 0x7f44c405bf60 2021-11-21T22:02:43.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.538+0000 7f44c0ff9700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44a40535d0 0x7f44a4055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.542+0000 7f44c0ff9700 1 -- 172.21.2.116:0/877702699 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f44b406b360 con 0x7f44c405bf60 2021-11-21T22:02:43.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.542+0000 7f44c2ffd700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44a40535d0 0x7f44a4055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.542+0000 7f44c2ffd700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44a40535d0 0x7f44a4055a90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f44b800f000 tx=0x7f44b800f040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.542+0000 7f44c0ff9700 1 -- 172.21.2.116:0/877702699 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44b40423b0 con 0x7f44c405bf60 2021-11-21T22:02:43.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.622+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44c409d2b0 con 0x7f44c405bf60 2021-11-21T22:02:43.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.622+0000 7f44c0ff9700 1 -- 172.21.2.116:0/877702699 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f44b4018070 con 0x7f44c405bf60 2021-11-21T22:02:43.629 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:43.629 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":141,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:37.337392+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:35.110650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:35.304135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:43.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44a40535d0 msgr2=0x7f44a4055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44a40535d0 0x7f44a4055a90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f44b800f000 tx=0x7f44b800f040).stop 2021-11-21T22:02:43.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44c405bf60 msgr2=0x7f44c409baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44c405bf60 0x7f44c409baa0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f44b4000ed0 tx=0x7f44b4007af0).stop 2021-11-21T22:02:43.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 shutdown_connections 2021-11-21T22:02:43.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44c4148fd0 0x7f44c40aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44c4065b00 0x7f44c40ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44a40535d0 0x7f44a4055a90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 --2- 172.21.2.116:0/877702699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44c405bf60 0x7f44c409baa0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 >> 172.21.2.116:0/877702699 conn(0x7f44c40b7290 msgr2=0x7f44c4143c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:43.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 shutdown_connections 2021-11-21T22:02:43.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.626+0000 7f44c9cef700 1 -- 172.21.2.116:0/877702699 wait complete. 2021-11-21T22:02:43.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:02:35.110650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:02:35.304135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:02:43.637 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:02:43.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842c46d700 1 Processor -- start 2021-11-21T22:02:43.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842c46d700 1 -- start start 2021-11-21T22:02:43.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842c46d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 0x7f84240adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842c46d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8424097c30 con 0x7f8424147dc0 2021-11-21T22:02:43.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842c46d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84240603d0 con 0x7f84240afe50 2021-11-21T22:02:43.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842c46d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8424148250 con 0x7f84240ad860 2021-11-21T22:02:43.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842aa0a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8424147dc0 0x7f84240b1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37068/0 (socket says 172.21.2.116:37068) 2021-11-21T22:02:43.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842aa0a700 1 -- 172.21.2.116:0/3036923993 learned_addr learned my addr 172.21.2.116:0/3036923993 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:43.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261088072 0 0) 0x7f8424097c30 con 0x7f8424147dc0 2021-11-21T22:02:43.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f840c003260 con 0x7f8424147dc0 2021-11-21T22:02:43.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1556537092 0 0) 0x7f840c003260 con 0x7f8424147dc0 2021-11-21T22:02:43.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 msgr2=0x7f84240adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:43.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 0x7f84240adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 >> v1:172.21.2.117:6789/0 conn(0x7f84240afe50 legacy=0x7f842405fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:43.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84241473e0 con 0x7f8424147dc0 2021-11-21T22:02:43.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f841c0019e0 con 0x7f8424147dc0 2021-11-21T22:02:43.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 >> v1:172.21.2.116:6789/0 conn(0x7f8424147dc0 legacy=0x7f84240b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c003780 0x7f840c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f840c004500 0x7f840c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 0x7f840c004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8424148250 con 0x7f840c004500 2021-11-21T22:02:43.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c00b910 con 0x7f840c003780 2021-11-21T22:02:43.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c00bad0 con 0x7f84240ad860 2021-11-21T22:02:43.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429a08700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f840c004500 0x7f840c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842a209700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c003780 0x7f840c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.708 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842aa0a700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 0x7f840c004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429a08700 1 -- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 msgr2=0x7f840c004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:43.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429a08700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 0x7f840c004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429a08700 1 -- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c003780 msgr2=0x7f840c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:43.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429a08700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c003780 0x7f840c003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429a08700 1 -- 172.21.2.116:0/3036923993 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c003260 con 0x7f840c004500 2021-11-21T22:02:43.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f842aa0a700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f84240ad860 0x7f840c004ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:43.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429a08700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f840c004500 0x7f840c004980 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f8418000f70 tx=0x7f84180079e0).ready entity=mon.0 client_cookie=bbc45eedc35a0897 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418001d80 con 0x7f840c004500 2021-11-21T22:02:43.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418014410 con 0x7f840c004500 2021-11-21T22:02:43.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.694+0000 7f8429207700 1 -- 172.21.2.116:0/3036923993 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418007cf0 con 0x7f840c004500 2021-11-21T22:02:43.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/3036923993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f840c004500 msgr2=0x7f840c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 --2- 172.21.2.116:0/3036923993 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f840c004500 0x7f840c004980 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f8418000f70 tx=0x7f84180079e0).stop 2021-11-21T22:02:43.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842aa0a700 1 -- 172.21.2.116:0/3036923993 reap_dead start 2021-11-21T22:02:43.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/3036923993 shutdown_connections 2021-11-21T22:02:43.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/3036923993 >> 172.21.2.116:0/3036923993 conn(0x7f8424062f90 msgr2=0x7f842405bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:43.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/3036923993 shutdown_connections 2021-11-21T22:02:43.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/3036923993 wait complete. 2021-11-21T22:02:43.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 Processor -- start 2021-11-21T22:02:43.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- start start 2021-11-21T22:02:43.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c004500 0x7f8424099f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84240ab2a0 0x7f84240b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f842414b2e0 0x7f84240a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842415a990 con 0x7f84240ab2a0 2021-11-21T22:02:43.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842415a450 con 0x7f840c004500 2021-11-21T22:02:43.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842415a6f0 con 0x7f842414b2e0 2021-11-21T22:02:43.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84240ab2a0 0x7f84240b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84240ab2a0 0x7f84240b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53356/0 (socket says 172.21.2.116:53356) 2021-11-21T22:02:43.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 -- 172.21.2.116:0/408388148 learned_addr learned my addr 172.21.2.116:0/408388148 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:43.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842a209700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c004500 0x7f8424099f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 -- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f842414b2e0 msgr2=0x7f84240a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f842414b2e0 0x7f84240a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 -- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c004500 msgr2=0x7f8424099f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c004500 0x7f8424099f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 -- 172.21.2.116:0/408388148 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84240b5b90 con 0x7f84240ab2a0 2021-11-21T22:02:43.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f8429a08700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84240ab2a0 0x7f84240b2c90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f8418000f70 tx=0x7f8418015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f84177fe700 1 -- 172.21.2.116:0/408388148 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84180137c0 con 0x7f84240ab2a0 2021-11-21T22:02:43.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842409a460 con 0x7f84240ab2a0 2021-11-21T22:02:43.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f84177fe700 1 -- 172.21.2.116:0/408388148 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418013920 con 0x7f84240ab2a0 2021-11-21T22:02:43.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f84177fe700 1 -- 172.21.2.116:0/408388148 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418013ca0 con 0x7f84240ab2a0 2021-11-21T22:02:43.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84240b3180 con 0x7f84240ab2a0 2021-11-21T22:02:43.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8424147680 con 0x7f84240ab2a0 2021-11-21T22:02:43.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f84177fe700 1 -- 172.21.2.116:0/408388148 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8418022470 con 0x7f84240ab2a0 2021-11-21T22:02:43.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f84177fe700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84040535d0 0x7f8404055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.698+0000 7f84177fe700 1 -- 172.21.2.116:0/408388148 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f841801c040 con 0x7f84240ab2a0 2021-11-21T22:02:43.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.702+0000 7f842a209700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84040535d0 0x7f8404055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.702+0000 7f842a209700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84040535d0 0x7f8404055a90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8420008000 tx=0x7f8420008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.702+0000 7f84177fe700 1 -- 172.21.2.116:0/408388148 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8418043f30 con 0x7f84240ab2a0 2021-11-21T22:02:43.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f84240573d0 con 0x7f84240ab2a0 2021-11-21T22:02:43.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f84177fe700 1 -- 172.21.2.116:0/408388148 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8418047510 con 0x7f84240ab2a0 2021-11-21T22:02:43.790 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:02:43.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84040535d0 msgr2=0x7f8404055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.791 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84040535d0 0x7f8404055a90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8420008000 tx=0x7f8420008040).stop 2021-11-21T22:02:43.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84240ab2a0 msgr2=0x7f84240b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84240ab2a0 0x7f84240b2c90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f8418000f70 tx=0x7f8418015000).stop 2021-11-21T22:02:43.792 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 shutdown_connections 2021-11-21T22:02:43.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f842414b2e0 0x7f84240a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f84040535d0 0x7f8404055a90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f84240ab2a0 0x7f84240b2c90 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.793 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 --2- 172.21.2.116:0/408388148 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f840c004500 0x7f8424099f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 >> 172.21.2.116:0/408388148 conn(0x7f8424062f90 msgr2=0x7f84240b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:43.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 shutdown_connections 2021-11-21T22:02:43.794 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.786+0000 7f842c46d700 1 -- 172.21.2.116:0/408388148 wait complete. 2021-11-21T22:02:43.798 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:02:43.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 Processor -- start 2021-11-21T22:02:43.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- start start 2021-11-21T22:02:43.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f17e0065b00 0x7f17e00b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e0063550 con 0x7f17e005c100 2021-11-21T22:02:43.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e00b26b0 con 0x7f17e014ab00 2021-11-21T22:02:43.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e0067b10 con 0x7f17e0065b00 2021-11-21T22:02:43.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17dffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f17e0065b00 0x7f17e00b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17dffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f17e0065b00 0x7f17e00b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53364/0 (socket says 172.21.2.116:53364) 2021-11-21T22:02:43.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17dffff700 1 -- 172.21.2.116:0/4039207557 learned_addr learned my addr 172.21.2.116:0/4039207557 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:43.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17dffff700 1 -- 172.21.2.116:0/4039207557 >> v1:172.21.2.117:6789/0 conn(0x7f17e014ab00 legacy=0x7f17e014cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:43.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17dffff700 1 -- 172.21.2.116:0/4039207557 >> v1:172.21.2.116:6789/0 conn(0x7f17e005c100 legacy=0x7f17e00b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17dffff700 1 -- 172.21.2.116:0/4039207557 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e005c4e0 con 0x7f17e0065b00 2021-11-21T22:02:43.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17dffff700 1 --2- 172.21.2.116:0/4039207557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00b31f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f17c8001110 tx=0x7f17d0006eb0).ready entity=mon.0 client_cookie=5020972d56c95471 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17df7fe700 1 -- 172.21.2.116:0/4039207557 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17d000d5c0 con 0x7f17e0065b00 2021-11-21T22:02:43.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17df7fe700 1 -- 172.21.2.116:0/4039207557 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d0000e90 con 0x7f17e0065b00 2021-11-21T22:02:43.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17df7fe700 1 -- 172.21.2.116:0/4039207557 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17d000d950 con 0x7f17e0065b00 2021-11-21T22:02:43.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- 172.21.2.116:0/4039207557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 msgr2=0x7f17e00b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 --2- 172.21.2.116:0/4039207557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00b31f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f17c8001110 tx=0x7f17d0006eb0).stop 2021-11-21T22:02:43.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- 172.21.2.116:0/4039207557 shutdown_connections 2021-11-21T22:02:43.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 --2- 172.21.2.116:0/4039207557 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00b31f0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- 172.21.2.116:0/4039207557 >> 172.21.2.116:0/4039207557 conn(0x7f17e00b7290 msgr2=0x7f17e00b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:43.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- 172.21.2.116:0/4039207557 shutdown_connections 2021-11-21T22:02:43.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.846+0000 7f17e6a93700 1 -- 172.21.2.116:0/4039207557 wait complete. 2021-11-21T22:02:43.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 Processor -- start 2021-11-21T22:02:43.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 -- start start 2021-11-21T22:02:43.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17e005c100 0x7f17e00aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17e014ab00 0x7f17e00adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00af400 con 0x7f17e0065b00 2021-11-21T22:02:43.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00ae090 con 0x7f17e005c100 2021-11-21T22:02:43.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00abf10 con 0x7f17e014ab00 2021-11-21T22:02:43.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53368/0 (socket says 172.21.2.116:53368) 2021-11-21T22:02:43.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 -- 172.21.2.116:0/494696017 learned_addr learned my addr 172.21.2.116:0/494696017 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:43.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 -- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17e014ab00 msgr2=0x7f17e00adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17e014ab00 0x7f17e00adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 -- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17e005c100 msgr2=0x7f17e00aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17e005c100 0x7f17e00aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 -- 172.21.2.116:0/494696017 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d000b1e0 con 0x7f17e0065b00 2021-11-21T22:02:43.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17dffff700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00ad6b0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f17c8001110 tx=0x7f17d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17ddffb700 1 -- 172.21.2.116:0/494696017 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17d000de90 con 0x7f17e0065b00 2021-11-21T22:02:43.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17ddffb700 1 -- 172.21.2.116:0/494696017 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d0002f70 con 0x7f17e0065b00 2021-11-21T22:02:43.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e00b14d0 con 0x7f17e0065b00 2021-11-21T22:02:43.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17ddffb700 1 -- 172.21.2.116:0/494696017 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17d001e460 con 0x7f17e0065b00 2021-11-21T22:02:43.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e00ac310 con 0x7f17e0065b00 2021-11-21T22:02:43.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e00b1300 con 0x7f17e0065b00 2021-11-21T22:02:43.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17ddffb700 1 -- 172.21.2.116:0/494696017 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f17d0025030 con 0x7f17e0065b00 2021-11-21T22:02:43.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17ddffb700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f17c40534f0 0x7f17c40559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:43.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17ddffb700 1 -- 172.21.2.116:0/494696017 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f17d006cd20 con 0x7f17e0065b00 2021-11-21T22:02:43.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.850+0000 7f17e482f700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f17c40534f0 0x7f17c40559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:43.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.854+0000 7f17e482f700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f17c40534f0 0x7f17c40559b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f17c80011d0 tx=0x7f17c8007770).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:43.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.854+0000 7f17ddffb700 1 -- 172.21.2.116:0/494696017 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f17d0043e80 con 0x7f17e0065b00 2021-11-21T22:02:43.939 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.934+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f17e00b1db0 con 0x7f17e0065b00 2021-11-21T22:02:43.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17ddffb700 1 -- 172.21.2.116:0/494696017 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7f17d0023070 con 0x7f17e0065b00 2021-11-21T22:02:43.941 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:43.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f17c40534f0 msgr2=0x7f17c40559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f17c40534f0 0x7f17c40559b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f17c80011d0 tx=0x7f17c8007770).stop 2021-11-21T22:02:43.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 msgr2=0x7f17e00ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:43.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00ad6b0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f17c8001110 tx=0x7f17d0015000).stop 2021-11-21T22:02:43.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 shutdown_connections 2021-11-21T22:02:43.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f17e014ab00 0x7f17e00adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f17c40534f0 0x7f17c40559b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f17e0065b00 0x7f17e00ad6b0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:43.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 --2- 172.21.2.116:0/494696017 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f17e005c100 0x7f17e00aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:43.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 >> 172.21.2.116:0/494696017 conn(0x7f17e00b7290 msgr2=0x7f17e00b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:43.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 shutdown_connections 2021-11-21T22:02:43.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.938+0000 7f17e6a93700 1 -- 172.21.2.116:0/494696017 wait complete. 2021-11-21T22:02:43.948 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:02:44.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 Processor -- start 2021-11-21T22:02:44.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- start start 2021-11-21T22:02:44.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f08840af570 0x7f08840b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840b5bf0 con 0x7f08840b5810 2021-11-21T22:02:44.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840ae740 con 0x7f0884068320 2021-11-21T22:02:44.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08840ae9e0 con 0x7f08840af570 2021-11-21T22:02:44.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f08840b5810 0x7f08840b0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37098/0 (socket says 172.21.2.116:37098) 2021-11-21T22:02:44.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 -- 172.21.2.116:0/2089707729 learned_addr learned my addr 172.21.2.116:0/2089707729 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:44.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181586597 0 0) 0x7f08840b5bf0 con 0x7f08840b5810 2021-11-21T22:02:44.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0864003260 con 0x7f08840b5810 2021-11-21T22:02:44.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f0882ffd700 1 --2- 172.21.2.116:0/2089707729 >> v2:172.21.2.117:3300/0 conn(0x7f08840af570 0x7f08840b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3942801874 0 0) 0x7f0864003260 con 0x7f08840b5810 2021-11-21T22:02:44.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 >> v2:172.21.2.117:3300/0 conn(0x7f08840af570 msgr2=0x7f08840b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:44.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 --2- 172.21.2.116:0/2089707729 >> v2:172.21.2.117:3300/0 conn(0x7f08840af570 0x7f08840b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 >> v1:172.21.2.117:6789/0 conn(0x7f0884068320 legacy=0x7f088413ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:44.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840aec80 con 0x7f08840b5810 2021-11-21T22:02:44.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f08740016b0 con 0x7f08840b5810 2021-11-21T22:02:44.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 >> v1:172.21.2.116:6789/0 conn(0x7f08840b5810 legacy=0x7f08840b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0864003820 0x7f0864003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0864004500 0x7f0864004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864004f70 0x7f086400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08840ae9e0 con 0x7f0864003820 2021-11-21T22:02:44.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086400be50 con 0x7f0864004f70 2021-11-21T22:02:44.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086400c010 con 0x7f0864004500 2021-11-21T22:02:44.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0864003820 0x7f0864003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 -- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0864004500 msgr2=0x7f0864004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:44.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0864004500 0x7f0864004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:44.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f0883fff700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864004f70 0x7f086400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 -- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864004f70 msgr2=0x7f086400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:44.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864004f70 0x7f086400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.015 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 -- 172.21.2.116:0/2089707729 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0864003260 con 0x7f0864003820 2021-11-21T22:02:44.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f0883fff700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864004f70 0x7f086400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:44.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f0883fff700 1 -- 172.21.2.116:0/2089707729 reap_dead start 2021-11-21T22:02:44.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08837fe700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0864003820 0x7f0864003c20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f08840b61a0 tx=0x7f08840aee10).ready entity=mon.0 client_cookie=f143a46ddab59b69 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08740070d0 con 0x7f0864003820 2021-11-21T22:02:44.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874007230 con 0x7f0864003820 2021-11-21T22:02:44.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f08827fc700 1 -- 172.21.2.116:0/2089707729 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f087400c8f0 con 0x7f0864003820 2021-11-21T22:02:44.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- 172.21.2.116:0/2089707729 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0864003820 msgr2=0x7f0864003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0864003820 0x7f0864003c20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f08840b61a0 tx=0x7f08840aee10).stop 2021-11-21T22:02:44.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- 172.21.2.116:0/2089707729 shutdown_connections 2021-11-21T22:02:44.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 --2- 172.21.2.116:0/2089707729 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0864003820 0x7f0864003c20 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- 172.21.2.116:0/2089707729 >> 172.21.2.116:0/2089707729 conn(0x7f088405fc40 msgr2=0x7f0884060020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:44.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- 172.21.2.116:0/2089707729 shutdown_connections 2021-11-21T22:02:44.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 -- 172.21.2.116:0/2089707729 wait complete. 2021-11-21T22:02:44.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:43.998+0000 7f088a12c700 1 Processor -- start 2021-11-21T22:02:44.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 -- start start 2021-11-21T22:02:44.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864003820 0x7f08840b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0884141e60 0x7f0884140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f088412ce20 0x7f0884140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884142840 con 0x7f088412ce20 2021-11-21T22:02:44.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088413f4d0 con 0x7f0864003820 2021-11-21T22:02:44.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088413f770 con 0x7f0884141e60 2021-11-21T22:02:44.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f088412ce20 0x7f0884140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f088412ce20 0x7f0884140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53386/0 (socket says 172.21.2.116:53386) 2021-11-21T22:02:44.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 -- 172.21.2.116:0/3286544083 learned_addr learned my addr 172.21.2.116:0/3286544083 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:44.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f08837fe700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864003820 0x7f08840b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0882ffd700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0884141e60 0x7f0884140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 -- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0884141e60 msgr2=0x7f0884140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0884141e60 0x7f0884140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 -- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864003820 msgr2=0x7f08840b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864003820 0x7f08840b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 -- 172.21.2.116:0/3286544083 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088413fa10 con 0x7f088412ce20 2021-11-21T22:02:44.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0883fff700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f088412ce20 0x7f0884140a80 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f0884060920 tx=0x7f087c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0880ff9700 1 -- 172.21.2.116:0/3286544083 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f087c00de50 con 0x7f088412ce20 2021-11-21T22:02:44.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0880ff9700 1 -- 172.21.2.116:0/3286544083 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c0104e0 con 0x7f088412ce20 2021-11-21T22:02:44.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0880ff9700 1 -- 172.21.2.116:0/3286544083 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f087c010840 con 0x7f088412ce20 2021-11-21T22:02:44.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08841400f0 con 0x7f088412ce20 2021-11-21T22:02:44.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0884064830 con 0x7f088412ce20 2021-11-21T22:02:44.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0884140280 con 0x7f088412ce20 2021-11-21T22:02:44.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0880ff9700 1 -- 172.21.2.116:0/3286544083 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f087c01a030 con 0x7f088412ce20 2021-11-21T22:02:44.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0880ff9700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f08680535b0 0x7f0868055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f0880ff9700 1 -- 172.21.2.116:0/3286544083 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f087c06b270 con 0x7f088412ce20 2021-11-21T22:02:44.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f08837fe700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f08680535b0 0x7f0868055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.002+0000 7f08837fe700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f08680535b0 0x7f0868055a70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f0874016000 tx=0x7f0874013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.006+0000 7f0880ff9700 1 -- 172.21.2.116:0/3286544083 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f087c0423d0 con 0x7f088412ce20 2021-11-21T22:02:44.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08840aef20 con 0x7f088412ce20 2021-11-21T22:02:44.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f0880ff9700 1 -- 172.21.2.116:0/3286544083 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+10 (secure 0 0 0) 0x7f087c0459f0 con 0x7f088412ce20 2021-11-21T22:02:44.094 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:44.095 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f08680535b0 msgr2=0x7f0868055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f08680535b0 0x7f0868055a70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f0874016000 tx=0x7f0874013000).stop 2021-11-21T22:02:44.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f088412ce20 msgr2=0x7f0884140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f088412ce20 0x7f0884140a80 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f0884060920 tx=0x7f087c009040).stop 2021-11-21T22:02:44.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 shutdown_connections 2021-11-21T22:02:44.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f08680535b0 0x7f0868055a70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f088412ce20 0x7f0884140a80 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0884141e60 0x7f0884140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 --2- 172.21.2.116:0/3286544083 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0864003820 0x7f08840b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 >> 172.21.2.116:0/3286544083 conn(0x7f088405fc40 msgr2=0x7f08840b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:44.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 shutdown_connections 2021-11-21T22:02:44.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.090+0000 7f088a12c700 1 -- 172.21.2.116:0/3286544083 wait complete. 2021-11-21T22:02:44.101 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:02:44.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 Processor -- start 2021-11-21T22:02:44.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- start start 2021-11-21T22:02:44.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f42080ad860 0x7f42080adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f42080afe50 0x7f4208147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4208097c30 con 0x7f420805fcc0 2021-11-21T22:02:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42081473e0 con 0x7f42080afe50 2021-11-21T22:02:44.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4208147680 con 0x7f42080ad860 2021-11-21T22:02:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f42080afe50 0x7f4208147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f42080afe50 0x7f4208147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53394/0 (socket says 172.21.2.116:53394) 2021-11-21T22:02:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 -- 172.21.2.116:0/725282865 learned_addr learned my addr 172.21.2.116:0/725282865 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:44.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f4207fff700 1 --2- 172.21.2.116:0/725282865 >> v2:172.21.2.117:3300/0 conn(0x7f42080ad860 0x7f42080adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 -- 172.21.2.116:0/725282865 >> v2:172.21.2.117:3300/0 conn(0x7f42080ad860 msgr2=0x7f42080adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 --2- 172.21.2.116:0/725282865 >> v2:172.21.2.117:3300/0 conn(0x7f42080ad860 0x7f42080adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 -- 172.21.2.116:0/725282865 >> v1:172.21.2.117:6789/0 conn(0x7f420805fcc0 legacy=0x7f42080b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:44.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 -- 172.21.2.116:0/725282865 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208147920 con 0x7f42080afe50 2021-11-21T22:02:44.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f42077fe700 1 --2- 172.21.2.116:0/725282865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f42080afe50 0x7f4208147dc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f42080af920 tx=0x7f41f0006eb0).ready entity=mon.0 client_cookie=61a9400ad6b38ef4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f4206ffd700 1 -- 172.21.2.116:0/725282865 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41f000d5c0 con 0x7f42080afe50 2021-11-21T22:02:44.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f4206ffd700 1 -- 172.21.2.116:0/725282865 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0000e90 con 0x7f42080afe50 2021-11-21T22:02:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f4206ffd700 1 -- 172.21.2.116:0/725282865 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41f000d910 con 0x7f42080afe50 2021-11-21T22:02:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- 172.21.2.116:0/725282865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f42080afe50 msgr2=0x7f4208147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 --2- 172.21.2.116:0/725282865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f42080afe50 0x7f4208147dc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f42080af920 tx=0x7f41f0006eb0).stop 2021-11-21T22:02:44.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- 172.21.2.116:0/725282865 shutdown_connections 2021-11-21T22:02:44.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 --2- 172.21.2.116:0/725282865 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f42080afe50 0x7f4208147dc0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 --2- 172.21.2.116:0/725282865 >> v2:172.21.2.117:3300/0 conn(0x7f42080ad860 0x7f42080adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- 172.21.2.116:0/725282865 >> 172.21.2.116:0/725282865 conn(0x7f4208062f90 msgr2=0x7f420805bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:44.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- 172.21.2.116:0/725282865 shutdown_connections 2021-11-21T22:02:44.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.150+0000 7f420e4b4700 1 -- 172.21.2.116:0/725282865 wait complete. 2021-11-21T22:02:44.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 Processor -- start 2021-11-21T22:02:44.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 -- start start 2021-11-21T22:02:44.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f420805fcc0 0x7f420814a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f42080ad860 0x7f42080b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f42080afe50 0x7f420814ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420809aa00 con 0x7f420805fcc0 2021-11-21T22:02:44.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420809aca0 con 0x7f42080afe50 2021-11-21T22:02:44.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42080997f0 con 0x7f42080ad860 2021-11-21T22:02:44.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f420805fcc0 0x7f420814a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f420805fcc0 0x7f420814a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53400/0 (socket says 172.21.2.116:53400) 2021-11-21T22:02:44.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 -- 172.21.2.116:0/718995930 learned_addr learned my addr 172.21.2.116:0/718995930 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:44.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 -- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f42080ad860 msgr2=0x7f42080b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420ca51700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f42080afe50 0x7f420814ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f42080ad860 0x7f42080b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:44.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 -- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f42080afe50 msgr2=0x7f420814ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f42080afe50 0x7f420814ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 -- 172.21.2.116:0/718995930 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f000b1e0 con 0x7f420805fcc0 2021-11-21T22:02:44.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420ca51700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f42080afe50 0x7f420814ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:44.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f4207fff700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f420805fcc0 0x7f420814a100 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f41f800a000 tx=0x7f41f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f42057fa700 1 -- 172.21.2.116:0/718995930 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41f8011780 con 0x7f420805fcc0 2021-11-21T22:02:44.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f42057fa700 1 -- 172.21.2.116:0/718995930 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f80118e0 con 0x7f420805fcc0 2021-11-21T22:02:44.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f42057fa700 1 -- 172.21.2.116:0/718995930 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41f8011c40 con 0x7f420805fcc0 2021-11-21T22:02:44.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4208099a90 con 0x7f420805fcc0 2021-11-21T22:02:44.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420814b410 con 0x7f420805fcc0 2021-11-21T22:02:44.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42080b1f20 con 0x7f420805fcc0 2021-11-21T22:02:44.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f42057fa700 1 -- 172.21.2.116:0/718995930 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f41f8012420 con 0x7f420805fcc0 2021-11-21T22:02:44.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f42057fa700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f41ec053610 0x7f41ec055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f42057fa700 1 -- 172.21.2.116:0/718995930 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f41f806b880 con 0x7f420805fcc0 2021-11-21T22:02:44.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.154+0000 7f42077fe700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f41ec053610 0x7f41ec055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.158+0000 7f42077fe700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f41ec053610 0x7f41ec055ad0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f42080af920 tx=0x7f41f0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.158+0000 7f42057fa700 1 -- 172.21.2.116:0/718995930 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41f8042980 con 0x7f420805fcc0 2021-11-21T22:02:44.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.238+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42080573d0 con 0x7f420805fcc0 2021-11-21T22:02:44.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.238+0000 7f42057fa700 1 -- 172.21.2.116:0/718995930 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+10 (secure 0 0 0) 0x7f41f8045fa0 con 0x7f420805fcc0 2021-11-21T22:02:44.245 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:44.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f41ec053610 msgr2=0x7f41ec055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f41ec053610 0x7f41ec055ad0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f42080af920 tx=0x7f41f0015000).stop 2021-11-21T22:02:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f420805fcc0 msgr2=0x7f420814a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f420805fcc0 0x7f420814a100 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f41f800a000 tx=0x7f41f8008040).stop 2021-11-21T22:02:44.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 shutdown_connections 2021-11-21T22:02:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f42080afe50 0x7f420814ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f42080ad860 0x7f42080b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f41ec053610 0x7f41ec055ad0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 --2- 172.21.2.116:0/718995930 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f420805fcc0 0x7f420814a100 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 >> 172.21.2.116:0/718995930 conn(0x7f4208062f90 msgr2=0x7f420814db50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:44.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 shutdown_connections 2021-11-21T22:02:44.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.242+0000 7f420e4b4700 1 -- 172.21.2.116:0/718995930 wait complete. 2021-11-21T22:02:44.252 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:02:44.252 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:44.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.298+0000 7f817832b700 1 Processor -- start 2021-11-21T22:02:44.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.298+0000 7f817832b700 1 -- start start 2021-11-21T22:02:44.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f81700aa040 0x7f81700aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 0x7f81701446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81700aaae0 con 0x7f8170144ba0 2021-11-21T22:02:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81701480f0 con 0x7f81700aa040 2021-11-21T22:02:44.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8170147e50 con 0x7f81700b11e0 2021-11-21T22:02:44.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81758c6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 0x7f81701446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f81700aa040 0x7f81700aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f81700aa040 0x7f81700aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:54576/0 (socket says 172.21.2.116:54576) 2021-11-21T22:02:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 -- 172.21.2.116:0/4079804705 learned_addr learned my addr 172.21.2.116:0/4079804705 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81768c8700 1 --1- 172.21.2.116:0/4079804705 >> v1:172.21.2.117:6789/0 conn(0x7f8170144ba0 0x7f817005f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:36044/0 (socket says 172.21.2.116:36044) 2021-11-21T22:02:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 -- 172.21.2.116:0/4079804705 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 msgr2=0x7f81701446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 --2- 172.21.2.116:0/4079804705 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 0x7f81701446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 -- 172.21.2.116:0/4079804705 >> v1:172.21.2.117:6789/0 conn(0x7f8170144ba0 legacy=0x7f817005f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 -- 172.21.2.116:0/4079804705 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170147c20 con 0x7f81700aa040 2021-11-21T22:02:44.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81760c7700 1 --2- 172.21.2.116:0/4079804705 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81700aa040 0x7f81700aa420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f8170147130 tx=0x7f816c006eb0).ready entity=mon.1 client_cookie=a4c9d13fe86cb113 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81750c5700 1 -- 172.21.2.116:0/4079804705 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816c00d5c0 con 0x7f81700aa040 2021-11-21T22:02:44.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81750c5700 1 -- 172.21.2.116:0/4079804705 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c000e90 con 0x7f81700aa040 2021-11-21T22:02:44.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f81750c5700 1 -- 172.21.2.116:0/4079804705 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816c00d950 con 0x7f81700aa040 2021-11-21T22:02:44.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- 172.21.2.116:0/4079804705 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81700aa040 msgr2=0x7f81700aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 --2- 172.21.2.116:0/4079804705 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81700aa040 0x7f81700aa420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f8170147130 tx=0x7f816c006eb0).stop 2021-11-21T22:02:44.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- 172.21.2.116:0/4079804705 shutdown_connections 2021-11-21T22:02:44.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 --2- 172.21.2.116:0/4079804705 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 0x7f81701446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 --2- 172.21.2.116:0/4079804705 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81700aa040 0x7f81700aa420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- 172.21.2.116:0/4079804705 >> 172.21.2.116:0/4079804705 conn(0x7f81700b5060 msgr2=0x7f81700b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:44.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- 172.21.2.116:0/4079804705 shutdown_connections 2021-11-21T22:02:44.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.302+0000 7f817832b700 1 -- 172.21.2.116:0/4079804705 wait complete. 2021-11-21T22:02:44.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 Processor -- start 2021-11-21T22:02:44.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 -- start start 2021-11-21T22:02:44.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81700aa040 0x7f81700ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 0x7f8170153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8170144ba0 0x7f8170153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170098470 con 0x7f81700aa040 2021-11-21T22:02:44.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81700ad350 con 0x7f8170144ba0 2021-11-21T22:02:44.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8170154140 con 0x7f81700b11e0 2021-11-21T22:02:44.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81700aa040 0x7f81700ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81700aa040 0x7f81700ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53416/0 (socket says 172.21.2.116:53416) 2021-11-21T22:02:44.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 -- 172.21.2.116:0/1870695648 learned_addr learned my addr 172.21.2.116:0/1870695648 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:44.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 -- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 msgr2=0x7f8170153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 0x7f8170153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:44.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 -- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8170144ba0 msgr2=0x7f8170153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8170144ba0 0x7f8170153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:44.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 -- 172.21.2.116:0/1870695648 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c00b1e0 con 0x7f81700aa040 2021-11-21T22:02:44.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81760c7700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81700aa040 0x7f81700ae890 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f8170147130 tx=0x7f816c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81637fe700 1 -- 172.21.2.116:0/1870695648 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816c011990 con 0x7f81700aa040 2021-11-21T22:02:44.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81637fe700 1 -- 172.21.2.116:0/1870695648 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c011af0 con 0x7f81700aa040 2021-11-21T22:02:44.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81637fe700 1 -- 172.21.2.116:0/1870695648 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816c011e50 con 0x7f81700aa040 2021-11-21T22:02:44.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81700980f0 con 0x7f81700aa040 2021-11-21T22:02:44.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81700adb00 con 0x7f81700aa040 2021-11-21T22:02:44.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81700aed80 con 0x7f81700aa040 2021-11-21T22:02:44.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81637fe700 1 -- 172.21.2.116:0/1870695648 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f816c025030 con 0x7f81700aa040 2021-11-21T22:02:44.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81637fe700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f81500534f0 0x7f81500559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:44.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81637fe700 1 -- 172.21.2.116:0/1870695648 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f816c06cd50 con 0x7f81700aa040 2021-11-21T22:02:44.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.306+0000 7f81758c6700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f81500534f0 0x7f81500559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:44.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.310+0000 7f81758c6700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f81500534f0 0x7f81500559b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f816400c000 tx=0x7f8164006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:44.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.310+0000 7f81637fe700 1 -- 172.21.2.116:0/1870695648 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f816c043d20 con 0x7f81700aa040 2021-11-21T22:02:44.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.418+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81700b0080 con 0x7f81700aa040 2021-11-21T22:02:44.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.418+0000 7f81637fe700 1 -- 172.21.2.116:0/1870695648 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f816c047340 con 0x7f81700aa040 2021-11-21T22:02:44.425 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:44.425 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:36.333834+0000","modified":"2021-11-21T22:02:37.338971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/1832322648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1832322648},{"type":"v1","addr":"172.21.2.116:6835","nonce":1832322648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-11-21T22:02:44.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f81500534f0 msgr2=0x7f81500559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f81500534f0 0x7f81500559b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f816400c000 tx=0x7f8164006df0).stop 2021-11-21T22:02:44.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81700aa040 msgr2=0x7f81700ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:44.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81700aa040 0x7f81700ae890 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f8170147130 tx=0x7f816c015000).stop 2021-11-21T22:02:44.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 shutdown_connections 2021-11-21T22:02:44.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8170144ba0 0x7f8170153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:44.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81700b11e0 0x7f8170153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:44.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f81500534f0 0x7f81500559b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 --2- 172.21.2.116:0/1870695648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81700aa040 0x7f81700ae890 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:44.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 >> 172.21.2.116:0/1870695648 conn(0x7f81700b5060 msgr2=0x7f81701440e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:44.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 shutdown_connections 2021-11-21T22:02:44.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:44.422+0000 7f817832b700 1 -- 172.21.2.116:0/1870695648 wait complete. 2021-11-21T22:02:44.429 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 65 2021-11-21T22:02:44.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:36.333834+0000', 'modified': '2021-11-21T22:02:37.338971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8109': {'gid': 8109, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.116:6835/1832322648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 1832322648}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 1832322648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:02:44.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:02:44.735 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-11-21T22:02:44.800 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8109,"want_state":"up:active","state":"up:active","rank_uptime":8.4585219649999992,"mdsmap_epoch":65,"osdmap_epoch":141,"osdmap_epoch_barrier":141,"uptime":11.704508371999999} 2021-11-21T22:02:44.800 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8109, "mdsmap_epoch": 65, "osdmap_epoch": 141, "osdmap_epoch_barrier": 141, "rank_uptime": 8.458521965, "state": "up:active", "uptime": 11.704508372, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:02:44.801 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:02:44.801 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:02:44.802 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:02:44.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:44.803 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:44.813 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:02:44.825 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:44.826 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:44.875 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:44.876 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:44.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:44.878 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:02:44.884 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:02:44.884 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:44.887 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:02:44.897 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:02:44.898 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:02:44.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:02:44.926+0000 7fa3aeaab080 -1 init, newargv = 0x55f5bcd0ad90 newargc=9ceph-fuse[14682]: starting ceph client 2021-11-21T22:02:44.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr: 2021-11-21T22:02:44.933 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:02:44.934 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:02:44.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:44.935 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:02:44.937 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14682]: starting fuse 2021-11-21T22:02:44.940 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:02:44.940 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:02:44.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:44.941 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:02:44.942 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:02:44.942 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:02:44.942 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:02:44.943 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:02:44.943 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:44.944 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:02:44.944 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:02:44.945 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:02:44.945 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:44.946 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:02:44.946 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:02:44.946 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:02:44.947 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:02:44.947 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:44.948 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:02:44.948 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:02:44.949 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:02:44.949 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:02:44.950 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:02:44.950 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:02:44.950 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:02:44.951 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:02:45.010 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14682.asok 2021-11-21T22:02:45.012 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14682.asok 2021-11-21T22:02:45.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:45.013 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14682.asok status 2021-11-21T22:02:45.062 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:02:45.062 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:02:45.062 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:02:45.062 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:02:45.063 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:02:45.063 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:02:45.063 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:02:45.063 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14682", 2021-11-21T22:02:45.064 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:02:45.064 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:45.064 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:02:45.064 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:02:45.064 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 5329, 2021-11-21T22:02:45.065 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:02:45.065 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:02:45.065 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:02:45.065 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 5329 2021-11-21T22:02:45.066 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:45.066 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:02:45.066 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:02:45.066 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:02:45.067 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 3277363880 2021-11-21T22:02:45.067 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:02:45.067 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:45.067 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:02:45.067 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:02:45.068 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:02:45.068 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 3277363880 2021-11-21T22:02:45.068 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:02:45.069 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.5329 172.21.2.136:0/3277363880", 2021-11-21T22:02:45.069 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/3277363880", 2021-11-21T22:02:45.069 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:02:45.070 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 65, 2021-11-21T22:02:45.070 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 141, 2021-11-21T22:02:45.070 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:02:45.071 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:02:45.071 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:02:45.072 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:02:45.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:45.074 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:45.074 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:45.081 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:45.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:45.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:45.082 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:45.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:45.133 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:45.134 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:45.145 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:02:45.146 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:45.147 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-11-21T22:02:45.147 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-11-21T22:02:45.148 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:02:45.148 DEBUG:teuthology.orchestra.run.gibba019:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2021-11-21T22:02:45.148 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-11-21T22:02:45.149 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-11-21T22:02:45.149 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content2'"'"') 2021-11-21T22:02:45.150 DEBUG:teuthology.orchestra.run.gibba019:> while True: 2021-11-21T22:02:45.150 DEBUG:teuthology.orchestra.run.gibba019:> time.sleep(1) 2021-11-21T22:02:45.150 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:02:45.192 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2021-11-21T22:02:45.205 INFO:teuthology.orchestra.run.gibba019.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2021-11-21T22:02:45.205 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-11-21T22:02:46.208 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2021-11-21T22:02:46.219 INFO:teuthology.orchestra.run.gibba019.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2021-11-21T22:02:46.221 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2021-11-21T22:02:46.221 DEBUG:teuthology.orchestra.run.gibba036:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2021-11-21T22:02:46.264 INFO:teuthology.orchestra.run.gibba036.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2021-11-21T22:02:46.265 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2021-11-21T22:02:46.266 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-11-21T22:02:46.266 DEBUG:teuthology.orchestra.run.gibba036:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2021-11-21T22:02:46.370 INFO:teuthology.orchestra.run.gibba036.stdout:content 2021-11-21T22:02:46.382 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:02:46.383 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:46.397 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:46.413 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:46.436 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:46.466 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:02:46.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:46.467 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:46.467 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:46.478 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:02:46.480 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:46.480 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:02:46.480 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:02:46.481 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:46.534 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15164]: fuse finished with error 0 and tester_r 0 2021-11-21T22:02:46.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:46.536 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:46.536 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:46.540 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:02:46.540 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:02:46.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:02:46.663 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:46.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:46.715 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:46.715 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:46.727 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:02:46.728 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:46.729 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:02:46.729 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:02:46.730 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:46.784 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14682]: fuse finished with error 0 and tester_r 0 2021-11-21T22:02:46.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:46.786 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:46.786 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:46.791 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:02:46.791 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:02:46.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:02:46.902 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:46.947 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:02:47.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbf6b33700 1 Processor -- start 2021-11-21T22:02:47.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbf6b33700 1 -- start start 2021-11-21T22:02:47.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbf6b33700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbbf00b11e0 0x7fbbf00aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbf6b33700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 0x7fbbf005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbf6b33700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00aaae0 con 0x7fbbf00aa040 2021-11-21T22:02:47.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbf6b33700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf01480f0 con 0x7fbbf00b11e0 2021-11-21T22:02:47.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbf6b33700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0144dc0 con 0x7fbbf00b07d0 2021-11-21T22:02:47.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbbf00b11e0 0x7fbbf00aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbbf00b11e0 0x7fbbf00aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53426/0 (socket says 172.21.2.116:53426) 2021-11-21T22:02:47.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 -- 172.21.2.116:0/2754351000 learned_addr learned my addr 172.21.2.116:0/2754351000 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:47.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 -- 172.21.2.116:0/2754351000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 msgr2=0x7fbbf005f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:47.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 --2- 172.21.2.116:0/2754351000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 0x7fbbf005f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 -- 172.21.2.116:0/2754351000 >> v1:172.21.2.116:6789/0 conn(0x7fbbf00aa040 legacy=0x7fbbf01446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 -- 172.21.2.116:0/2754351000 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf0147c20 con 0x7fbbf00b11e0 2021-11-21T22:02:47.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:46.998+0000 7fbbeffff700 1 --2- 172.21.2.116:0/2754351000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf00aa420 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7fbbd8002f70 tx=0x7fbbd8006eb0).ready entity=mon.0 client_cookie=8670abf90494766c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbef7fe700 1 -- 172.21.2.116:0/2754351000 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd800d5c0 con 0x7fbbf00b11e0 2021-11-21T22:02:47.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbef7fe700 1 -- 172.21.2.116:0/2754351000 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd8000e90 con 0x7fbbf00b11e0 2021-11-21T22:02:47.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbef7fe700 1 -- 172.21.2.116:0/2754351000 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd800d950 con 0x7fbbf00b11e0 2021-11-21T22:02:47.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- 172.21.2.116:0/2754351000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 msgr2=0x7fbbf00aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/2754351000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf00aa420 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7fbbd8002f70 tx=0x7fbbd8006eb0).stop 2021-11-21T22:02:47.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- 172.21.2.116:0/2754351000 shutdown_connections 2021-11-21T22:02:47.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/2754351000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 0x7fbbf005f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/2754351000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf00aa420 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- 172.21.2.116:0/2754351000 >> 172.21.2.116:0/2754351000 conn(0x7fbbf00b5060 msgr2=0x7fbbf00b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:47.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- 172.21.2.116:0/2754351000 shutdown_connections 2021-11-21T22:02:47.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- 172.21.2.116:0/2754351000 wait complete. 2021-11-21T22:02:47.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 Processor -- start 2021-11-21T22:02:47.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- start start 2021-11-21T22:02:47.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbbf00aa040 0x7fbbf0098430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 0x7fbbf0159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0146f40 con 0x7fbbf00b11e0 2021-11-21T22:02:47.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf015a880 con 0x7fbbf00aa040 2021-11-21T22:02:47.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf014d130 con 0x7fbbf00b07d0 2021-11-21T22:02:47.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53430/0 (socket says 172.21.2.116:53430) 2021-11-21T22:02:47.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 -- 172.21.2.116:0/1130426811 learned_addr learned my addr 172.21.2.116:0/1130426811 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:47.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf48cf700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbbf00aa040 0x7fbbf0098430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbeffff700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 0x7fbbf0159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 -- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 msgr2=0x7fbbf0159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 0x7fbbf0159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 -- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbbf00aa040 msgr2=0x7fbbf0098430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbbf00aa040 0x7fbbf0098430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 -- 172.21.2.116:0/1130426811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd800b1e0 con 0x7fbbf00b11e0 2021-11-21T22:02:47.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf50d0700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf015a390 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fbbf015a9c0 tx=0x7fbbe80091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbedffb700 1 -- 172.21.2.116:0/1130426811 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe800d4a0 con 0x7fbbf00b11e0 2021-11-21T22:02:47.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbedffb700 1 -- 172.21.2.116:0/1130426811 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe800d600 con 0x7fbbf00b11e0 2021-11-21T22:02:47.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbedffb700 1 -- 172.21.2.116:0/1130426811 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe800d960 con 0x7fbbf00b11e0 2021-11-21T22:02:47.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf014d3d0 con 0x7fbbf00b11e0 2021-11-21T22:02:47.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.002+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf014d910 con 0x7fbbf00b11e0 2021-11-21T22:02:47.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.006+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf0159ba0 con 0x7fbbf00b11e0 2021-11-21T22:02:47.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.006+0000 7fbbedffb700 1 -- 172.21.2.116:0/1130426811 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbbe8018030 con 0x7fbbf00b11e0 2021-11-21T22:02:47.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.006+0000 7fbbedffb700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbbd40536a0 0x7fbbd4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.006+0000 7fbbedffb700 1 -- 172.21.2.116:0/1130426811 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbbe806b280 con 0x7fbbf00b11e0 2021-11-21T22:02:47.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.006+0000 7fbbf48cf700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbbd40536a0 0x7fbbd4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.006+0000 7fbbf48cf700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbbd40536a0 0x7fbbd4055b60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fbbe000b000 tx=0x7fbbe0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.006+0000 7fbbedffb700 1 -- 172.21.2.116:0/1130426811 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbbe8042490 con 0x7fbbf00b11e0 2021-11-21T22:02:47.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.090+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbbf0147e50 con 0x7fbbf00b11e0 2021-11-21T22:02:47.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.090+0000 7fbbedffb700 1 -- 172.21.2.116:0/1130426811 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+33659 (secure 0 0 0) 0x7fbbe8028390 con 0x7fbbf00b11e0 2021-11-21T22:02:47.097 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:47.097 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:47.097 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 141, 2021-11-21T22:02:47.098 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:02:47.098 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:02:47.099 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:02:37.337392+0000", 2021-11-21T22:02:47.100 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:47.100 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:47.100 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:02:47.100 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:02:47.101 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:02:47.101 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:02:47.101 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:02:47.101 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:02:47.102 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:02:47.102 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:47.102 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:02:47.102 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:02:47.103 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:02:47.103 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:02:47.103 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:02:47.104 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 26, 2021-11-21T22:02:47.104 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:02:47.104 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:02:47.104 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:02:47.105 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:02:47.105 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:02:47.105 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.105 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:02:47.106 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:02:47.106 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:02:47.106 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:47.106 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:47.106 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:47.107 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:47.107 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:47.107 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:47.107 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:47.108 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:47.108 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:02:47.108 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:02:47.108 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:02:47.109 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:02:47.109 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:02:47.109 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:47.109 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:47.110 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:47.110 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:47.110 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:47.110 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:47.111 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:47.111 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.111 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:02:47.111 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:47.111 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:47.112 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:47.112 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:47.112 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:47.112 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:47.112 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:47.113 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:47.113 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:47.113 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:47.113 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:47.114 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:47.114 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:47.114 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:47.115 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:47.115 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:47.115 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:47.116 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:47.116 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:47.116 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:47.116 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:47.117 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:47.117 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:47.117 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:47.117 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:47.117 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:47.118 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.118 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:47.118 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:47.118 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:47.119 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:47.119 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:47.119 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:47.119 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:47.120 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:47.120 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:47.120 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:47.121 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:47.121 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:02:47.122 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:02:47.122 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.122 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:47.123 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:02:47.123 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.124 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.124 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.124 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:02:47.125 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:02:47.125 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:02:47.125 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:47.126 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:47.126 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:47.126 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:47.127 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:47.127 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:47.127 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:47.128 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:47.128 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:47.128 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:47.129 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:47.129 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:47.129 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:47.129 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:47.130 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:47.130 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:47.130 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:47.130 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:47.131 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:47.131 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:47.131 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.131 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:02:47.132 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:47.132 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:47.132 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:47.132 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:47.133 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:47.133 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:47.133 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:47.133 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:47.133 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:47.134 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:47.134 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:47.134 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:47.134 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:47.134 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:47.135 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:47.135 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:47.135 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:47.136 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:47.136 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:47.136 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:47.136 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:47.136 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:47.137 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:47.137 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:47.137 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:47.137 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:47.138 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.138 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:47.138 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:47.138 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:47.138 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:47.139 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:47.139 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:47.139 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:47.139 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:47.140 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:47.140 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:47.140 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:47.140 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:02:47.140 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:47.141 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:02:47.141 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.141 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.141 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.142 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 25, 2021-11-21T22:02:47.142 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:02:47.142 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:02:35.110650+0000", 2021-11-21T22:02:47.142 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:47.142 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:47.143 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:47.143 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:47.143 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:47.144 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:47.144 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:47.144 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:47.144 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:47.144 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:47.145 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:47.145 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:47.145 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:47.145 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:47.145 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:47.146 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:47.146 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:47.146 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:47.146 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:47.147 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:47.147 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.147 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "140", 2021-11-21T22:02:47.148 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:47.148 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:47.148 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:47.148 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:47.148 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:47.149 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:47.149 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:47.149 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:47.150 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:47.150 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:47.150 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:47.150 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:47.150 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:47.151 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:47.151 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:47.151 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:47.151 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:47.152 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:47.152 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:47.152 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:47.152 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:47.152 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:47.153 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:47.153 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:47.153 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:47.153 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:47.153 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.154 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:47.154 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:47.154 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:47.154 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:47.155 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:47.155 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:47.155 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:47.155 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:47.155 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:47.156 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:47.156 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:47.156 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:02:47.156 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:02:47.157 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:02:47.157 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:02:47.157 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.157 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:47.158 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:02:47.158 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:02:47.158 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.158 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.159 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.159 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.159 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 26, 2021-11-21T22:02:47.159 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:02:47.159 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:02:35.304135+0000", 2021-11-21T22:02:47.160 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:47.160 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:47.160 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:47.161 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:47.161 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:47.161 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:47.161 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:47.162 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:47.162 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:47.162 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:47.162 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:47.163 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:47.163 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:47.163 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:47.163 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:47.163 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:47.164 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:47.164 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:47.164 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:47.164 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:47.164 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.165 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "141", 2021-11-21T22:02:47.165 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:47.165 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:47.165 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:47.166 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:47.166 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:47.166 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:47.166 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:47.166 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:47.167 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:47.167 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:47.167 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:47.167 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:47.167 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:47.168 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:47.168 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:47.168 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:47.168 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:47.169 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:47.169 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:47.169 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:47.169 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:47.169 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:47.170 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:47.170 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:47.170 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:47.170 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:47.170 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.171 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:47.171 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:47.171 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:47.171 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:47.172 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:47.172 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:47.172 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:47.172 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:47.172 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:47.173 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:47.173 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:47.173 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:02:47.173 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:47.173 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:02:47.174 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:02:47.174 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.174 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.174 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.175 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:47.175 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:02:47.175 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.175 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:47.175 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:02:47.176 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.176 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.176 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.176 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.176 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.177 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.177 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.177 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 138, 2021-11-21T22:02:47.177 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.178 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.178 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.178 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.178 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.179 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.179 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:02:47.179 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.179 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.180 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.181 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.181 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:02:47.181 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.182 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.182 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.182 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.182 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.183 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.183 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.183 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.184 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:02:47.184 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.184 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.184 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.184 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.185 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:02:47.185 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.185 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.185 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.186 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.186 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.186 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.186 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.186 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.187 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:02:47.187 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.187 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.187 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.187 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.188 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:02:47.188 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.188 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.188 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.189 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.189 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.189 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.189 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.189 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.190 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:02:47.190 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.190 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.190 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.190 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.191 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:02:47.191 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:47.191 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.191 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.192 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.192 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:02:47.192 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:02:47.192 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:02:47.192 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:02:47.193 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.193 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.193 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.193 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.193 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.194 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.194 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:47.194 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:02:47.194 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.194 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.195 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.195 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.195 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.195 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.196 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.196 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:47.196 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.196 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.196 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.197 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.197 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.197 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.197 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:02:47.197 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.198 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.198 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.198 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.198 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:02:47.199 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.199 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.199 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.199 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.199 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.200 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.200 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.200 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.200 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:02:47.200 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.201 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.201 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.201 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.201 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:02:47.202 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.202 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.202 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.202 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.202 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.203 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.203 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.203 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.203 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:02:47.203 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.204 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.204 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.204 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.204 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:02:47.205 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.205 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.205 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.205 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.205 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.206 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.206 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.206 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.206 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:02:47.206 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.207 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.207 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.207 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.207 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:02:47.207 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:47.208 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.208 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.208 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.208 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:02:47.209 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:02:47.209 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:02:47.209 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:02:47.209 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.209 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.210 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.210 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.210 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.210 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.211 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:47.211 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:02:47.211 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.211 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.215 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.215 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.215 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.216 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.216 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.216 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:47.216 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.217 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.217 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.217 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.217 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.217 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.218 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:02:47.218 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.218 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.218 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.218 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.219 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:02:47.219 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.219 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.219 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.220 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.220 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.220 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.220 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.220 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.221 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:02:47.221 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.221 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.221 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.222 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.222 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:02:47.222 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.222 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.222 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.223 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.223 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.223 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.223 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.223 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.224 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:02:47.224 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.224 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.224 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.225 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.225 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:02:47.225 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.225 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.225 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.226 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.226 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.226 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.226 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.226 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.227 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:02:47.227 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.227 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.227 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.227 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.228 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:02:47.228 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:47.228 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.228 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.229 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.229 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:02:47.229 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:02:47.229 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:02:47.229 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:02:47.230 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.230 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.230 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.230 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.230 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.231 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.231 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:47.231 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:02:47.231 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.232 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.232 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.232 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.232 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.232 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.233 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.233 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 138, 2021-11-21T22:02:47.233 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.233 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.233 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.234 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.234 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.234 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.234 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:02:47.235 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.235 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.235 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.235 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.235 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:02:47.236 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.236 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.236 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.236 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.236 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.237 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.237 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.237 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.237 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:02:47.238 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.238 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.238 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.238 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.238 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:02:47.239 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.239 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.239 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.239 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.239 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.240 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.240 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.240 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.240 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:02:47.241 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.241 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.241 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.241 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.241 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:02:47.242 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.242 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.242 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.242 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.242 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.243 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.243 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.243 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.243 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:02:47.244 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.244 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.244 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.244 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.244 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:02:47.245 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:47.245 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.245 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.245 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.245 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:02:47.246 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:02:47.246 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:02:47.246 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:02:47.246 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.246 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.247 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.247 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.247 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.247 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.248 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:47.248 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:02:47.248 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.248 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.248 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.249 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.249 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.249 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.249 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.249 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:47.250 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.250 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.250 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.250 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.251 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.251 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.251 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:02:47.251 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.251 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.252 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.252 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.252 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:02:47.252 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.252 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.253 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.253 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.253 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.253 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.253 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.254 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.254 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:02:47.254 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.254 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.255 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.255 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.255 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:02:47.255 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.255 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.256 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.256 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.256 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.256 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.256 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.257 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.257 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:02:47.257 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.257 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.257 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.258 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.258 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:02:47.258 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.258 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.259 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.259 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.259 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.259 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.259 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.260 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.260 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:02:47.260 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.260 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.260 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.261 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.261 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:02:47.261 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:47.261 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.261 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.262 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.262 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:02:47.262 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:02:47.262 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:02:47.263 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:02:47.263 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.263 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.263 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.263 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.264 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.264 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.264 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:47.264 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:02:47.264 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.265 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.265 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.265 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.265 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.265 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.266 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.266 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:47.266 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.266 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.267 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.267 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.267 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.267 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.267 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:02:47.268 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.268 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.268 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.268 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.268 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:02:47.269 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.269 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.269 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.269 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.269 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.270 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.270 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.270 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.270 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:02:47.271 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.271 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.271 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.271 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.271 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:02:47.272 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.272 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.272 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.272 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.272 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.273 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.273 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.273 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.273 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:02:47.273 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.274 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.274 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.274 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.274 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:02:47.275 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.275 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.275 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.275 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.275 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.276 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.276 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.276 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.276 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:02:47.276 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.277 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.277 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.277 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.277 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:02:47.278 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:47.278 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.278 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.278 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.278 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:02:47.279 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:02:47.279 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:02:47.279 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:02:47.279 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.280 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.280 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.280 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.280 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.280 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.281 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:47.281 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:02:47.281 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.282 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.282 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.282 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.282 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.283 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.283 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.283 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:47.283 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.284 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.284 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.284 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.284 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.284 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.285 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:02:47.285 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.285 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.285 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.285 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.286 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:02:47.286 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.286 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.286 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.287 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.287 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.287 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.287 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.287 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.288 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:02:47.288 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.288 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.288 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.288 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.289 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:02:47.289 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.289 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.289 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.290 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.290 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.290 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.290 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.290 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.291 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:02:47.291 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.291 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.291 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.291 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.292 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:02:47.292 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.292 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.292 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.293 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.293 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.293 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.293 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.293 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.294 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:02:47.294 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.294 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.294 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.294 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.295 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:02:47.295 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:47.295 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.295 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.295 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.296 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:02:47.296 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:02:47.296 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:02:47.296 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:02:47.297 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.297 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.297 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.297 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.297 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.298 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.298 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:47.298 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:02:47.298 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:47.298 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:47.299 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:47.299 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:47.299 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:47.299 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:47.300 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:47.300 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:47.300 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:47.300 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:47.300 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:47.301 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.301 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.301 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.301 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:02:47.301 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.302 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.302 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.302 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.302 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:02:47.302 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.303 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.303 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.303 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.303 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:47.304 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.304 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.304 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.304 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:02:47.304 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.305 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.305 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.305 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.305 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:02:47.305 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.306 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.306 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.306 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.306 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:47.307 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.307 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.307 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.307 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:02:47.307 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.308 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.308 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.308 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.308 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:02:47.308 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.309 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.309 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.309 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.309 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:47.309 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:47.310 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.310 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:47.310 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:02:47.310 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.311 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.311 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.311 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:47.311 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:02:47.311 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:47.312 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.312 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.312 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.312 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:02:47.312 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:02:47.313 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:02:47.313 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:02:47.313 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:47.313 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:47.313 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:47.314 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:47.314 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.314 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:47.314 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:02:47.315 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.315 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:47.315 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.315 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.315 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.316 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.316 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.316 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:02:47.316 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.316 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.317 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.317 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:47.317 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.317 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.317 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.318 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.318 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.318 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:02:47.318 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.319 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.319 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.319 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:47.319 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.319 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.320 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.320 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.320 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.320 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:02:47.320 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.321 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.321 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.321 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:47.321 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.321 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.322 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.322 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.322 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.322 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:02:47.323 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.323 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.323 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.323 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:47.323 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.324 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.324 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.324 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.324 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.324 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:02:47.325 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.325 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.325 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.325 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:47.326 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.326 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.326 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.326 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.326 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.327 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:02:47.327 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.327 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.327 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.327 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:47.328 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.328 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.328 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.328 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.328 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.329 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:02:47.329 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.329 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.329 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:47.330 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:47.330 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:47.330 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:47.330 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:47.330 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:47.331 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:47.331 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:02:47.331 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:47.331 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.331 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:47.332 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:02:47.332 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:02:47.332 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:02:47.332 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:02:47.332 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T22:02:47.333 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:02:47.333 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:02:47.333 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:02:47.333 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:02:47.334 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:02:47.334 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:02:47.334 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:02:47.334 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:02:47.334 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:47.335 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:47.335 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:02:47.335 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:02:47.335 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:02:47.335 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:02:47.336 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:02:47.336 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:47.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.090+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbbd40536a0 msgr2=0x7fbbd4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.090+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbbd40536a0 0x7fbbd4055b60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fbbe000b000 tx=0x7fbbe0009040).stop 2021-11-21T22:02:47.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.090+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 msgr2=0x7fbbf015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.090+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf015a390 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fbbf015a9c0 tx=0x7fbbe80091a0).stop 2021-11-21T22:02:47.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 shutdown_connections 2021-11-21T22:02:47.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbbd40536a0 0x7fbbd4055b60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbbf00b11e0 0x7fbbf015a390 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbbf00b07d0 0x7fbbf0159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 --2- 172.21.2.116:0/1130426811 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbbf00aa040 0x7fbbf0098430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 >> 172.21.2.116:0/1130426811 conn(0x7fbbf00b5060 msgr2=0x7fbbf00b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:47.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 shutdown_connections 2021-11-21T22:02:47.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.094+0000 7fbbf6b33700 1 -- 172.21.2.116:0/1130426811 wait complete. 2021-11-21T22:02:47.341 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:47.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.390+0000 7f347f773700 1 Processor -- start 2021-11-21T22:02:47.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.390+0000 7f347f773700 1 -- start start 2021-11-21T22:02:47.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.390+0000 7f347f773700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780b11e0 0x7f34780aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.390+0000 7f347f773700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f34780b07d0 0x7f347805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.390+0000 7f347f773700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780aaae0 con 0x7f34780aa040 2021-11-21T22:02:47.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.390+0000 7f347f773700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34781480f0 con 0x7f34780b07d0 2021-11-21T22:02:47.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.390+0000 7f347f773700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3478144dc0 con 0x7f34780b11e0 2021-11-21T22:02:47.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f34780b07d0 0x7f347805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f34780b07d0 0x7f347805f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53438/0 (socket says 172.21.2.116:53438) 2021-11-21T22:02:47.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 -- 172.21.2.116:0/2508852063 learned_addr learned my addr 172.21.2.116:0/2508852063 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:47.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 -- 172.21.2.116:0/2508852063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780b11e0 msgr2=0x7f34780aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:47.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 --2- 172.21.2.116:0/2508852063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780b11e0 0x7f34780aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 -- 172.21.2.116:0/2508852063 >> v1:172.21.2.117:6789/0 conn(0x7f34780aa040 legacy=0x7f34781446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:47.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 -- 172.21.2.116:0/2508852063 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3478147c20 con 0x7f34780b07d0 2021-11-21T22:02:47.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347dd10700 1 --2- 172.21.2.116:0/2508852063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f347805f730 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f34780afa70 tx=0x7f3470009290).ready entity=mon.0 client_cookie=9e774ec55bce43ae server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f3467fff700 1 -- 172.21.2.116:0/2508852063 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347000b860 con 0x7f34780b07d0 2021-11-21T22:02:47.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f3467fff700 1 -- 172.21.2.116:0/2508852063 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347000b9c0 con 0x7f34780b07d0 2021-11-21T22:02:47.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f3467fff700 1 -- 172.21.2.116:0/2508852063 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347000bb70 con 0x7f34780b07d0 2021-11-21T22:02:47.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- 172.21.2.116:0/2508852063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 msgr2=0x7f347805f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 --2- 172.21.2.116:0/2508852063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f347805f730 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f34780afa70 tx=0x7f3470009290).stop 2021-11-21T22:02:47.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- 172.21.2.116:0/2508852063 shutdown_connections 2021-11-21T22:02:47.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 --2- 172.21.2.116:0/2508852063 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f347805f730 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 --2- 172.21.2.116:0/2508852063 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780b11e0 0x7f34780aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- 172.21.2.116:0/2508852063 >> 172.21.2.116:0/2508852063 conn(0x7f34780b5060 msgr2=0x7f34780b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:47.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- 172.21.2.116:0/2508852063 shutdown_connections 2021-11-21T22:02:47.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- 172.21.2.116:0/2508852063 wait complete. 2021-11-21T22:02:47.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 Processor -- start 2021-11-21T22:02:47.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- start start 2021-11-21T22:02:47.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780aa040 0x7f3478159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f34781499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f34780b11e0 0x7f34781570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347815ae20 con 0x7f34780b07d0 2021-11-21T22:02:47.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3478096e00 con 0x7f34780b11e0 2021-11-21T22:02:47.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780970a0 con 0x7f34780aa040 2021-11-21T22:02:47.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f34781499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f34781499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53442/0 (socket says 172.21.2.116:53442) 2021-11-21T22:02:47.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 -- 172.21.2.116:0/720534314 learned_addr learned my addr 172.21.2.116:0/720534314 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:47.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 -- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780aa040 msgr2=0x7f3478159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780aa040 0x7f3478159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 -- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f34780b11e0 msgr2=0x7f34781570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f34780b11e0 0x7f34781570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 -- 172.21.2.116:0/720534314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009530 con 0x7f34780b07d0 2021-11-21T22:02:47.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347cd0e700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f34781499f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f3468002f70 tx=0x7f3468006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f34667fc700 1 -- 172.21.2.116:0/720534314 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f346800e470 con 0x7f34780b07d0 2021-11-21T22:02:47.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f34667fc700 1 -- 172.21.2.116:0/720534314 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346800e5d0 con 0x7f34780b07d0 2021-11-21T22:02:47.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3478156c50 con 0x7f34780b07d0 2021-11-21T22:02:47.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f34667fc700 1 -- 172.21.2.116:0/720534314 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f346800e930 con 0x7f34780b07d0 2021-11-21T22:02:47.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.394+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3478156940 con 0x7f34780b07d0 2021-11-21T22:02:47.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.398+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3478156de0 con 0x7f34780b07d0 2021-11-21T22:02:47.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.398+0000 7f34667fc700 1 -- 172.21.2.116:0/720534314 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3468018030 con 0x7f34780b07d0 2021-11-21T22:02:47.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.398+0000 7f34667fc700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34580536a0 0x7f3458055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.398+0000 7f34667fc700 1 -- 172.21.2.116:0/720534314 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f346806aea0 con 0x7f34780b07d0 2021-11-21T22:02:47.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.398+0000 7f347d50f700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34580536a0 0x7f3458055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.398+0000 7f347d50f700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34580536a0 0x7f3458055b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f3474002f90 tx=0x7f34740069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.398+0000 7f34667fc700 1 -- 172.21.2.116:0/720534314 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3468041fa0 con 0x7f34780b07d0 2021-11-21T22:02:47.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3478147e50 con 0x7f34780b07d0 2021-11-21T22:02:47.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f34667fc700 1 -- 172.21.2.116:0/720534314 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f34680455c0 con 0x7f34780b07d0 2021-11-21T22:02:47.512 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:47.512 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":65,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:36.333834+0000","modified":"2021-11-21T22:02:37.338971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8109":{"gid":8109,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":2,"addr":"172.21.2.116:6835/1832322648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":1832322648},{"type":"v1","addr":"172.21.2.116:6835","nonce":1832322648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-11-21T22:02:47.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34580536a0 msgr2=0x7f3458055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34580536a0 0x7f3458055b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f3474002f90 tx=0x7f34740069b0).stop 2021-11-21T22:02:47.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 msgr2=0x7f34781499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f34781499f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f3468002f70 tx=0x7f3468006e30).stop 2021-11-21T22:02:47.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 shutdown_connections 2021-11-21T22:02:47.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f34780b11e0 0x7f34781570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f34580536a0 0x7f3458055b60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f34780b07d0 0x7f34781499f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 --2- 172.21.2.116:0/720534314 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f34780aa040 0x7f3478159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 >> 172.21.2.116:0/720534314 conn(0x7f34780b5060 msgr2=0x7f347814ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:47.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 shutdown_connections 2021-11-21T22:02:47.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.506+0000 7f347f773700 1 -- 172.21.2.116:0/720534314 wait complete. 2021-11-21T22:02:47.516 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 65 2021-11-21T22:02:47.519 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:02:47.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 Processor -- start 2021-11-21T22:02:47.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- start start 2021-11-21T22:02:47.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6000b3670 0x7fe6000b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe6000613e0 0x7fe600143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.575 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6000992a0 con 0x7fe6000988c0 2021-11-21T22:02:47.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6000ac5c0 con 0x7fe6000613e0 2021-11-21T22:02:47.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000b3a50 con 0x7fe6000b3670 2021-11-21T22:02:47.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6058ba700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe6000988c0 0x7fe600060cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37176/0 (socket says 172.21.2.116:37176) 2021-11-21T22:02:47.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6058ba700 1 -- 172.21.2.116:0/1951019513 learned_addr learned my addr 172.21.2.116:0/1951019513 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:47.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288323097 0 0) 0x7fe6000992a0 con 0x7fe6000988c0 2021-11-21T22:02:47.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e8003260 con 0x7fe6000988c0 2021-11-21T22:02:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6068bc700 1 --2- 172.21.2.116:0/1951019513 >> v2:172.21.2.117:3300/0 conn(0x7fe6000613e0 0x7fe600143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2719923925 0 0) 0x7fe5e8003260 con 0x7fe6000988c0 2021-11-21T22:02:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6000b3670 0x7fe6000b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6000b3670 msgr2=0x7fe6000b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:47.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6000b3670 0x7fe6000b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 >> v2:172.21.2.117:3300/0 conn(0x7fe6000613e0 msgr2=0x7fe600143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:47.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 --2- 172.21.2.116:0/1951019513 >> v2:172.21.2.117:3300/0 conn(0x7fe6000613e0 0x7fe600143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6000b1130 con 0x7fe6000988c0 2021-11-21T22:02:47.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe5f4001660 con 0x7fe6000988c0 2021-11-21T22:02:47.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 >> v1:172.21.2.116:6789/0 conn(0x7fe6000988c0 legacy=0x7fe600060cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5e80037d0 0x7fe5e8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5e8004500 0x7fe5e8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6000b3670 0x7fe5e800d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000ac5c0 con 0x7fe5e80037d0 2021-11-21T22:02:47.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000b3a50 con 0x7fe5e8004500 2021-11-21T22:02:47.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e8005300 con 0x7fe6000b3670 2021-11-21T22:02:47.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6068bc700 1 --2- 172.21.2.116:0/1951019513 >> v2:172.21.2.117:3300/0 conn(0x7fe6000613e0 0x7fe600143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:47.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5e80037d0 0x7fe5e8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6058ba700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5e8004500 0x7fe5e8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 -- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6000b3670 msgr2=0x7fe5e800d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:47.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe6000b3670 0x7fe5e800d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 -- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5e8004500 msgr2=0x7fe5e8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:47.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5e8004500 0x7fe5e8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 -- 172.21.2.116:0/1951019513 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8003260 con 0x7fe5e80037d0 2021-11-21T22:02:47.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6058ba700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe5e8004500 0x7fe5e8004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:47.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6060bb700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5e80037d0 0x7fe5e8003bf0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7fe5fc007000 tx=0x7fe5fc008040).ready entity=mon.0 client_cookie=93e2cb1fcd84c96 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5fc012470 con 0x7fe5e80037d0 2021-11-21T22:02:47.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc00c030 con 0x7fe5e80037d0 2021-11-21T22:02:47.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6050b9700 1 -- 172.21.2.116:0/1951019513 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5fc0127c0 con 0x7fe5e80037d0 2021-11-21T22:02:47.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- 172.21.2.116:0/1951019513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5e80037d0 msgr2=0x7fe5e8003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 --2- 172.21.2.116:0/1951019513 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5e80037d0 0x7fe5e8003bf0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7fe5fc007000 tx=0x7fe5fc008040).stop 2021-11-21T22:02:47.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe6068bc700 1 -- 172.21.2.116:0/1951019513 reap_dead start 2021-11-21T22:02:47.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- 172.21.2.116:0/1951019513 shutdown_connections 2021-11-21T22:02:47.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- 172.21.2.116:0/1951019513 >> 172.21.2.116:0/1951019513 conn(0x7fe6000b6c20 msgr2=0x7fe6000b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:47.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- 172.21.2.116:0/1951019513 shutdown_connections 2021-11-21T22:02:47.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- 172.21.2.116:0/1951019513 wait complete. 2021-11-21T22:02:47.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 Processor -- start 2021-11-21T22:02:47.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.566+0000 7fe60831f700 1 -- start start 2021-11-21T22:02:47.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5e80037d0 0x7fe6000adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6000ad4b0 0x7fe6000ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6000ae610 0x7fe600064280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe600152b00 con 0x7fe6000ae610 2021-11-21T22:02:47.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe600146d10 con 0x7fe6000ad4b0 2021-11-21T22:02:47.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe600152d90 con 0x7fe5e80037d0 2021-11-21T22:02:47.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6000ae610 0x7fe600064280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6000ae610 0x7fe600064280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53462/0 (socket says 172.21.2.116:53462) 2021-11-21T22:02:47.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 -- 172.21.2.116:0/1893312541 learned_addr learned my addr 172.21.2.116:0/1893312541 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:47.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 -- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5e80037d0 msgr2=0x7fe6000adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5e80037d0 0x7fe6000adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 -- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6000ad4b0 msgr2=0x7fe6000ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6000ad4b0 0x7fe6000ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 -- 172.21.2.116:0/1893312541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600153030 con 0x7fe6000ae610 2021-11-21T22:02:47.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6068bc700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6000ae610 0x7fe600064280 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fe5f40017c0 tx=0x7fe5f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe5f37fe700 1 -- 172.21.2.116:0/1893312541 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f800ce90 con 0x7fe6000ae610 2021-11-21T22:02:47.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe5f37fe700 1 -- 172.21.2.116:0/1893312541 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8010580 con 0x7fe6000ae610 2021-11-21T22:02:47.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe5f37fe700 1 -- 172.21.2.116:0/1893312541 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f80108e0 con 0x7fe6000ae610 2021-11-21T22:02:47.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe600146040 con 0x7fe6000ae610 2021-11-21T22:02:47.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe600064910 con 0x7fe6000ae610 2021-11-21T22:02:47.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6000994d0 con 0x7fe6000ae610 2021-11-21T22:02:47.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe5f37fe700 1 -- 172.21.2.116:0/1893312541 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe5f8010a40 con 0x7fe6000ae610 2021-11-21T22:02:47.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe5f37fe700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5e0053610 0x7fe5e0055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:47.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe5f37fe700 1 -- 172.21.2.116:0/1893312541 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe5f806b8c0 con 0x7fe6000ae610 2021-11-21T22:02:47.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.570+0000 7fe6060bb700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5e0053610 0x7fe5e0055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:47.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.574+0000 7fe6060bb700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5e0053610 0x7fe5e0055ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fe5fc007000 tx=0x7fe5fc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:47.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.574+0000 7fe5f37fe700 1 -- 172.21.2.116:0/1893312541 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5f8042a20 con 0x7fe6000ae610 2021-11-21T22:02:47.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.682+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe6000573d0 con 0x7fe6000ae610 2021-11-21T22:02:47.953 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:02:47.950+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:02:47.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe5f37fe700 1 -- 172.21.2.116:0/1893312541 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe5f8046040 con 0x7fe6000ae610 2021-11-21T22:02:47.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5e0053610 msgr2=0x7fe5e0055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5e0053610 0x7fe5e0055ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fe5fc007000 tx=0x7fe5fc008040).stop 2021-11-21T22:02:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6000ae610 msgr2=0x7fe600064280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6000ae610 0x7fe600064280 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fe5f40017c0 tx=0x7fe5f8008040).stop 2021-11-21T22:02:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 shutdown_connections 2021-11-21T22:02:47.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5e0053610 0x7fe5e0055ad0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe6000ae610 0x7fe600064280 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe6000ad4b0 0x7fe6000ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 --2- 172.21.2.116:0/1893312541 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe5e80037d0 0x7fe6000adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 >> 172.21.2.116:0/1893312541 conn(0x7fe6000b6c20 msgr2=0x7fe60005bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:47.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 shutdown_connections 2021-11-21T22:02:47.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:47.962+0000 7fe60831f700 1 -- 172.21.2.116:0/1893312541 wait complete. 2021-11-21T22:02:47.971 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:02:47.974 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:47.983 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:02:48.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd669dd2700 1 Processor -- start 2021-11-21T22:02:48.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd669dd2700 1 -- start start 2021-11-21T22:02:48.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd669dd2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6640b7360 0x7fd6640b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd669dd2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6640b2e50 con 0x7fd6640b39c0 2021-11-21T22:02:48.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd669dd2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6640b7740 con 0x7fd6640ad930 2021-11-21T22:02:48.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd669dd2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640acfa0 con 0x7fd6640b7360 2021-11-21T22:02:48.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd6695d1700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd6640b39c0 0x7fd66413dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37208/0 (socket says 172.21.2.116:37208) 2021-11-21T22:02:48.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd6695d1700 1 -- 172.21.2.116:0/3686923427 learned_addr learned my addr 172.21.2.116:0/3686923427 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:48.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480641820 0 0) 0x7fd6640b2e50 con 0x7fd6640b39c0 2021-11-21T22:02:48.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd650003260 con 0x7fd6640b39c0 2021-11-21T22:02:48.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816699225 0 0) 0x7fd650003260 con 0x7fd6640b39c0 2021-11-21T22:02:48.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6640b7360 msgr2=0x7fd6640b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:48.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd668dd0700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6640b7360 0x7fd6640b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6640b7360 0x7fd6640b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 >> v1:172.21.2.117:6789/0 conn(0x7fd6640ad930 legacy=0x7fd664141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664140a30 con 0x7fd6640b39c0 2021-11-21T22:02:48.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd65c0019e0 con 0x7fd6640b39c0 2021-11-21T22:02:48.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 >> v1:172.21.2.116:6789/0 conn(0x7fd6640b39c0 legacy=0x7fd66413dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd650003800 0x7fd650003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd650004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd650005050 0x7fd65000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640acfa0 con 0x7fd650004500 2021-11-21T22:02:48.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65000bf50 con 0x7fd650003800 2021-11-21T22:02:48.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65000c110 con 0x7fd650005050 2021-11-21T22:02:48.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd668dd0700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd6640b7360 0x7fd6640b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd650004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd6695d1700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd650005050 0x7fd65000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd668dd0700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd650003800 0x7fd650003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65bfff700 1 -- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd650005050 msgr2=0x7fd65000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd650005050 0x7fd65000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65bfff700 1 -- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd650003800 msgr2=0x7fd650003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd650003800 0x7fd650003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65bfff700 1 -- 172.21.2.116:0/3686923427 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650003260 con 0x7fd650004500 2021-11-21T22:02:48.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd650004980 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fd664064780 tx=0x7fd654007a00).ready entity=mon.0 client_cookie=ec6a4004b57c9370 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd654007d10 con 0x7fd650004500 2021-11-21T22:02:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654007030 con 0x7fd650004500 2021-11-21T22:02:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.022+0000 7fd65b7fe700 1 -- 172.21.2.116:0/3686923427 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd654007390 con 0x7fd650004500 2021-11-21T22:02:48.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3686923427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 msgr2=0x7fd650004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 --2- 172.21.2.116:0/3686923427 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd650004980 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fd664064780 tx=0x7fd654007a00).stop 2021-11-21T22:02:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd6695d1700 1 -- 172.21.2.116:0/3686923427 reap_dead start 2021-11-21T22:02:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3686923427 shutdown_connections 2021-11-21T22:02:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3686923427 >> 172.21.2.116:0/3686923427 conn(0x7fd664064c20 msgr2=0x7fd664065030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:48.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3686923427 shutdown_connections 2021-11-21T22:02:48.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3686923427 wait complete. 2021-11-21T22:02:48.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 Processor -- start 2021-11-21T22:02:48.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- start start 2021-11-21T22:02:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd66412cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6640b6850 0x7fd664066950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd664067640 0x7fd664060ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66412d6b0 con 0x7fd650004500 2021-11-21T22:02:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd664066010 con 0x7fd6640b6850 2021-11-21T22:02:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640b6cf0 con 0x7fd664067640 2021-11-21T22:02:48.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd66412cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd66412cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53492/0 (socket says 172.21.2.116:53492) 2021-11-21T22:02:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 -- 172.21.2.116:0/3650582843 learned_addr learned my addr 172.21.2.116:0/3650582843 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 -- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd664067640 msgr2=0x7fd664060ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd664067640 0x7fd664060ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 -- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6640b6850 msgr2=0x7fd664066950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6640b6850 0x7fd664066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6640b6850 0x7fd664066950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 -- 172.21.2.116:0/3650582843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664066240 con 0x7fd650004500 2021-11-21T22:02:48.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6640b6850 0x7fd664066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd668dd0700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd66412cf20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7fd66000b000 tx=0x7fd660008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd659ffb700 1 -- 172.21.2.116:0/3650582843 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd660016070 con 0x7fd650004500 2021-11-21T22:02:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd659ffb700 1 -- 172.21.2.116:0/3650582843 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660000e90 con 0x7fd650004500 2021-11-21T22:02:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd659ffb700 1 -- 172.21.2.116:0/3650582843 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd660011750 con 0x7fd650004500 2021-11-21T22:02:48.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6640664e0 con 0x7fd650004500 2021-11-21T22:02:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6640613e0 con 0x7fd650004500 2021-11-21T22:02:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd664066670 con 0x7fd650004500 2021-11-21T22:02:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd659ffb700 1 -- 172.21.2.116:0/3650582843 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd6600118b0 con 0x7fd650004500 2021-11-21T22:02:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd659ffb700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd6440536a0 0x7fd644055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd659ffb700 1 -- 172.21.2.116:0/3650582843 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fd66006b530 con 0x7fd650004500 2021-11-21T22:02:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.026+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd6440536a0 0x7fd644055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.030+0000 7fd65bfff700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd6440536a0 0x7fd644055b60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fd654016000 tx=0x7fd654015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:48.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.030+0000 7fd659ffb700 1 -- 172.21.2.116:0/3650582843 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd660042640 con 0x7fd650004500 2021-11-21T22:02:48.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd664140cd0 con 0x7fd650004500 2021-11-21T22:02:48.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd659ffb700 1 -- 172.21.2.116:0/3650582843 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fd660045c60 con 0x7fd650004500 2021-11-21T22:02:48.144 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:48.144 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":66,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:36.333834+0000","modified":"2021-11-21T22:02:47.953260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":142,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-11-21T22:02:48.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd6440536a0 msgr2=0x7fd644055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd6440536a0 0x7fd644055b60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fd654016000 tx=0x7fd654015000).stop 2021-11-21T22:02:48.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 msgr2=0x7fd66412cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd66412cf20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7fd66000b000 tx=0x7fd660008040).stop 2021-11-21T22:02:48.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 shutdown_connections 2021-11-21T22:02:48.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd664067640 0x7fd664060ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd6640b6850 0x7fd664066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd6440536a0 0x7fd644055b60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 --2- 172.21.2.116:0/3650582843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd650004500 0x7fd66412cf20 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.139+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 >> 172.21.2.116:0/3650582843 conn(0x7fd664064c20 msgr2=0x7fd664131bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:48.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.143+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 shutdown_connections 2021-11-21T22:02:48.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.143+0000 7fd669dd2700 1 -- 172.21.2.116:0/3650582843 wait complete. 2021-11-21T22:02:48.148 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 66 2021-11-21T22:02:48.152 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:02:48.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 Processor -- start 2021-11-21T22:02:48.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- start start 2021-11-21T22:02:48.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180b3670 0x7fe1180b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe1180988c0 0x7fe118060cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1180992a0 con 0x7fe1180611c0 2021-11-21T22:02:48.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1180ac5c0 con 0x7fe1180988c0 2021-11-21T22:02:48.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180b3a50 con 0x7fe1180b3670 2021-11-21T22:02:48.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe1180988c0 0x7fe118060cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe1180988c0 0x7fe118060cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53502/0 (socket says 172.21.2.116:53502) 2021-11-21T22:02:48.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 -- 172.21.2.116:0/577791674 learned_addr learned my addr 172.21.2.116:0/577791674 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:48.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 -- 172.21.2.116:0/577791674 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180b3670 msgr2=0x7fe1180b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:48.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 --2- 172.21.2.116:0/577791674 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180b3670 0x7fe1180b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 -- 172.21.2.116:0/577791674 >> v1:172.21.2.116:6789/0 conn(0x7fe1180611c0 legacy=0x7fe1180b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 -- 172.21.2.116:0/577791674 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1180b1130 con 0x7fe1180988c0 2021-11-21T22:02:48.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe116ffd700 1 --2- 172.21.2.116:0/577791674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180988c0 0x7fe118060cd0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100006f40).ready entity=mon.0 client_cookie=12dd75f3476007e4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe1167fc700 1 -- 172.21.2.116:0/577791674 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000dc70 con 0x7fe1180988c0 2021-11-21T22:02:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe1167fc700 1 -- 172.21.2.116:0/577791674 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100000e90 con 0x7fe1180988c0 2021-11-21T22:02:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe1167fc700 1 -- 172.21.2.116:0/577791674 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000be90 con 0x7fe1180988c0 2021-11-21T22:02:48.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- 172.21.2.116:0/577791674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180988c0 msgr2=0x7fe118060cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- 172.21.2.116:0/577791674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180988c0 0x7fe118060cd0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100006f40).stop 2021-11-21T22:02:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- 172.21.2.116:0/577791674 shutdown_connections 2021-11-21T22:02:48.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- 172.21.2.116:0/577791674 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180988c0 0x7fe118060cd0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- 172.21.2.116:0/577791674 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180b3670 0x7fe1180b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- 172.21.2.116:0/577791674 >> 172.21.2.116:0/577791674 conn(0x7fe1180b6c20 msgr2=0x7fe1180b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- 172.21.2.116:0/577791674 shutdown_connections 2021-11-21T22:02:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- 172.21.2.116:0/577791674 wait complete. 2021-11-21T22:02:48.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 Processor -- start 2021-11-21T22:02:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- start start 2021-11-21T22:02:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1180611c0 0x7fe1180907d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180988c0 0x7fe11814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180b3670 0x7fe11814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11814bfb0 con 0x7fe1180b3670 2021-11-21T22:02:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180adab0 con 0x7fe1180611c0 2021-11-21T22:02:48.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe11e134700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11814eee0 con 0x7fe1180988c0 2021-11-21T22:02:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180b3670 0x7fe11814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180b3670 0x7fe11814bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53510/0 (socket says 172.21.2.116:53510) 2021-11-21T22:02:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 -- 172.21.2.116:0/2660243218 learned_addr learned my addr 172.21.2.116:0/2660243218 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:48.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 -- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180988c0 msgr2=0x7fe11814b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180988c0 0x7fe11814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 -- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1180611c0 msgr2=0x7fe1180907d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1180611c0 0x7fe1180907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 -- 172.21.2.116:0/2660243218 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10000b460 con 0x7fe1180b3670 2021-11-21T22:02:48.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.207+0000 7fe117fff700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180b3670 0x7fe11814bac0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fe118063960 tx=0x7fe110007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe114ff9700 1 -- 172.21.2.116:0/2660243218 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe11000e4a0 con 0x7fe1180b3670 2021-11-21T22:02:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe114ff9700 1 -- 172.21.2.116:0/2660243218 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11000e600 con 0x7fe1180b3670 2021-11-21T22:02:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11814f180 con 0x7fe1180b3670 2021-11-21T22:02:48.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe114ff9700 1 -- 172.21.2.116:0/2660243218 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe11000e960 con 0x7fe1180b3670 2021-11-21T22:02:48.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe118066740 con 0x7fe1180b3670 2021-11-21T22:02:48.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1180994d0 con 0x7fe1180b3670 2021-11-21T22:02:48.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe114ff9700 1 -- 172.21.2.116:0/2660243218 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe110019030 con 0x7fe1180b3670 2021-11-21T22:02:48.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe114ff9700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0fc053580 0x7fe0fc055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:48.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe114ff9700 1 -- 172.21.2.116:0/2660243218 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fe11006b0c0 con 0x7fe1180b3670 2021-11-21T22:02:48.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe1177fe700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0fc053580 0x7fe0fc055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:48.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe1177fe700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0fc053580 0x7fe0fc055a40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe108002f70 tx=0x7fe1080069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:48.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.211+0000 7fe114ff9700 1 -- 172.21.2.116:0/2660243218 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1100421d0 con 0x7fe1180b3670 2021-11-21T22:02:48.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.323+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1180573d0 con 0x7fe1180b3670 2021-11-21T22:02:48.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.967+0000 7fe114ff9700 1 -- 172.21.2.116:0/2660243218 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1100457f0 con 0x7fe1180b3670 2021-11-21T22:02:48.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0fc053580 msgr2=0x7fe0fc055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0fc053580 0x7fe0fc055a40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe108002f70 tx=0x7fe1080069e0).stop 2021-11-21T22:02:48.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180b3670 msgr2=0x7fe11814bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:48.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180b3670 0x7fe11814bac0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fe118063960 tx=0x7fe110007f40).stop 2021-11-21T22:02:48.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 shutdown_connections 2021-11-21T22:02:48.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe0fc053580 0x7fe0fc055a40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1180b3670 0x7fe11814bac0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:48.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1180988c0 0x7fe11814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 --2- 172.21.2.116:0/2660243218 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1180611c0 0x7fe1180907d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:48.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 >> 172.21.2.116:0/2660243218 conn(0x7fe1180b6c20 msgr2=0x7fe118141850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:48.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 shutdown_connections 2021-11-21T22:02:48.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:48.971+0000 7fe11e134700 1 -- 172.21.2.116:0/2660243218 wait complete. 2021-11-21T22:02:48.981 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:02:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 Processor -- start 2021-11-21T22:02:49.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- start start 2021-11-21T22:02:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f70a40ac810 0x7f70a40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40adf80 0x7f70a40ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a40acbf0 con 0x7f70a40abc00 2021-11-21T22:02:49.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a41481f0 con 0x7f70a40ac810 2021-11-21T22:02:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4147570 con 0x7f70a40adf80 2021-11-21T22:02:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9626700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f70a40abc00 0x7f70a40b11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37244/0 (socket says 172.21.2.116:37244) 2021-11-21T22:02:49.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9626700 1 -- 172.21.2.116:0/3424821978 learned_addr learned my addr 172.21.2.116:0/3424821978 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751436377 0 0) 0x7f70a40acbf0 con 0x7f70a40abc00 2021-11-21T22:02:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f708c003260 con 0x7f70a40abc00 2021-11-21T22:02:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3826762705 0 0) 0x7f708c003260 con 0x7f70a40abc00 2021-11-21T22:02:49.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40adf80 msgr2=0x7f70a40ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40adf80 0x7f70a40ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 >> v2:172.21.2.117:3300/0 conn(0x7f70a40ac810 msgr2=0x7f70a40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 --2- 172.21.2.116:0/3424821978 >> v2:172.21.2.117:3300/0 conn(0x7f70a40ac810 0x7f70a40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4147810 con 0x7f70a40abc00 2021-11-21T22:02:49.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f7098001660 con 0x7f70a40abc00 2021-11-21T22:02:49.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 >> v1:172.21.2.116:6789/0 conn(0x7f70a40abc00 legacy=0x7f70a40b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:49.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f708c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40adf80 0x7f708c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f708c004b10 0x7f708c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41481f0 con 0x7f708c0037d0 2021-11-21T22:02:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4147570 con 0x7f708c004b10 2021-11-21T22:02:49.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c00b9e0 con 0x7f70a40adf80 2021-11-21T22:02:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9e27700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f708c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9e27700 1 -- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40adf80 msgr2=0x7f708c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9e27700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40adf80 0x7f708c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9e27700 1 -- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f708c004b10 msgr2=0x7f708c004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9e27700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f708c004b10 0x7f708c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9e27700 1 -- 172.21.2.116:0/3424821978 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4147810 con 0x7f708c0037d0 2021-11-21T22:02:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a9e27700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f708c003bf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7f70a0002f70 tx=0x7f70a0006ca0).ready entity=mon.0 client_cookie=99417d88d02f98f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70a0012410 con 0x7f708c0037d0 2021-11-21T22:02:49.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0012570 con 0x7f708c0037d0 2021-11-21T22:02:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70a8e25700 1 -- 172.21.2.116:0/3424821978 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70a00128d0 con 0x7f708c0037d0 2021-11-21T22:02:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- 172.21.2.116:0/3424821978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 msgr2=0x7f708c003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 --2- 172.21.2.116:0/3424821978 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f708c003bf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7f70a0002f70 tx=0x7f70a0006ca0).stop 2021-11-21T22:02:49.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70aa628700 1 -- 172.21.2.116:0/3424821978 reap_dead start 2021-11-21T22:02:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- 172.21.2.116:0/3424821978 shutdown_connections 2021-11-21T22:02:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- 172.21.2.116:0/3424821978 >> 172.21.2.116:0/3424821978 conn(0x7f70a40b5090 msgr2=0x7f70a40b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- 172.21.2.116:0/3424821978 shutdown_connections 2021-11-21T22:02:49.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.043+0000 7f70ac08b700 1 -- 172.21.2.116:0/3424821978 wait complete. 2021-11-21T22:02:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 Processor -- start 2021-11-21T22:02:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 -- start start 2021-11-21T22:02:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f70a40a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70a40a7230 0x7f70a4156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40666f0 0x7f70a4156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4065f00 con 0x7f708c0037d0 2021-11-21T22:02:49.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4067230 con 0x7f70a40a7230 2021-11-21T22:02:49.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40b0e10 con 0x7f70a40666f0 2021-11-21T22:02:49.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f70a40a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:49.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f70a40a8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53528/0 (socket says 172.21.2.116:53528) 2021-11-21T22:02:49.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 -- 172.21.2.116:0/3369072193 learned_addr learned my addr 172.21.2.116:0/3369072193 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:49.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 -- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40666f0 msgr2=0x7f70a4156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:49.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40666f0 0x7f70a4156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 -- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70a40a7230 msgr2=0x7f70a4156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:49.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70a40a7230 0x7f70a4156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 -- 172.21.2.116:0/3369072193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a40b0fd0 con 0x7f708c0037d0 2021-11-21T22:02:49.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9e27700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f70a40a8c10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f70a0002f70 tx=0x7f70a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:49.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f7096ffd700 1 -- 172.21.2.116:0/3369072193 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70a0012410 con 0x7f708c0037d0 2021-11-21T22:02:49.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f7096ffd700 1 -- 172.21.2.116:0/3369072193 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0011e00 con 0x7f708c0037d0 2021-11-21T22:02:49.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a40a9100 con 0x7f708c0037d0 2021-11-21T22:02:49.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f7096ffd700 1 -- 172.21.2.116:0/3369072193 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70a001d610 con 0x7f708c0037d0 2021-11-21T22:02:49.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a4064710 con 0x7f708c0037d0 2021-11-21T22:02:49.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a40673a0 con 0x7f708c0037d0 2021-11-21T22:02:49.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f7096ffd700 1 -- 172.21.2.116:0/3369072193 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f70a0024030 con 0x7f708c0037d0 2021-11-21T22:02:49.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f7096ffd700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7084053580 0x7f7084055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:49.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f7096ffd700 1 -- 172.21.2.116:0/3369072193 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f70a006c8f0 con 0x7f708c0037d0 2021-11-21T22:02:49.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.047+0000 7f70a9626700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7084053580 0x7f7084055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:49.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.051+0000 7f70a9626700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7084053580 0x7f7084055a40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f7098001370 tx=0x7f7098001640).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:49.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.051+0000 7f7096ffd700 1 -- 172.21.2.116:0/3369072193 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70a40673a0 con 0x7f708c0037d0 2021-11-21T22:02:49.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.131+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f70a40673a0 con 0x7f708c0037d0 2021-11-21T22:02:49.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f7096ffd700 1 -- 172.21.2.116:0/3369072193 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v143) v1 ==== 185+0+0 (secure 0 0 0) 0x7f70a40673a0 con 0x7f708c0037d0 2021-11-21T22:02:49.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7084053580 msgr2=0x7f7084055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:49.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7084053580 0x7f7084055a40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f7098001370 tx=0x7f7098001640).stop 2021-11-21T22:02:49.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 msgr2=0x7f70a40a8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:49.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f70a40a8c10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f70a0002f70 tx=0x7f70a0015000).stop 2021-11-21T22:02:49.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 shutdown_connections 2021-11-21T22:02:49.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70a40666f0 0x7f70a4156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70a40a7230 0x7f70a4156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:49.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7084053580 0x7f7084055a40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:49.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 --2- 172.21.2.116:0/3369072193 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f708c0037d0 0x7f70a40a8c10 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:49.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 >> 172.21.2.116:0/3369072193 conn(0x7f70a40b5090 msgr2=0x7f70a40b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:49.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 shutdown_connections 2021-11-21T22:02:49.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:49.975+0000 7f70ac08b700 1 -- 172.21.2.116:0/3369072193 wait complete. 2021-11-21T22:02:49.983 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:02:49.986 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:02:50.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 Processor -- start 2021-11-21T22:02:50.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- start start 2021-11-21T22:02:50.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc680abc00 0x7fdc680b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:50.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680adf80 0x7fdc680ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:50.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc680acbf0 con 0x7fdc680ac810 2021-11-21T22:02:50.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc681481f0 con 0x7fdc680abc00 2021-11-21T22:02:50.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68147570 con 0x7fdc680adf80 2021-11-21T22:02:50.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc680abc00 0x7fdc680b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:50.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc680abc00 0x7fdc680b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53540/0 (socket says 172.21.2.116:53540) 2021-11-21T22:02:50.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/3257321648 learned_addr learned my addr 172.21.2.116:0/3257321648 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:50.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6cb5a700 1 --2- 172.21.2.116:0/3257321648 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680adf80 0x7fdc680ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:50.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/3257321648 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680adf80 msgr2=0x7fdc680ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:50.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- 172.21.2.116:0/3257321648 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680adf80 0x7fdc680ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:50.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/3257321648 >> v1:172.21.2.117:6789/0 conn(0x7fdc680ac810 legacy=0x7fdc680b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:50.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/3257321648 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68147810 con 0x7fdc680abc00 2021-11-21T22:02:50.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc66ffd700 1 -- 172.21.2.116:0/3257321648 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723931036 0 0) 0x7fdc680acbf0 con 0x7fdc680ac810 2021-11-21T22:02:50.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- 172.21.2.116:0/3257321648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680abc00 0x7fdc680b13f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7fdc600032f0 tx=0x7fdc60009290).ready entity=mon.0 client_cookie=cd5f5681a6d6a915 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:50.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc66ffd700 1 -- 172.21.2.116:0/3257321648 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc6000b8a0 con 0x7fdc680abc00 2021-11-21T22:02:50.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc66ffd700 1 -- 172.21.2.116:0/3257321648 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000ba00 con 0x7fdc680abc00 2021-11-21T22:02:50.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc66ffd700 1 -- 172.21.2.116:0/3257321648 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc6000bb70 con 0x7fdc680abc00 2021-11-21T22:02:50.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/3257321648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680abc00 msgr2=0x7fdc680b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:50.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/3257321648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680abc00 0x7fdc680b13f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7fdc600032f0 tx=0x7fdc60009290).stop 2021-11-21T22:02:50.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/3257321648 shutdown_connections 2021-11-21T22:02:50.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/3257321648 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680adf80 0x7fdc680ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:50.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/3257321648 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680abc00 0x7fdc680b13f0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:50.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/3257321648 >> 172.21.2.116:0/3257321648 conn(0x7fdc680b5090 msgr2=0x7fdc680b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:50.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/3257321648 shutdown_connections 2021-11-21T22:02:50.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/3257321648 wait complete. 2021-11-21T22:02:50.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 Processor -- start 2021-11-21T22:02:50.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- start start 2021-11-21T22:02:50.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680abc00 0x7fdc6814ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:50.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680ac810 0x7fdc68064680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:50.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc680adf80 0x7fdc68064b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:50.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680656d0 con 0x7fdc680ac810 2021-11-21T22:02:50.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68066420 con 0x7fdc680adf80 2021-11-21T22:02:50.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68063700 con 0x7fdc680abc00 2021-11-21T22:02:50.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680ac810 0x7fdc68064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:50.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680ac810 0x7fdc68064680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53542/0 (socket says 172.21.2.116:53542) 2021-11-21T22:02:50.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/135464658 learned_addr learned my addr 172.21.2.116:0/135464658 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:50.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6cb5a700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc680adf80 0x7fdc68064b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:50.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc67fff700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680abc00 0x7fdc6814ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:50.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680abc00 msgr2=0x7fdc6814ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:50.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680abc00 0x7fdc6814ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:50.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc680adf80 msgr2=0x7fdc68064b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:50.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc680adf80 0x7fdc68064b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:50.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 -- 172.21.2.116:0/135464658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60009530 con 0x7fdc680ac810 2021-11-21T22:02:50.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6cb5a700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc680adf80 0x7fdc68064b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-21T22:02:50.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc677fe700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680ac810 0x7fdc68064680 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7fdc68143aa0 tx=0x7fdc60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:50.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc657fa700 1 -- 172.21.2.116:0/135464658 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc6000b8a0 con 0x7fdc680ac810 2021-11-21T22:02:50.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc657fa700 1 -- 172.21.2.116:0/135464658 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc60021630 con 0x7fdc680ac810 2021-11-21T22:02:50.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.039+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc68066650 con 0x7fdc680ac810 2021-11-21T22:02:50.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc657fa700 1 -- 172.21.2.116:0/135464658 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc60021990 con 0x7fdc680ac810 2021-11-21T22:02:50.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc68066b90 con 0x7fdc680ac810 2021-11-21T22:02:50.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc68064480 con 0x7fdc680ac810 2021-11-21T22:02:50.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc657fa700 1 -- 172.21.2.116:0/135464658 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdc6000f070 con 0x7fdc680ac810 2021-11-21T22:02:50.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc657fa700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc48053580 0x7fdc48055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:50.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc657fa700 1 -- 172.21.2.116:0/135464658 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fdc6006c9f0 con 0x7fdc680ac810 2021-11-21T22:02:50.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc67fff700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc48053580 0x7fdc48055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:50.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc67fff700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc48053580 0x7fdc48055a40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fdc5c00b000 tx=0x7fdc5c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:50.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.043+0000 7fdc657fa700 1 -- 172.21.2.116:0/135464658 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc60043be0 con 0x7fdc680ac810 2021-11-21T22:02:50.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.127+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdc680573d0 con 0x7fdc680ac810 2021-11-21T22:02:50.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.995+0000 7fdc657fa700 1 -- 172.21.2.116:0/135464658 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v144) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdc60015030 con 0x7fdc680ac810 2021-11-21T22:02:51.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc48053580 msgr2=0x7fdc48055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc48053580 0x7fdc48055a40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fdc5c00b000 tx=0x7fdc5c009040).stop 2021-11-21T22:02:51.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680ac810 msgr2=0x7fdc68064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680ac810 0x7fdc68064680 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7fdc68143aa0 tx=0x7fdc60016000).stop 2021-11-21T22:02:51.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 shutdown_connections 2021-11-21T22:02:51.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc680adf80 0x7fdc68064b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc48053580 0x7fdc48055a40 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc680ac810 0x7fdc68064680 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 --2- 172.21.2.116:0/135464658 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc680abc00 0x7fdc6814ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 >> 172.21.2.116:0/135464658 conn(0x7fdc680b5090 msgr2=0x7fdc680aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:51.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 shutdown_connections 2021-11-21T22:02:51.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:50.999+0000 7fdc6e5bd700 1 -- 172.21.2.116:0/135464658 wait complete. 2021-11-21T22:02:51.007 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:02:51.018 DEBUG:teuthology.orchestra.run.gibba016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:51.020 DEBUG:teuthology.orchestra.run.gibba017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:51.022 DEBUG:teuthology.orchestra.run.gibba019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:51.026 DEBUG:teuthology.orchestra.run.gibba036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-11-21T22:02:51.031 INFO:teuthology.orchestra.run.gibba017.stdout:[global] 2021-11-21T22:02:51.031 INFO:teuthology.orchestra.run.gibba017.stdout: chdir = "" 2021-11-21T22:02:51.032 INFO:teuthology.orchestra.run.gibba017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:51.032 INFO:teuthology.orchestra.run.gibba017.stdout: auth supported = cephx 2021-11-21T22:02:51.032 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.032 INFO:teuthology.orchestra.run.gibba017.stdout: filestore xattr use omap = true 2021-11-21T22:02:51.033 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.033 INFO:teuthology.orchestra.run.gibba017.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:51.033 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.033 INFO:teuthology.orchestra.run.gibba017.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:51.033 INFO:teuthology.orchestra.run.gibba017.stdout: auth debug = true 2021-11-21T22:02:51.034 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.034 INFO:teuthology.orchestra.run.gibba017.stdout: ms die on old message = true 2021-11-21T22:02:51.034 INFO:teuthology.orchestra.run.gibba017.stdout: ms die on bug = true 2021-11-21T22:02:51.034 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.035 INFO:teuthology.orchestra.run.gibba017.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:51.035 INFO:teuthology.orchestra.run.gibba017.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:51.035 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.036 INFO:teuthology.orchestra.run.gibba017.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:51.037 INFO:teuthology.orchestra.run.gibba017.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:51.037 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.037 INFO:teuthology.orchestra.run.gibba017.stdout: osd pool default size = 2 2021-11-21T22:02:51.038 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.038 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:51.038 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd allow pg remap = true 2021-11-21T22:02:51.038 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:51.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:51.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:51.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:51.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon warn on too few osds = false 2021-11-21T22:02:51.039 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:51.040 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:51.040 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.040 INFO:teuthology.orchestra.run.gibba017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:51.040 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.041 INFO:teuthology.orchestra.run.gibba017.stdout: osd default data pool replay window = 5 2021-11-21T22:02:51.041 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.041 INFO:teuthology.orchestra.run.gibba017.stdout: mon allow pool delete = true 2021-11-21T22:02:51.041 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.041 INFO:teuthology.orchestra.run.gibba017.stdout: mon cluster log file level = debug 2021-11-21T22:02:51.042 INFO:teuthology.orchestra.run.gibba017.stdout: debug asserts on shutdown = true 2021-11-21T22:02:51.042 INFO:teuthology.orchestra.run.gibba017.stdout: mon health detail to clog = false 2021-11-21T22:02:51.042 INFO:teuthology.orchestra.run.gibba017.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:51.042 INFO:teuthology.orchestra.run.gibba017.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:51.043 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.043 INFO:teuthology.orchestra.run.gibba017.stdout:[osd] 2021-11-21T22:02:51.043 INFO:teuthology.orchestra.run.gibba017.stdout: osd journal size = 100 2021-11-21T22:02:51.043 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.044 INFO:teuthology.orchestra.run.gibba017.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:51.044 INFO:teuthology.orchestra.run.gibba017.stdout: osd scrub max interval = 600 2021-11-21T22:02:51.044 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.045 INFO:teuthology.orchestra.run.gibba017.stdout: osd recover clone overlap = true 2021-11-21T22:02:51.045 INFO:teuthology.orchestra.run.gibba017.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:51.045 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.045 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug shutdown = true 2021-11-21T22:02:51.046 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug op order = true 2021-11-21T22:02:51.046 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:51.046 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.046 INFO:teuthology.orchestra.run.gibba017.stdout: osd open classes on start = true 2021-11-21T22:02:51.046 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug pg log writeout = true 2021-11-21T22:02:51.047 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.047 INFO:teuthology.orchestra.run.gibba017.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:51.047 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.047 INFO:teuthology.orchestra.run.gibba017.stdout: osd map max advance = 10 2021-11-21T22:02:51.048 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.048 INFO:teuthology.orchestra.run.gibba017.stdout: journal zero on create = true 2021-11-21T22:02:51.048 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.048 INFO:teuthology.orchestra.run.gibba017.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:51.049 INFO:teuthology.orchestra.run.gibba017.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:51.049 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.049 INFO:teuthology.orchestra.run.gibba017.stdout: bdev debug aio = true 2021-11-21T22:02:51.049 INFO:teuthology.orchestra.run.gibba017.stdout: osd debug misdirected ops = true 2021-11-21T22:02:51.049 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:51.050 INFO:teuthology.orchestra.run.gibba017.stdout: debug osd = 20 2021-11-21T22:02:51.050 INFO:teuthology.orchestra.run.gibba017.stdout: osd objectstore = filestore 2021-11-21T22:02:51.050 INFO:teuthology.orchestra.run.gibba017.stdout: osd op complaint time = 180 2021-11-21T22:02:51.050 INFO:teuthology.orchestra.run.gibba017.stdout: osd sloppy crc = True 2021-11-21T22:02:51.051 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.051 INFO:teuthology.orchestra.run.gibba017.stdout:[mgr] 2021-11-21T22:02:51.051 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:51.051 INFO:teuthology.orchestra.run.gibba017.stdout: debug mgr = 20 2021-11-21T22:02:51.051 INFO:teuthology.orchestra.run.gibba017.stdout: debug mon = 20 2021-11-21T22:02:51.052 INFO:teuthology.orchestra.run.gibba017.stdout: debug auth = 20 2021-11-21T22:02:51.052 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.052 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.052 INFO:teuthology.orchestra.run.gibba017.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:51.053 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.053 INFO:teuthology.orchestra.run.gibba017.stdout:[mon] 2021-11-21T22:02:51.053 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:51.053 INFO:teuthology.orchestra.run.gibba017.stdout: debug mon = 20 2021-11-21T22:02:51.053 INFO:teuthology.orchestra.run.gibba017.stdout: debug paxos = 20 2021-11-21T22:02:51.054 INFO:teuthology.orchestra.run.gibba017.stdout: debug auth = 20 2021-11-21T22:02:51.054 INFO:teuthology.orchestra.run.gibba017.stdout: mon data avail warn = 5 2021-11-21T22:02:51.055 INFO:teuthology.orchestra.run.gibba017.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:51.055 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.055 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:51.055 INFO:teuthology.orchestra.run.gibba017.stdout: mon osd prime pg temp = true 2021-11-21T22:02:51.056 INFO:teuthology.orchestra.run.gibba017.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.056 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.056 INFO:teuthology.orchestra.run.gibba017.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:51.056 INFO:teuthology.orchestra.run.gibba017.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:51.056 INFO:teuthology.orchestra.run.gibba017.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:51.057 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.057 INFO:teuthology.orchestra.run.gibba017.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:51.057 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:51.057 INFO:teuthology.orchestra.run.gibba017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:51.058 INFO:teuthology.orchestra.run.gibba017.stdout: mon op complaint time = 120 2021-11-21T22:02:51.058 INFO:teuthology.orchestra.run.gibba017.stdout: 2021-11-21T22:02:51.058 INFO:teuthology.orchestra.run.gibba017.stdout:[client] 2021-11-21T22:02:51.058 INFO:teuthology.orchestra.run.gibba017.stdout: rgw cache enabled = true 2021-11-21T22:02:51.059 INFO:teuthology.orchestra.run.gibba017.stdout: rgw enable ops log = true 2021-11-21T22:02:51.059 INFO:teuthology.orchestra.run.gibba017.stdout: rgw enable usage log = true 2021-11-21T22:02:51.059 INFO:teuthology.orchestra.run.gibba017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:51.059 INFO:teuthology.orchestra.run.gibba017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:51.060 INFO:teuthology.orchestra.run.gibba017.stdout: client mount timeout = 600 2021-11-21T22:02:51.060 INFO:teuthology.orchestra.run.gibba017.stdout: debug client = 20 2021-11-21T22:02:51.060 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:51.060 INFO:teuthology.orchestra.run.gibba017.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.060 INFO:teuthology.orchestra.run.gibba017.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.061 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.a] 2021-11-21T22:02:51.061 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.b] 2021-11-21T22:02:51.061 INFO:teuthology.orchestra.run.gibba017.stdout:[mon.c] 2021-11-21T22:02:51.061 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.a] 2021-11-21T22:02:51.062 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.c] 2021-11-21T22:02:51.062 INFO:teuthology.orchestra.run.gibba017.stdout:[mds.b] 2021-11-21T22:02:51.062 INFO:teuthology.orchestra.run.gibba017.stdout:[mds] 2021-11-21T22:02:51.062 INFO:teuthology.orchestra.run.gibba017.stdout: debug mds = 20 2021-11-21T22:02:51.063 INFO:teuthology.orchestra.run.gibba017.stdout: debug ms = 1 2021-11-21T22:02:51.063 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal frag = True 2021-11-21T22:02:51.063 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:51.063 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal merge size = 5 2021-11-21T22:02:51.064 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal split bits = 3 2021-11-21T22:02:51.064 INFO:teuthology.orchestra.run.gibba017.stdout: mds bal split size = 100 2021-11-21T22:02:51.064 INFO:teuthology.orchestra.run.gibba017.stdout: mds debug frag = True 2021-11-21T22:02:51.065 INFO:teuthology.orchestra.run.gibba017.stdout: mds debug scatterstat = True 2021-11-21T22:02:51.065 INFO:teuthology.orchestra.run.gibba017.stdout: mds op complaint time = 180 2021-11-21T22:02:51.065 INFO:teuthology.orchestra.run.gibba017.stdout: mds verify scatter = True 2021-11-21T22:02:51.065 INFO:teuthology.orchestra.run.gibba017.stdout: osd op complaint time = 180 2021-11-21T22:02:51.065 INFO:teuthology.orchestra.run.gibba017.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.066 INFO:teuthology.orchestra.run.gibba017.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.066 INFO:teuthology.orchestra.run.gibba016.stdout:[global] 2021-11-21T22:02:51.066 INFO:teuthology.orchestra.run.gibba016.stdout: chdir = "" 2021-11-21T22:02:51.066 INFO:teuthology.orchestra.run.gibba016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: auth supported = cephx 2021-11-21T22:02:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: filestore xattr use omap = true 2021-11-21T22:02:51.067 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:51.068 INFO:teuthology.orchestra.run.gibba016.stdout: auth debug = true 2021-11-21T22:02:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: ms die on old message = true 2021-11-21T22:02:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: ms die on bug = true 2021-11-21T22:02:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.069 INFO:teuthology.orchestra.run.gibba016.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:51.070 INFO:teuthology.orchestra.run.gibba016.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: osd pool default size = 2 2021-11-21T22:02:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:51.071 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd allow pg remap = true 2021-11-21T22:02:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:51.072 INFO:teuthology.orchestra.run.gibba016.stdout: mon warn on too few osds = false 2021-11-21T22:02:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.073 INFO:teuthology.orchestra.run.gibba016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: osd default data pool replay window = 5 2021-11-21T22:02:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.074 INFO:teuthology.orchestra.run.gibba016.stdout: mon allow pool delete = true 2021-11-21T22:02:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: mon cluster log file level = debug 2021-11-21T22:02:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: debug asserts on shutdown = true 2021-11-21T22:02:51.075 INFO:teuthology.orchestra.run.gibba016.stdout: mon health detail to clog = false 2021-11-21T22:02:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.076 INFO:teuthology.orchestra.run.gibba016.stdout:[osd] 2021-11-21T22:02:51.076 INFO:teuthology.orchestra.run.gibba016.stdout: osd journal size = 100 2021-11-21T22:02:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: osd scrub max interval = 600 2021-11-21T22:02:51.077 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.078 INFO:teuthology.orchestra.run.gibba016.stdout: osd recover clone overlap = true 2021-11-21T22:02:51.078 INFO:teuthology.orchestra.run.gibba016.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:51.078 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.079 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug shutdown = true 2021-11-21T22:02:51.079 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug op order = true 2021-11-21T22:02:51.079 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: osd open classes on start = true 2021-11-21T22:02:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug pg log writeout = true 2021-11-21T22:02:51.080 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: osd map max advance = 10 2021-11-21T22:02:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.081 INFO:teuthology.orchestra.run.gibba016.stdout: journal zero on create = true 2021-11-21T22:02:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.082 INFO:teuthology.orchestra.run.gibba016.stdout: bdev debug aio = true 2021-11-21T22:02:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: osd debug misdirected ops = true 2021-11-21T22:02:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: debug osd = 20 2021-11-21T22:02:51.083 INFO:teuthology.orchestra.run.gibba016.stdout: osd objectstore = filestore 2021-11-21T22:02:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: osd op complaint time = 180 2021-11-21T22:02:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: osd sloppy crc = True 2021-11-21T22:02:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.084 INFO:teuthology.orchestra.run.gibba016.stdout:[mgr] 2021-11-21T22:02:51.084 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: debug mgr = 20 2021-11-21T22:02:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: debug mon = 20 2021-11-21T22:02:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: debug auth = 20 2021-11-21T22:02:51.085 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.086 INFO:teuthology.orchestra.run.gibba016.stdout:[mon] 2021-11-21T22:02:51.086 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: debug mon = 20 2021-11-21T22:02:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: debug paxos = 20 2021-11-21T22:02:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: debug auth = 20 2021-11-21T22:02:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: mon data avail warn = 5 2021-11-21T22:02:51.087 INFO:teuthology.orchestra.run.gibba016.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: mon osd prime pg temp = true 2021-11-21T22:02:51.088 INFO:teuthology.orchestra.run.gibba016.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:51.089 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: mon op complaint time = 120 2021-11-21T22:02:51.090 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:51.091 INFO:teuthology.orchestra.run.gibba016.stdout:[client] 2021-11-21T22:02:51.091 INFO:teuthology.orchestra.run.gibba016.stdout: rgw cache enabled = true 2021-11-21T22:02:51.091 INFO:teuthology.orchestra.run.gibba016.stdout: rgw enable ops log = true 2021-11-21T22:02:51.091 INFO:teuthology.orchestra.run.gibba016.stdout: rgw enable usage log = true 2021-11-21T22:02:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: client mount timeout = 600 2021-11-21T22:02:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: debug client = 20 2021-11-21T22:02:51.092 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:51.093 INFO:teuthology.orchestra.run.gibba016.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.093 INFO:teuthology.orchestra.run.gibba016.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.093 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.a] 2021-11-21T22:02:51.093 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.b] 2021-11-21T22:02:51.093 INFO:teuthology.orchestra.run.gibba016.stdout:[mon.c] 2021-11-21T22:02:51.094 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.a] 2021-11-21T22:02:51.094 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.c] 2021-11-21T22:02:51.094 INFO:teuthology.orchestra.run.gibba016.stdout:[mds.b] 2021-11-21T22:02:51.094 INFO:teuthology.orchestra.run.gibba016.stdout:[mds] 2021-11-21T22:02:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: debug mds = 20 2021-11-21T22:02:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: debug ms = 1 2021-11-21T22:02:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal frag = True 2021-11-21T22:02:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:51.095 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal merge size = 5 2021-11-21T22:02:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal split bits = 3 2021-11-21T22:02:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: mds bal split size = 100 2021-11-21T22:02:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: mds debug frag = True 2021-11-21T22:02:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: mds debug scatterstat = True 2021-11-21T22:02:51.096 INFO:teuthology.orchestra.run.gibba016.stdout: mds op complaint time = 180 2021-11-21T22:02:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: mds verify scatter = True 2021-11-21T22:02:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: osd op complaint time = 180 2021-11-21T22:02:51.097 INFO:teuthology.orchestra.run.gibba016.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.098 INFO:teuthology.orchestra.run.gibba016.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.101 INFO:teuthology.orchestra.run.gibba036.stdout:[global] 2021-11-21T22:02:51.101 INFO:teuthology.orchestra.run.gibba036.stdout: chdir = "" 2021-11-21T22:02:51.101 INFO:teuthology.orchestra.run.gibba036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:51.102 INFO:teuthology.orchestra.run.gibba036.stdout: auth supported = cephx 2021-11-21T22:02:51.102 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.102 INFO:teuthology.orchestra.run.gibba036.stdout: filestore xattr use omap = true 2021-11-21T22:02:51.102 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.103 INFO:teuthology.orchestra.run.gibba036.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:51.103 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.103 INFO:teuthology.orchestra.run.gibba036.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:51.103 INFO:teuthology.orchestra.run.gibba036.stdout: auth debug = true 2021-11-21T22:02:51.104 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.104 INFO:teuthology.orchestra.run.gibba036.stdout: ms die on old message = true 2021-11-21T22:02:51.104 INFO:teuthology.orchestra.run.gibba036.stdout: ms die on bug = true 2021-11-21T22:02:51.104 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.104 INFO:teuthology.orchestra.run.gibba036.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:51.105 INFO:teuthology.orchestra.run.gibba036.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:51.105 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.105 INFO:teuthology.orchestra.run.gibba036.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:51.105 INFO:teuthology.orchestra.run.gibba036.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:51.106 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.106 INFO:teuthology.orchestra.run.gibba036.stdout: osd pool default size = 2 2021-11-21T22:02:51.106 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.106 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:51.107 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd allow pg remap = true 2021-11-21T22:02:51.107 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:51.107 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:51.107 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:51.108 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:51.108 INFO:teuthology.orchestra.run.gibba036.stdout: mon warn on too few osds = false 2021-11-21T22:02:51.108 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:51.108 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:51.109 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.109 INFO:teuthology.orchestra.run.gibba036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:51.109 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.109 INFO:teuthology.orchestra.run.gibba036.stdout: osd default data pool replay window = 5 2021-11-21T22:02:51.109 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.110 INFO:teuthology.orchestra.run.gibba036.stdout: mon allow pool delete = true 2021-11-21T22:02:51.110 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.110 INFO:teuthology.orchestra.run.gibba036.stdout: mon cluster log file level = debug 2021-11-21T22:02:51.110 INFO:teuthology.orchestra.run.gibba036.stdout: debug asserts on shutdown = true 2021-11-21T22:02:51.111 INFO:teuthology.orchestra.run.gibba036.stdout: mon health detail to clog = false 2021-11-21T22:02:51.111 INFO:teuthology.orchestra.run.gibba036.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:51.111 INFO:teuthology.orchestra.run.gibba036.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:51.111 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.112 INFO:teuthology.orchestra.run.gibba036.stdout:[osd] 2021-11-21T22:02:51.112 INFO:teuthology.orchestra.run.gibba036.stdout: osd journal size = 100 2021-11-21T22:02:51.112 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.112 INFO:teuthology.orchestra.run.gibba036.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:51.112 INFO:teuthology.orchestra.run.gibba036.stdout: osd scrub max interval = 600 2021-11-21T22:02:51.113 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.113 INFO:teuthology.orchestra.run.gibba036.stdout: osd recover clone overlap = true 2021-11-21T22:02:51.113 INFO:teuthology.orchestra.run.gibba036.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:51.113 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.114 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug shutdown = true 2021-11-21T22:02:51.114 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug op order = true 2021-11-21T22:02:51.114 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:51.114 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.115 INFO:teuthology.orchestra.run.gibba036.stdout: osd open classes on start = true 2021-11-21T22:02:51.115 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug pg log writeout = true 2021-11-21T22:02:51.115 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.115 INFO:teuthology.orchestra.run.gibba036.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:51.116 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.116 INFO:teuthology.orchestra.run.gibba036.stdout: osd map max advance = 10 2021-11-21T22:02:51.116 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.116 INFO:teuthology.orchestra.run.gibba036.stdout: journal zero on create = true 2021-11-21T22:02:51.117 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.117 INFO:teuthology.orchestra.run.gibba036.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:51.117 INFO:teuthology.orchestra.run.gibba036.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:51.117 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.117 INFO:teuthology.orchestra.run.gibba036.stdout: bdev debug aio = true 2021-11-21T22:02:51.118 INFO:teuthology.orchestra.run.gibba036.stdout: osd debug misdirected ops = true 2021-11-21T22:02:51.118 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:51.118 INFO:teuthology.orchestra.run.gibba036.stdout: debug osd = 20 2021-11-21T22:02:51.118 INFO:teuthology.orchestra.run.gibba036.stdout: osd objectstore = filestore 2021-11-21T22:02:51.119 INFO:teuthology.orchestra.run.gibba036.stdout: osd op complaint time = 180 2021-11-21T22:02:51.119 INFO:teuthology.orchestra.run.gibba036.stdout: osd sloppy crc = True 2021-11-21T22:02:51.119 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.119 INFO:teuthology.orchestra.run.gibba036.stdout:[mgr] 2021-11-21T22:02:51.119 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:51.120 INFO:teuthology.orchestra.run.gibba036.stdout: debug mgr = 20 2021-11-21T22:02:51.120 INFO:teuthology.orchestra.run.gibba036.stdout: debug mon = 20 2021-11-21T22:02:51.120 INFO:teuthology.orchestra.run.gibba036.stdout: debug auth = 20 2021-11-21T22:02:51.120 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.121 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.121 INFO:teuthology.orchestra.run.gibba036.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:51.121 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.121 INFO:teuthology.orchestra.run.gibba036.stdout:[mon] 2021-11-21T22:02:51.122 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:51.122 INFO:teuthology.orchestra.run.gibba036.stdout: debug mon = 20 2021-11-21T22:02:51.122 INFO:teuthology.orchestra.run.gibba036.stdout: debug paxos = 20 2021-11-21T22:02:51.122 INFO:teuthology.orchestra.run.gibba036.stdout: debug auth = 20 2021-11-21T22:02:51.122 INFO:teuthology.orchestra.run.gibba036.stdout: mon data avail warn = 5 2021-11-21T22:02:51.123 INFO:teuthology.orchestra.run.gibba036.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:51.123 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.123 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:51.123 INFO:teuthology.orchestra.run.gibba036.stdout: mon osd prime pg temp = true 2021-11-21T22:02:51.123 INFO:teuthology.orchestra.run.gibba036.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.124 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.124 INFO:teuthology.orchestra.run.gibba036.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:51.124 INFO:teuthology.orchestra.run.gibba036.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:51.124 INFO:teuthology.orchestra.run.gibba036.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:51.125 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.125 INFO:teuthology.orchestra.run.gibba036.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:51.125 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:51.125 INFO:teuthology.orchestra.run.gibba036.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:51.125 INFO:teuthology.orchestra.run.gibba036.stdout: mon op complaint time = 120 2021-11-21T22:02:51.126 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-11-21T22:02:51.126 INFO:teuthology.orchestra.run.gibba036.stdout:[client] 2021-11-21T22:02:51.126 INFO:teuthology.orchestra.run.gibba036.stdout: rgw cache enabled = true 2021-11-21T22:02:51.126 INFO:teuthology.orchestra.run.gibba036.stdout: rgw enable ops log = true 2021-11-21T22:02:51.127 INFO:teuthology.orchestra.run.gibba036.stdout: rgw enable usage log = true 2021-11-21T22:02:51.127 INFO:teuthology.orchestra.run.gibba036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:51.127 INFO:teuthology.orchestra.run.gibba036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:51.127 INFO:teuthology.orchestra.run.gibba036.stdout: client mount timeout = 600 2021-11-21T22:02:51.127 INFO:teuthology.orchestra.run.gibba036.stdout: debug client = 20 2021-11-21T22:02:51.128 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:51.128 INFO:teuthology.orchestra.run.gibba036.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.128 INFO:teuthology.orchestra.run.gibba036.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.128 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.a] 2021-11-21T22:02:51.129 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.b] 2021-11-21T22:02:51.129 INFO:teuthology.orchestra.run.gibba036.stdout:[mon.c] 2021-11-21T22:02:51.129 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.a] 2021-11-21T22:02:51.129 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.c] 2021-11-21T22:02:51.130 INFO:teuthology.orchestra.run.gibba036.stdout:[mds.b] 2021-11-21T22:02:51.130 INFO:teuthology.orchestra.run.gibba036.stdout:[mds] 2021-11-21T22:02:51.130 INFO:teuthology.orchestra.run.gibba036.stdout: debug mds = 20 2021-11-21T22:02:51.130 INFO:teuthology.orchestra.run.gibba036.stdout: debug ms = 1 2021-11-21T22:02:51.130 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal frag = True 2021-11-21T22:02:51.131 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:51.131 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal merge size = 5 2021-11-21T22:02:51.131 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal split bits = 3 2021-11-21T22:02:51.131 INFO:teuthology.orchestra.run.gibba036.stdout: mds bal split size = 100 2021-11-21T22:02:51.131 INFO:teuthology.orchestra.run.gibba036.stdout: mds debug frag = True 2021-11-21T22:02:51.132 INFO:teuthology.orchestra.run.gibba036.stdout: mds debug scatterstat = True 2021-11-21T22:02:51.132 INFO:teuthology.orchestra.run.gibba036.stdout: mds op complaint time = 180 2021-11-21T22:02:51.132 INFO:teuthology.orchestra.run.gibba036.stdout: mds verify scatter = True 2021-11-21T22:02:51.132 INFO:teuthology.orchestra.run.gibba036.stdout: osd op complaint time = 180 2021-11-21T22:02:51.133 INFO:teuthology.orchestra.run.gibba036.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.133 INFO:teuthology.orchestra.run.gibba036.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.133 INFO:teuthology.orchestra.run.gibba019.stdout:[global] 2021-11-21T22:02:51.133 INFO:teuthology.orchestra.run.gibba019.stdout: chdir = "" 2021-11-21T22:02:51.133 INFO:teuthology.orchestra.run.gibba019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-11-21T22:02:51.134 INFO:teuthology.orchestra.run.gibba019.stdout: auth supported = cephx 2021-11-21T22:02:51.134 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.134 INFO:teuthology.orchestra.run.gibba019.stdout: filestore xattr use omap = true 2021-11-21T22:02:51.134 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.135 INFO:teuthology.orchestra.run.gibba019.stdout: mon clock drift allowed = 1.000 2021-11-21T22:02:51.135 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.135 INFO:teuthology.orchestra.run.gibba019.stdout: osd crush chooseleaf type = 0 2021-11-21T22:02:51.135 INFO:teuthology.orchestra.run.gibba019.stdout: auth debug = true 2021-11-21T22:02:51.135 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.136 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on old message = true 2021-11-21T22:02:51.136 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on bug = true 2021-11-21T22:02:51.136 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.136 INFO:teuthology.orchestra.run.gibba019.stdout: mon max pg per osd = 10000 # >= luminous 2021-11-21T22:02:51.137 INFO:teuthology.orchestra.run.gibba019.stdout: mon pg warn max object skew = 0 2021-11-21T22:02:51.137 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.137 INFO:teuthology.orchestra.run.gibba019.stdout: # disable pg_autoscaler by default for new pools 2021-11-21T22:02:51.137 INFO:teuthology.orchestra.run.gibba019.stdout: osd_pool_default_pg_autoscale_mode = off 2021-11-21T22:02:51.137 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.138 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default size = 2 2021-11-21T22:02:51.138 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.138 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow primary affinity = true 2021-11-21T22:02:51.138 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow pg remap = true 2021-11-21T22:02:51.138 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on legacy crush tunables = false 2021-11-21T22:02:51.139 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on crush straw calc version zero = false 2021-11-21T22:02:51.139 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on no sortbitwise = false 2021-11-21T22:02:51.139 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on osd down out interval zero = false 2021-11-21T22:02:51.139 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on too few osds = false 2021-11-21T22:02:51.140 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-11-21T22:02:51.140 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_no_redundancy = false 2021-11-21T22:02:51.140 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.140 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-11-21T22:02:51.140 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.141 INFO:teuthology.orchestra.run.gibba019.stdout: osd default data pool replay window = 5 2021-11-21T22:02:51.141 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.141 INFO:teuthology.orchestra.run.gibba019.stdout: mon allow pool delete = true 2021-11-21T22:02:51.141 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.141 INFO:teuthology.orchestra.run.gibba019.stdout: mon cluster log file level = debug 2021-11-21T22:02:51.142 INFO:teuthology.orchestra.run.gibba019.stdout: debug asserts on shutdown = true 2021-11-21T22:02:51.142 INFO:teuthology.orchestra.run.gibba019.stdout: mon health detail to clog = false 2021-11-21T22:02:51.142 INFO:teuthology.orchestra.run.gibba019.stdout: mon host = "172.21.2.116,172.21.2.117,[v2:172.21.2.117:3301,v1:172.21.2.117:6790]" 2021-11-21T22:02:51.142 INFO:teuthology.orchestra.run.gibba019.stdout: fsid = fe3c4e1d-e687-4491-9a93-95ac7299e2b3 2021-11-21T22:02:51.143 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.143 INFO:teuthology.orchestra.run.gibba019.stdout:[osd] 2021-11-21T22:02:51.143 INFO:teuthology.orchestra.run.gibba019.stdout: osd journal size = 100 2021-11-21T22:02:51.143 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.143 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub load threshold = 5.0 2021-11-21T22:02:51.144 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub max interval = 600 2021-11-21T22:02:51.144 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.144 INFO:teuthology.orchestra.run.gibba019.stdout: osd recover clone overlap = true 2021-11-21T22:02:51.144 INFO:teuthology.orchestra.run.gibba019.stdout: osd recovery max chunk = 1048576 2021-11-21T22:02:51.144 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.145 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug shutdown = true 2021-11-21T22:02:51.145 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug op order = true 2021-11-21T22:02:51.145 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug verify stray on activate = true 2021-11-21T22:02:51.145 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.146 INFO:teuthology.orchestra.run.gibba019.stdout: osd open classes on start = true 2021-11-21T22:02:51.146 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug pg log writeout = true 2021-11-21T22:02:51.146 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.146 INFO:teuthology.orchestra.run.gibba019.stdout: osd deep scrub update digest min age = 30 2021-11-21T22:02:51.146 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.147 INFO:teuthology.orchestra.run.gibba019.stdout: osd map max advance = 10 2021-11-21T22:02:51.147 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.147 INFO:teuthology.orchestra.run.gibba019.stdout: journal zero on create = true 2021-11-21T22:02:51.147 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.147 INFO:teuthology.orchestra.run.gibba019.stdout: filestore ondisk finisher threads = 3 2021-11-21T22:02:51.148 INFO:teuthology.orchestra.run.gibba019.stdout: filestore apply finisher threads = 3 2021-11-21T22:02:51.148 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.148 INFO:teuthology.orchestra.run.gibba019.stdout: bdev debug aio = true 2021-11-21T22:02:51.148 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug misdirected ops = true 2021-11-21T22:02:51.149 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:51.149 INFO:teuthology.orchestra.run.gibba019.stdout: debug osd = 20 2021-11-21T22:02:51.149 INFO:teuthology.orchestra.run.gibba019.stdout: osd objectstore = filestore 2021-11-21T22:02:51.149 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-11-21T22:02:51.149 INFO:teuthology.orchestra.run.gibba019.stdout: osd sloppy crc = True 2021-11-21T22:02:51.150 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.150 INFO:teuthology.orchestra.run.gibba019.stdout:[mgr] 2021-11-21T22:02:51.150 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:51.150 INFO:teuthology.orchestra.run.gibba019.stdout: debug mgr = 20 2021-11-21T22:02:51.150 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-11-21T22:02:51.151 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-11-21T22:02:51.151 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.151 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.151 INFO:teuthology.orchestra.run.gibba019.stdout: mgr/telemetry/nag = false 2021-11-21T22:02:51.152 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.152 INFO:teuthology.orchestra.run.gibba019.stdout:[mon] 2021-11-21T22:02:51.152 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:51.152 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-11-21T22:02:51.152 INFO:teuthology.orchestra.run.gibba019.stdout: debug paxos = 20 2021-11-21T22:02:51.153 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-11-21T22:02:51.153 INFO:teuthology.orchestra.run.gibba019.stdout: mon data avail warn = 5 2021-11-21T22:02:51.153 INFO:teuthology.orchestra.run.gibba019.stdout: mon mgr mkfs grace = 240 2021-11-21T22:02:51.153 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-11-21T22:02:51.153 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd reporter subtree level = osd 2021-11-21T22:02:51.154 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd prime pg temp = true 2021-11-21T22:02:51.154 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-11-21T22:02:51.154 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.154 INFO:teuthology.orchestra.run.gibba019.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-11-21T22:02:51.155 INFO:teuthology.orchestra.run.gibba019.stdout: auth mon ticket ttl = 660 # 11m 2021-11-21T22:02:51.155 INFO:teuthology.orchestra.run.gibba019.stdout: auth service ticket ttl = 240 # 4m 2021-11-21T22:02:51.155 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.155 INFO:teuthology.orchestra.run.gibba019.stdout: # don't complain about insecure global_id in the test suite 2021-11-21T22:02:51.155 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-11-21T22:02:51.156 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-11-21T22:02:51.156 INFO:teuthology.orchestra.run.gibba019.stdout: mon op complaint time = 120 2021-11-21T22:02:51.156 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-11-21T22:02:51.156 INFO:teuthology.orchestra.run.gibba019.stdout:[client] 2021-11-21T22:02:51.157 INFO:teuthology.orchestra.run.gibba019.stdout: rgw cache enabled = true 2021-11-21T22:02:51.157 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable ops log = true 2021-11-21T22:02:51.157 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable usage log = true 2021-11-21T22:02:51.157 INFO:teuthology.orchestra.run.gibba019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-11-21T22:02:51.158 INFO:teuthology.orchestra.run.gibba019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-11-21T22:02:51.158 INFO:teuthology.orchestra.run.gibba019.stdout: client mount timeout = 600 2021-11-21T22:02:51.158 INFO:teuthology.orchestra.run.gibba019.stdout: debug client = 20 2021-11-21T22:02:51.159 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:51.159 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.160 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.160 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.a] 2021-11-21T22:02:51.161 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.b] 2021-11-21T22:02:51.161 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.c] 2021-11-21T22:02:51.161 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.a] 2021-11-21T22:02:51.162 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.c] 2021-11-21T22:02:51.162 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.b] 2021-11-21T22:02:51.162 INFO:teuthology.orchestra.run.gibba019.stdout:[mds] 2021-11-21T22:02:51.162 INFO:teuthology.orchestra.run.gibba019.stdout: debug mds = 20 2021-11-21T22:02:51.163 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-11-21T22:02:51.163 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal frag = True 2021-11-21T22:02:51.163 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal fragment size max = 10000 2021-11-21T22:02:51.163 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal merge size = 5 2021-11-21T22:02:51.163 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split bits = 3 2021-11-21T22:02:51.164 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split size = 100 2021-11-21T22:02:51.164 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug frag = True 2021-11-21T22:02:51.164 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug scatterstat = True 2021-11-21T22:02:51.164 INFO:teuthology.orchestra.run.gibba019.stdout: mds op complaint time = 180 2021-11-21T22:02:51.165 INFO:teuthology.orchestra.run.gibba019.stdout: mds verify scatter = True 2021-11-21T22:02:51.165 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-11-21T22:02:51.165 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-11-21T22:02:51.165 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-11-21T22:02:51.166 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T22:02:51.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.259+0000 7fbf6b238700 1 Processor -- start 2021-11-21T22:02:51.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.259+0000 7fbf6b238700 1 -- start start 2021-11-21T22:02:51.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbf64144810 0x7fbf641478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf640ac600 0x7fbf640aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf64147e80 con 0x7fbf640abce0 2021-11-21T22:02:51.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640b1490 con 0x7fbf64144810 2021-11-21T22:02:51.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640b1fd0 con 0x7fbf640ac600 2021-11-21T22:02:51.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbf64144810 0x7fbf641478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:51.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbf64144810 0x7fbf641478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53550/0 (socket says 172.21.2.116:53550) 2021-11-21T22:02:51.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 -- 172.21.2.116:0/3122862272 learned_addr learned my addr 172.21.2.116:0/3122862272 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:51.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 -- 172.21.2.116:0/3122862272 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf640ac600 msgr2=0x7fbf640aca90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:51.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 --2- 172.21.2.116:0/3122862272 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf640ac600 0x7fbf640aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 -- 172.21.2.116:0/3122862272 >> v1:172.21.2.116:6789/0 conn(0x7fbf640abce0 legacy=0x7fbf640b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 -- 172.21.2.116:0/3122862272 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640af5a0 con 0x7fbf64144810 2021-11-21T22:02:51.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf68fd4700 1 --2- 172.21.2.116:0/3122862272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf64144810 0x7fbf641478c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54006eb0).ready entity=mon.0 client_cookie=c6440a5320f6d0da server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:51.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf637fe700 1 -- 172.21.2.116:0/3122862272 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf5400d5c0 con 0x7fbf64144810 2021-11-21T22:02:51.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf637fe700 1 -- 172.21.2.116:0/3122862272 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf54000e90 con 0x7fbf64144810 2021-11-21T22:02:51.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf637fe700 1 -- 172.21.2.116:0/3122862272 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf5400d950 con 0x7fbf64144810 2021-11-21T22:02:51.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/3122862272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf64144810 msgr2=0x7fbf641478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- 172.21.2.116:0/3122862272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf64144810 0x7fbf641478c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54006eb0).stop 2021-11-21T22:02:51.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/3122862272 shutdown_connections 2021-11-21T22:02:51.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- 172.21.2.116:0/3122862272 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf640ac600 0x7fbf640aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- 172.21.2.116:0/3122862272 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf64144810 0x7fbf641478c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/3122862272 >> 172.21.2.116:0/3122862272 conn(0x7fbf6405bd20 msgr2=0x7fbf6405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:51.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/3122862272 shutdown_connections 2021-11-21T22:02:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/3122862272 wait complete. 2021-11-21T22:02:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 Processor -- start 2021-11-21T22:02:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- start start 2021-11-21T22:02:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf640abce0 0x7fbf64143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.274 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf640ac600 0x7fbf6414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf64144810 0x7fbf64144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6414a560 con 0x7fbf640ac600 2021-11-21T22:02:51.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf641467e0 con 0x7fbf640abce0 2021-11-21T22:02:51.275 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64146a80 con 0x7fbf64144810 2021-11-21T22:02:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf640ac600 0x7fbf6414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf640ac600 0x7fbf6414a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53558/0 (socket says 172.21.2.116:53558) 2021-11-21T22:02:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 -- 172.21.2.116:0/2395707711 learned_addr learned my addr 172.21.2.116:0/2395707711 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:51.277 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 -- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf64144810 msgr2=0x7fbf64144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf64144810 0x7fbf64144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 -- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf640abce0 msgr2=0x7fbf64143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf640abce0 0x7fbf64143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.278 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 -- 172.21.2.116:0/2395707711 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5400b1e0 con 0x7fbf640ac600 2021-11-21T22:02:51.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf63fff700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf640ac600 0x7fbf6414a070 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7fbf640b4e00 tx=0x7fbf4c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:51.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf61ffb700 1 -- 172.21.2.116:0/2395707711 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf4c00ce30 con 0x7fbf640ac600 2021-11-21T22:02:51.279 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf61ffb700 1 -- 172.21.2.116:0/2395707711 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c00f470 con 0x7fbf640ac600 2021-11-21T22:02:51.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf64149bd0 con 0x7fbf640ac600 2021-11-21T22:02:51.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf61ffb700 1 -- 172.21.2.116:0/2395707711 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf4c00f7d0 con 0x7fbf640ac600 2021-11-21T22:02:51.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf64146510 con 0x7fbf640ac600 2021-11-21T22:02:51.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.263+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf64146c80 con 0x7fbf640ac600 2021-11-21T22:02:51.280 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.267+0000 7fbf61ffb700 1 -- 172.21.2.116:0/2395707711 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbf4c019030 con 0x7fbf640ac600 2021-11-21T22:02:51.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.267+0000 7fbf61ffb700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf480536a0 0x7fbf48055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.267+0000 7fbf61ffb700 1 -- 172.21.2.116:0/2395707711 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbf4c06aac0 con 0x7fbf640ac600 2021-11-21T22:02:51.281 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.267+0000 7fbf68fd4700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf480536a0 0x7fbf48055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:51.282 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.267+0000 7fbf68fd4700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf480536a0 0x7fbf48055b60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:51.283 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.267+0000 7fbf61ffb700 1 -- 172.21.2.116:0/2395707711 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf4c041ea0 con 0x7fbf640ac600 2021-11-21T22:02:51.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.351+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbf64149d60 con 0x7fbf640ac600 2021-11-21T22:02:51.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.355+0000 7fbf61ffb700 1 -- 172.21.2.116:0/2395707711 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf4c0454c0 con 0x7fbf640ac600 2021-11-21T22:02:51.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf480536a0 msgr2=0x7fbf48055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf480536a0 0x7fbf48055b60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fbf54002f70 tx=0x7fbf54015000).stop 2021-11-21T22:02:51.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf640ac600 msgr2=0x7fbf6414a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf640ac600 0x7fbf6414a070 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7fbf640b4e00 tx=0x7fbf4c007c50).stop 2021-11-21T22:02:51.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 shutdown_connections 2021-11-21T22:02:51.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbf64144810 0x7fbf64144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbf480536a0 0x7fbf48055b60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbf640ac600 0x7fbf6414a070 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 --2- 172.21.2.116:0/2395707711 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbf640abce0 0x7fbf64143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 >> 172.21.2.116:0/2395707711 conn(0x7fbf6405bd20 msgr2=0x7fbf64140950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:51.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 shutdown_connections 2021-11-21T22:02:51.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.359+0000 7fbf6b238700 1 -- 172.21.2.116:0/2395707711 wait complete. 2021-11-21T22:02:51.368 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2021-11-21T22:02:51.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd890599700 1 Processor -- start 2021-11-21T22:02:51.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd890599700 1 -- start start 2021-11-21T22:02:51.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd890599700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a86b0 0x7fd88814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd890599700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8880a6e80 con 0x7fd8880a6aa0 2021-11-21T22:02:51.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd890599700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88814a7b0 con 0x7fd888062410 2021-11-21T22:02:51.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd890599700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880a8bc0 con 0x7fd8880a86b0 2021-11-21T22:02:51.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd88e335700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd8880a6aa0 0x7fd8881470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37288/0 (socket says 172.21.2.116:37288) 2021-11-21T22:02:51.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd88e335700 1 -- 172.21.2.116:0/3468136253 learned_addr learned my addr 172.21.2.116:0/3468136253 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:51.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596520528 0 0) 0x7fd8880a6e80 con 0x7fd8880a6aa0 2021-11-21T22:02:51.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.415+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd870003260 con 0x7fd8880a6aa0 2021-11-21T22:02:51.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3157623788 0 0) 0x7fd870003260 con 0x7fd8880a6aa0 2021-11-21T22:02:51.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a86b0 msgr2=0x7fd88814b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a86b0 0x7fd88814b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 >> v1:172.21.2.117:6789/0 conn(0x7fd888062410 legacy=0x7fd8881443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88814aa10 con 0x7fd8880a6aa0 2021-11-21T22:02:51.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fd87c001680 con 0x7fd8880a6aa0 2021-11-21T22:02:51.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 >> v1:172.21.2.116:6789/0 conn(0x7fd8880a6aa0 legacy=0x7fd8881470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd870003780 0x7fd870003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a86b0 0x7fd870004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870004ac0 0x7fd870004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880a8bc0 con 0x7fd870003780 2021-11-21T22:02:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87000b910 con 0x7fd870004ac0 2021-11-21T22:02:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87000bad0 con 0x7fd8880a86b0 2021-11-21T22:02:51.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88e335700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd870003780 0x7fd870003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870004ac0 0x7fd870004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88e335700 1 -- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a86b0 msgr2=0x7fd870004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88e335700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a86b0 0x7fd870004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88e335700 1 -- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870004ac0 msgr2=0x7fd870004f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88e335700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870004ac0 0x7fd870004f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88e335700 1 -- 172.21.2.116:0/3468136253 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870003260 con 0x7fd870003780 2021-11-21T22:02:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870004ac0 0x7fd870004f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:51.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88e335700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd870003780 0x7fd870003b80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fd888061b50 tx=0x7fd87c001e70).ready entity=mon.0 client_cookie=d10d2d7eac1a48fd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87c007170 con 0x7fd870003780 2021-11-21T22:02:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c0072d0 con 0x7fd870003780 2021-11-21T22:02:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88d333700 1 -- 172.21.2.116:0/3468136253 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87c00c8f0 con 0x7fd870003780 2021-11-21T22:02:51.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3468136253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd870003780 msgr2=0x7fd870003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 --2- 172.21.2.116:0/3468136253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd870003780 0x7fd870003b80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fd888061b50 tx=0x7fd87c001e70).stop 2021-11-21T22:02:51.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 -- 172.21.2.116:0/3468136253 reap_dead start 2021-11-21T22:02:51.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3468136253 shutdown_connections 2021-11-21T22:02:51.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3468136253 >> 172.21.2.116:0/3468136253 conn(0x7fd88805bd20 msgr2=0x7fd88805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:51.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3468136253 shutdown_connections 2021-11-21T22:02:51.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3468136253 wait complete. 2021-11-21T22:02:51.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 Processor -- start 2021-11-21T22:02:51.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- start start 2021-11-21T22:02:51.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870003780 0x7fd888143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a9260 0x7fd888156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd888156810 0x7fd888156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd888147ac0 con 0x7fd888156810 2021-11-21T22:02:51.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88815a4c0 con 0x7fd870003780 2021-11-21T22:02:51.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880a99f0 con 0x7fd8880a9260 2021-11-21T22:02:51.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd888156810 0x7fd888156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:51.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd888156810 0x7fd888156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53580/0 (socket says 172.21.2.116:53580) 2021-11-21T22:02:51.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 -- 172.21.2.116:0/3430657324 learned_addr learned my addr 172.21.2.116:0/3430657324 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:51.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 -- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a9260 msgr2=0x7fd888156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a9260 0x7fd888156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 -- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870003780 msgr2=0x7fd888143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:51.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870003780 0x7fd888143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:51.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 -- 172.21.2.116:0/3430657324 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880a9c90 con 0x7fd888156810 2021-11-21T22:02:51.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd88eb36700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd888156810 0x7fd888156c90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7fd87c0018e0 tx=0x7fd880008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:51.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd87b7fe700 1 -- 172.21.2.116:0/3430657324 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd880001520 con 0x7fd888156810 2021-11-21T22:02:51.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd87b7fe700 1 -- 172.21.2.116:0/3430657324 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd880001680 con 0x7fd888156810 2021-11-21T22:02:51.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88815a1d0 con 0x7fd888156810 2021-11-21T22:02:51.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd87b7fe700 1 -- 172.21.2.116:0/3430657324 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8800019e0 con 0x7fd888156810 2021-11-21T22:02:51.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888146090 con 0x7fd888156810 2021-11-21T22:02:51.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8880573d0 con 0x7fd888156810 2021-11-21T22:02:51.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.419+0000 7fd87b7fe700 1 -- 172.21.2.116:0/3430657324 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd880019030 con 0x7fd888156810 2021-11-21T22:02:51.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.423+0000 7fd87b7fe700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd868053580 0x7fd868055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:51.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.423+0000 7fd87b7fe700 1 -- 172.21.2.116:0/3430657324 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd88006b1d0 con 0x7fd888156810 2021-11-21T22:02:51.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.423+0000 7fd88e335700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd868053580 0x7fd868055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:51.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.423+0000 7fd88e335700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd868053580 0x7fd868055a40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fd888061b50 tx=0x7fd87c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:51.442 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.423+0000 7fd87b7fe700 1 -- 172.21.2.116:0/3430657324 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd880042610 con 0x7fd888156810 2021-11-21T22:02:51.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.535+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7fd88805db70 con 0x7fd888156810 2021-11-21T22:02:52.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd87b7fe700 1 -- 172.21.2.116:0/3430657324 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v377) v1 ==== 128+0+0 (secure 0 0 0) 0x7fd880045c30 con 0x7fd888156810 2021-11-21T22:02:52.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd868053580 msgr2=0x7fd868055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:52.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd868053580 0x7fd868055a40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fd888061b50 tx=0x7fd87c015000).stop 2021-11-21T22:02:52.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd888156810 msgr2=0x7fd888156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:52.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd888156810 0x7fd888156c90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7fd87c0018e0 tx=0x7fd880008040).stop 2021-11-21T22:02:52.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 shutdown_connections 2021-11-21T22:02:52.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd868053580 0x7fd868055a40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:52.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd888156810 0x7fd888156c90 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:52.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8880a9260 0x7fd888156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:52.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 --2- 172.21.2.116:0/3430657324 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd870003780 0x7fd888143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:52.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 >> 172.21.2.116:0/3430657324 conn(0x7fd88805bd20 msgr2=0x7fd888148fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:52.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 shutdown_connections 2021-11-21T22:02:52.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:51.999+0000 7fd890599700 1 -- 172.21.2.116:0/3430657324 wait complete. 2021-11-21T22:02:52.009 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2021-11-21T22:02:52.010 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:02:52.011 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2021-11-21T22:02:52.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 Processor -- start 2021-11-21T22:02:52.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- start start 2021-11-21T22:02:52.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a6aa0 0x7f965014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:52.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9650062410 0x7f96501443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:52.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96500a6e80 con 0x7f96500a86b0 2021-11-21T22:02:52.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96500a8bc0 con 0x7f9650062410 2021-11-21T22:02:52.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965014a7b0 con 0x7f96500a6aa0 2021-11-21T22:02:52.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9650062410 0x7f96501443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:52.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f9650062410 0x7f96501443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53584/0 (socket says 172.21.2.116:53584) 2021-11-21T22:02:52.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 -- 172.21.2.116:0/40101152 learned_addr learned my addr 172.21.2.116:0/40101152 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:52.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 -- 172.21.2.116:0/40101152 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a6aa0 msgr2=0x7f965014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:52.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 --2- 172.21.2.116:0/40101152 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a6aa0 0x7f965014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:52.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 -- 172.21.2.116:0/40101152 >> v1:172.21.2.116:6789/0 conn(0x7f96500a86b0 legacy=0x7f96501470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:52.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 -- 172.21.2.116:0/40101152 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965014aa10 con 0x7f9650062410 2021-11-21T22:02:52.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f9654907700 1 --2- 172.21.2.116:0/40101152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9650062410 0x7f96501443f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f96480032f0 tx=0x7f9648009290).ready entity=mon.0 client_cookie=6707d34f496633c3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:52.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f964effd700 1 -- 172.21.2.116:0/40101152 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f964800b8a0 con 0x7f9650062410 2021-11-21T22:02:52.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f964effd700 1 -- 172.21.2.116:0/40101152 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964800ba00 con 0x7f9650062410 2021-11-21T22:02:52.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f964effd700 1 -- 172.21.2.116:0/40101152 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f964800bb70 con 0x7f9650062410 2021-11-21T22:02:52.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- 172.21.2.116:0/40101152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9650062410 msgr2=0x7f96501443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:52.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- 172.21.2.116:0/40101152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9650062410 0x7f96501443f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f96480032f0 tx=0x7f9648009290).stop 2021-11-21T22:02:52.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- 172.21.2.116:0/40101152 shutdown_connections 2021-11-21T22:02:52.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- 172.21.2.116:0/40101152 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9650062410 0x7f96501443f0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:52.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- 172.21.2.116:0/40101152 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a6aa0 0x7f965014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:52.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- 172.21.2.116:0/40101152 >> 172.21.2.116:0/40101152 conn(0x7f965005bd20 msgr2=0x7f965005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:52.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- 172.21.2.116:0/40101152 shutdown_connections 2021-11-21T22:02:52.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- 172.21.2.116:0/40101152 wait complete. 2021-11-21T22:02:52.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 Processor -- start 2021-11-21T22:02:52.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- start start 2021-11-21T22:02:52.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9650062410 0x7f96500a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:52.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96500a6aa0 0x7f96500aa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:52.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a86b0 0x7f9650144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:52.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96500aa8d0 con 0x7f96500a6aa0 2021-11-21T22:02:52.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650156e70 con 0x7f9650062410 2021-11-21T22:02:52.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f965636a700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96500a9870 con 0x7f96500a86b0 2021-11-21T22:02:52.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f964f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96500a6aa0 0x7f96500aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:52.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.059+0000 7f964f7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96500a6aa0 0x7f96500aa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53594/0 (socket says 172.21.2.116:53594) 2021-11-21T22:02:52.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964f7fe700 1 -- 172.21.2.116:0/2264484598 learned_addr learned my addr 172.21.2.116:0/2264484598 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:52.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964f7fe700 1 -- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a86b0 msgr2=0x7f9650144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:52.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964f7fe700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a86b0 0x7f9650144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:52.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964f7fe700 1 -- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9650062410 msgr2=0x7f96500a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:52.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964f7fe700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9650062410 0x7f96500a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:52.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964f7fe700 1 -- 172.21.2.116:0/2264484598 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648009530 con 0x7f96500a6aa0 2021-11-21T22:02:52.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964f7fe700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96500a6aa0 0x7f96500aa120 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f9650061ab0 tx=0x7f9640007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:52.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964d7fa700 1 -- 172.21.2.116:0/2264484598 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f964000ce30 con 0x7f96500a6aa0 2021-11-21T22:02:52.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964d7fa700 1 -- 172.21.2.116:0/2264484598 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9640013450 con 0x7f96500a6aa0 2021-11-21T22:02:52.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96501570a0 con 0x7f96500a6aa0 2021-11-21T22:02:52.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964d7fa700 1 -- 172.21.2.116:0/2264484598 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96400137b0 con 0x7f96500a6aa0 2021-11-21T22:02:52.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501472e0 con 0x7f96500a6aa0 2021-11-21T22:02:52.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9650157230 con 0x7f96500a6aa0 2021-11-21T22:02:52.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964d7fa700 1 -- 172.21.2.116:0/2264484598 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9640010070 con 0x7f96500a6aa0 2021-11-21T22:02:52.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964d7fa700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9634053640 0x7f9634055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:52.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964d7fa700 1 -- 172.21.2.116:0/2264484598 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f964006ad00 con 0x7f96500a6aa0 2021-11-21T22:02:52.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964ffff700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9634053640 0x7f9634055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:52.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.063+0000 7f964ffff700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9634053640 0x7f9634055b00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f9638002f70 tx=0x7f9638006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:52.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.067+0000 7f964d7fa700 1 -- 172.21.2.116:0/2264484598 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f964006e030 con 0x7f96500a6aa0 2021-11-21T22:02:52.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:52.175+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f96500573d0 con 0x7f96500a6aa0 2021-11-21T22:02:53.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f964d7fa700 1 -- 172.21.2.116:0/2264484598 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v378) v1 ==== 130+0+0 (secure 0 0 0) 0x7f96500573d0 con 0x7f96500a6aa0 2021-11-21T22:02:53.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9634053640 msgr2=0x7f9634055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9634053640 0x7f9634055b00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f9638002f70 tx=0x7f9638006a80).stop 2021-11-21T22:02:53.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96500a6aa0 msgr2=0x7f96500aa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96500a6aa0 0x7f96500aa120 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f9650061ab0 tx=0x7f9640007bf0).stop 2021-11-21T22:02:53.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 shutdown_connections 2021-11-21T22:02:53.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f96500a86b0 0x7f9650144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9634053640 0x7f9634055b00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f96500a6aa0 0x7f96500aa120 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 --2- 172.21.2.116:0/2264484598 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9650062410 0x7f96500a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 >> 172.21.2.116:0/2264484598 conn(0x7f965005bd20 msgr2=0x7f9650148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 shutdown_connections 2021-11-21T22:02:53.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.003+0000 7f965636a700 1 -- 172.21.2.116:0/2264484598 wait complete. 2021-11-21T22:02:53.015 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T22:02:53.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9382399700 1 Processor -- start 2021-11-21T22:02:53.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9382399700 1 -- start start 2021-11-21T22:02:53.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9382399700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f937c0abce0 0x7f937c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9382399700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c0ac600 0x7f937c0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9382399700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937c147e80 con 0x7f937c144810 2021-11-21T22:02:53.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9382399700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937c0b1490 con 0x7f937c0abce0 2021-11-21T22:02:53.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9382399700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0b1fd0 con 0x7f937c0ac600 2021-11-21T22:02:53.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937b7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f937c0abce0 0x7f937c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f9380936700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c0ac600 0x7f937c0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937bfff700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f937c144810 0x7f937c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:36232/0 (socket says 172.21.2.116:36232) 2021-11-21T22:02:53.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937bfff700 1 -- 172.21.2.116:0/441958608 learned_addr learned my addr 172.21.2.116:0/441958608 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:53.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937b7fe700 1 -- 172.21.2.116:0/441958608 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c0ac600 msgr2=0x7f937c0aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:53.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937b7fe700 1 --2- 172.21.2.116:0/441958608 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c0ac600 0x7f937c0aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937b7fe700 1 -- 172.21.2.116:0/441958608 >> v1:172.21.2.117:6789/0 conn(0x7f937c144810 legacy=0x7f937c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937b7fe700 1 -- 172.21.2.116:0/441958608 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0af5a0 con 0x7f937c0abce0 2021-11-21T22:02:53.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937b7fe700 1 --2- 172.21.2.116:0/441958608 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 0x7f937c1478c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9364002f70 tx=0x7f9364006df0).ready entity=mon.1 client_cookie=a421fea4d7247e99 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937affd700 1 -- 172.21.2.116:0/441958608 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f936400d570 con 0x7f937c0abce0 2021-11-21T22:02:53.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937affd700 1 -- 172.21.2.116:0/441958608 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364000e90 con 0x7f937c0abce0 2021-11-21T22:02:53.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.067+0000 7f937affd700 1 -- 172.21.2.116:0/441958608 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f936400d8c0 con 0x7f937c0abce0 2021-11-21T22:02:53.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/441958608 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 msgr2=0x7f937c1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 --2- 172.21.2.116:0/441958608 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 0x7f937c1478c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9364002f70 tx=0x7f9364006df0).stop 2021-11-21T22:02:53.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/441958608 shutdown_connections 2021-11-21T22:02:53.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 --2- 172.21.2.116:0/441958608 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c0ac600 0x7f937c0aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 --2- 172.21.2.116:0/441958608 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 0x7f937c1478c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/441958608 >> 172.21.2.116:0/441958608 conn(0x7f937c05bd20 msgr2=0x7f937c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/441958608 shutdown_connections 2021-11-21T22:02:53.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/441958608 wait complete. 2021-11-21T22:02:53.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 Processor -- start 2021-11-21T22:02:53.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- start start 2021-11-21T22:02:53.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 0x7f937c0668c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f937c0ac600 0x7f937c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c144810 0x7f937c1488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0648e0 con 0x7f937c0ac600 2021-11-21T22:02:53.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0b4310 con 0x7f937c0abce0 2021-11-21T22:02:53.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0b3a30 con 0x7f937c144810 2021-11-21T22:02:53.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f937c0ac600 0x7f937c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f937c0ac600 0x7f937c0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53606/0 (socket says 172.21.2.116:53606) 2021-11-21T22:02:53.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 -- 172.21.2.116:0/3996631259 learned_addr learned my addr 172.21.2.116:0/3996631259 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:53.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937bfff700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 0x7f937c0668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9380936700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c144810 0x7f937c1488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 -- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c144810 msgr2=0x7f937c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c144810 0x7f937c1488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 -- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 msgr2=0x7f937c0668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 0x7f937c0668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 -- 172.21.2.116:0/3996631259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936400b180 con 0x7f937c0ac600 2021-11-21T22:02:53.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f937b7fe700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f937c0ac600 0x7f937c0b2a90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f937c064c80 tx=0x7f9364014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f93797fa700 1 -- 172.21.2.116:0/3996631259 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9364016070 con 0x7f937c0ac600 2021-11-21T22:02:53.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f93797fa700 1 -- 172.21.2.116:0/3996631259 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364013c60 con 0x7f937c0ac600 2021-11-21T22:02:53.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c0b4540 con 0x7f937c0ac600 2021-11-21T22:02:53.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f93797fa700 1 -- 172.21.2.116:0/3996631259 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f936401e450 con 0x7f937c0ac600 2021-11-21T22:02:53.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c0b0a70 con 0x7f937c0ac600 2021-11-21T22:02:53.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937c0b49e0 con 0x7f937c0ac600 2021-11-21T22:02:53.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f93797fa700 1 -- 172.21.2.116:0/3996631259 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9364025030 con 0x7f937c0ac600 2021-11-21T22:02:53.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f93797fa700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9360053580 0x7f9360055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.071+0000 7f93797fa700 1 -- 172.21.2.116:0/3996631259 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f936406c5c0 con 0x7f937c0ac600 2021-11-21T22:02:53.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.075+0000 7f937bfff700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9360053580 0x7f9360055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.075+0000 7f937bfff700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9360053580 0x7f9360055a40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f936c00d000 tx=0x7f936c000bc0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.075+0000 7f93797fa700 1 -- 172.21.2.116:0/3996631259 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9364043a00 con 0x7f937c0ac600 2021-11-21T22:02:53.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.159+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f937c0af060 con 0x7f937c0ac600 2021-11-21T22:02:53.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.163+0000 7f93797fa700 1 -- 172.21.2.116:0/3996631259 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9364023020 con 0x7f937c0ac600 2021-11-21T22:02:53.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.163+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9360053580 msgr2=0x7f9360055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.163+0000 7f9382399700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9360053580 0x7f9360055a40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f936c00d000 tx=0x7f936c000bc0).stop 2021-11-21T22:02:53.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.163+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f937c0ac600 msgr2=0x7f937c0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.163+0000 7f9382399700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f937c0ac600 0x7f937c0b2a90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f937c064c80 tx=0x7f9364014000).stop 2021-11-21T22:02:53.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 shutdown_connections 2021-11-21T22:02:53.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f937c144810 0x7f937c1488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9360053580 0x7f9360055a40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f937c0ac600 0x7f937c0b2a90 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 --2- 172.21.2.116:0/3996631259 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f937c0abce0 0x7f937c0668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 >> 172.21.2.116:0/3996631259 conn(0x7f937c05bd20 msgr2=0x7f937c0ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 shutdown_connections 2021-11-21T22:02:53.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.167+0000 7f9382399700 1 -- 172.21.2.116:0/3996631259 wait complete. 2021-11-21T22:02:53.175 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:02:53.176 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:53.183 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:53.191 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:53.205 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:02:53.219 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:02:53.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:53.220 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:53.221 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:02:53.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:53.225 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:02:53.226 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:02:53.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:02:53.227 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:02:53.227 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:02:53.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:02:53.232 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:02:53.232 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:02:53.233 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:02:53.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 Processor -- start 2021-11-21T22:02:53.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- start start 2021-11-21T22:02:53.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f81ac0b1a50 0x7f81ac060b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac144410 con 0x7f81ac141530 2021-11-21T22:02:53.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac1446b0 con 0x7f81ac060740 2021-11-21T22:02:53.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac144950 con 0x7f81ac0b1a50 2021-11-21T22:02:53.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b1252700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f81ac0b1a50 0x7f81ac060b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81abfff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f81ac141530 0x7f81ac067200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37338/0 (socket says 172.21.2.116:37338) 2021-11-21T22:02:53.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81abfff700 1 -- 172.21.2.116:0/2236314243 learned_addr learned my addr 172.21.2.116:0/2236314243 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:53.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b1252700 1 -- 172.21.2.116:0/2236314243 >> v1:172.21.2.117:6789/0 conn(0x7f81ac060740 legacy=0x7f81ac140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:53.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b1252700 1 -- 172.21.2.116:0/2236314243 >> v1:172.21.2.116:6789/0 conn(0x7f81ac141530 legacy=0x7f81ac067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b1252700 1 -- 172.21.2.116:0/2236314243 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac144bf0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81ab7fe700 1 -- 172.21.2.116:0/2236314243 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380713082 0 0) 0x7f81ac144410 con 0x7f81ac141530 2021-11-21T22:02:53.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b1252700 1 --2- 172.21.2.116:0/2236314243 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac060b20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f81ac063a90 tx=0x7f81a40091d0).ready entity=mon.0 client_cookie=bf2d2a9f5fe0c1e5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81ab7fe700 1 -- 172.21.2.116:0/2236314243 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a4009ea0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81ab7fe700 1 -- 172.21.2.116:0/2236314243 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400b860 con 0x7f81ac0b1a50 2021-11-21T22:02:53.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81ab7fe700 1 -- 172.21.2.116:0/2236314243 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a400b9d0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- 172.21.2.116:0/2236314243 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 msgr2=0x7f81ac060b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/2236314243 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac060b20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f81ac063a90 tx=0x7f81a40091d0).stop 2021-11-21T22:02:53.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- 172.21.2.116:0/2236314243 shutdown_connections 2021-11-21T22:02:53.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/2236314243 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac060b20 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- 172.21.2.116:0/2236314243 >> 172.21.2.116:0/2236314243 conn(0x7f81ac0b7b80 msgr2=0x7f81ac0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- 172.21.2.116:0/2236314243 shutdown_connections 2021-11-21T22:02:53.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- 172.21.2.116:0/2236314243 wait complete. 2021-11-21T22:02:53.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 Processor -- start 2021-11-21T22:02:53.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.303+0000 7f81b2cb5700 1 -- start start 2021-11-21T22:02:53.315 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81ac060740 0x7f81ac09bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac062890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81ac141530 0x7f81ac0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac063720 con 0x7f81ac0b1a50 2021-11-21T22:02:53.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac09c9a0 con 0x7f81ac141530 2021-11-21T22:02:53.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac1362a0 con 0x7f81ac060740 2021-11-21T22:02:53.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac062890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac062890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53620/0 (socket says 172.21.2.116:53620) 2021-11-21T22:02:53.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 -- 172.21.2.116:0/3816673290 learned_addr learned my addr 172.21.2.116:0/3816673290 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:53.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 -- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81ac060740 msgr2=0x7f81ac09bf40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:53.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81ac060740 0x7f81ac09bf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 -- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81ac141530 msgr2=0x7f81ac0ab240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:53.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81ac141530 0x7f81ac0ab240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 -- 172.21.2.116:0/3816673290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a4009470 con 0x7f81ac0b1a50 2021-11-21T22:02:53.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81abfff700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac062890 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f819c000ed0 tx=0x7f819c007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81a9ffb700 1 -- 172.21.2.116:0/3816673290 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f819c00ce20 con 0x7f81ac0b1a50 2021-11-21T22:02:53.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81a9ffb700 1 -- 172.21.2.116:0/3816673290 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c0072e0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81a9ffb700 1 -- 172.21.2.116:0/3816673290 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f819c005610 con 0x7f81ac0b1a50 2021-11-21T22:02:53.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac0b5bd0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.319 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac0b13b0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac09bce0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81a9ffb700 1 -- 172.21.2.116:0/3816673290 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f819c010070 con 0x7f81ac0b1a50 2021-11-21T22:02:53.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81a9ffb700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8190053520 0x7f81900559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81a9ffb700 1 -- 172.21.2.116:0/3816673290 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f819c06aea0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b0a51700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8190053520 0x7f81900559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.307+0000 7f81b0a51700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8190053520 0x7f81900559e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f8194002f90 tx=0x7f8194006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.311+0000 7f81a9ffb700 1 -- 172.21.2.116:0/3816673290 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f819c0421d0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.391+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f81ac0573d0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.391+0000 7f81a9ffb700 1 -- 172.21.2.116:0/3816673290 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) v1 ==== 81+0+28837 (secure 0 0 0) 0x7f819c0457f0 con 0x7f81ac0b1a50 2021-11-21T22:02:53.397 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:53.397 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:53.397 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 144, 2021-11-21T22:02:53.398 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:02:53.398 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:02:53.398 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:02:50.980855+0000", 2021-11-21T22:02:53.398 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:53.399 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:02:53.399 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:02:53.399 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:02:53.399 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:02:53.399 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:02:53.400 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:02:53.400 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:02:53.400 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:02:53.400 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:53.401 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:02:53.401 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:02:53.401 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:02:53.401 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:02:53.401 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:02:53.402 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 26, 2021-11-21T22:02:53.402 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:02:53.402 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:02:53.403 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:02:53.403 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:02:53.404 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:02:53.404 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.404 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:02:53.405 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:02:53.405 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:02:53.405 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:53.405 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:53.406 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:53.406 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:53.406 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:53.406 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:53.406 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:53.407 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:53.407 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:02:53.407 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:02:53.407 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:02:53.408 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:02:53.408 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:02:53.408 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:53.408 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:53.408 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:53.409 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:53.409 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:53.409 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:53.409 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:53.409 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.410 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:02:53.410 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:53.410 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:53.411 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:53.411 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:53.411 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:53.411 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:53.412 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:53.412 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:53.412 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:53.412 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:53.412 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:53.413 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:53.413 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:53.413 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:53.413 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:53.413 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:53.414 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:53.414 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:53.415 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:53.415 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:53.415 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:53.415 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:53.416 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:53.416 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:53.416 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:53.416 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:53.416 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.417 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:53.417 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:53.417 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:53.417 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:53.418 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:53.418 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:53.418 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:53.418 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:53.418 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:53.419 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:53.419 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:53.419 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:02:53.419 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:02:53.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.420 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:53.420 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:02:53.420 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.420 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.421 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.421 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:02:53.421 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:02:53.421 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:02:53.421 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:02:53.422 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:02:53.422 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:02:53.422 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:02:53.422 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:02:53.422 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:02:53.423 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:02:53.423 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:02:53.423 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:02:53.423 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:02:53.423 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:02:53.424 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:02:53.424 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:02:53.424 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:02:53.424 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:02:53.425 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:02:53.425 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:02:53.425 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:02:53.425 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:02:53.425 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:02:53.426 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.426 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:02:53.426 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:02:53.426 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:02:53.426 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:02:53.427 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:02:53.427 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:02:53.427 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:02:53.427 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:02:53.428 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:02:53.428 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:02:53.428 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:02:53.428 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:02:53.428 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:02:53.429 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:02:53.429 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:02:53.429 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:02:53.429 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:02:53.429 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:02:53.430 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:02:53.430 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:02:53.430 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:02:53.430 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:02:53.431 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:02:53.431 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:02:53.431 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:02:53.431 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:02:53.431 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:02:53.432 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.432 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:02:53.432 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:02:53.432 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:02:53.432 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:02:53.433 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:02:53.433 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:02:53.433 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:02:53.433 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:02:53.434 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:02:53.434 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:02:53.434 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:02:53.434 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:02:53.434 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:02:53.435 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:02:53.435 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.435 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.435 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:53.435 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:02:53.436 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.436 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:53.436 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:02:53.436 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.436 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.437 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.437 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.437 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.437 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.438 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.438 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 138, 2021-11-21T22:02:53.438 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.438 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.438 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.439 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.439 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.439 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.439 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:02:53.439 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.440 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.440 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.440 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.440 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:02:53.441 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.441 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.441 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.441 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.441 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.442 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.442 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.442 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.442 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:02:53.442 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.443 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.443 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.443 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.443 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:02:53.443 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.444 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.444 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.444 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.444 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.445 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.445 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.445 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.445 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:02:53.445 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.446 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.446 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.446 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.446 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:02:53.446 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.447 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.447 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.447 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.447 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.448 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.448 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.448 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.448 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:02:53.448 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.449 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.449 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.449 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:02:53.449 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:02:53.450 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.450 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.450 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.450 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:02:53.451 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:02:53.451 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:02:53.451 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:02:53.451 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.451 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.452 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.452 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.452 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.453 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:53.453 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:02:53.453 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.453 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.454 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.454 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.454 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.454 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.454 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.455 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:53.455 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.455 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.455 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.455 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.456 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.456 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.456 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:02:53.456 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.457 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.457 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.457 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.457 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:02:53.457 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.458 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.458 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.458 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.458 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.458 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.459 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.459 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.459 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:02:53.459 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.460 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.460 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.460 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.460 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:02:53.460 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.461 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.461 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.461 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.461 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.461 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.462 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.462 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:02:53.462 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.463 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.463 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.463 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.463 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:02:53.463 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.464 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.464 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.464 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.464 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.464 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.465 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.465 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.465 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:02:53.465 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.465 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.466 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.466 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.466 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:02:53.467 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:02:53.467 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.467 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.467 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.467 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:02:53.468 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:02:53.468 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:02:53.468 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:02:53.468 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.469 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.469 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.469 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.469 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.469 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.470 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:53.470 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:02:53.470 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.470 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.470 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.471 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.471 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.471 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.471 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.471 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:53.472 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.472 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.472 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.472 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.473 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.473 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.473 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:02:53.473 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.473 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.474 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.474 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:02:53.474 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.474 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.475 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.475 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.475 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.475 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.475 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.476 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.476 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:02:53.476 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.477 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.477 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.477 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:02:53.477 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.477 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.478 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.478 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.478 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.478 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.479 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.479 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:02:53.479 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.479 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.480 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.480 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:02:53.480 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.480 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.481 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.481 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.481 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.481 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.481 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.482 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.482 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:02:53.482 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.482 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.482 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.483 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.483 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:02:53.483 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:02:53.484 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.484 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.484 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.484 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:02:53.484 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:02:53.485 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:02:53.485 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:02:53.485 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.485 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.486 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.486 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.486 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.486 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.486 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:53.487 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:02:53.487 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.487 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.487 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.487 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.488 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.488 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.488 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.488 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 138, 2021-11-21T22:02:53.488 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.489 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.489 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.489 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.489 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.489 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.490 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:02:53.490 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.490 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.490 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.491 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.491 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:02:53.491 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.491 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.491 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.492 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.492 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.492 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.492 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.492 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.493 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:02:53.493 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.493 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.493 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.493 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.494 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:02:53.494 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.494 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.494 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.495 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.495 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.495 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.495 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.495 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.496 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:02:53.496 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.496 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.497 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.497 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.497 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:02:53.497 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.498 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.498 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.498 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.498 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.498 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.499 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.499 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:02:53.499 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.499 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.500 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.500 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.500 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:02:53.500 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:02:53.500 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.501 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.501 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.501 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:02:53.501 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:02:53.502 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:02:53.502 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:02:53.502 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.502 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.502 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.503 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.503 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.503 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.503 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:53.503 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:02:53.504 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.504 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.504 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.504 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.505 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.505 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.505 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.505 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:53.505 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.506 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.506 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.506 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.506 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.506 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.507 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:02:53.507 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.507 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.507 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.508 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.508 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:02:53.508 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.508 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.508 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.509 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.509 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.509 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.509 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:02:53.510 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.510 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.510 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.511 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.511 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:02:53.511 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.511 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.511 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.512 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.512 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.512 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.512 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.512 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.513 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:02:53.513 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.513 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.513 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.513 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.514 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:02:53.514 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.514 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.514 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.515 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.515 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.515 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.515 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.515 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.516 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:02:53.516 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.516 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.516 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.516 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.517 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:02:53.517 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:02:53.517 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.517 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.517 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.518 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:02:53.518 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:02:53.518 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:02:53.518 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:02:53.519 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.519 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.519 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.519 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.519 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.520 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.520 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:53.520 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:02:53.520 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.520 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.521 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.521 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.521 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.521 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.521 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.522 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 132, 2021-11-21T22:02:53.522 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.522 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.522 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.523 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.523 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.523 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.523 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:02:53.523 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.524 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.524 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.524 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.524 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:02:53.524 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.525 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.525 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.525 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.525 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.525 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.526 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.526 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.526 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:02:53.526 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.527 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.527 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:02:53.527 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.528 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.528 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.528 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.528 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.528 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.529 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.529 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.529 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:02:53.529 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.529 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.530 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.530 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.530 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:02:53.530 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.531 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.531 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.531 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.531 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.531 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.532 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.532 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.532 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:02:53.532 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.532 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.533 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.533 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.533 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:02:53.533 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:02:53.534 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.534 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.534 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.534 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:02:53.534 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:02:53.535 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:02:53.535 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:02:53.535 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.535 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.535 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.536 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.536 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.536 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.536 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:53.537 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:02:53.537 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.537 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.537 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.537 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.538 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.538 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.538 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.538 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:53.539 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.539 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.539 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.539 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.539 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.540 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.540 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:02:53.540 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.540 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.540 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.541 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.541 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:02:53.541 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.541 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.541 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.542 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.542 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.542 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.542 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.543 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.543 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:02:53.543 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.543 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.543 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.544 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:02:53.544 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.544 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.544 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.545 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.545 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.545 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.546 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.546 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:02:53.546 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.546 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.546 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.547 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.547 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:02:53.547 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.547 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.547 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.548 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.548 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.548 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.548 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.549 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.549 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:02:53.549 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.549 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.549 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.550 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.550 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:02:53.550 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:02:53.550 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.550 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.551 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.551 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:02:53.551 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:02:53.551 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:02:53.551 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:02:53.552 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.552 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.552 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.552 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.553 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.553 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.553 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:53.553 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:02:53.553 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:02:53.554 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:02:53.554 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:02:53.554 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:02:53.554 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:02:53.554 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:02:53.555 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:02:53.555 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 139, 2021-11-21T22:02:53.555 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:02:53.555 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:02:53.555 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:02:53.556 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.556 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.556 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.556 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:02:53.557 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.557 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.557 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.557 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.557 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:02:53.558 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.558 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.558 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.558 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.558 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:02:53.559 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.559 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.559 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.559 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:02:53.560 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.560 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.560 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.560 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.560 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:02:53.561 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.561 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.561 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.561 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.561 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:02:53.562 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.562 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.562 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.562 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:02:53.562 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.563 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.563 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.563 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.563 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:02:53.564 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.564 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.564 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.564 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.564 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:02:53.565 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:02:53.565 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.565 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:02:53.565 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:02:53.565 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.566 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.566 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.566 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:02:53.566 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:02:53.567 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:02:53.567 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.567 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.567 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.567 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:02:53.568 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:02:53.568 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:02:53.568 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:02:53.568 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:02:53.568 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:02:53.569 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:02:53.569 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:53.569 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.569 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:53.570 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:02:53.570 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.570 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:02:53.570 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.570 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.571 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.571 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.571 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.571 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:02:53.571 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.572 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.572 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.572 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:02:53.572 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.573 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.573 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.573 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.573 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.573 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:02:53.574 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.574 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.574 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.574 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:02:53.575 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.575 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.575 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.575 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.575 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.576 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:02:53.576 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.576 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.576 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.576 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:02:53.577 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.577 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.577 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.577 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.577 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.578 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:02:53.578 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.578 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.578 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.579 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:02:53.579 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.579 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.579 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.579 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.580 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.580 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:02:53.580 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.580 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.580 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.581 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:02:53.581 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.581 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.581 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.581 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.582 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.582 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:02:53.582 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.582 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.583 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.583 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:02:53.583 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.583 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.583 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.584 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.584 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.584 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:02:53.584 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.584 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.585 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:53.585 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:02:53.585 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:02:53.585 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:02:53.585 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:02:53.586 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:02:53.586 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:02:53.586 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:02:53.586 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:02:53.586 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.587 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:02:53.587 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:02:53.587 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:02:53.587 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:02:53.588 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:02:53.588 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:02:53.588 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/1832322648": "2021-11-22T22:02:47.686723+0000", 2021-11-21T22:02:53.588 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/1832322648": "2021-11-22T22:02:47.686723+0000" 2021-11-21T22:02:53.588 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.589 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:02:53.589 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:02:53.589 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:02:53.589 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:02:53.589 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:02:53.590 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:02:53.590 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:02:53.590 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:02:53.590 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:53.590 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:53.591 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:02:53.591 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:02:53.591 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:02:53.591 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:02:53.591 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:02:53.592 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:53.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8190053520 msgr2=0x7f81900559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8190053520 0x7f81900559e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f8194002f90 tx=0x7f8194006930).stop 2021-11-21T22:02:53.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 msgr2=0x7f81ac062890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac062890 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f819c000ed0 tx=0x7f819c007ce0).stop 2021-11-21T22:02:53.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 shutdown_connections 2021-11-21T22:02:53.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f81ac141530 0x7f81ac0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8190053520 0x7f81900559e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f81ac0b1a50 0x7f81ac062890 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 --2- 172.21.2.116:0/3816673290 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f81ac060740 0x7f81ac09bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 >> 172.21.2.116:0/3816673290 conn(0x7f81ac0b7b80 msgr2=0x7f81ac0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 shutdown_connections 2021-11-21T22:02:53.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.395+0000 7f81b2cb5700 1 -- 172.21.2.116:0/3816673290 wait complete. 2021-11-21T22:02:53.597 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:53.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 Processor -- start 2021-11-21T22:02:53.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- start start 2021-11-21T22:02:53.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1ae0096bd0 0x7f1ae0096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1ae0141730 0x7f1ae00974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae0140b20 con 0x7f1ae0098c90 2021-11-21T22:02:53.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae00b2340 con 0x7f1ae0096bd0 2021-11-21T22:02:53.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae0140d50 con 0x7f1ae0141730 2021-11-21T22:02:53.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1ae0096bd0 0x7f1ae0096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1ae0096bd0 0x7f1ae0096fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53626/0 (socket says 172.21.2.116:53626) 2021-11-21T22:02:53.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/3491186967 learned_addr learned my addr 172.21.2.116:0/3491186967 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:53.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/3491186967 >> v2:172.21.2.117:3300/0 conn(0x7f1ae0141730 msgr2=0x7f1ae00974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:53.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- 172.21.2.116:0/3491186967 >> v2:172.21.2.117:3300/0 conn(0x7f1ae0141730 0x7f1ae00974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/3491186967 >> v1:172.21.2.116:6789/0 conn(0x7f1ae0098c90 legacy=0x7f1ae00b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/3491186967 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0140ff0 con 0x7f1ae0096bd0 2021-11-21T22:02:53.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- 172.21.2.116:0/3491186967 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0096fb0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f1ad0002f70 tx=0x7f1ad0006e50).ready entity=mon.0 client_cookie=71ac1eedef9677a4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1add59a700 1 -- 172.21.2.116:0/3491186967 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad000d610 con 0x7f1ae0096bd0 2021-11-21T22:02:53.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1add59a700 1 -- 172.21.2.116:0/3491186967 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0000e90 con 0x7f1ae0096bd0 2021-11-21T22:02:53.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1add59a700 1 -- 172.21.2.116:0/3491186967 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad000d9a0 con 0x7f1ae0096bd0 2021-11-21T22:02:53.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- 172.21.2.116:0/3491186967 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 msgr2=0x7f1ae0096fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3491186967 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0096fb0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f1ad0002f70 tx=0x7f1ad0006e50).stop 2021-11-21T22:02:53.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- 172.21.2.116:0/3491186967 shutdown_connections 2021-11-21T22:02:53.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3491186967 >> v2:172.21.2.117:3300/0 conn(0x7f1ae0141730 0x7f1ae00974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- 172.21.2.116:0/3491186967 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0096fb0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- 172.21.2.116:0/3491186967 >> 172.21.2.116:0/3491186967 conn(0x7f1ae00b44c0 msgr2=0x7f1ae005bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- 172.21.2.116:0/3491186967 shutdown_connections 2021-11-21T22:02:53.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- 172.21.2.116:0/3491186967 wait complete. 2021-11-21T22:02:53.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 Processor -- start 2021-11-21T22:02:53.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- start start 2021-11-21T22:02:53.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0098c90 0x7f1ae01598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae0141730 0x7f1ae0153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae00ac420 con 0x7f1ae0096bd0 2021-11-21T22:02:53.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae009b9a0 con 0x7f1ae0098c90 2021-11-21T22:02:53.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1adf59e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae009a310 con 0x7f1ae0141730 2021-11-21T22:02:53.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53632/0 (socket says 172.21.2.116:53632) 2021-11-21T22:02:53.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/4243152744 learned_addr learned my addr 172.21.2.116:0/4243152744 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:53.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae0141730 msgr2=0x7f1ae0153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae0141730 0x7f1ae0153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0098c90 msgr2=0x7f1ae01598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0098c90 0x7f1ae01598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.651+0000 7f1ade59c700 1 -- 172.21.2.116:0/4243152744 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad000b1e0 con 0x7f1ae0096bd0 2021-11-21T22:02:53.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1ade59c700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0142650 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f1ad0002f70 tx=0x7f1ad0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1acf7fe700 1 -- 172.21.2.116:0/4243152744 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad0006b80 con 0x7f1ae0096bd0 2021-11-21T22:02:53.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1acf7fe700 1 -- 172.21.2.116:0/4243152744 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000d290 con 0x7f1ae0096bd0 2021-11-21T22:02:53.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae009a070 con 0x7f1ae0096bd0 2021-11-21T22:02:53.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1acf7fe700 1 -- 172.21.2.116:0/4243152744 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad001e460 con 0x7f1ae0096bd0 2021-11-21T22:02:53.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae00abaa0 con 0x7f1ae0096bd0 2021-11-21T22:02:53.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae00573d0 con 0x7f1ae0096bd0 2021-11-21T22:02:53.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1acf7fe700 1 -- 172.21.2.116:0/4243152744 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1ad0025030 con 0x7f1ae0096bd0 2021-11-21T22:02:53.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1acf7fe700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1abc053580 0x7f1abc055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1acf7fe700 1 -- 172.21.2.116:0/4243152744 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1ad006ca00 con 0x7f1ae0096bd0 2021-11-21T22:02:53.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1addd9b700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1abc053580 0x7f1abc055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1addd9b700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1abc053580 0x7f1abc055a40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f1ad4002f70 tx=0x7f1ad4006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.655+0000 7f1acf7fe700 1 -- 172.21.2.116:0/4243152744 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ad0043d30 con 0x7f1ae0096bd0 2021-11-21T22:02:53.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ae01413f0 con 0x7f1ae0096bd0 2021-11-21T22:02:53.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1acf7fe700 1 -- 172.21.2.116:0/4243152744 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f1ad0023020 con 0x7f1ae0096bd0 2021-11-21T22:02:53.767 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:53.768 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2009161131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2009161131},{"type":"v1","addr":"172.21.2.116:6835","nonce":2009161131}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67}],"filesystems":[]} 2021-11-21T22:02:53.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1abc053580 msgr2=0x7f1abc055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1abc053580 0x7f1abc055a40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f1ad4002f70 tx=0x7f1ad4006a80).stop 2021-11-21T22:02:53.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 msgr2=0x7f1ae0142650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0142650 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f1ad0002f70 tx=0x7f1ad0015000).stop 2021-11-21T22:02:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 shutdown_connections 2021-11-21T22:02:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ae0141730 0x7f1ae0153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ae0098c90 0x7f1ae01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1abc053580 0x7f1abc055a40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 --2- 172.21.2.116:0/4243152744 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ae0096bd0 0x7f1ae0142650 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 >> 172.21.2.116:0/4243152744 conn(0x7f1ae00b44c0 msgr2=0x7f1ae00aa270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 shutdown_connections 2021-11-21T22:02:53.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.763+0000 7f1adf59e700 1 -- 172.21.2.116:0/4243152744 wait complete. 2021-11-21T22:02:53.773 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 67 2021-11-21T22:02:53.775 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:53.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 Processor -- start 2021-11-21T22:02:53.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- start start 2021-11-21T22:02:53.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fcff00a86b0 0x7fcff014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcff0062410 0x7fcff01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff00a6e80 con 0x7fcff00a6aa0 2021-11-21T22:02:53.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff00a8bc0 con 0x7fcff0062410 2021-11-21T22:02:53.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff014a7b0 con 0x7fcff00a86b0 2021-11-21T22:02:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcff0062410 0x7fcff01443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fcff0062410 0x7fcff01443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53644/0 (socket says 172.21.2.116:53644) 2021-11-21T22:02:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 -- 172.21.2.116:0/1299091852 learned_addr learned my addr 172.21.2.116:0/1299091852 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:53.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 -- 172.21.2.116:0/1299091852 >> v2:172.21.2.117:3300/0 conn(0x7fcff00a86b0 msgr2=0x7fcff014b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:53.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 --2- 172.21.2.116:0/1299091852 >> v2:172.21.2.117:3300/0 conn(0x7fcff00a86b0 0x7fcff014b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 -- 172.21.2.116:0/1299091852 >> v1:172.21.2.117:6789/0 conn(0x7fcff00a6aa0 legacy=0x7fcff01470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:53.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 -- 172.21.2.116:0/1299091852 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff014aa10 con 0x7fcff0062410 2021-11-21T22:02:53.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff6b15700 1 --2- 172.21.2.116:0/1299091852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff0062410 0x7fcff01443f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7fcff0061ab0 tx=0x7fcfe8009290).ready entity=mon.0 client_cookie=62f46cd7ae5b6a8a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff5312700 1 -- 172.21.2.116:0/1299091852 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe800b830 con 0x7fcff0062410 2021-11-21T22:02:53.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff5312700 1 -- 172.21.2.116:0/1299091852 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe800b990 con 0x7fcff0062410 2021-11-21T22:02:53.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff5312700 1 -- 172.21.2.116:0/1299091852 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe800bb40 con 0x7fcff0062410 2021-11-21T22:02:53.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- 172.21.2.116:0/1299091852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff0062410 msgr2=0x7fcff01443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.833 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 --2- 172.21.2.116:0/1299091852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff0062410 0x7fcff01443f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7fcff0061ab0 tx=0x7fcfe8009290).stop 2021-11-21T22:02:53.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- 172.21.2.116:0/1299091852 shutdown_connections 2021-11-21T22:02:53.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 --2- 172.21.2.116:0/1299091852 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff0062410 0x7fcff01443f0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 --2- 172.21.2.116:0/1299091852 >> v2:172.21.2.117:3300/0 conn(0x7fcff00a86b0 0x7fcff014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.834 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- 172.21.2.116:0/1299091852 >> 172.21.2.116:0/1299091852 conn(0x7fcff005bd20 msgr2=0x7fcff005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- 172.21.2.116:0/1299091852 shutdown_connections 2021-11-21T22:02:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- 172.21.2.116:0/1299091852 wait complete. 2021-11-21T22:02:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 Processor -- start 2021-11-21T22:02:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.823+0000 7fcff8578700 1 -- start start 2021-11-21T22:02:53.835 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff0062410 0x7fcff00641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff00a6aa0 0x7fcff00646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff00a86b0 0x7fcff0066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0064bb0 con 0x7fcff00a6aa0 2021-11-21T22:02:53.836 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff01571a0 con 0x7fcff0062410 2021-11-21T22:02:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0144f50 con 0x7fcff00a86b0 2021-11-21T22:02:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff00a6aa0 0x7fcff00646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff00a6aa0 0x7fcff00646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53648/0 (socket says 172.21.2.116:53648) 2021-11-21T22:02:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 -- 172.21.2.116:0/1745749403 learned_addr learned my addr 172.21.2.116:0/1745749403 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:53.837 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 -- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff00a86b0 msgr2=0x7fcff0066d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff00a86b0 0x7fcff0066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 -- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff0062410 msgr2=0x7fcff00641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff0062410 0x7fcff00641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.838 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 -- 172.21.2.116:0/1745749403 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8009530 con 0x7fcff00a6aa0 2021-11-21T22:02:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff5b13700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff00a6aa0 0x7fcff00646c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7fcff0063f80 tx=0x7fcfe0006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcfdf7fe700 1 -- 172.21.2.116:0/1745749403 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe0012410 con 0x7fcff00a6aa0 2021-11-21T22:02:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcfdf7fe700 1 -- 172.21.2.116:0/1745749403 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe0012570 con 0x7fcff00a6aa0 2021-11-21T22:02:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcfdf7fe700 1 -- 172.21.2.116:0/1745749403 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe00128d0 con 0x7fcff00a6aa0 2021-11-21T22:02:53.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff01573d0 con 0x7fcff00a6aa0 2021-11-21T22:02:53.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff01451f0 con 0x7fcff00a6aa0 2021-11-21T22:02:53.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff00573d0 con 0x7fcff00a6aa0 2021-11-21T22:02:53.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcfdf7fe700 1 -- 172.21.2.116:0/1745749403 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fcfe000f030 con 0x7fcff00a6aa0 2021-11-21T22:02:53.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcfdf7fe700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd00535b0 0x7fcfd0055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:53.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcfdf7fe700 1 -- 172.21.2.116:0/1745749403 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcfe006b350 con 0x7fcff00a6aa0 2021-11-21T22:02:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.827+0000 7fcff6314700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd00535b0 0x7fcfd0055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.831+0000 7fcff6314700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd00535b0 0x7fcfd0055a70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fcfec002fb0 tx=0x7fcfec0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:53.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.831+0000 7fcfdf7fe700 1 -- 172.21.2.116:0/1745749403 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcfe0042790 con 0x7fcff00a6aa0 2021-11-21T22:02:53.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.939+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcff014b470 con 0x7fcff00a6aa0 2021-11-21T22:02:53.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.939+0000 7fcfdf7fe700 1 -- 172.21.2.116:0/1745749403 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fcfe0045db0 con 0x7fcff00a6aa0 2021-11-21T22:02:53.945 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:53.945 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/2735390614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2735390614},{"type":"v1","addr":"172.21.2.117:6835","nonce":2735390614}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/2577058045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":2577058045},{"type":"v1","addr":"172.21.2.116:6833","nonce":2577058045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":63},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6835/2009161131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2009161131},{"type":"v1","addr":"172.21.2.116:6835","nonce":2009161131}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67}],"filesystems":[]} 2021-11-21T22:02:53.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd00535b0 msgr2=0x7fcfd0055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd00535b0 0x7fcfd0055a70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fcfec002fb0 tx=0x7fcfec0069b0).stop 2021-11-21T22:02:53.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff00a6aa0 msgr2=0x7fcff00646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:53.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff00a6aa0 0x7fcff00646c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7fcff0063f80 tx=0x7fcfe0006d50).stop 2021-11-21T22:02:53.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 shutdown_connections 2021-11-21T22:02:53.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fcff00a86b0 0x7fcff0066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fcfd00535b0 0x7fcfd0055a70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fcff00a6aa0 0x7fcff00646c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:53.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 --2- 172.21.2.116:0/1745749403 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fcff0062410 0x7fcff00641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:53.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 >> 172.21.2.116:0/1745749403 conn(0x7fcff005bd20 msgr2=0x7fcff01441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:53.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 shutdown_connections 2021-11-21T22:02:53.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:53.943+0000 7fcff8578700 1 -- 172.21.2.116:0/1745749403 wait complete. 2021-11-21T22:02:53.949 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 67 2021-11-21T22:02:53.952 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T22:02:53.952 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T22:02:53.953 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:02:53.953 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:02:53.954 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T22:02:53.954 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T22:02:53.954 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:02:53.954 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:02:53.955 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T22:02:53.955 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T22:02:53.955 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:02:53.956 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:02:53.958 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:02:53.959 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T22:02:53.961 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:02:53.961 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T22:02:53.962 INFO:tasks.ceph.mds.a:Started 2021-11-21T22:02:53.963 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:53.964 INFO:tasks.ceph.mds.c:Started 2021-11-21T22:02:53.964 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:53.966 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:02:53.967 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T22:02:53.969 INFO:tasks.ceph.mds.b:Started 2021-11-21T22:02:53.969 DEBUG:teuthology.parallel:result is None 2021-11-21T22:02:53.970 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T22:02:53.995 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:02:53.995 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:02:54.004 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T22:02:54.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 Processor -- start 2021-11-21T22:02:54.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- start start 2021-11-21T22:02:54.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff0fc0abce0 0x7ff0fc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:54.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc0ac600 0x7ff0fc0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:54.027 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc147e80 con 0x7ff0fc144810 2021-11-21T22:02:54.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc0b1490 con 0x7ff0fc0abce0 2021-11-21T22:02:54.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc0b1fd0 con 0x7ff0fc0ac600 2021-11-21T22:02:54.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0faffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff0fc144810 0x7ff0fc0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37402/0 (socket says 172.21.2.116:37402) 2021-11-21T22:02:54.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0faffd700 1 -- 172.21.2.116:0/3151388169 learned_addr learned my addr 172.21.2.116:0/3151388169 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:54.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0fb7fe700 1 --2- 172.21.2.116:0/3151388169 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc0ac600 0x7ff0fc0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:54.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0fa7fc700 1 --2- 172.21.2.116:0/3151388169 >> v2:172.21.2.116:3300/0 conn(0x7ff0fc0abce0 0x7ff0fc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:54.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0f9ffb700 1 -- 172.21.2.116:0/3151388169 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676640850 0 0) 0x7ff0fc147e80 con 0x7ff0fc144810 2021-11-21T22:02:54.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0f9ffb700 1 -- 172.21.2.116:0/3151388169 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e0003260 con 0x7ff0fc144810 2021-11-21T22:02:54.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0fb7fe700 1 -- 172.21.2.116:0/3151388169 >> v2:172.21.2.116:3300/0 conn(0x7ff0fc0abce0 msgr2=0x7ff0fc1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:54.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0fb7fe700 1 --2- 172.21.2.116:0/3151388169 >> v2:172.21.2.116:3300/0 conn(0x7ff0fc0abce0 0x7ff0fc1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:54.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0fb7fe700 1 -- 172.21.2.116:0/3151388169 >> v1:172.21.2.116:6789/0 conn(0x7ff0fc144810 legacy=0x7ff0fc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:54.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0fb7fe700 1 -- 172.21.2.116:0/3151388169 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc0af5a0 con 0x7ff0fc0ac600 2021-11-21T22:02:54.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0fb7fe700 1 --2- 172.21.2.116:0/3151388169 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc0ac600 0x7ff0fc0aca90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ff0f40032f0 tx=0x7ff0f40091d0).ready entity=mon.2 client_cookie=9a8a7e300a9d73f0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:54.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0f9ffb700 1 -- 172.21.2.116:0/3151388169 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f400b600 con 0x7ff0fc0ac600 2021-11-21T22:02:54.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0f9ffb700 1 -- 172.21.2.116:0/3151388169 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f400b760 con 0x7ff0fc0ac600 2021-11-21T22:02:54.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff0f9ffb700 1 -- 172.21.2.116:0/3151388169 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f400b8d0 con 0x7ff0fc0ac600 2021-11-21T22:02:54.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- 172.21.2.116:0/3151388169 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc0ac600 msgr2=0x7ff0fc0aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:54.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 --2- 172.21.2.116:0/3151388169 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc0ac600 0x7ff0fc0aca90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ff0f40032f0 tx=0x7ff0f40091d0).stop 2021-11-21T22:02:54.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- 172.21.2.116:0/3151388169 shutdown_connections 2021-11-21T22:02:54.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 --2- 172.21.2.116:0/3151388169 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc0ac600 0x7ff0fc0aca90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:54.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 --2- 172.21.2.116:0/3151388169 >> v2:172.21.2.116:3300/0 conn(0x7ff0fc0abce0 0x7ff0fc1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:54.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- 172.21.2.116:0/3151388169 >> 172.21.2.116:0/3151388169 conn(0x7ff0fc05bd20 msgr2=0x7ff0fc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:54.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- 172.21.2.116:0/3151388169 shutdown_connections 2021-11-21T22:02:54.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.023+0000 7ff1014dc700 1 -- 172.21.2.116:0/3151388169 wait complete. 2021-11-21T22:02:54.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 Processor -- start 2021-11-21T22:02:54.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 -- start start 2021-11-21T22:02:54.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0fc0abce0 0x7ff0fc0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:54.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0fc0ac600 0x7ff0fc1452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:54.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc144810 0x7ff0fc156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:54.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc149310 con 0x7ff0fc0ac600 2021-11-21T22:02:54.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc149540 con 0x7ff0fc0abce0 2021-11-21T22:02:54.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc1497e0 con 0x7ff0fc144810 2021-11-21T22:02:54.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0fc0ac600 0x7ff0fc1452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:54.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0fc0ac600 0x7ff0fc1452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53688/0 (socket says 172.21.2.116:53688) 2021-11-21T22:02:54.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 -- 172.21.2.116:0/1142557639 learned_addr learned my addr 172.21.2.116:0/1142557639 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:54.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0faffd700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0fc0abce0 0x7ff0fc0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:54.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fb7fe700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc144810 0x7ff0fc156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:54.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 -- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc144810 msgr2=0x7ff0fc156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:54.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc144810 0x7ff0fc156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:54.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 -- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0fc0abce0 msgr2=0x7ff0fc0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:54.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0fc0abce0 0x7ff0fc0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:54.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 -- 172.21.2.116:0/1142557639 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009470 con 0x7ff0fc0ac600 2021-11-21T22:02:54.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0fa7fc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0fc0ac600 0x7ff0fc1452d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7ff0f000c000 tx=0x7ff0f0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:54.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0ebfff700 1 -- 172.21.2.116:0/1142557639 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f0010590 con 0x7ff0fc0ac600 2021-11-21T22:02:54.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0ebfff700 1 -- 172.21.2.116:0/1142557639 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f00106f0 con 0x7ff0fc0ac600 2021-11-21T22:02:54.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc144e30 con 0x7ff0fc0ac600 2021-11-21T22:02:54.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0ebfff700 1 -- 172.21.2.116:0/1142557639 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f0010a50 con 0x7ff0fc0ac600 2021-11-21T22:02:54.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc14ae50 con 0x7ff0fc0ac600 2021-11-21T22:02:54.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc1499e0 con 0x7ff0fc0ac600 2021-11-21T22:02:54.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0ebfff700 1 -- 172.21.2.116:0/1142557639 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff0f0010bb0 con 0x7ff0fc0ac600 2021-11-21T22:02:54.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0ebfff700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff0d80534f0 0x7ff0d80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:54.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0ebfff700 1 -- 172.21.2.116:0/1142557639 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff0f006b5f0 con 0x7ff0fc0ac600 2021-11-21T22:02:54.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0faffd700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff0d80534f0 0x7ff0d80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:54.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.027+0000 7ff0faffd700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff0d80534f0 0x7ff0d80559b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7ff0ec00d000 tx=0x7ff0ec009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:54.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.031+0000 7ff0ebfff700 1 -- 172.21.2.116:0/1142557639 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff0f0042a30 con 0x7ff0fc0ac600 2021-11-21T22:02:54.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:54.127+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7ff0fc144fc0 con 0x7ff0fc0ac600 2021-11-21T22:02:55.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff0ebfff700 1 -- 172.21.2.116:0/1142557639 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v148) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff0f0046050 con 0x7ff0fc0ac600 2021-11-21T22:02:55.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff0d80534f0 msgr2=0x7ff0d80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff0d80534f0 0x7ff0d80559b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7ff0ec00d000 tx=0x7ff0ec009040).stop 2021-11-21T22:02:55.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0fc0ac600 msgr2=0x7ff0fc1452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0fc0ac600 0x7ff0fc1452d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7ff0f000c000 tx=0x7ff0f0006df0).stop 2021-11-21T22:02:55.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 shutdown_connections 2021-11-21T22:02:55.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff0fc144810 0x7ff0fc156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff0d80534f0 0x7ff0d80559b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff0fc0ac600 0x7ff0fc1452d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 --2- 172.21.2.116:0/1142557639 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff0fc0abce0 0x7ff0fc0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 >> 172.21.2.116:0/1142557639 conn(0x7ff0fc05bd20 msgr2=0x7ff0fc0af920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 shutdown_connections 2021-11-21T22:02:55.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.031+0000 7ff1014dc700 1 -- 172.21.2.116:0/1142557639 wait complete. 2021-11-21T22:02:55.041 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T22:02:55.042 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T22:02:55.043 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T22:02:55.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c54c45700 1 Processor -- start 2021-11-21T22:02:55.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c54c45700 1 -- start start 2021-11-21T22:02:55.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c54c45700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7c500a86b0 0x7f7c5014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c54c45700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c501443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c54c45700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c500a6e80 con 0x7f7c500a6aa0 2021-11-21T22:02:55.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c54c45700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c500a8bc0 con 0x7f7c500a86b0 2021-11-21T22:02:55.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c54c45700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5014a7b0 con 0x7f7c50062410 2021-11-21T22:02:55.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c4ed9d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c501443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c4e59c700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f7c500a6aa0 0x7f7c501470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:36330/0 (socket says 172.21.2.116:36330) 2021-11-21T22:02:55.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c4e59c700 1 -- 172.21.2.116:0/4244099868 learned_addr learned my addr 172.21.2.116:0/4244099868 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:55.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c4ed9d700 1 --2- 172.21.2.116:0/4244099868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c501443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:57044/0 (socket says 172.21.2.116:57044) 2021-11-21T22:02:55.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.135+0000 7f7c4dd9b700 1 --2- 172.21.2.116:0/4244099868 >> v2:172.21.2.117:3300/0 conn(0x7f7c500a86b0 0x7f7c5014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4ed9d700 1 -- 172.21.2.116:0/4244099868 >> v2:172.21.2.117:3300/0 conn(0x7f7c500a86b0 msgr2=0x7f7c5014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4ed9d700 1 --2- 172.21.2.116:0/4244099868 >> v2:172.21.2.117:3300/0 conn(0x7f7c500a86b0 0x7f7c5014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4ed9d700 1 -- 172.21.2.116:0/4244099868 >> v1:172.21.2.117:6789/0 conn(0x7f7c500a6aa0 legacy=0x7f7c501470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4ed9d700 1 -- 172.21.2.116:0/4244099868 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5014aa10 con 0x7f7c50062410 2021-11-21T22:02:55.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4ed9d700 1 --2- 172.21.2.116:0/4244099868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c501443f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f7c480032f0 tx=0x7f7c480091d0).ready entity=mon.2 client_cookie=2fe7553db97dd0b6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4d59a700 1 -- 172.21.2.116:0/4244099868 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c4800b6c0 con 0x7f7c50062410 2021-11-21T22:02:55.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4d59a700 1 -- 172.21.2.116:0/4244099868 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4800b820 con 0x7f7c50062410 2021-11-21T22:02:55.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4d59a700 1 -- 172.21.2.116:0/4244099868 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c4800b990 con 0x7f7c50062410 2021-11-21T22:02:55.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- 172.21.2.116:0/4244099868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 msgr2=0x7f7c501443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 --2- 172.21.2.116:0/4244099868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c501443f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f7c480032f0 tx=0x7f7c480091d0).stop 2021-11-21T22:02:55.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- 172.21.2.116:0/4244099868 shutdown_connections 2021-11-21T22:02:55.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 --2- 172.21.2.116:0/4244099868 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c501443f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 --2- 172.21.2.116:0/4244099868 >> v2:172.21.2.117:3300/0 conn(0x7f7c500a86b0 0x7f7c5014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- 172.21.2.116:0/4244099868 >> 172.21.2.116:0/4244099868 conn(0x7f7c5005bd20 msgr2=0x7f7c5005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- 172.21.2.116:0/4244099868 shutdown_connections 2021-11-21T22:02:55.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- 172.21.2.116:0/4244099868 wait complete. 2021-11-21T22:02:55.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 Processor -- start 2021-11-21T22:02:55.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- start start 2021-11-21T22:02:55.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c500b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c500a6aa0 0x7f7c500b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c500a86b0 0x7f7c500b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500a9a20 con 0x7f7c500a6aa0 2021-11-21T22:02:55.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50065db0 con 0x7f7c500a86b0 2021-11-21T22:02:55.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c54c45700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50066050 con 0x7f7c50062410 2021-11-21T22:02:55.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c500a6aa0 0x7f7c500b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c500a6aa0 0x7f7c500b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53706/0 (socket says 172.21.2.116:53706) 2021-11-21T22:02:55.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 -- 172.21.2.116:0/1930159885 learned_addr learned my addr 172.21.2.116:0/1930159885 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 -- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 msgr2=0x7f7c500b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c500b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 -- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c500a86b0 msgr2=0x7f7c500b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c500a86b0 0x7f7c500b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.139+0000 7f7c4dd9b700 1 -- 172.21.2.116:0/1930159885 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c48009470 con 0x7f7c500a6aa0 2021-11-21T22:02:55.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c4dd9b700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c500a6aa0 0x7f7c500b0050 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f7c4000c000 tx=0x7f7c40006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c477fe700 1 -- 172.21.2.116:0/1930159885 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c40010780 con 0x7f7c500a6aa0 2021-11-21T22:02:55.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c477fe700 1 -- 172.21.2.116:0/1930159885 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c400108e0 con 0x7f7c500a6aa0 2021-11-21T22:02:55.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c500662f0 con 0x7f7c500a6aa0 2021-11-21T22:02:55.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c477fe700 1 -- 172.21.2.116:0/1930159885 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c40010c40 con 0x7f7c500a6aa0 2021-11-21T22:02:55.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c500a9790 con 0x7f7c500a6aa0 2021-11-21T22:02:55.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c50066480 con 0x7f7c500a6aa0 2021-11-21T22:02:55.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c477fe700 1 -- 172.21.2.116:0/1930159885 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7c4001c460 con 0x7f7c500a6aa0 2021-11-21T22:02:55.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c477fe700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c477fe700 1 -- 172.21.2.116:0/1930159885 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7c4006bac0 con 0x7f7c500a6aa0 2021-11-21T22:02:55.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c4e59c700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c4e59c700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f7c50061ab0 tx=0x7f7c38007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.143+0000 7f7c477fe700 1 -- 172.21.2.116:0/1930159885 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c40042ef0 con 0x7f7c500a6aa0 2021-11-21T22:02:55.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7c500b1170 con 0x7f7c500a6aa0 2021-11-21T22:02:55.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c477fe700 1 -- 172.21.2.116:0/1930159885 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f7c40046510 con 0x7f7c500a6aa0 2021-11-21T22:02:55.259 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:55.260 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:02:55.260 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T22:02:55.260 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.260 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T22:02:55.261 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T22:02:55.261 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.261 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:55.261 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.261 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.262 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.262 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.262 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.262 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.263 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T22:02:55.263 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T22:02:55.263 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.263 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:55.263 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.264 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.264 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.264 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.264 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.264 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.265 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T22:02:55.265 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T22:02:55.265 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.265 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:55.266 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.266 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.266 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.266 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.266 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.267 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.267 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T22:02:55.267 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T22:02:55.267 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.268 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.268 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.268 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.268 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.269 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.269 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.269 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T22:02:55.269 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T22:02:55.269 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.270 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.270 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.270 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.270 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.270 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.271 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.271 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T22:02:55.271 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T22:02:55.274 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.274 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.275 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.275 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.275 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.278 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.278 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.278 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T22:02:55.279 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T22:02:55.279 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.279 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.279 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.280 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.280 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.280 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.280 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.281 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T22:02:55.281 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T22:02:55.281 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.281 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.281 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.282 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.282 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.282 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.282 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.282 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T22:02:55.283 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T22:02:55.283 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.283 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.283 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.284 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.284 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.284 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.284 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.284 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T22:02:55.285 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T22:02:55.285 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.285 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.285 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.285 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.286 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.286 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.286 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.286 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T22:02:55.286 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T22:02:55.287 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.287 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.287 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.287 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.288 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.288 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.288 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.288 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T22:02:55.288 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T22:02:55.289 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.289 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:55.289 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:02:55.289 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:02:55.289 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.290 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.290 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.290 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T22:02:55.290 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T22:02:55.290 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.291 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:02:55.291 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:02:55.291 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:02:55.291 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.292 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.292 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.292 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T22:02:55.292 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T22:02:55.292 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.293 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:02:55.293 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:02:55.293 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:02:55.293 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.293 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.294 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.294 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.294 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T22:02:55.294 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T22:02:55.295 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.295 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:02:55.295 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:02:55.295 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.295 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.296 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.296 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.296 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T22:02:55.296 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T22:02:55.297 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:02:55.297 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:02:55.297 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:02:55.297 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:02:55.297 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.298 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.298 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:02:55.298 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:02:55.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 msgr2=0x7f7c34055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f7c50061ab0 tx=0x7f7c38007820).stop 2021-11-21T22:02:55.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c500a6aa0 msgr2=0x7f7c500b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c500a6aa0 0x7f7c500b0050 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f7c4000c000 tx=0x7f7c40006df0).stop 2021-11-21T22:02:55.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 shutdown_connections 2021-11-21T22:02:55.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7c500a86b0 0x7f7c500b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7c34053610 0x7f7c34055ad0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7c500a6aa0 0x7f7c500b0050 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 --2- 172.21.2.116:0/1930159885 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7c50062410 0x7f7c500b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 >> 172.21.2.116:0/1930159885 conn(0x7f7c5005bd20 msgr2=0x7f7c500b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 shutdown_connections 2021-11-21T22:02:55.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.255+0000 7f7c54c45700 1 -- 172.21.2.116:0/1930159885 wait complete. 2021-11-21T22:02:55.304 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:55.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 Processor -- start 2021-11-21T22:02:55.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- start start 2021-11-21T22:02:55.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4ecc0a6aa0 0x7f4ecc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f4ecc0a86b0 0x7f4ecc1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0a6e80 con 0x7f4ecc1475d0 2021-11-21T22:02:55.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0a8bc0 con 0x7f4ecc0a6aa0 2021-11-21T22:02:55.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc14a7b0 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecb7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f4ecc1475d0 0x7f4ecc1443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37436/0 (socket says 172.21.2.116:37436) 2021-11-21T22:02:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecb7fe700 1 -- 172.21.2.116:0/2790359391 learned_addr learned my addr 172.21.2.116:0/2790359391 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecaffd700 1 --2- 172.21.2.116:0/2790359391 >> v2:172.21.2.116:3300/0 conn(0x7f4ecc0a6aa0 0x7f4ecc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ec9ffb700 1 -- 172.21.2.116:0/2790359391 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338946076 0 0) 0x7f4ecc0a6e80 con 0x7f4ecc1475d0 2021-11-21T22:02:55.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ec9ffb700 1 -- 172.21.2.116:0/2790359391 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eac003260 con 0x7f4ecc1475d0 2021-11-21T22:02:55.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecaffd700 1 -- 172.21.2.116:0/2790359391 >> v2:172.21.2.117:3300/0 conn(0x7f4ecc0a86b0 msgr2=0x7f4ecc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecaffd700 1 --2- 172.21.2.116:0/2790359391 >> v2:172.21.2.117:3300/0 conn(0x7f4ecc0a86b0 0x7f4ecc1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecaffd700 1 -- 172.21.2.116:0/2790359391 >> v1:172.21.2.116:6789/0 conn(0x7f4ecc1475d0 legacy=0x7f4ecc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecaffd700 1 -- 172.21.2.116:0/2790359391 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc14aa10 con 0x7f4ecc0a6aa0 2021-11-21T22:02:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ecaffd700 1 --2- 172.21.2.116:0/2790359391 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a6aa0 0x7f4ecc14b150 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f4ec4001490 tx=0x7f4ebc006df0).ready entity=mon.0 client_cookie=66d81d96519dbf8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ec9ffb700 1 -- 172.21.2.116:0/2790359391 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ebc00d570 con 0x7f4ecc0a6aa0 2021-11-21T22:02:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ec9ffb700 1 -- 172.21.2.116:0/2790359391 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc000e90 con 0x7f4ecc0a6aa0 2021-11-21T22:02:55.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ec9ffb700 1 -- 172.21.2.116:0/2790359391 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ebc00d8c0 con 0x7f4ecc0a6aa0 2021-11-21T22:02:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- 172.21.2.116:0/2790359391 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a6aa0 msgr2=0x7f4ecc14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2790359391 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a6aa0 0x7f4ecc14b150 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f4ec4001490 tx=0x7f4ebc006df0).stop 2021-11-21T22:02:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- 172.21.2.116:0/2790359391 shutdown_connections 2021-11-21T22:02:55.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2790359391 >> v2:172.21.2.117:3300/0 conn(0x7f4ecc0a86b0 0x7f4ecc1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2790359391 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a6aa0 0x7f4ecc14b150 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- 172.21.2.116:0/2790359391 >> 172.21.2.116:0/2790359391 conn(0x7f4ecc05bd20 msgr2=0x7f4ecc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- 172.21.2.116:0/2790359391 shutdown_connections 2021-11-21T22:02:55.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.351+0000 7f4ed1910700 1 -- 172.21.2.116:0/2790359391 wait complete. 2021-11-21T22:02:55.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 Processor -- start 2021-11-21T22:02:55.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 -- start start 2021-11-21T22:02:55.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4ecc0a6aa0 0x7f4ecc0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a86b0 0x7f4ecc13b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4ecc1475d0 0x7f4ecc13b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc13e070 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0a97c0 con 0x7f4ecc0a6aa0 2021-11-21T22:02:55.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0a9a60 con 0x7f4ecc1475d0 2021-11-21T22:02:55.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a86b0 0x7f4ecc13b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a86b0 0x7f4ecc13b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53720/0 (socket says 172.21.2.116:53720) 2021-11-21T22:02:55.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 -- 172.21.2.116:0/2882807545 learned_addr learned my addr 172.21.2.116:0/2882807545 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 -- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4ecc1475d0 msgr2=0x7f4ecc13b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4ecc1475d0 0x7f4ecc13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 -- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4ecc0a6aa0 msgr2=0x7f4ecc0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4ecc0a6aa0 0x7f4ecc0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 -- 172.21.2.116:0/2882807545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc00b180 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4eca7fc700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a86b0 0x7f4ecc13b360 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f4eb4002f70 tx=0x7f4eb4006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ec3fff700 1 -- 172.21.2.116:0/2882807545 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb4013460 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ec3fff700 1 -- 172.21.2.116:0/2882807545 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb40135c0 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ec3fff700 1 -- 172.21.2.116:0/2882807545 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb4013920 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc0a9d00 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc0aa6d0 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc0573d0 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ec3fff700 1 -- 172.21.2.116:0/2882807545 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4eb401a060 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ec3fff700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4eb00536f0 0x7f4eb0055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ec3fff700 1 -- 172.21.2.116:0/2882807545 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4eb406b1e0 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.355+0000 7f4ecaffd700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4eb00536f0 0x7f4eb0055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.359+0000 7f4ecaffd700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4eb00536f0 0x7f4eb0055bb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f4ec4001490 tx=0x7f4ebc014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.359+0000 7f4ec3fff700 1 -- 172.21.2.116:0/2882807545 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4eb4042560 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ecc0ac030 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ec3fff700 1 -- 172.21.2.116:0/2882807545 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f4eb4018070 con 0x7f4ecc0a86b0 2021-11-21T22:02:55.447 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:55.447 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":148,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:55.029458+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:55.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4eb00536f0 msgr2=0x7f4eb0055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4eb00536f0 0x7f4eb0055bb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f4ec4001490 tx=0x7f4ebc014000).stop 2021-11-21T22:02:55.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a86b0 msgr2=0x7f4ecc13b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a86b0 0x7f4ecc13b360 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f4eb4002f70 tx=0x7f4eb4006ec0).stop 2021-11-21T22:02:55.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 shutdown_connections 2021-11-21T22:02:55.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4ecc1475d0 0x7f4ecc13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4eb00536f0 0x7f4eb0055bb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4ecc0a86b0 0x7f4ecc13b360 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 --2- 172.21.2.116:0/2882807545 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4ecc0a6aa0 0x7f4ecc0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 >> 172.21.2.116:0/2882807545 conn(0x7f4ecc05bd20 msgr2=0x7f4ecc141be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 shutdown_connections 2021-11-21T22:02:55.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.443+0000 7f4ed1910700 1 -- 172.21.2.116:0/2882807545 wait complete. 2021-11-21T22:02:55.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T22:02:55.455 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:02:55.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f04238d9700 1 Processor -- start 2021-11-21T22:02:55.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f04238d9700 1 -- start start 2021-11-21T22:02:55.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f04238d9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c144810 0x7f041c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f04238d9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f041c0abce0 0x7f041c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f04238d9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c147e80 con 0x7f041c0b1db0 2021-11-21T22:02:55.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f04238d9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0b1490 con 0x7f041c0abce0 2021-11-21T22:02:55.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f04238d9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0af060 con 0x7f041c144810 2021-11-21T22:02:55.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f0420e74700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f041c0abce0 0x7f041c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f0420e74700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f041c0abce0 0x7f041c0b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53728/0 (socket says 172.21.2.116:53728) 2021-11-21T22:02:55.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.503+0000 7f0420e74700 1 -- 172.21.2.116:0/767098546 learned_addr learned my addr 172.21.2.116:0/767098546 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0420e74700 1 -- 172.21.2.116:0/767098546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c144810 msgr2=0x7f041c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0420e74700 1 --2- 172.21.2.116:0/767098546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c144810 0x7f041c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0420e74700 1 -- 172.21.2.116:0/767098546 >> v1:172.21.2.117:6789/0 conn(0x7f041c0b1db0 legacy=0x7f041c0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0420e74700 1 -- 172.21.2.116:0/767098546 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0af5a0 con 0x7f041c0abce0 2021-11-21T22:02:55.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0420e74700 1 --2- 172.21.2.116:0/767098546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c0b18c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f041c0b2a30 tx=0x7f0418006eb0).ready entity=mon.0 client_cookie=d578809adea7e943 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f040ffff700 1 -- 172.21.2.116:0/767098546 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f041800d570 con 0x7f041c0abce0 2021-11-21T22:02:55.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f040ffff700 1 -- 172.21.2.116:0/767098546 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0418000e90 con 0x7f041c0abce0 2021-11-21T22:02:55.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f040ffff700 1 -- 172.21.2.116:0/767098546 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f041800d8c0 con 0x7f041c0abce0 2021-11-21T22:02:55.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/767098546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 msgr2=0x7f041c0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 --2- 172.21.2.116:0/767098546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c0b18c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f041c0b2a30 tx=0x7f0418006eb0).stop 2021-11-21T22:02:55.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/767098546 shutdown_connections 2021-11-21T22:02:55.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 --2- 172.21.2.116:0/767098546 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c0b18c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 --2- 172.21.2.116:0/767098546 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c144810 0x7f041c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/767098546 >> 172.21.2.116:0/767098546 conn(0x7f041c05bd20 msgr2=0x7f041c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/767098546 shutdown_connections 2021-11-21T22:02:55.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/767098546 wait complete. 2021-11-21T22:02:55.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 Processor -- start 2021-11-21T22:02:55.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- start start 2021-11-21T22:02:55.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c13cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f041c144810 0x7f041c14a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c156700 0x7f041c144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c14a310 con 0x7f041c0abce0 2021-11-21T22:02:55.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c146360 con 0x7f041c144810 2021-11-21T22:02:55.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c14a4f0 con 0x7f041c156700 2021-11-21T22:02:55.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c13cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c13cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53732/0 (socket says 172.21.2.116:53732) 2021-11-21T22:02:55.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 -- 172.21.2.116:0/3161180941 learned_addr learned my addr 172.21.2.116:0/3161180941 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 -- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c156700 msgr2=0x7f041c144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c156700 0x7f041c144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 -- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f041c144810 msgr2=0x7f041c14a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f041c144810 0x7f041c14a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 -- 172.21.2.116:0/3161180941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041800b1e0 con 0x7f041c0abce0 2021-11-21T22:02:55.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f0421675700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c13cd00 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7f0410002f70 tx=0x7f0410006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f040e7fc700 1 -- 172.21.2.116:0/3161180941 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0410013460 con 0x7f041c0abce0 2021-11-21T22:02:55.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f040e7fc700 1 -- 172.21.2.116:0/3161180941 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04100135c0 con 0x7f041c0abce0 2021-11-21T22:02:55.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c14a0b0 con 0x7f041c0abce0 2021-11-21T22:02:55.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c146990 con 0x7f041c0abce0 2021-11-21T22:02:55.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f040e7fc700 1 -- 172.21.2.116:0/3161180941 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0410013920 con 0x7f041c0abce0 2021-11-21T22:02:55.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c0b2c50 con 0x7f041c0abce0 2021-11-21T22:02:55.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.507+0000 7f040e7fc700 1 -- 172.21.2.116:0/3161180941 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f041001a060 con 0x7f041c0abce0 2021-11-21T22:02:55.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.511+0000 7f040e7fc700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03fc053660 0x7f03fc055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.511+0000 7f040e7fc700 1 -- 172.21.2.116:0/3161180941 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f041006b120 con 0x7f041c0abce0 2021-11-21T22:02:55.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.511+0000 7f0420e74700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03fc053660 0x7f03fc055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.511+0000 7f0420e74700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03fc053660 0x7f03fc055b20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f041c0b2a30 tx=0x7f0418015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.511+0000 7f040e7fc700 1 -- 172.21.2.116:0/3161180941 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04100424a0 con 0x7f041c0abce0 2021-11-21T22:02:55.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f041c0af300 con 0x7f041c0abce0 2021-11-21T22:02:55.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f040e7fc700 1 -- 172.21.2.116:0/3161180941 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0410018020 con 0x7f041c0abce0 2021-11-21T22:02:55.598 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:02:55.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03fc053660 msgr2=0x7f03fc055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03fc053660 0x7f03fc055b20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f041c0b2a30 tx=0x7f0418015000).stop 2021-11-21T22:02:55.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 msgr2=0x7f041c13cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c13cd00 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7f0410002f70 tx=0x7f0410006e20).stop 2021-11-21T22:02:55.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 shutdown_connections 2021-11-21T22:02:55.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f041c156700 0x7f041c144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f041c144810 0x7f041c14a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f03fc053660 0x7f03fc055b20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 --2- 172.21.2.116:0/3161180941 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f041c0abce0 0x7f041c13cd00 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 >> 172.21.2.116:0/3161180941 conn(0x7f041c05bd20 msgr2=0x7f041c0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 shutdown_connections 2021-11-21T22:02:55.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.595+0000 7f04238d9700 1 -- 172.21.2.116:0/3161180941 wait complete. 2021-11-21T22:02:55.605 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:02:55.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 Processor -- start 2021-11-21T22:02:55.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- start start 2021-11-21T22:02:55.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aff10 0x7f33600ab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f33600aab70 0x7f33600b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f33601477c0 0x7f3360144250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33600b1160 con 0x7f33600aab70 2021-11-21T22:02:55.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3360146e90 con 0x7f33601477c0 2021-11-21T22:02:55.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33600aba40 con 0x7f33600aff10 2021-11-21T22:02:55.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f33600aab70 0x7f33600b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f33600aab70 0x7f33600b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53738/0 (socket says 172.21.2.116:53738) 2021-11-21T22:02:55.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 -- 172.21.2.116:0/1463448684 learned_addr learned my addr 172.21.2.116:0/1463448684 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 -- 172.21.2.116:0/1463448684 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aff10 msgr2=0x7f33600ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 --2- 172.21.2.116:0/1463448684 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aff10 0x7f33600ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 -- 172.21.2.116:0/1463448684 >> v2:172.21.2.117:3300/0 conn(0x7f33601477c0 msgr2=0x7f3360144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 --2- 172.21.2.116:0/1463448684 >> v2:172.21.2.117:3300/0 conn(0x7f33601477c0 0x7f3360144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 -- 172.21.2.116:0/1463448684 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600b5960 con 0x7f33600aab70 2021-11-21T22:02:55.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335effd700 1 --2- 172.21.2.116:0/1463448684 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aab70 0x7f33600b5470 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f3350002f70 tx=0x7f3350006eb0).ready entity=mon.0 client_cookie=e42fd6d6f0c7db59 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335e7fc700 1 -- 172.21.2.116:0/1463448684 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f335000d570 con 0x7f33600aab70 2021-11-21T22:02:55.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335e7fc700 1 -- 172.21.2.116:0/1463448684 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3350000e90 con 0x7f33600aab70 2021-11-21T22:02:55.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f335e7fc700 1 -- 172.21.2.116:0/1463448684 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f335000d900 con 0x7f33600aab70 2021-11-21T22:02:55.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- 172.21.2.116:0/1463448684 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aab70 msgr2=0x7f33600b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 --2- 172.21.2.116:0/1463448684 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aab70 0x7f33600b5470 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f3350002f70 tx=0x7f3350006eb0).stop 2021-11-21T22:02:55.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- 172.21.2.116:0/1463448684 shutdown_connections 2021-11-21T22:02:55.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 --2- 172.21.2.116:0/1463448684 >> v2:172.21.2.117:3300/0 conn(0x7f33601477c0 0x7f3360144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 --2- 172.21.2.116:0/1463448684 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aab70 0x7f33600b5470 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 --2- 172.21.2.116:0/1463448684 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aff10 0x7f33600ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- 172.21.2.116:0/1463448684 >> 172.21.2.116:0/1463448684 conn(0x7f3360069e70 msgr2=0x7f3360063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- 172.21.2.116:0/1463448684 shutdown_connections 2021-11-21T22:02:55.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- 172.21.2.116:0/1463448684 wait complete. 2021-11-21T22:02:55.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 Processor -- start 2021-11-21T22:02:55.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.655+0000 7f336614c700 1 -- start start 2021-11-21T22:02:55.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aab70 0x7f33600aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aff10 0x7f3360159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33601477c0 0x7f336015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336014a420 con 0x7f33600aff10 2021-11-21T22:02:55.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336014a6c0 con 0x7f33601477c0 2021-11-21T22:02:55.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336014a960 con 0x7f33600aab70 2021-11-21T22:02:55.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aff10 0x7f3360159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aff10 0x7f3360159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53744/0 (socket says 172.21.2.116:53744) 2021-11-21T22:02:55.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 -- 172.21.2.116:0/215779409 learned_addr learned my addr 172.21.2.116:0/215779409 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 -- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aab70 msgr2=0x7f33600aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aab70 0x7f33600aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 -- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33601477c0 msgr2=0x7f336015a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33601477c0 0x7f336015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 -- 172.21.2.116:0/215779409 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335000b1e0 con 0x7f33600aff10 2021-11-21T22:02:55.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335effd700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aff10 0x7f3360159c70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f3350002f70 tx=0x7f3350015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335cff9700 1 -- 172.21.2.116:0/215779409 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3350011990 con 0x7f33600aff10 2021-11-21T22:02:55.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335cff9700 1 -- 172.21.2.116:0/215779409 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3350011af0 con 0x7f33600aff10 2021-11-21T22:02:55.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336014ac00 con 0x7f33600aff10 2021-11-21T22:02:55.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3360149d50 con 0x7f33600aff10 2021-11-21T22:02:55.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335cff9700 1 -- 172.21.2.116:0/215779409 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3350011e50 con 0x7f33600aff10 2021-11-21T22:02:55.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336014a030 con 0x7f33600aff10 2021-11-21T22:02:55.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335cff9700 1 -- 172.21.2.116:0/215779409 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3350025030 con 0x7f33600aff10 2021-11-21T22:02:55.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335cff9700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33440535d0 0x7f3344055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335cff9700 1 -- 172.21.2.116:0/215779409 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f335006caa0 con 0x7f33600aff10 2021-11-21T22:02:55.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.659+0000 7f335f7fe700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33440535d0 0x7f3344055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.663+0000 7f335f7fe700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33440535d0 0x7f3344055a90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f3348002f70 tx=0x7f3348006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.663+0000 7f335cff9700 1 -- 172.21.2.116:0/215779409 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3350043f30 con 0x7f33600aff10 2021-11-21T22:02:55.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.743+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f336014ad90 con 0x7f33600aff10 2021-11-21T22:02:55.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.743+0000 7f335cff9700 1 -- 172.21.2.116:0/215779409 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7f33500230a0 con 0x7f33600aff10 2021-11-21T22:02:55.748 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:02:55.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.743+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33440535d0 msgr2=0x7f3344055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.743+0000 7f336614c700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33440535d0 0x7f3344055a90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f3348002f70 tx=0x7f3348006a80).stop 2021-11-21T22:02:55.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.743+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aff10 msgr2=0x7f3360159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.743+0000 7f336614c700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aff10 0x7f3360159c70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f3350002f70 tx=0x7f3350015000).stop 2021-11-21T22:02:55.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 shutdown_connections 2021-11-21T22:02:55.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f33601477c0 0x7f336015a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f33440535d0 0x7f3344055a90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f33600aff10 0x7f3360159c70 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 --2- 172.21.2.116:0/215779409 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f33600aab70 0x7f33600aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 >> 172.21.2.116:0/215779409 conn(0x7f3360069e70 msgr2=0x7f3360062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 shutdown_connections 2021-11-21T22:02:55.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.747+0000 7f336614c700 1 -- 172.21.2.116:0/215779409 wait complete. 2021-11-21T22:02:55.755 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T22:02:55.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 Processor -- start 2021-11-21T22:02:55.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- start start 2021-11-21T22:02:55.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fdb880ad780 0x7fdb8805f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 0x7fdb88143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdb880af7e0 0x7fdb880b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb880adb60 con 0x7fdb880af7e0 2021-11-21T22:02:55.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb88143160 con 0x7fdb880ad780 2021-11-21T22:02:55.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880b0ce0 con 0x7fdb880affc0 2021-11-21T22:02:55.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdb880af7e0 0x7fdb880b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdb880af7e0 0x7fdb880b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53752/0 (socket says 172.21.2.116:53752) 2021-11-21T22:02:55.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 -- 172.21.2.116:0/1774181875 learned_addr learned my addr 172.21.2.116:0/1774181875 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 -- 172.21.2.116:0/1774181875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 msgr2=0x7fdb88143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 --2- 172.21.2.116:0/1774181875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 0x7fdb88143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 -- 172.21.2.116:0/1774181875 >> v2:172.21.2.117:3300/0 conn(0x7fdb880ad780 msgr2=0x7fdb8805f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 --2- 172.21.2.116:0/1774181875 >> v2:172.21.2.117:3300/0 conn(0x7fdb880ad780 0x7fdb8805f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 -- 172.21.2.116:0/1774181875 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb880b0a40 con 0x7fdb880af7e0 2021-11-21T22:02:55.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e814700 1 --2- 172.21.2.116:0/1774181875 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880af7e0 0x7fdb880b16c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7fdb800032f0 tx=0x7fdb8000b530).ready entity=mon.0 client_cookie=e906effc68d355a7 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8d011700 1 -- 172.21.2.116:0/1774181875 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb8000be70 con 0x7fdb880af7e0 2021-11-21T22:02:55.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8d011700 1 -- 172.21.2.116:0/1774181875 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb80010470 con 0x7fdb880af7e0 2021-11-21T22:02:55.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8d011700 1 -- 172.21.2.116:0/1774181875 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb800107d0 con 0x7fdb880af7e0 2021-11-21T22:02:55.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- 172.21.2.116:0/1774181875 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880af7e0 msgr2=0x7fdb880b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- 172.21.2.116:0/1774181875 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880af7e0 0x7fdb880b16c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7fdb800032f0 tx=0x7fdb8000b530).stop 2021-11-21T22:02:55.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- 172.21.2.116:0/1774181875 shutdown_connections 2021-11-21T22:02:55.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- 172.21.2.116:0/1774181875 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880af7e0 0x7fdb880b16c0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- 172.21.2.116:0/1774181875 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 0x7fdb88143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- 172.21.2.116:0/1774181875 >> v2:172.21.2.117:3300/0 conn(0x7fdb880ad780 0x7fdb8805f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- 172.21.2.116:0/1774181875 >> 172.21.2.116:0/1774181875 conn(0x7fdb880b6000 msgr2=0x7fdb880b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- 172.21.2.116:0/1774181875 shutdown_connections 2021-11-21T22:02:55.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- 172.21.2.116:0/1774181875 wait complete. 2021-11-21T22:02:55.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 Processor -- start 2021-11-21T22:02:55.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- start start 2021-11-21T22:02:55.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880ad780 0x7fdb8809ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb880af7e0 0x7fdb88159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 0x7fdb880a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88066ec0 con 0x7fdb880ad780 2021-11-21T22:02:55.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8809e710 con 0x7fdb880af7e0 2021-11-21T22:02:55.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb90277700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8809e9b0 con 0x7fdb880affc0 2021-11-21T22:02:55.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e013700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880ad780 0x7fdb8809ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e013700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880ad780 0x7fdb8809ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53756/0 (socket says 172.21.2.116:53756) 2021-11-21T22:02:55.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.803+0000 7fdb8e013700 1 -- 172.21.2.116:0/3824136932 learned_addr learned my addr 172.21.2.116:0/3824136932 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8e013700 1 -- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 msgr2=0x7fdb880a8d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:55.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8e013700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 0x7fdb880a8d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8e013700 1 -- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb880af7e0 msgr2=0x7fdb88159bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:02:55.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8e013700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb880af7e0 0x7fdb88159bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8e013700 1 -- 172.21.2.116:0/3824136932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8000b7d0 con 0x7fdb880ad780 2021-11-21T22:02:55.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8e013700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880ad780 0x7fdb8809ee50 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fdb84002f70 tx=0x7fdb84006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb777fe700 1 -- 172.21.2.116:0/3824136932 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb84013460 con 0x7fdb880ad780 2021-11-21T22:02:55.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb777fe700 1 -- 172.21.2.116:0/3824136932 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb840135c0 con 0x7fdb880ad780 2021-11-21T22:02:55.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb777fe700 1 -- 172.21.2.116:0/3824136932 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb84013920 con 0x7fdb880ad780 2021-11-21T22:02:55.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb880672c0 con 0x7fdb880ad780 2021-11-21T22:02:55.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8809e0f0 con 0x7fdb880ad780 2021-11-21T22:02:55.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8809eb70 con 0x7fdb880ad780 2021-11-21T22:02:55.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb777fe700 1 -- 172.21.2.116:0/3824136932 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdb8400f070 con 0x7fdb880ad780 2021-11-21T22:02:55.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb777fe700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdb68053610 0x7fdb68055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb777fe700 1 -- 172.21.2.116:0/3824136932 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdb8406b120 con 0x7fdb880ad780 2021-11-21T22:02:55.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8d812700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdb68053610 0x7fdb68055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.807+0000 7fdb8d812700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdb68053610 0x7fdb68055ad0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fdb78002f70 tx=0x7fdb780068f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.811+0000 7fdb777fe700 1 -- 172.21.2.116:0/3824136932 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdb840424a0 con 0x7fdb880ad780 2021-11-21T22:02:55.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdb881433c0 con 0x7fdb880ad780 2021-11-21T22:02:55.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb777fe700 1 -- 172.21.2.116:0/3824136932 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v148) v1 ==== 84+0+143 (secure 0 0 0) 0x7fdb84045ac0 con 0x7fdb880ad780 2021-11-21T22:02:55.899 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:55.899 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T22:02:55.899 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.899 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T22:02:55.900 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T22:02:55.900 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:02:55.900 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:02:55.901 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T22:02:55.901 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T22:02:55.901 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:02:55.901 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T22:02:55.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdb68053610 msgr2=0x7fdb68055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdb68053610 0x7fdb68055ad0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fdb78002f70 tx=0x7fdb780068f0).stop 2021-11-21T22:02:55.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880ad780 msgr2=0x7fdb8809ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880ad780 0x7fdb8809ee50 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fdb84002f70 tx=0x7fdb84006ec0).stop 2021-11-21T22:02:55.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 shutdown_connections 2021-11-21T22:02:55.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdb880affc0 0x7fdb880a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdb880af7e0 0x7fdb88159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:55.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdb68053610 0x7fdb68055ad0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 --2- 172.21.2.116:0/3824136932 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdb880ad780 0x7fdb8809ee50 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 >> 172.21.2.116:0/3824136932 conn(0x7fdb880b6000 msgr2=0x7fdb880b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 shutdown_connections 2021-11-21T22:02:55.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.895+0000 7fdb90277700 1 -- 172.21.2.116:0/3824136932 wait complete. 2021-11-21T22:02:55.906 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T22:02:55.906 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:02:55.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.955+0000 7fa4f70bc700 1 Processor -- start 2021-11-21T22:02:55.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.955+0000 7fa4f70bc700 1 -- start start 2021-11-21T22:02:55.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.955+0000 7fa4f70bc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa4f005f730 0x7fa4f0141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.955+0000 7fa4f70bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f0142370 0x7fa4f0142800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.955+0000 7fa4f70bc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f00b06c0 con 0x7fa4f00b02e0 2021-11-21T22:02:55.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.955+0000 7fa4f70bc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f00adf20 con 0x7fa4f005f730 2021-11-21T22:02:55.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.955+0000 7fa4f70bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00b2470 con 0x7fa4f0142370 2021-11-21T22:02:55.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f4e58700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa4f00b02e0 0x7fa4f00af620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37488/0 (socket says 172.21.2.116:37488) 2021-11-21T22:02:55.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f4e58700 1 -- 172.21.2.116:0/3120870408 learned_addr learned my addr 172.21.2.116:0/3120870408 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:55.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4effff700 1 --2- 172.21.2.116:0/3120870408 >> v2:172.21.2.116:3300/0 conn(0x7fa4f005f730 0x7fa4f0141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 --2- 172.21.2.116:0/3120870408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f0142370 0x7fa4f0142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4ef7fe700 1 -- 172.21.2.116:0/3120870408 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447854751 0 0) 0x7fa4f00b06c0 con 0x7fa4f00b02e0 2021-11-21T22:02:55.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4ef7fe700 1 -- 172.21.2.116:0/3120870408 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d0003260 con 0x7fa4f00b02e0 2021-11-21T22:02:55.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4effff700 1 -- 172.21.2.116:0/3120870408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f0142370 msgr2=0x7fa4f0142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:55.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4effff700 1 --2- 172.21.2.116:0/3120870408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f0142370 0x7fa4f0142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4effff700 1 -- 172.21.2.116:0/3120870408 >> v1:172.21.2.116:6789/0 conn(0x7fa4f00b02e0 legacy=0x7fa4f00af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4effff700 1 -- 172.21.2.116:0/3120870408 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00b26d0 con 0x7fa4f005f730 2021-11-21T22:02:55.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4effff700 1 --2- 172.21.2.116:0/3120870408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f005f730 0x7fa4f0141e80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7fa4e0002f70 tx=0x7fa4e0006df0).ready entity=mon.0 client_cookie=6f51ced08a7c3433 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4ef7fe700 1 -- 172.21.2.116:0/3120870408 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e000d4a0 con 0x7fa4f005f730 2021-11-21T22:02:55.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4ef7fe700 1 -- 172.21.2.116:0/3120870408 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e0000e90 con 0x7fa4f005f730 2021-11-21T22:02:55.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4ef7fe700 1 -- 172.21.2.116:0/3120870408 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e000d7f0 con 0x7fa4f005f730 2021-11-21T22:02:55.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- 172.21.2.116:0/3120870408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f005f730 msgr2=0x7fa4f0141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/3120870408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f005f730 0x7fa4f0141e80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7fa4e0002f70 tx=0x7fa4e0006df0).stop 2021-11-21T22:02:55.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- 172.21.2.116:0/3120870408 shutdown_connections 2021-11-21T22:02:55.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/3120870408 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f0142370 0x7fa4f0142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/3120870408 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f005f730 0x7fa4f0141e80 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- 172.21.2.116:0/3120870408 >> 172.21.2.116:0/3120870408 conn(0x7fa4f00b5ff0 msgr2=0x7fa4f00b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:55.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- 172.21.2.116:0/3120870408 shutdown_connections 2021-11-21T22:02:55.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- 172.21.2.116:0/3120870408 wait complete. 2021-11-21T22:02:55.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 Processor -- start 2021-11-21T22:02:55.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- start start 2021-11-21T22:02:55.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f005f730 0x7fa4f00654a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4f00b02e0 0x7fa4f0145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f0142370 0x7fa4f0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0151d90 con 0x7fa4f0142370 2021-11-21T22:02:55.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0145970 con 0x7fa4f00b02e0 2021-11-21T22:02:55.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0151f80 con 0x7fa4f005f730 2021-11-21T22:02:55.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f4e58700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f005f730 0x7fa4f00654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f0142370 0x7fa4f0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f4e58700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f005f730 0x7fa4f00654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:57114/0 (socket says 172.21.2.116:57114) 2021-11-21T22:02:55.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f4e58700 1 -- 172.21.2.116:0/163422759 learned_addr learned my addr 172.21.2.116:0/163422759 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:55.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4effff700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4f00b02e0 0x7fa4f0145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 -- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f005f730 msgr2=0x7fa4f00654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f005f730 0x7fa4f00654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 -- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4f00b02e0 msgr2=0x7fa4f0145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:55.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4f00b02e0 0x7fa4f0145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:55.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 -- 172.21.2.116:0/163422759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e000b180 con 0x7fa4f0142370 2021-11-21T22:02:55.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f5659700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f0142370 0x7fa4f0155ba0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fa4f00aac70 tx=0x7fa4e80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4edffb700 1 -- 172.21.2.116:0/163422759 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e8012600 con 0x7fa4f0142370 2021-11-21T22:02:55.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4edffb700 1 -- 172.21.2.116:0/163422759 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8012760 con 0x7fa4f0142370 2021-11-21T22:02:55.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4edffb700 1 -- 172.21.2.116:0/163422759 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e8012ac0 con 0x7fa4f0142370 2021-11-21T22:02:55.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f0155460 con 0x7fa4f0142370 2021-11-21T22:02:55.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.959+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f0065fa0 con 0x7fa4f0142370 2021-11-21T22:02:55.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.963+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f0152140 con 0x7fa4f0142370 2021-11-21T22:02:55.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.963+0000 7fa4edffb700 1 -- 172.21.2.116:0/163422759 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa4e8022070 con 0x7fa4f0142370 2021-11-21T22:02:55.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.963+0000 7fa4edffb700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4d4053640 0x7fa4d4055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:55.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.963+0000 7fa4edffb700 1 -- 172.21.2.116:0/163422759 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa4e806bc20 con 0x7fa4f0142370 2021-11-21T22:02:55.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.963+0000 7fa4f4e58700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4d4053640 0x7fa4d4055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:55.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.963+0000 7fa4f4e58700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4d4053640 0x7fa4d4055b00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fa4d800d000 tx=0x7fa4d8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:55.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:55.963+0000 7fa4edffb700 1 -- 172.21.2.116:0/163422759 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa4e8042fa0 con 0x7fa4f0142370 2021-11-21T22:02:56.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:56.047+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa4f0142cf0 con 0x7fa4f0142370 2021-11-21T22:02:57.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4edffb700 1 -- 172.21.2.116:0/163422759 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v149) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa4e8016070 con 0x7fa4f0142370 2021-11-21T22:02:57.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4d4053640 msgr2=0x7fa4d4055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:57.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4d4053640 0x7fa4d4055b00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fa4d800d000 tx=0x7fa4d8009040).stop 2021-11-21T22:02:57.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f0142370 msgr2=0x7fa4f0155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:57.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f0142370 0x7fa4f0155ba0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fa4f00aac70 tx=0x7fa4e80091d0).stop 2021-11-21T22:02:57.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 shutdown_connections 2021-11-21T22:02:57.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa4d4053640 0x7fa4d4055b00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa4f0142370 0x7fa4f0155ba0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa4f00b02e0 0x7fa4f0145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 --2- 172.21.2.116:0/163422759 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa4f005f730 0x7fa4f00654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 >> 172.21.2.116:0/163422759 conn(0x7fa4f00b5ff0 msgr2=0x7fa4f00b63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:57.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 shutdown_connections 2021-11-21T22:02:57.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.055+0000 7fa4f70bc700 1 -- 172.21.2.116:0/163422759 wait complete. 2021-11-21T22:02:57.062 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:02:57.066 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:02:57.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 Processor -- start 2021-11-21T22:02:57.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 -- start start 2021-11-21T22:02:57.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbaf80abce0 0x7fbaf81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:57.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf8147e80 con 0x7fbaf8144810 2021-11-21T22:02:57.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf80af5a0 con 0x7fbaf80ac600 2021-11-21T22:02:57.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf80b1490 con 0x7fbaf80abce0 2021-11-21T22:02:57.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbafd0fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fbaf8144810 0x7fbaf80b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37502/0 (socket says 172.21.2.116:37502) 2021-11-21T22:02:57.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbafd0fe700 1 -- 172.21.2.116:0/1807826085 learned_addr learned my addr 172.21.2.116:0/1807826085 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:57.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbafc8fd700 1 --2- 172.21.2.116:0/1807826085 >> v2:172.21.2.116:3300/0 conn(0x7fbaf80abce0 0x7fbaf81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:57.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbafc8fd700 1 -- 172.21.2.116:0/1807826085 >> v1:172.21.2.117:6789/0 conn(0x7fbaf80ac600 legacy=0x7fbaf80aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:57.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbafc8fd700 1 -- 172.21.2.116:0/1807826085 >> v1:172.21.2.116:6789/0 conn(0x7fbaf8144810 legacy=0x7fbaf80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:57.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbafc8fd700 1 -- 172.21.2.116:0/1807826085 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80b1fd0 con 0x7fbaf80abce0 2021-11-21T22:02:57.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbafc8fd700 1 --2- 172.21.2.116:0/1807826085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf81478c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fbaf80b4e00 tx=0x7fbaf4006df0).ready entity=mon.0 client_cookie=c9caf4ba2fe94b3e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:57.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaebfff700 1 -- 172.21.2.116:0/1807826085 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaf400d5c0 con 0x7fbaf80abce0 2021-11-21T22:02:57.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaebfff700 1 -- 172.21.2.116:0/1807826085 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf4000e90 con 0x7fbaf80abce0 2021-11-21T22:02:57.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaebfff700 1 -- 172.21.2.116:0/1807826085 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaf400d910 con 0x7fbaf80abce0 2021-11-21T22:02:57.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 -- 172.21.2.116:0/1807826085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 msgr2=0x7fbaf81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:57.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 --2- 172.21.2.116:0/1807826085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf81478c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fbaf80b4e00 tx=0x7fbaf4006df0).stop 2021-11-21T22:02:57.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 -- 172.21.2.116:0/1807826085 shutdown_connections 2021-11-21T22:02:57.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.115+0000 7fbaff362700 1 --2- 172.21.2.116:0/1807826085 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf81478c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:57.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- 172.21.2.116:0/1807826085 >> 172.21.2.116:0/1807826085 conn(0x7fbaf805bd20 msgr2=0x7fbaf805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:57.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- 172.21.2.116:0/1807826085 shutdown_connections 2021-11-21T22:02:57.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- 172.21.2.116:0/1807826085 wait complete. 2021-11-21T22:02:57.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 Processor -- start 2021-11-21T22:02:57.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- start start 2021-11-21T22:02:57.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf8145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:57.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbaf80ac600 0x7fbaf8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:57.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf8144810 0x7fbaf8157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:57.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8156b00 con 0x7fbaf80abce0 2021-11-21T22:02:57.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf81431d0 con 0x7fbaf80ac600 2021-11-21T22:02:57.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8143470 con 0x7fbaf8144810 2021-11-21T22:02:57.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf8145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:57.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf8145310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53788/0 (socket says 172.21.2.116:53788) 2021-11-21T22:02:57.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 -- 172.21.2.116:0/3938208413 learned_addr learned my addr 172.21.2.116:0/3938208413 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:57.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 -- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf8144810 msgr2=0x7fbaf8157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:57.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf8144810 0x7fbaf8157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:57.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 -- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbaf80ac600 msgr2=0x7fbaf8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:57.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbaf80ac600 0x7fbaf8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:57.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 -- 172.21.2.116:0/3938208413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf400b180 con 0x7fbaf80abce0 2021-11-21T22:02:57.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafd0fe700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf8145310 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fbaf8062500 tx=0x7fbaec007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:57.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaea7fc700 1 -- 172.21.2.116:0/3938208413 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaec00ce30 con 0x7fbaf80abce0 2021-11-21T22:02:57.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaea7fc700 1 -- 172.21.2.116:0/3938208413 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec0072e0 con 0x7fbaf80abce0 2021-11-21T22:02:57.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf8143710 con 0x7fbaf80abce0 2021-11-21T22:02:57.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf8157910 con 0x7fbaf80abce0 2021-11-21T22:02:57.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaea7fc700 1 -- 172.21.2.116:0/3938208413 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaec005800 con 0x7fbaf80abce0 2021-11-21T22:02:57.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf81474f0 con 0x7fbaf80abce0 2021-11-21T22:02:57.133 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaea7fc700 1 -- 172.21.2.116:0/3938208413 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbaec010070 con 0x7fbaf80abce0 2021-11-21T22:02:57.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaea7fc700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad80536f0 0x7fbad8055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:57.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbaea7fc700 1 -- 172.21.2.116:0/3938208413 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fbaec06af60 con 0x7fbaf80abce0 2021-11-21T22:02:57.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.119+0000 7fbafc8fd700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad80536f0 0x7fbad8055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:57.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.123+0000 7fbafc8fd700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad80536f0 0x7fbad8055bb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fbaf80b4e00 tx=0x7fbaf4012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:57.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.123+0000 7fbaea7fc700 1 -- 172.21.2.116:0/3938208413 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbaec042180 con 0x7fbaf80abce0 2021-11-21T22:02:57.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:57.207+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fbaf80573d0 con 0x7fbaf80abce0 2021-11-21T22:02:58.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaea7fc700 1 -- 172.21.2.116:0/3938208413 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v150) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbaec0457a0 con 0x7fbaf80abce0 2021-11-21T22:02:58.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad80536f0 msgr2=0x7fbad8055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad80536f0 0x7fbad8055bb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fbaf80b4e00 tx=0x7fbaf4012000).stop 2021-11-21T22:02:58.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 msgr2=0x7fbaf8145310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf8145310 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fbaf8062500 tx=0x7fbaec007ca0).stop 2021-11-21T22:02:58.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 shutdown_connections 2021-11-21T22:02:58.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbaf8144810 0x7fbaf8157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbaf80ac600 0x7fbaf8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbad80536f0 0x7fbad8055bb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 --2- 172.21.2.116:0/3938208413 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbaf80abce0 0x7fbaf8145310 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 >> 172.21.2.116:0/3938208413 conn(0x7fbaf805bd20 msgr2=0x7fbaf81401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 shutdown_connections 2021-11-21T22:02:58.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.067+0000 7fbaff362700 1 -- 172.21.2.116:0/3938208413 wait complete. 2021-11-21T22:02:58.074 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:02:58.078 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:02:58.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f21bd6700 1 Processor -- start 2021-11-21T22:02:58.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f21bd6700 1 -- start start 2021-11-21T22:02:58.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f21bd6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c14fbe0 0x7f1f1c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f21bd6700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c0b1940 con 0x7f1f1c0b11c0 2021-11-21T22:02:58.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f21bd6700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c0b15a0 con 0x7f1f1c0a8670 2021-11-21T22:02:58.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f21bd6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0b1b70 con 0x7f1f1c14fbe0 2021-11-21T22:02:58.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1b7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f1f1c0b11c0 0x7f1f1c147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37516/0 (socket says 172.21.2.116:37516) 2021-11-21T22:02:58.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1b7fe700 1 -- 172.21.2.116:0/3703947971 learned_addr learned my addr 172.21.2.116:0/3703947971 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:58.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836505042 0 0) 0x7f1f1c0b1940 con 0x7f1f1c0b11c0 2021-11-21T22:02:58.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1efc003260 con 0x7f1f1c0b11c0 2021-11-21T22:02:58.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2656361398 0 0) 0x7f1efc003260 con 0x7f1f1c0b11c0 2021-11-21T22:02:58.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c14fbe0 msgr2=0x7f1f1c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c14fbe0 0x7f1f1c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 >> v1:172.21.2.117:6789/0 conn(0x7f1f1c0a8670 legacy=0x7f1f1c147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c0b1e10 con 0x7f1f1c0b11c0 2021-11-21T22:02:58.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.127+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f1f04001660 con 0x7f1f1c0b11c0 2021-11-21T22:02:58.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 >> v1:172.21.2.116:6789/0 conn(0x7f1f1c0b11c0 legacy=0x7f1f1c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1efc003780 0x7f1efc003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c14fbe0 0x7f1efc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1efc004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0b1b70 con 0x7f1efc004ac0 2021-11-21T22:02:58.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc00b910 con 0x7f1efc003780 2021-11-21T22:02:58.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc00bad0 con 0x7f1f1c14fbe0 2021-11-21T22:02:58.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1efc004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1affd700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c14fbe0 0x7f1efc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 -- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c14fbe0 msgr2=0x7f1efc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c14fbe0 0x7f1efc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 -- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1efc003780 msgr2=0x7f1efc003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1efc003780 0x7f1efc003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 -- 172.21.2.116:0/3703947971 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc003260 con 0x7f1efc004ac0 2021-11-21T22:02:58.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1efc004f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7f1f140032f0 tx=0x7f1f14006f90).ready entity=mon.0 client_cookie=e6e5d73edb98f28b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f14011400 con 0x7f1efc004ac0 2021-11-21T22:02:58.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14011560 con 0x7f1efc004ac0 2021-11-21T22:02:58.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1a7fc700 1 -- 172.21.2.116:0/3703947971 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f140118c0 con 0x7f1efc004ac0 2021-11-21T22:02:58.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3703947971 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 msgr2=0x7f1efc004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 --2- 172.21.2.116:0/3703947971 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1efc004f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7f1f140032f0 tx=0x7f1f14006f90).stop 2021-11-21T22:02:58.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1bfff700 1 -- 172.21.2.116:0/3703947971 reap_dead start 2021-11-21T22:02:58.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3703947971 shutdown_connections 2021-11-21T22:02:58.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3703947971 >> 172.21.2.116:0/3703947971 conn(0x7f1f1c05bd20 msgr2=0x7f1f1c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3703947971 shutdown_connections 2021-11-21T22:02:58.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3703947971 wait complete. 2021-11-21T22:02:58.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 Processor -- start 2021-11-21T22:02:58.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- start start 2021-11-21T22:02:58.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1f1c13c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c0aef90 0x7f1f1c0648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1f1c064de0 0x7f1f1c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0b5980 con 0x7f1efc004ac0 2021-11-21T22:02:58.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0b56e0 con 0x7f1f1c064de0 2021-11-21T22:02:58.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c15a330 con 0x7f1f1c0aef90 2021-11-21T22:02:58.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1f1c13c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1f1c13c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53804/0 (socket says 172.21.2.116:53804) 2021-11-21T22:02:58.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 -- 172.21.2.116:0/3506810172 learned_addr learned my addr 172.21.2.116:0/3506810172 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:58.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 -- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c0aef90 msgr2=0x7f1f1c0648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c0aef90 0x7f1f1c0648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 -- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1f1c064de0 msgr2=0x7f1f1c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1f1c064de0 0x7f1f1c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 -- 172.21.2.116:0/3506810172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c15a5d0 con 0x7f1efc004ac0 2021-11-21T22:02:58.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f1b7fe700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1f1c13c920 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7f1f1c061da0 tx=0x7f1f1c0b1fa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f18ff9700 1 -- 172.21.2.116:0/3506810172 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f040071d0 con 0x7f1efc004ac0 2021-11-21T22:02:58.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f18ff9700 1 -- 172.21.2.116:0/3506810172 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04007330 con 0x7f1efc004ac0 2021-11-21T22:02:58.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c065290 con 0x7f1efc004ac0 2021-11-21T22:02:58.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f18ff9700 1 -- 172.21.2.116:0/3506810172 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f04004920 con 0x7f1efc004ac0 2021-11-21T22:02:58.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c0b3280 con 0x7f1efc004ac0 2021-11-21T22:02:58.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.131+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c0b20b0 con 0x7f1efc004ac0 2021-11-21T22:02:58.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.135+0000 7f1f18ff9700 1 -- 172.21.2.116:0/3506810172 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1f04010070 con 0x7f1efc004ac0 2021-11-21T22:02:58.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.135+0000 7f1f18ff9700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1f000536a0 0x7f1f00055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.135+0000 7f1f18ff9700 1 -- 172.21.2.116:0/3506810172 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f1f0406b4d0 con 0x7f1efc004ac0 2021-11-21T22:02:58.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.135+0000 7f1f1affd700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1f000536a0 0x7f1f00055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.135+0000 7f1f1affd700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1f000536a0 0x7f1f00055b60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1f0c00b000 tx=0x7f1f0c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.135+0000 7f1f18ff9700 1 -- 172.21.2.116:0/3506810172 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1f040426a0 con 0x7f1efc004ac0 2021-11-21T22:02:58.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.247+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f1f1c0b3280 con 0x7f1efc004ac0 2021-11-21T22:02:58.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.251+0000 7f1f18ff9700 1 -- 172.21.2.116:0/3506810172 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v69) v1 ==== 189+0+0 (secure 0 0 0) 0x7f1f1c0b3280 con 0x7f1efc004ac0 2021-11-21T22:02:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1f000536a0 msgr2=0x7f1f00055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1f000536a0 0x7f1f00055b60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1f0c00b000 tx=0x7f1f0c009040).stop 2021-11-21T22:02:58.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 msgr2=0x7f1f1c13c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1f1c13c920 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7f1f1c061da0 tx=0x7f1f1c0b1fa0).stop 2021-11-21T22:02:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 shutdown_connections 2021-11-21T22:02:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1f1c064de0 0x7f1f1c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1f1c0aef90 0x7f1f1c0648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1f000536a0 0x7f1f00055b60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 --2- 172.21.2.116:0/3506810172 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1efc004ac0 0x7f1f1c13c920 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 >> 172.21.2.116:0/3506810172 conn(0x7f1f1c05bd20 msgr2=0x7f1f1c1485e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 shutdown_connections 2021-11-21T22:02:58.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.255+0000 7f1f21bd6700 1 -- 172.21.2.116:0/3506810172 wait complete. 2021-11-21T22:02:58.260 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 27 and data pool 28 2021-11-21T22:02:58.264 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:58.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748cfea700 1 Processor -- start 2021-11-21T22:02:58.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748cfea700 1 -- start start 2021-11-21T22:02:58.316 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748cfea700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f74880b5660 0x7f748813bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748cfea700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f74880aff60 0x7f74880b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748cfea700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74880b5a40 con 0x7f74880b1660 2021-11-21T22:02:58.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748cfea700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880aef20 con 0x7f74880aff60 2021-11-21T22:02:58.317 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748cfea700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880ae540 con 0x7f74880b5660 2021-11-21T22:02:58.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f7485d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f74880b1660 0x7f74880b0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37548/0 (socket says 172.21.2.116:37548) 2021-11-21T22:02:58.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f7485d9b700 1 -- 172.21.2.116:0/1693695514 learned_addr learned my addr 172.21.2.116:0/1693695514 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:58.318 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f7486d9d700 1 --2- 172.21.2.116:0/1693695514 >> v2:172.21.2.116:3300/0 conn(0x7f74880aff60 0x7f74880b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748559a700 1 -- 172.21.2.116:0/1693695514 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395055509 0 0) 0x7f74880b5a40 con 0x7f74880b1660 2021-11-21T22:02:58.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f748559a700 1 -- 172.21.2.116:0/1693695514 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7468003260 con 0x7f74880b1660 2021-11-21T22:02:58.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f7486d9d700 1 -- 172.21.2.116:0/1693695514 >> v2:172.21.2.117:3300/0 conn(0x7f74880b5660 msgr2=0x7f748813bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f7486d9d700 1 --2- 172.21.2.116:0/1693695514 >> v2:172.21.2.117:3300/0 conn(0x7f74880b5660 0x7f748813bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f7486d9d700 1 -- 172.21.2.116:0/1693695514 >> v1:172.21.2.116:6789/0 conn(0x7f74880b1660 legacy=0x7f74880b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.311+0000 7f7486d9d700 1 -- 172.21.2.116:0/1693695514 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880ae7e0 con 0x7f74880aff60 2021-11-21T22:02:58.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 --2- 172.21.2.116:0/1693695514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880aff60 0x7f74880b0e30 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f74800032f0 tx=0x7f748000a370).ready entity=mon.0 client_cookie=b85f63161b4cc48 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748559a700 1 -- 172.21.2.116:0/1693695514 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f748000cb70 con 0x7f74880aff60 2021-11-21T22:02:58.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748559a700 1 -- 172.21.2.116:0/1693695514 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748000ccd0 con 0x7f74880aff60 2021-11-21T22:02:58.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748559a700 1 -- 172.21.2.116:0/1693695514 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7480010600 con 0x7f74880aff60 2021-11-21T22:02:58.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/1693695514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880aff60 msgr2=0x7f74880b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 --2- 172.21.2.116:0/1693695514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880aff60 0x7f74880b0e30 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f74800032f0 tx=0x7f748000a370).stop 2021-11-21T22:02:58.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/1693695514 shutdown_connections 2021-11-21T22:02:58.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 --2- 172.21.2.116:0/1693695514 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880aff60 0x7f74880b0e30 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 --2- 172.21.2.116:0/1693695514 >> v2:172.21.2.117:3300/0 conn(0x7f74880b5660 0x7f748813bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/1693695514 >> 172.21.2.116:0/1693695514 conn(0x7f748805f730 msgr2=0x7f7488063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/1693695514 shutdown_connections 2021-11-21T22:02:58.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/1693695514 wait complete. 2021-11-21T22:02:58.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 Processor -- start 2021-11-21T22:02:58.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- start start 2021-11-21T22:02:58.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74880aff60 0x7f74881399c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74880b1660 0x7f7488144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880b5660 0x7f748814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880a9660 con 0x7f74880b5660 2021-11-21T22:02:58.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748814dd60 con 0x7f74880aff60 2021-11-21T22:02:58.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74881383e0 con 0x7f74880b1660 2021-11-21T22:02:58.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880b5660 0x7f748814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880b5660 0x7f748814e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53830/0 (socket says 172.21.2.116:53830) 2021-11-21T22:02:58.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 -- 172.21.2.116:0/3722412125 learned_addr learned my addr 172.21.2.116:0/3722412125 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:58.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 -- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74880b1660 msgr2=0x7f7488144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74880b1660 0x7f7488144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 -- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74880aff60 msgr2=0x7f74881399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74880aff60 0x7f74881399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 -- 172.21.2.116:0/3722412125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748000a610 con 0x7f74880b5660 2021-11-21T22:02:58.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f7486d9d700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880b5660 0x7f748814e430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f74880aa310 tx=0x7f74880ae2e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f747f7fe700 1 -- 172.21.2.116:0/3722412125 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f748001f070 con 0x7f74880b5660 2021-11-21T22:02:58.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f747f7fe700 1 -- 172.21.2.116:0/3722412125 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7480019660 con 0x7f74880b5660 2021-11-21T22:02:58.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f747f7fe700 1 -- 172.21.2.116:0/3722412125 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74800199c0 con 0x7f74880b5660 2021-11-21T22:02:58.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748813b460 con 0x7f74880b5660 2021-11-21T22:02:58.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880aadd0 con 0x7f74880b5660 2021-11-21T22:02:58.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880aea80 con 0x7f74880b5660 2021-11-21T22:02:58.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f747f7fe700 1 -- 172.21.2.116:0/3722412125 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f748000e040 con 0x7f74880b5660 2021-11-21T22:02:58.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.315+0000 7f747f7fe700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f746c057a60 0x7f746c059f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.319+0000 7f747f7fe700 1 -- 172.21.2.116:0/3722412125 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f748006d1f0 con 0x7f74880b5660 2021-11-21T22:02:58.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.319+0000 7f748659c700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f746c057a60 0x7f746c059f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.319+0000 7f748659c700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f746c057a60 0x7f746c059f20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f7470002f70 tx=0x7f74700069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.319+0000 7f747f7fe700 1 -- 172.21.2.116:0/3722412125 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7480044350 con 0x7f74880b5660 2021-11-21T22:02:58.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.403+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74880573d0 con 0x7f74880b5660 2021-11-21T22:02:58.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f747f7fe700 1 -- 172.21.2.116:0/3722412125 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f7480047970 con 0x7f74880b5660 2021-11-21T22:02:58.410 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:58.410 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":151,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:58.250659+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:56.060658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:57.220549+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:58.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f746c057a60 msgr2=0x7f746c059f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f746c057a60 0x7f746c059f20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f7470002f70 tx=0x7f74700069e0).stop 2021-11-21T22:02:58.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880b5660 msgr2=0x7f748814e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880b5660 0x7f748814e430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f74880aa310 tx=0x7f74880ae2e0).stop 2021-11-21T22:02:58.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 shutdown_connections 2021-11-21T22:02:58.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f746c057a60 0x7f746c059f20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74880b5660 0x7f748814e430 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74880b1660 0x7f7488144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 --2- 172.21.2.116:0/3722412125 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74880aff60 0x7f74881399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 >> 172.21.2.116:0/3722412125 conn(0x7f748805f730 msgr2=0x7f748813bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 shutdown_connections 2021-11-21T22:02:58.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.407+0000 7f748cfea700 1 -- 172.21.2.116:0/3722412125 wait complete. 2021-11-21T22:02:58.417 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:58.469 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.463+0000 7fc27412e700 1 Processor -- start 2021-11-21T22:02:58.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.463+0000 7fc27412e700 1 -- start start 2021-11-21T22:02:58.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc26c0b24a0 0x7fc26c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.470 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 0x7fc26c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26c0b00d0 con 0x7fc26c0aeb70 2021-11-21T22:02:58.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc26c0adba0 con 0x7fc26c0b24a0 2021-11-21T22:02:58.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c0af7a0 con 0x7fc26c0af360 2021-11-21T22:02:58.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fc26c0aeb70 0x7fc26c069e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37562/0 (socket says 172.21.2.116:37562) 2021-11-21T22:02:58.471 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 -- 172.21.2.116:0/872746128 learned_addr learned my addr 172.21.2.116:0/872746128 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:58.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc271eca700 1 --2- 172.21.2.116:0/872746128 >> v2:172.21.2.116:3300/0 conn(0x7fc26c0b24a0 0x7fc26c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc271eca700 1 -- 172.21.2.116:0/872746128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 msgr2=0x7fc26c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc271eca700 1 --2- 172.21.2.116:0/872746128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 0x7fc26c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.472 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc271eca700 1 -- 172.21.2.116:0/872746128 >> v1:172.21.2.116:6789/0 conn(0x7fc26c0aeb70 legacy=0x7fc26c069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc271eca700 1 -- 172.21.2.116:0/872746128 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c0b2880 con 0x7fc26c0b24a0 2021-11-21T22:02:58.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc271eca700 1 --2- 172.21.2.116:0/872746128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0b1a40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7fc260002f70 tx=0x7fc260006df0).ready entity=mon.0 client_cookie=77ad504bc831dcf9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc270ec8700 1 -- 172.21.2.116:0/872746128 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc26000d570 con 0x7fc26c0b24a0 2021-11-21T22:02:58.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc270ec8700 1 -- 172.21.2.116:0/872746128 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260000e90 con 0x7fc26c0b24a0 2021-11-21T22:02:58.473 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc270ec8700 1 -- 172.21.2.116:0/872746128 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc26000d8c0 con 0x7fc26c0b24a0 2021-11-21T22:02:58.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- 172.21.2.116:0/872746128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 msgr2=0x7fc26c0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- 172.21.2.116:0/872746128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0b1a40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7fc260002f70 tx=0x7fc260006df0).stop 2021-11-21T22:02:58.477 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- 172.21.2.116:0/872746128 shutdown_connections 2021-11-21T22:02:58.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- 172.21.2.116:0/872746128 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 0x7fc26c0ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- 172.21.2.116:0/872746128 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0b1a40 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- 172.21.2.116:0/872746128 >> 172.21.2.116:0/872746128 conn(0x7fc26c05fd00 msgr2=0x7fc26c0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.478 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- 172.21.2.116:0/872746128 shutdown_connections 2021-11-21T22:02:58.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- 172.21.2.116:0/872746128 wait complete. 2021-11-21T22:02:58.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 Processor -- start 2021-11-21T22:02:58.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- start start 2021-11-21T22:02:58.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc26c0aeb70 0x7fc26c140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 0x7fc26c1404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0656b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c066440 con 0x7fc26c0b24a0 2021-11-21T22:02:58.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c066bb0 con 0x7fc26c0aeb70 2021-11-21T22:02:58.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c066670 con 0x7fc26c0af360 2021-11-21T22:02:58.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0656b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53846/0 (socket says 172.21.2.116:53846) 2021-11-21T22:02:58.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 -- 172.21.2.116:0/1569160006 learned_addr learned my addr 172.21.2.116:0/1569160006 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:58.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 -- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 msgr2=0x7fc26c1404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.481 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 0x7fc26c1404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 -- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc26c0aeb70 msgr2=0x7fc26c140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc26c0aeb70 0x7fc26c140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 -- 172.21.2.116:0/1569160006 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26000b180 con 0x7fc26c0b24a0 2021-11-21T22:02:58.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc2726cb700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0656b0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fc26c0630d0 tx=0x7fc264007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc25effd700 1 -- 172.21.2.116:0/1569160006 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc264005ba0 con 0x7fc26c0b24a0 2021-11-21T22:02:58.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc25effd700 1 -- 172.21.2.116:0/1569160006 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc264005d00 con 0x7fc26c0b24a0 2021-11-21T22:02:58.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c066910 con 0x7fc26c0b24a0 2021-11-21T22:02:58.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc25effd700 1 -- 172.21.2.116:0/1569160006 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2640072c0 con 0x7fc26c0b24a0 2021-11-21T22:02:58.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.467+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c065ba0 con 0x7fc26c0b24a0 2021-11-21T22:02:58.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.471+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c142690 con 0x7fc26c0b24a0 2021-11-21T22:02:58.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.471+0000 7fc25effd700 1 -- 172.21.2.116:0/1569160006 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc264010070 con 0x7fc26c0b24a0 2021-11-21T22:02:58.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.471+0000 7fc25effd700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc24c0536d0 0x7fc24c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.471+0000 7fc25effd700 1 -- 172.21.2.116:0/1569160006 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc26406b330 con 0x7fc26c0b24a0 2021-11-21T22:02:58.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.471+0000 7fc271eca700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc24c0536d0 0x7fc24c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.471+0000 7fc271eca700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc24c0536d0 0x7fc24c055b90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fc260002f70 tx=0x7fc260015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.485 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.471+0000 7fc25effd700 1 -- 172.21.2.116:0/1569160006 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc26c142690 con 0x7fc26c0b24a0 2021-11-21T22:02:58.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.555+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc26c142690 con 0x7fc26c0b24a0 2021-11-21T22:02:58.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.555+0000 7fc25effd700 1 -- 172.21.2.116:0/1569160006 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fc26c142690 con 0x7fc26c0b24a0 2021-11-21T22:02:58.560 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:58.561 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":151,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:58.250659+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:56.060658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:57.220549+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc24c0536d0 msgr2=0x7fc24c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc24c0536d0 0x7fc24c055b90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fc260002f70 tx=0x7fc260015000).stop 2021-11-21T22:02:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 msgr2=0x7fc26c0656b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0656b0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fc26c0630d0 tx=0x7fc264007f00).stop 2021-11-21T22:02:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 shutdown_connections 2021-11-21T22:02:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc24c0536d0 0x7fc24c055b90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc26c0b24a0 0x7fc26c0656b0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc26c0af360 0x7fc26c1404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 --2- 172.21.2.116:0/1569160006 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc26c0aeb70 0x7fc26c140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 >> 172.21.2.116:0/1569160006 conn(0x7fc26c05fd00 msgr2=0x7fc26c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 shutdown_connections 2021-11-21T22:02:58.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.559+0000 7fc27412e700 1 -- 172.21.2.116:0/1569160006 wait complete. 2021-11-21T22:02:58.568 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:02:58.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 Processor -- start 2021-11-21T22:02:58.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- start start 2021-11-21T22:02:58.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec40adc10 0x7f7ec40b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec40b3c50 con 0x7f7ec4144a10 2021-11-21T22:02:58.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec40b3ef0 con 0x7f7ec40b3690 2021-11-21T22:02:58.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40b4190 con 0x7f7ec40adc10 2021-11-21T22:02:58.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7ec4144a10 0x7f7ec4141020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37574/0 (socket says 172.21.2.116:37574) 2021-11-21T22:02:58.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 -- 172.21.2.116:0/2258389313 learned_addr learned my addr 172.21.2.116:0/2258389313 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:58.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9d18700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec40adc10 0x7f7ec40b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642638549 0 0) 0x7f7ec40b3c50 con 0x7f7ec4144a10 2021-11-21T22:02:58.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac003260 con 0x7f7ec4144a10 2021-11-21T22:02:58.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990885956 0 0) 0x7f7eac003260 con 0x7f7ec4144a10 2021-11-21T22:02:58.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec40adc10 msgr2=0x7f7ec40b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec40adc10 0x7f7ec40b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 >> v1:172.21.2.117:6789/0 conn(0x7f7ec40b3690 legacy=0x7f7ec413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec4140b80 con 0x7f7ec4144a10 2021-11-21T22:02:58.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f7ec0001660 con 0x7f7ec4144a10 2021-11-21T22:02:58.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 >> v1:172.21.2.116:6789/0 conn(0x7f7ec4144a10 legacy=0x7f7ec4141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7eac003780 0x7f7eac003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7eac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec40adc10 0x7f7eac004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40b4190 con 0x7f7eac004500 2021-11-21T22:02:58.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac00fb90 con 0x7f7eac003780 2021-11-21T22:02:58.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac00fd50 con 0x7f7ec40adc10 2021-11-21T22:02:58.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7eac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 -- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec40adc10 msgr2=0x7f7eac004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec40adc10 0x7f7eac004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 -- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7eac003780 msgr2=0x7f7eac003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:58.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7eac003780 0x7f7eac003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:58.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 -- 172.21.2.116:0/2258389313 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec4140b80 con 0x7f7eac004500 2021-11-21T22:02:58.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec9517700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7eac004980 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f7ec0001370 tx=0x7f7ec0001dc0).ready entity=mon.0 client_cookie=3cff9443eac06986 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec00071a0 con 0x7f7eac004500 2021-11-21T22:02:58.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec0007300 con 0x7f7eac004500 2021-11-21T22:02:58.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ec8d16700 1 -- 172.21.2.116:0/2258389313 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec000cae0 con 0x7f7eac004500 2021-11-21T22:02:58.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/2258389313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 msgr2=0x7f7eac004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 --2- 172.21.2.116:0/2258389313 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7eac004980 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f7ec0001370 tx=0x7f7ec0001dc0).stop 2021-11-21T22:02:58.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7eca519700 1 -- 172.21.2.116:0/2258389313 reap_dead start 2021-11-21T22:02:58.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/2258389313 shutdown_connections 2021-11-21T22:02:58.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/2258389313 >> 172.21.2.116:0/2258389313 conn(0x7f7ec405bd20 msgr2=0x7f7ec405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:58.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/2258389313 shutdown_connections 2021-11-21T22:02:58.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.619+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/2258389313 wait complete. 2021-11-21T22:02:58.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 Processor -- start 2021-11-21T22:02:58.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 -- start start 2021-11-21T22:02:58.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7ec4066d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ec40b1110 0x7f7ec41599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec4159eb0 0x7f7ec40b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec4118c30 con 0x7f7eac004500 2021-11-21T22:02:58.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40aefc0 con 0x7f7ec40b1110 2021-11-21T22:02:58.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40af260 con 0x7f7ec4159eb0 2021-11-21T22:02:58.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7ec4066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7ec4066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53864/0 (socket says 172.21.2.116:53864) 2021-11-21T22:02:58.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 -- 172.21.2.116:0/3066042147 learned_addr learned my addr 172.21.2.116:0/3066042147 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:58.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eca519700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec4159eb0 0x7f7ec40b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9517700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ec40b1110 0x7f7ec41599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 -- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec4159eb0 msgr2=0x7f7ec40b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec4159eb0 0x7f7ec40b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 -- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ec40b1110 msgr2=0x7f7ec41599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:58.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ec40b1110 0x7f7ec41599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:58.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 -- 172.21.2.116:0/3066042147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec4067270 con 0x7f7eac004500 2021-11-21T22:02:58.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eca519700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec4159eb0 0x7f7ec40b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:58.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9d18700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7ec4066d80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f7eb800c000 tx=0x7f7eb8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eb6ffd700 1 -- 172.21.2.116:0/3066042147 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb8016070 con 0x7f7eac004500 2021-11-21T22:02:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eb6ffd700 1 -- 172.21.2.116:0/3066042147 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb8000e90 con 0x7f7eac004500 2021-11-21T22:02:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec40af750 con 0x7f7eac004500 2021-11-21T22:02:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec415a330 con 0x7f7eac004500 2021-11-21T22:02:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eb6ffd700 1 -- 172.21.2.116:0/3066042147 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb800ca20 con 0x7f7eac004500 2021-11-21T22:02:58.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec40af420 con 0x7f7eac004500 2021-11-21T22:02:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eb6ffd700 1 -- 172.21.2.116:0/3066042147 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7eb800cb80 con 0x7f7eac004500 2021-11-21T22:02:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eb6ffd700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ea40536d0 0x7f7ea4055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7eb6ffd700 1 -- 172.21.2.116:0/3066042147 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7eb806b9c0 con 0x7f7eac004500 2021-11-21T22:02:58.642 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.623+0000 7f7ec9517700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ea40536d0 0x7f7ea4055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:58.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.627+0000 7f7ec9517700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ea40536d0 0x7f7ea4055b90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f7ec0016000 tx=0x7f7ec0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:58.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.627+0000 7f7eb6ffd700 1 -- 172.21.2.116:0/3066042147 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7eb8042b20 con 0x7f7eac004500 2021-11-21T22:02:58.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:58.735+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7ec40573d0 con 0x7f7eac004500 2021-11-21T22:02:59.269 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7eb6ffd700 1 -- 172.21.2.116:0/3066042147 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7eb8046140 con 0x7f7eac004500 2021-11-21T22:02:59.270 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ea40536d0 msgr2=0x7f7ea4055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ea40536d0 0x7f7ea4055b90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f7ec0016000 tx=0x7f7ec0015000).stop 2021-11-21T22:02:59.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 msgr2=0x7f7ec4066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7ec4066d80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f7eb800c000 tx=0x7f7eb8006df0).stop 2021-11-21T22:02:59.271 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 shutdown_connections 2021-11-21T22:02:59.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7ec4159eb0 0x7f7ec40b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7ec40b1110 0x7f7ec41599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7ea40536d0 0x7f7ea4055b90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.272 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 --2- 172.21.2.116:0/3066042147 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7eac004500 0x7f7ec4066d80 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 >> 172.21.2.116:0/3066042147 conn(0x7f7ec405bd20 msgr2=0x7f7ec413ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 shutdown_connections 2021-11-21T22:02:59.273 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.267+0000 7f7ecbf7c700 1 -- 172.21.2.116:0/3066042147 wait complete. 2021-11-21T22:02:59.277 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:02:59.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.323+0000 7f3894713700 1 Processor -- start 2021-11-21T22:02:59.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.323+0000 7f3894713700 1 -- start start 2021-11-21T22:02:59.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c0a9d00 0x7f388c13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f388c05c840 0x7f388c13e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c144970 con 0x7f388c05cef0 2021-11-21T22:02:59.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c13e750 con 0x7f388c05c840 2021-11-21T22:02:59.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0b1200 con 0x7f388c0a9d00 2021-11-21T22:02:59.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f388c05c840 0x7f388c13e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f388c05c840 0x7f388c13e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53872/0 (socket says 172.21.2.116:53872) 2021-11-21T22:02:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 -- 172.21.2.116:0/1538953386 learned_addr learned my addr 172.21.2.116:0/1538953386 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 -- 172.21.2.116:0/1538953386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c0a9d00 msgr2=0x7f388c13dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 --2- 172.21.2.116:0/1538953386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c0a9d00 0x7f388c13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 --2- 172.21.2.116:0/1538953386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c0a9d00 0x7f388c13dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 -- 172.21.2.116:0/1538953386 >> v1:172.21.2.117:6789/0 conn(0x7f388c05cef0 legacy=0x7f388c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:59.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 -- 172.21.2.116:0/1538953386 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0aa0e0 con 0x7f388c05c840 2021-11-21T22:02:59.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 --2- 172.21.2.116:0/1538953386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c0a9d00 0x7f388c13dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:59.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 --2- 172.21.2.116:0/1538953386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c13e260 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f388c061ab0 tx=0x7f3880006eb0).ready entity=mon.0 client_cookie=80059c63e0a86e1f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38914ad700 1 -- 172.21.2.116:0/1538953386 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f388000d570 con 0x7f388c05c840 2021-11-21T22:02:59.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38914ad700 1 -- 172.21.2.116:0/1538953386 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3880000e90 con 0x7f388c05c840 2021-11-21T22:02:59.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38914ad700 1 -- 172.21.2.116:0/1538953386 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f388000d8c0 con 0x7f388c05c840 2021-11-21T22:02:59.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- 172.21.2.116:0/1538953386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 msgr2=0x7f388c13e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- 172.21.2.116:0/1538953386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c13e260 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f388c061ab0 tx=0x7f3880006eb0).stop 2021-11-21T22:02:59.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- 172.21.2.116:0/1538953386 shutdown_connections 2021-11-21T22:02:59.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- 172.21.2.116:0/1538953386 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c13e260 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- 172.21.2.116:0/1538953386 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c0a9d00 0x7f388c13dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- 172.21.2.116:0/1538953386 >> 172.21.2.116:0/1538953386 conn(0x7f388c05bd20 msgr2=0x7f388c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- 172.21.2.116:0/1538953386 shutdown_connections 2021-11-21T22:02:59.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- 172.21.2.116:0/1538953386 wait complete. 2021-11-21T22:02:59.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 Processor -- start 2021-11-21T22:02:59.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- start start 2021-11-21T22:02:59.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c05cef0 0x7f388c0a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f388c0a9d00 0x7f388c0ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c156200 con 0x7f388c05c840 2021-11-21T22:02:59.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c153c60 con 0x7f388c0a9d00 2021-11-21T22:02:59.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c1563f0 con 0x7f388c05cef0 2021-11-21T22:02:59.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c0ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53878/0 (socket says 172.21.2.116:53878) 2021-11-21T22:02:59.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 -- 172.21.2.116:0/1932282661 learned_addr learned my addr 172.21.2.116:0/1932282661 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:59.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3891cae700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c05cef0 0x7f388c0a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3892cb0700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f388c0a9d00 0x7f388c0ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 -- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c05cef0 msgr2=0x7f388c0a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c05cef0 0x7f388c0a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 -- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f388c0a9d00 msgr2=0x7f388c0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f388c0a9d00 0x7f388c0ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 -- 172.21.2.116:0/1932282661 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388000b1e0 con 0x7f388c05c840 2021-11-21T22:02:59.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f38924af700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c0ade50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7f3888007000 tx=0x7f3888008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f387f7fe700 1 -- 172.21.2.116:0/1932282661 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3888010650 con 0x7f388c05c840 2021-11-21T22:02:59.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f387f7fe700 1 -- 172.21.2.116:0/1932282661 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38880107b0 con 0x7f388c05c840 2021-11-21T22:02:59.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c1566f0 con 0x7f388c05c840 2021-11-21T22:02:59.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f387f7fe700 1 -- 172.21.2.116:0/1932282661 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3888010b10 con 0x7f388c05c840 2021-11-21T22:02:59.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.327+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c1570f0 con 0x7f388c05c840 2021-11-21T22:02:59.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.331+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c0abb80 con 0x7f388c05c840 2021-11-21T22:02:59.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.331+0000 7f387f7fe700 1 -- 172.21.2.116:0/1932282661 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f3888010c70 con 0x7f388c05c840 2021-11-21T22:02:59.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.331+0000 7f387f7fe700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f386c0534f0 0x7f386c0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.331+0000 7f387f7fe700 1 -- 172.21.2.116:0/1932282661 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f388806b970 con 0x7f388c05c840 2021-11-21T22:02:59.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.331+0000 7f3891cae700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f386c0534f0 0x7f386c0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.331+0000 7f3891cae700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f386c0534f0 0x7f386c0559b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f388001c000 tx=0x7f3880015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.331+0000 7f387f7fe700 1 -- 172.21.2.116:0/1932282661 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3888042ad0 con 0x7f388c05c840 2021-11-21T22:02:59.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f388c0573d0 con 0x7f388c05c840 2021-11-21T22:02:59.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f387f7fe700 1 -- 172.21.2.116:0/1932282661 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f38880460f0 con 0x7f388c05c840 2021-11-21T22:02:59.446 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:59.447 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":70,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:58.250794+0000","modified":"2021-11-21T22:02:59.255376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/139139464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":139139464},{"type":"v1","addr":"172.21.2.116:6835","nonce":139139464}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-11-21T22:02:59.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f386c0534f0 msgr2=0x7f386c0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f386c0534f0 0x7f386c0559b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f388001c000 tx=0x7f3880015000).stop 2021-11-21T22:02:59.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 msgr2=0x7f388c0ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c0ade50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7f3888007000 tx=0x7f3888008040).stop 2021-11-21T22:02:59.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 shutdown_connections 2021-11-21T22:02:59.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f388c0a9d00 0x7f388c0ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f388c05cef0 0x7f388c0a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f386c0534f0 0x7f386c0559b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 --2- 172.21.2.116:0/1932282661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f388c05c840 0x7f388c0ade50 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 >> 172.21.2.116:0/1932282661 conn(0x7f388c05bd20 msgr2=0x7f388c0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 shutdown_connections 2021-11-21T22:02:59.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.443+0000 7f3894713700 1 -- 172.21.2.116:0/1932282661 wait complete. 2021-11-21T22:02:59.451 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 70 2021-11-21T22:02:59.453 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:02:59.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.499+0000 7f432c083700 1 Processor -- start 2021-11-21T22:02:59.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.499+0000 7f432c083700 1 -- start start 2021-11-21T22:02:59.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.499+0000 7f432c083700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a6aa0 0x7f432414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f4324062410 0x7f43241443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240a6e80 con 0x7f43240a86b0 2021-11-21T22:02:59.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240a8bc0 con 0x7f4324062410 2021-11-21T22:02:59.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432414a7b0 con 0x7f43240a6aa0 2021-11-21T22:02:59.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432961e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f43240a86b0 0x7f43241470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37608/0 (socket says 172.21.2.116:37608) 2021-11-21T22:02:59.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432961e700 1 -- 172.21.2.116:0/371675713 learned_addr learned my addr 172.21.2.116:0/371675713 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:59.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432a620700 1 --2- 172.21.2.116:0/371675713 >> v2:172.21.2.116:3300/0 conn(0x7f4324062410 0x7f43241443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4328e1d700 1 -- 172.21.2.116:0/371675713 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366979604 0 0) 0x7f43240a6e80 con 0x7f43240a86b0 2021-11-21T22:02:59.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4328e1d700 1 -- 172.21.2.116:0/371675713 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f430c003260 con 0x7f43240a86b0 2021-11-21T22:02:59.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432a620700 1 -- 172.21.2.116:0/371675713 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a6aa0 msgr2=0x7f432414b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:02:59.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432a620700 1 --2- 172.21.2.116:0/371675713 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a6aa0 0x7f432414b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432a620700 1 -- 172.21.2.116:0/371675713 >> v1:172.21.2.116:6789/0 conn(0x7f43240a86b0 legacy=0x7f43241470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432a620700 1 -- 172.21.2.116:0/371675713 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432414aa10 con 0x7f4324062410 2021-11-21T22:02:59.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432a620700 1 --2- 172.21.2.116:0/371675713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f43241443f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f431c0032f0 tx=0x7f431c0091d0).ready entity=mon.0 client_cookie=ae15a7a96710f341 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4328e1d700 1 -- 172.21.2.116:0/371675713 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f431c00b6c0 con 0x7f4324062410 2021-11-21T22:02:59.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4328e1d700 1 -- 172.21.2.116:0/371675713 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c00b820 con 0x7f4324062410 2021-11-21T22:02:59.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4328e1d700 1 -- 172.21.2.116:0/371675713 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f431c00b9d0 con 0x7f4324062410 2021-11-21T22:02:59.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- 172.21.2.116:0/371675713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 msgr2=0x7f43241443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 --2- 172.21.2.116:0/371675713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f43241443f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f431c0032f0 tx=0x7f431c0091d0).stop 2021-11-21T22:02:59.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- 172.21.2.116:0/371675713 shutdown_connections 2021-11-21T22:02:59.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 --2- 172.21.2.116:0/371675713 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f43241443f0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 --2- 172.21.2.116:0/371675713 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a6aa0 0x7f432414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- 172.21.2.116:0/371675713 >> 172.21.2.116:0/371675713 conn(0x7f432405bd20 msgr2=0x7f432405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- 172.21.2.116:0/371675713 shutdown_connections 2021-11-21T22:02:59.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- 172.21.2.116:0/371675713 wait complete. 2021-11-21T22:02:59.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 Processor -- start 2021-11-21T22:02:59.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- start start 2021-11-21T22:02:59.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f432414d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43240a6aa0 0x7f43240b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a86b0 0x7f432413bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240aa1e0 con 0x7f4324062410 2021-11-21T22:02:59.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240aa480 con 0x7f43240a6aa0 2021-11-21T22:02:59.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240aa720 con 0x7f43240a86b0 2021-11-21T22:02:59.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f432414d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f432414d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53890/0 (socket says 172.21.2.116:53890) 2021-11-21T22:02:59.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 -- 172.21.2.116:0/2829053047 learned_addr learned my addr 172.21.2.116:0/2829053047 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:59.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 -- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a86b0 msgr2=0x7f432413bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a86b0 0x7f432413bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 -- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43240a6aa0 msgr2=0x7f43240b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43240a6aa0 0x7f43240b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 -- 172.21.2.116:0/2829053047 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c009470 con 0x7f4324062410 2021-11-21T22:02:59.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4329e1f700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f432414d0e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f4320002fb0 tx=0x7f4320006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4312ffd700 1 -- 172.21.2.116:0/2829053047 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4320013460 con 0x7f4324062410 2021-11-21T22:02:59.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4312ffd700 1 -- 172.21.2.116:0/2829053047 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43200135c0 con 0x7f4324062410 2021-11-21T22:02:59.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f4312ffd700 1 -- 172.21.2.116:0/2829053047 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4320013920 con 0x7f4324062410 2021-11-21T22:02:59.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432413d510 con 0x7f4324062410 2021-11-21T22:02:59.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.503+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43241598c0 con 0x7f4324062410 2021-11-21T22:02:59.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.507+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432413d6a0 con 0x7f4324062410 2021-11-21T22:02:59.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.507+0000 7f4312ffd700 1 -- 172.21.2.116:0/2829053047 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f432000f070 con 0x7f4324062410 2021-11-21T22:02:59.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.507+0000 7f4312ffd700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4304053610 0x7f4304055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.507+0000 7f4312ffd700 1 -- 172.21.2.116:0/2829053047 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f432006b090 con 0x7f4324062410 2021-11-21T22:02:59.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.507+0000 7f432961e700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4304053610 0x7f4304055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.507+0000 7f432961e700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4304053610 0x7f4304055ad0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f4314001110 tx=0x7f43140076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.507+0000 7f4312ffd700 1 -- 172.21.2.116:0/2829053047 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43200421f0 con 0x7f4324062410 2021-11-21T22:02:59.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.591+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43240ac030 con 0x7f4324062410 2021-11-21T22:02:59.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f4312ffd700 1 -- 172.21.2.116:0/2829053047 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f4320018030 con 0x7f4324062410 2021-11-21T22:02:59.597 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:02:59.598 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":152,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:59.253897+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:56.060658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:57.220549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:02:59.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4304053610 msgr2=0x7f4304055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4304053610 0x7f4304055ad0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f4314001110 tx=0x7f43140076c0).stop 2021-11-21T22:02:59.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 msgr2=0x7f432414d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f432414d0e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f4320002fb0 tx=0x7f4320006df0).stop 2021-11-21T22:02:59.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 shutdown_connections 2021-11-21T22:02:59.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f43240a86b0 0x7f432413bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f43240a6aa0 0x7f43240b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4304053610 0x7f4304055ad0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 --2- 172.21.2.116:0/2829053047 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4324062410 0x7f432414d0e0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 >> 172.21.2.116:0/2829053047 conn(0x7f432405bd20 msgr2=0x7f4324149bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 shutdown_connections 2021-11-21T22:02:59.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.595+0000 7f432c083700 1 -- 172.21.2.116:0/2829053047 wait complete. 2021-11-21T22:02:59.605 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:02:59.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c5a3ad700 1 Processor -- start 2021-11-21T22:02:59.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c5a3ad700 1 -- start start 2021-11-21T22:02:59.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c5a3ad700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8c540b11c0 0x7f8c54147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c5a3ad700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 0x7f8c541442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c5a3ad700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540b1940 con 0x7f8c540a8670 2021-11-21T22:02:59.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c5a3ad700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c540b15a0 con 0x7f8c540b11c0 2021-11-21T22:02:59.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c5a3ad700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b1b70 con 0x7f8c54143e60 2021-11-21T22:02:59.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c537fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8c540a8670 0x7f8c54147720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37622/0 (socket says 172.21.2.116:37622) 2021-11-21T22:02:59.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c537fe700 1 -- 172.21.2.116:0/437813212 learned_addr learned my addr 172.21.2.116:0/437813212 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:59.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249858367 0 0) 0x7f8c540b1940 con 0x7f8c540a8670 2021-11-21T22:02:59.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.655+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c34003260 con 0x7f8c540a8670 2021-11-21T22:02:59.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 0x7f8c541442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1728433527 0 0) 0x7f8c34003260 con 0x7f8c540a8670 2021-11-21T22:02:59.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c53fff700 1 --2- 172.21.2.116:0/437813212 >> v2:172.21.2.117:3300/0 conn(0x7f8c540b11c0 0x7f8c54147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 msgr2=0x7f8c541442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:59.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 0x7f8c541442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 >> v2:172.21.2.117:3300/0 conn(0x7f8c540b11c0 msgr2=0x7f8c54147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:59.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 --2- 172.21.2.116:0/437813212 >> v2:172.21.2.117:3300/0 conn(0x7f8c540b11c0 0x7f8c54147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540b1e10 con 0x7f8c540a8670 2021-11-21T22:02:59.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8c44001660 con 0x7f8c540a8670 2021-11-21T22:02:59.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 >> v1:172.21.2.116:6789/0 conn(0x7f8c540a8670 legacy=0x7f8c54147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 0x7f8c34003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34003e10 0x7f8c34004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c34004b10 0x7f8c34004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b15a0 con 0x7f8c34004b10 2021-11-21T22:02:59.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b1b70 con 0x7f8c34003e10 2021-11-21T22:02:59.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3400b9e0 con 0x7f8c54143e60 2021-11-21T22:02:59.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c53fff700 1 --2- 172.21.2.116:0/437813212 >> v2:172.21.2.117:3300/0 conn(0x7f8c540b11c0 0x7f8c54147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:59.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c34004b10 0x7f8c34004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 msgr2=0x7f8c34003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:59.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c53fff700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 0x7f8c34003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c537fe700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34003e10 0x7f8c34004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 0x7f8c34003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34003e10 msgr2=0x7f8c34004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:59.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34003e10 0x7f8c34004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/437813212 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34003260 con 0x7f8c34004b10 2021-11-21T22:02:59.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c537fe700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34003e10 0x7f8c34004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:59.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c53fff700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54143e60 0x7f8c34003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:02:59.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c34004b10 0x7f8c34004f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f8c4c007000 tx=0x7f8c4c008040).ready entity=mon.0 client_cookie=a057306af6fd2ff1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4c00c640 con 0x7f8c34004b10 2021-11-21T22:02:59.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c00c7a0 con 0x7f8c34004b10 2021-11-21T22:02:59.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c52ffd700 1 -- 172.21.2.116:0/437813212 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4c00cb00 con 0x7f8c34004b10 2021-11-21T22:02:59.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/437813212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c34004b10 msgr2=0x7f8c34004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 --2- 172.21.2.116:0/437813212 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c34004b10 0x7f8c34004f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f8c4c007000 tx=0x7f8c4c008040).stop 2021-11-21T22:02:59.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/437813212 reap_dead start 2021-11-21T22:02:59.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/437813212 shutdown_connections 2021-11-21T22:02:59.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/437813212 >> 172.21.2.116:0/437813212 conn(0x7f8c5405bd20 msgr2=0x7f8c5405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/437813212 shutdown_connections 2021-11-21T22:02:59.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/437813212 wait complete. 2021-11-21T22:02:59.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 Processor -- start 2021-11-21T22:02:59.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- start start 2021-11-21T22:02:59.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34004b10 0x7f8c5414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54159c40 0x7f8c5415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c54141870 0x7f8c54141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5413ed70 con 0x7f8c54141870 2021-11-21T22:02:59.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5413e750 con 0x7f8c34004b10 2021-11-21T22:02:59.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5a3ad700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c541597e0 con 0x7f8c54159c40 2021-11-21T22:02:59.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c54141870 0x7f8c54141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c54141870 0x7f8c54141cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53912/0 (socket says 172.21.2.116:53912) 2021-11-21T22:02:59.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/2821910531 learned_addr learned my addr 172.21.2.116:0/2821910531 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:59.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c537fe700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54159c40 0x7f8c5415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c53fff700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34004b10 0x7f8c5414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54159c40 msgr2=0x7f8c5415a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54159c40 0x7f8c5415a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34004b10 msgr2=0x7f8c5414ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34004b10 0x7f8c5414ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 -- 172.21.2.116:0/2821910531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5413e4b0 con 0x7f8c54141870 2021-11-21T22:02:59.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.659+0000 7f8c5894a700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c54141870 0x7f8c54141cf0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7f8c54068660 tx=0x7f8c4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c517fa700 1 -- 172.21.2.116:0/2821910531 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4c00a4a0 con 0x7f8c54141870 2021-11-21T22:02:59.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c517fa700 1 -- 172.21.2.116:0/2821910531 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c020070 con 0x7f8c54141870 2021-11-21T22:02:59.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5413ead0 con 0x7f8c54141870 2021-11-21T22:02:59.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c517fa700 1 -- 172.21.2.116:0/2821910531 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4c006e10 con 0x7f8c54141870 2021-11-21T22:02:59.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5413f1c0 con 0x7f8c54141870 2021-11-21T22:02:59.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c540b20b0 con 0x7f8c54141870 2021-11-21T22:02:59.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c517fa700 1 -- 172.21.2.116:0/2821910531 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8c4c02c070 con 0x7f8c54141870 2021-11-21T22:02:59.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c517fa700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c38053640 0x7f8c38055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c517fa700 1 -- 172.21.2.116:0/2821910531 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8c4c06cf30 con 0x7f8c54141870 2021-11-21T22:02:59.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c53fff700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c38053640 0x7f8c38055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c53fff700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c38053640 0x7f8c38055b00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f8c3c00c000 tx=0x7f8c3c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.663+0000 7f8c517fa700 1 -- 172.21.2.116:0/2821910531 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c4c044090 con 0x7f8c54141870 2021-11-21T22:02:59.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.775+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8c540573d0 con 0x7f8c54141870 2021-11-21T22:02:59.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c517fa700 1 -- 172.21.2.116:0/2821910531 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8c4c0476b0 con 0x7f8c54141870 2021-11-21T22:02:59.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c38053640 msgr2=0x7f8c38055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c38053640 0x7f8c38055b00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f8c3c00c000 tx=0x7f8c3c008040).stop 2021-11-21T22:02:59.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c54141870 msgr2=0x7f8c54141cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c54141870 0x7f8c54141cf0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7f8c54068660 tx=0x7f8c4c008040).stop 2021-11-21T22:02:59.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 shutdown_connections 2021-11-21T22:02:59.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c38053640 0x7f8c38055b00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c54141870 0x7f8c54141cf0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c54159c40 0x7f8c5415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 --2- 172.21.2.116:0/2821910531 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c34004b10 0x7f8c5414ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 >> 172.21.2.116:0/2821910531 conn(0x7f8c5405bd20 msgr2=0x7f8c540af590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.789 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 shutdown_connections 2021-11-21T22:02:59.790 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.779+0000 7f8c5a3ad700 1 -- 172.21.2.116:0/2821910531 wait complete. 2021-11-21T22:02:59.790 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:02:59.792 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:02:59.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.843+0000 7fec9762d700 1 Processor -- start 2021-11-21T22:02:59.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.843+0000 7fec9762d700 1 -- start start 2021-11-21T22:02:59.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.843+0000 7fec9762d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fec900a8670 0x7fec90147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.843+0000 7fec9762d700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec900b1940 con 0x7fec900b11c0 2021-11-21T22:02:59.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.843+0000 7fec9762d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec900b15a0 con 0x7fec90143e60 2021-11-21T22:02:59.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.843+0000 7fec9762d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec900b1b70 con 0x7fec900a8670 2021-11-21T22:02:59.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec94bc8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fec900a8670 0x7fec90147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fec900b11c0 0x7fec90147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37640/0 (socket says 172.21.2.116:37640) 2021-11-21T22:02:59.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 -- 172.21.2.116:0/2490077550 learned_addr learned my addr 172.21.2.116:0/2490077550 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:02:59.853 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec94bc8700 1 -- 172.21.2.116:0/2490077550 >> v1:172.21.2.117:6789/0 conn(0x7fec90143e60 legacy=0x7fec901442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:02:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec94bc8700 1 -- 172.21.2.116:0/2490077550 >> v1:172.21.2.116:6789/0 conn(0x7fec900b11c0 legacy=0x7fec90147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec94bc8700 1 -- 172.21.2.116:0/2490077550 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec900b1e10 con 0x7fec900a8670 2021-11-21T22:02:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec94bc8700 1 --2- 172.21.2.116:0/2490077550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec90147940 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7fec90068600 tx=0x7fec84006df0).ready entity=mon.0 client_cookie=447ae2b22d87ea4b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec83fff700 1 -- 172.21.2.116:0/2490077550 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8400d5c0 con 0x7fec900a8670 2021-11-21T22:02:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec83fff700 1 -- 172.21.2.116:0/2490077550 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84000e90 con 0x7fec900a8670 2021-11-21T22:02:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec83fff700 1 -- 172.21.2.116:0/2490077550 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8400d910 con 0x7fec900a8670 2021-11-21T22:02:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- 172.21.2.116:0/2490077550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 msgr2=0x7fec90147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 --2- 172.21.2.116:0/2490077550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec90147940 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7fec90068600 tx=0x7fec84006df0).stop 2021-11-21T22:02:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- 172.21.2.116:0/2490077550 shutdown_connections 2021-11-21T22:02:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 --2- 172.21.2.116:0/2490077550 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec90147940 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- 172.21.2.116:0/2490077550 >> 172.21.2.116:0/2490077550 conn(0x7fec9005bd20 msgr2=0x7fec9005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- 172.21.2.116:0/2490077550 shutdown_connections 2021-11-21T22:02:59.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- 172.21.2.116:0/2490077550 wait complete. 2021-11-21T22:02:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 Processor -- start 2021-11-21T22:02:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- start start 2021-11-21T22:02:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec900b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec900b11c0 0x7fec9014a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec90143e60 0x7fec90159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec90065910 con 0x7fec900a8670 2021-11-21T22:02:59.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec901599f0 con 0x7fec900b11c0 2021-11-21T22:02:59.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec90065b70 con 0x7fec90143e60 2021-11-21T22:02:59.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec900b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec900b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53922/0 (socket says 172.21.2.116:53922) 2021-11-21T22:02:59.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 -- 172.21.2.116:0/4085191948 learned_addr learned my addr 172.21.2.116:0/4085191948 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:02:59.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 -- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec90143e60 msgr2=0x7fec90159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec90143e60 0x7fec90159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 -- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec900b11c0 msgr2=0x7fec9014a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec900b11c0 0x7fec9014a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 -- 172.21.2.116:0/4085191948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8400b180 con 0x7fec900a8670 2021-11-21T22:02:59.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec953c9700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec900b5a00 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7fec90147180 tx=0x7fec8c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec827fc700 1 -- 172.21.2.116:0/4085191948 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8c00ce30 con 0x7fec900a8670 2021-11-21T22:02:59.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec827fc700 1 -- 172.21.2.116:0/4085191948 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c0072e0 con 0x7fec900a8670 2021-11-21T22:02:59.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec90065e10 con 0x7fec900a8670 2021-11-21T22:02:59.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec827fc700 1 -- 172.21.2.116:0/4085191948 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8c005800 con 0x7fec900a8670 2021-11-21T22:02:59.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.847+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec900b5280 con 0x7fec900a8670 2021-11-21T22:02:59.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.851+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec900b20b0 con 0x7fec900a8670 2021-11-21T22:02:59.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.851+0000 7fec827fc700 1 -- 172.21.2.116:0/4085191948 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fec8c010070 con 0x7fec900a8670 2021-11-21T22:02:59.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.851+0000 7fec827fc700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec70053610 0x7fec70055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:02:59.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.851+0000 7fec827fc700 1 -- 172.21.2.116:0/4085191948 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fec8c06b060 con 0x7fec900a8670 2021-11-21T22:02:59.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.851+0000 7fec94bc8700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec70053610 0x7fec70055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:02:59.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.851+0000 7fec94bc8700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec70053610 0x7fec70055ad0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fec90068600 tx=0x7fec84012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:02:59.864 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.851+0000 7fec827fc700 1 -- 172.21.2.116:0/4085191948 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec8c042140 con 0x7fec900a8670 2021-11-21T22:02:59.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.963+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fec900573d0 con 0x7fec900a8670 2021-11-21T22:02:59.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec827fc700 1 -- 172.21.2.116:0/4085191948 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7fec8c045760 con 0x7fec900a8670 2021-11-21T22:02:59.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec70053610 msgr2=0x7fec70055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec70053610 0x7fec70055ad0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fec90068600 tx=0x7fec84012000).stop 2021-11-21T22:02:59.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 msgr2=0x7fec900b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:02:59.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec900b5a00 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7fec90147180 tx=0x7fec8c007c40).stop 2021-11-21T22:02:59.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 shutdown_connections 2021-11-21T22:02:59.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fec90143e60 0x7fec90159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fec900b11c0 0x7fec9014a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:02:59.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fec70053610 0x7fec70055ad0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 --2- 172.21.2.116:0/4085191948 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fec900a8670 0x7fec900b5a00 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:02:59.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 >> 172.21.2.116:0/4085191948 conn(0x7fec9005bd20 msgr2=0x7fec9013f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:02:59.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 shutdown_connections 2021-11-21T22:02:59.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:02:59.971+0000 7fec9762d700 1 -- 172.21.2.116:0/4085191948 wait complete. 2021-11-21T22:02:59.978 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:02:59.982 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:00.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 Processor -- start 2021-11-21T22:03:00.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- start start 2021-11-21T22:03:00.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f2d24062410 0x7f2d24144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240a6e80 con 0x7f2d240a6aa0 2021-11-21T22:03:00.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2414a7b0 con 0x7f2d240a86b0 2021-11-21T22:03:00.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d240a8bc0 con 0x7f2d24062410 2021-11-21T22:03:00.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2937a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f2d240a6aa0 0x7f2d241470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37654/0 (socket says 172.21.2.116:37654) 2021-11-21T22:03:00.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2937a700 1 -- 172.21.2.116:0/4167659636 learned_addr learned my addr 172.21.2.116:0/4167659636 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:00.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d29b7b700 1 --2- 172.21.2.116:0/4167659636 >> v2:172.21.2.116:3300/0 conn(0x7f2d24062410 0x7f2d24144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d29b7b700 1 -- 172.21.2.116:0/4167659636 >> v1:172.21.2.117:6789/0 conn(0x7f2d240a86b0 legacy=0x7f2d241443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d29b7b700 1 -- 172.21.2.116:0/4167659636 >> v1:172.21.2.116:6789/0 conn(0x7f2d240a6aa0 legacy=0x7f2d241470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d29b7b700 1 -- 172.21.2.116:0/4167659636 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2414aa10 con 0x7f2d24062410 2021-11-21T22:03:00.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d29b7b700 1 --2- 172.21.2.116:0/4167659636 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d24144b00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f2d18001110 tx=0x7f2d1c0091d0).ready entity=mon.0 client_cookie=8ab5fcea39093377 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d17fff700 1 -- 172.21.2.116:0/4167659636 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d1c009ea0 con 0x7f2d24062410 2021-11-21T22:03:00.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d17fff700 1 -- 172.21.2.116:0/4167659636 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c00b760 con 0x7f2d24062410 2021-11-21T22:03:00.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d17fff700 1 -- 172.21.2.116:0/4167659636 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d1c00b8d0 con 0x7f2d24062410 2021-11-21T22:03:00.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- 172.21.2.116:0/4167659636 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 msgr2=0x7f2d24144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/4167659636 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d24144b00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f2d18001110 tx=0x7f2d1c0091d0).stop 2021-11-21T22:03:00.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- 172.21.2.116:0/4167659636 shutdown_connections 2021-11-21T22:03:00.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/4167659636 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d24144b00 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- 172.21.2.116:0/4167659636 >> 172.21.2.116:0/4167659636 conn(0x7f2d2405bd20 msgr2=0x7f2d2405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- 172.21.2.116:0/4167659636 shutdown_connections 2021-11-21T22:03:00.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.031+0000 7f2d2b5de700 1 -- 172.21.2.116:0/4167659636 wait complete. 2021-11-21T22:03:00.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 Processor -- start 2021-11-21T22:03:00.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 -- start start 2021-11-21T22:03:00.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d240b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d240a6aa0 0x7f2d24159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d240a86b0 0x7f2d2415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240aa8c0 con 0x7f2d24062410 2021-11-21T22:03:00.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24066d30 con 0x7f2d240a6aa0 2021-11-21T22:03:00.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24159990 con 0x7f2d240a86b0 2021-11-21T22:03:00.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d240b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d240b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53936/0 (socket says 172.21.2.116:53936) 2021-11-21T22:03:00.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 -- 172.21.2.116:0/3800568556 learned_addr learned my addr 172.21.2.116:0/3800568556 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:00.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d28b79700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d240a6aa0 0x7f2d24159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d29b7b700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d240a86b0 0x7f2d2415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 -- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d240a86b0 msgr2=0x7f2d2415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d240a86b0 0x7f2d2415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 -- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d240a6aa0 msgr2=0x7f2d24159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d240a6aa0 0x7f2d24159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 -- 172.21.2.116:0/3800568556 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c009470 con 0x7f2d24062410 2021-11-21T22:03:00.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d29b7b700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d240a86b0 0x7f2d2415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:00.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2937a700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d240b3ca0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f2d180011d0 tx=0x7f2d18007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d167fc700 1 -- 172.21.2.116:0/3800568556 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d180072e0 con 0x7f2d24062410 2021-11-21T22:03:00.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d167fc700 1 -- 172.21.2.116:0/3800568556 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18007440 con 0x7f2d24062410 2021-11-21T22:03:00.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d24066430 con 0x7f2d24062410 2021-11-21T22:03:00.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d167fc700 1 -- 172.21.2.116:0/3800568556 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d180057e0 con 0x7f2d24062410 2021-11-21T22:03:00.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d24066f20 con 0x7f2d24062410 2021-11-21T22:03:00.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d24159ad0 con 0x7f2d24062410 2021-11-21T22:03:00.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d167fc700 1 -- 172.21.2.116:0/3800568556 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2d1801a060 con 0x7f2d24062410 2021-11-21T22:03:00.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d167fc700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d04053660 0x7f2d04055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d167fc700 1 -- 172.21.2.116:0/3800568556 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2d1806af60 con 0x7f2d24062410 2021-11-21T22:03:00.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.035+0000 7f2d28b79700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d04053660 0x7f2d04055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.039+0000 7f2d28b79700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d04053660 0x7f2d04055b20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f2d2000b000 tx=0x7f2d20009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.039+0000 7f2d167fc700 1 -- 172.21.2.116:0/3800568556 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d180420c0 con 0x7f2d24062410 2021-11-21T22:03:00.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d2405db70 con 0x7f2d24062410 2021-11-21T22:03:00.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d167fc700 1 -- 172.21.2.116:0/3800568556 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f2d18018020 con 0x7f2d24062410 2021-11-21T22:03:00.151 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:00.152 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":70,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:58.250794+0000","modified":"2021-11-21T22:02:59.255376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/139139464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":139139464},{"type":"v1","addr":"172.21.2.116:6835","nonce":139139464}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-11-21T22:03:00.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d04053660 msgr2=0x7f2d04055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d04053660 0x7f2d04055b20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f2d2000b000 tx=0x7f2d20009040).stop 2021-11-21T22:03:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 msgr2=0x7f2d240b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d240b3ca0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f2d180011d0 tx=0x7f2d18007c00).stop 2021-11-21T22:03:00.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 shutdown_connections 2021-11-21T22:03:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2d240a86b0 0x7f2d2415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2d240a6aa0 0x7f2d24159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2d04053660 0x7f2d04055b20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 --2- 172.21.2.116:0/3800568556 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2d24062410 0x7f2d240b3ca0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 >> 172.21.2.116:0/3800568556 conn(0x7f2d2405bd20 msgr2=0x7f2d24142720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 shutdown_connections 2021-11-21T22:03:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.147+0000 7f2d2b5de700 1 -- 172.21.2.116:0/3800568556 wait complete. 2021-11-21T22:03:00.155 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 70 2021-11-21T22:03:00.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:58.250794+0000', 'modified': '2021-11-21T22:02:59.255376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8472': {'gid': 8472, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/139139464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 139139464}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 139139464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:03:00.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:03:00.462 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:03:00.532 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8472,"want_state":"up:active","state":"up:active","rank_uptime":2.273260762,"mdsmap_epoch":70,"osdmap_epoch":152,"osdmap_epoch_barrier":152,"uptime":6.5296948969999997} 2021-11-21T22:03:00.532 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8472, "mdsmap_epoch": 70, "osdmap_epoch": 152, "osdmap_epoch_barrier": 152, "rank_uptime": 2.273260762, "state": "up:active", "uptime": 6.529694897, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:03:00.534 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:00.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 Processor -- start 2021-11-21T22:03:00.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- start start 2021-11-21T22:03:00.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 0x7f8c980b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8c980b7070 0x7f8c980ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c980b29b0 con 0x7f8c981443a0 2021-11-21T22:03:00.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c98067c30 con 0x7f8c980b7070 2021-11-21T22:03:00.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980b7910 con 0x7f8c98067850 2021-11-21T22:03:00.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9cb69700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8c981443a0 0x7f8c98144830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37666/0 (socket says 172.21.2.116:37666) 2021-11-21T22:03:00.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9cb69700 1 -- 172.21.2.116:0/1128018099 learned_addr learned my addr 172.21.2.116:0/1128018099 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:00.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c977fe700 1 --2- 172.21.2.116:0/1128018099 >> v2:172.21.2.116:3300/0 conn(0x7f8c980b7070 0x7f8c980ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c96ffd700 1 -- 172.21.2.116:0/1128018099 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039281208 0 0) 0x7f8c980b29b0 con 0x7f8c981443a0 2021-11-21T22:03:00.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c96ffd700 1 -- 172.21.2.116:0/1128018099 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c78003260 con 0x7f8c981443a0 2021-11-21T22:03:00.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c977fe700 1 -- 172.21.2.116:0/1128018099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 msgr2=0x7f8c980b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c977fe700 1 --2- 172.21.2.116:0/1128018099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 0x7f8c980b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c977fe700 1 -- 172.21.2.116:0/1128018099 >> v1:172.21.2.116:6789/0 conn(0x7f8c981443a0 legacy=0x7f8c98144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c977fe700 1 -- 172.21.2.116:0/1128018099 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c980b2be0 con 0x7f8c980b7070 2021-11-21T22:03:00.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c977fe700 1 --2- 172.21.2.116:0/1128018099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c980ab010 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7f8c88002f70 tx=0x7f8c88006df0).ready entity=mon.0 client_cookie=2544ff7e14affdf3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c96ffd700 1 -- 172.21.2.116:0/1128018099 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8800d570 con 0x7f8c980b7070 2021-11-21T22:03:00.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c96ffd700 1 -- 172.21.2.116:0/1128018099 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c88000e90 con 0x7f8c980b7070 2021-11-21T22:03:00.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c96ffd700 1 -- 172.21.2.116:0/1128018099 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8800d8c0 con 0x7f8c980b7070 2021-11-21T22:03:00.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/1128018099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 msgr2=0x7f8c980ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/1128018099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c980ab010 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7f8c88002f70 tx=0x7f8c88006df0).stop 2021-11-21T22:03:00.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/1128018099 shutdown_connections 2021-11-21T22:03:00.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/1128018099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c980ab010 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/1128018099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 0x7f8c980b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/1128018099 >> 172.21.2.116:0/1128018099 conn(0x7f8c98064d60 msgr2=0x7f8c98065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/1128018099 shutdown_connections 2021-11-21T22:03:00.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/1128018099 wait complete. 2021-11-21T22:03:00.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 Processor -- start 2021-11-21T22:03:00.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.583+0000 7f8c9e5cc700 1 -- start start 2021-11-21T22:03:00.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 0x7f8c981466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c98150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c981443a0 0x7f8c98153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98151130 con 0x7f8c980b7070 2021-11-21T22:03:00.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c981536b0 con 0x7f8c981443a0 2021-11-21T22:03:00.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980b0c40 con 0x7f8c98067850 2021-11-21T22:03:00.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c98150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c98150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53948/0 (socket says 172.21.2.116:53948) 2021-11-21T22:03:00.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 -- 172.21.2.116:0/4015596699 learned_addr learned my addr 172.21.2.116:0/4015596699 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:00.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 -- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 msgr2=0x7f8c981466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 0x7f8c981466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 -- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c981443a0 msgr2=0x7f8c98153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c981443a0 0x7f8c98153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 -- 172.21.2.116:0/4015596699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8800b180 con 0x7f8c980b7070 2021-11-21T22:03:00.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c977fe700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c98150120 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f8c88002f70 tx=0x7f8c88014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c957fa700 1 -- 172.21.2.116:0/4015596699 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c88016070 con 0x7f8c980b7070 2021-11-21T22:03:00.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c957fa700 1 -- 172.21.2.116:0/4015596699 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8800de40 con 0x7f8c980b7070 2021-11-21T22:03:00.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c957fa700 1 -- 172.21.2.116:0/4015596699 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c88006b20 con 0x7f8c980b7070 2021-11-21T22:03:00.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c980b0ee0 con 0x7f8c980b7070 2021-11-21T22:03:00.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c980b0930 con 0x7f8c980b7070 2021-11-21T22:03:00.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c980573d0 con 0x7f8c980b7070 2021-11-21T22:03:00.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c957fa700 1 -- 172.21.2.116:0/4015596699 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8c8800f030 con 0x7f8c980b7070 2021-11-21T22:03:00.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c957fa700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c7c0536d0 0x7f8c7c055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c957fa700 1 -- 172.21.2.116:0/4015596699 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8c8806c790 con 0x7f8c980b7070 2021-11-21T22:03:00.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c97fff700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c7c0536d0 0x7f8c7c055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.587+0000 7f8c97fff700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c7c0536d0 0x7f8c7c055b90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f8c80002f70 tx=0x7f8c800069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.591+0000 7f8c957fa700 1 -- 172.21.2.116:0/4015596699 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c88043760 con 0x7f8c980b7070 2021-11-21T22:03:00.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.671+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c98068740 con 0x7f8c980b7070 2021-11-21T22:03:00.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.671+0000 7f8c957fa700 1 -- 172.21.2.116:0/4015596699 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f8c88023020 con 0x7f8c980b7070 2021-11-21T22:03:00.676 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:00.677 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":152,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:59.253897+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:56.060658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:57.220549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:00.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c7c0536d0 msgr2=0x7f8c7c055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c7c0536d0 0x7f8c7c055b90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f8c80002f70 tx=0x7f8c800069e0).stop 2021-11-21T22:03:00.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 msgr2=0x7f8c98150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c98150120 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f8c88002f70 tx=0x7f8c88014000).stop 2021-11-21T22:03:00.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 shutdown_connections 2021-11-21T22:03:00.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c981443a0 0x7f8c98153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c7c0536d0 0x7f8c7c055b90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c980b7070 0x7f8c98150120 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 --2- 172.21.2.116:0/4015596699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c98067850 0x7f8c981466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 >> 172.21.2.116:0/4015596699 conn(0x7f8c98064d60 msgr2=0x7f8c98065140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 shutdown_connections 2021-11-21T22:03:00.680 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.675+0000 7f8c9e5cc700 1 -- 172.21.2.116:0/4015596699 wait complete. 2021-11-21T22:03:00.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:02:56.060658+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:02:57.220549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:03:00.686 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:03:00.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 Processor -- start 2021-11-21T22:03:00.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- start start 2021-11-21T22:03:00.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fca00144810 0x7fca001478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca000abce0 0x7fca000b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fca000b1db0 0x7fca000ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca00147e80 con 0x7fca000b1db0 2021-11-21T22:03:00.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca000af5a0 con 0x7fca00144810 2021-11-21T22:03:00.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca000b1490 con 0x7fca000abce0 2021-11-21T22:03:00.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fca000b1db0 0x7fca000ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fca000b1db0 0x7fca000ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53954/0 (socket says 172.21.2.116:53954) 2021-11-21T22:03:00.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 -- 172.21.2.116:0/2887679963 learned_addr learned my addr 172.21.2.116:0/2887679963 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:00.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 -- 172.21.2.116:0/2887679963 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca000abce0 msgr2=0x7fca000b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 --2- 172.21.2.116:0/2887679963 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca000abce0 0x7fca000b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 -- 172.21.2.116:0/2887679963 >> v2:172.21.2.117:3300/0 conn(0x7fca00144810 msgr2=0x7fca001478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 --2- 172.21.2.116:0/2887679963 >> v2:172.21.2.117:3300/0 conn(0x7fca00144810 0x7fca001478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 -- 172.21.2.116:0/2887679963 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca000af060 con 0x7fca000b1db0 2021-11-21T22:03:00.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca0589a700 1 --2- 172.21.2.116:0/2887679963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca000b1db0 0x7fca000ac600 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7fc9f80032f0 tx=0x7fc9f8009290).ready entity=mon.0 client_cookie=fd88b3fdbda2d79f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fc9f3fff700 1 -- 172.21.2.116:0/2887679963 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f800b870 con 0x7fca000b1db0 2021-11-21T22:03:00.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fc9f3fff700 1 -- 172.21.2.116:0/2887679963 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f800b9d0 con 0x7fca000b1db0 2021-11-21T22:03:00.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fc9f3fff700 1 -- 172.21.2.116:0/2887679963 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f800bb40 con 0x7fca000b1db0 2021-11-21T22:03:00.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- 172.21.2.116:0/2887679963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca000b1db0 msgr2=0x7fca000ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 --2- 172.21.2.116:0/2887679963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca000b1db0 0x7fca000ac600 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7fc9f80032f0 tx=0x7fc9f8009290).stop 2021-11-21T22:03:00.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- 172.21.2.116:0/2887679963 shutdown_connections 2021-11-21T22:03:00.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 --2- 172.21.2.116:0/2887679963 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca000b1db0 0x7fca000ac600 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 --2- 172.21.2.116:0/2887679963 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca000abce0 0x7fca000b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 --2- 172.21.2.116:0/2887679963 >> v2:172.21.2.117:3300/0 conn(0x7fca00144810 0x7fca001478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- 172.21.2.116:0/2887679963 >> 172.21.2.116:0/2887679963 conn(0x7fca0005bd20 msgr2=0x7fca0005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- 172.21.2.116:0/2887679963 shutdown_connections 2021-11-21T22:03:00.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.735+0000 7fca072fd700 1 -- 172.21.2.116:0/2887679963 wait complete. 2021-11-21T22:03:00.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 Processor -- start 2021-11-21T22:03:00.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 -- start start 2021-11-21T22:03:00.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca000abce0 0x7fca0014a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca00144810 0x7fca00144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca001467f0 0x7fca00145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca00145700 con 0x7fca00144810 2021-11-21T22:03:00.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca000b2f20 con 0x7fca000abce0 2021-11-21T22:03:00.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca00063910 con 0x7fca001467f0 2021-11-21T22:03:00.751 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca00144810 0x7fca00144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca00144810 0x7fca00144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53960/0 (socket says 172.21.2.116:53960) 2021-11-21T22:03:00.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 -- 172.21.2.116:0/3375716872 learned_addr learned my addr 172.21.2.116:0/3375716872 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:00.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 -- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca001467f0 msgr2=0x7fca00145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca001467f0 0x7fca00145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.752 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 -- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca000abce0 msgr2=0x7fca0014a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca000abce0 0x7fca0014a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 -- 172.21.2.116:0/3375716872 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8009530 con 0x7fca00144810 2021-11-21T22:03:00.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca04898700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca00144810 0x7fca00144d20 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7fc9f4002f70 tx=0x7fc9f4006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.753 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fc9f27fc700 1 -- 172.21.2.116:0/3375716872 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f4013460 con 0x7fca00144810 2021-11-21T22:03:00.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fc9f27fc700 1 -- 172.21.2.116:0/3375716872 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f40135c0 con 0x7fca00144810 2021-11-21T22:03:00.754 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca000b3150 con 0x7fca00144810 2021-11-21T22:03:00.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca00063e10 con 0x7fca00144810 2021-11-21T22:03:00.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fc9f27fc700 1 -- 172.21.2.116:0/3375716872 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f4013920 con 0x7fca00144810 2021-11-21T22:03:00.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca000af300 con 0x7fca00144810 2021-11-21T22:03:00.755 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fc9f27fc700 1 -- 172.21.2.116:0/3375716872 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc9f401a060 con 0x7fca00144810 2021-11-21T22:03:00.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fc9f27fc700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9e0053660 0x7fc9e0055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fc9f27fc700 1 -- 172.21.2.116:0/3375716872 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc9f406b4e0 con 0x7fca00144810 2021-11-21T22:03:00.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.739+0000 7fca05099700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9e0053660 0x7fc9e0055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.743+0000 7fca05099700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9e0053660 0x7fc9e0055b20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fc9fc002f70 tx=0x7fc9fc006970).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.743+0000 7fc9f27fc700 1 -- 172.21.2.116:0/3375716872 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9f40424d0 con 0x7fca00144810 2021-11-21T22:03:00.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.823+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fca000573d0 con 0x7fca00144810 2021-11-21T22:03:00.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.823+0000 7fc9f27fc700 1 -- 172.21.2.116:0/3375716872 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v152) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc9f4045af0 con 0x7fca00144810 2021-11-21T22:03:00.828 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:03:00.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.823+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9e0053660 msgr2=0x7fc9e0055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.829 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.823+0000 7fca072fd700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9e0053660 0x7fc9e0055b20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fc9fc002f70 tx=0x7fc9fc006970).stop 2021-11-21T22:03:00.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.823+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca00144810 msgr2=0x7fca00144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.823+0000 7fca072fd700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca00144810 0x7fca00144d20 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7fc9f4002f70 tx=0x7fc9f4006e90).stop 2021-11-21T22:03:00.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 shutdown_connections 2021-11-21T22:03:00.830 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca001467f0 0x7fca00145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc9e0053660 0x7fc9e0055b20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca00144810 0x7fca00144d20 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 --2- 172.21.2.116:0/3375716872 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca000abce0 0x7fca0014a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 >> 172.21.2.116:0/3375716872 conn(0x7fca0005bd20 msgr2=0x7fca001476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.831 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 shutdown_connections 2021-11-21T22:03:00.832 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.827+0000 7fca072fd700 1 -- 172.21.2.116:0/3375716872 wait complete. 2021-11-21T22:03:00.835 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:03:00.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed3b431700 1 Processor -- start 2021-11-21T22:03:00.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed3b431700 1 -- start start 2021-11-21T22:03:00.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed3b431700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed340adc10 0x7fed340b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed3b431700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fed34144a10 0x7fed34141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed3b431700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed340b3c50 con 0x7fed34141510 2021-11-21T22:03:00.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed3b431700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed340b3ef0 con 0x7fed34144a10 2021-11-21T22:03:00.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed3b431700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed340b4190 con 0x7fed340adc10 2021-11-21T22:03:00.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed399ce700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fed34141510 0x7fed3413a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37692/0 (socket says 172.21.2.116:37692) 2021-11-21T22:03:00.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed399ce700 1 -- 172.21.2.116:0/1680925974 learned_addr learned my addr 172.21.2.116:0/1680925974 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:00.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058900001 0 0) 0x7fed340b3c50 con 0x7fed34141510 2021-11-21T22:03:00.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed1c003260 con 0x7fed34141510 2021-11-21T22:03:00.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467199517 0 0) 0x7fed1c003260 con 0x7fed34141510 2021-11-21T22:03:00.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed340adc10 msgr2=0x7fed340b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed340adc10 0x7fed340b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 >> v2:172.21.2.117:3300/0 conn(0x7fed34144a10 msgr2=0x7fed34141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 --2- 172.21.2.116:0/1680925974 >> v2:172.21.2.117:3300/0 conn(0x7fed34144a10 0x7fed34141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34140b80 con 0x7fed34141510 2021-11-21T22:03:00.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fed2c0019e0 con 0x7fed34141510 2021-11-21T22:03:00.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 >> v1:172.21.2.116:6789/0 conn(0x7fed34141510 legacy=0x7fed3413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed340adc10 0x7fed1c0037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed1c003ee0 0x7fed1c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed1c00fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed340b3ef0 con 0x7fed1c004bf0 2021-11-21T22:03:00.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed340b4190 con 0x7fed1c003ee0 2021-11-21T22:03:00.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.883+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c0053a0 con 0x7fed340adc10 2021-11-21T22:03:00.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed1c00fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 -- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed340adc10 msgr2=0x7fed1c0037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed340adc10 0x7fed1c0037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 -- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed1c003ee0 msgr2=0x7fed1c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:00.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed1c003ee0 0x7fed1c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:00.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 -- 172.21.2.116:0/1680925974 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c003260 con 0x7fed1c004bf0 2021-11-21T22:03:00.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed1c00fb60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7fed2c0016f0 tx=0x7fed3413e3d0).ready entity=mon.0 client_cookie=5473357c5dad88b6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2c00cc50 con 0x7fed1c004bf0 2021-11-21T22:03:00.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c00cdb0 con 0x7fed1c004bf0 2021-11-21T22:03:00.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed27fff700 1 -- 172.21.2.116:0/1680925974 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2c005b80 con 0x7fed1c004bf0 2021-11-21T22:03:00.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/1680925974 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 msgr2=0x7fed1c00fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 --2- 172.21.2.116:0/1680925974 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed1c00fb60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7fed2c0016f0 tx=0x7fed3413e3d0).stop 2021-11-21T22:03:00.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 -- 172.21.2.116:0/1680925974 reap_dead start 2021-11-21T22:03:00.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/1680925974 shutdown_connections 2021-11-21T22:03:00.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/1680925974 >> 172.21.2.116:0/1680925974 conn(0x7fed3405bd20 msgr2=0x7fed3405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/1680925974 shutdown_connections 2021-11-21T22:03:00.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/1680925974 wait complete. 2021-11-21T22:03:00.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 Processor -- start 2021-11-21T22:03:00.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- start start 2021-11-21T22:03:00.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed340ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed340af040 0x7fed3413c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed3413ceb0 0x7fed34155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed340b11f0 con 0x7fed1c004bf0 2021-11-21T22:03:00.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed34153cc0 con 0x7fed340af040 2021-11-21T22:03:00.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed340b63f0 con 0x7fed3413ceb0 2021-11-21T22:03:00.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed340ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed340ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53976/0 (socket says 172.21.2.116:53976) 2021-11-21T22:03:00.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 -- 172.21.2.116:0/2941670665 learned_addr learned my addr 172.21.2.116:0/2941670665 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:00.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed3413ceb0 0x7fed34155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 -- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed3413ceb0 msgr2=0x7fed34155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed3413ceb0 0x7fed34155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 -- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed340af040 msgr2=0x7fed3413c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed389cc700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed340af040 0x7fed3413c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed340af040 0x7fed3413c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 -- 172.21.2.116:0/2941670665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3415a330 con 0x7fed1c004bf0 2021-11-21T22:03:00.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed399ce700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed3413ceb0 0x7fed34155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:00.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed389cc700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed340af040 0x7fed3413c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:00.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed391cd700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed340ae760 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7fed340b6c40 tx=0x7fed30006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed267fc700 1 -- 172.21.2.116:0/2941670665 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed3000f510 con 0x7fed1c004bf0 2021-11-21T22:03:00.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed267fc700 1 -- 172.21.2.116:0/2941670665 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed30000e90 con 0x7fed1c004bf0 2021-11-21T22:03:00.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed267fc700 1 -- 172.21.2.116:0/2941670665 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed3000f6b0 con 0x7fed1c004bf0 2021-11-21T22:03:00.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed340b6620 con 0x7fed1c004bf0 2021-11-21T22:03:00.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3413d330 con 0x7fed1c004bf0 2021-11-21T22:03:00.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.887+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3415a4c0 con 0x7fed1c004bf0 2021-11-21T22:03:00.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.891+0000 7fed267fc700 1 -- 172.21.2.116:0/2941670665 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fed30017020 con 0x7fed1c004bf0 2021-11-21T22:03:00.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.891+0000 7fed267fc700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fed140536d0 0x7fed14055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:00.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.891+0000 7fed267fc700 1 -- 172.21.2.116:0/2941670665 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fed3006b000 con 0x7fed1c004bf0 2021-11-21T22:03:00.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.891+0000 7fed389cc700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fed140536d0 0x7fed14055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:00.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.891+0000 7fed389cc700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fed140536d0 0x7fed14055b90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fed28007000 tx=0x7fed28008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:00.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.891+0000 7fed267fc700 1 -- 172.21.2.116:0/2941670665 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed3415a4c0 con 0x7fed1c004bf0 2021-11-21T22:03:00.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fed3415a4c0 con 0x7fed1c004bf0 2021-11-21T22:03:00.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed267fc700 1 -- 172.21.2.116:0/2941670665 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v152) v1 ==== 92+0+10 (secure 0 0 0) 0x7fed3415a4c0 con 0x7fed1c004bf0 2021-11-21T22:03:00.980 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:00.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fed140536d0 msgr2=0x7fed14055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fed140536d0 0x7fed14055b90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fed28007000 tx=0x7fed28008040).stop 2021-11-21T22:03:00.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 msgr2=0x7fed340ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:00.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed340ae760 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7fed340b6c40 tx=0x7fed30006e70).stop 2021-11-21T22:03:00.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 shutdown_connections 2021-11-21T22:03:00.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fed3413ceb0 0x7fed34155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fed340af040 0x7fed3413c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fed140536d0 0x7fed14055b90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 --2- 172.21.2.116:0/2941670665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fed1c004bf0 0x7fed340ae760 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:00.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 >> 172.21.2.116:0/2941670665 conn(0x7fed3405bd20 msgr2=0x7fed340b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:00.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 shutdown_connections 2021-11-21T22:03:00.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:00.975+0000 7fed3b431700 1 -- 172.21.2.116:0/2941670665 wait complete. 2021-11-21T22:03:00.987 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:03:01.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.035+0000 7fb6d3da1700 1 Processor -- start 2021-11-21T22:03:01.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.035+0000 7fb6d3da1700 1 -- start start 2021-11-21T22:03:01.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6cc144810 0x7fb6cc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6cc147e80 con 0x7fb6cc0ac600 2021-11-21T22:03:01.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6cc0af5a0 con 0x7fb6cc0abce0 2021-11-21T22:03:01.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6cc0b1490 con 0x7fb6cc144810 2021-11-21T22:03:01.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d1b3d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb6cc144810 0x7fb6cc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d233e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb6cc0ac600 0x7fb6cc0aca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37710/0 (socket says 172.21.2.116:37710) 2021-11-21T22:03:01.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d233e700 1 -- 172.21.2.116:0/3430684421 learned_addr learned my addr 172.21.2.116:0/3430684421 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:01.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d1b3d700 1 -- 172.21.2.116:0/3430684421 >> v1:172.21.2.117:6789/0 conn(0x7fb6cc0abce0 legacy=0x7fb6cc0b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:01.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d1b3d700 1 -- 172.21.2.116:0/3430684421 >> v1:172.21.2.116:6789/0 conn(0x7fb6cc0ac600 legacy=0x7fb6cc0aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d0b3b700 1 -- 172.21.2.116:0/3430684421 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251125741 0 0) 0x7fb6cc147e80 con 0x7fb6cc0ac600 2021-11-21T22:03:01.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d1b3d700 1 -- 172.21.2.116:0/3430684421 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc0b1fd0 con 0x7fb6cc144810 2021-11-21T22:03:01.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d1b3d700 1 --2- 172.21.2.116:0/3430684421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc144810 0x7fb6cc1478c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fb6cc05d170 tx=0x7fb6c0006df0).ready entity=mon.0 client_cookie=66454b7305a55964 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d0b3b700 1 -- 172.21.2.116:0/3430684421 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c000d5c0 con 0x7fb6cc144810 2021-11-21T22:03:01.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d0b3b700 1 -- 172.21.2.116:0/3430684421 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0000e90 con 0x7fb6cc144810 2021-11-21T22:03:01.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d0b3b700 1 -- 172.21.2.116:0/3430684421 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c000d910 con 0x7fb6cc144810 2021-11-21T22:03:01.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3430684421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc144810 msgr2=0x7fb6cc1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3430684421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc144810 0x7fb6cc1478c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fb6cc05d170 tx=0x7fb6c0006df0).stop 2021-11-21T22:03:01.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3430684421 shutdown_connections 2021-11-21T22:03:01.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3430684421 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc144810 0x7fb6cc1478c0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3430684421 >> 172.21.2.116:0/3430684421 conn(0x7fb6cc05bd20 msgr2=0x7fb6cc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:01.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3430684421 shutdown_connections 2021-11-21T22:03:01.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3430684421 wait complete. 2021-11-21T22:03:01.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 Processor -- start 2021-11-21T22:03:01.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- start start 2021-11-21T22:03:01.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6cc0abce0 0x7fb6cc0acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc0ac600 0x7fb6cc0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6cc144810 0x7fb6cc159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc156700 con 0x7fb6cc0ac600 2021-11-21T22:03:01.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc1569a0 con 0x7fb6cc0abce0 2021-11-21T22:03:01.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc156c40 con 0x7fb6cc144810 2021-11-21T22:03:01.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc0ac600 0x7fb6cc0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc0ac600 0x7fb6cc0b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:53990/0 (socket says 172.21.2.116:53990) 2021-11-21T22:03:01.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 -- 172.21.2.116:0/3748548855 learned_addr learned my addr 172.21.2.116:0/3748548855 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:01.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 -- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6cc144810 msgr2=0x7fb6cc159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6cc144810 0x7fb6cc159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 -- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6cc0abce0 msgr2=0x7fb6cc0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6cc0abce0 0x7fb6cc0acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 -- 172.21.2.116:0/3748548855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c000b180 con 0x7fb6cc0ac600 2021-11-21T22:03:01.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d133c700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc0ac600 0x7fb6cc0b2ab0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7fb6c8002fb0 tx=0x7fb6c8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6beffd700 1 -- 172.21.2.116:0/3748548855 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c8013460 con 0x7fb6cc0ac600 2021-11-21T22:03:01.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6beffd700 1 -- 172.21.2.116:0/3748548855 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c80135c0 con 0x7fb6cc0ac600 2021-11-21T22:03:01.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc156ee0 con 0x7fb6cc0ac600 2021-11-21T22:03:01.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6beffd700 1 -- 172.21.2.116:0/3748548855 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c8013920 con 0x7fb6cc0ac600 2021-11-21T22:03:01.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.039+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc157380 con 0x7fb6cc0ac600 2021-11-21T22:03:01.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.043+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6cc157070 con 0x7fb6cc0ac600 2021-11-21T22:03:01.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.043+0000 7fb6beffd700 1 -- 172.21.2.116:0/3748548855 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb6c801b030 con 0x7fb6cc0ac600 2021-11-21T22:03:01.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.043+0000 7fb6beffd700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6b4053640 0x7fb6b4055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.043+0000 7fb6beffd700 1 -- 172.21.2.116:0/3748548855 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb6c806b6c0 con 0x7fb6cc0ac600 2021-11-21T22:03:01.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.043+0000 7fb6d1b3d700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6b4053640 0x7fb6b4055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.043+0000 7fb6d1b3d700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6b4053640 0x7fb6b4055b00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fb6cc05d170 tx=0x7fb6c0012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.043+0000 7fb6beffd700 1 -- 172.21.2.116:0/3748548855 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6c8042820 con 0x7fb6cc0ac600 2021-11-21T22:03:01.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6cc0573d0 con 0x7fb6cc0ac600 2021-11-21T22:03:01.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6beffd700 1 -- 172.21.2.116:0/3748548855 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v152) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb6c8045e40 con 0x7fb6cc0ac600 2021-11-21T22:03:01.132 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:01.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6b4053640 msgr2=0x7fb6b4055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.134 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6b4053640 0x7fb6b4055b00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fb6cc05d170 tx=0x7fb6c0012000).stop 2021-11-21T22:03:01.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc0ac600 msgr2=0x7fb6cc0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc0ac600 0x7fb6cc0b2ab0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7fb6c8002fb0 tx=0x7fb6c8006df0).stop 2021-11-21T22:03:01.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 shutdown_connections 2021-11-21T22:03:01.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb6cc144810 0x7fb6cc159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb6b4053640 0x7fb6b4055b00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb6cc0ac600 0x7fb6cc0b2ab0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 --2- 172.21.2.116:0/3748548855 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb6cc0abce0 0x7fb6cc0acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 >> 172.21.2.116:0/3748548855 conn(0x7fb6cc05bd20 msgr2=0x7fb6cc140aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:01.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 shutdown_connections 2021-11-21T22:03:01.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.127+0000 7fb6d3da1700 1 -- 172.21.2.116:0/3748548855 wait complete. 2021-11-21T22:03:01.140 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:03:01.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 Processor -- start 2021-11-21T22:03:01.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- start start 2021-11-21T22:03:01.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0081426e0 0x7fe008142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe0080ad470 0x7fe0080ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080afae0 con 0x7fe008062e60 2021-11-21T22:03:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0080b0f60 con 0x7fe0080ad470 2021-11-21T22:03:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe008063470 con 0x7fe0081426e0 2021-11-21T22:03:01.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe0080ad470 0x7fe0080ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe0080ad470 0x7fe0080ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54000/0 (socket says 172.21.2.116:54000) 2021-11-21T22:03:01.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 -- 172.21.2.116:0/2595067545 learned_addr learned my addr 172.21.2.116:0/2595067545 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:01.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 -- 172.21.2.116:0/2595067545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0081426e0 msgr2=0x7fe008142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:01.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 --2- 172.21.2.116:0/2595067545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0081426e0 0x7fe008142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 -- 172.21.2.116:0/2595067545 >> v1:172.21.2.117:6789/0 conn(0x7fe008062e60 legacy=0x7fe0080af3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:01.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 -- 172.21.2.116:0/2595067545 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008143050 con 0x7fe0080ad470 2021-11-21T22:03:01.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00e79d700 1 --2- 172.21.2.116:0/2595067545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0080ad470 0x7fe0080ad920 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fe0081432f0 tx=0x7fe00000b530).ready entity=mon.0 client_cookie=797b0e0d15c4bd5 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00cf9a700 1 -- 172.21.2.116:0/2595067545 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0000155e0 con 0x7fe0080ad470 2021-11-21T22:03:01.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00cf9a700 1 -- 172.21.2.116:0/2595067545 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000015740 con 0x7fe0080ad470 2021-11-21T22:03:01.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00cf9a700 1 -- 172.21.2.116:0/2595067545 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe000015ae0 con 0x7fe0080ad470 2021-11-21T22:03:01.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- 172.21.2.116:0/2595067545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0080ad470 msgr2=0x7fe0080ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- 172.21.2.116:0/2595067545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0080ad470 0x7fe0080ad920 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fe0081432f0 tx=0x7fe00000b530).stop 2021-11-21T22:03:01.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- 172.21.2.116:0/2595067545 shutdown_connections 2021-11-21T22:03:01.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- 172.21.2.116:0/2595067545 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0080ad470 0x7fe0080ad920 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- 172.21.2.116:0/2595067545 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0081426e0 0x7fe008142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- 172.21.2.116:0/2595067545 >> 172.21.2.116:0/2595067545 conn(0x7fe0080b5730 msgr2=0x7fe0080b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:01.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- 172.21.2.116:0/2595067545 shutdown_connections 2021-11-21T22:03:01.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- 172.21.2.116:0/2595067545 wait complete. 2021-11-21T22:03:01.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 Processor -- start 2021-11-21T22:03:01.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- start start 2021-11-21T22:03:01.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0080ad470 0x7fe0080669f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0081426e0 0x7fe008066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe008159a90 0x7fe008159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008156fe0 con 0x7fe0081426e0 2021-11-21T22:03:01.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008157410 con 0x7fe008159a90 2021-11-21T22:03:01.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe010200700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00813bbd0 con 0x7fe0080ad470 2021-11-21T22:03:01.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00d79b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0081426e0 0x7fe008066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00d79b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0081426e0 0x7fe008066ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54008/0 (socket says 172.21.2.116:54008) 2021-11-21T22:03:01.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.191+0000 7fe00d79b700 1 -- 172.21.2.116:0/2557186862 learned_addr learned my addr 172.21.2.116:0/2557186862 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:01.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00d79b700 1 -- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0080ad470 msgr2=0x7fe0080669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00d79b700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0080ad470 0x7fe0080669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00d79b700 1 -- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe008159a90 msgr2=0x7fe008159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00d79b700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe008159a90 0x7fe008159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00d79b700 1 -- 172.21.2.116:0/2557186862 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00000b7d0 con 0x7fe0081426e0 2021-11-21T22:03:01.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00d79b700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0081426e0 0x7fe008066ee0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7fdff8002fb0 tx=0x7fdff8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fdff6ffd700 1 -- 172.21.2.116:0/2557186862 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdff8013460 con 0x7fe0081426e0 2021-11-21T22:03:01.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fdff6ffd700 1 -- 172.21.2.116:0/2557186862 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff80135c0 con 0x7fe0081426e0 2021-11-21T22:03:01.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fdff6ffd700 1 -- 172.21.2.116:0/2557186862 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdff8013920 con 0x7fe0081426e0 2021-11-21T22:03:01.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00813f860 con 0x7fe0081426e0 2021-11-21T22:03:01.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00813f550 con 0x7fe0081426e0 2021-11-21T22:03:01.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008067570 con 0x7fe0081426e0 2021-11-21T22:03:01.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fdff6ffd700 1 -- 172.21.2.116:0/2557186862 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdff800f070 con 0x7fe0081426e0 2021-11-21T22:03:01.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fdff6ffd700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdfe8053660 0x7fdfe8055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fdff6ffd700 1 -- 172.21.2.116:0/2557186862 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdff806b4e0 con 0x7fe0081426e0 2021-11-21T22:03:01.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00df9c700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdfe8053660 0x7fdfe8055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.195+0000 7fe00df9c700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdfe8053660 0x7fdfe8055b20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fe004000ed0 tx=0x7fe004006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.199+0000 7fdff6ffd700 1 -- 172.21.2.116:0/2557186862 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdff8042570 con 0x7fe0081426e0 2021-11-21T22:03:01.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.279+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe008157120 con 0x7fe0081426e0 2021-11-21T22:03:01.284 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.279+0000 7fdff6ffd700 1 -- 172.21.2.116:0/2557186862 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v152) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdff8045b90 con 0x7fe0081426e0 2021-11-21T22:03:01.285 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:01.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdfe8053660 msgr2=0x7fdfe8055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdfe8053660 0x7fdfe8055b20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fe004000ed0 tx=0x7fe004006930).stop 2021-11-21T22:03:01.286 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0081426e0 msgr2=0x7fe008066ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0081426e0 0x7fe008066ee0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7fdff8002fb0 tx=0x7fdff8006df0).stop 2021-11-21T22:03:01.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 shutdown_connections 2021-11-21T22:03:01.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe008159a90 0x7fe008159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.287 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdfe8053660 0x7fdfe8055b20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe0081426e0 0x7fe008066ee0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 --2- 172.21.2.116:0/2557186862 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe0080ad470 0x7fe0080669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 >> 172.21.2.116:0/2557186862 conn(0x7fe0080b5730 msgr2=0x7fe0080b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:01.288 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 shutdown_connections 2021-11-21T22:03:01.289 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.283+0000 7fe010200700 1 -- 172.21.2.116:0/2557186862 wait complete. 2021-11-21T22:03:01.292 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:03:01.292 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:01.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c2737b700 1 Processor -- start 2021-11-21T22:03:01.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c2737b700 1 -- start start 2021-11-21T22:03:01.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c2737b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8c200a6aa0 0x7f8c2014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c2737b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 0x7f8c201443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c2737b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c200a6e80 con 0x7f8c200a86b0 2021-11-21T22:03:01.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c2737b700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c200a8bc0 con 0x7f8c200a6aa0 2021-11-21T22:03:01.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c2737b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2014a7b0 con 0x7f8c20062410 2021-11-21T22:03:01.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c24916700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8c200a86b0 0x7f8c201470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37740/0 (socket says 172.21.2.116:37740) 2021-11-21T22:03:01.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c24916700 1 -- 172.21.2.116:0/95484242 learned_addr learned my addr 172.21.2.116:0/95484242 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:01.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699182946 0 0) 0x7f8c200a6e80 con 0x7f8c200a86b0 2021-11-21T22:03:01.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.339+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c08003260 con 0x7f8c200a86b0 2021-11-21T22:03:01.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25918700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 0x7f8c201443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 --2- 172.21.2.116:0/95484242 >> v2:172.21.2.117:3300/0 conn(0x7f8c200a6aa0 0x7f8c2014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286440154 0 0) 0x7f8c08003260 con 0x7f8c200a86b0 2021-11-21T22:03:01.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 msgr2=0x7f8c201443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:01.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 0x7f8c201443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 >> v2:172.21.2.117:3300/0 conn(0x7f8c200a6aa0 msgr2=0x7f8c2014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:01.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 --2- 172.21.2.116:0/95484242 >> v2:172.21.2.117:3300/0 conn(0x7f8c200a6aa0 0x7f8c2014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2014aa10 con 0x7f8c200a86b0 2021-11-21T22:03:01.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8c1c001680 con 0x7f8c200a86b0 2021-11-21T22:03:01.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 >> v1:172.21.2.116:6789/0 conn(0x7f8c200a86b0 legacy=0x7f8c201470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c080037d0 0x7f8c08003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c08004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 0x7f8c08004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200a8bc0 con 0x7f8c08004500 2021-11-21T22:03:01.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2014a7b0 con 0x7f8c080037d0 2021-11-21T22:03:01.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0800b900 con 0x7f8c20062410 2021-11-21T22:03:01.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c24916700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c08004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25918700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 0x7f8c08004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c080037d0 0x7f8c08003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c24916700 1 -- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 msgr2=0x7f8c08004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:01.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c24916700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20062410 0x7f8c08004ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c24916700 1 -- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c080037d0 msgr2=0x7f8c08003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:01.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c24916700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c080037d0 0x7f8c08003bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c24916700 1 -- 172.21.2.116:0/95484242 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08003260 con 0x7f8c08004500 2021-11-21T22:03:01.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c24916700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c08004980 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f8c1c001390 tx=0x7f8c1c001da0).ready entity=mon.0 client_cookie=6e0183de92ecb2e6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1c007170 con 0x7f8c08004500 2021-11-21T22:03:01.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c0072d0 con 0x7f8c08004500 2021-11-21T22:03:01.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c13fff700 1 -- 172.21.2.116:0/95484242 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1c00c8f0 con 0x7f8c08004500 2021-11-21T22:03:01.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/95484242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 msgr2=0x7f8c08004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 --2- 172.21.2.116:0/95484242 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c08004980 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7f8c1c001390 tx=0x7f8c1c001da0).stop 2021-11-21T22:03:01.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25918700 1 -- 172.21.2.116:0/95484242 reap_dead start 2021-11-21T22:03:01.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/95484242 shutdown_connections 2021-11-21T22:03:01.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/95484242 >> 172.21.2.116:0/95484242 conn(0x7f8c2005bd20 msgr2=0x7f8c2005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:01.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/95484242 shutdown_connections 2021-11-21T22:03:01.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/95484242 wait complete. 2021-11-21T22:03:01.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 Processor -- start 2021-11-21T22:03:01.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- start start 2021-11-21T22:03:01.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c2014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.366 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c20066570 0x7f8c20065a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20065f40 0x7f8c20142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20064d30 con 0x7f8c08004500 2021-11-21T22:03:01.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200a9c70 con 0x7f8c20066570 2021-11-21T22:03:01.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20064010 con 0x7f8c20065f40 2021-11-21T22:03:01.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c2014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c2014b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54024/0 (socket says 172.21.2.116:54024) 2021-11-21T22:03:01.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 -- 172.21.2.116:0/1375486094 learned_addr learned my addr 172.21.2.116:0/1375486094 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:01.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25918700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20065f40 0x7f8c20142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 -- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20065f40 msgr2=0x7f8c20142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20065f40 0x7f8c20142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 -- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c20066570 msgr2=0x7f8c20065a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c20066570 0x7f8c20065a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 -- 172.21.2.116:0/1375486094 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200642b0 con 0x7f8c08004500 2021-11-21T22:03:01.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25918700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20065f40 0x7f8c20142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:01.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c25117700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c2014b9f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7f8c14008000 tx=0x7f8c14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c127fc700 1 -- 172.21.2.116:0/1375486094 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1400c070 con 0x7f8c08004500 2021-11-21T22:03:01.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c127fc700 1 -- 172.21.2.116:0/1375486094 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c14000e90 con 0x7f8c08004500 2021-11-21T22:03:01.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c20064550 con 0x7f8c08004500 2021-11-21T22:03:01.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c127fc700 1 -- 172.21.2.116:0/1375486094 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c140135e0 con 0x7f8c08004500 2021-11-21T22:03:01.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c20064f30 con 0x7f8c08004500 2021-11-21T22:03:01.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c20066350 con 0x7f8c08004500 2021-11-21T22:03:01.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.343+0000 7f8c127fc700 1 -- 172.21.2.116:0/1375486094 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8c14019020 con 0x7f8c08004500 2021-11-21T22:03:01.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.347+0000 7f8c127fc700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c000535b0 0x7f8c00055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:01.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.347+0000 7f8c127fc700 1 -- 172.21.2.116:0/1375486094 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8c1406aec0 con 0x7f8c08004500 2021-11-21T22:03:01.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.347+0000 7f8c24916700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c000535b0 0x7f8c00055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:01.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.347+0000 7f8c24916700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c000535b0 0x7f8c00055a70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f8c1c001390 tx=0x7f8c1c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:01.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.347+0000 7f8c127fc700 1 -- 172.21.2.116:0/1375486094 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c14041fc0 con 0x7f8c08004500 2021-11-21T22:03:01.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c200657f0 con 0x7f8c08004500 2021-11-21T22:03:01.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c127fc700 1 -- 172.21.2.116:0/1375486094 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f8c140455e0 con 0x7f8c08004500 2021-11-21T22:03:01.460 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:01.460 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":70,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:58.250794+0000","modified":"2021-11-21T22:02:59.255376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/139139464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":139139464},{"type":"v1","addr":"172.21.2.116:6835","nonce":139139464}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-11-21T22:03:01.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c000535b0 msgr2=0x7f8c00055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c000535b0 0x7f8c00055a70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f8c1c001390 tx=0x7f8c1c015000).stop 2021-11-21T22:03:01.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 msgr2=0x7f8c2014b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:01.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c2014b9f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7f8c14008000 tx=0x7f8c14008040).stop 2021-11-21T22:03:01.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 shutdown_connections 2021-11-21T22:03:01.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8c20065f40 0x7f8c20142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8c20066570 0x7f8c20065a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:01.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8c000535b0 0x7f8c00055a70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 --2- 172.21.2.116:0/1375486094 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8c08004500 0x7f8c2014b9f0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:01.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 >> 172.21.2.116:0/1375486094 conn(0x7f8c2005bd20 msgr2=0x7f8c200b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:01.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 shutdown_connections 2021-11-21T22:03:01.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:01.455+0000 7f8c2737b700 1 -- 172.21.2.116:0/1375486094 wait complete. 2021-11-21T22:03:01.465 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 70 2021-11-21T22:03:01.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:58.250794+0000', 'modified': '2021-11-21T22:02:59.255376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8472': {'gid': 8472, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/139139464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 139139464}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 139139464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:03:01.770 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:03:01.771 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:03:01.836 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8472,"want_state":"up:active","state":"up:active","rank_uptime":3.577448231,"mdsmap_epoch":70,"osdmap_epoch":152,"osdmap_epoch_barrier":152,"uptime":7.8338820309999999} 2021-11-21T22:03:01.837 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8472, "mdsmap_epoch": 70, "osdmap_epoch": 152, "osdmap_epoch_barrier": 152, "rank_uptime": 3.577448231, "state": "up:active", "uptime": 7.833882031, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:03:01.838 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:03:01.838 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:03:01.839 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:03:01.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:01.841 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:01.853 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:03:01.864 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:01.865 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:01.912 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:01.913 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:01.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:01.915 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:03:01.920 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:03:01.920 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:01.964 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:03:01.975 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:01.976 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:01.981 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:01.982 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:01.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:01.983 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:03:02.006 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:03:02.006+0000 7f85fa9f7080 -1 init, newargv = 0x56427a648a90 newargc=9ceph-fuse[ 2021-11-21T22:03:02.006 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:15278]: starting ceph client 2021-11-21T22:03:02.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15278]: starting fuse 2021-11-21T22:03:03.110 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:03:03.124 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:03.125 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:03.173 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:03.174 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:03.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:03.176 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:03:03.182 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:03:03.182 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:03:03.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:03.183 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:03:03.184 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:03:03.184 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:03:03.184 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:03:03.185 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:03:03.185 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:03.186 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:03:03.187 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:03:03.188 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:03:03.188 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:03.189 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:03:03.190 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:03:03.191 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:03:03.191 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:03:03.192 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:03.193 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:03:03.193 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:03:03.194 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:03:03.195 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:03:03.195 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:03:03.196 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:03.197 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:03:03.198 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:03:03.222 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15278.asok 2021-11-21T22:03:03.224 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15278.asok 2021-11-21T22:03:03.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:03.226 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15278.asok status 2021-11-21T22:03:03.315 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:03:03.315 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:03:03.315 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:03:03.316 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:03:03.316 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:03:03.316 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:03:03.316 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:03:03.316 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15278", 2021-11-21T22:03:03.317 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:03:03.317 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:03.317 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:03:03.317 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:03:03.318 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5404, 2021-11-21T22:03:03.318 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:03:03.318 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:03:03.318 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:03:03.318 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5404 2021-11-21T22:03:03.319 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:03.319 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:03:03.319 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:03:03.319 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:03:03.320 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 560036346 2021-11-21T22:03:03.320 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:03:03.320 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:03.321 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:03:03.321 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:03:03.322 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:03:03.322 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 560036346 2021-11-21T22:03:03.322 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:03.323 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5404 172.21.2.119:0/560036346", 2021-11-21T22:03:03.323 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/560036346", 2021-11-21T22:03:03.323 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:03:03.323 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 70, 2021-11-21T22:03:03.324 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 152, 2021-11-21T22:03:03.324 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:03:03.324 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:03:03.324 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:03:03.325 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:03:03.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:03.326 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:03.327 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:03.335 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:03:03.336 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:03:03.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:03.337 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:03.355 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:03.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.407+0000 7f4fdca8f700 1 Processor -- start 2021-11-21T22:03:03.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.407+0000 7f4fdca8f700 1 -- start start 2021-11-21T22:03:03.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.407+0000 7f4fdca8f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a6aa0 0x7f4fd814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.407+0000 7f4fdca8f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f4fd80a86b0 0x7f4fd81470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.407+0000 7f4fdca8f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd80a6e80 con 0x7f4fd81475d0 2021-11-21T22:03:03.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.407+0000 7f4fdca8f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd80a8bc0 con 0x7f4fd80a86b0 2021-11-21T22:03:03.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.407+0000 7f4fdca8f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd814a7b0 con 0x7f4fd80a6aa0 2021-11-21T22:03:03.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd6d9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f4fd81475d0 0x7f4fd81443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37760/0 (socket says 172.21.2.116:37760) 2021-11-21T22:03:03.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd6d9d700 1 -- 172.21.2.116:0/1973057730 learned_addr learned my addr 172.21.2.116:0/1973057730 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213412849 0 0) 0x7f4fd80a6e80 con 0x7f4fd81475d0 2021-11-21T22:03:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb8003260 con 0x7f4fd81475d0 2021-11-21T22:03:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd659c700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a6aa0 0x7f4fd814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 --2- 172.21.2.116:0/1973057730 >> v2:172.21.2.117:3300/0 conn(0x7f4fd80a86b0 0x7f4fd81470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 134303946 0 0) 0x7f4fb8003260 con 0x7f4fd81475d0 2021-11-21T22:03:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a6aa0 msgr2=0x7f4fd814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a6aa0 0x7f4fd814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 >> v2:172.21.2.117:3300/0 conn(0x7f4fd80a86b0 msgr2=0x7f4fd81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 --2- 172.21.2.116:0/1973057730 >> v2:172.21.2.117:3300/0 conn(0x7f4fd80a86b0 0x7f4fd81470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd814aa10 con 0x7f4fd81475d0 2021-11-21T22:03:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f4fd00019e0 con 0x7f4fd81475d0 2021-11-21T22:03:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 >> v1:172.21.2.116:6789/0 conn(0x7f4fd81475d0 legacy=0x7f4fd81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a6aa0 0x7f4fb80037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fb8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4fb80049f0 0x7f4fb8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80a8bc0 con 0x7f4fb8003e00 2021-11-21T22:03:03.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd814a7b0 con 0x7f4fb80049f0 2021-11-21T22:03:03.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb800b8e0 con 0x7f4fd80a6aa0 2021-11-21T22:03:03.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fb8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 -- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a6aa0 msgr2=0x7f4fb80037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a6aa0 0x7f4fb80037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 -- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4fb80049f0 msgr2=0x7f4fb8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4fb80049f0 0x7f4fb8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 -- 172.21.2.116:0/1973057730 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8003260 con 0x7f4fb8003e00 2021-11-21T22:03:03.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd5d9b700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fb8004500 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f4fc800a000 tx=0x7f4fc8008040).ready entity=mon.0 client_cookie=b6ac29a667170632 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc8013670 con 0x7f4fb8003e00 2021-11-21T22:03:03.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc80137d0 con 0x7f4fb8003e00 2021-11-21T22:03:03.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd559a700 1 -- 172.21.2.116:0/1973057730 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc8013b30 con 0x7f4fb8003e00 2021-11-21T22:03:03.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- 172.21.2.116:0/1973057730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 msgr2=0x7f4fb8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 --2- 172.21.2.116:0/1973057730 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fb8004500 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f4fc800a000 tx=0x7f4fc8008040).stop 2021-11-21T22:03:03.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd6d9d700 1 -- 172.21.2.116:0/1973057730 reap_dead start 2021-11-21T22:03:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- 172.21.2.116:0/1973057730 shutdown_connections 2021-11-21T22:03:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- 172.21.2.116:0/1973057730 >> 172.21.2.116:0/1973057730 conn(0x7f4fd805bd20 msgr2=0x7f4fd805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- 172.21.2.116:0/1973057730 shutdown_connections 2021-11-21T22:03:03.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- 172.21.2.116:0/1973057730 wait complete. 2021-11-21T22:03:03.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 Processor -- start 2021-11-21T22:03:03.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- start start 2021-11-21T22:03:03.429 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fd80aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a99b0 0x7f4fd80aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4fd8145e70 0x7f4fd81462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.430 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80a93c0 con 0x7f4fb8003e00 2021-11-21T22:03:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80a9120 con 0x7f4fd8145e70 2021-11-21T22:03:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fdca8f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd8156e00 con 0x7f4fd80a99b0 2021-11-21T22:03:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fd80aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fd80aa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54044/0 (socket says 172.21.2.116:54044) 2021-11-21T22:03:03.431 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.411+0000 7f4fd659c700 1 -- 172.21.2.116:0/445766864 learned_addr learned my addr 172.21.2.116:0/445766864 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:03.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd659c700 1 -- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a99b0 msgr2=0x7f4fd80aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd659c700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a99b0 0x7f4fd80aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd659c700 1 -- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4fd8145e70 msgr2=0x7f4fd81462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd659c700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4fd8145e70 0x7f4fd81462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.432 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd659c700 1 -- 172.21.2.116:0/445766864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd81570a0 con 0x7f4fb8003e00 2021-11-21T22:03:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd659c700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fd80aa210 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f4fc000a000 tx=0x7f4fc0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fcf7fe700 1 -- 172.21.2.116:0/445766864 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc0016070 con 0x7f4fb8003e00 2021-11-21T22:03:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fcf7fe700 1 -- 172.21.2.116:0/445766864 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc0000e90 con 0x7f4fb8003e00 2021-11-21T22:03:03.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd81467e0 con 0x7f4fb8003e00 2021-11-21T22:03:03.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fcf7fe700 1 -- 172.21.2.116:0/445766864 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc0011970 con 0x7f4fb8003e00 2021-11-21T22:03:03.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd80a8d20 con 0x7f4fb8003e00 2021-11-21T22:03:03.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd8156c70 con 0x7f4fb8003e00 2021-11-21T22:03:03.434 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fcf7fe700 1 -- 172.21.2.116:0/445766864 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4fc0011ad0 con 0x7f4fb8003e00 2021-11-21T22:03:03.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fcf7fe700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4fbc053520 0x7f4fbc0559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fcf7fe700 1 -- 172.21.2.116:0/445766864 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4fc006b6c0 con 0x7f4fb8003e00 2021-11-21T22:03:03.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd5d9b700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4fbc053520 0x7f4fbc0559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.415+0000 7f4fd5d9b700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4fbc053520 0x7f4fbc0559e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f4fc800a000 tx=0x7f4fc8017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.419+0000 7f4fcf7fe700 1 -- 172.21.2.116:0/445766864 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fc0042820 con 0x7f4fb8003e00 2021-11-21T22:03:03.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fd8157230 con 0x7f4fb8003e00 2021-11-21T22:03:03.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fcf7fe700 1 -- 172.21.2.116:0/445766864 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f4fc0045e40 con 0x7f4fb8003e00 2021-11-21T22:03:03.506 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:03.506 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":152,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:02:59.253897+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:02:56.060658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-11-21T22:02:57.220549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:03.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4fbc053520 msgr2=0x7f4fbc0559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4fbc053520 0x7f4fbc0559e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f4fc800a000 tx=0x7f4fc8017000).stop 2021-11-21T22:03:03.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 msgr2=0x7f4fd80aa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fd80aa210 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f4fc000a000 tx=0x7f4fc0008040).stop 2021-11-21T22:03:03.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 shutdown_connections 2021-11-21T22:03:03.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f4fd8145e70 0x7f4fd81462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f4fd80a99b0 0x7f4fd80aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f4fbc053520 0x7f4fbc0559e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 --2- 172.21.2.116:0/445766864 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f4fb8003e00 0x7f4fd80aa210 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 >> 172.21.2.116:0/445766864 conn(0x7f4fd805bd20 msgr2=0x7f4fd80b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 shutdown_connections 2021-11-21T22:03:03.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.503+0000 7f4fdca8f700 1 -- 172.21.2.116:0/445766864 wait complete. 2021-11-21T22:03:03.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:02:56.060658+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:02:57.220549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:03:03.514 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:03:03.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfede8700 1 Processor -- start 2021-11-21T22:03:03.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfede8700 1 -- start start 2021-11-21T22:03:03.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfede8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf81478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfede8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7efcf80ac600 0x7efcf80acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfede8700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf8147e80 con 0x7efcf8144810 2021-11-21T22:03:03.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfede8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf80b1490 con 0x7efcf80ac600 2021-11-21T22:03:03.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfede8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80b1fd0 con 0x7efcf80abce0 2021-11-21T22:03:03.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcfd385700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7efcf80ac600 0x7efcf80acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcf7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf81478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcf7fff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf81478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:57400/0 (socket says 172.21.2.116:57400) 2021-11-21T22:03:03.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.563+0000 7efcf7fff700 1 -- 172.21.2.116:0/3246195448 learned_addr learned my addr 172.21.2.116:0/3246195448 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:03.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 -- 172.21.2.116:0/3246195448 >> v2:172.21.2.117:3300/0 conn(0x7efcf80ac600 msgr2=0x7efcf80acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 --2- 172.21.2.116:0/3246195448 >> v2:172.21.2.117:3300/0 conn(0x7efcf80ac600 0x7efcf80acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 -- 172.21.2.116:0/3246195448 >> v1:172.21.2.117:6789/0 conn(0x7efcf8144810 legacy=0x7efcf80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 -- 172.21.2.116:0/3246195448 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80af5a0 con 0x7efcf80abce0 2021-11-21T22:03:03.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfd385700 1 --2- 172.21.2.116:0/3246195448 >> v2:172.21.2.117:3300/0 conn(0x7efcf80ac600 0x7efcf80acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:03:03.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 --2- 172.21.2.116:0/3246195448 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf81478c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efce0002f70 tx=0x7efce0006eb0).ready entity=mon.2 client_cookie=b17f46f7701ded4d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf77fe700 1 -- 172.21.2.116:0/3246195448 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce000d570 con 0x7efcf80abce0 2021-11-21T22:03:03.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf77fe700 1 -- 172.21.2.116:0/3246195448 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce0000e90 con 0x7efcf80abce0 2021-11-21T22:03:03.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf77fe700 1 -- 172.21.2.116:0/3246195448 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce000d900 con 0x7efcf80abce0 2021-11-21T22:03:03.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- 172.21.2.116:0/3246195448 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 msgr2=0x7efcf81478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 --2- 172.21.2.116:0/3246195448 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf81478c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efce0002f70 tx=0x7efce0006eb0).stop 2021-11-21T22:03:03.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- 172.21.2.116:0/3246195448 shutdown_connections 2021-11-21T22:03:03.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 --2- 172.21.2.116:0/3246195448 >> v2:172.21.2.117:3300/0 conn(0x7efcf80ac600 0x7efcf80acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 --2- 172.21.2.116:0/3246195448 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf81478c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- 172.21.2.116:0/3246195448 >> 172.21.2.116:0/3246195448 conn(0x7efcf805bd20 msgr2=0x7efcf805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- 172.21.2.116:0/3246195448 shutdown_connections 2021-11-21T22:03:03.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- 172.21.2.116:0/3246195448 wait complete. 2021-11-21T22:03:03.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 Processor -- start 2021-11-21T22:03:03.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- start start 2021-11-21T22:03:03.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf8143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efcf80ac600 0x7efcf81566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efcf8144810 0x7efcf815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf8145230 con 0x7efcf80ac600 2021-11-21T22:03:03.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf81454d0 con 0x7efcf8144810 2021-11-21T22:03:03.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcfede8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf8159c40 con 0x7efcf80abce0 2021-11-21T22:03:03.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efcf80ac600 0x7efcf81566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efcf80ac600 0x7efcf81566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54062/0 (socket says 172.21.2.116:54062) 2021-11-21T22:03:03.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.567+0000 7efcf7fff700 1 -- 172.21.2.116:0/2298837038 learned_addr learned my addr 172.21.2.116:0/2298837038 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:03.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf7fff700 1 -- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 msgr2=0x7efcf8143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf7fff700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf8143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf7fff700 1 -- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efcf8144810 msgr2=0x7efcf815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf7fff700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efcf8144810 0x7efcf815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf7fff700 1 -- 172.21.2.116:0/2298837038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf0009470 con 0x7efcf80ac600 2021-11-21T22:03:03.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf7fff700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efcf80ac600 0x7efcf81566b0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7efce0002f70 tx=0x7efce0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf5ffb700 1 -- 172.21.2.116:0/2298837038 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce00117e0 con 0x7efcf80ac600 2021-11-21T22:03:03.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf5ffb700 1 -- 172.21.2.116:0/2298837038 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce0011940 con 0x7efcf80ac600 2021-11-21T22:03:03.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce000b1e0 con 0x7efcf80ac600 2021-11-21T22:03:03.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf5ffb700 1 -- 172.21.2.116:0/2298837038 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce0011ca0 con 0x7efcf80ac600 2021-11-21T22:03:03.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf81457e0 con 0x7efcf80ac600 2021-11-21T22:03:03.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf80af060 con 0x7efcf80ac600 2021-11-21T22:03:03.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf5ffb700 1 -- 172.21.2.116:0/2298837038 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7efce0025030 con 0x7efcf80ac600 2021-11-21T22:03:03.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf5ffb700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efcdc053600 0x7efcdc055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcf5ffb700 1 -- 172.21.2.116:0/2298837038 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7efce006cb10 con 0x7efcf80ac600 2021-11-21T22:03:03.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcfcb84700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efcdc053600 0x7efcdc055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.571+0000 7efcfcb84700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efcdc053600 0x7efcdc055ac0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7efcf80b2a30 tx=0x7efce8007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.575+0000 7efcf5ffb700 1 -- 172.21.2.116:0/2298837038 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efce0043d20 con 0x7efcf80ac600 2021-11-21T22:03:03.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efcf80573d0 con 0x7efcf80ac600 2021-11-21T22:03:03.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcf5ffb700 1 -- 172.21.2.116:0/2298837038 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v152) v1 ==== 110+0+10 (secure 0 0 0) 0x7efce0023020 con 0x7efcf80ac600 2021-11-21T22:03:03.661 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:03:03.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efcdc053600 msgr2=0x7efcdc055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efcdc053600 0x7efcdc055ac0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7efcf80b2a30 tx=0x7efce8007810).stop 2021-11-21T22:03:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efcf80ac600 msgr2=0x7efcf81566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efcf80ac600 0x7efcf81566b0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7efce0002f70 tx=0x7efce0015000).stop 2021-11-21T22:03:03.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 shutdown_connections 2021-11-21T22:03:03.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7efcf8144810 0x7efcf815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7efcdc053600 0x7efcdc055ac0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7efcf80ac600 0x7efcf81566b0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 --2- 172.21.2.116:0/2298837038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7efcf80abce0 0x7efcf8143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 >> 172.21.2.116:0/2298837038 conn(0x7efcf805bd20 msgr2=0x7efcf8148280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 shutdown_connections 2021-11-21T22:03:03.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.655+0000 7efcfede8700 1 -- 172.21.2.116:0/2298837038 wait complete. 2021-11-21T22:03:03.669 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:03:03.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 Processor -- start 2021-11-21T22:03:03.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- start start 2021-11-21T22:03:03.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbd78144810 0x7fbd781478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd780ac600 0x7fbd780aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd78147e80 con 0x7fbd780abce0 2021-11-21T22:03:03.723 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd780b1490 con 0x7fbd78144810 2021-11-21T22:03:03.724 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780b1fd0 con 0x7fbd780ac600 2021-11-21T22:03:03.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbd78144810 0x7fbd781478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbd78144810 0x7fbd781478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54066/0 (socket says 172.21.2.116:54066) 2021-11-21T22:03:03.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2235957111 learned_addr learned my addr 172.21.2.116:0/2235957111 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:03.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2235957111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd780ac600 msgr2=0x7fbd780aca90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:03.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 --2- 172.21.2.116:0/2235957111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd780ac600 0x7fbd780aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2235957111 >> v1:172.21.2.117:6789/0 conn(0x7fbd780abce0 legacy=0x7fbd780b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:03.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2235957111 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd780af5a0 con 0x7fbd78144810 2021-11-21T22:03:03.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7cfb0700 1 --2- 172.21.2.116:0/2235957111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd78144810 0x7fbd781478c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fbd780b4e00 tx=0x7fbd68006eb0).ready entity=mon.0 client_cookie=e251e724d4f83855 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd777fe700 1 -- 172.21.2.116:0/2235957111 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd6800d570 con 0x7fbd78144810 2021-11-21T22:03:03.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd777fe700 1 -- 172.21.2.116:0/2235957111 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd68000e90 con 0x7fbd78144810 2021-11-21T22:03:03.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd777fe700 1 -- 172.21.2.116:0/2235957111 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd6800d8c0 con 0x7fbd78144810 2021-11-21T22:03:03.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- 172.21.2.116:0/2235957111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd78144810 msgr2=0x7fbd781478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2235957111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd78144810 0x7fbd781478c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fbd780b4e00 tx=0x7fbd68006eb0).stop 2021-11-21T22:03:03.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- 172.21.2.116:0/2235957111 shutdown_connections 2021-11-21T22:03:03.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2235957111 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd780ac600 0x7fbd780aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2235957111 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd78144810 0x7fbd781478c0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- 172.21.2.116:0/2235957111 >> 172.21.2.116:0/2235957111 conn(0x7fbd7805bd20 msgr2=0x7fbd7805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- 172.21.2.116:0/2235957111 shutdown_connections 2021-11-21T22:03:03.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.719+0000 7fbd7f214700 1 -- 172.21.2.116:0/2235957111 wait complete. 2021-11-21T22:03:03.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 Processor -- start 2021-11-21T22:03:03.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 -- start start 2021-11-21T22:03:03.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd780abce0 0x7fbd780b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbd780ac600 0x7fbd78139280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd78144810 0x7fbd781464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780b3980 con 0x7fbd780abce0 2021-11-21T22:03:03.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780b3160 con 0x7fbd780ac600 2021-11-21T22:03:03.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780b3bb0 con 0x7fbd78144810 2021-11-21T22:03:03.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd780abce0 0x7fbd780b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd780abce0 0x7fbd780b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54072/0 (socket says 172.21.2.116:54072) 2021-11-21T22:03:03.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2012945612 learned_addr learned my addr 172.21.2.116:0/2012945612 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:03.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd78144810 msgr2=0x7fbd781464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd78144810 0x7fbd781464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbd780ac600 msgr2=0x7fbd78139280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbd780ac600 0x7fbd78139280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 -- 172.21.2.116:0/2012945612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6800b1e0 con 0x7fbd780abce0 2021-11-21T22:03:03.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7cfb0700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd780abce0 0x7fbd780b3350 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fbd780b4e00 tx=0x7fbd68015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd75ffb700 1 -- 172.21.2.116:0/2012945612 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd68002f70 con 0x7fbd780abce0 2021-11-21T22:03:03.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd75ffb700 1 -- 172.21.2.116:0/2012945612 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6800de20 con 0x7fbd780abce0 2021-11-21T22:03:03.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd780b3e50 con 0x7fbd780abce0 2021-11-21T22:03:03.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd75ffb700 1 -- 172.21.2.116:0/2012945612 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd6801e460 con 0x7fbd780abce0 2021-11-21T22:03:03.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd78142930 con 0x7fbd780abce0 2021-11-21T22:03:03.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd780af060 con 0x7fbd780abce0 2021-11-21T22:03:03.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd75ffb700 1 -- 172.21.2.116:0/2012945612 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbd68025030 con 0x7fbd780abce0 2021-11-21T22:03:03.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd75ffb700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbd5c053690 0x7fbd5c055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd75ffb700 1 -- 172.21.2.116:0/2012945612 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbd6806ce70 con 0x7fbd780abce0 2021-11-21T22:03:03.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.723+0000 7fbd77fff700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbd5c053690 0x7fbd5c055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.727+0000 7fbd77fff700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbd5c053690 0x7fbd5c055b50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fbd60002fb0 tx=0x7fbd600069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.727+0000 7fbd75ffb700 1 -- 172.21.2.116:0/2012945612 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbd68043ec0 con 0x7fbd780abce0 2021-11-21T22:03:03.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.807+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd780573d0 con 0x7fbd780abce0 2021-11-21T22:03:03.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd75ffb700 1 -- 172.21.2.116:0/2012945612 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v152) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd68023070 con 0x7fbd780abce0 2021-11-21T22:03:03.814 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbd5c053690 msgr2=0x7fbd5c055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbd5c053690 0x7fbd5c055b50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fbd60002fb0 tx=0x7fbd600069b0).stop 2021-11-21T22:03:03.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd780abce0 msgr2=0x7fbd780b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd780abce0 0x7fbd780b3350 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fbd780b4e00 tx=0x7fbd68015000).stop 2021-11-21T22:03:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 shutdown_connections 2021-11-21T22:03:03.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbd78144810 0x7fbd781464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbd780ac600 0x7fbd78139280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbd5c053690 0x7fbd5c055b50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 --2- 172.21.2.116:0/2012945612 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbd780abce0 0x7fbd780b3350 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 >> 172.21.2.116:0/2012945612 conn(0x7fbd7805bd20 msgr2=0x7fbd780b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 shutdown_connections 2021-11-21T22:03:03.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.811+0000 7fbd7f214700 1 -- 172.21.2.116:0/2012945612 wait complete. 2021-11-21T22:03:03.821 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:03:03.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.867+0000 7f1bc59e9700 1 Processor -- start 2021-11-21T22:03:03.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.867+0000 7f1bc59e9700 1 -- start start 2021-11-21T22:03:03.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1bc01426c0 0x7f1bc00ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc01433b0 con 0x7f1bc00ae1c0 2021-11-21T22:03:03.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc00b2040 con 0x7f1bc00ac2f0 2021-11-21T22:03:03.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc0143110 con 0x7f1bc01426c0 2021-11-21T22:03:03.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f1bc00ae1c0 0x7f1bc00b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37806/0 (socket says 172.21.2.116:37806) 2021-11-21T22:03:03.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 -- 172.21.2.116:0/1375794842 learned_addr learned my addr 172.21.2.116:0/1375794842 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:03.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472545735 0 0) 0x7f1bc01433b0 con 0x7f1bc00ae1c0 2021-11-21T22:03:03.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba0003260 con 0x7f1bc00ae1c0 2021-11-21T22:03:03.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1246141676 0 0) 0x7f1ba0003260 con 0x7f1bc00ae1c0 2021-11-21T22:03:03.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 >> v2:172.21.2.117:3300/0 conn(0x7f1bc01426c0 msgr2=0x7f1bc00ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 --2- 172.21.2.116:0/1375794842 >> v2:172.21.2.117:3300/0 conn(0x7f1bc01426c0 0x7f1bc00ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 >> v1:172.21.2.117:6789/0 conn(0x7f1bc00ac2f0 legacy=0x7f1bc00af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00ada80 con 0x7f1bc00ae1c0 2021-11-21T22:03:03.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f1bb80019e0 con 0x7f1bc00ae1c0 2021-11-21T22:03:03.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 >> v1:172.21.2.116:6789/0 conn(0x7f1bc00ae1c0 legacy=0x7f1bc00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ba0003750 0x7f1ba0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ba0004500 0x7f1ba0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1ba000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc0143110 con 0x7f1ba0004f90 2021-11-21T22:03:03.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba000be80 con 0x7f1ba0004500 2021-11-21T22:03:03.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba000c040 con 0x7f1ba0003750 2021-11-21T22:03:03.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1ba000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 -- 172.21.2.116:0/1375794842 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ba0003750 msgr2=0x7f1ba0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1ba0003750 0x7f1ba0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 -- 172.21.2.116:0/1375794842 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ba0004500 msgr2=0x7f1ba0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:03.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1ba0004500 0x7f1ba0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 -- 172.21.2.116:0/1375794842 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba0003260 con 0x7f1ba0004f90 2021-11-21T22:03:03.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 -- 172.21.2.116:0/1375794842 reap_dead start 2021-11-21T22:03:03.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbf7fe700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1ba000b750 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f1bc005d170 tx=0x7f1bc00adc10).ready entity=mon.0 client_cookie=9d07d559e67326a2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bb800cd10 con 0x7f1ba0004f90 2021-11-21T22:03:03.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb800ce70 con 0x7f1ba0004f90 2021-11-21T22:03:03.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bbdffb700 1 -- 172.21.2.116:0/1375794842 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bb8005b80 con 0x7f1ba0004f90 2021-11-21T22:03:03.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1375794842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 msgr2=0x7f1ba000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1ba000b750 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f1bc005d170 tx=0x7f1bc00adc10).stop 2021-11-21T22:03:03.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1375794842 shutdown_connections 2021-11-21T22:03:03.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1375794842 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1ba000b750 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1375794842 >> 172.21.2.116:0/1375794842 conn(0x7f1bc005bd20 msgr2=0x7f1bc005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1375794842 shutdown_connections 2021-11-21T22:03:03.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1375794842 wait complete. 2021-11-21T22:03:03.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 Processor -- start 2021-11-21T22:03:03.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- start start 2021-11-21T22:03:03.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1bc012e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bc01553f0 0x7f1bc013d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bc0066c00 0x7f1bc00670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc013f610 con 0x7f1ba0004f90 2021-11-21T22:03:03.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc013f800 con 0x7f1bc01553f0 2021-11-21T22:03:03.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.871+0000 7f1bc59e9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc013dc10 con 0x7f1bc0066c00 2021-11-21T22:03:03.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1bc012e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1bc012e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54092/0 (socket says 172.21.2.116:54092) 2021-11-21T22:03:03.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 -- 172.21.2.116:0/1185445960 learned_addr learned my addr 172.21.2.116:0/1185445960 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:03.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 -- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bc0066c00 msgr2=0x7f1bc00670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bc0066c00 0x7f1bc00670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 -- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bc01553f0 msgr2=0x7f1bc013d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bc01553f0 0x7f1bc013d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 -- 172.21.2.116:0/1185445960 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc013deb0 con 0x7f1ba0004f90 2021-11-21T22:03:03.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbeffd700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1bc012e1e0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f1ba8002f70 tx=0x7f1ba8006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bb7fff700 1 -- 172.21.2.116:0/1185445960 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba8013460 con 0x7f1ba0004f90 2021-11-21T22:03:03.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bb7fff700 1 -- 172.21.2.116:0/1185445960 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba80135c0 con 0x7f1ba0004f90 2021-11-21T22:03:03.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc00aeac0 con 0x7f1ba0004f90 2021-11-21T22:03:03.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bb7fff700 1 -- 172.21.2.116:0/1185445960 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba8013920 con 0x7f1ba0004f90 2021-11-21T22:03:03.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc013f240 con 0x7f1ba0004f90 2021-11-21T22:03:03.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc013e040 con 0x7f1ba0004f90 2021-11-21T22:03:03.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bb7fff700 1 -- 172.21.2.116:0/1185445960 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1ba801a060 con 0x7f1ba0004f90 2021-11-21T22:03:03.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bb7fff700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ba40536f0 0x7f1ba4055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:03.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bb7fff700 1 -- 172.21.2.116:0/1185445960 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1ba806b360 con 0x7f1ba0004f90 2021-11-21T22:03:03.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbe7fc700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ba40536f0 0x7f1ba4055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:03.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.875+0000 7f1bbe7fc700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ba40536f0 0x7f1ba4055bb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f1bb0000f70 tx=0x7f1bb00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:03.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.879+0000 7f1bb7fff700 1 -- 172.21.2.116:0/1185445960 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ba80424c0 con 0x7f1ba0004f90 2021-11-21T22:03:03.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1bc00add20 con 0x7f1ba0004f90 2021-11-21T22:03:03.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bb7fff700 1 -- 172.21.2.116:0/1185445960 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v152) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1ba8045ae0 con 0x7f1ba0004f90 2021-11-21T22:03:03.966 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:03.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ba40536f0 msgr2=0x7f1ba4055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ba40536f0 0x7f1ba4055bb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f1bb0000f70 tx=0x7f1bb00069b0).stop 2021-11-21T22:03:03.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 msgr2=0x7f1bc012e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:03.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1bc012e1e0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f1ba8002f70 tx=0x7f1ba8006e40).stop 2021-11-21T22:03:03.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 shutdown_connections 2021-11-21T22:03:03.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1bc0066c00 0x7f1bc00670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1bc01553f0 0x7f1bc013d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:03.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1ba40536f0 0x7f1ba4055bb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 --2- 172.21.2.116:0/1185445960 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1ba0004f90 0x7f1bc012e1e0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:03.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 >> 172.21.2.116:0/1185445960 conn(0x7f1bc005bd20 msgr2=0x7f1bc0140ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:03.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 shutdown_connections 2021-11-21T22:03:03.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:03.963+0000 7f1bc59e9700 1 -- 172.21.2.116:0/1185445960 wait complete. 2021-11-21T22:03:03.973 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:03:04.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 Processor -- start 2021-11-21T22:03:04.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 -- start start 2021-11-21T22:03:04.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff3b0144810 0x7ff3b01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff3b00ac600 0x7ff3b00aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b0147e80 con 0x7ff3b00abce0 2021-11-21T22:03:04.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00b1490 con 0x7ff3b00ac600 2021-11-21T22:03:04.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00b1fd0 con 0x7ff3b0144810 2021-11-21T22:03:04.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3add9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ff3b00abce0 0x7ff3b00b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37826/0 (socket says 172.21.2.116:37826) 2021-11-21T22:03:04.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3add9b700 1 -- 172.21.2.116:0/1268587082 learned_addr learned my addr 172.21.2.116:0/1268587082 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:04.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3aed9d700 1 --2- 172.21.2.116:0/1268587082 >> v2:172.21.2.116:3300/0 conn(0x7ff3b00ac600 0x7ff3b00aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:04.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3aed9d700 1 -- 172.21.2.116:0/1268587082 >> v2:172.21.2.117:3300/0 conn(0x7ff3b0144810 msgr2=0x7ff3b01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:04.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3aed9d700 1 --2- 172.21.2.116:0/1268587082 >> v2:172.21.2.117:3300/0 conn(0x7ff3b0144810 0x7ff3b01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3aed9d700 1 -- 172.21.2.116:0/1268587082 >> v1:172.21.2.116:6789/0 conn(0x7ff3b00abce0 legacy=0x7ff3b00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3aed9d700 1 -- 172.21.2.116:0/1268587082 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00af5a0 con 0x7ff3b00ac600 2021-11-21T22:03:04.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3aed9d700 1 --2- 172.21.2.116:0/1268587082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00ac600 0x7ff3b00aca50 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7ff3a80032f0 tx=0x7ff3a80091d0).ready entity=mon.0 client_cookie=4f438ec6fb98726d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:04.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3ad59a700 1 -- 172.21.2.116:0/1268587082 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a8009ea0 con 0x7ff3b00ac600 2021-11-21T22:03:04.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3ad59a700 1 -- 172.21.2.116:0/1268587082 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800b870 con 0x7ff3b00ac600 2021-11-21T22:03:04.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3ad59a700 1 -- 172.21.2.116:0/1268587082 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a800b9e0 con 0x7ff3b00ac600 2021-11-21T22:03:04.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 -- 172.21.2.116:0/1268587082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00ac600 msgr2=0x7ff3b00aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.023+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/1268587082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00ac600 0x7ff3b00aca50 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7ff3a80032f0 tx=0x7ff3a80091d0).stop 2021-11-21T22:03:04.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- 172.21.2.116:0/1268587082 shutdown_connections 2021-11-21T22:03:04.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/1268587082 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00ac600 0x7ff3b00aca50 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:04.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/1268587082 >> v2:172.21.2.117:3300/0 conn(0x7ff3b0144810 0x7ff3b01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- 172.21.2.116:0/1268587082 >> 172.21.2.116:0/1268587082 conn(0x7ff3b005bd20 msgr2=0x7ff3b005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:04.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- 172.21.2.116:0/1268587082 shutdown_connections 2021-11-21T22:03:04.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- 172.21.2.116:0/1268587082 wait complete. 2021-11-21T22:03:04.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 Processor -- start 2021-11-21T22:03:04.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- start start 2021-11-21T22:03:04.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00abce0 0x7ff3b0143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff3b00ac600 0x7ff3b00b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff3b0144810 0x7ff3b013a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0146660 con 0x7ff3b00abce0 2021-11-21T22:03:04.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00b2f10 con 0x7ff3b00ac600 2021-11-21T22:03:04.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b014a2a0 con 0x7ff3b0144810 2021-11-21T22:03:04.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00abce0 0x7ff3b0143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:04.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00abce0 0x7ff3b0143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54108/0 (socket says 172.21.2.116:54108) 2021-11-21T22:03:04.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 -- 172.21.2.116:0/2681332045 learned_addr learned my addr 172.21.2.116:0/2681332045 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:04.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 -- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff3b0144810 msgr2=0x7ff3b013a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff3b0144810 0x7ff3b013a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 -- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff3b00ac600 msgr2=0x7ff3b00b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff3b00ac600 0x7ff3b00b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 -- 172.21.2.116:0/2681332045 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8009470 con 0x7ff3b00abce0 2021-11-21T22:03:04.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3ae59c700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00abce0 0x7ff3b0143320 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7ff3a0002f70 tx=0x7ff3a0006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:04.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3a77fe700 1 -- 172.21.2.116:0/2681332045 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a0013460 con 0x7ff3b00abce0 2021-11-21T22:03:04.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3a77fe700 1 -- 172.21.2.116:0/2681332045 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a00135c0 con 0x7ff3b00abce0 2021-11-21T22:03:04.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3a77fe700 1 -- 172.21.2.116:0/2681332045 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a0013920 con 0x7ff3b00abce0 2021-11-21T22:03:04.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b00b42f0 con 0x7ff3b00abce0 2021-11-21T22:03:04.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b014a7b0 con 0x7ff3b00abce0 2021-11-21T22:03:04.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b00b30d0 con 0x7ff3b00abce0 2021-11-21T22:03:04.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3a77fe700 1 -- 172.21.2.116:0/2681332045 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff3a000f070 con 0x7ff3b00abce0 2021-11-21T22:03:04.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3a77fe700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff394053610 0x7ff394055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3a77fe700 1 -- 172.21.2.116:0/2681332045 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff3a006b220 con 0x7ff3b00abce0 2021-11-21T22:03:04.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.027+0000 7ff3add9b700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff394053610 0x7ff394055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:04.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.031+0000 7ff3add9b700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff394053610 0x7ff394055ad0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7ff3b00b4e00 tx=0x7ff398007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:04.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.031+0000 7ff3a77fe700 1 -- 172.21.2.116:0/2681332045 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3a0042380 con 0x7ff3b00abce0 2021-11-21T22:03:04.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3b00af060 con 0x7ff3b00abce0 2021-11-21T22:03:04.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3a77fe700 1 -- 172.21.2.116:0/2681332045 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v152) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff3a00459a0 con 0x7ff3b00abce0 2021-11-21T22:03:04.118 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:04.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff394053610 msgr2=0x7ff394055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff394053610 0x7ff394055ad0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7ff3b00b4e00 tx=0x7ff398007800).stop 2021-11-21T22:03:04.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00abce0 msgr2=0x7ff3b0143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00abce0 0x7ff3b0143320 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7ff3a0002f70 tx=0x7ff3a0006e20).stop 2021-11-21T22:03:04.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 shutdown_connections 2021-11-21T22:03:04.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff3b0144810 0x7ff3b013a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff3b00ac600 0x7ff3b00b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff394053610 0x7ff394055ad0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:04.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 --2- 172.21.2.116:0/2681332045 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff3b00abce0 0x7ff3b0143320 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:04.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 >> 172.21.2.116:0/2681332045 conn(0x7ff3b005bd20 msgr2=0x7ff3b013ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:04.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 shutdown_connections 2021-11-21T22:03:04.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.115+0000 7ff3b4f65700 1 -- 172.21.2.116:0/2681332045 wait complete. 2021-11-21T22:03:04.125 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:03:04.126 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:04.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 Processor -- start 2021-11-21T22:03:04.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 -- start start 2021-11-21T22:03:04.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc080a6aa0 0x7fdc0814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a86b0 0x7fdc081470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc080a6e80 con 0x7fdc081475d0 2021-11-21T22:03:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc080a8bc0 con 0x7fdc080a6aa0 2021-11-21T22:03:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0814a7b0 con 0x7fdc080a86b0 2021-11-21T22:03:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0dccb700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fdc081475d0 0x7fdc081443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37838/0 (socket says 172.21.2.116:37838) 2021-11-21T22:03:04.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0dccb700 1 -- 172.21.2.116:0/680452755 learned_addr learned my addr 172.21.2.116:0/680452755 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0d4ca700 1 --2- 172.21.2.116:0/680452755 >> v2:172.21.2.116:3300/0 conn(0x7fdc080a6aa0 0x7fdc0814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdbf7fff700 1 -- 172.21.2.116:0/680452755 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228419849 0 0) 0x7fdc080a6e80 con 0x7fdc081475d0 2021-11-21T22:03:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0d4ca700 1 -- 172.21.2.116:0/680452755 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a86b0 msgr2=0x7fdc081470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:04.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0d4ca700 1 --2- 172.21.2.116:0/680452755 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a86b0 0x7fdc081470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0d4ca700 1 -- 172.21.2.116:0/680452755 >> v1:172.21.2.116:6789/0 conn(0x7fdc081475d0 legacy=0x7fdc081443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0d4ca700 1 -- 172.21.2.116:0/680452755 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0814aa10 con 0x7fdc080a6aa0 2021-11-21T22:03:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0d4ca700 1 --2- 172.21.2.116:0/680452755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc080a6aa0 0x7fdc0814b150 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7fdbf8002f70 tx=0x7fdbf8006df0).ready entity=mon.0 client_cookie=c6dcbe8ef1559ecf server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdbf7fff700 1 -- 172.21.2.116:0/680452755 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbf800d570 con 0x7fdc080a6aa0 2021-11-21T22:03:04.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdbf7fff700 1 -- 172.21.2.116:0/680452755 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf8000e90 con 0x7fdc080a6aa0 2021-11-21T22:03:04.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdbf7fff700 1 -- 172.21.2.116:0/680452755 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbf800d8c0 con 0x7fdc080a6aa0 2021-11-21T22:03:04.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 -- 172.21.2.116:0/680452755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc080a6aa0 msgr2=0x7fdc0814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/680452755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc080a6aa0 0x7fdc0814b150 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7fdbf8002f70 tx=0x7fdbf8006df0).stop 2021-11-21T22:03:04.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 -- 172.21.2.116:0/680452755 shutdown_connections 2021-11-21T22:03:04.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/680452755 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a86b0 0x7fdc081470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.175+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/680452755 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc080a6aa0 0x7fdc0814b150 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:04.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- 172.21.2.116:0/680452755 >> 172.21.2.116:0/680452755 conn(0x7fdc0805bd20 msgr2=0x7fdc0805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:04.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- 172.21.2.116:0/680452755 shutdown_connections 2021-11-21T22:03:04.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- 172.21.2.116:0/680452755 wait complete. 2021-11-21T22:03:04.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 Processor -- start 2021-11-21T22:03:04.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- start start 2021-11-21T22:03:04.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a6aa0 0x7fdc0814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc080a86b0 0x7fdc08139320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc081475d0 0x7fdc08139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0813a460 con 0x7fdc081475d0 2021-11-21T22:03:04.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0813c730 con 0x7fdc080a86b0 2021-11-21T22:03:04.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0813c9d0 con 0x7fdc080a6aa0 2021-11-21T22:03:04.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc081475d0 0x7fdc08139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:04.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc081475d0 0x7fdc08139810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54122/0 (socket says 172.21.2.116:54122) 2021-11-21T22:03:04.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 -- 172.21.2.116:0/3487985439 learned_addr learned my addr 172.21.2.116:0/3487985439 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:04.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 -- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a6aa0 msgr2=0x7fdc0814b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a6aa0 0x7fdc0814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 -- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc080a86b0 msgr2=0x7fdc08139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.193 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc080a86b0 0x7fdc08139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 -- 172.21.2.116:0/3487985439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf800b180 con 0x7fdc081475d0 2021-11-21T22:03:04.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0dccb700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc081475d0 0x7fdc08139810 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7fdc04000ed0 tx=0x7fdc04007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:04.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdbf67fc700 1 -- 172.21.2.116:0/3487985439 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc04001d50 con 0x7fdc081475d0 2021-11-21T22:03:04.194 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdbf67fc700 1 -- 172.21.2.116:0/3487985439 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04007320 con 0x7fdc081475d0 2021-11-21T22:03:04.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0813cc70 con 0x7fdc081475d0 2021-11-21T22:03:04.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdbf67fc700 1 -- 172.21.2.116:0/3487985439 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc04005220 con 0x7fdc081475d0 2021-11-21T22:03:04.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0813f2f0 con 0x7fdc081475d0 2021-11-21T22:03:04.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc080573d0 con 0x7fdc081475d0 2021-11-21T22:03:04.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdbf67fc700 1 -- 172.21.2.116:0/3487985439 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdc04005610 con 0x7fdc081475d0 2021-11-21T22:03:04.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdbf67fc700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdbe8053610 0x7fdbe8055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:04.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdbf67fc700 1 -- 172.21.2.116:0/3487985439 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdc0406b2e0 con 0x7fdc081475d0 2021-11-21T22:03:04.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.179+0000 7fdc0d4ca700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdbe8053610 0x7fdbe8055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:04.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.183+0000 7fdc0d4ca700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdbe8053610 0x7fdbe8055ad0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fdbf8002f70 tx=0x7fdbf8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:04.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.183+0000 7fdbf67fc700 1 -- 172.21.2.116:0/3487985439 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc040423e0 con 0x7fdc081475d0 2021-11-21T22:03:04.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.291+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc0805db70 con 0x7fdc081475d0 2021-11-21T22:03:04.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.291+0000 7fdbf67fc700 1 -- 172.21.2.116:0/3487985439 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fdc04007490 con 0x7fdc081475d0 2021-11-21T22:03:04.297 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:04.297 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":70,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:58.250794+0000","modified":"2021-11-21T22:02:59.255376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/139139464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":139139464},{"type":"v1","addr":"172.21.2.116:6835","nonce":139139464}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-11-21T22:03:04.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdbe8053610 msgr2=0x7fdbe8055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdbe8053610 0x7fdbe8055ad0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fdbf8002f70 tx=0x7fdbf8015000).stop 2021-11-21T22:03:04.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc081475d0 msgr2=0x7fdc08139810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:04.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc081475d0 0x7fdc08139810 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7fdc04000ed0 tx=0x7fdc04007c60).stop 2021-11-21T22:03:04.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 shutdown_connections 2021-11-21T22:03:04.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdbe8053610 0x7fdbe8055ad0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:04.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc081475d0 0x7fdc08139810 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:04.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc080a86b0 0x7fdc08139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 --2- 172.21.2.116:0/3487985439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc080a6aa0 0x7fdc0814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:04.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 >> 172.21.2.116:0/3487985439 conn(0x7fdc0805bd20 msgr2=0x7fdc080b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:04.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 shutdown_connections 2021-11-21T22:03:04.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:04.295+0000 7fdc0f72e700 1 -- 172.21.2.116:0/3487985439 wait complete. 2021-11-21T22:03:04.301 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 70 2021-11-21T22:03:04.582 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:03:04.586 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:03:04.589 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:03:04.593 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:03:04.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:02:58.250794+0000', 'modified': '2021-11-21T22:02:59.255376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8472': {'gid': 8472, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/139139464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 139139464}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 139139464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:03:04.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:03:04.607 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:03:04.669 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8472,"want_state":"up:active","state":"up:active","rank_uptime":6.410935856,"mdsmap_epoch":70,"osdmap_epoch":152,"osdmap_epoch_barrier":152,"uptime":10.667368140000001} 2021-11-21T22:03:04.669 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8472, "mdsmap_epoch": 70, "osdmap_epoch": 152, "osdmap_epoch_barrier": 152, "rank_uptime": 6.410935856, "state": "up:active", "uptime": 10.66736814, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:03:04.670 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:03:04.670 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:03:04.671 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:03:04.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:04.672 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:04.678 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:03:04.742 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:04.742 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:04.763 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:04.765 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:04.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:04.766 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:03:04.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:03:04.777 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:04.822 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:03:04.834 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:04.834 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:04.840 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:04.840 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:04.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:04.842 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:03:04.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:03:04.866+0000 7fc05dccc080 -1 init, newargv = 0x55fb901efc40 newargc=9 2021-11-21T22:03:04.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14777]: starting ceph client 2021-11-21T22:03:04.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14777]: starting fuse 2021-11-21T22:03:05.971 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:03:05.983 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:05.984 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:06.034 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:06.035 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:06.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.036 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:03:06.041 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:03:06.042 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:03:06.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.043 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:03:06.043 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:03:06.044 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:03:06.044 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:03:06.044 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:03:06.045 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:06.045 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:03:06.046 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:03:06.046 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:03:06.046 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:06.047 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:03:06.047 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:03:06.048 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:03:06.048 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:03:06.048 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:06.049 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:03:06.049 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:03:06.050 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:03:06.050 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:03:06.050 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:03:06.051 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:06.051 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:03:06.052 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:03:06.117 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14777.asok 2021-11-21T22:03:06.120 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14777.asok 2021-11-21T22:03:06.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.121 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14777.asok status 2021-11-21T22:03:06.168 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:03:06.169 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:03:06.169 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:03:06.169 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:03:06.169 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:03:06.170 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:03:06.170 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:03:06.170 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14777", 2021-11-21T22:03:06.170 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:03:06.171 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:06.171 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:03:06.172 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:03:06.172 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 5081, 2021-11-21T22:03:06.172 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:03:06.173 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:03:06.173 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:03:06.173 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 5081 2021-11-21T22:03:06.173 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:06.174 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:03:06.174 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:03:06.174 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:03:06.175 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1246944395 2021-11-21T22:03:06.175 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:03:06.175 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:06.176 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:03:06.176 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:03:06.176 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:03:06.177 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1246944395 2021-11-21T22:03:06.177 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:06.177 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.5081 172.21.2.136:0/1246944395", 2021-11-21T22:03:06.177 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/1246944395", 2021-11-21T22:03:06.178 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:03:06.178 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 70, 2021-11-21T22:03:06.178 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 152, 2021-11-21T22:03:06.179 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:03:06.179 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:03:06.179 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:03:06.179 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:03:06.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.181 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:06.182 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:06.187 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:03:06.188 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:03:06.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.189 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:06.242 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-11-21T22:03:06.243 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2021-11-21T22:03:06.280 INFO:teuthology.orchestra.run.gibba019.stdout:1637532186.241346211 2021-11-21T22:03:06.280 INFO:tasks.cephfs.test_misc:rctime = 1637532186.241346211 2021-11-21T22:03:06.281 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:03:06.281 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:06.294 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:06.308 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:06.333 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:06.362 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:03:06.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.363 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:06.364 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:06.370 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:03:06.370 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:03:06.371 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-11-21T22:03:06.371 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:03:06.371 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:06.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15278]: fuse finished with error 0 and tester_r 0 2021-11-21T22:03:06.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.429 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:06.430 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:06.434 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-11-21T22:03:06.435 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:03:06.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:03:06.556 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:06.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.567 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:06.567 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:06.584 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:03:06.584 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:03:06.585 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:03:06.585 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:03:06.585 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:06.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14777]: fuse finished with error 0 and tester_r 0 2021-11-21T22:03:06.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:06.643 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:06.643 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:06.647 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:03:06.648 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:03:06.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:03:06.666 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:06.712 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:03:06.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.763+0000 7f8f5690f700 1 Processor -- start 2021-11-21T22:03:06.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.763+0000 7f8f5690f700 1 -- start start 2021-11-21T22:03:06.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.763+0000 7f8f5690f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a6aa0 0x7f8f5014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:06.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.763+0000 7f8f5690f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8f500a86b0 0x7f8f501470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:06.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.763+0000 7f8f5690f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f500a6e80 con 0x7f8f501475d0 2021-11-21T22:03:06.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.763+0000 7f8f5690f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f500a8bc0 con 0x7f8f500a86b0 2021-11-21T22:03:06.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.763+0000 7f8f5690f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5014a7b0 con 0x7f8f500a6aa0 2021-11-21T22:03:06.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4ffff700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a6aa0 0x7f8f5014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:06.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4f7fe700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8f500a86b0 0x7f8f501470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:06.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f54eac700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f8f501475d0 0x7f8f501443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:36762/0 (socket says 172.21.2.116:36762) 2021-11-21T22:03:06.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f54eac700 1 -- 172.21.2.116:0/2155151251 learned_addr learned my addr 172.21.2.116:0/2155151251 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:06.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4f7fe700 1 -- 172.21.2.116:0/2155151251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a6aa0 msgr2=0x7f8f5014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:06.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4f7fe700 1 --2- 172.21.2.116:0/2155151251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a6aa0 0x7f8f5014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:06.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4f7fe700 1 -- 172.21.2.116:0/2155151251 >> v1:172.21.2.117:6789/0 conn(0x7f8f501475d0 legacy=0x7f8f501443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:06.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4f7fe700 1 -- 172.21.2.116:0/2155151251 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5014aa10 con 0x7f8f500a86b0 2021-11-21T22:03:06.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4f7fe700 1 --2- 172.21.2.116:0/2155151251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a86b0 0x7f8f501470e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f8f38002f70 tx=0x7f8f38006df0).ready entity=mon.1 client_cookie=d60ef30c86b5132c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:06.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4effd700 1 -- 172.21.2.116:0/2155151251 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f3800d570 con 0x7f8f500a86b0 2021-11-21T22:03:06.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4effd700 1 -- 172.21.2.116:0/2155151251 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38000e90 con 0x7f8f500a86b0 2021-11-21T22:03:06.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f4effd700 1 -- 172.21.2.116:0/2155151251 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f3800d8c0 con 0x7f8f500a86b0 2021-11-21T22:03:06.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 -- 172.21.2.116:0/2155151251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a86b0 msgr2=0x7f8f501470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:06.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 --2- 172.21.2.116:0/2155151251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a86b0 0x7f8f501470e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f8f38002f70 tx=0x7f8f38006df0).stop 2021-11-21T22:03:06.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 -- 172.21.2.116:0/2155151251 shutdown_connections 2021-11-21T22:03:06.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 --2- 172.21.2.116:0/2155151251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a86b0 0x7f8f501470e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:06.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 --2- 172.21.2.116:0/2155151251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a6aa0 0x7f8f5014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:06.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 -- 172.21.2.116:0/2155151251 >> 172.21.2.116:0/2155151251 conn(0x7f8f5005bd20 msgr2=0x7f8f5005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:06.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 -- 172.21.2.116:0/2155151251 shutdown_connections 2021-11-21T22:03:06.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.767+0000 7f8f5690f700 1 -- 172.21.2.116:0/2155151251 wait complete. 2021-11-21T22:03:06.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 Processor -- start 2021-11-21T22:03:06.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 -- start start 2021-11-21T22:03:06.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a6aa0 0x7f8f500672a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:06.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a86b0 0x7f8f50146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:06.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8f501475d0 0x7f8f50142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:06.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50146470 con 0x7f8f501475d0 2021-11-21T22:03:06.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50064950 con 0x7f8f500a6aa0 2021-11-21T22:03:06.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50066d10 con 0x7f8f500a86b0 2021-11-21T22:03:06.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8f501475d0 0x7f8f50142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:06.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8f501475d0 0x7f8f50142e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54138/0 (socket says 172.21.2.116:54138) 2021-11-21T22:03:06.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 -- 172.21.2.116:0/218284572 learned_addr learned my addr 172.21.2.116:0/218284572 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:06.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 -- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a86b0 msgr2=0x7f8f50146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:06.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a86b0 0x7f8f50146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:06.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 -- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a6aa0 msgr2=0x7f8f500672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:06.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a6aa0 0x7f8f500672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:06.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 -- 172.21.2.116:0/218284572 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3800b180 con 0x7f8f501475d0 2021-11-21T22:03:06.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f54eac700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8f501475d0 0x7f8f50142e20 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f8f50061ab0 tx=0x7f8f480013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:06.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f4d7fa700 1 -- 172.21.2.116:0/218284572 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f48005ba0 con 0x7f8f501475d0 2021-11-21T22:03:06.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f4d7fa700 1 -- 172.21.2.116:0/218284572 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f48005d00 con 0x7f8f501475d0 2021-11-21T22:03:06.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f500b1db0 con 0x7f8f501475d0 2021-11-21T22:03:06.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f4d7fa700 1 -- 172.21.2.116:0/218284572 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f480072c0 con 0x7f8f501475d0 2021-11-21T22:03:06.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f501472f0 con 0x7f8f501475d0 2021-11-21T22:03:06.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f50146f00 con 0x7f8f501475d0 2021-11-21T22:03:06.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f4d7fa700 1 -- 172.21.2.116:0/218284572 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8f4801a060 con 0x7f8f501475d0 2021-11-21T22:03:06.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f4d7fa700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8f34053570 0x7f8f34055a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:06.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f4d7fa700 1 -- 172.21.2.116:0/218284572 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8f4806b3f0 con 0x7f8f501475d0 2021-11-21T22:03:06.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.771+0000 7f8f4ffff700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8f34053570 0x7f8f34055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:06.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.775+0000 7f8f4ffff700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8f34053570 0x7f8f34055a30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f8f4000c000 tx=0x7f8f40006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:06.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.775+0000 7f8f4d7fa700 1 -- 172.21.2.116:0/218284572 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f48042550 con 0x7f8f501475d0 2021-11-21T22:03:06.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.855+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8f50064b50 con 0x7f8f501475d0 2021-11-21T22:03:06.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.855+0000 7f8f4d7fa700 1 -- 172.21.2.116:0/218284572 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v152) v1 ==== 81+0+33659 (secure 0 0 0) 0x7f8f48018070 con 0x7f8f501475d0 2021-11-21T22:03:06.861 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:06.861 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:03:06.861 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 152, 2021-11-21T22:03:06.862 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:03:06.862 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:03:06.862 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:02:59.253897+0000", 2021-11-21T22:03:06.862 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:03:06.863 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:03:06.863 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:03:06.863 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:03:06.863 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:03:06.863 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:03:06.864 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:03:06.864 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:03:06.865 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:03:06.865 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:06.865 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:03:06.865 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:03:06.865 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:03:06.866 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:03:06.866 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:03:06.866 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 28, 2021-11-21T22:03:06.866 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:03:06.867 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:03:06.867 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:03:06.867 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:03:06.867 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:03:06.867 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.868 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:03:06.868 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:03:06.868 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:03:06.868 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:06.869 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:06.869 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:06.869 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:06.869 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:06.869 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:06.870 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:06.870 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:06.870 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:03:06.870 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:03:06.870 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:03:06.871 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:03:06.871 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:03:06.871 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:06.871 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:06.872 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:06.872 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:06.872 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:06.872 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:06.872 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:06.873 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.873 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:03:06.873 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:06.873 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:06.873 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:06.874 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:06.874 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:06.874 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:06.875 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:06.875 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:06.875 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:06.875 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:06.876 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:06.876 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:06.876 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:06.876 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:06.877 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:06.877 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:06.877 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:06.877 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:06.877 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:06.878 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:06.878 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:06.878 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:06.878 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:06.878 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:06.879 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:06.879 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:06.879 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.879 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:06.880 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:06.880 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:06.880 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:06.880 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:06.880 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:06.881 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:06.881 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:06.881 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:06.881 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:06.881 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:06.882 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:03:06.882 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:03:06.882 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.882 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:06.883 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:03:06.883 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.883 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.883 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.883 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:03:06.884 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:03:06.884 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:03:06.884 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:06.884 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:06.884 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:06.885 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:06.885 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:06.885 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:06.885 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:06.886 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:06.886 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:03:06.886 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:03:06.886 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:03:06.886 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:03:06.887 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:03:06.887 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:06.887 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:06.887 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:06.887 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:06.888 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:06.888 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:06.888 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:06.888 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.888 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:03:06.889 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:06.889 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:06.889 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:06.889 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:06.890 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:06.890 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:06.890 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:06.890 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:06.890 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:06.891 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:06.891 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:06.891 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:06.891 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:06.891 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:06.892 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:06.892 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:06.892 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:06.892 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:06.892 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:06.893 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:06.893 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:06.893 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:06.893 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:06.894 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:06.894 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:06.894 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:06.894 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.894 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:06.895 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:06.895 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:06.895 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:06.895 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:06.895 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:06.896 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:06.896 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:06.896 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:06.896 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:06.896 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:06.897 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:03:06.897 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:06.897 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:03:06.897 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.898 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.898 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.898 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 27, 2021-11-21T22:03:06.898 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:03:06.898 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:02:56.060658+0000", 2021-11-21T22:03:06.899 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:06.899 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:06.899 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:06.899 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:06.899 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:06.900 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:06.900 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:06.900 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:06.900 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:03:06.900 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:03:06.901 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:03:06.901 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:03:06.901 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:03:06.901 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:06.902 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:06.902 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:06.902 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:06.902 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:06.902 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:06.903 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:06.903 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.903 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "151", 2021-11-21T22:03:06.903 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:06.903 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:06.904 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:06.904 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:06.904 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:06.904 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:06.904 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:06.905 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:06.905 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:06.905 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:06.905 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:06.906 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:06.906 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:06.906 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:06.906 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:06.906 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:06.907 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:06.907 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:06.907 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:06.907 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:06.907 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:06.908 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:06.908 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:06.908 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:06.908 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:06.909 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:06.909 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.909 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:06.909 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:06.909 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:06.910 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:06.910 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:06.910 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:06.910 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:06.910 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:06.911 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:06.911 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:06.911 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:06.911 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:03:06.911 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:03:06.912 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:03:06.912 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:03:06.912 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.912 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:06.913 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:03:06.913 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:03:06.913 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.913 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.913 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.914 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.914 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 28, 2021-11-21T22:03:06.914 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:03:06.914 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:02:57.220549+0000", 2021-11-21T22:03:06.914 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:06.915 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:06.915 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:06.915 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:06.915 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:06.916 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:06.916 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:06.916 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:06.916 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:03:06.916 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:03:06.917 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:03:06.917 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:03:06.917 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:03:06.917 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:06.917 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:06.918 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:06.918 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:06.918 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:06.918 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:06.919 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:06.919 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.919 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "152", 2021-11-21T22:03:06.919 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:06.919 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:06.920 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:06.920 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:06.920 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:06.920 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:06.920 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:06.921 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:06.921 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:06.921 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:06.921 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:06.921 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:06.922 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:06.922 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:06.922 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:06.922 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:06.923 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:06.923 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:06.923 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:06.923 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:06.923 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:06.924 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:06.924 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:06.924 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:06.924 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:06.924 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:06.925 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.925 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:06.925 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:06.925 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:06.926 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:06.926 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:06.926 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:06.926 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:06.926 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:06.927 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:06.927 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:06.927 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:06.927 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:03:06.927 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:06.928 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:03:06.928 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:03:06.928 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.928 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.928 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.929 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:06.929 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:03:06.929 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.929 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:03:06.930 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:03:06.930 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:06.930 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:06.930 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:06.930 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:06.931 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:06.931 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:06.931 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:06.931 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:06.931 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:06.932 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:06.932 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:06.932 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.932 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.932 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.933 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:03:06.933 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.933 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.933 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.934 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.934 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:03:06.934 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.934 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.934 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.935 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.935 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:06.935 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.935 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.935 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.936 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:03:06.936 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.936 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.936 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.937 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.937 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:03:06.937 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.937 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.937 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.938 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.938 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:06.938 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.938 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.938 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.939 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:03:06.939 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.939 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.939 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.939 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.940 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:03:06.940 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.940 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.940 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.941 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.941 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:06.941 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.941 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.941 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.942 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:03:06.942 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.942 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.942 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.942 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.943 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:03:06.943 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:06.943 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.943 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.943 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.944 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:03:06.944 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:03:06.944 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:03:06.944 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:03:06.945 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:06.945 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:06.945 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:06.945 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.945 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.946 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.946 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:03:06.946 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:03:06.946 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:06.946 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:06.947 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:06.947 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:06.947 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:06.947 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:06.948 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:06.948 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:06.948 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:06.948 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:06.948 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:06.949 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.949 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.949 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.949 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:03:06.949 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.950 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.950 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.950 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.950 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:03:06.950 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.951 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.951 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.951 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.951 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:06.952 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.952 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.952 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.952 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:03:06.952 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.953 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.953 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.953 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.953 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:03:06.953 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.954 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.954 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.954 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.954 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:06.954 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.955 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.955 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.955 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:03:06.955 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.956 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.956 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.956 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.956 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:03:06.956 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.957 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.957 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.957 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.957 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:06.957 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.958 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.958 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.958 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:03:06.958 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.958 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.959 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.959 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.959 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:03:06.959 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:06.960 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.960 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.960 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.960 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:03:06.960 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:03:06.961 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:03:06.961 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:03:06.961 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:06.961 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:06.961 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:06.962 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.962 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.962 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.962 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:03:06.963 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:03:06.963 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:06.963 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:06.963 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:06.963 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:06.964 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:06.964 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:06.964 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:06.964 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 149, 2021-11-21T22:03:06.964 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:06.965 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:06.965 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:06.965 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.965 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.965 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.966 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:03:06.966 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.966 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.966 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.966 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.967 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:03:06.967 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.967 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.967 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.968 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.968 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:06.968 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.968 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.968 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.969 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:03:06.969 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.969 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.969 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.969 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.970 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:03:06.970 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.970 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.970 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.971 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.971 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:06.971 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.971 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.971 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.972 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:03:06.972 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.972 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.972 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.972 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.973 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:03:06.973 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.973 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.973 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.973 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.974 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:06.974 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.974 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.974 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.975 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:03:06.975 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.975 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.975 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.975 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.976 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:03:06.976 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:06.976 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.976 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.976 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.977 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:03:06.977 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:03:06.977 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:03:06.977 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:03:06.977 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:06.978 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:06.978 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:06.978 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.978 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.979 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.979 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:03:06.979 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:03:06.979 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:06.979 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:06.980 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:06.980 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:06.980 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:06.980 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:06.980 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:06.981 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 138, 2021-11-21T22:03:06.981 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:06.981 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:06.981 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:06.981 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.982 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.982 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.982 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:03:06.982 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.983 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.983 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.983 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.983 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:03:06.983 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.984 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.984 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.984 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.984 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:06.984 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.985 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.985 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.985 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:03:06.985 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.985 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.986 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.986 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.986 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:03:06.986 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.987 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.987 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.987 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.987 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:06.987 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.988 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.988 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.988 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:03:06.988 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.988 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.989 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.989 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.989 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:03:06.989 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.989 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.990 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.990 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.990 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:06.990 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.991 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.991 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.991 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:03:06.991 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.991 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.992 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.992 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:06.992 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:03:06.992 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:06.992 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:06.993 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.993 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.993 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:03:06.993 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:03:06.993 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:03:06.994 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:03:06.994 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:06.994 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:06.994 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:06.995 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:06.995 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.995 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.995 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:03:06.995 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:03:06.996 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:06.996 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:06.996 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:06.996 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:06.996 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:06.997 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:06.997 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:06.997 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 149, 2021-11-21T22:03:06.997 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:06.997 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:06.998 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:06.998 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:06.998 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.998 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:06.999 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:03:06.999 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:06.999 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:06.999 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:06.999 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.000 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:03:07.000 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:07.000 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.000 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.000 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.001 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:07.001 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.001 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.001 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.001 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:03:07.002 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:07.002 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.002 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.002 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.003 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:03:07.003 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:07.003 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.003 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.003 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.004 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:07.004 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.004 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.004 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.004 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:03:07.005 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:07.005 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.005 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.005 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.005 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:03:07.006 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:07.006 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.006 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.006 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.007 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:07.007 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.007 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.007 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.007 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:03:07.008 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:07.008 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.008 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.008 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.009 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:03:07.009 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:07.009 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.009 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.009 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.010 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:03:07.010 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:03:07.010 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:03:07.010 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:03:07.010 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:07.011 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:07.011 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:07.011 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.011 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.011 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.012 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:03:07.012 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:03:07.012 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:07.012 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:07.013 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:07.013 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:07.013 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:07.013 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:07.013 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:07.014 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:07.014 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:07.014 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:07.014 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:07.014 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.015 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.015 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.015 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:03:07.015 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.015 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.016 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.016 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.016 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:03:07.016 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.016 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.017 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.017 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.017 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:07.017 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.018 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.018 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.018 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:03:07.018 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.018 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.019 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.019 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.019 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:03:07.019 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.019 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.020 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.020 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.020 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:07.020 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.020 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.021 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.021 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:03:07.021 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.021 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.021 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.022 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.022 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:03:07.022 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.022 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.023 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.023 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.023 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:07.023 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.023 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.024 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.024 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:03:07.024 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.024 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.024 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.025 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.025 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:03:07.025 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:07.025 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.026 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.026 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.026 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:03:07.026 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:03:07.026 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:03:07.027 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:03:07.027 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:07.027 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:07.027 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:07.027 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.028 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.028 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.028 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:03:07.028 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:03:07.028 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:07.029 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:07.029 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:07.029 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:07.029 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:07.030 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:07.030 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:07.030 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:07.030 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:07.030 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:07.031 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:07.031 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.031 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.031 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.031 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:03:07.032 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.032 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.032 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.032 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.032 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:03:07.033 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.033 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.033 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.033 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.034 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:07.034 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.034 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.034 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.034 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:03:07.035 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.035 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.035 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.035 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.035 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:03:07.036 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.036 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.036 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.036 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.036 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:07.037 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.037 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.037 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.037 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:03:07.037 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.038 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.038 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.038 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.038 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:03:07.039 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.039 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.039 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.039 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.039 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:07.040 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.040 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.040 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.040 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:03:07.040 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.041 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.041 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.041 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.041 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:03:07.041 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:07.042 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.042 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.042 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.042 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:03:07.043 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:03:07.043 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:03:07.043 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:03:07.043 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:07.043 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:07.044 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:07.044 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.044 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.044 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.044 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:03:07.045 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:03:07.045 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:07.045 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:07.045 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:07.045 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:07.046 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:07.046 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:07.046 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:07.046 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:07.046 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:07.047 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:07.047 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:07.047 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.047 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.048 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.048 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:03:07.048 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.048 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.048 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.049 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.049 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:03:07.049 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.049 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.049 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.050 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.050 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:07.050 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.050 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.050 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.051 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:03:07.051 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.051 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.051 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.051 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.052 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:03:07.052 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.052 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.052 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.053 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.053 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:07.053 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.053 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.053 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.054 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:03:07.054 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.054 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.054 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.054 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.055 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:03:07.055 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.055 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.055 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.055 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.056 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:07.056 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:07.056 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.056 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:07.056 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:03:07.057 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.057 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.057 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.057 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:07.058 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:03:07.058 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:07.058 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.058 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.059 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.059 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:03:07.059 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:03:07.059 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:03:07.060 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:03:07.060 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:07.060 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:07.060 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:07.061 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:07.061 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.061 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:07.061 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:03:07.062 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.062 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:03:07.062 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.062 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.062 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.063 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.063 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.063 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:03:07.063 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.064 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.064 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.064 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:03:07.064 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.065 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.065 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.065 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.065 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.065 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:03:07.066 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.066 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.066 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.066 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:03:07.067 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.067 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.067 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.067 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.068 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.068 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:03:07.068 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.068 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.068 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.069 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:03:07.069 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.069 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.069 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.070 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.070 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.070 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:03:07.070 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.071 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.071 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.071 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:03:07.071 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.071 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.072 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.072 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.072 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.072 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:03:07.073 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.073 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.073 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.073 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:03:07.074 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.074 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.074 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.074 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.074 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.075 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:03:07.075 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.075 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.075 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.076 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:03:07.076 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.076 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.076 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.077 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.077 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.077 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:03:07.077 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.078 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.078 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:07.078 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:03:07.078 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:07.078 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:07.079 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:07.079 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:07.079 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:07.079 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:03:07.080 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:07.080 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.080 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:07.080 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:03:07.080 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:03:07.081 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:03:07.081 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:03:07.081 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T22:03:07.081 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:03:07.082 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:03:07.082 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:03:07.082 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:03:07.082 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:03:07.083 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:03:07.083 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:03:07.083 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:03:07.083 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:07.083 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:07.084 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:03:07.084 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:03:07.084 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:03:07.084 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:03:07.085 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:03:07.085 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:03:07.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8f34053570 msgr2=0x7f8f34055a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8f34053570 0x7f8f34055a30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f8f4000c000 tx=0x7f8f40006df0).stop 2021-11-21T22:03:07.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8f501475d0 msgr2=0x7f8f50142e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8f501475d0 0x7f8f50142e20 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f8f50061ab0 tx=0x7f8f480013a0).stop 2021-11-21T22:03:07.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 shutdown_connections 2021-11-21T22:03:07.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8f34053570 0x7f8f34055a30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8f501475d0 0x7f8f50142e20 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8f500a86b0 0x7f8f50146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:07.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 --2- 172.21.2.116:0/218284572 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8f500a6aa0 0x7f8f500672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:07.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 >> 172.21.2.116:0/218284572 conn(0x7f8f5005bd20 msgr2=0x7f8f50148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:07.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 shutdown_connections 2021-11-21T22:03:07.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:06.859+0000 7f8f5690f700 1 -- 172.21.2.116:0/218284572 wait complete. 2021-11-21T22:03:07.090 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:07.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a479d700 1 Processor -- start 2021-11-21T22:03:07.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a479d700 1 -- start start 2021-11-21T22:03:07.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a479d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0b24a0 0x7fa89c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a479d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa89c0aed90 0x7fa89c069e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a479d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c0b00d0 con 0x7fa89c0af360 2021-11-21T22:03:07.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a479d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89c0adba0 con 0x7fa89c0aed90 2021-11-21T22:03:07.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a479d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0af7a0 con 0x7fa89c0b24a0 2021-11-21T22:03:07.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a2539700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0b24a0 0x7fa89c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a1d38700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fa89c0af360 0x7fa89c0ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:36780/0 (socket says 172.21.2.116:36780) 2021-11-21T22:03:07.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a2d3a700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa89c0aed90 0x7fa89c069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a1d38700 1 -- 172.21.2.116:0/3923343602 learned_addr learned my addr 172.21.2.116:0/3923343602 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:07.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a2539700 1 -- 172.21.2.116:0/3923343602 >> v2:172.21.2.117:3300/0 conn(0x7fa89c0aed90 msgr2=0x7fa89c069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:07.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a2539700 1 --2- 172.21.2.116:0/3923343602 >> v2:172.21.2.117:3300/0 conn(0x7fa89c0aed90 0x7fa89c069e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a2539700 1 -- 172.21.2.116:0/3923343602 >> v1:172.21.2.117:6789/0 conn(0x7fa89c0af360 legacy=0x7fa89c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a2539700 1 -- 172.21.2.116:0/3923343602 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0b2880 con 0x7fa89c0b24a0 2021-11-21T22:03:07.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.143+0000 7fa8a2539700 1 --2- 172.21.2.116:0/3923343602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0b24a0 0x7fa89c0b1a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fa898002f70 tx=0x7fa898006df0).ready entity=mon.2 client_cookie=19299c0090dbb51b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:07.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1537700 1 -- 172.21.2.116:0/3923343602 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa89800d570 con 0x7fa89c0b24a0 2021-11-21T22:03:07.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1537700 1 -- 172.21.2.116:0/3923343602 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa898000e90 con 0x7fa89c0b24a0 2021-11-21T22:03:07.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1537700 1 -- 172.21.2.116:0/3923343602 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa89800d900 con 0x7fa89c0b24a0 2021-11-21T22:03:07.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- 172.21.2.116:0/3923343602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0b24a0 msgr2=0x7fa89c0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 --2- 172.21.2.116:0/3923343602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0b24a0 0x7fa89c0b1a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fa898002f70 tx=0x7fa898006df0).stop 2021-11-21T22:03:07.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- 172.21.2.116:0/3923343602 shutdown_connections 2021-11-21T22:03:07.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 --2- 172.21.2.116:0/3923343602 >> v2:172.21.2.117:3300/0 conn(0x7fa89c0aed90 0x7fa89c069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 --2- 172.21.2.116:0/3923343602 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0b24a0 0x7fa89c0b1a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- 172.21.2.116:0/3923343602 >> 172.21.2.116:0/3923343602 conn(0x7fa89c05fd00 msgr2=0x7fa89c0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:07.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- 172.21.2.116:0/3923343602 shutdown_connections 2021-11-21T22:03:07.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- 172.21.2.116:0/3923343602 wait complete. 2021-11-21T22:03:07.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 Processor -- start 2021-11-21T22:03:07.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- start start 2021-11-21T22:03:07.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0aed90 0x7fa89c13fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa89c0af360 0x7fa89c13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa89c0b24a0 0x7fa89c130f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0664b0 con 0x7fa89c0af360 2021-11-21T22:03:07.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c13d4d0 con 0x7fa89c0b24a0 2021-11-21T22:03:07.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c065ad0 con 0x7fa89c0aed90 2021-11-21T22:03:07.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa89c0af360 0x7fa89c13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa89c0af360 0x7fa89c13cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54148/0 (socket says 172.21.2.116:54148) 2021-11-21T22:03:07.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 -- 172.21.2.116:0/2966420210 learned_addr learned my addr 172.21.2.116:0/2966420210 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:07.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a2539700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0aed90 0x7fa89c13fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a2d3a700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa89c0b24a0 0x7fa89c130f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 -- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0aed90 msgr2=0x7fa89c13fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0aed90 0x7fa89c13fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 -- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa89c0b24a0 msgr2=0x7fa89c130f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa89c0b24a0 0x7fa89c130f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 -- 172.21.2.116:0/2966420210 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89800b180 con 0x7fa89c0af360 2021-11-21T22:03:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a1d38700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa89c0af360 0x7fa89c13cfe0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7fa89c0630d0 tx=0x7fa890007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa88f7fe700 1 -- 172.21.2.116:0/2966420210 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa89000ce30 con 0x7fa89c0af360 2021-11-21T22:03:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa88f7fe700 1 -- 172.21.2.116:0/2966420210 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8900072e0 con 0x7fa89c0af360 2021-11-21T22:03:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa88f7fe700 1 -- 172.21.2.116:0/2966420210 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa890005800 con 0x7fa89c0af360 2021-11-21T22:03:07.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c065d70 con 0x7fa89c0af360 2021-11-21T22:03:07.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.147+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89c065040 con 0x7fa89c0af360 2021-11-21T22:03:07.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.151+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89c0573d0 con 0x7fa89c0af360 2021-11-21T22:03:07.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.151+0000 7fa88f7fe700 1 -- 172.21.2.116:0/2966420210 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa890010070 con 0x7fa89c0af360 2021-11-21T22:03:07.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.151+0000 7fa88f7fe700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa87c0535d0 0x7fa87c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.151+0000 7fa88f7fe700 1 -- 172.21.2.116:0/2966420210 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa89006b130 con 0x7fa89c0af360 2021-11-21T22:03:07.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.151+0000 7fa8a2539700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa87c0535d0 0x7fa87c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.151+0000 7fa8a2539700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa87c0535d0 0x7fa87c055a90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fa898018000 tx=0x7fa898014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:07.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.151+0000 7fa88f7fe700 1 -- 172.21.2.116:0/2966420210 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa890042180 con 0x7fa89c0af360 2021-11-21T22:03:07.262 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa89c0658a0 con 0x7fa89c0af360 2021-11-21T22:03:07.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa88f7fe700 1 -- 172.21.2.116:0/2966420210 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fa8900457a0 con 0x7fa89c0af360 2021-11-21T22:03:07.264 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:07.264 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":70,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:58.250794+0000","modified":"2021-11-21T22:02:59.255376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/139139464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":139139464},{"type":"v1","addr":"172.21.2.116:6835","nonce":139139464}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-11-21T22:03:07.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa87c0535d0 msgr2=0x7fa87c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa87c0535d0 0x7fa87c055a90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fa898018000 tx=0x7fa898014000).stop 2021-11-21T22:03:07.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa89c0af360 msgr2=0x7fa89c13cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa89c0af360 0x7fa89c13cfe0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7fa89c0630d0 tx=0x7fa890007c40).stop 2021-11-21T22:03:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 shutdown_connections 2021-11-21T22:03:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa89c0b24a0 0x7fa89c130f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa87c0535d0 0x7fa87c055a90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa89c0af360 0x7fa89c13cfe0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 --2- 172.21.2.116:0/2966420210 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa89c0aed90 0x7fa89c13fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 >> 172.21.2.116:0/2966420210 conn(0x7fa89c05fd00 msgr2=0x7fa89c0b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:07.267 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 shutdown_connections 2021-11-21T22:03:07.268 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.259+0000 7fa8a479d700 1 -- 172.21.2.116:0/2966420210 wait complete. 2021-11-21T22:03:07.268 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 70 2021-11-21T22:03:07.271 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:03:07.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 Processor -- start 2021-11-21T22:03:07.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- start start 2021-11-21T22:03:07.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f74b41478c0 0x7f74b40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b4147e80 con 0x7f74b4144810 2021-11-21T22:03:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b40af5a0 con 0x7f74b40abce0 2021-11-21T22:03:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b40b1490 con 0x7f74b41478c0 2021-11-21T22:03:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b88b8700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f74b4144810 0x7f74b40b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37880/0 (socket says 172.21.2.116:37880) 2021-11-21T22:03:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b88b8700 1 -- 172.21.2.116:0/638620655 learned_addr learned my addr 172.21.2.116:0/638620655 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b90b9700 1 --2- 172.21.2.116:0/638620655 >> v2:172.21.2.116:3300/0 conn(0x7f74b41478c0 0x7f74b40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b90b9700 1 -- 172.21.2.116:0/638620655 >> v1:172.21.2.117:6789/0 conn(0x7f74b40abce0 legacy=0x7f74b40ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b90b9700 1 -- 172.21.2.116:0/638620655 >> v1:172.21.2.116:6789/0 conn(0x7f74b4144810 legacy=0x7f74b40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b90b9700 1 -- 172.21.2.116:0/638620655 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40b1fd0 con 0x7f74b41478c0 2021-11-21T22:03:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b90b9700 1 --2- 172.21.2.116:0/638620655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b41478c0 0x7f74b40acd10 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f74b40b4e00 tx=0x7f74ac0091d0).ready entity=mon.0 client_cookie=193b3c679b08c7f3 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b37fe700 1 -- 172.21.2.116:0/638620655 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ac009ea0 con 0x7f74b41478c0 2021-11-21T22:03:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b37fe700 1 -- 172.21.2.116:0/638620655 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac00b860 con 0x7f74b41478c0 2021-11-21T22:03:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74b37fe700 1 -- 172.21.2.116:0/638620655 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ac00b9d0 con 0x7f74b41478c0 2021-11-21T22:03:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- 172.21.2.116:0/638620655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b41478c0 msgr2=0x7f74b40acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 --2- 172.21.2.116:0/638620655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b41478c0 0x7f74b40acd10 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f74b40b4e00 tx=0x7f74ac0091d0).stop 2021-11-21T22:03:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- 172.21.2.116:0/638620655 shutdown_connections 2021-11-21T22:03:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 --2- 172.21.2.116:0/638620655 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b41478c0 0x7f74b40acd10 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.328 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- 172.21.2.116:0/638620655 >> 172.21.2.116:0/638620655 conn(0x7f74b405bd20 msgr2=0x7f74b405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:07.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- 172.21.2.116:0/638620655 shutdown_connections 2021-11-21T22:03:07.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- 172.21.2.116:0/638620655 wait complete. 2021-11-21T22:03:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 Processor -- start 2021-11-21T22:03:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.319+0000 7f74bab1c700 1 -- start start 2021-11-21T22:03:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b40abce0 0x7f74b40b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.332 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74b4144810 0x7f74b415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74b41478c0 0x7f74b415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4062f30 con 0x7f74b40abce0 2021-11-21T22:03:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40b0f80 con 0x7f74b4144810 2021-11-21T22:03:07.333 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4144ea0 con 0x7f74b41478c0 2021-11-21T22:03:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b40abce0 0x7f74b40b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b40abce0 0x7f74b40b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54166/0 (socket says 172.21.2.116:54166) 2021-11-21T22:03:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 -- 172.21.2.116:0/2632009521 learned_addr learned my addr 172.21.2.116:0/2632009521 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 -- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74b41478c0 msgr2=0x7f74b415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74b41478c0 0x7f74b415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:07.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 -- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74b4144810 msgr2=0x7f74b415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74b4144810 0x7f74b415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:07.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 -- 172.21.2.116:0/2632009521 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac009470 con 0x7f74b40abce0 2021-11-21T22:03:07.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b88b8700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b40abce0 0x7f74b40b0890 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f74b4062500 tx=0x7f74a4007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:07.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b1ffb700 1 -- 172.21.2.116:0/2632009521 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a400ce30 con 0x7f74b40abce0 2021-11-21T22:03:07.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b1ffb700 1 -- 172.21.2.116:0/2632009521 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a40072e0 con 0x7f74b40abce0 2021-11-21T22:03:07.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b1ffb700 1 -- 172.21.2.116:0/2632009521 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a4005800 con 0x7f74b40abce0 2021-11-21T22:03:07.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b4159df0 con 0x7f74b40abce0 2021-11-21T22:03:07.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b4145910 con 0x7f74b40abce0 2021-11-21T22:03:07.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b413fe60 con 0x7f74b40abce0 2021-11-21T22:03:07.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b1ffb700 1 -- 172.21.2.116:0/2632009521 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f74a4010070 con 0x7f74b40abce0 2021-11-21T22:03:07.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b1ffb700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74980534f0 0x7f74980559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:07.337 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b1ffb700 1 -- 172.21.2.116:0/2632009521 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f74a406aff0 con 0x7f74b40abce0 2021-11-21T22:03:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.323+0000 7f74b3fff700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74980534f0 0x7f74980559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.327+0000 7f74b3fff700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74980534f0 0x7f74980559b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f749c002f90 tx=0x7f749c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:07.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.327+0000 7f74b1ffb700 1 -- 172.21.2.116:0/2632009521 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74a4042150 con 0x7f74b40abce0 2021-11-21T22:03:07.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.435+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f74b4145060 con 0x7f74b40abce0 2021-11-21T22:03:07.940 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:03:07.935+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:03:07.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74b1ffb700 1 -- 172.21.2.116:0/2632009521 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7f74a4018030 con 0x7f74b40abce0 2021-11-21T22:03:07.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74980534f0 msgr2=0x7f74980559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74980534f0 0x7f74980559b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f749c002f90 tx=0x7f749c0069b0).stop 2021-11-21T22:03:07.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b40abce0 msgr2=0x7f74b40b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:07.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b40abce0 0x7f74b40b0890 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f74b4062500 tx=0x7f74a4007c00).stop 2021-11-21T22:03:07.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 shutdown_connections 2021-11-21T22:03:07.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74b41478c0 0x7f74b415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:07.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74b4144810 0x7f74b415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:07.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74980534f0 0x7f74980559b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 --2- 172.21.2.116:0/2632009521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74b40abce0 0x7f74b40b0890 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:07.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 >> 172.21.2.116:0/2632009521 conn(0x7f74b405bd20 msgr2=0x7f74b405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:07.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 shutdown_connections 2021-11-21T22:03:07.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:07.951+0000 7f74bab1c700 1 -- 172.21.2.116:0/2632009521 wait complete. 2021-11-21T22:03:07.964 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:03:07.967 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:07.968 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:03:08.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.011+0000 7f97b2887700 1 Processor -- start 2021-11-21T22:03:08.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.011+0000 7f97b2887700 1 -- start start 2021-11-21T22:03:08.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f97ac062410 0x7f97ac1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac0a6e80 con 0x7f97ac0a86b0 2021-11-21T22:03:08.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0a8bc0 con 0x7f97ac062410 2021-11-21T22:03:08.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac14a7b0 con 0x7f97ac0a6aa0 2021-11-21T22:03:08.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97ab7fe700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f97ac0a86b0 0x7f97ac1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:36818/0 (socket says 172.21.2.116:36818) 2021-11-21T22:03:08.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97ab7fe700 1 -- 172.21.2.116:0/4272673641 learned_addr learned my addr 172.21.2.116:0/4272673641 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b0e24700 1 --2- 172.21.2.116:0/4272673641 >> v2:172.21.2.117:3300/0 conn(0x7f97ac062410 0x7f97ac1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97abfff700 1 --2- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b0e24700 1 -- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 msgr2=0x7f97ac14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:08.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b0e24700 1 --2- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b0e24700 1 -- 172.21.2.116:0/4272673641 >> v1:172.21.2.117:6789/0 conn(0x7f97ac0a86b0 legacy=0x7f97ac1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b0e24700 1 -- 172.21.2.116:0/4272673641 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac14aa10 con 0x7f97ac062410 2021-11-21T22:03:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97abfff700 1 --2- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:03:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b0e24700 1 --2- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac062410 0x7f97ac1443f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f97a40032f0 tx=0x7f97a40091d0).ready entity=mon.1 client_cookie=212a7225ceaed4a8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:08.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97aaffd700 1 -- 172.21.2.116:0/4272673641 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a400b6c0 con 0x7f97ac062410 2021-11-21T22:03:08.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97aaffd700 1 -- 172.21.2.116:0/4272673641 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400b820 con 0x7f97ac062410 2021-11-21T22:03:08.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97aaffd700 1 -- 172.21.2.116:0/4272673641 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a400b990 con 0x7f97ac062410 2021-11-21T22:03:08.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 -- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac062410 msgr2=0x7f97ac1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 --2- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac062410 0x7f97ac1443f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f97a40032f0 tx=0x7f97a40091d0).stop 2021-11-21T22:03:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 -- 172.21.2.116:0/4272673641 shutdown_connections 2021-11-21T22:03:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.015+0000 7f97b2887700 1 --2- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac062410 0x7f97ac1443f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 --2- 172.21.2.116:0/4272673641 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- 172.21.2.116:0/4272673641 >> 172.21.2.116:0/4272673641 conn(0x7f97ac05bd20 msgr2=0x7f97ac05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:08.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- 172.21.2.116:0/4272673641 shutdown_connections 2021-11-21T22:03:08.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- 172.21.2.116:0/4272673641 wait complete. 2021-11-21T22:03:08.028 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 Processor -- start 2021-11-21T22:03:08.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- start start 2021-11-21T22:03:08.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f97ac062410 0x7f97ac14ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac064d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac0a86b0 0x7f97ac065210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac065700 con 0x7f97ac062410 2021-11-21T22:03:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac146760 con 0x7f97ac0a86b0 2021-11-21T22:03:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac146410 con 0x7f97ac0a6aa0 2021-11-21T22:03:08.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f97ac062410 0x7f97ac14ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f97ac062410 0x7f97ac14ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54190/0 (socket says 172.21.2.116:54190) 2021-11-21T22:03:08.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 -- 172.21.2.116:0/160560633 learned_addr learned my addr 172.21.2.116:0/160560633 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:08.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97ab7fe700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac064d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 -- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 msgr2=0x7f97ac064d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.031 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac064d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b0e24700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac0a86b0 0x7f97ac065210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 -- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac0a86b0 msgr2=0x7f97ac065210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac0a86b0 0x7f97ac065210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.032 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 -- 172.21.2.116:0/160560633 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c00b180 con 0x7f97ac062410 2021-11-21T22:03:08.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97abfff700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f97ac062410 0x7f97ac14ce20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7f979c00c000 tx=0x7f979c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:08.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97a97fa700 1 -- 172.21.2.116:0/160560633 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c00ca40 con 0x7f97ac062410 2021-11-21T22:03:08.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97a97fa700 1 -- 172.21.2.116:0/160560633 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00cba0 con 0x7f97ac062410 2021-11-21T22:03:08.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a4009470 con 0x7f97ac062410 2021-11-21T22:03:08.033 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac145a10 con 0x7f97ac062410 2021-11-21T22:03:08.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97a97fa700 1 -- 172.21.2.116:0/160560633 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c0123f0 con 0x7f97ac062410 2021-11-21T22:03:08.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac145e50 con 0x7f97ac062410 2021-11-21T22:03:08.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97a97fa700 1 -- 172.21.2.116:0/160560633 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f979c012550 con 0x7f97ac062410 2021-11-21T22:03:08.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97a97fa700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97900535d0 0x7f9790055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97a97fa700 1 -- 172.21.2.116:0/160560633 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f979c06c180 con 0x7f97ac062410 2021-11-21T22:03:08.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.019+0000 7f97ab7fe700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97900535d0 0x7f9790055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.023+0000 7f97ab7fe700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97900535d0 0x7f9790055a90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f979400b000 tx=0x7f9794009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:08.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.023+0000 7f97a97fa700 1 -- 172.21.2.116:0/160560633 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f979c043120 con 0x7f97ac062410 2021-11-21T22:03:08.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.131+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97ac0573d0 con 0x7f97ac062410 2021-11-21T22:03:08.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.131+0000 7f97a97fa700 1 -- 172.21.2.116:0/160560633 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f979c046740 con 0x7f97ac062410 2021-11-21T22:03:08.137 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:08.137 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":71,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:02:58.250794+0000","modified":"2021-11-21T22:03:07.940541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":153,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-11-21T22:03:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97900535d0 msgr2=0x7f9790055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97900535d0 0x7f9790055a90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f979400b000 tx=0x7f9794009040).stop 2021-11-21T22:03:08.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f97ac062410 msgr2=0x7f97ac14ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f97ac062410 0x7f97ac14ce20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7f979c00c000 tx=0x7f979c006df0).stop 2021-11-21T22:03:08.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 shutdown_connections 2021-11-21T22:03:08.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f97ac0a86b0 0x7f97ac065210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f97ac0a6aa0 0x7f97ac064d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f97900535d0 0x7f9790055a90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 --2- 172.21.2.116:0/160560633 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f97ac062410 0x7f97ac14ce20 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 >> 172.21.2.116:0/160560633 conn(0x7f97ac05bd20 msgr2=0x7f97ac149090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:08.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 shutdown_connections 2021-11-21T22:03:08.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.135+0000 7f97b2887700 1 -- 172.21.2.116:0/160560633 wait complete. 2021-11-21T22:03:08.141 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 71 2021-11-21T22:03:08.144 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:03:08.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 Processor -- start 2021-11-21T22:03:08.196 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 -- start start 2021-11-21T22:03:08.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb478144810 0x7fb4781478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb4780abce0 0x7fb4780b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.198 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb4780b1db0 0x7fb4780ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb478147e80 con 0x7fb478144810 2021-11-21T22:03:08.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780af5a0 con 0x7fb4780abce0 2021-11-21T22:03:08.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47fdc6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780b1490 con 0x7fb4780b1db0 2021-11-21T22:03:08.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47db62700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb478144810 0x7fb4781478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47db62700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb478144810 0x7fb4781478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54196/0 (socket says 172.21.2.116:54196) 2021-11-21T22:03:08.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.191+0000 7fb47db62700 1 -- 172.21.2.116:0/1468872950 learned_addr learned my addr 172.21.2.116:0/1468872950 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:08.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 -- 172.21.2.116:0/1468872950 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb4780b1db0 msgr2=0x7fb4780ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:08.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- 172.21.2.116:0/1468872950 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb4780b1db0 0x7fb4780ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:08.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 -- 172.21.2.116:0/1468872950 >> v2:172.21.2.117:3300/0 conn(0x7fb4780abce0 msgr2=0x7fb4780b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:08.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- 172.21.2.116:0/1468872950 >> v2:172.21.2.117:3300/0 conn(0x7fb4780abce0 0x7fb4780b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:08.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 -- 172.21.2.116:0/1468872950 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780af060 con 0x7fb478144810 2021-11-21T22:03:08.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- 172.21.2.116:0/1468872950 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb478144810 0x7fb4781478c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fb46c002f70 tx=0x7fb46c006eb0).ready entity=mon.0 client_cookie=6d7825a6a000ce29 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:08.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47cb60700 1 -- 172.21.2.116:0/1468872950 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46c00d570 con 0x7fb478144810 2021-11-21T22:03:08.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47cb60700 1 -- 172.21.2.116:0/1468872950 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c000e90 con 0x7fb478144810 2021-11-21T22:03:08.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47cb60700 1 -- 172.21.2.116:0/1468872950 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46c00d900 con 0x7fb478144810 2021-11-21T22:03:08.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/1468872950 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb478144810 msgr2=0x7fb4781478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/1468872950 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb478144810 0x7fb4781478c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fb46c002f70 tx=0x7fb46c006eb0).stop 2021-11-21T22:03:08.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/1468872950 shutdown_connections 2021-11-21T22:03:08.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/1468872950 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb4780b1db0 0x7fb4780ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:08.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/1468872950 >> v2:172.21.2.117:3300/0 conn(0x7fb4780abce0 0x7fb4780b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:08.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/1468872950 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb478144810 0x7fb4781478c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/1468872950 >> 172.21.2.116:0/1468872950 conn(0x7fb47805bd20 msgr2=0x7fb47805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:08.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/1468872950 shutdown_connections 2021-11-21T22:03:08.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/1468872950 wait complete. 2021-11-21T22:03:08.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 Processor -- start 2021-11-21T22:03:08.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- start start 2021-11-21T22:03:08.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb4780abce0 0x7fb47813f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb478144810 0x7fb478146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb478144d20 0x7fb478146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47815a6a0 con 0x7fb4780abce0 2021-11-21T22:03:08.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478145570 con 0x7fb478144810 2021-11-21T22:03:08.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4781452d0 con 0x7fb478144d20 2021-11-21T22:03:08.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb4780abce0 0x7fb47813f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb4780abce0 0x7fb47813f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54202/0 (socket says 172.21.2.116:54202) 2021-11-21T22:03:08.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 -- 172.21.2.116:0/3469107136 learned_addr learned my addr 172.21.2.116:0/3469107136 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:08.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47e363700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb478144d20 0x7fb478146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 -- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb478144d20 msgr2=0x7fb478146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47d361700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb478144810 0x7fb478146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb478144d20 0x7fb478146a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 -- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb478144810 msgr2=0x7fb478146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb478144810 0x7fb478146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 -- 172.21.2.116:0/3469107136 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c00b1e0 con 0x7fb4780abce0 2021-11-21T22:03:08.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47d361700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb478144810 0x7fb478146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:08.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47e363700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb478144d20 0x7fb478146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:08.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47db62700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb4780abce0 0x7fb47813f640 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fb46c002f70 tx=0x7fb46c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:08.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb46affd700 1 -- 172.21.2.116:0/3469107136 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46c011990 con 0x7fb4780abce0 2021-11-21T22:03:08.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb46affd700 1 -- 172.21.2.116:0/3469107136 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c011af0 con 0x7fb4780abce0 2021-11-21T22:03:08.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb46affd700 1 -- 172.21.2.116:0/3469107136 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46c011e50 con 0x7fb4780abce0 2021-11-21T22:03:08.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47815a900 con 0x7fb4780abce0 2021-11-21T22:03:08.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4780b4950 con 0x7fb4780abce0 2021-11-21T22:03:08.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb478146330 con 0x7fb4780abce0 2021-11-21T22:03:08.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.195+0000 7fb46affd700 1 -- 172.21.2.116:0/3469107136 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb46c01a040 con 0x7fb4780abce0 2021-11-21T22:03:08.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.199+0000 7fb46affd700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4580536a0 0x7fb458055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:08.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.199+0000 7fb46affd700 1 -- 172.21.2.116:0/3469107136 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fb46c06c990 con 0x7fb4780abce0 2021-11-21T22:03:08.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.199+0000 7fb47d361700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4580536a0 0x7fb458055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:08.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.199+0000 7fb47d361700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4580536a0 0x7fb458055b60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fb474007000 tx=0x7fb474008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:08.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.199+0000 7fb46affd700 1 -- 172.21.2.116:0/3469107136 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb46c043aa0 con 0x7fb4780abce0 2021-11-21T22:03:08.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.311+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb4780af300 con 0x7fb4780abce0 2021-11-21T22:03:08.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb46affd700 1 -- 172.21.2.116:0/3469107136 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb46c0470c0 con 0x7fb4780abce0 2021-11-21T22:03:08.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4580536a0 msgr2=0x7fb458055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4580536a0 0x7fb458055b60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fb474007000 tx=0x7fb474008040).stop 2021-11-21T22:03:08.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb4780abce0 msgr2=0x7fb47813f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:08.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb4780abce0 0x7fb47813f640 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fb46c002f70 tx=0x7fb46c015000).stop 2021-11-21T22:03:08.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 shutdown_connections 2021-11-21T22:03:08.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb478144d20 0x7fb478146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb478144810 0x7fb478146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb4580536a0 0x7fb458055b60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 --2- 172.21.2.116:0/3469107136 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb4780abce0 0x7fb47813f640 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:08.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 >> 172.21.2.116:0/3469107136 conn(0x7fb47805bd20 msgr2=0x7fb478148660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:08.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 shutdown_connections 2021-11-21T22:03:08.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:08.963+0000 7fb47fdc6700 1 -- 172.21.2.116:0/3469107136 wait complete. 2021-11-21T22:03:08.975 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:03:09.029 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 Processor -- start 2021-11-21T22:03:09.030 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- start start 2021-11-21T22:03:09.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 0x7fe5280b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:09.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe528060ca0 0x7fe52809a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:09.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52809ab10 con 0x7fe5280b5610 2021-11-21T22:03:09.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe528142b20 con 0x7fe528060ca0 2021-11-21T22:03:09.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe528142dc0 con 0x7fe52805ced0 2021-11-21T22:03:09.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe528060ca0 0x7fe52809a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:09.035 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe528060ca0 0x7fe52809a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54210/0 (socket says 172.21.2.116:54210) 2021-11-21T22:03:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 -- 172.21.2.116:0/1594021317 learned_addr learned my addr 172.21.2.116:0/1594021317 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe527fff700 1 --2- 172.21.2.116:0/1594021317 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 0x7fe5280b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 -- 172.21.2.116:0/1594021317 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 msgr2=0x7fe5280b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:09.036 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 --2- 172.21.2.116:0/1594021317 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 0x7fe5280b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 -- 172.21.2.116:0/1594021317 >> v1:172.21.2.117:6789/0 conn(0x7fe5280b5610 legacy=0x7fe528099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 -- 172.21.2.116:0/1594021317 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528143060 con 0x7fe528060ca0 2021-11-21T22:03:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 --2- 172.21.2.116:0/1594021317 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe528060ca0 0x7fe52809a620 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7fe528143920 tx=0x7fe518006eb0).ready entity=mon.0 client_cookie=720583d6c810de5a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:09.037 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe526ffd700 1 -- 172.21.2.116:0/1594021317 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe51800d570 con 0x7fe528060ca0 2021-11-21T22:03:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe526ffd700 1 -- 172.21.2.116:0/1594021317 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe518000e90 con 0x7fe528060ca0 2021-11-21T22:03:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe526ffd700 1 -- 172.21.2.116:0/1594021317 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe51800d8c0 con 0x7fe528060ca0 2021-11-21T22:03:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- 172.21.2.116:0/1594021317 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe528060ca0 msgr2=0x7fe52809a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:09.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- 172.21.2.116:0/1594021317 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe528060ca0 0x7fe52809a620 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7fe528143920 tx=0x7fe518006eb0).stop 2021-11-21T22:03:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- 172.21.2.116:0/1594021317 shutdown_connections 2021-11-21T22:03:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- 172.21.2.116:0/1594021317 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe528060ca0 0x7fe52809a620 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- 172.21.2.116:0/1594021317 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 0x7fe5280b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- 172.21.2.116:0/1594021317 >> 172.21.2.116:0/1594021317 conn(0x7fe5280b9da0 msgr2=0x7fe5280b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:09.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- 172.21.2.116:0/1594021317 shutdown_connections 2021-11-21T22:03:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- 172.21.2.116:0/1594021317 wait complete. 2021-11-21T22:03:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 Processor -- start 2021-11-21T22:03:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- start start 2021-11-21T22:03:09.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 0x7fe52809b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:09.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe528060ca0 0x7fe5280aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:09.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5280b5610 0x7fe5280af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:09.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281448c0 con 0x7fe5280b5610 2021-11-21T22:03:09.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52809e710 con 0x7fe528060ca0 2021-11-21T22:03:09.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52e573700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52809e470 con 0x7fe52805ced0 2021-11-21T22:03:09.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5280b5610 0x7fe5280af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:09.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5280b5610 0x7fe5280af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54218/0 (socket says 172.21.2.116:54218) 2021-11-21T22:03:09.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.027+0000 7fe52cb10700 1 -- 172.21.2.116:0/2184180490 learned_addr learned my addr 172.21.2.116:0/2184180490 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:09.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52cb10700 1 -- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 msgr2=0x7fe52809b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:09.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52cb10700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 0x7fe52809b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:09.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52cb10700 1 -- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe528060ca0 msgr2=0x7fe5280aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:09.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5277fe700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe528060ca0 0x7fe5280aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:09.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52cb10700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe528060ca0 0x7fe5280aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:09.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52cb10700 1 -- 172.21.2.116:0/2184180490 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51800b1e0 con 0x7fe5280b5610 2021-11-21T22:03:09.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5277fe700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe528060ca0 0x7fe5280aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:09.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52cb10700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5280b5610 0x7fe5280af280 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7fe528143920 tx=0x7fe518015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:09.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5257fa700 1 -- 172.21.2.116:0/2184180490 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe518002f70 con 0x7fe5280b5610 2021-11-21T22:03:09.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5257fa700 1 -- 172.21.2.116:0/2184180490 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51800bec0 con 0x7fe5280b5610 2021-11-21T22:03:09.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe528144b20 con 0x7fe5280b5610 2021-11-21T22:03:09.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52809dba0 con 0x7fe5280b5610 2021-11-21T22:03:09.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5257fa700 1 -- 172.21.2.116:0/2184180490 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe51800d8f0 con 0x7fe5280b5610 2021-11-21T22:03:09.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528143300 con 0x7fe5280b5610 2021-11-21T22:03:09.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5257fa700 1 -- 172.21.2.116:0/2184180490 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe518025030 con 0x7fe5280b5610 2021-11-21T22:03:09.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5257fa700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5080534f0 0x7fe5080559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:09.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe5257fa700 1 -- 172.21.2.116:0/2184180490 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fe51806cd50 con 0x7fe5280b5610 2021-11-21T22:03:09.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe527fff700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5080534f0 0x7fe5080559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:09.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.031+0000 7fe527fff700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5080534f0 0x7fe5080559b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fe51c00a000 tx=0x7fe51c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:09.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.035+0000 7fe5257fa700 1 -- 172.21.2.116:0/2184180490 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe518043d50 con 0x7fe5280b5610 2021-11-21T22:03:09.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.115+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5280573d0 con 0x7fe5280b5610 2021-11-21T22:03:09.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe5257fa700 1 -- 172.21.2.116:0/2184180490 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v154) v1 ==== 185+0+0 (secure 0 0 0) 0x7fe518023020 con 0x7fe5280b5610 2021-11-21T22:03:09.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5080534f0 msgr2=0x7fe5080559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:09.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5080534f0 0x7fe5080559b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fe51c00a000 tx=0x7fe51c008040).stop 2021-11-21T22:03:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5280b5610 msgr2=0x7fe5280af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5280b5610 0x7fe5280af280 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7fe528143920 tx=0x7fe518015000).stop 2021-11-21T22:03:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 shutdown_connections 2021-11-21T22:03:09.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe5080534f0 0x7fe5080559b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe5280b5610 0x7fe5280af280 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe528060ca0 0x7fe5280aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 --2- 172.21.2.116:0/2184180490 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe52805ced0 0x7fe52809b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 >> 172.21.2.116:0/2184180490 conn(0x7fe5280b9da0 msgr2=0x7fe52805bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:09.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 shutdown_connections 2021-11-21T22:03:09.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:09.971+0000 7fe52e573700 1 -- 172.21.2.116:0/2184180490 wait complete. 2021-11-21T22:03:09.978 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:03:09.982 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:03:10.038 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1da3dd700 1 Processor -- start 2021-11-21T22:03:10.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1da3dd700 1 -- start start 2021-11-21T22:03:10.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1da3dd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1d40a8670 0x7fc1d4147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:10.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1da3dd700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fc1d4143e60 0x7fc1d41442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:10.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1da3dd700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d40b1940 con 0x7fc1d40b11c0 2021-11-21T22:03:10.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1da3dd700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d40b15a0 con 0x7fc1d40a8670 2021-11-21T22:03:10.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1da3dd700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d40b1b70 con 0x7fc1d4143e60 2021-11-21T22:03:10.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1d37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1d40a8670 0x7fc1d4147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:10.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1d37fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc1d40a8670 0x7fc1d4147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54224/0 (socket says 172.21.2.116:54224) 2021-11-21T22:03:10.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1d37fe700 1 -- 172.21.2.116:0/1573335911 learned_addr learned my addr 172.21.2.116:0/1573335911 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:10.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1d37fe700 1 -- 172.21.2.116:0/1573335911 >> v2:172.21.2.117:3300/0 conn(0x7fc1d4143e60 msgr2=0x7fc1d41442d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:10.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1d37fe700 1 --2- 172.21.2.116:0/1573335911 >> v2:172.21.2.117:3300/0 conn(0x7fc1d4143e60 0x7fc1d41442d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:10.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.031+0000 7fc1d37fe700 1 -- 172.21.2.116:0/1573335911 >> v1:172.21.2.117:6789/0 conn(0x7fc1d40b11c0 legacy=0x7fc1d4147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:10.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d37fe700 1 -- 172.21.2.116:0/1573335911 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d40b1e10 con 0x7fc1d40a8670 2021-11-21T22:03:10.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d37fe700 1 --2- 172.21.2.116:0/1573335911 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d4147940 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7fc1d4147180 tx=0x7fc1c4006eb0).ready entity=mon.0 client_cookie=3d90192b88894b85 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:10.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d2ffd700 1 -- 172.21.2.116:0/1573335911 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c400d5c0 con 0x7fc1d40a8670 2021-11-21T22:03:10.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d2ffd700 1 -- 172.21.2.116:0/1573335911 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c4000e90 con 0x7fc1d40a8670 2021-11-21T22:03:10.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d2ffd700 1 -- 172.21.2.116:0/1573335911 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c400d910 con 0x7fc1d40a8670 2021-11-21T22:03:10.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1573335911 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 msgr2=0x7fc1d4147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:10.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1573335911 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d4147940 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7fc1d4147180 tx=0x7fc1c4006eb0).stop 2021-11-21T22:03:10.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1573335911 shutdown_connections 2021-11-21T22:03:10.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1573335911 >> v2:172.21.2.117:3300/0 conn(0x7fc1d4143e60 0x7fc1d41442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:10.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1573335911 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d4147940 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:10.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1573335911 >> 172.21.2.116:0/1573335911 conn(0x7fc1d405bd20 msgr2=0x7fc1d405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:10.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1573335911 shutdown_connections 2021-11-21T22:03:10.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1573335911 wait complete. 2021-11-21T22:03:10.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 Processor -- start 2021-11-21T22:03:10.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- start start 2021-11-21T22:03:10.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d414a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:10.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1d40b11c0 0x7fc1d4066960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:10.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1d4143e60 0x7fc1d4066e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:10.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d405fe00 con 0x7fc1d40a8670 2021-11-21T22:03:10.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d40b4590 con 0x7fc1d4143e60 2021-11-21T22:03:10.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d40b5cd0 con 0x7fc1d40b11c0 2021-11-21T22:03:10.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d414a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:10.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d414a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54228/0 (socket says 172.21.2.116:54228) 2021-11-21T22:03:10.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 -- 172.21.2.116:0/1067634745 learned_addr learned my addr 172.21.2.116:0/1067634745 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:10.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 -- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1d40b11c0 msgr2=0x7fc1d4066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:10.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1d40b11c0 0x7fc1d4066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:10.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 -- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1d4143e60 msgr2=0x7fc1d4066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:10.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1d4143e60 0x7fc1d4066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:10.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 -- 172.21.2.116:0/1067634745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c400b1e0 con 0x7fc1d40a8670 2021-11-21T22:03:10.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d3fff700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d414a7c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7fc1bc002f90 tx=0x7fc1bc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:10.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d17fa700 1 -- 172.21.2.116:0/1067634745 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1bc013460 con 0x7fc1d40a8670 2021-11-21T22:03:10.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d17fa700 1 -- 172.21.2.116:0/1067634745 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0135c0 con 0x7fc1d40a8670 2021-11-21T22:03:10.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d17fa700 1 -- 172.21.2.116:0/1067634745 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1bc013920 con 0x7fc1d40a8670 2021-11-21T22:03:10.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d40b47f0 con 0x7fc1d40a8670 2021-11-21T22:03:10.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d40b5950 con 0x7fc1d40a8670 2021-11-21T22:03:10.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d40b20b0 con 0x7fc1d40a8670 2021-11-21T22:03:10.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d17fa700 1 -- 172.21.2.116:0/1067634745 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc1bc01a060 con 0x7fc1d40a8670 2021-11-21T22:03:10.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.035+0000 7fc1d17fa700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b8053600 0x7fc1b8055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:10.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.039+0000 7fc1d17fa700 1 -- 172.21.2.116:0/1067634745 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fc1bc06b2c0 con 0x7fc1d40a8670 2021-11-21T22:03:10.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.039+0000 7fc1d37fe700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b8053600 0x7fc1b8055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:10.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.039+0000 7fc1d37fe700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b8053600 0x7fc1b8055ac0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fc1d4147180 tx=0x7fc1c4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:10.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.039+0000 7fc1d17fa700 1 -- 172.21.2.116:0/1067634745 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc1bc042470 con 0x7fc1d40a8670 2021-11-21T22:03:10.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.123+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1d40573d0 con 0x7fc1d40a8670 2021-11-21T22:03:10.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1d17fa700 1 -- 172.21.2.116:0/1067634745 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v155) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc1bc018030 con 0x7fc1d40a8670 2021-11-21T22:03:10.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b8053600 msgr2=0x7fc1b8055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:10.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b8053600 0x7fc1b8055ac0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fc1d4147180 tx=0x7fc1c4015000).stop 2021-11-21T22:03:10.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 msgr2=0x7fc1d414a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:10.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d414a7c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7fc1bc002f90 tx=0x7fc1bc006df0).stop 2021-11-21T22:03:10.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 shutdown_connections 2021-11-21T22:03:10.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc1d4143e60 0x7fc1d4066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:10.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc1d40b11c0 0x7fc1d4066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:10.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc1b8053600 0x7fc1b8055ac0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:10.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.975+0000 7fc1da3dd700 1 --2- 172.21.2.116:0/1067634745 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc1d40a8670 0x7fc1d414a7c0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:10.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.979+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 >> 172.21.2.116:0/1067634745 conn(0x7fc1d405bd20 msgr2=0x7fc1d413ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:10.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.979+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 shutdown_connections 2021-11-21T22:03:10.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:10.979+0000 7fc1da3dd700 1 -- 172.21.2.116:0/1067634745 wait complete. 2021-11-21T22:03:10.984 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:03:10.989 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-11-21T22:03:11.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9ba2225700 1 Processor -- start 2021-11-21T22:03:11.040 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9ba2225700 1 -- start start 2021-11-21T22:03:11.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9ba2225700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9b9c0b02e0 0x7f9b9c05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9ba2225700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0afab0 0x7f9b9c143680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9ba2225700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c0b06c0 con 0x7f9b9c0af620 2021-11-21T22:03:11.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9ba2225700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c0adf20 con 0x7f9b9c0b02e0 2021-11-21T22:03:11.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9ba2225700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0b2470 con 0x7f9b9c0afab0 2021-11-21T22:03:11.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9b9affd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9b9c0af620 0x7f9b9c142f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37962/0 (socket says 172.21.2.116:37962) 2021-11-21T22:03:11.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.035+0000 7f9b9affd700 1 -- 172.21.2.116:0/1407487489 learned_addr learned my addr 172.21.2.116:0/1407487489 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:11.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757381081 0 0) 0x7f9b9c0b06c0 con 0x7f9b9c0af620 2021-11-21T22:03:11.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b7c003260 con 0x7f9b9c0af620 2021-11-21T22:03:11.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0afab0 0x7f9b9c143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 --2- 172.21.2.116:0/1407487489 >> v2:172.21.2.117:3300/0 conn(0x7f9b9c0b02e0 0x7f9b9c05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707689269 0 0) 0x7f9b7c003260 con 0x7f9b9c0af620 2021-11-21T22:03:11.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0afab0 msgr2=0x7f9b9c143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:11.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0afab0 0x7f9b9c143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 >> v2:172.21.2.117:3300/0 conn(0x7f9b9c0b02e0 msgr2=0x7f9b9c05f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:11.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 --2- 172.21.2.116:0/1407487489 >> v2:172.21.2.117:3300/0 conn(0x7f9b9c0b02e0 0x7f9b9c05f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0b26d0 con 0x7f9b9c0af620 2021-11-21T22:03:11.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f9b84001680 con 0x7f9b9c0af620 2021-11-21T22:03:11.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 >> v1:172.21.2.116:6789/0 conn(0x7f9b9c0af620 legacy=0x7f9b9c142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b7c0037d0 0x7f9b7c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0afab0 0x7f9b7c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b7c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0adf20 con 0x7f9b7c004b10 2021-11-21T22:03:11.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0b2470 con 0x7f9b7c0037d0 2021-11-21T22:03:11.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c00db70 con 0x7f9b9c0afab0 2021-11-21T22:03:11.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b7c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b7c0037d0 0x7f9b7c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 -- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0afab0 msgr2=0x7f9b7c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:11.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0afab0 0x7f9b7c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 -- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b7c0037d0 msgr2=0x7f9b7c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:11.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b7c0037d0 0x7f9b7c003bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 -- 172.21.2.116:0/1407487489 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c003260 con 0x7f9b7c004b10 2021-11-21T22:03:11.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b7c004f90 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f9b9400a000 tx=0x7f9b94008040).ready entity=mon.0 client_cookie=d7588ed910b5c31b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:11.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9400c070 con 0x7f9b7c004b10 2021-11-21T22:03:11.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9400e5f0 con 0x7f9b7c004b10 2021-11-21T22:03:11.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9a7fc700 1 -- 172.21.2.116:0/1407487489 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9400e950 con 0x7f9b7c004b10 2021-11-21T22:03:11.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- 172.21.2.116:0/1407487489 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 msgr2=0x7f9b7c004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 --2- 172.21.2.116:0/1407487489 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b7c004f90 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f9b9400a000 tx=0x7f9b94008040).stop 2021-11-21T22:03:11.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9bfff700 1 -- 172.21.2.116:0/1407487489 reap_dead start 2021-11-21T22:03:11.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- 172.21.2.116:0/1407487489 shutdown_connections 2021-11-21T22:03:11.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- 172.21.2.116:0/1407487489 >> 172.21.2.116:0/1407487489 conn(0x7f9b9c0b5ff0 msgr2=0x7f9b9c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:11.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- 172.21.2.116:0/1407487489 shutdown_connections 2021-11-21T22:03:11.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- 172.21.2.116:0/1407487489 wait complete. 2021-11-21T22:03:11.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 Processor -- start 2021-11-21T22:03:11.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- start start 2021-11-21T22:03:11.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b9c0603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b9c09f200 0x7f9b9c0ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0ab720 0x7f9b9c142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c09e1f0 con 0x7f9b7c004b10 2021-11-21T22:03:11.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c09e490 con 0x7f9b9c09f200 2021-11-21T22:03:11.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c09e730 con 0x7f9b9c0ab720 2021-11-21T22:03:11.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b9c0603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b9c0603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54248/0 (socket says 172.21.2.116:54248) 2021-11-21T22:03:11.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 -- 172.21.2.116:0/2742859437 learned_addr learned my addr 172.21.2.116:0/2742859437 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:11.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 -- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0ab720 msgr2=0x7f9b9c142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0ab720 0x7f9b9c142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 -- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b9c09f200 msgr2=0x7f9b9c0ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b9c09f200 0x7f9b9c0ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 -- 172.21.2.116:0/2742859437 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0a2470 con 0x7f9b7c004b10 2021-11-21T22:03:11.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b9b7fe700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b9c0603b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 rx=0x7f9b8c008000 tx=0x7f9b8c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:11.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b98ff9700 1 -- 172.21.2.116:0/2742859437 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8c010550 con 0x7f9b7c004b10 2021-11-21T22:03:11.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b98ff9700 1 -- 172.21.2.116:0/2742859437 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c000e90 con 0x7f9b7c004b10 2021-11-21T22:03:11.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c14db80 con 0x7f9b7c004b10 2021-11-21T22:03:11.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9b98ff9700 1 -- 172.21.2.116:0/2742859437 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8c0106b0 con 0x7f9b7c004b10 2021-11-21T22:03:11.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.039+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c0abba0 con 0x7f9b7c004b10 2021-11-21T22:03:11.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.043+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c09e8f0 con 0x7f9b7c004b10 2021-11-21T22:03:11.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.043+0000 7f9b98ff9700 1 -- 172.21.2.116:0/2742859437 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9b8c018020 con 0x7f9b7c004b10 2021-11-21T22:03:11.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.043+0000 7f9b98ff9700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9b80053640 0x7f9b80055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.043+0000 7f9b98ff9700 1 -- 172.21.2.116:0/2742859437 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9b8c06ac70 con 0x7f9b7c004b10 2021-11-21T22:03:11.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.043+0000 7f9b9affd700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9b80053640 0x7f9b80055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.043+0000 7f9b9affd700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9b80053640 0x7f9b80055b00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f9b84001390 tx=0x7f9b9c0b7000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:11.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.043+0000 7f9b98ff9700 1 -- 172.21.2.116:0/2742859437 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b8c0420b0 con 0x7f9b7c004b10 2021-11-21T22:03:11.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.127+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9b9c0b0da0 con 0x7f9b7c004b10 2021-11-21T22:03:11.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9b98ff9700 1 -- 172.21.2.116:0/2742859437 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9b8c0456d0 con 0x7f9b7c004b10 2021-11-21T22:03:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9b80053640 msgr2=0x7f9b80055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9b80053640 0x7f9b80055b00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f9b84001390 tx=0x7f9b9c0b7000).stop 2021-11-21T22:03:11.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 msgr2=0x7f9b9c0603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b9c0603b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 rx=0x7f9b8c008000 tx=0x7f9b8c008040).stop 2021-11-21T22:03:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 shutdown_connections 2021-11-21T22:03:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9b9c0ab720 0x7f9b9c142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9b9c09f200 0x7f9b9c0ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9b80053640 0x7f9b80055b00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 --2- 172.21.2.116:0/2742859437 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9b7c004b10 0x7f9b9c0603b0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 >> 172.21.2.116:0/2742859437 conn(0x7f9b9c0b5ff0 msgr2=0x7f9b9c0b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:11.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 shutdown_connections 2021-11-21T22:03:11.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.135+0000 7f9ba2225700 1 -- 172.21.2.116:0/2742859437 wait complete. 2021-11-21T22:03:11.145 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2021-11-21T22:03:11.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 Processor -- start 2021-11-21T22:03:11.197 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- start start 2021-11-21T22:03:11.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f8cc8060740 0x7f8cc8060b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.199 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc80b1a50 0x7f8cc8140e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8cc8065b70 0x7f8cc8141310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc8141800 con 0x7f8cc8065b70 2021-11-21T22:03:11.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc8144410 con 0x7f8cc8060740 2021-11-21T22:03:11.200 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81446b0 con 0x7f8cc80b1a50 2021-11-21T22:03:11.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8cc8065b70 0x7f8cc8141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8cc8065b70 0x7f8cc8141310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54256/0 (socket says 172.21.2.116:54256) 2021-11-21T22:03:11.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 -- 172.21.2.116:0/2278530848 learned_addr learned my addr 172.21.2.116:0/2278530848 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:11.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 -- 172.21.2.116:0/2278530848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc80b1a50 msgr2=0x7f8cc8140e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:11.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 --2- 172.21.2.116:0/2278530848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc80b1a50 0x7f8cc8140e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 -- 172.21.2.116:0/2278530848 >> v2:172.21.2.117:3300/0 conn(0x7f8cc8060740 msgr2=0x7f8cc8060b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:11.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 --2- 172.21.2.116:0/2278530848 >> v2:172.21.2.117:3300/0 conn(0x7f8cc8060740 0x7f8cc8060b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 -- 172.21.2.116:0/2278530848 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8144950 con 0x7f8cc8065b70 2021-11-21T22:03:11.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc77fe700 1 --2- 172.21.2.116:0/2278530848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc8141310 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f8cb80032f0 tx=0x7f8cb8009290).ready entity=mon.0 client_cookie=629fef1859961258 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:11.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc5ffb700 1 -- 172.21.2.116:0/2278530848 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cb800b870 con 0x7f8cc8065b70 2021-11-21T22:03:11.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc5ffb700 1 -- 172.21.2.116:0/2278530848 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800b9d0 con 0x7f8cc8065b70 2021-11-21T22:03:11.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc5ffb700 1 -- 172.21.2.116:0/2278530848 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cb800bb80 con 0x7f8cc8065b70 2021-11-21T22:03:11.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- 172.21.2.116:0/2278530848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 msgr2=0x7f8cc8141310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- 172.21.2.116:0/2278530848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc8141310 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f8cb80032f0 tx=0x7f8cb8009290).stop 2021-11-21T22:03:11.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- 172.21.2.116:0/2278530848 shutdown_connections 2021-11-21T22:03:11.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- 172.21.2.116:0/2278530848 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc8141310 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- 172.21.2.116:0/2278530848 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc80b1a50 0x7f8cc8140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- 172.21.2.116:0/2278530848 >> v2:172.21.2.117:3300/0 conn(0x7f8cc8060740 0x7f8cc8060b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- 172.21.2.116:0/2278530848 >> 172.21.2.116:0/2278530848 conn(0x7f8cc80b7b80 msgr2=0x7f8cc80b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:11.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- 172.21.2.116:0/2278530848 shutdown_connections 2021-11-21T22:03:11.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- 172.21.2.116:0/2278530848 wait complete. 2021-11-21T22:03:11.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 Processor -- start 2021-11-21T22:03:11.206 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- start start 2021-11-21T22:03:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc8060740 0x7f8cc8137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc809c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8cc80b1a50 0x7f8cc813ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc809d160 con 0x7f8cc8065b70 2021-11-21T22:03:11.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81366c0 con 0x7f8cc80b1a50 2021-11-21T22:03:11.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc805d3d0 con 0x7f8cc8060740 2021-11-21T22:03:11.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc809c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc809c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54262/0 (socket says 172.21.2.116:54262) 2021-11-21T22:03:11.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 -- 172.21.2.116:0/1732120511 learned_addr learned my addr 172.21.2.116:0/1732120511 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:11.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 -- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc8060740 msgr2=0x7f8cc8137860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:11.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc8060740 0x7f8cc8137860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 -- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8cc80b1a50 msgr2=0x7f8cc813ec80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8cc80b1a50 0x7f8cc813ec80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 -- 172.21.2.116:0/1732120511 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb8009530 con 0x7f8cc8065b70 2021-11-21T22:03:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cc67fc700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc809c7f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f8cc0002ef0 tx=0x7f8cc0006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:11.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cbffff700 1 -- 172.21.2.116:0/1732120511 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cc0009e00 con 0x7f8cc8065b70 2021-11-21T22:03:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cbffff700 1 -- 172.21.2.116:0/1732120511 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc0002320 con 0x7f8cc8065b70 2021-11-21T22:03:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc8063780 con 0x7f8cc8065b70 2021-11-21T22:03:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8cbffff700 1 -- 172.21.2.116:0/1732120511 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cc0002680 con 0x7f8cc8065b70 2021-11-21T22:03:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.195+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc81387d0 con 0x7f8cc8065b70 2021-11-21T22:03:11.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.199+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc8144bf0 con 0x7f8cc8065b70 2021-11-21T22:03:11.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.199+0000 7f8cbffff700 1 -- 172.21.2.116:0/1732120511 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8cc000f030 con 0x7f8cc8065b70 2021-11-21T22:03:11.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.199+0000 7f8cbffff700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cac0534f0 0x7f8cac0559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:11.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.199+0000 7f8cbffff700 1 -- 172.21.2.116:0/1732120511 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8cc006b2e0 con 0x7f8cc8065b70 2021-11-21T22:03:11.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.199+0000 7f8cc6ffd700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cac0534f0 0x7f8cac0559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:11.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.199+0000 7f8cc6ffd700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cac0534f0 0x7f8cac0559b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f8cb0002fb0 tx=0x7f8cb0006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:11.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.199+0000 7f8cbffff700 1 -- 172.21.2.116:0/1732120511 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8cc00426c0 con 0x7f8cc8065b70 2021-11-21T22:03:11.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.311+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f8cc80573d0 con 0x7f8cc8065b70 2021-11-21T22:03:11.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.983+0000 7f8cbffff700 1 -- 172.21.2.116:0/1732120511 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v395) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8cc0045ce0 con 0x7f8cc8065b70 2021-11-21T22:03:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cac0534f0 msgr2=0x7f8cac0559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cac0534f0 0x7f8cac0559b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f8cb0002fb0 tx=0x7f8cb0006930).stop 2021-11-21T22:03:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 msgr2=0x7f8cc809c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:11.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc809c7f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f8cc0002ef0 tx=0x7f8cc0006d40).stop 2021-11-21T22:03:11.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 shutdown_connections 2021-11-21T22:03:11.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8cc80b1a50 0x7f8cc813ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8cac0534f0 0x7f8cac0559b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8cc8065b70 0x7f8cc809c7f0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 --2- 172.21.2.116:0/1732120511 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8cc8060740 0x7f8cc8137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 >> 172.21.2.116:0/1732120511 conn(0x7f8cc80b7b80 msgr2=0x7f8cc80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 shutdown_connections 2021-11-21T22:03:11.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:11.987+0000 7f8ccd651700 1 -- 172.21.2.116:0/1732120511 wait complete. 2021-11-21T22:03:11.997 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2021-11-21T22:03:11.997 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:03:11.998 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2021-11-21T22:03:12.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 Processor -- start 2021-11-21T22:03:12.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- start start 2021-11-21T22:03:12.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa3800a6aa0 0x7fa38014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:12.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa380062410 0x7fa3801443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:12.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3800a6e80 con 0x7fa3800a86b0 2021-11-21T22:03:12.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3800a8bc0 con 0x7fa3800a6aa0 2021-11-21T22:03:12.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38014a7b0 con 0x7fa380062410 2021-11-21T22:03:12.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa385c12700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fa3800a86b0 0x7fa3801470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:37994/0 (socket says 172.21.2.116:37994) 2021-11-21T22:03:12.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa385c12700 1 -- 172.21.2.116:0/3150641686 learned_addr learned my addr 172.21.2.116:0/3150641686 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:12.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa386413700 1 --2- 172.21.2.116:0/3150641686 >> v2:172.21.2.116:3300/0 conn(0x7fa3800a6aa0 0x7fa38014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:12.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa386413700 1 -- 172.21.2.116:0/3150641686 >> v2:172.21.2.117:3300/0 conn(0x7fa380062410 msgr2=0x7fa3801443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:12.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa386413700 1 --2- 172.21.2.116:0/3150641686 >> v2:172.21.2.117:3300/0 conn(0x7fa380062410 0x7fa3801443f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:12.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa386413700 1 -- 172.21.2.116:0/3150641686 >> v1:172.21.2.116:6789/0 conn(0x7fa3800a86b0 legacy=0x7fa3801470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:12.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa386413700 1 -- 172.21.2.116:0/3150641686 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38014aa10 con 0x7fa3800a6aa0 2021-11-21T22:03:12.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa386413700 1 --2- 172.21.2.116:0/3150641686 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a6aa0 0x7fa38014b150 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fa374002f70 tx=0x7fa374006df0).ready entity=mon.0 client_cookie=b44fa3465f950c06 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:12.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa385411700 1 -- 172.21.2.116:0/3150641686 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37400d5c0 con 0x7fa3800a6aa0 2021-11-21T22:03:12.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa385411700 1 -- 172.21.2.116:0/3150641686 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374000e90 con 0x7fa3800a6aa0 2021-11-21T22:03:12.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa385411700 1 -- 172.21.2.116:0/3150641686 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37400d950 con 0x7fa3800a6aa0 2021-11-21T22:03:12.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- 172.21.2.116:0/3150641686 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a6aa0 msgr2=0x7fa38014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:12.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 --2- 172.21.2.116:0/3150641686 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a6aa0 0x7fa38014b150 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fa374002f70 tx=0x7fa374006df0).stop 2021-11-21T22:03:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- 172.21.2.116:0/3150641686 shutdown_connections 2021-11-21T22:03:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 --2- 172.21.2.116:0/3150641686 >> v2:172.21.2.117:3300/0 conn(0x7fa380062410 0x7fa3801443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 --2- 172.21.2.116:0/3150641686 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a6aa0 0x7fa38014b150 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:12.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- 172.21.2.116:0/3150641686 >> 172.21.2.116:0/3150641686 conn(0x7fa38005bd20 msgr2=0x7fa38005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- 172.21.2.116:0/3150641686 shutdown_connections 2021-11-21T22:03:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.047+0000 7fa388677700 1 -- 172.21.2.116:0/3150641686 wait complete. 2021-11-21T22:03:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 Processor -- start 2021-11-21T22:03:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 -- start start 2021-11-21T22:03:12.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa380062410 0x7fa3800ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:12.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa3800a6aa0 0x7fa3800ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:12.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a86b0 0x7fa380159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:12.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800ace70 con 0x7fa3800a86b0 2021-11-21T22:03:12.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa380141f70 con 0x7fa3800a6aa0 2021-11-21T22:03:12.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa380063be0 con 0x7fa380062410 2021-11-21T22:03:12.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a86b0 0x7fa380159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:12.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a86b0 0x7fa380159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54276/0 (socket says 172.21.2.116:54276) 2021-11-21T22:03:12.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 -- 172.21.2.116:0/2156753785 learned_addr learned my addr 172.21.2.116:0/2156753785 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:12.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386413700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa380062410 0x7fa3800ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:12.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 -- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa380062410 msgr2=0x7fa3800ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:12.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa385c12700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa3800a6aa0 0x7fa3800ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:12.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa380062410 0x7fa3800ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:12.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 -- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa3800a6aa0 msgr2=0x7fa3800ac980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:12.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa3800a6aa0 0x7fa3800ac980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:12.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 -- 172.21.2.116:0/2156753785 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37400b180 con 0x7fa3800a86b0 2021-11-21T22:03:12.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386413700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa380062410 0x7fa3800ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:12.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa385c12700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa3800a6aa0 0x7fa3800ac980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:12.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa386c14700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a86b0 0x7fa380159910 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fa380067810 tx=0x7fa3780091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:12.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa3737fe700 1 -- 172.21.2.116:0/2156753785 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa378001520 con 0x7fa3800a86b0 2021-11-21T22:03:12.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa3737fe700 1 -- 172.21.2.116:0/2156753785 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378001680 con 0x7fa3800a86b0 2021-11-21T22:03:12.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa3737fe700 1 -- 172.21.2.116:0/2156753785 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3780019e0 con 0x7fa3800a86b0 2021-11-21T22:03:12.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3800642b0 con 0x7fa3800a86b0 2021-11-21T22:03:12.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3801571e0 con 0x7fa3800a86b0 2021-11-21T22:03:12.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38014abf0 con 0x7fa3800a86b0 2021-11-21T22:03:12.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa3737fe700 1 -- 172.21.2.116:0/2156753785 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa378018030 con 0x7fa3800a86b0 2021-11-21T22:03:12.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa3737fe700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa360053640 0x7fa360055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:12.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.051+0000 7fa3737fe700 1 -- 172.21.2.116:0/2156753785 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa37806b060 con 0x7fa3800a86b0 2021-11-21T22:03:12.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.055+0000 7fa386413700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa360053640 0x7fa360055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:12.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.055+0000 7fa386413700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa360053640 0x7fa360055b00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fa374015000 tx=0x7fa374014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:12.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.055+0000 7fa3737fe700 1 -- 172.21.2.116:0/2156753785 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa378042550 con 0x7fa3800a86b0 2021-11-21T22:03:12.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.167+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7fa3800573d0 con 0x7fa3800a86b0 2021-11-21T22:03:12.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa3737fe700 1 -- 172.21.2.116:0/2156753785 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v396) v1 ==== 139+0+0 (secure 0 0 0) 0x7fa378045b70 con 0x7fa3800a86b0 2021-11-21T22:03:12.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa360053640 msgr2=0x7fa360055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:12.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa360053640 0x7fa360055b00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fa374015000 tx=0x7fa374014000).stop 2021-11-21T22:03:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a86b0 msgr2=0x7fa380159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a86b0 0x7fa380159910 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fa380067810 tx=0x7fa3780091a0).stop 2021-11-21T22:03:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 shutdown_connections 2021-11-21T22:03:12.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa360053640 0x7fa360055b00 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:12.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa3800a86b0 0x7fa380159910 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:12.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa3800a6aa0 0x7fa3800ac980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:12.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 --2- 172.21.2.116:0/2156753785 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa380062410 0x7fa3800ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:12.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 >> 172.21.2.116:0/2156753785 conn(0x7fa38005bd20 msgr2=0x7fa380149090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:12.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 shutdown_connections 2021-11-21T22:03:12.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:12.991+0000 7fa388677700 1 -- 172.21.2.116:0/2156753785 wait complete. 2021-11-21T22:03:13.002 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-11-21T22:03:13.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 Processor -- start 2021-11-21T22:03:13.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- start start 2021-11-21T22:03:13.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c144810 0x7f720c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f720c0ac600 0x7f720c0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c147e80 con 0x7f720c0abce0 2021-11-21T22:03:13.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c0b1490 con 0x7f720c0ac600 2021-11-21T22:03:13.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c0b1fd0 con 0x7f720c144810 2021-11-21T22:03:13.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7211b4a700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f720c0abce0 0x7f720c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38010/0 (socket says 172.21.2.116:38010) 2021-11-21T22:03:13.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7211b4a700 1 -- 172.21.2.116:0/3691299905 learned_addr learned my addr 172.21.2.116:0/3691299905 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:13.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7212b4c700 1 --2- 172.21.2.116:0/3691299905 >> v2:172.21.2.116:3300/0 conn(0x7f720c0ac600 0x7f720c0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7212b4c700 1 -- 172.21.2.116:0/3691299905 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c144810 msgr2=0x7f720c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7212b4c700 1 --2- 172.21.2.116:0/3691299905 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c144810 0x7f720c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7212b4c700 1 -- 172.21.2.116:0/3691299905 >> v1:172.21.2.116:6789/0 conn(0x7f720c0abce0 legacy=0x7f720c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7212b4c700 1 -- 172.21.2.116:0/3691299905 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0af5a0 con 0x7f720c0ac600 2021-11-21T22:03:13.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7211349700 1 -- 172.21.2.116:0/3691299905 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673011733 0 0) 0x7f720c147e80 con 0x7f720c0abce0 2021-11-21T22:03:13.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7212b4c700 1 --2- 172.21.2.116:0/3691299905 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c0ac600 0x7f720c0aca90 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f72040032f0 tx=0x7f72040091d0).ready entity=mon.0 client_cookie=adb2e01ebe6723ad server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7211349700 1 -- 172.21.2.116:0/3691299905 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7204009ea0 con 0x7f720c0ac600 2021-11-21T22:03:13.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7211349700 1 -- 172.21.2.116:0/3691299905 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400b8a0 con 0x7f720c0ac600 2021-11-21T22:03:13.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f7211349700 1 -- 172.21.2.116:0/3691299905 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f720400ba10 con 0x7f720c0ac600 2021-11-21T22:03:13.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- 172.21.2.116:0/3691299905 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c0ac600 msgr2=0x7f720c0aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- 172.21.2.116:0/3691299905 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c0ac600 0x7f720c0aca90 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f72040032f0 tx=0x7f72040091d0).stop 2021-11-21T22:03:13.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- 172.21.2.116:0/3691299905 shutdown_connections 2021-11-21T22:03:13.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- 172.21.2.116:0/3691299905 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c0ac600 0x7f720c0aca90 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- 172.21.2.116:0/3691299905 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c144810 0x7f720c1478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- 172.21.2.116:0/3691299905 >> 172.21.2.116:0/3691299905 conn(0x7f720c05bd20 msgr2=0x7f720c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- 172.21.2.116:0/3691299905 shutdown_connections 2021-11-21T22:03:13.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- 172.21.2.116:0/3691299905 wait complete. 2021-11-21T22:03:13.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 Processor -- start 2021-11-21T22:03:13.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 -- start start 2021-11-21T22:03:13.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c0abce0 0x7f720c13f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f720c0ac600 0x7f720c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.051+0000 7f72145af700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c144810 0x7f720c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f72145af700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c156410 con 0x7f720c144810 2021-11-21T22:03:13.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f72145af700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c13ccd0 con 0x7f720c0ac600 2021-11-21T22:03:13.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f72145af700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c15a690 con 0x7f720c0abce0 2021-11-21T22:03:13.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c144810 0x7f720c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c144810 0x7f720c156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54290/0 (socket says 172.21.2.116:54290) 2021-11-21T22:03:13.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 -- 172.21.2.116:0/1603704904 learned_addr learned my addr 172.21.2.116:0/1603704904 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:13.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 -- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c0abce0 msgr2=0x7f720c13f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c0abce0 0x7f720c13f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 -- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f720c0ac600 msgr2=0x7f720c0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f720c0ac600 0x7f720c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 -- 172.21.2.116:0/1603704904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7204009470 con 0x7f720c144810 2021-11-21T22:03:13.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f7212b4c700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c144810 0x7f720c156d40 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f72040032f0 tx=0x7f7204016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f71fb7fe700 1 -- 172.21.2.116:0/1603704904 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7204006ea0 con 0x7f720c144810 2021-11-21T22:03:13.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f71fb7fe700 1 -- 172.21.2.116:0/1603704904 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72040107a0 con 0x7f720c144810 2021-11-21T22:03:13.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f71fb7fe700 1 -- 172.21.2.116:0/1603704904 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7204009ea0 con 0x7f720c144810 2021-11-21T22:03:13.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c156600 con 0x7f720c144810 2021-11-21T22:03:13.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c15a3d0 con 0x7f720c144810 2021-11-21T22:03:13.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c1573d0 con 0x7f720c144810 2021-11-21T22:03:13.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f71fb7fe700 1 -- 172.21.2.116:0/1603704904 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7204029070 con 0x7f720c144810 2021-11-21T22:03:13.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f71fb7fe700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f71ec053610 0x7f71ec055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f71fb7fe700 1 -- 172.21.2.116:0/1603704904 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f720406c710 con 0x7f720c144810 2021-11-21T22:03:13.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f721234b700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f71ec053610 0x7f71ec055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.055+0000 7f721234b700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f71ec053610 0x7f71ec055ad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f7208002f70 tx=0x7f72080069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.059+0000 7f71fb7fe700 1 -- 172.21.2.116:0/1603704904 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7204043b50 con 0x7f720c144810 2021-11-21T22:03:13.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.139+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f720c15a850 con 0x7f720c144810 2021-11-21T22:03:13.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f71fb7fe700 1 -- 172.21.2.116:0/1603704904 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7204015020 con 0x7f720c144810 2021-11-21T22:03:13.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f71ec053610 msgr2=0x7f71ec055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f71ec053610 0x7f71ec055ad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f7208002f70 tx=0x7f72080069e0).stop 2021-11-21T22:03:13.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c144810 msgr2=0x7f720c156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c144810 0x7f720c156d40 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f72040032f0 tx=0x7f7204016000).stop 2021-11-21T22:03:13.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 shutdown_connections 2021-11-21T22:03:13.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f71ec053610 0x7f71ec055ad0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f720c144810 0x7f720c156d40 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f720c0ac600 0x7f720c0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 --2- 172.21.2.116:0/1603704904 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f720c0abce0 0x7f720c13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 >> 172.21.2.116:0/1603704904 conn(0x7f720c05bd20 msgr2=0x7f720c148280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 shutdown_connections 2021-11-21T22:03:13.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.147+0000 7f72145af700 1 -- 172.21.2.116:0/1603704904 wait complete. 2021-11-21T22:03:13.157 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:03:13.157 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:13.165 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:13.193 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:13.206 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:03:13.220 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:03:13.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:13.221 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:13.222 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:13.253 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-11-21T22:03:13.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:03:13.255 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-11-21T22:03:13.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:13.258 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:13.258 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:13.263 INFO:teuthology.orchestra.run.gibba036.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-11-21T22:03:13.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:03:13.265 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-11-21T22:03:13.265 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:03:13.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f7d4c1700 1 Processor -- start 2021-11-21T22:03:13.334 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f7d4c1700 1 -- start start 2021-11-21T22:03:13.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f7d4c1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 0x7f6f7814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f7d4c1700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6f780a86b0 0x7f6f781470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f7d4c1700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780a6e80 con 0x7f6f781475d0 2021-11-21T22:03:13.335 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f7d4c1700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f780a8bc0 con 0x7f6f780a86b0 2021-11-21T22:03:13.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f7d4c1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7814a7b0 con 0x7f6f780a6aa0 2021-11-21T22:03:13.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f777fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f6f781475d0 0x7f6f781443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38024/0 (socket says 172.21.2.116:38024) 2021-11-21T22:03:13.336 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f777fe700 1 -- 172.21.2.116:0/4182151768 learned_addr learned my addr 172.21.2.116:0/4182151768 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:13.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466227757 0 0) 0x7f6f780a6e80 con 0x7f6f781475d0 2021-11-21T22:03:13.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f58003260 con 0x7f6f781475d0 2021-11-21T22:03:13.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 0x7f6f7814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f767fc700 1 --2- 172.21.2.116:0/4182151768 >> v2:172.21.2.117:3300/0 conn(0x7f6f780a86b0 0x7f6f781470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1670772689 0 0) 0x7f6f58003260 con 0x7f6f781475d0 2021-11-21T22:03:13.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 msgr2=0x7f6f7814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 0x7f6f7814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 >> v2:172.21.2.117:3300/0 conn(0x7f6f780a86b0 msgr2=0x7f6f781470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 --2- 172.21.2.116:0/4182151768 >> v2:172.21.2.117:3300/0 conn(0x7f6f780a86b0 0x7f6f781470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7814aa10 con 0x7f6f781475d0 2021-11-21T22:03:13.342 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f6f700019e0 con 0x7f6f781475d0 2021-11-21T22:03:13.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 >> v1:172.21.2.116:6789/0 conn(0x7f6f781475d0 legacy=0x7f6f781443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f580037d0 0x7f6f58003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 0x7f6f58004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580049f0 0x7f6f58004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780a8bc0 con 0x7f6f580037d0 2021-11-21T22:03:13.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7814a7b0 con 0x7f6f580049f0 2021-11-21T22:03:13.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5800b880 con 0x7f6f780a6aa0 2021-11-21T22:03:13.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f580037d0 0x7f6f58003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f767fc700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 0x7f6f58004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f777fe700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580049f0 0x7f6f58004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 -- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 msgr2=0x7f6f58004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f780a6aa0 0x7f6f58004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 -- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580049f0 msgr2=0x7f6f58004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580049f0 0x7f6f58004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 -- 172.21.2.116:0/4182151768 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f58003260 con 0x7f6f580037d0 2021-11-21T22:03:13.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f777fe700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580049f0 0x7f6f58004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:13.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f76ffd700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f580037d0 0x7f6f58003bf0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f6f6000a000 tx=0x7f6f60008040).ready entity=mon.0 client_cookie=c99f1e30b474ead2 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f60013670 con 0x7f6f580037d0 2021-11-21T22:03:13.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f600137d0 con 0x7f6f580037d0 2021-11-21T22:03:13.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.331+0000 7f6f75ffb700 1 -- 172.21.2.116:0/4182151768 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f60013b30 con 0x7f6f580037d0 2021-11-21T22:03:13.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/4182151768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f580037d0 msgr2=0x7f6f58003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 --2- 172.21.2.116:0/4182151768 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f580037d0 0x7f6f58003bf0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f6f6000a000 tx=0x7f6f60008040).stop 2021-11-21T22:03:13.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 -- 172.21.2.116:0/4182151768 reap_dead start 2021-11-21T22:03:13.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/4182151768 shutdown_connections 2021-11-21T22:03:13.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/4182151768 >> 172.21.2.116:0/4182151768 conn(0x7f6f7805bd20 msgr2=0x7f6f7805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/4182151768 shutdown_connections 2021-11-21T22:03:13.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/4182151768 wait complete. 2021-11-21T22:03:13.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 Processor -- start 2021-11-21T22:03:13.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- start start 2021-11-21T22:03:13.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580037d0 0x7f6f78156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f78143510 0x7f6f780a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f780ab1d0 0x7f6f78159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78063cb0 con 0x7f6f780ab1d0 2021-11-21T22:03:13.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78143d30 con 0x7f6f580037d0 2021-11-21T22:03:13.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78156640 con 0x7f6f78143510 2021-11-21T22:03:13.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f780ab1d0 0x7f6f78159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f780ab1d0 0x7f6f78159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54308/0 (socket says 172.21.2.116:54308) 2021-11-21T22:03:13.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 -- 172.21.2.116:0/32941625 learned_addr learned my addr 172.21.2.116:0/32941625 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:13.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 -- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f78143510 msgr2=0x7f6f780a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f78143510 0x7f6f780a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 -- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580037d0 msgr2=0x7f6f78156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580037d0 0x7f6f78156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 -- 172.21.2.116:0/32941625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78143a90 con 0x7f6f780ab1d0 2021-11-21T22:03:13.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f777fe700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f780ab1d0 0x7f6f78159f70 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f6f78159e50 tx=0x7f6f70009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f6ffff700 1 -- 172.21.2.116:0/32941625 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f70004b80 con 0x7f6f780ab1d0 2021-11-21T22:03:13.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f6ffff700 1 -- 172.21.2.116:0/32941625 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70004ce0 con 0x7f6f780ab1d0 2021-11-21T22:03:13.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f6ffff700 1 -- 172.21.2.116:0/32941625 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f700072c0 con 0x7f6f780ab1d0 2021-11-21T22:03:13.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f78159830 con 0x7f6f780ab1d0 2021-11-21T22:03:13.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f780a93b0 con 0x7f6f780ab1d0 2021-11-21T22:03:13.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f780573d0 con 0x7f6f780ab1d0 2021-11-21T22:03:13.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f6ffff700 1 -- 172.21.2.116:0/32941625 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6f70007420 con 0x7f6f780ab1d0 2021-11-21T22:03:13.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f6ffff700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6f5c053570 0x7f6f5c055a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f6ffff700 1 -- 172.21.2.116:0/32941625 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6f7006b420 con 0x7f6f780ab1d0 2021-11-21T22:03:13.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.335+0000 7f6f76ffd700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6f5c053570 0x7f6f5c055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.339+0000 7f6f76ffd700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6f5c053570 0x7f6f5c055a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f6f6000a000 tx=0x7f6f60008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.339+0000 7f6f6ffff700 1 -- 172.21.2.116:0/32941625 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6f70042860 con 0x7f6f780ab1d0 2021-11-21T22:03:13.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6f7805db70 con 0x7f6f780ab1d0 2021-11-21T22:03:13.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f6ffff700 1 -- 172.21.2.116:0/32941625 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v155) v1 ==== 81+0+28835 (secure 0 0 0) 0x7f6f70045e80 con 0x7f6f780ab1d0 2021-11-21T22:03:13.427 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:13.427 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:03:13.428 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 155, 2021-11-21T22:03:13.428 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:03:13.428 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:03:13.428 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:03:10.976442+0000", 2021-11-21T22:03:13.429 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:03:13.429 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:03:13.429 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:03:13.429 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:03:13.429 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:03:13.430 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:03:13.430 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:03:13.430 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:03:13.430 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:03:13.431 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:13.431 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:03:13.431 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:03:13.431 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:03:13.431 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:03:13.432 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:03:13.432 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 28, 2021-11-21T22:03:13.432 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:03:13.432 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:03:13.432 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:03:13.433 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:03:13.433 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:03:13.433 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.433 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:03:13.434 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:03:13.434 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:03:13.434 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:13.434 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:13.434 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:13.435 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:13.435 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:13.435 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:13.435 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:13.436 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:13.436 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:03:13.436 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:03:13.436 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:03:13.436 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:03:13.437 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:03:13.437 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:13.437 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:13.437 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:13.437 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:13.438 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:13.438 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:13.438 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:13.438 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.439 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:03:13.439 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:13.439 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:13.439 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:13.439 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:13.440 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:13.440 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:13.440 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:13.440 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:13.441 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:13.441 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:13.441 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:13.441 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:13.441 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:13.442 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:13.442 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:13.442 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:13.442 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:13.443 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:13.443 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:13.443 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:13.443 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:13.444 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:13.444 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:13.444 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:13.444 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:13.445 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:13.445 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.445 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:13.445 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:13.445 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:13.446 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:13.446 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:13.446 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:13.446 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:13.446 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:13.447 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:13.447 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:13.447 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:13.447 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:03:13.448 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:03:13.448 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.448 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:13.448 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:03:13.448 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.449 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.449 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:03:13.449 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:03:13.449 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:03:13.450 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:13.450 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:13.450 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:13.450 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:13.450 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:13.451 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:13.451 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:13.451 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:13.451 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:03:13.452 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:03:13.452 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:03:13.452 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:03:13.452 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:03:13.453 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:13.453 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:13.453 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:13.453 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:13.453 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:13.454 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:13.454 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:13.454 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.454 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:03:13.455 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:13.455 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:13.455 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:13.455 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:13.455 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:13.456 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:13.456 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:13.456 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:13.456 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:13.457 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:13.457 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:13.457 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:13.457 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:13.457 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:13.458 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:13.458 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:13.458 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:13.458 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:13.459 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:13.459 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:13.459 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:13.459 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:13.459 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:13.460 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:13.460 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:13.460 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:13.460 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.460 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:13.461 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:13.461 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:13.461 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:13.461 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:13.462 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:13.462 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:13.462 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:13.462 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:13.462 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:13.463 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:13.463 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:03:13.463 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:13.463 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:03:13.464 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.464 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.464 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:13.464 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:03:13.464 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.465 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:03:13.465 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:03:13.465 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.465 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.465 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.466 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.466 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.466 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.466 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.467 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:13.467 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.467 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.467 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.467 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.468 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.468 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.468 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:03:13.468 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.468 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.469 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.469 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.469 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:03:13.469 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.469 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.470 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.470 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.470 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.470 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.471 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.471 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.471 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:03:13.471 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.471 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.472 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.472 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.472 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:03:13.472 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.473 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.473 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.473 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.473 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.473 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.474 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.474 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:03:13.474 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.474 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.475 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.475 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.475 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:03:13.475 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.476 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.476 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.476 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.476 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.476 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.477 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.477 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.477 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:03:13.477 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.477 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.478 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.478 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.478 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:03:13.478 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:13.479 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.479 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.479 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:03:13.479 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:03:13.480 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:03:13.480 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:03:13.480 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.480 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.480 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.481 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.481 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.481 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.481 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:03:13.482 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:03:13.482 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.482 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.482 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.482 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.483 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.483 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.483 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.483 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:13.484 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.484 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.484 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.484 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.484 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.485 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.485 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:03:13.485 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.485 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.486 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.486 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.486 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:03:13.486 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.486 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.487 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.487 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.487 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.487 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.488 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.488 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.488 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:03:13.488 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.488 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.489 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.489 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.489 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:03:13.489 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.489 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.490 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.490 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.490 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.490 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.491 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.491 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.491 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:03:13.491 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.491 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.492 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.492 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.492 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:03:13.492 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.492 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.493 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.493 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.493 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.493 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.494 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.494 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.494 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:03:13.494 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.494 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.495 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.495 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.495 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:03:13.495 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:13.495 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.496 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.496 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.496 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:03:13.496 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:03:13.497 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:03:13.497 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:03:13.497 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.497 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.497 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.498 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.498 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.498 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.498 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:03:13.498 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:03:13.499 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.499 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.499 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.499 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.500 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.500 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.500 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.500 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 149, 2021-11-21T22:03:13.500 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.501 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.501 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.501 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.501 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.502 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.502 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:03:13.502 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.502 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.502 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.503 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.503 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:03:13.503 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.503 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.503 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.504 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.504 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.504 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.504 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.505 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.505 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:03:13.505 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.505 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.505 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.506 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.506 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:03:13.506 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.506 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.507 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.507 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.507 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.507 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.507 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.508 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.508 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:03:13.508 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.508 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.508 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.509 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:03:13.509 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.509 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.510 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.510 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.510 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.510 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.511 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.511 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:03:13.511 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.511 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.511 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.512 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.512 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:03:13.512 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:13.512 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.513 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.513 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.513 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:03:13.513 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:03:13.513 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:03:13.514 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:03:13.514 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.514 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.514 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.515 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.515 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.515 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.515 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:03:13.515 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:03:13.516 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.516 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.516 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.516 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.517 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.517 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.517 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.517 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 138, 2021-11-21T22:03:13.517 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.518 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.518 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.518 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.518 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.519 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.519 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:03:13.519 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.519 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.519 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.520 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.520 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:03:13.520 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.520 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.520 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.521 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.521 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.521 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.521 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.521 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.522 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:03:13.522 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.522 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.522 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.523 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.523 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:03:13.523 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.523 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.523 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.524 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.524 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.524 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.524 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.524 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.525 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:03:13.525 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.525 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.525 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.525 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.526 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:03:13.526 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.526 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.526 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.527 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.527 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.527 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.527 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.528 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:03:13.528 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.528 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.528 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.528 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.529 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:03:13.529 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:13.529 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.529 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.530 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.530 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:03:13.530 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:03:13.530 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:03:13.530 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:03:13.531 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.531 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.531 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.531 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.531 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.532 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.532 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:03:13.532 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:03:13.532 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.533 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.533 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.533 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.533 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.533 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.534 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.534 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 149, 2021-11-21T22:03:13.534 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.534 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.534 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.535 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.535 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.535 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:03:13.535 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.536 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.536 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.536 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.536 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:03:13.537 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.537 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.537 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.537 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.537 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.538 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.538 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.538 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.538 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:03:13.538 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.539 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.539 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.539 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.539 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:03:13.540 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.540 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.540 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.540 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.540 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.541 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.541 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.541 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.541 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:03:13.541 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.542 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.542 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.542 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.542 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:03:13.543 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.543 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.543 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.543 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.543 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.544 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.544 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.544 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:03:13.544 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.545 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.545 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.545 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.545 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:03:13.545 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:13.546 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.546 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.546 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.546 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:03:13.547 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:03:13.547 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:03:13.547 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:03:13.547 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.547 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.548 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.548 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.548 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.548 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.548 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:03:13.549 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:03:13.549 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.549 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.549 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.550 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.550 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.550 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.550 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.551 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:13.551 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.551 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.551 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.551 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.552 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.552 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.552 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:03:13.552 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.552 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.553 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.553 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.553 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:03:13.553 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.553 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.554 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.554 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.554 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.554 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.555 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.555 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.555 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:03:13.555 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.555 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.556 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.556 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.556 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:03:13.556 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.557 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.557 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.557 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.557 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.557 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.558 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.558 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.558 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:03:13.558 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.558 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.559 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.559 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.559 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:03:13.559 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.560 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.560 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.560 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.560 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.560 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.561 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.561 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.561 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:03:13.561 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.561 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.562 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.562 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.562 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:03:13.562 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:13.562 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.563 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.563 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.563 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:03:13.563 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:03:13.564 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:03:13.564 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:03:13.564 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.564 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.564 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.565 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.565 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.565 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.565 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:03:13.565 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:03:13.566 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.566 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.566 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.566 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.567 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.567 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.567 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.567 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:13.567 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.568 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.568 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.568 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.568 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.568 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.569 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:03:13.569 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.569 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.569 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.570 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.570 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:03:13.570 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.570 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.570 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.571 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.571 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.571 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.571 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.571 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.572 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:03:13.572 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.572 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.572 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.573 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.573 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:03:13.573 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.573 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.573 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.574 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.574 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.574 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.574 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.574 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.575 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:03:13.575 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.575 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.575 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.575 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.576 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:03:13.576 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.576 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.576 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.577 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.577 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.577 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.577 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.577 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.578 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:03:13.578 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.578 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.578 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.578 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.579 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:03:13.579 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:13.579 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.579 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.579 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.580 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:03:13.580 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:03:13.580 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:03:13.580 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:03:13.581 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.581 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.581 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.581 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.581 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.582 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.582 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:03:13.582 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:03:13.582 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:13.582 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:13.583 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:13.583 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:13.583 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:13.583 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:13.583 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:13.584 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 150, 2021-11-21T22:03:13.584 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:13.584 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:13.584 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:13.585 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.585 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.585 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.585 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:03:13.585 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.586 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.586 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.586 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.586 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:03:13.586 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.587 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.587 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.587 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.587 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:13.587 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.588 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.588 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.588 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:03:13.588 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.589 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.589 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.589 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.589 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:03:13.589 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.590 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.590 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.590 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.590 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:13.590 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.591 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.591 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.591 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:03:13.591 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.591 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.592 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.592 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.592 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:03:13.592 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.593 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.593 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.593 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.593 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:13.593 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:13.594 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.594 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:13.594 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:03:13.594 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.594 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.595 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.595 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:13.595 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:03:13.595 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:13.595 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.596 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.596 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.596 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:03:13.596 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:03:13.597 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:03:13.597 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:03:13.597 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:13.597 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:13.597 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:13.598 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:13.598 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.598 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:13.598 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:03:13.598 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.599 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:03:13.599 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.599 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.599 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.600 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.600 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.600 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:03:13.600 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.600 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.601 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.601 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:03:13.601 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.601 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.601 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.602 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.602 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.602 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:03:13.602 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.602 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.603 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.603 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:03:13.603 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.603 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.604 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.604 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.604 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.604 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:03:13.604 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.605 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.605 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.605 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:03:13.605 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.605 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.606 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.606 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.606 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.606 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:03:13.607 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.607 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.607 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.607 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:03:13.607 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.608 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.608 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.608 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.608 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.608 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:03:13.609 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.609 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.609 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.609 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:03:13.609 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.610 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.610 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.610 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.610 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.611 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:03:13.611 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.611 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.611 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.611 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:03:13.612 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.612 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.612 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.612 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.613 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.613 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:03:13.613 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.613 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.613 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:13.614 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:03:13.614 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:13.614 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:13.614 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:13.615 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:13.615 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:13.615 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:03:13.615 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:13.615 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.616 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:13.616 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:03:13.616 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:03:13.616 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:03:13.617 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:03:13.617 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": { 2021-11-21T22:03:13.617 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6835/139139464": "2021-11-22T22:03:07.439133+0000", 2021-11-21T22:03:13.617 INFO:teuthology.orchestra.run.gibba016.stdout: "172.21.2.116:6834/139139464": "2021-11-22T22:03:07.439133+0000" 2021-11-21T22:03:13.617 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.618 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:03:13.618 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:03:13.618 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:03:13.618 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:03:13.619 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:03:13.619 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:03:13.619 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:03:13.619 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:03:13.619 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:13.620 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:13.620 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:03:13.620 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:03:13.620 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:03:13.621 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:03:13.621 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:03:13.621 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:03:13.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6f5c053570 msgr2=0x7f6f5c055a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6f5c053570 0x7f6f5c055a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f6f6000a000 tx=0x7f6f60008040).stop 2021-11-21T22:03:13.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f780ab1d0 msgr2=0x7f6f78159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f780ab1d0 0x7f6f78159f70 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f6f78159e50 tx=0x7f6f70009040).stop 2021-11-21T22:03:13.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 shutdown_connections 2021-11-21T22:03:13.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6f5c053570 0x7f6f5c055a30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6f780ab1d0 0x7f6f78159f70 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6f78143510 0x7f6f780a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 --2- 172.21.2.116:0/32941625 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6f580037d0 0x7f6f78156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 >> 172.21.2.116:0/32941625 conn(0x7f6f7805bd20 msgr2=0x7f6f780b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 shutdown_connections 2021-11-21T22:03:13.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.423+0000 7f6f7d4c1700 1 -- 172.21.2.116:0/32941625 wait complete. 2021-11-21T22:03:13.625 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:13.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 Processor -- start 2021-11-21T22:03:13.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- start start 2021-11-21T22:03:13.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 0x7f74c814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80a6e80 con 0x7f74c80a86b0 2021-11-21T22:03:13.685 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c814a7b0 con 0x7f74c8062410 2021-11-21T22:03:13.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80a8bc0 con 0x7f74c80a6aa0 2021-11-21T22:03:13.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c77fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f74c80a86b0 0x7f74c81470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38040/0 (socket says 172.21.2.116:38040) 2021-11-21T22:03:13.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c77fe700 1 -- 172.21.2.116:0/659198699 learned_addr learned my addr 172.21.2.116:0/659198699 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:13.686 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257038405 0 0) 0x7f74c80a6e80 con 0x7f74c80a86b0 2021-11-21T22:03:13.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a8003260 con 0x7f74c80a86b0 2021-11-21T22:03:13.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 0x7f74c814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 325180537 0 0) 0x7f74a8003260 con 0x7f74c80a86b0 2021-11-21T22:03:13.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 msgr2=0x7f74c814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.687 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 0x7f74c814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 >> v1:172.21.2.117:6789/0 conn(0x7f74c8062410 legacy=0x7f74c81443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c814aa10 con 0x7f74c80a86b0 2021-11-21T22:03:13.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f74b8001680 con 0x7f74c80a86b0 2021-11-21T22:03:13.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 >> v1:172.21.2.116:6789/0 conn(0x7f74c80a86b0 legacy=0x7f74c81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74a8003800 0x7f74a8003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 0x7f74a8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74a8004b10 0x7f74a8004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80a8bc0 con 0x7f74a8003800 2021-11-21T22:03:13.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a800b910 con 0x7f74a8004b10 2021-11-21T22:03:13.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a800bad0 con 0x7f74c80a6aa0 2021-11-21T22:03:13.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74a8003800 0x7f74a8003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c77fe700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 0x7f74a8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 -- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 msgr2=0x7f74a8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74c80a6aa0 0x7f74a8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 -- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74a8004b10 msgr2=0x7f74a8004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74ccfd2700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74a8004b10 0x7f74a8004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74a8004b10 0x7f74a8004f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 -- 172.21.2.116:0/659198699 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8003260 con 0x7f74a8003800 2021-11-21T22:03:13.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c7fff700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74a8003800 0x7f74a8003c00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f74b000a000 tx=0x7f74b0008040).ready entity=mon.0 client_cookie=ea9fdd00a050e672 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b0013670 con 0x7f74a8003800 2021-11-21T22:03:13.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b00137d0 con 0x7f74a8003800 2021-11-21T22:03:13.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74c6ffd700 1 -- 172.21.2.116:0/659198699 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b0013b30 con 0x7f74a8003800 2021-11-21T22:03:13.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- 172.21.2.116:0/659198699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74a8003800 msgr2=0x7f74a8003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 --2- 172.21.2.116:0/659198699 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74a8003800 0x7f74a8003c00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f74b000a000 tx=0x7f74b0008040).stop 2021-11-21T22:03:13.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74ccfd2700 1 -- 172.21.2.116:0/659198699 reap_dead start 2021-11-21T22:03:13.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- 172.21.2.116:0/659198699 shutdown_connections 2021-11-21T22:03:13.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- 172.21.2.116:0/659198699 >> 172.21.2.116:0/659198699 conn(0x7f74c805bd20 msgr2=0x7f74c805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- 172.21.2.116:0/659198699 shutdown_connections 2021-11-21T22:03:13.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.679+0000 7f74cea35700 1 -- 172.21.2.116:0/659198699 wait complete. 2021-11-21T22:03:13.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 Processor -- start 2021-11-21T22:03:13.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 -- start start 2021-11-21T22:03:13.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74a8003800 0x7f74c8064030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74c80a9230 0x7f74c8066c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74c8064520 0x7f74c8067130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c8159ac0 con 0x7f74c80a9230 2021-11-21T22:03:13.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c815a5f0 con 0x7f74c8064520 2021-11-21T22:03:13.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80a8dc0 con 0x7f74a8003800 2021-11-21T22:03:13.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74c80a9230 0x7f74c8066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74c80a9230 0x7f74c8066c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54332/0 (socket says 172.21.2.116:54332) 2021-11-21T22:03:13.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 -- 172.21.2.116:0/1653492009 learned_addr learned my addr 172.21.2.116:0/1653492009 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:13.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 -- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74a8003800 msgr2=0x7f74c8064030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74a8003800 0x7f74c8064030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 -- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74c8064520 msgr2=0x7f74c8067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74c8064520 0x7f74c8067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 -- 172.21.2.116:0/1653492009 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80a8ff0 con 0x7f74c80a9230 2021-11-21T22:03:13.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c77fe700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74c80a9230 0x7f74c8066c40 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f74c8063880 tx=0x7f74b8001e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c57fa700 1 -- 172.21.2.116:0/1653492009 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b8010710 con 0x7f74c80a9230 2021-11-21T22:03:13.701 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c57fa700 1 -- 172.21.2.116:0/1653492009 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8010870 con 0x7f74c80a9230 2021-11-21T22:03:13.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c8159820 con 0x7f74c80a9230 2021-11-21T22:03:13.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c57fa700 1 -- 172.21.2.116:0/1653492009 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b8010bd0 con 0x7f74c80a9230 2021-11-21T22:03:13.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c81470e0 con 0x7f74c80a9230 2021-11-21T22:03:13.702 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c80573d0 con 0x7f74c80a9230 2021-11-21T22:03:13.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c57fa700 1 -- 172.21.2.116:0/1653492009 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f74b801a020 con 0x7f74c80a9230 2021-11-21T22:03:13.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c57fa700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74ac0535b0 0x7f74ac055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c57fa700 1 -- 172.21.2.116:0/1653492009 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f74b806b600 con 0x7f74c80a9230 2021-11-21T22:03:13.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.683+0000 7f74c7fff700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74ac0535b0 0x7f74ac055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.687+0000 7f74c7fff700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74ac0535b0 0x7f74ac055a70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f74b000a000 tx=0x7f74b0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.687+0000 7f74c57fa700 1 -- 172.21.2.116:0/1653492009 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74b80428d0 con 0x7f74c80a9230 2021-11-21T22:03:13.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74c8068320 con 0x7f74c80a9230 2021-11-21T22:03:13.798 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74c57fa700 1 -- 172.21.2.116:0/1653492009 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f74b8045ef0 con 0x7f74c80a9230 2021-11-21T22:03:13.799 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:13.799 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":8715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/2548389324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2548389324},{"type":"v1","addr":"172.21.2.116:6835","nonce":2548389324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[]} 2021-11-21T22:03:13.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74ac0535b0 msgr2=0x7f74ac055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.800 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74ac0535b0 0x7f74ac055a70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f74b000a000 tx=0x7f74b0008040).stop 2021-11-21T22:03:13.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74c80a9230 msgr2=0x7f74c8066c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74c80a9230 0x7f74c8066c40 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f74c8063880 tx=0x7f74b8001e00).stop 2021-11-21T22:03:13.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 shutdown_connections 2021-11-21T22:03:13.801 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f74c8064520 0x7f74c8067130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f74ac0535b0 0x7f74ac055a70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f74c80a9230 0x7f74c8066c40 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 --2- 172.21.2.116:0/1653492009 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f74a8003800 0x7f74c8064030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.802 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 >> 172.21.2.116:0/1653492009 conn(0x7f74c805bd20 msgr2=0x7f74c80b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 shutdown_connections 2021-11-21T22:03:13.803 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.795+0000 7f74cea35700 1 -- 172.21.2.116:0/1653492009 wait complete. 2021-11-21T22:03:13.803 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 73 2021-11-21T22:03:13.806 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:13.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 Processor -- start 2021-11-21T22:03:13.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- start start 2021-11-21T22:03:13.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f16ec0b24a0 0x7f16ec0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0af360 0x7f16ec0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec0b00d0 con 0x7f16ec0aeb70 2021-11-21T22:03:13.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec0adba0 con 0x7f16ec0b24a0 2021-11-21T22:03:13.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0af7a0 con 0x7f16ec0af360 2021-11-21T22:03:13.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eb7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f16ec0aeb70 0x7f16ec13c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38064/0 (socket says 172.21.2.116:38064) 2021-11-21T22:03:13.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eb7fe700 1 -- 172.21.2.116:0/1459795163 learned_addr learned my addr 172.21.2.116:0/1459795163 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:13.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eaffd700 1 --2- 172.21.2.116:0/1459795163 >> v2:172.21.2.116:3300/0 conn(0x7f16ec0b24a0 0x7f16ec0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eaffd700 1 -- 172.21.2.116:0/1459795163 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0af360 msgr2=0x7f16ec0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:13.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eaffd700 1 --2- 172.21.2.116:0/1459795163 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0af360 0x7f16ec0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eaffd700 1 -- 172.21.2.116:0/1459795163 >> v1:172.21.2.116:6789/0 conn(0x7f16ec0aeb70 legacy=0x7f16ec13c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eaffd700 1 -- 172.21.2.116:0/1459795163 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec0b2880 con 0x7f16ec0b24a0 2021-11-21T22:03:13.862 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eaffd700 1 --2- 172.21.2.116:0/1459795163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec0b24a0 0x7f16ec0b1a40 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f16dc002f70 tx=0x7f16dc006df0).ready entity=mon.0 client_cookie=ac8863812a5a8474 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16e9ffb700 1 -- 172.21.2.116:0/1459795163 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16dc00d570 con 0x7f16ec0b24a0 2021-11-21T22:03:13.865 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16e9ffb700 1 -- 172.21.2.116:0/1459795163 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc000e90 con 0x7f16ec0b24a0 2021-11-21T22:03:13.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16e9ffb700 1 -- 172.21.2.116:0/1459795163 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16dc00d8c0 con 0x7f16ec0b24a0 2021-11-21T22:03:13.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- 172.21.2.116:0/1459795163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec0b24a0 msgr2=0x7f16ec0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- 172.21.2.116:0/1459795163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec0b24a0 0x7f16ec0b1a40 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f16dc002f70 tx=0x7f16dc006df0).stop 2021-11-21T22:03:13.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- 172.21.2.116:0/1459795163 shutdown_connections 2021-11-21T22:03:13.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- 172.21.2.116:0/1459795163 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0af360 0x7f16ec0ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- 172.21.2.116:0/1459795163 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec0b24a0 0x7f16ec0b1a40 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- 172.21.2.116:0/1459795163 >> 172.21.2.116:0/1459795163 conn(0x7f16ec05fd00 msgr2=0x7f16ec0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- 172.21.2.116:0/1459795163 shutdown_connections 2021-11-21T22:03:13.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- 172.21.2.116:0/1459795163 wait complete. 2021-11-21T22:03:13.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 Processor -- start 2021-11-21T22:03:13.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- start start 2021-11-21T22:03:13.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0aeb70 0x7f16ec066530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16ec0af360 0x7f16ec155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec13fa50 0x7f16ec13d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec067210 con 0x7f16ec13fa50 2021-11-21T22:03:13.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec156410 con 0x7f16ec0af360 2021-11-21T22:03:13.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16f17ba700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec065b50 con 0x7f16ec0aeb70 2021-11-21T22:03:13.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec13fa50 0x7f16ec13d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eb7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec13fa50 0x7f16ec13d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54342/0 (socket says 172.21.2.116:54342) 2021-11-21T22:03:13.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.855+0000 7f16eb7fe700 1 -- 172.21.2.116:0/4125987878 learned_addr learned my addr 172.21.2.116:0/4125987878 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:13.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eb7fe700 1 -- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0aeb70 msgr2=0x7f16ec066530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:13.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eb7fe700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0aeb70 0x7f16ec066530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eb7fe700 1 -- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16ec0af360 msgr2=0x7f16ec155b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:13.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eb7fe700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16ec0af360 0x7f16ec155b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eb7fe700 1 -- 172.21.2.116:0/4125987878 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc00b180 con 0x7f16ec13fa50 2021-11-21T22:03:13.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eb7fe700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec13fa50 0x7f16ec13d8b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f16ec0630d0 tx=0x7f16e4007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16e3fff700 1 -- 172.21.2.116:0/4125987878 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16e4005ba0 con 0x7f16ec13fa50 2021-11-21T22:03:13.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16e3fff700 1 -- 172.21.2.116:0/4125987878 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e4005d00 con 0x7f16ec13fa50 2021-11-21T22:03:13.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec065df0 con 0x7f16ec13fa50 2021-11-21T22:03:13.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1560d0 con 0x7f16ec13fa50 2021-11-21T22:03:13.872 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16e3fff700 1 -- 172.21.2.116:0/4125987878 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16e40072c0 con 0x7f16ec13fa50 2021-11-21T22:03:13.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c4003410 con 0x7f16ec13fa50 2021-11-21T22:03:13.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16e3fff700 1 -- 172.21.2.116:0/4125987878 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f16e401a060 con 0x7f16ec13fa50 2021-11-21T22:03:13.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16e3fff700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16d00536f0 0x7f16d0055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:13.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16e3fff700 1 -- 172.21.2.116:0/4125987878 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f16e406b170 con 0x7f16ec13fa50 2021-11-21T22:03:13.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eaffd700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16d00536f0 0x7f16d0055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:13.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.859+0000 7f16eaffd700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16d00536f0 0x7f16d0055bb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f16dc002f70 tx=0x7f16dc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:13.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.863+0000 7f16e3fff700 1 -- 172.21.2.116:0/4125987878 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f16e40425b0 con 0x7f16ec13fa50 2021-11-21T22:03:13.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16c4003c40 con 0x7f16ec13fa50 2021-11-21T22:03:13.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16e3fff700 1 -- 172.21.2.116:0/4125987878 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f16e4018070 con 0x7f16ec13fa50 2021-11-21T22:03:13.976 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:13.976 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/3288253723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":3288253723},{"type":"v1","addr":"172.21.2.116:6833","nonce":3288253723}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":5371,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.117:6835/2661874539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":2661874539},{"type":"v1","addr":"172.21.2.117:6835","nonce":2661874539}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":8715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.116:6835/2548389324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":2548389324},{"type":"v1","addr":"172.21.2.116:6835","nonce":2548389324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[]} 2021-11-21T22:03:13.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16d00536f0 msgr2=0x7f16d0055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16d00536f0 0x7f16d0055bb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f16dc002f70 tx=0x7f16dc015000).stop 2021-11-21T22:03:13.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec13fa50 msgr2=0x7f16ec13d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec13fa50 0x7f16ec13d8b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f16ec0630d0 tx=0x7f16e4007f40).stop 2021-11-21T22:03:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 shutdown_connections 2021-11-21T22:03:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f16d00536f0 0x7f16d0055bb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f16ec13fa50 0x7f16ec13d8b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:13.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f16ec0af360 0x7f16ec155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 --2- 172.21.2.116:0/4125987878 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f16ec0aeb70 0x7f16ec066530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 >> 172.21.2.116:0/4125987878 conn(0x7f16ec05fd00 msgr2=0x7f16ec0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 shutdown_connections 2021-11-21T22:03:13.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:13.971+0000 7f16f17ba700 1 -- 172.21.2.116:0/4125987878 wait complete. 2021-11-21T22:03:13.980 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 73 2021-11-21T22:03:13.984 INFO:tasks.ceph.mds.a:Restarting daemon 2021-11-21T22:03:13.984 INFO:tasks.ceph.mds.a:Stopping old one... 2021-11-21T22:03:13.984 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:03:13.985 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:03:13.985 INFO:tasks.ceph.mds.c:Restarting daemon 2021-11-21T22:03:13.985 INFO:tasks.ceph.mds.c:Stopping old one... 2021-11-21T22:03:13.985 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:03:13.986 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:03:13.986 INFO:tasks.ceph.mds.b:Restarting daemon 2021-11-21T22:03:13.986 INFO:tasks.ceph.mds.b:Stopping old one... 2021-11-21T22:03:13.987 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:03:13.987 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:03:13.992 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:03:13.993 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-11-21T22:03:13.994 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:03:13.995 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-11-21T22:03:13.996 INFO:tasks.ceph.mds.a:Started 2021-11-21T22:03:13.996 DEBUG:teuthology.parallel:result is None 2021-11-21T22:03:13.997 INFO:tasks.ceph.mds.c:Started 2021-11-21T22:03:13.998 DEBUG:teuthology.parallel:result is None 2021-11-21T22:03:13.999 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:03:13.999 DEBUG:teuthology.orchestra.run.gibba017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-11-21T22:03:14.028 INFO:tasks.ceph.mds.a.gibba016.stdout:starting mds.a at 2021-11-21T22:03:14.029 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:03:14.041 INFO:tasks.ceph.mds.b:Started 2021-11-21T22:03:14.041 DEBUG:teuthology.parallel:result is None 2021-11-21T22:03:14.042 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-11-21T22:03:14.083 INFO:tasks.ceph.mds.b.gibba017.stdout:starting mds.b at 2021-11-21T22:03:14.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 Processor -- start 2021-11-21T22:03:14.094 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 -- start start 2021-11-21T22:03:14.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f7324144810 0x7f73241478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7324147e80 con 0x7f73240ac600 2021-11-21T22:03:14.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73240af5a0 con 0x7f73240abce0 2021-11-21T22:03:14.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240b1490 con 0x7f7324144810 2021-11-21T22:03:14.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f73240ac600 0x7f73240aca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38100/0 (socket says 172.21.2.116:38100) 2021-11-21T22:03:14.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 -- 172.21.2.116:0/4200633399 learned_addr learned my addr 172.21.2.116:0/4200633399 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:14.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716699804 0 0) 0x7f7324147e80 con 0x7f73240ac600 2021-11-21T22:03:14.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7304003260 con 0x7f73240ac600 2021-11-21T22:03:14.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2923691571 0 0) 0x7f7304003260 con 0x7f73240ac600 2021-11-21T22:03:14.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 >> v2:172.21.2.117:3300/0 conn(0x7f7324144810 msgr2=0x7f73241478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:14.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73237fe700 1 --2- 172.21.2.116:0/4200633399 >> v2:172.21.2.117:3300/0 conn(0x7f7324144810 0x7f73241478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:14.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 --2- 172.21.2.116:0/4200633399 >> v2:172.21.2.117:3300/0 conn(0x7f7324144810 0x7f73241478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:14.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 >> v1:172.21.2.117:6789/0 conn(0x7f73240abce0 legacy=0x7f73240b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:14.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73240b1fd0 con 0x7f73240ac600 2021-11-21T22:03:14.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f731c0019e0 con 0x7f73240ac600 2021-11-21T22:03:14.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 >> v1:172.21.2.116:6789/0 conn(0x7f73240ac600 legacy=0x7f73240aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:14.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7304003820 0x7f7304003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7304004500 0x7f7304004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f730400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240b1490 con 0x7f7304005050 2021-11-21T22:03:14.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730400bf40 con 0x7f7304004500 2021-11-21T22:03:14.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730400c100 con 0x7f7304003820 2021-11-21T22:03:14.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f730400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:14.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7322ffd700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7304004500 0x7f7304004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:14.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 -- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7304003820 msgr2=0x7f7304003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:14.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7304003820 0x7f7304003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:14.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 -- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7304004500 msgr2=0x7f7304004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:14.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7304004500 0x7f7304004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:14.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 -- 172.21.2.116:0/4200633399 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7304003260 con 0x7f7304005050 2021-11-21T22:03:14.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7322ffd700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7304004500 0x7f7304004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:14.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 -- 172.21.2.116:0/4200633399 reap_dead start 2021-11-21T22:03:14.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7323fff700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f730400b750 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f732405d170 tx=0x7f731c0020e0).ready entity=mon.0 client_cookie=efb7120c792b6523 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:14.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f731c00cc60 con 0x7f7304005050 2021-11-21T22:03:14.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00cdc0 con 0x7f7304005050 2021-11-21T22:03:14.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f73227fc700 1 -- 172.21.2.116:0/4200633399 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f731c005b80 con 0x7f7304005050 2021-11-21T22:03:14.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 -- 172.21.2.116:0/4200633399 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 msgr2=0x7f730400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:14.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.091+0000 7f7329ea0700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f730400b750 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f732405d170 tx=0x7f731c0020e0).stop 2021-11-21T22:03:14.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- 172.21.2.116:0/4200633399 shutdown_connections 2021-11-21T22:03:14.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 --2- 172.21.2.116:0/4200633399 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f730400b750 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:14.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- 172.21.2.116:0/4200633399 >> 172.21.2.116:0/4200633399 conn(0x7f732405bd20 msgr2=0x7f732405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:14.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- 172.21.2.116:0/4200633399 shutdown_connections 2021-11-21T22:03:14.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- 172.21.2.116:0/4200633399 wait complete. 2021-11-21T22:03:14.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 Processor -- start 2021-11-21T22:03:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- start start 2021-11-21T22:03:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f7324063ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7324145ff0 0x7f7324159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732415a2f0 0x7f73240629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324146450 con 0x7f7304005050 2021-11-21T22:03:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324064710 con 0x7f732415a2f0 2021-11-21T22:03:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73240649b0 con 0x7f7324145ff0 2021-11-21T22:03:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f7324063ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:14.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f7324063ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54386/0 (socket says 172.21.2.116:54386) 2021-11-21T22:03:14.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 -- 172.21.2.116:0/1830066846 learned_addr learned my addr 172.21.2.116:0/1830066846 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:14.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 -- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7324145ff0 msgr2=0x7f7324159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:14.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7324145ff0 0x7f7324159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:14.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 -- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732415a2f0 msgr2=0x7f73240629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:14.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732415a2f0 0x7f73240629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:14.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 -- 172.21.2.116:0/1830066846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324159960 con 0x7f7304005050 2021-11-21T22:03:14.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f73237fe700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f7324063ff0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f7314007000 tx=0x7f7314008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:14.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7320ff9700 1 -- 172.21.2.116:0/1830066846 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f731400f4e0 con 0x7f7304005050 2021-11-21T22:03:14.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7320ff9700 1 -- 172.21.2.116:0/1830066846 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314000e90 con 0x7f7304005050 2021-11-21T22:03:14.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73240a7130 con 0x7f7304005050 2021-11-21T22:03:14.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7320ff9700 1 -- 172.21.2.116:0/1830066846 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f731400f830 con 0x7f7304005050 2021-11-21T22:03:14.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7324062eb0 con 0x7f7304005050 2021-11-21T22:03:14.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7324159af0 con 0x7f7304005050 2021-11-21T22:03:14.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7320ff9700 1 -- 172.21.2.116:0/1830066846 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f731400f990 con 0x7f7304005050 2021-11-21T22:03:14.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7320ff9700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7308053640 0x7f7308055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:14.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7320ff9700 1 -- 172.21.2.116:0/1830066846 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f731406b140 con 0x7f7304005050 2021-11-21T22:03:14.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.095+0000 7f7322ffd700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7308053640 0x7f7308055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:14.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.099+0000 7f7322ffd700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7308053640 0x7f7308055b00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f730c008000 tx=0x7f730c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:14.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.099+0000 7f7320ff9700 1 -- 172.21.2.116:0/1830066846 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7314042580 con 0x7f7304005050 2021-11-21T22:03:14.195 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:14.191+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f73240af060 con 0x7f7304005050 2021-11-21T22:03:15.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7320ff9700 1 -- 172.21.2.116:0/1830066846 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v158) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7314045ba0 con 0x7f7304005050 2021-11-21T22:03:15.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7308053640 msgr2=0x7f7308055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7308053640 0x7f7308055b00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f730c008000 tx=0x7f730c009040).stop 2021-11-21T22:03:15.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 msgr2=0x7f7324063ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f7324063ff0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f7314007000 tx=0x7f7314008040).stop 2021-11-21T22:03:15.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 shutdown_connections 2021-11-21T22:03:15.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f732415a2f0 0x7f73240629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7324145ff0 0x7f7324159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7308053640 0x7f7308055b00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 --2- 172.21.2.116:0/1830066846 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7304005050 0x7f7324063ff0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 >> 172.21.2.116:0/1830066846 conn(0x7f732405bd20 msgr2=0x7f7324141800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 shutdown_connections 2021-11-21T22:03:15.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.159+0000 7f7329ea0700 1 -- 172.21.2.116:0/1830066846 wait complete. 2021-11-21T22:03:15.168 INFO:teuthology.orchestra.run.gibba016.stderr: removed all blacklist entries 2021-11-21T22:03:15.171 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-11-21T22:03:15.171 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-11-21T22:03:15.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 Processor -- start 2021-11-21T22:03:15.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 -- start start 2021-11-21T22:03:15.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdf1c0a6aa0 0x7fdf1c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fdf1c0a86b0 0x7fdf1c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c1475d0 0x7fdf1c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c0a8bc0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c0a6e80 con 0x7fdf1c0a86b0 2021-11-21T22:03:15.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf20862700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c14a7b0 con 0x7fdf1c1475d0 2021-11-21T22:03:15.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf1a59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdf1c0a6aa0 0x7fdf1c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf1a59c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdf1c0a6aa0 0x7fdf1c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54398/0 (socket says 172.21.2.116:54398) 2021-11-21T22:03:15.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.227+0000 7fdf1a59c700 1 -- 172.21.2.116:0/2029070166 learned_addr learned my addr 172.21.2.116:0/2029070166 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:15.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 -- 172.21.2.116:0/2029070166 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c1475d0 msgr2=0x7fdf1c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- 172.21.2.116:0/2029070166 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c1475d0 0x7fdf1c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 -- 172.21.2.116:0/2029070166 >> v2:172.21.2.117:3300/0 conn(0x7fdf1c0a86b0 msgr2=0x7fdf1c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- 172.21.2.116:0/2029070166 >> v2:172.21.2.117:3300/0 conn(0x7fdf1c0a86b0 0x7fdf1c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 -- 172.21.2.116:0/2029070166 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c14aa10 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- 172.21.2.116:0/2029070166 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c14b150 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7fdf0c002f70 tx=0x7fdf0c006eb0).ready entity=mon.0 client_cookie=f1830d5bdfbaf59 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1959a700 1 -- 172.21.2.116:0/2029070166 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0c00d570 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1959a700 1 -- 172.21.2.116:0/2029070166 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c000e90 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1959a700 1 -- 172.21.2.116:0/2029070166 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0c00d900 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/2029070166 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 msgr2=0x7fdf1c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 --2- 172.21.2.116:0/2029070166 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c14b150 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7fdf0c002f70 tx=0x7fdf0c006eb0).stop 2021-11-21T22:03:15.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/2029070166 shutdown_connections 2021-11-21T22:03:15.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 --2- 172.21.2.116:0/2029070166 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c1475d0 0x7fdf1c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 --2- 172.21.2.116:0/2029070166 >> v2:172.21.2.117:3300/0 conn(0x7fdf1c0a86b0 0x7fdf1c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 --2- 172.21.2.116:0/2029070166 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c14b150 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/2029070166 >> 172.21.2.116:0/2029070166 conn(0x7fdf1c05bd20 msgr2=0x7fdf1c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/2029070166 shutdown_connections 2021-11-21T22:03:15.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/2029070166 wait complete. 2021-11-21T22:03:15.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 Processor -- start 2021-11-21T22:03:15.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- start start 2021-11-21T22:03:15.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c0a86b0 0x7fdf1c0a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf1c1475d0 0x7fdf1c0a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1451a0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0abb40 con 0x7fdf1c1475d0 2021-11-21T22:03:15.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c146540 con 0x7fdf1c0a86b0 2021-11-21T22:03:15.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c1437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54404/0 (socket says 172.21.2.116:54404) 2021-11-21T22:03:15.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 -- 172.21.2.116:0/4205385120 learned_addr learned my addr 172.21.2.116:0/4205385120 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:15.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf19d9b700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c0a86b0 0x7fdf1c0a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1ad9d700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf1c1475d0 0x7fdf1c0a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 -- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c0a86b0 msgr2=0x7fdf1c0a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c0a86b0 0x7fdf1c0a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 -- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf1c1475d0 msgr2=0x7fdf1c0a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf1c1475d0 0x7fdf1c0a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 -- 172.21.2.116:0/4205385120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c00b1e0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf1a59c700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c1437a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7fdf0c002f70 tx=0x7fdf0c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf137fe700 1 -- 172.21.2.116:0/4205385120 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0c011690 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf137fe700 1 -- 172.21.2.116:0/4205385120 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c0117f0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c1457b0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf137fe700 1 -- 172.21.2.116:0/4205385120 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0c011b50 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c145cf0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c0a8f00 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.231+0000 7fdf137fe700 1 -- 172.21.2.116:0/4205385120 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdf0c025030 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.235+0000 7fdf137fe700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf000536d0 0x7fdf00055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.235+0000 7fdf137fe700 1 -- 172.21.2.116:0/4205385120 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdf0c06c6d0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.235+0000 7fdf19d9b700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf000536d0 0x7fdf00055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.235+0000 7fdf19d9b700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf000536d0 0x7fdf00055b90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fdf0400c000 tx=0x7fdf04008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.235+0000 7fdf137fe700 1 -- 172.21.2.116:0/4205385120 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf0c043b60 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdf1c0573d0 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf137fe700 1 -- 172.21.2.116:0/4205385120 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fdf0c023020 con 0x7fdf1c0a6aa0 2021-11-21T22:03:15.350 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:15.350 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:03:15.351 INFO:teuthology.orchestra.run.gibba016.stdout: "auth_dump": [ 2021-11-21T22:03:15.351 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.351 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.a", 2021-11-21T22:03:15.351 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phyOXiHxAACq/TYRLEBIgZRX0d/UScbQ==", 2021-11-21T22:03:15.352 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.352 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:03:15.352 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.352 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.352 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.353 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.353 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.353 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.353 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.b", 2021-11-21T22:03:15.354 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph2Wd6JhAAyOHB2glVy8I6JltIRfgHkg==", 2021-11-21T22:03:15.354 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.354 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:03:15.354 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.355 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.355 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.355 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.355 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.356 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.356 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mds.c", 2021-11-21T22:03:15.356 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5ph7PtQIRAAwfPVEb31Jb7pXz4Nik2+HQ==", 2021-11-21T22:03:15.356 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.357 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:03:15.357 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.357 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.357 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.357 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.358 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.358 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.359 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.0", 2021-11-21T22:03:15.359 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5pheLYkIhAA/8keDNhC2DZCnnGsoUf+lA==", 2021-11-21T22:03:15.359 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.359 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.360 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.360 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.360 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.360 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.360 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.361 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.1", 2021-11-21T22:03:15.361 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phjcnLKBAAH67SoT+ZSUGGreHvk1wjWQ==", 2021-11-21T22:03:15.361 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.362 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.362 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.362 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.362 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.363 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.363 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.363 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.2", 2021-11-21T22:03:15.363 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD9v5phLSJ5NBAAegQR/oqWGBGPpjMnL07vOg==", 2021-11-21T22:03:15.364 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.364 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.364 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.364 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.364 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.365 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.365 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.365 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.3", 2021-11-21T22:03:15.365 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phD9FmABAAv5rOM7rxykb8DgTOgm9A3A==", 2021-11-21T22:03:15.366 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.366 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.366 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.366 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.367 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.368 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.368 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.368 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.4", 2021-11-21T22:03:15.368 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD+v5phCbLvMBAAnnNxf1rC6CAvo1d21fx2KA==", 2021-11-21T22:03:15.369 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.369 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.369 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.369 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.370 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.370 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.370 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.370 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.5", 2021-11-21T22:03:15.371 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phNO2UAxAAil6k3y58BGoEfxedoJH/qA==", 2021-11-21T22:03:15.371 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.371 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.371 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.371 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.372 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.372 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.372 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.372 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.6", 2021-11-21T22:03:15.372 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5phTnghDRAA/fGoz7UWBHhZmWq3YXd6vA==", 2021-11-21T22:03:15.373 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.373 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.373 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.373 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.374 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.374 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.374 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.374 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "osd.7", 2021-11-21T22:03:15.374 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD/v5ph1NaFGxAAlKb7prjh917m2Tn07Rtwcg==", 2021-11-21T22:03:15.375 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.375 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.375 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.375 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.376 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.376 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.376 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.376 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.0", 2021-11-21T22:03:15.376 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phj+3sKRAAKfjTu8eVaRNhd0yCC6lGzg==", 2021-11-21T22:03:15.377 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.377 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:03:15.377 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:03:15.377 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:03:15.378 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.378 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.378 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.378 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.1", 2021-11-21T22:03:15.378 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phHPuHLRAA9FnjgJqnVmBomZUefoi8Eg==", 2021-11-21T22:03:15.379 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.379 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow", 2021-11-21T22:03:15.379 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow r", 2021-11-21T22:03:15.379 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow rw pool=cephfs_data" 2021-11-21T22:03:15.380 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.380 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.380 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.380 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "client.admin", 2021-11-21T22:03:15.380 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phf11OBhAAt23PtpOFnvSmYD0b9O28JQ==", 2021-11-21T22:03:15.381 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:03:15.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr": "allow *", 2021-11-21T22:03:15.381 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow *", 2021-11-21T22:03:15.382 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.382 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.382 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.382 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.382 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.x", 2021-11-21T22:03:15.383 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phvQ87HhAAlWky4+pqbp5i1KV4UpcZxQ==", 2021-11-21T22:03:15.383 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.383 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:03:15.383 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:03:15.384 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.384 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.384 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:15.384 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:15.384 INFO:teuthology.orchestra.run.gibba016.stdout: "entity": "mgr.y", 2021-11-21T22:03:15.385 INFO:teuthology.orchestra.run.gibba016.stdout: "key": "AQD8v5phesScGxAAaEpv+0/065kCRaIjSeMBeA==", 2021-11-21T22:03:15.385 INFO:teuthology.orchestra.run.gibba016.stdout: "caps": { 2021-11-21T22:03:15.385 INFO:teuthology.orchestra.run.gibba016.stdout: "mds": "allow *", 2021-11-21T22:03:15.385 INFO:teuthology.orchestra.run.gibba016.stdout: "mon": "allow profile mgr", 2021-11-21T22:03:15.386 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": "allow *" 2021-11-21T22:03:15.386 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.386 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:15.386 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:15.386 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:03:15.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf000536d0 msgr2=0x7fdf00055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf000536d0 0x7fdf00055b90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fdf0400c000 tx=0x7fdf04008040).stop 2021-11-21T22:03:15.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 msgr2=0x7fdf1c1437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c1437a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7fdf0c002f70 tx=0x7fdf0c015000).stop 2021-11-21T22:03:15.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 shutdown_connections 2021-11-21T22:03:15.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdf1c1475d0 0x7fdf1c0a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdf1c0a86b0 0x7fdf1c0a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdf000536d0 0x7fdf00055b90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 --2- 172.21.2.116:0/4205385120 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdf1c0a6aa0 0x7fdf1c1437a0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 >> 172.21.2.116:0/4205385120 conn(0x7fdf1c05bd20 msgr2=0x7fdf1c148460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 shutdown_connections 2021-11-21T22:03:15.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.347+0000 7fdf20862700 1 -- 172.21.2.116:0/4205385120 wait complete. 2021-11-21T22:03:15.391 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:15.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 Processor -- start 2021-11-21T22:03:15.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- start start 2021-11-21T22:03:15.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7bfc062410 0x7f7bfc144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc0a6e80 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc14a7b0 con 0x7f7bfc0a86b0 2021-11-21T22:03:15.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc0a8bc0 con 0x7f7bfc062410 2021-11-21T22:03:15.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c01ce7700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7bfc0a6aa0 0x7f7bfc1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38136/0 (socket says 172.21.2.116:38136) 2021-11-21T22:03:15.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c01ce7700 1 -- 172.21.2.116:0/1567993688 learned_addr learned my addr 172.21.2.116:0/1567993688 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:15.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c024e8700 1 --2- 172.21.2.116:0/1567993688 >> v2:172.21.2.116:3300/0 conn(0x7f7bfc062410 0x7f7bfc144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c00ce5700 1 -- 172.21.2.116:0/1567993688 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871046665 0 0) 0x7f7bfc0a6e80 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c00ce5700 1 -- 172.21.2.116:0/1567993688 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be4003260 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c024e8700 1 -- 172.21.2.116:0/1567993688 >> v1:172.21.2.117:6789/0 conn(0x7f7bfc0a86b0 legacy=0x7f7bfc1443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:15.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c024e8700 1 -- 172.21.2.116:0/1567993688 >> v1:172.21.2.116:6789/0 conn(0x7f7bfc0a6aa0 legacy=0x7f7bfc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c024e8700 1 -- 172.21.2.116:0/1567993688 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc14aa10 con 0x7f7bfc062410 2021-11-21T22:03:15.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c024e8700 1 --2- 172.21.2.116:0/1567993688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc062410 0x7f7bfc144b00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7f7bfc061b50 tx=0x7f7bf40091d0).ready entity=mon.0 client_cookie=26bfd9da5ff0320f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c00ce5700 1 -- 172.21.2.116:0/1567993688 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bf400b680 con 0x7f7bfc062410 2021-11-21T22:03:15.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c00ce5700 1 -- 172.21.2.116:0/1567993688 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf400b7e0 con 0x7f7bfc062410 2021-11-21T22:03:15.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c00ce5700 1 -- 172.21.2.116:0/1567993688 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bf400b950 con 0x7f7bfc062410 2021-11-21T22:03:15.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- 172.21.2.116:0/1567993688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc062410 msgr2=0x7f7bfc144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/1567993688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc062410 0x7f7bfc144b00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7f7bfc061b50 tx=0x7f7bf40091d0).stop 2021-11-21T22:03:15.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- 172.21.2.116:0/1567993688 shutdown_connections 2021-11-21T22:03:15.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/1567993688 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc062410 0x7f7bfc144b00 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- 172.21.2.116:0/1567993688 >> 172.21.2.116:0/1567993688 conn(0x7f7bfc05bd20 msgr2=0x7f7bfc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- 172.21.2.116:0/1567993688 shutdown_connections 2021-11-21T22:03:15.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 -- 172.21.2.116:0/1567993688 wait complete. 2021-11-21T22:03:15.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.447+0000 7f7c03f4b700 1 Processor -- start 2021-11-21T22:03:15.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 -- start start 2021-11-21T22:03:15.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7bfc062410 0x7f7bfc142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc0a6aa0 0x7f7bfc064b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7bfc0a86b0 0x7f7bfc065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0673b0 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc064730 con 0x7f7bfc062410 2021-11-21T22:03:15.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc067110 con 0x7f7bfc0a86b0 2021-11-21T22:03:15.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc0a6aa0 0x7f7bfc064b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc0a6aa0 0x7f7bfc064b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54420/0 (socket says 172.21.2.116:54420) 2021-11-21T22:03:15.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 -- 172.21.2.116:0/396441527 learned_addr learned my addr 172.21.2.116:0/396441527 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:15.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 -- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7bfc0a86b0 msgr2=0x7f7bfc065060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7bfc0a86b0 0x7f7bfc065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 -- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7bfc062410 msgr2=0x7f7bfc142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7bfc062410 0x7f7bfc142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 -- 172.21.2.116:0/396441527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4009470 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c014e6700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc0a6aa0 0x7f7bfc064b70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f7bf8002fb0 tx=0x7f7bf8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7beeffd700 1 -- 172.21.2.116:0/396441527 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bf8013460 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7beeffd700 1 -- 172.21.2.116:0/396441527 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf80135c0 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc146b80 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc063850 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7beeffd700 1 -- 172.21.2.116:0/396441527 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bf8013920 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc0648f0 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7beeffd700 1 -- 172.21.2.116:0/396441527 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7bf800f070 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7beeffd700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7bdc053660 0x7f7bdc055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7beeffd700 1 -- 172.21.2.116:0/396441527 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7bf806b170 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c01ce7700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7bdc053660 0x7f7bdc055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.451+0000 7f7c01ce7700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7bdc053660 0x7f7bdc055b20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f7bf0001110 tx=0x7f7bf00076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.455+0000 7f7beeffd700 1 -- 172.21.2.116:0/396441527 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7bf80424f0 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.535+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bfc0573d0 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.541 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.535+0000 7f7beeffd700 1 -- 172.21.2.116:0/396441527 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f7bf8045b10 con 0x7f7bfc0a6aa0 2021-11-21T22:03:15.541 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:15.542 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":158,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:15.150786+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:15.542 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7bdc053660 msgr2=0x7f7bdc055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7bdc053660 0x7f7bdc055b20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f7bf0001110 tx=0x7f7bf00076c0).stop 2021-11-21T22:03:15.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc0a6aa0 msgr2=0x7f7bfc064b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc0a6aa0 0x7f7bfc064b70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f7bf8002fb0 tx=0x7f7bf8006df0).stop 2021-11-21T22:03:15.543 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 shutdown_connections 2021-11-21T22:03:15.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7bfc0a86b0 0x7f7bfc065060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7bdc053660 0x7f7bdc055b20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7bfc0a6aa0 0x7f7bfc064b70 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.544 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 --2- 172.21.2.116:0/396441527 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7bfc062410 0x7f7bfc142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 >> 172.21.2.116:0/396441527 conn(0x7f7bfc05bd20 msgr2=0x7f7bfc1472f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 shutdown_connections 2021-11-21T22:03:15.545 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.539+0000 7f7c03f4b700 1 -- 172.21.2.116:0/396441527 wait complete. 2021-11-21T22:03:15.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-11-21T22:03:15.549 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:03:15.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 Processor -- start 2021-11-21T22:03:15.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- start start 2021-11-21T22:03:15.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1ec0abce0 0x7fb1ec1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ec147e80 con 0x7fb1ec144810 2021-11-21T22:03:15.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ec0af5a0 con 0x7fb1ec0ac600 2021-11-21T22:03:15.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0b1490 con 0x7fb1ec0abce0 2021-11-21T22:03:15.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1eaffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fb1ec144810 0x7fb1ec0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38150/0 (socket says 172.21.2.116:38150) 2021-11-21T22:03:15.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1eaffd700 1 -- 172.21.2.116:0/4265911000 learned_addr learned my addr 172.21.2.116:0/4265911000 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:15.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531708882 0 0) 0x7fb1ec147e80 con 0x7fb1ec144810 2021-11-21T22:03:15.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d0003260 con 0x7fb1ec144810 2021-11-21T22:03:15.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2700295990 0 0) 0x7fb1d0003260 con 0x7fb1ec144810 2021-11-21T22:03:15.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1ec0abce0 msgr2=0x7fb1ec1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1ec0abce0 0x7fb1ec1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 >> v1:172.21.2.117:6789/0 conn(0x7fb1ec0ac600 legacy=0x7fb1ec0acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec0b1fd0 con 0x7fb1ec144810 2021-11-21T22:03:15.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fb1e0001660 con 0x7fb1ec144810 2021-11-21T22:03:15.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 >> v1:172.21.2.116:6789/0 conn(0x7fb1ec144810 legacy=0x7fb1ec0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1d0003800 0x7fb1d0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1d0004500 0x7fb1d0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1d0005070 0x7fb1d000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0b1490 con 0x7fb1d0004500 2021-11-21T22:03:15.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d000bea0 con 0x7fb1d0005070 2021-11-21T22:03:15.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d000c060 con 0x7fb1d0003800 2021-11-21T22:03:15.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1ea7fc700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1d0004500 0x7fb1d0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1eb7fe700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1d0005070 0x7fb1d000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1eaffd700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1d0003800 0x7fb1d0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1ea7fc700 1 -- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1d0003800 msgr2=0x7fb1d0003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1ea7fc700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1d0003800 0x7fb1d0003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1ea7fc700 1 -- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1d0005070 msgr2=0x7fb1d000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1ea7fc700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1d0005070 0x7fb1d000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1ea7fc700 1 -- 172.21.2.116:0/4265911000 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0003260 con 0x7fb1d0004500 2021-11-21T22:03:15.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1ea7fc700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1d0004500 0x7fb1d0004980 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fb1e4000b60 tx=0x7fb1e4006f70).ready entity=mon.0 client_cookie=5bddf17f20bd7230 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e4011400 con 0x7fb1d0004500 2021-11-21T22:03:15.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e4011560 con 0x7fb1d0004500 2021-11-21T22:03:15.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1e9ffb700 1 -- 172.21.2.116:0/4265911000 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e40118c0 con 0x7fb1d0004500 2021-11-21T22:03:15.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- 172.21.2.116:0/4265911000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1d0004500 msgr2=0x7fb1d0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 --2- 172.21.2.116:0/4265911000 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1d0004500 0x7fb1d0004980 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fb1e4000b60 tx=0x7fb1e4006f70).stop 2021-11-21T22:03:15.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1eb7fe700 1 -- 172.21.2.116:0/4265911000 reap_dead start 2021-11-21T22:03:15.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- 172.21.2.116:0/4265911000 shutdown_connections 2021-11-21T22:03:15.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- 172.21.2.116:0/4265911000 >> 172.21.2.116:0/4265911000 conn(0x7fb1ec05bd20 msgr2=0x7fb1ec05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- 172.21.2.116:0/4265911000 shutdown_connections 2021-11-21T22:03:15.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- 172.21.2.116:0/4265911000 wait complete. 2021-11-21T22:03:15.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 Processor -- start 2021-11-21T22:03:15.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.599+0000 7fb1f16b5700 1 -- start start 2021-11-21T22:03:15.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1ec0b0a30 0x7fb1ec156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1ec0b4770 0x7fb1ec157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1ec0b2ab0 0x7fb1ec0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec1426e0 con 0x7fb1ec0b4770 2021-11-21T22:03:15.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0b0e10 con 0x7fb1ec0b0a30 2021-11-21T22:03:15.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec142900 con 0x7fb1ec0b2ab0 2021-11-21T22:03:15.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1ec0b4770 0x7fb1ec157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1ec0b4770 0x7fb1ec157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54442/0 (socket says 172.21.2.116:54442) 2021-11-21T22:03:15.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 -- 172.21.2.116:0/1262538065 learned_addr learned my addr 172.21.2.116:0/1262538065 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:15.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 -- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1ec0b2ab0 msgr2=0x7fb1ec0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1ec0b2ab0 0x7fb1ec0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 -- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1ec0b0a30 msgr2=0x7fb1ec156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1ec0b0a30 0x7fb1ec156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 -- 172.21.2.116:0/1262538065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec15a8e0 con 0x7fb1ec0b4770 2021-11-21T22:03:15.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1ea7fc700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1ec0b4770 0x7fb1ec157200 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7fb1e4000b60 tx=0x7fb1e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1dbfff700 1 -- 172.21.2.116:0/1262538065 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e4011400 con 0x7fb1ec0b4770 2021-11-21T22:03:15.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1dbfff700 1 -- 172.21.2.116:0/1262538065 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e4010e30 con 0x7fb1ec0b4770 2021-11-21T22:03:15.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec15ab80 con 0x7fb1ec0b4770 2021-11-21T22:03:15.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ec1576f0 con 0x7fb1ec0b4770 2021-11-21T22:03:15.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1dbfff700 1 -- 172.21.2.116:0/1262538065 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e401c610 con 0x7fb1ec0b4770 2021-11-21T22:03:15.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ec0af060 con 0x7fb1ec0b4770 2021-11-21T22:03:15.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1dbfff700 1 -- 172.21.2.116:0/1262538065 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb1e400f070 con 0x7fb1ec0b4770 2021-11-21T22:03:15.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1dbfff700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1c80535d0 0x7fb1c8055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1dbfff700 1 -- 172.21.2.116:0/1262538065 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb1e406cbf0 con 0x7fb1ec0b4770 2021-11-21T22:03:15.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1eaffd700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1c80535d0 0x7fb1c8055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.603+0000 7fb1eaffd700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1c80535d0 0x7fb1c8055a90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fb1e000b000 tx=0x7fb1ec146e80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.607+0000 7fb1dbfff700 1 -- 172.21.2.116:0/1262538065 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1e4047e60 con 0x7fb1ec0b4770 2021-11-21T22:03:15.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.687+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb1ec0573d0 con 0x7fb1ec0b4770 2021-11-21T22:03:15.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1dbfff700 1 -- 172.21.2.116:0/1262538065 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v158) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb1e4043f70 con 0x7fb1ec0b4770 2021-11-21T22:03:15.694 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:03:15.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1c80535d0 msgr2=0x7fb1c8055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1c80535d0 0x7fb1c8055a90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fb1e000b000 tx=0x7fb1ec146e80).stop 2021-11-21T22:03:15.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1ec0b4770 msgr2=0x7fb1ec157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1ec0b4770 0x7fb1ec157200 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7fb1e4000b60 tx=0x7fb1e4016000).stop 2021-11-21T22:03:15.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 shutdown_connections 2021-11-21T22:03:15.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1ec0b2ab0 0x7fb1ec0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1c80535d0 0x7fb1c8055a90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1ec0b4770 0x7fb1ec157200 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 --2- 172.21.2.116:0/1262538065 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1ec0b0a30 0x7fb1ec156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 >> 172.21.2.116:0/1262538065 conn(0x7fb1ec05bd20 msgr2=0x7fb1ec0acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 shutdown_connections 2021-11-21T22:03:15.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.691+0000 7fb1f16b5700 1 -- 172.21.2.116:0/1262538065 wait complete. 2021-11-21T22:03:15.701 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:03:15.756 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.751+0000 7f04f2ff0700 1 Processor -- start 2021-11-21T22:03:15.757 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.751+0000 7f04f2ff0700 1 -- start start 2021-11-21T22:03:15.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f04ec1426e0 0x7f04ec142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ec0afae0 con 0x7f04ec0ad470 2021-11-21T22:03:15.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ec0b0f60 con 0x7f04ec062e60 2021-11-21T22:03:15.760 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec063470 con 0x7f04ec1426e0 2021-11-21T22:03:15.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f158d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f04ec0ad470 0x7f04ec0ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38174/0 (socket says 172.21.2.116:38174) 2021-11-21T22:03:15.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f158d700 1 -- 172.21.2.116:0/3058320490 learned_addr learned my addr 172.21.2.116:0/3058320490 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:15.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431728777 0 0) 0x7f04ec0afae0 con 0x7f04ec0ad470 2021-11-21T22:03:15.761 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04cc003260 con 0x7f04ec0ad470 2021-11-21T22:03:15.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3269107681 0 0) 0x7f04cc003260 con 0x7f04ec0ad470 2021-11-21T22:03:15.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 >> v2:172.21.2.117:3300/0 conn(0x7f04ec1426e0 msgr2=0x7f04ec142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.762 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 --2- 172.21.2.116:0/3058320490 >> v2:172.21.2.117:3300/0 conn(0x7f04ec1426e0 0x7f04ec142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 >> v1:172.21.2.117:6789/0 conn(0x7f04ec062e60 legacy=0x7f04ec0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec143050 con 0x7f04ec0ad470 2021-11-21T22:03:15.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f04e40019e0 con 0x7f04ec0ad470 2021-11-21T22:03:15.763 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 >> v1:172.21.2.116:6789/0 conn(0x7f04ec0ad470 legacy=0x7f04ec0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04cc0037d0 0x7f04cc003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc004500 0x7f04cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04cc005070 0x7f04cc007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec063470 con 0x7f04cc0037d0 2021-11-21T22:03:15.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc007ed0 con 0x7f04cc004500 2021-11-21T22:03:15.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc008090 con 0x7f04cc005070 2021-11-21T22:03:15.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f0d8c700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04cc0037d0 0x7f04cc003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04ebfff700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc004500 0x7f04cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f158d700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04cc005070 0x7f04cc007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f0d8c700 1 -- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04cc005070 msgr2=0x7f04cc007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f0d8c700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04cc005070 0x7f04cc007730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f0d8c700 1 -- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc004500 msgr2=0x7f04cc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f0d8c700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc004500 0x7f04cc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f0d8c700 1 -- 172.21.2.116:0/3058320490 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec143050 con 0x7f04cc0037d0 2021-11-21T22:03:15.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f158d700 1 -- 172.21.2.116:0/3058320490 reap_dead start 2021-11-21T22:03:15.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f0d8c700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04cc0037d0 0x7f04cc003bd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f04d4002f70 tx=0x7f04d4006ca0).ready entity=mon.0 client_cookie=bd6c86aaca4b3dec server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04d4013410 con 0x7f04cc0037d0 2021-11-21T22:03:15.767 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d4013570 con 0x7f04cc0037d0 2021-11-21T22:03:15.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04eb7fe700 1 -- 172.21.2.116:0/3058320490 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04d40138d0 con 0x7f04cc0037d0 2021-11-21T22:03:15.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- 172.21.2.116:0/3058320490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04cc0037d0 msgr2=0x7f04cc003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04cc0037d0 0x7f04cc003bd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f04d4002f70 tx=0x7f04d4006ca0).stop 2021-11-21T22:03:15.768 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- 172.21.2.116:0/3058320490 shutdown_connections 2021-11-21T22:03:15.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/3058320490 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04cc0037d0 0x7f04cc003bd0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- 172.21.2.116:0/3058320490 >> 172.21.2.116:0/3058320490 conn(0x7f04ec0b5730 msgr2=0x7f04ec0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- 172.21.2.116:0/3058320490 shutdown_connections 2021-11-21T22:03:15.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- 172.21.2.116:0/3058320490 wait complete. 2021-11-21T22:03:15.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 Processor -- start 2021-11-21T22:03:15.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.755+0000 7f04f2ff0700 1 -- start start 2021-11-21T22:03:15.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc0037d0 0x7f04ec141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04ec150fa0 0x7f04ec0639b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04ec0670b0 0x7f04ec067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec13f810 con 0x7f04ec0670b0 2021-11-21T22:03:15.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec13e100 con 0x7f04cc0037d0 2021-11-21T22:03:15.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec13fa40 con 0x7f04ec150fa0 2021-11-21T22:03:15.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04ec0670b0 0x7f04ec067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04ec0670b0 0x7f04ec067560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54460/0 (socket says 172.21.2.116:54460) 2021-11-21T22:03:15.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 -- 172.21.2.116:0/2003354519 learned_addr learned my addr 172.21.2.116:0/2003354519 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:15.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 -- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04ec150fa0 msgr2=0x7f04ec0639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04ec150fa0 0x7f04ec0639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 -- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc0037d0 msgr2=0x7f04ec141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc0037d0 0x7f04ec141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 -- 172.21.2.116:0/2003354519 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec13f0d0 con 0x7f04ec0670b0 2021-11-21T22:03:15.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f158d700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04ec0670b0 0x7f04ec067560 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 rx=0x7f04e40016f0 tx=0x7f04e4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04e9ffb700 1 -- 172.21.2.116:0/2003354519 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04e4007e20 con 0x7f04ec0670b0 2021-11-21T22:03:15.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04e9ffb700 1 -- 172.21.2.116:0/2003354519 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4004ab0 con 0x7f04ec0670b0 2021-11-21T22:03:15.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec13e530 con 0x7f04ec0670b0 2021-11-21T22:03:15.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec13d0a0 con 0x7f04ec0670b0 2021-11-21T22:03:15.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04e9ffb700 1 -- 172.21.2.116:0/2003354519 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04e4004e10 con 0x7f04ec0670b0 2021-11-21T22:03:15.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c4003410 con 0x7f04ec0670b0 2021-11-21T22:03:15.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04e9ffb700 1 -- 172.21.2.116:0/2003354519 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f04e401c060 con 0x7f04ec0670b0 2021-11-21T22:03:15.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04e9ffb700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f04d00534f0 0x7f04d00559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04e9ffb700 1 -- 172.21.2.116:0/2003354519 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f04e406b090 con 0x7f04ec0670b0 2021-11-21T22:03:15.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f0d8c700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f04d00534f0 0x7f04d00559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.759+0000 7f04f0d8c700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f04d00534f0 0x7f04d00559b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f04d4002f70 tx=0x7f04d4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.763+0000 7f04e9ffb700 1 -- 172.21.2.116:0/2003354519 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04e4042520 con 0x7f04ec0670b0 2021-11-21T22:03:15.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.843+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f04c4003c40 con 0x7f04ec0670b0 2021-11-21T22:03:15.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.843+0000 7f04e9ffb700 1 -- 172.21.2.116:0/2003354519 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7f04e401a070 con 0x7f04ec0670b0 2021-11-21T22:03:15.849 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:15.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f04d00534f0 msgr2=0x7f04d00559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f04d00534f0 0x7f04d00559b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f04d4002f70 tx=0x7f04d4014000).stop 2021-11-21T22:03:15.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04ec0670b0 msgr2=0x7f04ec067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04ec0670b0 0x7f04ec067560 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 rx=0x7f04e40016f0 tx=0x7f04e4009040).stop 2021-11-21T22:03:15.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 shutdown_connections 2021-11-21T22:03:15.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f04d00534f0 0x7f04d00559b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.851 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f04ec0670b0 0x7f04ec067560 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f04ec150fa0 0x7f04ec0639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 --2- 172.21.2.116:0/2003354519 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f04cc0037d0 0x7f04ec141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 >> 172.21.2.116:0/2003354519 conn(0x7f04ec0b5730 msgr2=0x7f04ec0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 shutdown_connections 2021-11-21T22:03:15.852 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.847+0000 7f04f2ff0700 1 -- 172.21.2.116:0/2003354519 wait complete. 2021-11-21T22:03:15.857 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-11-21T22:03:15.908 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.903+0000 7f9dc305c700 1 Processor -- start 2021-11-21T22:03:15.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.903+0000 7f9dc305c700 1 -- start start 2021-11-21T22:03:15.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.903+0000 7f9dc305c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9dbc0affc0 0x7f9dbc05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.903+0000 7f9dc305c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0af7e0 0x7f9dbc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.903+0000 7f9dc305c700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc143f70 con 0x7f9dbc0ad780 2021-11-21T22:03:15.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.903+0000 7f9dc305c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc0adb60 con 0x7f9dbc0affc0 2021-11-21T22:03:15.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.903+0000 7f9dc305c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc143160 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc15f9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0af7e0 0x7f9dbc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbbfff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9dbc0affc0 0x7f9dbc05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc15f9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0af7e0 0x7f9dbc13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3301/0 says I am v2:172.21.2.116:57812/0 (socket says 172.21.2.116:57812) 2021-11-21T22:03:15.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbbfff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f9dbc0affc0 0x7f9dbc05f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:55636/0 (socket says 172.21.2.116:55636) 2021-11-21T22:03:15.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc15f9700 1 -- 172.21.2.116:0/717512548 learned_addr learned my addr 172.21.2.116:0/717512548 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:15.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbbfff700 1 -- 172.21.2.116:0/717512548 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0af7e0 msgr2=0x7f9dbc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:15.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbbfff700 1 --2- 172.21.2.116:0/717512548 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0af7e0 0x7f9dbc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbbfff700 1 -- 172.21.2.116:0/717512548 >> v1:172.21.2.117:6789/0 conn(0x7f9dbc0ad780 legacy=0x7f9dbc143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbbfff700 1 -- 172.21.2.116:0/717512548 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0b0ce0 con 0x7f9dbc0affc0 2021-11-21T22:03:15.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbbfff700 1 --2- 172.21.2.116:0/717512548 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 0x7f9dbc05f730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f9dac002f70 tx=0x7f9dac006df0).ready entity=mon.1 client_cookie=59c1c8eb1bd5f93e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbb7fe700 1 -- 172.21.2.116:0/717512548 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dac00d570 con 0x7f9dbc0affc0 2021-11-21T22:03:15.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbb7fe700 1 -- 172.21.2.116:0/717512548 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac000e90 con 0x7f9dbc0affc0 2021-11-21T22:03:15.917 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dbb7fe700 1 -- 172.21.2.116:0/717512548 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dac00d8c0 con 0x7f9dbc0affc0 2021-11-21T22:03:15.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 -- 172.21.2.116:0/717512548 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 msgr2=0x7f9dbc05f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 --2- 172.21.2.116:0/717512548 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 0x7f9dbc05f730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f9dac002f70 tx=0x7f9dac006df0).stop 2021-11-21T22:03:15.918 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 -- 172.21.2.116:0/717512548 shutdown_connections 2021-11-21T22:03:15.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 --2- 172.21.2.116:0/717512548 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0af7e0 0x7f9dbc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 --2- 172.21.2.116:0/717512548 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 0x7f9dbc05f730 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:15.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 -- 172.21.2.116:0/717512548 >> 172.21.2.116:0/717512548 conn(0x7f9dbc0b6000 msgr2=0x7f9dbc0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:15.919 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 -- 172.21.2.116:0/717512548 shutdown_connections 2021-11-21T22:03:15.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.907+0000 7f9dc305c700 1 -- 172.21.2.116:0/717512548 wait complete. 2021-11-21T22:03:15.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 Processor -- start 2021-11-21T22:03:15.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 -- start start 2021-11-21T22:03:15.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0ad780 0x7f9dbc1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dbc0af7e0 0x7f9dbc0a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 0x7f9dbc0a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0a7150 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0a2fb0 con 0x7f9dbc0affc0 2021-11-21T22:03:15.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc152ee0 con 0x7f9dbc0ad780 2021-11-21T22:03:15.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dbc0af7e0 0x7f9dbc0a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dbc0af7e0 0x7f9dbc0a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54474/0 (socket says 172.21.2.116:54474) 2021-11-21T22:03:15.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 -- 172.21.2.116:0/1024107706 learned_addr learned my addr 172.21.2.116:0/1024107706 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:15.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 -- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0ad780 msgr2=0x7f9dbc1529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0ad780 0x7f9dbc1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 -- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 msgr2=0x7f9dbc0a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:15.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 0x7f9dbc0a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:15.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 -- 172.21.2.116:0/1024107706 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac00b180 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dbbfff700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dbc0af7e0 0x7f9dbc0a6770 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f9dbc0a7290 tx=0x7f9dac014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9db9ffb700 1 -- 172.21.2.116:0/1024107706 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dac016070 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9db9ffb700 1 -- 172.21.2.116:0/1024107706 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac00d8f0 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc0a2a70 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9db9ffb700 1 -- 172.21.2.116:0/1024107706 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dac01e450 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc0ac9f0 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc0ab000 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9db9ffb700 1 -- 172.21.2.116:0/1024107706 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f9dac025030 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9db9ffb700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9da0053610 0x7f9da0055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:15.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9db9ffb700 1 -- 172.21.2.116:0/1024107706 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9dac06c750 con 0x7f9dbc0af7e0 2021-11-21T22:03:15.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.911+0000 7f9dc0df8700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9da0053610 0x7f9da0055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:15.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.915+0000 7f9dc0df8700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9da0053610 0x7f9da0055ad0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f9dbc068820 tx=0x7f9da4007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:15.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.915+0000 7f9db9ffb700 1 -- 172.21.2.116:0/1024107706 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9dac043b10 con 0x7f9dbc0af7e0 2021-11-21T22:03:16.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9dbc0b0a40 con 0x7f9dbc0af7e0 2021-11-21T22:03:16.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9db9ffb700 1 -- 172.21.2.116:0/1024107706 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v158) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9dac023070 con 0x7f9dbc0af7e0 2021-11-21T22:03:16.003 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:16.003 INFO:teuthology.orchestra.run.gibba016.stdout:[ 2021-11-21T22:03:16.003 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:16.011 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 1, 2021-11-21T22:03:16.011 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "device_health_metrics" 2021-11-21T22:03:16.012 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:16.012 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:16.012 INFO:teuthology.orchestra.run.gibba016.stdout: "poolnum": 2, 2021-11-21T22:03:16.012 INFO:teuthology.orchestra.run.gibba016.stdout: "poolname": "rbd" 2021-11-21T22:03:16.013 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:16.013 INFO:teuthology.orchestra.run.gibba016.stdout:] 2021-11-21T22:03:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9da0053610 msgr2=0x7f9da0055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9da0053610 0x7f9da0055ad0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f9dbc068820 tx=0x7f9da4007810).stop 2021-11-21T22:03:16.016 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dbc0af7e0 msgr2=0x7f9dbc0a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dbc0af7e0 0x7f9dbc0a6770 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f9dbc0a7290 tx=0x7f9dac014000).stop 2021-11-21T22:03:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 shutdown_connections 2021-11-21T22:03:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9dbc0affc0 0x7f9dbc0a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:16.017 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9da0053610 0x7f9da0055ad0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9dbc0af7e0 0x7f9dbc0a6770 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 --2- 172.21.2.116:0/1024107706 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9dbc0ad780 0x7f9dbc1529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 >> 172.21.2.116:0/1024107706 conn(0x7f9dbc0b6000 msgr2=0x7f9dbc05bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:16.018 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 shutdown_connections 2021-11-21T22:03:16.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:15.999+0000 7f9dc305c700 1 -- 172.21.2.116:0/1024107706 wait complete. 2021-11-21T22:03:16.019 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-11-21T22:03:16.020 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-11-21T22:03:16.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7e1c92700 1 Processor -- start 2021-11-21T22:03:16.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7e1c92700 1 -- start start 2021-11-21T22:03:16.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7e1c92700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe7dc144810 0x7fe7dc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:16.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7e1c92700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 0x7fe7dc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:16.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7e1c92700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7dc147e80 con 0x7fe7dc0b1db0 2021-11-21T22:03:16.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7e1c92700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc0b1490 con 0x7fe7dc144810 2021-11-21T22:03:16.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7e1c92700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0af060 con 0x7fe7dc0abce0 2021-11-21T22:03:16.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe7dc144810 0x7fe7dc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:16.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe7dc144810 0x7fe7dc1478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54478/0 (socket says 172.21.2.116:54478) 2021-11-21T22:03:16.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 -- 172.21.2.116:0/3085952067 learned_addr learned my addr 172.21.2.116:0/3085952067 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:16.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 -- 172.21.2.116:0/3085952067 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 msgr2=0x7fe7dc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:16.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 --2- 172.21.2.116:0/3085952067 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 0x7fe7dc0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:16.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 -- 172.21.2.116:0/3085952067 >> v1:172.21.2.117:6789/0 conn(0x7fe7dc0b1db0 legacy=0x7fe7dc0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:16.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 -- 172.21.2.116:0/3085952067 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc0af5a0 con 0x7fe7dc144810 2021-11-21T22:03:16.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7db7fe700 1 --2- 172.21.2.116:0/3085952067 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc144810 0x7fe7dc1478c0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7fe7dc0b2a30 tx=0x7fe7c4006eb0).ready entity=mon.0 client_cookie=b7eb58209822e4bd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:16.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7da7fc700 1 -- 172.21.2.116:0/3085952067 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7c400d570 con 0x7fe7dc144810 2021-11-21T22:03:16.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7da7fc700 1 -- 172.21.2.116:0/3085952067 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c4000e90 con 0x7fe7dc144810 2021-11-21T22:03:16.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.067+0000 7fe7da7fc700 1 -- 172.21.2.116:0/3085952067 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7c400d8c0 con 0x7fe7dc144810 2021-11-21T22:03:16.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/3085952067 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc144810 msgr2=0x7fe7dc1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:16.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/3085952067 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc144810 0x7fe7dc1478c0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7fe7dc0b2a30 tx=0x7fe7c4006eb0).stop 2021-11-21T22:03:16.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/3085952067 shutdown_connections 2021-11-21T22:03:16.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/3085952067 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 0x7fe7dc0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:16.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/3085952067 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc144810 0x7fe7dc1478c0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:16.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/3085952067 >> 172.21.2.116:0/3085952067 conn(0x7fe7dc05bd20 msgr2=0x7fe7dc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:16.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/3085952067 shutdown_connections 2021-11-21T22:03:16.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/3085952067 wait complete. 2021-11-21T22:03:16.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 Processor -- start 2021-11-21T22:03:16.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- start start 2021-11-21T22:03:16.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 0x7fe7dc156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:16.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7dc144810 0x7fe7dc143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:16.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc15a2e0 0x7fe7dc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:16.085 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc159be0 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.086 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc145950 con 0x7fe7dc144810 2021-11-21T22:03:16.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc145bf0 con 0x7fe7dc0abce0 2021-11-21T22:03:16.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc15a2e0 0x7fe7dc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:16.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc15a2e0 0x7fe7dc15a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54486/0 (socket says 172.21.2.116:54486) 2021-11-21T22:03:16.087 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 -- 172.21.2.116:0/2856865869 learned_addr learned my addr 172.21.2.116:0/2856865869 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:16.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 -- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 msgr2=0x7fe7dc156690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:16.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 0x7fe7dc156690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:16.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 -- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7dc144810 msgr2=0x7fe7dc143510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:16.088 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7dc144810 0x7fe7dc143510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:16.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 -- 172.21.2.116:0/2856865869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c400b1e0 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7dbfff700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc15a2e0 0x7fe7dc15a770 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7fe7d4003310 tx=0x7fe7d4009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:16.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7d8ff9700 1 -- 172.21.2.116:0/2856865869 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7d4001500 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.089 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7d8ff9700 1 -- 172.21.2.116:0/2856865869 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d4001660 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc159e40 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7d8ff9700 1 -- 172.21.2.116:0/2856865869 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7d40019c0 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc145640 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.090 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7dc146300 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.091 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7d8ff9700 1 -- 172.21.2.116:0/2856865869 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe7d4018030 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7d8ff9700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe7c0053610 0x7fe7c0055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:16.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7d8ff9700 1 -- 172.21.2.116:0/2856865869 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe7d406b010 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.092 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.071+0000 7fe7db7fe700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe7c0053610 0x7fe7c0055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:16.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.075+0000 7fe7db7fe700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe7c0053610 0x7fe7c0055ad0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fe7dc0b2a30 tx=0x7fe7c4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:16.093 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.075+0000 7fe7d8ff9700 1 -- 172.21.2.116:0/2856865869 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe7d4042550 con 0x7fe7dc15a2e0 2021-11-21T22:03:16.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:16.159+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fe7dc159fd0 con 0x7fe7dc15a2e0 2021-11-21T22:03:17.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7d8ff9700 1 -- 172.21.2.116:0/2856865869 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v159) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe7d4016030 con 0x7fe7dc15a2e0 2021-11-21T22:03:17.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe7c0053610 msgr2=0x7fe7c0055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:17.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe7c0053610 0x7fe7c0055ad0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fe7dc0b2a30 tx=0x7fe7c4015000).stop 2021-11-21T22:03:17.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc15a2e0 msgr2=0x7fe7dc15a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:17.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc15a2e0 0x7fe7dc15a770 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7fe7d4003310 tx=0x7fe7d4009120).stop 2021-11-21T22:03:17.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 shutdown_connections 2021-11-21T22:03:17.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe7c0053610 0x7fe7c0055ad0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:17.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7dc15a2e0 0x7fe7dc15a770 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:17.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7dc144810 0x7fe7dc143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:17.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 --2- 172.21.2.116:0/2856865869 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe7dc0abce0 0x7fe7dc156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:17.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 >> 172.21.2.116:0/2856865869 conn(0x7fe7dc05bd20 msgr2=0x7fe7dc141870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:17.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 shutdown_connections 2021-11-21T22:03:17.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.171+0000 7fe7e1c92700 1 -- 172.21.2.116:0/2856865869 wait complete. 2021-11-21T22:03:17.182 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' created 2021-11-21T22:03:17.184 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-11-21T22:03:17.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 Processor -- start 2021-11-21T22:03:17.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- start start 2021-11-21T22:03:17.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8720144810 0x7f87201478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:17.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200ac600 0x7f87200aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:17.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8720147e80 con 0x7f87200abce0 2021-11-21T22:03:17.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87200b1490 con 0x7f8720144810 2021-11-21T22:03:17.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87200b1fd0 con 0x7f87200ac600 2021-11-21T22:03:17.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8720144810 0x7f87201478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:17.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f8720144810 0x7f87201478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54492/0 (socket says 172.21.2.116:54492) 2021-11-21T22:03:17.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 -- 172.21.2.116:0/2002014404 learned_addr learned my addr 172.21.2.116:0/2002014404 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:17.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 -- 172.21.2.116:0/2002014404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200ac600 msgr2=0x7f87200aca90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:17.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 --2- 172.21.2.116:0/2002014404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200ac600 0x7f87200aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:17.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 -- 172.21.2.116:0/2002014404 >> v1:172.21.2.117:6789/0 conn(0x7f87200abce0 legacy=0x7f87200b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:17.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 -- 172.21.2.116:0/2002014404 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200af5a0 con 0x7f8720144810 2021-11-21T22:03:17.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8724960700 1 --2- 172.21.2.116:0/2002014404 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8720144810 0x7f87201478c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f87200b4e00 tx=0x7f8708006eb0).ready entity=mon.0 client_cookie=565459e81e651436 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:17.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f871f7fe700 1 -- 172.21.2.116:0/2002014404 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870800d570 con 0x7f8720144810 2021-11-21T22:03:17.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f871f7fe700 1 -- 172.21.2.116:0/2002014404 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708000e90 con 0x7f8720144810 2021-11-21T22:03:17.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f871f7fe700 1 -- 172.21.2.116:0/2002014404 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870800d8c0 con 0x7f8720144810 2021-11-21T22:03:17.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- 172.21.2.116:0/2002014404 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8720144810 msgr2=0x7f87201478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:17.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2002014404 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8720144810 0x7f87201478c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f87200b4e00 tx=0x7f8708006eb0).stop 2021-11-21T22:03:17.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- 172.21.2.116:0/2002014404 shutdown_connections 2021-11-21T22:03:17.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2002014404 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200ac600 0x7f87200aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:17.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2002014404 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8720144810 0x7f87201478c0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:17.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- 172.21.2.116:0/2002014404 >> 172.21.2.116:0/2002014404 conn(0x7f872005bd20 msgr2=0x7f872005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:17.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- 172.21.2.116:0/2002014404 shutdown_connections 2021-11-21T22:03:17.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- 172.21.2.116:0/2002014404 wait complete. 2021-11-21T22:03:17.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 Processor -- start 2021-11-21T22:03:17.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- start start 2021-11-21T22:03:17.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200abce0 0x7f87200b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:17.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87200ac600 0x7f8720139280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:17.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8720144810 0x7f87201464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:17.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87200b3980 con 0x7f87200ac600 2021-11-21T22:03:17.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87200b3160 con 0x7f8720144810 2021-11-21T22:03:17.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f8726bc4700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87200b3bb0 con 0x7f87200abce0 2021-11-21T22:03:17.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f871ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87200ac600 0x7f8720139280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:17.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f871ffff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87200ac600 0x7f8720139280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54500/0 (socket says 172.21.2.116:54500) 2021-11-21T22:03:17.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.235+0000 7f871ffff700 1 -- 172.21.2.116:0/2323248833 learned_addr learned my addr 172.21.2.116:0/2323248833 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:17.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871ffff700 1 -- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200abce0 msgr2=0x7f87200b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:17.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871ffff700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200abce0 0x7f87200b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:17.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871ffff700 1 -- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8720144810 msgr2=0x7f87201464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:17.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871ffff700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8720144810 0x7f87201464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:17.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871ffff700 1 -- 172.21.2.116:0/2323248833 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870800b1e0 con 0x7f87200ac600 2021-11-21T22:03:17.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871ffff700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87200ac600 0x7f8720139280 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f8710002fb0 tx=0x7f8710006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:17.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871dffb700 1 -- 172.21.2.116:0/2323248833 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8710013460 con 0x7f87200ac600 2021-11-21T22:03:17.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871dffb700 1 -- 172.21.2.116:0/2323248833 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87100135c0 con 0x7f87200ac600 2021-11-21T22:03:17.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87200b3e50 con 0x7f87200ac600 2021-11-21T22:03:17.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871dffb700 1 -- 172.21.2.116:0/2323248833 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8710013920 con 0x7f87200ac600 2021-11-21T22:03:17.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8720142930 con 0x7f87200ac600 2021-11-21T22:03:17.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87200af060 con 0x7f87200ac600 2021-11-21T22:03:17.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871dffb700 1 -- 172.21.2.116:0/2323248833 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f871000f070 con 0x7f87200ac600 2021-11-21T22:03:17.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871dffb700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8704053690 0x7f8704055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:17.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f871dffb700 1 -- 172.21.2.116:0/2323248833 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f871006b1f0 con 0x7f87200ac600 2021-11-21T22:03:17.254 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f8724960700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8704053690 0x7f8704055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:17.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.239+0000 7f8724960700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8704053690 0x7f8704055b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f87200b4e00 tx=0x7f8708015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:17.256 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.243+0000 7f871dffb700 1 -- 172.21.2.116:0/2323248833 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8710042520 con 0x7f87200ac600 2021-11-21T22:03:17.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:17.327+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f87200573d0 con 0x7f87200ac600 2021-11-21T22:03:18.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.135+0000 7f871dffb700 1 -- 172.21.2.116:0/2323248833 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v160) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8710045b40 con 0x7f87200ac600 2021-11-21T22:03:18.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.135+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8704053690 msgr2=0x7f8704055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.135+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8704053690 0x7f8704055b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f87200b4e00 tx=0x7f8708015000).stop 2021-11-21T22:03:18.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.135+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87200ac600 msgr2=0x7f8720139280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.135+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87200ac600 0x7f8720139280 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f8710002fb0 tx=0x7f8710006df0).stop 2021-11-21T22:03:18.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 shutdown_connections 2021-11-21T22:03:18.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8720144810 0x7f87201464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:18.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8704053690 0x7f8704055b50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f87200ac600 0x7f8720139280 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 --2- 172.21.2.116:0/2323248833 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f87200abce0 0x7f87200b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:18.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 >> 172.21.2.116:0/2323248833 conn(0x7f872005bd20 msgr2=0x7f87200b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 shutdown_connections 2021-11-21T22:03:18.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.139+0000 7f8726bc4700 1 -- 172.21.2.116:0/2323248833 wait complete. 2021-11-21T22:03:18.144 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' created 2021-11-21T22:03:18.148 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-11-21T22:03:18.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7d19ca700 1 Processor -- start 2021-11-21T22:03:18.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7d19ca700 1 -- start start 2021-11-21T22:03:18.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7d19ca700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd7cc062610 0x7fd7cc0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7d19ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc0b10e0 0x7fd7cc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7d19ca700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc05cb80 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7d19ca700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc0a94c0 con 0x7fd7cc062610 2021-11-21T22:03:18.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7d19ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b1590 con 0x7fd7cc0b10e0 2021-11-21T22:03:18.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7ca7fc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd7cc062610 0x7fd7cc0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7ca7fc700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd7cc062610 0x7fd7cc0adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:55676/0 (socket says 172.21.2.116:55676) 2021-11-21T22:03:18.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7ca7fc700 1 -- 172.21.2.116:0/3921560507 learned_addr learned my addr 172.21.2.116:0/3921560507 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:18.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7cb7fe700 1 --2- 172.21.2.116:0/3921560507 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc0b10e0 0x7fd7cc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7cb7fe700 1 -- 172.21.2.116:0/3921560507 >> v2:172.21.2.117:3300/0 conn(0x7fd7cc062610 msgr2=0x7fd7cc0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7cb7fe700 1 --2- 172.21.2.116:0/3921560507 >> v2:172.21.2.117:3300/0 conn(0x7fd7cc062610 0x7fd7cc0adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7c9ffb700 1 -- 172.21.2.116:0/3921560507 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179034666 0 0) 0x7fd7cc05cb80 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7cb7fe700 1 -- 172.21.2.116:0/3921560507 >> v1:172.21.2.117:6789/0 conn(0x7fd7cc0b0d00 legacy=0x7fd7cc1430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7cb7fe700 1 -- 172.21.2.116:0/3921560507 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0af320 con 0x7fd7cc0b10e0 2021-11-21T22:03:18.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7ca7fc700 1 --2- 172.21.2.116:0/3921560507 >> v2:172.21.2.117:3300/0 conn(0x7fd7cc062610 0x7fd7cc0adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:03:18.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7cb7fe700 1 --2- 172.21.2.116:0/3921560507 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc0b10e0 0x7fd7cc0b22a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd7c40032f0 tx=0x7fd7c40091d0).ready entity=mon.2 client_cookie=8a4c38c27b164920 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7c9ffb700 1 -- 172.21.2.116:0/3921560507 <== mon.2 v2:172.21.2.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c400b6c0 con 0x7fd7cc0b10e0 2021-11-21T22:03:18.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7c9ffb700 1 -- 172.21.2.116:0/3921560507 <== mon.2 v2:172.21.2.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c400b820 con 0x7fd7cc0b10e0 2021-11-21T22:03:18.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.199+0000 7fd7c9ffb700 1 -- 172.21.2.116:0/3921560507 <== mon.2 v2:172.21.2.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c400b990 con 0x7fd7cc0b10e0 2021-11-21T22:03:18.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/3921560507 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc0b10e0 msgr2=0x7fd7cc0b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/3921560507 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc0b10e0 0x7fd7cc0b22a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd7c40032f0 tx=0x7fd7c40091d0).stop 2021-11-21T22:03:18.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/3921560507 shutdown_connections 2021-11-21T22:03:18.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/3921560507 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc0b10e0 0x7fd7cc0b22a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/3921560507 >> v2:172.21.2.117:3300/0 conn(0x7fd7cc062610 0x7fd7cc0adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/3921560507 >> 172.21.2.116:0/3921560507 conn(0x7fd7cc05bd20 msgr2=0x7fd7cc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/3921560507 shutdown_connections 2021-11-21T22:03:18.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/3921560507 wait complete. 2021-11-21T22:03:18.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 Processor -- start 2021-11-21T22:03:18.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- start start 2021-11-21T22:03:18.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc062610 0x7fd7cc159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7cc0b0d00 0x7fd7cc159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7cc0b10e0 0x7fd7cc136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0aaee0 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0643b0 con 0x7fd7cc0b10e0 2021-11-21T22:03:18.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc064650 con 0x7fd7cc062610 2021-11-21T22:03:18.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7cc0b0d00 0x7fd7cc159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7cc0b0d00 0x7fd7cc159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54514/0 (socket says 172.21.2.116:54514) 2021-11-21T22:03:18.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 -- 172.21.2.116:0/2811845739 learned_addr learned my addr 172.21.2.116:0/2811845739 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:18.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7caffd700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc062610 0x7fd7cc159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7cb7fe700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7cc0b10e0 0x7fd7cc136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 -- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc062610 msgr2=0x7fd7cc159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc062610 0x7fd7cc159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 -- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7cc0b10e0 msgr2=0x7fd7cc136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7cc0b10e0 0x7fd7cc136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 -- 172.21.2.116:0/2811845739 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b400b1e0 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7ca7fc700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7cc0b0d00 0x7fd7cc159e50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7fd7b400c000 tx=0x7fd7b4006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7c3fff700 1 -- 172.21.2.116:0/2811845739 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b4012600 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7c3fff700 1 -- 172.21.2.116:0/2811845739 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4012760 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7c3fff700 1 -- 172.21.2.116:0/2811845739 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b4012ac0 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c4009470 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc0b4a00 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc0af5c0 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7c3fff700 1 -- 172.21.2.116:0/2811845739 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd7b4012c20 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.203+0000 7fd7c3fff700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7b0053600 0x7fd7b0055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.207+0000 7fd7c3fff700 1 -- 172.21.2.116:0/2811845739 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fd7b406ba00 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.207+0000 7fd7caffd700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7b0053600 0x7fd7b0055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.207+0000 7fd7caffd700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7b0053600 0x7fd7b0055ac0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fd7bc00b000 tx=0x7fd7bc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.207+0000 7fd7c3fff700 1 -- 172.21.2.116:0/2811845739 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7b4042bd0 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.322 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.319+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd7cc05db70 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.327 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7c3fff700 1 -- 172.21.2.116:0/2811845739 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v76) v1 ==== 189+0+0 (secure 0 0 0) 0x7fd7b40461f0 con 0x7fd7cc0b0d00 2021-11-21T22:03:18.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7b0053600 msgr2=0x7fd7b0055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7b0053600 0x7fd7b0055ac0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fd7bc00b000 tx=0x7fd7bc009040).stop 2021-11-21T22:03:18.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7cc0b0d00 msgr2=0x7fd7cc159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.329 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7cc0b0d00 0x7fd7cc159e50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7fd7b400c000 tx=0x7fd7b4006eb0).stop 2021-11-21T22:03:18.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 shutdown_connections 2021-11-21T22:03:18.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd7cc0b10e0 0x7fd7cc136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd7b0053600 0x7fd7b0055ac0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd7cc0b0d00 0x7fd7cc159e50 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 --2- 172.21.2.116:0/2811845739 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd7cc062610 0x7fd7cc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 >> 172.21.2.116:0/2811845739 conn(0x7fd7cc05bd20 msgr2=0x7fd7cc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 shutdown_connections 2021-11-21T22:03:18.331 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.323+0000 7fd7d19ca700 1 -- 172.21.2.116:0/2811845739 wait complete. 2021-11-21T22:03:18.332 INFO:teuthology.orchestra.run.gibba016.stderr:new fs with metadata pool 29 and data pool 30 2021-11-21T22:03:18.335 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:18.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 Processor -- start 2021-11-21T22:03:18.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- start start 2021-11-21T22:03:18.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed0067850 0x7f8ed00b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed00b29b0 con 0x7f8ed01443a0 2021-11-21T22:03:18.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed0067c30 con 0x7f8ed00b7070 2021-11-21T22:03:18.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00ab720 con 0x7f8ed0067850 2021-11-21T22:03:18.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8eced9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f8ed01443a0 0x7f8ed0144830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38252/0 (socket says 172.21.2.116:38252) 2021-11-21T22:03:18.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8eced9d700 1 -- 172.21.2.116:0/3890942488 learned_addr learned my addr 172.21.2.116:0/3890942488 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:18.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794029767 0 0) 0x7f8ed00b29b0 con 0x7f8ed01443a0 2021-11-21T22:03:18.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb0003260 con 0x7f8ed01443a0 2021-11-21T22:03:18.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461477669 0 0) 0x7f8eb0003260 con 0x7f8ed01443a0 2021-11-21T22:03:18.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed0067850 msgr2=0x7f8ed00b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ece59c700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed0067850 0x7f8ed00b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed0067850 0x7f8ed00b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 >> v1:172.21.2.117:6789/0 conn(0x7f8ed00b7070 legacy=0x7f8ed00ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00b7910 con 0x7f8ed01443a0 2021-11-21T22:03:18.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f8ec80019e0 con 0x7f8ed01443a0 2021-11-21T22:03:18.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 >> v1:172.21.2.116:6789/0 conn(0x7f8ed01443a0 legacy=0x7f8ed0144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0003800 0x7f8eb0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8eb0004500 0x7f8eb0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8eb0005070 0x7f8eb000fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00ab720 con 0x7f8eb0004500 2021-11-21T22:03:18.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00102b0 con 0x7f8eb0003800 2021-11-21T22:03:18.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0010470 con 0x7f8eb0005070 2021-11-21T22:03:18.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ece59c700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed0067850 0x7f8ed00b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:18.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecdd9b700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8eb0004500 0x7f8eb0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8eced9d700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8eb0005070 0x7f8eb000fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ece59c700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0003800 0x7f8eb0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecdd9b700 1 -- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8eb0005070 msgr2=0x7f8eb000fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecdd9b700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8eb0005070 0x7f8eb000fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecdd9b700 1 -- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0003800 msgr2=0x7f8eb0003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecdd9b700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0003800 0x7f8eb0003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecdd9b700 1 -- 172.21.2.116:0/3890942488 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0003260 con 0x7f8eb0004500 2021-11-21T22:03:18.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecdd9b700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8eb0004500 0x7f8eb0004980 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7f8eb8001fa0 tx=0x7f8eb8007a00).ready entity=mon.0 client_cookie=63074083561d5cb0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb8007070 con 0x7f8eb0004500 2021-11-21T22:03:18.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb80071d0 con 0x7f8eb0004500 2021-11-21T22:03:18.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ecd59a700 1 -- 172.21.2.116:0/3890942488 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb8006510 con 0x7f8eb0004500 2021-11-21T22:03:18.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- 172.21.2.116:0/3890942488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8eb0004500 msgr2=0x7f8eb0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 --2- 172.21.2.116:0/3890942488 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8eb0004500 0x7f8eb0004980 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7f8eb8001fa0 tx=0x7f8eb8007a00).stop 2021-11-21T22:03:18.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8eced9d700 1 -- 172.21.2.116:0/3890942488 reap_dead start 2021-11-21T22:03:18.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- 172.21.2.116:0/3890942488 shutdown_connections 2021-11-21T22:03:18.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- 172.21.2.116:0/3890942488 >> 172.21.2.116:0/3890942488 conn(0x7f8ed0064d60 msgr2=0x7f8ed0065170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- 172.21.2.116:0/3890942488 shutdown_connections 2021-11-21T22:03:18.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.383+0000 7f8ed49b5700 1 -- 172.21.2.116:0/3890942488 wait complete. 2021-11-21T22:03:18.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 Processor -- start 2021-11-21T22:03:18.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 -- start start 2021-11-21T22:03:18.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0004500 0x7f8ed00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ed00a12f0 0x7f8ed01517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed009ecf0 0x7f8ed014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0150e50 con 0x7f8ed00a12f0 2021-11-21T22:03:18.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed009f270 con 0x7f8eb0004500 2021-11-21T22:03:18.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0151080 con 0x7f8ed009ecf0 2021-11-21T22:03:18.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ed00a12f0 0x7f8ed01517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ed00a12f0 0x7f8ed01517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54540/0 (socket says 172.21.2.116:54540) 2021-11-21T22:03:18.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 -- 172.21.2.116:0/569433038 learned_addr learned my addr 172.21.2.116:0/569433038 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:18.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ece59c700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0004500 0x7f8ed00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 -- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed009ecf0 msgr2=0x7f8ed014d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed009ecf0 0x7f8ed014d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:18.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 -- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0004500 msgr2=0x7f8ed00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0004500 0x7f8ed00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 -- 172.21.2.116:0/569433038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed0151320 con 0x7f8ed00a12f0 2021-11-21T22:03:18.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ece59c700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0004500 0x7f8ed00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:18.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ecdd9b700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ed00a12f0 0x7f8ed01517c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7f8eb8001fa0 tx=0x7f8eb8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ec77fe700 1 -- 172.21.2.116:0/569433038 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb8014810 con 0x7f8ed00a12f0 2021-11-21T22:03:18.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ec77fe700 1 -- 172.21.2.116:0/569433038 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb8018070 con 0x7f8ed00a12f0 2021-11-21T22:03:18.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed0151cb0 con 0x7f8ed00a12f0 2021-11-21T22:03:18.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ec77fe700 1 -- 172.21.2.116:0/569433038 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb8014b60 con 0x7f8ed00a12f0 2021-11-21T22:03:18.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed014dd70 con 0x7f8ed00a12f0 2021-11-21T22:03:18.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed01514b0 con 0x7f8ed00a12f0 2021-11-21T22:03:18.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ec77fe700 1 -- 172.21.2.116:0/569433038 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f8eb801d070 con 0x7f8ed00a12f0 2021-11-21T22:03:18.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ec77fe700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8eb40536d0 0x7f8eb4055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ec77fe700 1 -- 172.21.2.116:0/569433038 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8eb806cb80 con 0x7f8ed00a12f0 2021-11-21T22:03:18.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ece59c700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8eb40536d0 0x7f8eb4055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.387+0000 7f8ece59c700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8eb40536d0 0x7f8eb4055b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f8ec0009000 tx=0x7f8ec0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.391+0000 7f8ec77fe700 1 -- 172.21.2.116:0/569433038 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8eb8043ce0 con 0x7f8ed00a12f0 2021-11-21T22:03:18.479 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.475+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ed00573d0 con 0x7f8ed00a12f0 2021-11-21T22:03:18.480 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.475+0000 7f8ec77fe700 1 -- 172.21.2.116:0/569433038 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f8eb801a020 con 0x7f8ed00a12f0 2021-11-21T22:03:18.480 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:18.481 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":161,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:18.322563+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:03:16.171372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-11-21T22:03:17.340772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:18.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8eb40536d0 msgr2=0x7f8eb4055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8eb40536d0 0x7f8eb4055b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f8ec0009000 tx=0x7f8ec0008040).stop 2021-11-21T22:03:18.482 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ed00a12f0 msgr2=0x7f8ed01517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ed00a12f0 0x7f8ed01517c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7f8eb8001fa0 tx=0x7f8eb8015000).stop 2021-11-21T22:03:18.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 shutdown_connections 2021-11-21T22:03:18.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8ed009ecf0 0x7f8ed014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:18.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8eb40536d0 0x7f8eb4055b90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.483 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f8ed00a12f0 0x7f8ed01517c0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 --2- 172.21.2.116:0/569433038 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f8eb0004500 0x7f8ed00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 >> 172.21.2.116:0/569433038 conn(0x7f8ed0064d60 msgr2=0x7f8ed00ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 shutdown_connections 2021-11-21T22:03:18.484 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.479+0000 7f8ed49b5700 1 -- 172.21.2.116:0/569433038 wait complete. 2021-11-21T22:03:18.489 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:18.552 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 Processor -- start 2021-11-21T22:03:18.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- start start 2021-11-21T22:03:18.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640a86b0 0x7f6a6414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a640a6e80 con 0x7f6a640a6aa0 2021-11-21T22:03:18.554 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6414a7b0 con 0x7f6a64062410 2021-11-21T22:03:18.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a8bc0 con 0x7f6a640a86b0 2021-11-21T22:03:18.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f6a640a6aa0 0x7f6a641470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38270/0 (socket says 172.21.2.116:38270) 2021-11-21T22:03:18.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 -- 172.21.2.116:0/1245897646 learned_addr learned my addr 172.21.2.116:0/1245897646 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:18.555 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611036410 0 0) 0x7f6a640a6e80 con 0x7f6a640a6aa0 2021-11-21T22:03:18.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a44003260 con 0x7f6a640a6aa0 2021-11-21T22:03:18.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369808866 0 0) 0x7f6a44003260 con 0x7f6a640a6aa0 2021-11-21T22:03:18.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a627fc700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640a86b0 0x7f6a6414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.556 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640a86b0 msgr2=0x7f6a6414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640a86b0 0x7f6a6414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 >> v1:172.21.2.117:6789/0 conn(0x7f6a64062410 legacy=0x7f6a641443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6414aa10 con 0x7f6a640a6aa0 2021-11-21T22:03:18.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f6a54001680 con 0x7f6a640a6aa0 2021-11-21T22:03:18.557 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 >> v1:172.21.2.116:6789/0 conn(0x7f6a640a6aa0 legacy=0x7f6a641470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a44003800 0x7f6a44003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44004500 0x7f6a44004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640a86b0 0x7f6a4400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.558 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a8bc0 con 0x7f6a44003800 2021-11-21T22:03:18.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44005290 con 0x7f6a44004500 2021-11-21T22:03:18.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4400bc40 con 0x7f6a640a86b0 2021-11-21T22:03:18.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a44003800 0x7f6a44003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a627fc700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44004500 0x7f6a44004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.559 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 -- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640a86b0 msgr2=0x7f6a4400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640a86b0 0x7f6a4400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:18.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 -- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44004500 msgr2=0x7f6a44004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44004500 0x7f6a44004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 -- 172.21.2.116:0/1245897646 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a44003260 con 0x7f6a44003800 2021-11-21T22:03:18.560 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a627fc700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44004500 0x7f6a44004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:18.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a62ffd700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a44003800 0x7f6a44003c00 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7f6a64061b50 tx=0x7f6a54001d80).ready entity=mon.0 client_cookie=3c2cff1feb8fe046 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a54007170 con 0x7f6a44003800 2021-11-21T22:03:18.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a540072d0 con 0x7f6a44003800 2021-11-21T22:03:18.561 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a61ffb700 1 -- 172.21.2.116:0/1245897646 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5400c8f0 con 0x7f6a44003800 2021-11-21T22:03:18.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- 172.21.2.116:0/1245897646 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a44003800 msgr2=0x7f6a44003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 --2- 172.21.2.116:0/1245897646 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a44003800 0x7f6a44003c00 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7f6a64061b50 tx=0x7f6a54001d80).stop 2021-11-21T22:03:18.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a637fe700 1 -- 172.21.2.116:0/1245897646 reap_dead start 2021-11-21T22:03:18.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- 172.21.2.116:0/1245897646 shutdown_connections 2021-11-21T22:03:18.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- 172.21.2.116:0/1245897646 >> 172.21.2.116:0/1245897646 conn(0x7f6a6405bd20 msgr2=0x7f6a6405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- 172.21.2.116:0/1245897646 shutdown_connections 2021-11-21T22:03:18.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.539+0000 7f6a694c3700 1 -- 172.21.2.116:0/1245897646 wait complete. 2021-11-21T22:03:18.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 Processor -- start 2021-11-21T22:03:18.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 -- start start 2021-11-21T22:03:18.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44003800 0x7f6a64159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a64147580 0x7f6a640aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640aaeb0 0x7f6a64144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64146b10 con 0x7f6a64147580 2021-11-21T22:03:18.564 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6415a310 con 0x7f6a44003800 2021-11-21T22:03:18.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640aac60 con 0x7f6a640aaeb0 2021-11-21T22:03:18.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a64147580 0x7f6a640aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a64147580 0x7f6a640aa770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54560/0 (socket says 172.21.2.116:54560) 2021-11-21T22:03:18.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 -- 172.21.2.116:0/1014166615 learned_addr learned my addr 172.21.2.116:0/1014166615 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:18.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a62ffd700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44003800 0x7f6a64159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a637fe700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640aaeb0 0x7f6a64144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 -- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640aaeb0 msgr2=0x7f6a64144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640aaeb0 0x7f6a64144f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 -- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44003800 msgr2=0x7f6a64159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44003800 0x7f6a64159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 -- 172.21.2.116:0/1014166615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a640a8ea0 con 0x7f6a64147580 2021-11-21T22:03:18.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a627fc700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a64147580 0x7f6a640aa770 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f6a640ab320 tx=0x7f6a4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a5bfff700 1 -- 172.21.2.116:0/1014166615 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c016070 con 0x7f6a64147580 2021-11-21T22:03:18.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a5bfff700 1 -- 172.21.2.116:0/1014166615 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c000e90 con 0x7f6a64147580 2021-11-21T22:03:18.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a5bfff700 1 -- 172.21.2.116:0/1014166615 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c0117e0 con 0x7f6a64147580 2021-11-21T22:03:18.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6415a580 con 0x7f6a64147580 2021-11-21T22:03:18.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a64145400 con 0x7f6a64147580 2021-11-21T22:03:18.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a640a9030 con 0x7f6a64147580 2021-11-21T22:03:18.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a5bfff700 1 -- 172.21.2.116:0/1014166615 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6a4c011940 con 0x7f6a64147580 2021-11-21T22:03:18.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a5bfff700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a48053640 0x7f6a48055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a5bfff700 1 -- 172.21.2.116:0/1014166615 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6a4c06b9a0 con 0x7f6a64147580 2021-11-21T22:03:18.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.543+0000 7f6a62ffd700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a48053640 0x7f6a48055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.547+0000 7f6a62ffd700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a48053640 0x7f6a48055b00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f6a54019000 tx=0x7f6a54015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.547+0000 7f6a5bfff700 1 -- 172.21.2.116:0/1014166615 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a4c042b00 con 0x7f6a64147580 2021-11-21T22:03:18.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.627+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a640b4c80 con 0x7f6a64147580 2021-11-21T22:03:18.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.627+0000 7f6a5bfff700 1 -- 172.21.2.116:0/1014166615 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f6a4c046120 con 0x7f6a64147580 2021-11-21T22:03:18.632 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:18.633 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":161,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:18.322563+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:03:16.171372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-11-21T22:03:17.340772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:18.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a48053640 msgr2=0x7f6a48055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a48053640 0x7f6a48055b00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f6a54019000 tx=0x7f6a54015000).stop 2021-11-21T22:03:18.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a64147580 msgr2=0x7f6a640aa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a64147580 0x7f6a640aa770 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f6a640ab320 tx=0x7f6a4c008040).stop 2021-11-21T22:03:18.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 shutdown_connections 2021-11-21T22:03:18.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6a640aaeb0 0x7f6a64144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6a48053640 0x7f6a48055b00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.635 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6a64147580 0x7f6a640aa770 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 --2- 172.21.2.116:0/1014166615 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6a44003800 0x7f6a64159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 >> 172.21.2.116:0/1014166615 conn(0x7f6a6405bd20 msgr2=0x7f6a641441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 shutdown_connections 2021-11-21T22:03:18.636 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.631+0000 7f6a694c3700 1 -- 172.21.2.116:0/1014166615 wait complete. 2021-11-21T22:03:18.640 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-11-21T22:03:18.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8809e2d700 1 Processor -- start 2021-11-21T22:03:18.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8809e2d700 1 -- start start 2021-11-21T22:03:18.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8809e2d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88040abce0 0x7f88041478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8809e2d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f88040ac600 0x7f88040acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8809e2d700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8804147e80 con 0x7f8804144810 2021-11-21T22:03:18.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8809e2d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040b1490 con 0x7f88040ac600 2021-11-21T22:03:18.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8809e2d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040b1fd0 con 0x7f88040abce0 2021-11-21T22:03:18.703 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f88040ac600 0x7f88040acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8802ffd700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88040abce0 0x7f88041478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f88040ac600 0x7f88040acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.117:3300/0 says I am v2:172.21.2.116:55734/0 (socket says 172.21.2.116:55734) 2021-11-21T22:03:18.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 -- 172.21.2.116:0/1828898707 learned_addr learned my addr 172.21.2.116:0/1828898707 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:18.704 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 -- 172.21.2.116:0/1828898707 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88040abce0 msgr2=0x7f88041478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:18.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 --2- 172.21.2.116:0/1828898707 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88040abce0 0x7f88041478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 -- 172.21.2.116:0/1828898707 >> v1:172.21.2.117:6789/0 conn(0x7f8804144810 legacy=0x7f88040b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 -- 172.21.2.116:0/1828898707 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040af5a0 con 0x7f88040ac600 2021-11-21T22:03:18.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.691+0000 7f8803fff700 1 --2- 172.21.2.116:0/1828898707 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 0x7f88040acab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f87fc0032f0 tx=0x7f87fc009290).ready entity=mon.1 client_cookie=e02b640a80c0029d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.705 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88027fc700 1 -- 172.21.2.116:0/1828898707 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87fc00b8a0 con 0x7f88040ac600 2021-11-21T22:03:18.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88027fc700 1 -- 172.21.2.116:0/1828898707 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc00ba00 con 0x7f88040ac600 2021-11-21T22:03:18.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88027fc700 1 -- 172.21.2.116:0/1828898707 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87fc00bb70 con 0x7f88040ac600 2021-11-21T22:03:18.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/1828898707 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 msgr2=0x7f88040acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.706 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 --2- 172.21.2.116:0/1828898707 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 0x7f88040acab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f87fc0032f0 tx=0x7f87fc009290).stop 2021-11-21T22:03:18.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/1828898707 shutdown_connections 2021-11-21T22:03:18.707 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 --2- 172.21.2.116:0/1828898707 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 0x7f88040acab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 --2- 172.21.2.116:0/1828898707 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f88040abce0 0x7f88041478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:18.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/1828898707 >> 172.21.2.116:0/1828898707 conn(0x7f880405bd20 msgr2=0x7f880405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:18.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/1828898707 shutdown_connections 2021-11-21T22:03:18.709 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/1828898707 wait complete. 2021-11-21T22:03:18.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 Processor -- start 2021-11-21T22:03:18.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- start start 2021-11-21T22:03:18.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88040abce0 0x7f880414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.710 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 0x7f880415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8804144810 0x7f880413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804159a90 con 0x7f88040abce0 2021-11-21T22:03:18.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880414b060 con 0x7f88040ac600 2021-11-21T22:03:18.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8804159cc0 con 0x7f8804144810 2021-11-21T22:03:18.711 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88040abce0 0x7f880414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88040abce0 0x7f880414a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54570/0 (socket says 172.21.2.116:54570) 2021-11-21T22:03:18.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 -- 172.21.2.116:0/400162453 learned_addr learned my addr 172.21.2.116:0/400162453 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:18.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 -- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8804144810 msgr2=0x7f880413bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8804144810 0x7f880413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:18.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 -- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 msgr2=0x7f880415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:18.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 0x7f880415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:18.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 -- 172.21.2.116:0/400162453 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc009530 con 0x7f88040abce0 2021-11-21T22:03:18.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f88037fe700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88040abce0 0x7f880414a9c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f88040b2a30 tx=0x7f87f4007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.713 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8800ff9700 1 -- 172.21.2.116:0/400162453 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f4007030 con 0x7f88040abce0 2021-11-21T22:03:18.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8800ff9700 1 -- 172.21.2.116:0/400162453 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4007190 con 0x7f88040abce0 2021-11-21T22:03:18.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8800ff9700 1 -- 172.21.2.116:0/400162453 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f4006510 con 0x7f88040abce0 2021-11-21T22:03:18.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8804159f60 con 0x7f88040abce0 2021-11-21T22:03:18.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880413c4c0 con 0x7f88040abce0 2021-11-21T22:03:18.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.695+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88040af060 con 0x7f88040abce0 2021-11-21T22:03:18.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.699+0000 7f8800ff9700 1 -- 172.21.2.116:0/400162453 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f87f4010030 con 0x7f88040abce0 2021-11-21T22:03:18.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.699+0000 7f8800ff9700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87e80535d0 0x7f87e8055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:18.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.699+0000 7f8800ff9700 1 -- 172.21.2.116:0/400162453 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f87f406b5c0 con 0x7f88040abce0 2021-11-21T22:03:18.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.699+0000 7f8802ffd700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87e80535d0 0x7f87e8055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:18.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.699+0000 7f8802ffd700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87e80535d0 0x7f87e8055a90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f87ec00c000 tx=0x7f87ec006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:18.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.699+0000 7f8800ff9700 1 -- 172.21.2.116:0/400162453 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87f4042610 con 0x7f88040abce0 2021-11-21T22:03:18.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:18.811+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f88040573d0 con 0x7f88040abce0 2021-11-21T22:03:19.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8800ff9700 1 -- 172.21.2.116:0/400162453 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7f87f4045c30 con 0x7f88040abce0 2021-11-21T22:03:19.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87e80535d0 msgr2=0x7f87e8055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87e80535d0 0x7f87e8055a90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f87ec00c000 tx=0x7f87ec006df0).stop 2021-11-21T22:03:19.343 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88040abce0 msgr2=0x7f880414a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88040abce0 0x7f880414a9c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f88040b2a30 tx=0x7f87f4007c00).stop 2021-11-21T22:03:19.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 shutdown_connections 2021-11-21T22:03:19.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f8804144810 0x7f880413bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f88040ac600 0x7f880415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.344 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f87e80535d0 0x7f87e8055a90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 --2- 172.21.2.116:0/400162453 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f88040abce0 0x7f880414a9c0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 >> 172.21.2.116:0/400162453 conn(0x7f880405bd20 msgr2=0x7f880405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 shutdown_connections 2021-11-21T22:03:19.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.339+0000 7f8809e2d700 1 -- 172.21.2.116:0/400162453 wait complete. 2021-11-21T22:03:19.349 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:19.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.395+0000 7fe1fb130700 1 Processor -- start 2021-11-21T22:03:19.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.395+0000 7fe1fb130700 1 -- start start 2021-11-21T22:03:19.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe1f40a6aa0 0x7fe1f414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f4062410 0x7fe1f41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f40a6e80 con 0x7fe1f40a86b0 2021-11-21T22:03:19.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f40a8bc0 con 0x7fe1f40a6aa0 2021-11-21T22:03:19.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f414a7b0 con 0x7fe1f4062410 2021-11-21T22:03:19.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe1f40a6aa0 0x7fe1f414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fe1f40a6aa0 0x7fe1f414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54578/0 (socket says 172.21.2.116:54578) 2021-11-21T22:03:19.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3564801587 learned_addr learned my addr 172.21.2.116:0/3564801587 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3564801587 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f4062410 msgr2=0x7fe1f41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:19.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- 172.21.2.116:0/3564801587 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f4062410 0x7fe1f41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3564801587 >> v1:172.21.2.117:6789/0 conn(0x7fe1f40a86b0 legacy=0x7fe1f41470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:19.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3564801587 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f414aa10 con 0x7fe1f40a6aa0 2021-11-21T22:03:19.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- 172.21.2.116:0/3564801587 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f40a6aa0 0x7fe1f414b150 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7fe1f4061ab0 tx=0x7fe1e4006eb0).ready entity=mon.0 client_cookie=ac80e4b8524f44f8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f37fe700 1 -- 172.21.2.116:0/3564801587 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1e400d570 con 0x7fe1f40a6aa0 2021-11-21T22:03:19.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f37fe700 1 -- 172.21.2.116:0/3564801587 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4000e90 con 0x7fe1f40a6aa0 2021-11-21T22:03:19.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f37fe700 1 -- 172.21.2.116:0/3564801587 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1e400d8c0 con 0x7fe1f40a6aa0 2021-11-21T22:03:19.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- 172.21.2.116:0/3564801587 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f40a6aa0 msgr2=0x7fe1f414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3564801587 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f40a6aa0 0x7fe1f414b150 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7fe1f4061ab0 tx=0x7fe1e4006eb0).stop 2021-11-21T22:03:19.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- 172.21.2.116:0/3564801587 shutdown_connections 2021-11-21T22:03:19.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3564801587 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f4062410 0x7fe1f41443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3564801587 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f40a6aa0 0x7fe1f414b150 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- 172.21.2.116:0/3564801587 >> 172.21.2.116:0/3564801587 conn(0x7fe1f405bd20 msgr2=0x7fe1f405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- 172.21.2.116:0/3564801587 shutdown_connections 2021-11-21T22:03:19.416 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- 172.21.2.116:0/3564801587 wait complete. 2021-11-21T22:03:19.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 Processor -- start 2021-11-21T22:03:19.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- start start 2021-11-21T22:03:19.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f4062410 0x7fe1f413f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1f40a6aa0 0x7fe1f4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f40a86b0 0x7fe1f413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f4157400 con 0x7fe1f4062410 2021-11-21T22:03:19.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40a92e0 con 0x7fe1f40a6aa0 2021-11-21T22:03:19.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40a9580 con 0x7fe1f40a86b0 2021-11-21T22:03:19.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f4062410 0x7fe1f413f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f4062410 0x7fe1f413f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54584/0 (socket says 172.21.2.116:54584) 2021-11-21T22:03:19.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3568383814 learned_addr learned my addr 172.21.2.116:0/3568383814 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f40a86b0 msgr2=0x7fe1f413e0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:19.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f40a86b0 0x7fe1f413e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.422 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1f40a6aa0 msgr2=0x7fe1f4156f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:19.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1f40a6aa0 0x7fe1f4156f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 -- 172.21.2.116:0/3568383814 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e400b1e0 con 0x7fe1f4062410 2021-11-21T22:03:19.423 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f8ecc700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f4062410 0x7fe1f413f370 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fe1f4061ab0 tx=0x7fe1e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f1ffb700 1 -- 172.21.2.116:0/3568383814 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1e4006b80 con 0x7fe1f4062410 2021-11-21T22:03:19.424 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f1ffb700 1 -- 172.21.2.116:0/3568383814 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e400de10 con 0x7fe1f4062410 2021-11-21T22:03:19.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f40a9820 con 0x7fe1f4062410 2021-11-21T22:03:19.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1f1ffb700 1 -- 172.21.2.116:0/3568383814 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1e401e460 con 0x7fe1f4062410 2021-11-21T22:03:19.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.399+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f40ab820 con 0x7fe1f4062410 2021-11-21T22:03:19.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.403+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f40573d0 con 0x7fe1f4062410 2021-11-21T22:03:19.426 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.403+0000 7fe1f1ffb700 1 -- 172.21.2.116:0/3568383814 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe1e4025030 con 0x7fe1f4062410 2021-11-21T22:03:19.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.403+0000 7fe1f1ffb700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1d4053580 0x7fe1d4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.427 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.403+0000 7fe1f1ffb700 1 -- 172.21.2.116:0/3568383814 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe1e406ccb0 con 0x7fe1f4062410 2021-11-21T22:03:19.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.403+0000 7fe1f3fff700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1d4053580 0x7fe1d4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.403+0000 7fe1f3fff700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1d4053580 0x7fe1d4055a40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fe1e8002ef0 tx=0x7fe1e8006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.428 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.403+0000 7fe1f1ffb700 1 -- 172.21.2.116:0/3568383814 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1e4043d00 con 0x7fe1f4062410 2021-11-21T22:03:19.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.511+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1f405db70 con 0x7fe1f4062410 2021-11-21T22:03:19.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.511+0000 7fe1f1ffb700 1 -- 172.21.2.116:0/3568383814 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+2486 (secure 0 0 0) 0x7fe1e4023070 con 0x7fe1f4062410 2021-11-21T22:03:19.518 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:19.518 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":77,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1070380430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1070380430},{"type":"v1","addr":"172.21.2.117:6835","nonce":1070380430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75},{"gid":8796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/27380863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":27380863},{"type":"v1","addr":"172.21.2.116:6833","nonce":27380863}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:03:18.322670+0000","modified":"2021-11-21T22:03:19.327237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8799":{"gid":8799,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/403152574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":403152574},{"type":"v1","addr":"172.21.2.116:6835","nonce":403152574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-11-21T22:03:19.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1d4053580 msgr2=0x7fe1d4055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1d4053580 0x7fe1d4055a40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fe1e8002ef0 tx=0x7fe1e8006890).stop 2021-11-21T22:03:19.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f4062410 msgr2=0x7fe1f413f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f4062410 0x7fe1f413f370 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fe1f4061ab0 tx=0x7fe1e4015000).stop 2021-11-21T22:03:19.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 shutdown_connections 2021-11-21T22:03:19.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe1f40a86b0 0x7fe1f413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe1f40a6aa0 0x7fe1f4156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.522 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1d4053580 0x7fe1d4055a40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 --2- 172.21.2.116:0/3568383814 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe1f4062410 0x7fe1f413f370 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.523 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 >> 172.21.2.116:0/3568383814 conn(0x7fe1f405bd20 msgr2=0x7fe1f41434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 shutdown_connections 2021-11-21T22:03:19.524 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.515+0000 7fe1fb130700 1 -- 172.21.2.116:0/3568383814 wait complete. 2021-11-21T22:03:19.524 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 77 2021-11-21T22:03:19.527 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:19.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 Processor -- start 2021-11-21T22:03:19.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- start start 2021-11-21T22:03:19.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 0x7f858013bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f85800aff60 0x7f85800b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85800b5a40 con 0x7f85800b1660 2021-11-21T22:03:19.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85800aef20 con 0x7f85800aff60 2021-11-21T22:03:19.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85800ae540 con 0x7f85800b5660 2021-11-21T22:03:19.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f85800aff60 0x7f85800b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f85800aff60 0x7f85800b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54594/0 (socket says 172.21.2.116:54594) 2021-11-21T22:03:19.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 -- 172.21.2.116:0/1420270676 learned_addr learned my addr 172.21.2.116:0/1420270676 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 -- 172.21.2.116:0/1420270676 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 msgr2=0x7f858013bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:19.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 --2- 172.21.2.116:0/1420270676 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 0x7f858013bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 -- 172.21.2.116:0/1420270676 >> v1:172.21.2.116:6789/0 conn(0x7f85800b1660 legacy=0x7f85800b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 -- 172.21.2.116:0/1420270676 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800ae7e0 con 0x7f85800aff60 2021-11-21T22:03:19.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858596f700 1 --2- 172.21.2.116:0/1420270676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85800b0e30 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f85780032f0 tx=0x7f8578008310).ready entity=mon.0 client_cookie=1224eddea465f214 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f8573fff700 1 -- 172.21.2.116:0/1420270676 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8578010070 con 0x7f85800aff60 2021-11-21T22:03:19.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f8573fff700 1 -- 172.21.2.116:0/1420270676 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578015470 con 0x7f85800aff60 2021-11-21T22:03:19.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f8573fff700 1 -- 172.21.2.116:0/1420270676 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85780157d0 con 0x7f85800aff60 2021-11-21T22:03:19.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- 172.21.2.116:0/1420270676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 msgr2=0x7f85800b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- 172.21.2.116:0/1420270676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85800b0e30 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f85780032f0 tx=0x7f8578008310).stop 2021-11-21T22:03:19.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- 172.21.2.116:0/1420270676 shutdown_connections 2021-11-21T22:03:19.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- 172.21.2.116:0/1420270676 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85800b0e30 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- 172.21.2.116:0/1420270676 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 0x7f858013bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- 172.21.2.116:0/1420270676 >> 172.21.2.116:0/1420270676 conn(0x7f858005f730 msgr2=0x7f8580063080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- 172.21.2.116:0/1420270676 shutdown_connections 2021-11-21T22:03:19.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- 172.21.2.116:0/1420270676 wait complete. 2021-11-21T22:03:19.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 Processor -- start 2021-11-21T22:03:19.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- start start 2021-11-21T22:03:19.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85801361b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85800b1660 0x7f85800a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 0x7f85800a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858013ba50 con 0x7f85800aff60 2021-11-21T22:03:19.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800a9ec0 con 0x7f85800b1660 2021-11-21T22:03:19.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f85873d2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858013aac0 con 0x7f85800b5660 2021-11-21T22:03:19.599 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858516e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85801361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858516e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85801361b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54598/0 (socket says 172.21.2.116:54598) 2021-11-21T22:03:19.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.575+0000 7f858516e700 1 -- 172.21.2.116:0/3122227318 learned_addr learned my addr 172.21.2.116:0/3122227318 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858516e700 1 -- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 msgr2=0x7f85800a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858516e700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 0x7f85800a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858516e700 1 -- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85800b1660 msgr2=0x7f85800a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858516e700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85800b1660 0x7f85800a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858516e700 1 -- 172.21.2.116:0/3122227318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85780085b0 con 0x7f85800aff60 2021-11-21T22:03:19.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858516e700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85801361b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f857c002f70 tx=0x7f857c006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85727fc700 1 -- 172.21.2.116:0/3122227318 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c013460 con 0x7f85800aff60 2021-11-21T22:03:19.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85727fc700 1 -- 172.21.2.116:0/3122227318 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c0135c0 con 0x7f85800aff60 2021-11-21T22:03:19.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8580139c40 con 0x7f85800aff60 2021-11-21T22:03:19.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85800abb20 con 0x7f85800aff60 2021-11-21T22:03:19.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85727fc700 1 -- 172.21.2.116:0/3122227318 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c013920 con 0x7f85800aff60 2021-11-21T22:03:19.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85800abf60 con 0x7f85800aff60 2021-11-21T22:03:19.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85727fc700 1 -- 172.21.2.116:0/3122227318 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f857c01a060 con 0x7f85800aff60 2021-11-21T22:03:19.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85727fc700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8560053720 0x7f8560055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f85727fc700 1 -- 172.21.2.116:0/3122227318 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f857c06b300 con 0x7f85800aff60 2021-11-21T22:03:19.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858496d700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8560053720 0x7f8560055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.579+0000 7f858496d700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8560053720 0x7f8560055be0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f8580063a90 tx=0x7f85740077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.583+0000 7f85727fc700 1 -- 172.21.2.116:0/3122227318 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f857c042460 con 0x7f85800aff60 2021-11-21T22:03:19.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.663+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85800573d0 con 0x7f85800aff60 2021-11-21T22:03:19.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.663+0000 7f85727fc700 1 -- 172.21.2.116:0/3122227318 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f857c018020 con 0x7f85800aff60 2021-11-21T22:03:19.669 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:19.669 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":162,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:19.325611+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:03:16.171372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-11-21T22:03:17.340772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:19.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8560053720 msgr2=0x7f8560055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8560053720 0x7f8560055be0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f8580063a90 tx=0x7f85740077d0).stop 2021-11-21T22:03:19.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 msgr2=0x7f85801361b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85801361b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f857c002f70 tx=0x7f857c006e20).stop 2021-11-21T22:03:19.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 shutdown_connections 2021-11-21T22:03:19.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f85800b5660 0x7f85800a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f85800b1660 0x7f85800a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f8560053720 0x7f8560055be0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 --2- 172.21.2.116:0/3122227318 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f85800aff60 0x7f85801361b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 >> 172.21.2.116:0/3122227318 conn(0x7f858005f730 msgr2=0x7f8580139040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 shutdown_connections 2021-11-21T22:03:19.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.667+0000 7f85873d2700 1 -- 172.21.2.116:0/3122227318 wait complete. 2021-11-21T22:03:19.677 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:03:19.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc30452700 1 Processor -- start 2021-11-21T22:03:19.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc30452700 1 -- start start 2021-11-21T22:03:19.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc30452700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc280b24a0 0x7fdc280b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc30452700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fdc280af360 0x7fdc280ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc30452700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc280b00d0 con 0x7fdc280aeb70 2021-11-21T22:03:19.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc30452700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc280adba0 con 0x7fdc280b24a0 2021-11-21T22:03:19.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc30452700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc280af7a0 con 0x7fdc280af360 2021-11-21T22:03:19.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc2e1ee700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc280b24a0 0x7fdc280b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc2e1ee700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdc280b24a0 0x7fdc280b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54606/0 (socket says 172.21.2.116:54606) 2021-11-21T22:03:19.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc2e1ee700 1 -- 172.21.2.116:0/1436432994 learned_addr learned my addr 172.21.2.116:0/1436432994 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc2e1ee700 1 -- 172.21.2.116:0/1436432994 >> v2:172.21.2.117:3300/0 conn(0x7fdc280af360 msgr2=0x7fdc280ae680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:19.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc2e1ee700 1 --2- 172.21.2.116:0/1436432994 >> v2:172.21.2.117:3300/0 conn(0x7fdc280af360 0x7fdc280ae680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.727+0000 7fdc2e1ee700 1 -- 172.21.2.116:0/1436432994 >> v1:172.21.2.117:6789/0 conn(0x7fdc280aeb70 legacy=0x7fdc28069e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:19.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e1ee700 1 -- 172.21.2.116:0/1436432994 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280b2880 con 0x7fdc280b24a0 2021-11-21T22:03:19.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e1ee700 1 --2- 172.21.2.116:0/1436432994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc280b1a40 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7fdc280630d0 tx=0x7fdc1c006eb0).ready entity=mon.0 client_cookie=365bd270333b2114 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2d1ec700 1 -- 172.21.2.116:0/1436432994 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc1c00d5c0 con 0x7fdc280b24a0 2021-11-21T22:03:19.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2d1ec700 1 -- 172.21.2.116:0/1436432994 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c000e90 con 0x7fdc280b24a0 2021-11-21T22:03:19.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2d1ec700 1 -- 172.21.2.116:0/1436432994 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc1c00d910 con 0x7fdc280b24a0 2021-11-21T22:03:19.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/1436432994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 msgr2=0x7fdc280b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 --2- 172.21.2.116:0/1436432994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc280b1a40 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7fdc280630d0 tx=0x7fdc1c006eb0).stop 2021-11-21T22:03:19.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/1436432994 shutdown_connections 2021-11-21T22:03:19.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 --2- 172.21.2.116:0/1436432994 >> v2:172.21.2.117:3300/0 conn(0x7fdc280af360 0x7fdc280ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 --2- 172.21.2.116:0/1436432994 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc280b1a40 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/1436432994 >> 172.21.2.116:0/1436432994 conn(0x7fdc2805fd00 msgr2=0x7fdc280600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/1436432994 shutdown_connections 2021-11-21T22:03:19.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/1436432994 wait complete. 2021-11-21T22:03:19.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 Processor -- start 2021-11-21T22:03:19.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- start start 2021-11-21T22:03:19.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc280aeb70 0x7fdc280b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc280af360 0x7fdc28141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc281423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28066560 con 0x7fdc280b24a0 2021-11-21T22:03:19.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280657c0 con 0x7fdc280aeb70 2021-11-21T22:03:19.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28065a60 con 0x7fdc280af360 2021-11-21T22:03:19.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc281423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc281423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54614/0 (socket says 172.21.2.116:54614) 2021-11-21T22:03:19.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 -- 172.21.2.116:0/4099259375 learned_addr learned my addr 172.21.2.116:0/4099259375 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 -- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc280af360 msgr2=0x7fdc28141ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:19.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc280af360 0x7fdc28141ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 -- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc280aeb70 msgr2=0x7fdc280b4600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:19.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc280aeb70 0x7fdc280b4600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 -- 172.21.2.116:0/4099259375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c00b1e0 con 0x7fdc280b24a0 2021-11-21T22:03:19.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc2e9ef700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc281423e0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7fdc20003330 tx=0x7fdc200091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc1b7fe700 1 -- 172.21.2.116:0/4099259375 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc20001520 con 0x7fdc280b24a0 2021-11-21T22:03:19.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc1b7fe700 1 -- 172.21.2.116:0/4099259375 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc20001680 con 0x7fdc280b24a0 2021-11-21T22:03:19.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc28065d00 con 0x7fdc280b24a0 2021-11-21T22:03:19.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc1b7fe700 1 -- 172.21.2.116:0/4099259375 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc200019e0 con 0x7fdc280b24a0 2021-11-21T22:03:19.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc28137f10 con 0x7fdc280b24a0 2021-11-21T22:03:19.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc280573d0 con 0x7fdc280b24a0 2021-11-21T22:03:19.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc1b7fe700 1 -- 172.21.2.116:0/4099259375 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdc20018030 con 0x7fdc280b24a0 2021-11-21T22:03:19.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc1b7fe700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc080534f0 0x7fdc080559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.731+0000 7fdc1b7fe700 1 -- 172.21.2.116:0/4099259375 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdc2006b330 con 0x7fdc280b24a0 2021-11-21T22:03:19.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.735+0000 7fdc2e1ee700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc080534f0 0x7fdc080559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.735+0000 7fdc2e1ee700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc080534f0 0x7fdc080559b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fdc280630d0 tx=0x7fdc1c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.735+0000 7fdc1b7fe700 1 -- 172.21.2.116:0/4099259375 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc20042540 con 0x7fdc280b24a0 2021-11-21T22:03:19.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.847+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fdc2805f2d0 con 0x7fdc280b24a0 2021-11-21T22:03:19.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc1b7fe700 1 -- 172.21.2.116:0/4099259375 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7fdc20045b60 con 0x7fdc280b24a0 2021-11-21T22:03:19.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc080534f0 msgr2=0x7fdc080559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc080534f0 0x7fdc080559b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fdc280630d0 tx=0x7fdc1c015000).stop 2021-11-21T22:03:19.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 msgr2=0x7fdc281423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc281423e0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7fdc20003330 tx=0x7fdc200091a0).stop 2021-11-21T22:03:19.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 shutdown_connections 2021-11-21T22:03:19.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fdc080534f0 0x7fdc080559b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdc280b24a0 0x7fdc281423e0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdc280af360 0x7fdc28141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 --2- 172.21.2.116:0/4099259375 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdc280aeb70 0x7fdc280b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 >> 172.21.2.116:0/4099259375 conn(0x7fdc2805fd00 msgr2=0x7fdc2813f010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 shutdown_connections 2021-11-21T22:03:19.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.851+0000 7fdc30452700 1 -- 172.21.2.116:0/4099259375 wait complete. 2021-11-21T22:03:19.859 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.0 2021-11-21T22:03:19.862 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-11-21T22:03:19.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fc20c7700 1 Processor -- start 2021-11-21T22:03:19.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fc20c7700 1 -- start start 2021-11-21T22:03:19.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fc20c7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0fbc0b11c0 0x7f0fbc147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.920 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fc20c7700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0fbc143e60 0x7f0fbc1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fc20c7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc0b1940 con 0x7f0fbc0a8670 2021-11-21T22:03:19.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fc20c7700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0b15a0 con 0x7f0fbc143e60 2021-11-21T22:03:19.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fc20c7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0b1b70 con 0x7f0fbc0b11c0 2021-11-21T22:03:19.921 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fbbfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0fbc143e60 0x7f0fbc1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fbbfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0fbc143e60 0x7f0fbc1442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54620/0 (socket says 172.21.2.116:54620) 2021-11-21T22:03:19.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2156582718 learned_addr learned my addr 172.21.2.116:0/2156582718 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2156582718 >> v2:172.21.2.117:3300/0 conn(0x7f0fbc0b11c0 msgr2=0x7f0fbc147230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:19.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fbbfff700 1 --2- 172.21.2.116:0/2156582718 >> v2:172.21.2.117:3300/0 conn(0x7f0fbc0b11c0 0x7f0fbc147230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.922 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2156582718 >> v1:172.21.2.117:6789/0 conn(0x7f0fbc0a8670 legacy=0x7f0fbc147720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:19.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.911+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2156582718 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0b1e10 con 0x7f0fbc143e60 2021-11-21T22:03:19.923 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 --2- 172.21.2.116:0/2156582718 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc1442d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f0fbc147180 tx=0x7f0fb4009290).ready entity=mon.0 client_cookie=89b0c8c5c3d9725e server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fba7fc700 1 -- 172.21.2.116:0/2156582718 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb400b830 con 0x7f0fbc143e60 2021-11-21T22:03:19.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fba7fc700 1 -- 172.21.2.116:0/2156582718 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb400b990 con 0x7f0fbc143e60 2021-11-21T22:03:19.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fba7fc700 1 -- 172.21.2.116:0/2156582718 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb400bb40 con 0x7f0fbc143e60 2021-11-21T22:03:19.924 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2156582718 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 msgr2=0x7f0fbc1442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2156582718 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc1442d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f0fbc147180 tx=0x7f0fb4009290).stop 2021-11-21T22:03:19.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2156582718 shutdown_connections 2021-11-21T22:03:19.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2156582718 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc1442d0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:19.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2156582718 >> v2:172.21.2.117:3300/0 conn(0x7f0fbc0b11c0 0x7f0fbc147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.925 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2156582718 >> 172.21.2.116:0/2156582718 conn(0x7f0fbc05bd20 msgr2=0x7f0fbc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:19.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2156582718 shutdown_connections 2021-11-21T22:03:19.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2156582718 wait complete. 2021-11-21T22:03:19.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 Processor -- start 2021-11-21T22:03:19.926 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- start start 2021-11-21T22:03:19.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fbc0a8670 0x7f0fbc0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fbc0b11c0 0x7f0fbc0643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc0648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0b2f30 con 0x7f0fbc143e60 2021-11-21T22:03:19.927 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0b2c90 con 0x7f0fbc0a8670 2021-11-21T22:03:19.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0b3500 con 0x7f0fbc0b11c0 2021-11-21T22:03:19.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc0648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc0648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54630/0 (socket says 172.21.2.116:54630) 2021-11-21T22:03:19.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2593761933 learned_addr learned my addr 172.21.2.116:0/2593761933 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:19.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fbc0b11c0 msgr2=0x7f0fbc0643e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:19.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fbc0b11c0 0x7f0fbc0643e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fbc0a8670 msgr2=0x7f0fbc0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:19.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fbc0a8670 0x7f0fbc0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:19.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 -- 172.21.2.116:0/2593761933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4009530 con 0x7f0fbc143e60 2021-11-21T22:03:19.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fbbfff700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc0648d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f0fbc147180 tx=0x7f0fb4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fb8ff9700 1 -- 172.21.2.116:0/2593761933 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb400b830 con 0x7f0fbc143e60 2021-11-21T22:03:19.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fb8ff9700 1 -- 172.21.2.116:0/2593761933 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb40107a0 con 0x7f0fbc143e60 2021-11-21T22:03:19.930 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fb8ff9700 1 -- 172.21.2.116:0/2593761933 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb4010b00 con 0x7f0fbc143e60 2021-11-21T22:03:19.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc0b4dd0 con 0x7f0fbc143e60 2021-11-21T22:03:19.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc0b4370 con 0x7f0fbc143e60 2021-11-21T22:03:19.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc0b20b0 con 0x7f0fbc143e60 2021-11-21T22:03:19.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.915+0000 7f0fb8ff9700 1 -- 172.21.2.116:0/2593761933 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0fb4027020 con 0x7f0fbc143e60 2021-11-21T22:03:19.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.919+0000 7f0fb8ff9700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f9c0536f0 0x7f0f9c055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:19.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.919+0000 7f0fb8ff9700 1 -- 172.21.2.116:0/2593761933 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0fb406cdd0 con 0x7f0fbc143e60 2021-11-21T22:03:19.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.919+0000 7f0fbb7fe700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f9c0536f0 0x7f0f9c055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:19.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.919+0000 7f0fbb7fe700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f9c0536f0 0x7f0f9c055bb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f0fb0002fb0 tx=0x7f0fb00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:19.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:19.919+0000 7f0fb8ff9700 1 -- 172.21.2.116:0/2593761933 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fb4043f30 con 0x7f0fbc143e60 2021-11-21T22:03:20.034 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.031+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0fbc0573d0 con 0x7f0fbc143e60 2021-11-21T22:03:20.039 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.035+0000 7f0fb8ff9700 1 -- 172.21.2.116:0/2593761933 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0fb4015030 con 0x7f0fbc143e60 2021-11-21T22:03:20.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.035+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f9c0536f0 msgr2=0x7f0f9c055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.035+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f9c0536f0 0x7f0f9c055bb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f0fb0002fb0 tx=0x7f0fb00069b0).stop 2021-11-21T22:03:20.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.035+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 msgr2=0x7f0fbc0648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.041 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.035+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc0648d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f0fbc147180 tx=0x7f0fb4016000).stop 2021-11-21T22:03:20.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 shutdown_connections 2021-11-21T22:03:20.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0f9c0536f0 0x7f0f9c055bb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.042 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0fbc143e60 0x7f0fbc0648d0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0fbc0b11c0 0x7f0fbc0643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 --2- 172.21.2.116:0/2593761933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0fbc0a8670 0x7f0fbc0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 >> 172.21.2.116:0/2593761933 conn(0x7f0fbc05bd20 msgr2=0x7f0fbc142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:20.043 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 shutdown_connections 2021-11-21T22:03:20.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.039+0000 7f0fc20c7700 1 -- 172.21.2.116:0/2593761933 wait complete. 2021-11-21T22:03:20.044 INFO:teuthology.orchestra.run.gibba016.stderr:updated caps for client.1 2021-11-21T22:03:20.047 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:20.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 Processor -- start 2021-11-21T22:03:20.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- start start 2021-11-21T22:03:20.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f645c0b24a0 0x7f645c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f645c0aed90 0x7f645c13c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c0b00d0 con 0x7f645c0af360 2021-11-21T22:03:20.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0adba0 con 0x7f645c0aed90 2021-11-21T22:03:20.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0af7a0 con 0x7f645c0b24a0 2021-11-21T22:03:20.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64618ae700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f645c0af360 0x7f645c0ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38358/0 (socket says 172.21.2.116:38358) 2021-11-21T22:03:20.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64618ae700 1 -- 172.21.2.116:0/1398320486 learned_addr learned my addr 172.21.2.116:0/1398320486 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:20.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336916307 0 0) 0x7f645c0b00d0 con 0x7f645c0af360 2021-11-21T22:03:20.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6444003260 con 0x7f645c0af360 2021-11-21T22:03:20.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2060631704 0 0) 0x7f6444003260 con 0x7f645c0af360 2021-11-21T22:03:20.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f645c0b24a0 msgr2=0x7f645c0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:20.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f645c0b24a0 0x7f645c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 >> v2:172.21.2.117:3300/0 conn(0x7f645c0aed90 msgr2=0x7f645c13c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:20.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 --2- 172.21.2.116:0/1398320486 >> v2:172.21.2.117:3300/0 conn(0x7f645c0aed90 0x7f645c13c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0b2880 con 0x7f645c0af360 2021-11-21T22:03:20.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f6458001660 con 0x7f645c0af360 2021-11-21T22:03:20.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 >> v1:172.21.2.116:6789/0 conn(0x7f645c0af360 legacy=0x7f645c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f64440037d0 0x7f6444003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f645c0b24a0 0x7f6444004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6444004bf0 0x7f644400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0adba0 con 0x7f6444004bf0 2021-11-21T22:03:20.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0af7a0 con 0x7f64440037d0 2021-11-21T22:03:20.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6444005320 con 0x7f645c0b24a0 2021-11-21T22:03:20.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6444004bf0 0x7f644400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 -- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f645c0b24a0 msgr2=0x7f6444004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:20.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f645c0b24a0 0x7f6444004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 -- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f64440037d0 msgr2=0x7f6444003c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:20.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f64440037d0 0x7f6444003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 -- 172.21.2.116:0/1398320486 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444003260 con 0x7f6444004bf0 2021-11-21T22:03:20.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6444004bf0 0x7f644400b750 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f64540032f0 tx=0x7f6454006f90).ready entity=mon.0 client_cookie=918b185dbfaf8602 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64540108b0 con 0x7f6444004bf0 2021-11-21T22:03:20.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454010a10 con 0x7f6444004bf0 2021-11-21T22:03:20.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64610ad700 1 -- 172.21.2.116:0/1398320486 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6454010d70 con 0x7f6444004bf0 2021-11-21T22:03:20.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- 172.21.2.116:0/1398320486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6444004bf0 msgr2=0x7f644400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 --2- 172.21.2.116:0/1398320486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6444004bf0 0x7f644400b750 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f64540032f0 tx=0x7f6454006f90).stop 2021-11-21T22:03:20.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f64628b0700 1 -- 172.21.2.116:0/1398320486 reap_dead start 2021-11-21T22:03:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- 172.21.2.116:0/1398320486 shutdown_connections 2021-11-21T22:03:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- 172.21.2.116:0/1398320486 >> 172.21.2.116:0/1398320486 conn(0x7f645c05fd00 msgr2=0x7f645c0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- 172.21.2.116:0/1398320486 shutdown_connections 2021-11-21T22:03:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.135+0000 7f6464313700 1 -- 172.21.2.116:0/1398320486 wait complete. 2021-11-21T22:03:20.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 Processor -- start 2021-11-21T22:03:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 -- start start 2021-11-21T22:03:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6444004bf0 0x7f645c066960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f645c0659d0 0x7f645c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f645c159b30 0x7f645c141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c065ff0 con 0x7f645c159b30 2021-11-21T22:03:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0651f0 con 0x7f645c0659d0 2021-11-21T22:03:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c066220 con 0x7f6444004bf0 2021-11-21T22:03:20.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f645c159b30 0x7f645c141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f645c159b30 0x7f645c141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54650/0 (socket says 172.21.2.116:54650) 2021-11-21T22:03:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 -- 172.21.2.116:0/1814054185 learned_addr learned my addr 172.21.2.116:0/1814054185 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 -- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6444004bf0 msgr2=0x7f645c066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6444004bf0 0x7f645c066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 -- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f645c0659d0 msgr2=0x7f645c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f645c0659d0 0x7f645c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 -- 172.21.2.116:0/1814054185 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0664c0 con 0x7f645c159b30 2021-11-21T22:03:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64628b0700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f645c159b30 0x7f645c141fc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f645c062110 tx=0x7f6454019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f644f7fe700 1 -- 172.21.2.116:0/1814054185 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6454009370 con 0x7f645c159b30 2021-11-21T22:03:20.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f644f7fe700 1 -- 172.21.2.116:0/1814054185 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645401a9e0 con 0x7f645c159b30 2021-11-21T22:03:20.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c0b5010 con 0x7f645c159b30 2021-11-21T22:03:20.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c1540e0 con 0x7f645c159b30 2021-11-21T22:03:20.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f644f7fe700 1 -- 172.21.2.116:0/1814054185 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f645401ab50 con 0x7f645c159b30 2021-11-21T22:03:20.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c066650 con 0x7f645c159b30 2021-11-21T22:03:20.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f644f7fe700 1 -- 172.21.2.116:0/1814054185 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f645401acb0 con 0x7f645c159b30 2021-11-21T22:03:20.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f644f7fe700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f643c0535d0 0x7f643c055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f644f7fe700 1 -- 172.21.2.116:0/1814054185 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f645406c800 con 0x7f645c159b30 2021-11-21T22:03:20.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.139+0000 7f64620af700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f643c0535d0 0x7f643c055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.143+0000 7f64620af700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f643c0535d0 0x7f643c055a90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f6450002f70 tx=0x7f6450006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.143+0000 7f644f7fe700 1 -- 172.21.2.116:0/1814054185 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6454043850 con 0x7f645c159b30 2021-11-21T22:03:20.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f645c0573d0 con 0x7f645c159b30 2021-11-21T22:03:20.255 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f644f7fe700 1 -- 172.21.2.116:0/1814054185 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f6454046e70 con 0x7f645c159b30 2021-11-21T22:03:20.256 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:20.256 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":77,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1070380430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1070380430},{"type":"v1","addr":"172.21.2.117:6835","nonce":1070380430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75},{"gid":8796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/27380863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":27380863},{"type":"v1","addr":"172.21.2.116:6833","nonce":27380863}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:03:18.322670+0000","modified":"2021-11-21T22:03:19.327237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8799":{"gid":8799,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/403152574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":403152574},{"type":"v1","addr":"172.21.2.116:6835","nonce":403152574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-11-21T22:03:20.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f643c0535d0 msgr2=0x7f643c055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.257 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f643c0535d0 0x7f643c055a90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f6450002f70 tx=0x7f6450006a10).stop 2021-11-21T22:03:20.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f645c159b30 msgr2=0x7f645c141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f645c159b30 0x7f645c141fc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f645c062110 tx=0x7f6454019000).stop 2021-11-21T22:03:20.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 shutdown_connections 2021-11-21T22:03:20.258 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f643c0535d0 0x7f643c055a90 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f645c159b30 0x7f645c141fc0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f645c0659d0 0x7f645c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.259 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 --2- 172.21.2.116:0/1814054185 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6444004bf0 0x7f645c066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 >> 172.21.2.116:0/1814054185 conn(0x7f645c05fd00 msgr2=0x7f645c05c810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:20.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 shutdown_connections 2021-11-21T22:03:20.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.251+0000 7f6464313700 1 -- 172.21.2.116:0/1814054185 wait complete. 2021-11-21T22:03:20.260 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 77 2021-11-21T22:03:20.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:03:18.322670+0000', 'modified': '2021-11-21T22:03:19.327237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8799': {'gid': 8799, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/403152574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 403152574}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 403152574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:03:20.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:03:20.566 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:03:20.630 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8799,"want_state":"up:active","state":"up:active","rank_uptime":2.3003991890000002,"mdsmap_epoch":77,"osdmap_epoch":162,"osdmap_epoch_barrier":162,"uptime":6.5957807739999996} 2021-11-21T22:03:20.631 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8799, "mdsmap_epoch": 77, "osdmap_epoch": 162, "osdmap_epoch_barrier": 162, "rank_uptime": 2.300399189, "state": "up:active", "uptime": 6.595780774, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:03:20.632 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:20.681 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 Processor -- start 2021-11-21T22:03:20.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- start start 2021-11-21T22:03:20.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f12400b24a0 0x7f12400b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.682 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12400af360 0x7f12400ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400aeb70 0x7f124013c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12400adba0 con 0x7f12400af360 2021-11-21T22:03:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12400af7a0 con 0x7f12400b24a0 2021-11-21T22:03:20.683 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12400b00d0 con 0x7f12400aeb70 2021-11-21T22:03:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12400af360 0x7f12400ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f12400af360 0x7f12400ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54658/0 (socket says 172.21.2.116:54658) 2021-11-21T22:03:20.684 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 -- 172.21.2.116:0/3586586291 learned_addr learned my addr 172.21.2.116:0/3586586291 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:20.688 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 -- 172.21.2.116:0/3586586291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400aeb70 msgr2=0x7f124013c210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:20.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 --2- 172.21.2.116:0/3586586291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400aeb70 0x7f124013c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 -- 172.21.2.116:0/3586586291 >> v2:172.21.2.117:3300/0 conn(0x7f12400b24a0 msgr2=0x7f12400b1a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:20.689 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 --2- 172.21.2.116:0/3586586291 >> v2:172.21.2.117:3300/0 conn(0x7f12400b24a0 0x7f12400b1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 -- 172.21.2.116:0/3586586291 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12400b2880 con 0x7f12400af360 2021-11-21T22:03:20.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12458e8700 1 --2- 172.21.2.116:0/3586586291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400af360 0x7f12400ae680 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7f12380032f0 tx=0x7f123800b530).ready entity=mon.0 client_cookie=3638dd57c5b4be5b server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12450e7700 1 -- 172.21.2.116:0/3586586291 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123800dd00 con 0x7f12400af360 2021-11-21T22:03:20.690 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12450e7700 1 -- 172.21.2.116:0/3586586291 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123800de60 con 0x7f12400af360 2021-11-21T22:03:20.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12450e7700 1 -- 172.21.2.116:0/3586586291 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1238012640 con 0x7f12400af360 2021-11-21T22:03:20.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- 172.21.2.116:0/3586586291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400af360 msgr2=0x7f12400ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- 172.21.2.116:0/3586586291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400af360 0x7f12400ae680 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7f12380032f0 tx=0x7f123800b530).stop 2021-11-21T22:03:20.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- 172.21.2.116:0/3586586291 shutdown_connections 2021-11-21T22:03:20.691 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- 172.21.2.116:0/3586586291 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400aeb70 0x7f124013c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- 172.21.2.116:0/3586586291 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400af360 0x7f12400ae680 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- 172.21.2.116:0/3586586291 >> v2:172.21.2.117:3300/0 conn(0x7f12400b24a0 0x7f12400b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- 172.21.2.116:0/3586586291 >> 172.21.2.116:0/3586586291 conn(0x7f124005fd00 msgr2=0x7f12400600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:20.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- 172.21.2.116:0/3586586291 shutdown_connections 2021-11-21T22:03:20.692 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- 172.21.2.116:0/3586586291 wait complete. 2021-11-21T22:03:20.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 Processor -- start 2021-11-21T22:03:20.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- start start 2021-11-21T22:03:20.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400aeb70 0x7f12400b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.693 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400af360 0x7f1240066c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12400b24a0 0x7f1240065ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1240067130 con 0x7f12400aeb70 2021-11-21T22:03:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1240141880 con 0x7f12400b24a0 2021-11-21T22:03:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f124834d700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1240141b20 con 0x7f12400af360 2021-11-21T22:03:20.694 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12460e9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400aeb70 0x7f12400b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12460e9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400aeb70 0x7f12400b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54662/0 (socket says 172.21.2.116:54662) 2021-11-21T22:03:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.679+0000 7f12460e9700 1 -- 172.21.2.116:0/3558948131 learned_addr learned my addr 172.21.2.116:0/3558948131 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12460e9700 1 -- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400af360 msgr2=0x7f1240066c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.695 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12460e9700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400af360 0x7f1240066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12460e9700 1 -- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12400b24a0 msgr2=0x7f1240065ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12460e9700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12400b24a0 0x7f1240065ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12460e9700 1 -- 172.21.2.116:0/3558948131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123800b7d0 con 0x7f12400aeb70 2021-11-21T22:03:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12460e9700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400aeb70 0x7f12400b4600 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7f1230000ed0 tx=0x7f1230006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.696 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f122f7fe700 1 -- 172.21.2.116:0/3558948131 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1230012410 con 0x7f12400aeb70 2021-11-21T22:03:20.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f122f7fe700 1 -- 172.21.2.116:0/3558948131 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1230012570 con 0x7f12400aeb70 2021-11-21T22:03:20.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1240141ec0 con 0x7f12400aeb70 2021-11-21T22:03:20.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1240142400 con 0x7f12400aeb70 2021-11-21T22:03:20.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f122f7fe700 1 -- 172.21.2.116:0/3558948131 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12300128d0 con 0x7f12400aeb70 2021-11-21T22:03:20.697 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12400573d0 con 0x7f12400aeb70 2021-11-21T22:03:20.698 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f122f7fe700 1 -- 172.21.2.116:0/3558948131 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f123000f030 con 0x7f12400aeb70 2021-11-21T22:03:20.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f122f7fe700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1220053720 0x7f1220055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f122f7fe700 1 -- 172.21.2.116:0/3558948131 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f123006b4d0 con 0x7f12400aeb70 2021-11-21T22:03:20.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12458e8700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1220053720 0x7f1220055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.699 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.683+0000 7f12458e8700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1220053720 0x7f1220055be0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f12380032f0 tx=0x7f124013eff0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.700 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.687+0000 7f122f7fe700 1 -- 172.21.2.116:0/3558948131 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1230042630 con 0x7f12400aeb70 2021-11-21T22:03:20.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.767+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f124005f2d0 con 0x7f12400aeb70 2021-11-21T22:03:20.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.767+0000 7f122f7fe700 1 -- 172.21.2.116:0/3558948131 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f1230045c50 con 0x7f12400aeb70 2021-11-21T22:03:20.773 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:20.773 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":162,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:19.325611+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:03:16.171372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-11-21T22:03:17.340772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:20.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1220053720 msgr2=0x7f1220055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1220053720 0x7f1220055be0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f12380032f0 tx=0x7f124013eff0).stop 2021-11-21T22:03:20.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400aeb70 msgr2=0x7f12400b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400aeb70 0x7f12400b4600 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7f1230000ed0 tx=0x7f1230006d70).stop 2021-11-21T22:03:20.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 shutdown_connections 2021-11-21T22:03:20.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f12400b24a0 0x7f1240065ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f12400af360 0x7f1240066c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1220053720 0x7f1220055be0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 --2- 172.21.2.116:0/3558948131 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f12400aeb70 0x7f12400b4600 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 >> 172.21.2.116:0/3558948131 conn(0x7f124005fd00 msgr2=0x7f124005c710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:20.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 shutdown_connections 2021-11-21T22:03:20.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.771+0000 7f124834d700 1 -- 172.21.2.116:0/3558948131 wait complete. 2021-11-21T22:03:20.782 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:03:16.171372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:03:17.340772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:03:20.783 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:03:20.839 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.831+0000 7f35afb05700 1 Processor -- start 2021-11-21T22:03:20.840 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.831+0000 7f35afb05700 1 -- start start 2021-11-21T22:03:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35a8067890 0x7f35a80b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80ac5a0 0x7f35a80634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.841 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a8140930 con 0x7f35a80b0ca0 2021-11-21T22:03:20.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a80b1080 con 0x7f35a8067890 2021-11-21T22:03:20.842 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80acd20 con 0x7f35a80ac5a0 2021-11-21T22:03:20.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35a8067890 0x7f35a80b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.843 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f35a8067890 0x7f35a80b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54670/0 (socket says 172.21.2.116:54670) 2021-11-21T22:03:20.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 -- 172.21.2.116:0/4053336675 learned_addr learned my addr 172.21.2.116:0/4053336675 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:20.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 -- 172.21.2.116:0/4053336675 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80ac5a0 msgr2=0x7f35a80634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:20.844 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 --2- 172.21.2.116:0/4053336675 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80ac5a0 0x7f35a80634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 -- 172.21.2.116:0/4053336675 >> v1:172.21.2.117:6789/0 conn(0x7f35a80b0ca0 legacy=0x7f35a8140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:20.845 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 -- 172.21.2.116:0/4053336675 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80acfc0 con 0x7f35a8067890 2021-11-21T22:03:20.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ad0a0700 1 --2- 172.21.2.116:0/4053336675 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a8067890 0x7f35a80b1790 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 rx=0x7f35a8140100 tx=0x7f359c006f40).ready entity=mon.0 client_cookie=dd3dbb670ac15433 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ac89f700 1 -- 172.21.2.116:0/4053336675 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359c00dc30 con 0x7f35a8067890 2021-11-21T22:03:20.846 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ac89f700 1 -- 172.21.2.116:0/4053336675 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c000e90 con 0x7f35a8067890 2021-11-21T22:03:20.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35ac89f700 1 -- 172.21.2.116:0/4053336675 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359c00be90 con 0x7f35a8067890 2021-11-21T22:03:20.847 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- 172.21.2.116:0/4053336675 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a8067890 msgr2=0x7f35a80b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 --2- 172.21.2.116:0/4053336675 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a8067890 0x7f35a80b1790 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 rx=0x7f35a8140100 tx=0x7f359c006f40).stop 2021-11-21T22:03:20.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- 172.21.2.116:0/4053336675 shutdown_connections 2021-11-21T22:03:20.848 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 --2- 172.21.2.116:0/4053336675 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80ac5a0 0x7f35a80634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 --2- 172.21.2.116:0/4053336675 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a8067890 0x7f35a80b1790 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- 172.21.2.116:0/4053336675 >> 172.21.2.116:0/4053336675 conn(0x7f35a805bd20 msgr2=0x7f35a805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:20.849 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- 172.21.2.116:0/4053336675 shutdown_connections 2021-11-21T22:03:20.850 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- 172.21.2.116:0/4053336675 wait complete. 2021-11-21T22:03:20.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 Processor -- start 2021-11-21T22:03:20.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.835+0000 7f35afb05700 1 -- start start 2021-11-21T22:03:20.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35a8067890 0x7f35a80aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.855 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a80ac5a0 0x7f35a81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80b0ca0 0x7f35a81575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a8064950 con 0x7f35a80ac5a0 2021-11-21T22:03:20.856 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a8156a60 con 0x7f35a8067890 2021-11-21T22:03:20.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80b3570 con 0x7f35a80b0ca0 2021-11-21T22:03:20.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a80ac5a0 0x7f35a81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a80ac5a0 0x7f35a81570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54678/0 (socket says 172.21.2.116:54678) 2021-11-21T22:03:20.857 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 -- 172.21.2.116:0/2600722576 learned_addr learned my addr 172.21.2.116:0/2600722576 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:20.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 -- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80b0ca0 msgr2=0x7f35a81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80b0ca0 0x7f35a81575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 -- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35a8067890 msgr2=0x7f35a80aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.858 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35a8067890 0x7f35a80aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 -- 172.21.2.116:0/2600722576 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c00b460 con 0x7f35a80ac5a0 2021-11-21T22:03:20.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad0a0700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a80ac5a0 0x7f35a81570f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f35a80adc90 tx=0x7f35a813fd80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f359affd700 1 -- 172.21.2.116:0/2600722576 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359c01d070 con 0x7f35a80ac5a0 2021-11-21T22:03:20.859 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f359affd700 1 -- 172.21.2.116:0/2600722576 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c010e10 con 0x7f35a80ac5a0 2021-11-21T22:03:20.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f359affd700 1 -- 172.21.2.116:0/2600722576 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359c022600 con 0x7f35a80ac5a0 2021-11-21T22:03:20.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a8142a80 con 0x7f35a80ac5a0 2021-11-21T22:03:20.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a80646b0 con 0x7f35a80ac5a0 2021-11-21T22:03:20.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a80573d0 con 0x7f35a80ac5a0 2021-11-21T22:03:20.860 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f359affd700 1 -- 172.21.2.116:0/2600722576 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f359c029070 con 0x7f35a80ac5a0 2021-11-21T22:03:20.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f359affd700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3588053720 0x7f3588055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:20.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f359affd700 1 -- 172.21.2.116:0/2600722576 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f359c06ce20 con 0x7f35a80ac5a0 2021-11-21T22:03:20.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.839+0000 7f35ad8a1700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3588053720 0x7f3588055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:20.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.843+0000 7f35ad8a1700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3588053720 0x7f3588055be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f35a4002ef0 tx=0x7f35a4006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:20.861 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.843+0000 7f359affd700 1 -- 172.21.2.116:0/2600722576 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f359c043e70 con 0x7f35a80ac5a0 2021-11-21T22:03:20.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.923+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f35a80aeda0 con 0x7f35a80ac5a0 2021-11-21T22:03:20.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.923+0000 7f359affd700 1 -- 172.21.2.116:0/2600722576 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v162) v1 ==== 110+0+10 (secure 0 0 0) 0x7f359c047490 con 0x7f35a80ac5a0 2021-11-21T22:03:20.930 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:03:20.931 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3588053720 msgr2=0x7f3588055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3588053720 0x7f3588055be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f35a4002ef0 tx=0x7f35a4006910).stop 2021-11-21T22:03:20.932 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a80ac5a0 msgr2=0x7f35a81570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:20.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a80ac5a0 0x7f35a81570f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f35a80adc90 tx=0x7f35a813fd80).stop 2021-11-21T22:03:20.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 shutdown_connections 2021-11-21T22:03:20.933 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f35a80b0ca0 0x7f35a81575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f3588053720 0x7f3588055be0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f35a80ac5a0 0x7f35a81570f0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:20.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 --2- 172.21.2.116:0/2600722576 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f35a8067890 0x7f35a80aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:20.935 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 >> 172.21.2.116:0/2600722576 conn(0x7f35a805bd20 msgr2=0x7f35a8141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:20.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 shutdown_connections 2021-11-21T22:03:20.936 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.927+0000 7f35afb05700 1 -- 172.21.2.116:0/2600722576 wait complete. 2021-11-21T22:03:20.937 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:03:20.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 Processor -- start 2021-11-21T22:03:20.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- start start 2021-11-21T22:03:21.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f23980b1270 0x7f23980b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f23980ad580 0x7f23980ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980b0070 con 0x7f23980b6770 2021-11-21T22:03:21.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23980b05c0 con 0x7f23980b1270 2021-11-21T22:03:21.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23980b2230 con 0x7f23980ad580 2021-11-21T22:03:21.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f23980b1270 0x7f23980b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f23980b1270 0x7f23980b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54684/0 (socket says 172.21.2.116:54684) 2021-11-21T22:03:21.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 -- 172.21.2.116:0/4076965174 learned_addr learned my addr 172.21.2.116:0/4076965174 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:21.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 -- 172.21.2.116:0/4076965174 >> v2:172.21.2.117:3300/0 conn(0x7f23980ad580 msgr2=0x7f23980ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:21.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- 172.21.2.116:0/4076965174 >> v2:172.21.2.117:3300/0 conn(0x7f23980ad580 0x7f23980ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 -- 172.21.2.116:0/4076965174 >> v1:172.21.2.116:6789/0 conn(0x7f23980b6770 legacy=0x7f23981394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 -- 172.21.2.116:0/4076965174 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980b24d0 con 0x7f23980b1270 2021-11-21T22:03:21.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- 172.21.2.116:0/4076965174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f23980b6b50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7f238c002f70 tx=0x7f238c006f40).ready entity=mon.0 client_cookie=c01ac35ab9b71d33 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239cb26700 1 -- 172.21.2.116:0/4076965174 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f238c00dc70 con 0x7f23980b1270 2021-11-21T22:03:21.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239cb26700 1 -- 172.21.2.116:0/4076965174 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c000e90 con 0x7f23980b1270 2021-11-21T22:03:21.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239cb26700 1 -- 172.21.2.116:0/4076965174 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f238c00be90 con 0x7f23980b1270 2021-11-21T22:03:21.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- 172.21.2.116:0/4076965174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 msgr2=0x7f23980b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- 172.21.2.116:0/4076965174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f23980b6b50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7f238c002f70 tx=0x7f238c006f40).stop 2021-11-21T22:03:21.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- 172.21.2.116:0/4076965174 shutdown_connections 2021-11-21T22:03:21.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- 172.21.2.116:0/4076965174 >> v2:172.21.2.117:3300/0 conn(0x7f23980ad580 0x7f23980ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- 172.21.2.116:0/4076965174 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f23980b6b50 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- 172.21.2.116:0/4076965174 >> 172.21.2.116:0/4076965174 conn(0x7f2398062040 msgr2=0x7f23980b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- 172.21.2.116:0/4076965174 shutdown_connections 2021-11-21T22:03:21.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- 172.21.2.116:0/4076965174 wait complete. 2021-11-21T22:03:21.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 Processor -- start 2021-11-21T22:03:21.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- start start 2021-11-21T22:03:21.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23980ad580 0x7f239813b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f239813a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23980b6770 0x7f23981374d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398151b30 con 0x7f23980b1270 2021-11-21T22:03:21.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398151dd0 con 0x7f23980b6770 2021-11-21T22:03:21.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239fd8c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980b58b0 con 0x7f23980ad580 2021-11-21T22:03:21.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f239813a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f239813a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54690/0 (socket says 172.21.2.116:54690) 2021-11-21T22:03:21.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 -- 172.21.2.116:0/2251063520 learned_addr learned my addr 172.21.2.116:0/2251063520 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:21.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 -- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23980ad580 msgr2=0x7f239813b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23980ad580 0x7f239813b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 -- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23980b6770 msgr2=0x7f23981374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.987+0000 7f239d327700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23980b6770 0x7f23981374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f239d327700 1 -- 172.21.2.116:0/2251063520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c00b460 con 0x7f23980b1270 2021-11-21T22:03:21.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f239d327700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f239813a430 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f238c002f70 tx=0x7f239813f7c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f238affd700 1 -- 172.21.2.116:0/2251063520 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f238c01d070 con 0x7f23980b1270 2021-11-21T22:03:21.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f238affd700 1 -- 172.21.2.116:0/2251063520 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c022410 con 0x7f23980b1270 2021-11-21T22:03:21.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239813a920 con 0x7f23980b1270 2021-11-21T22:03:21.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239813af50 con 0x7f23980b1270 2021-11-21T22:03:21.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f238affd700 1 -- 172.21.2.116:0/2251063520 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f238c022770 con 0x7f23980b1270 2021-11-21T22:03:21.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239813a0d0 con 0x7f23980b1270 2021-11-21T22:03:21.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f238affd700 1 -- 172.21.2.116:0/2251063520 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f238c029070 con 0x7f23980b1270 2021-11-21T22:03:21.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f238affd700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2378053720 0x7f2378055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f238affd700 1 -- 172.21.2.116:0/2251063520 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f238c06cde0 con 0x7f23980b1270 2021-11-21T22:03:21.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f239db28700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2378053720 0x7f2378055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f239db28700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2378053720 0x7f2378055be0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f2398060900 tx=0x7f2394007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:20.991+0000 7f238affd700 1 -- 172.21.2.116:0/2251063520 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f239813a0d0 con 0x7f23980b1270 2021-11-21T22:03:21.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.075+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f239813b8b0 con 0x7f23980b1270 2021-11-21T22:03:21.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.075+0000 7f238affd700 1 -- 172.21.2.116:0/2251063520 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v162) v1 ==== 92+0+10 (secure 0 0 0) 0x7f239813b8b0 con 0x7f23980b1270 2021-11-21T22:03:21.081 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:21.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2378053720 msgr2=0x7f2378055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2378053720 0x7f2378055be0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f2398060900 tx=0x7f2394007810).stop 2021-11-21T22:03:21.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 msgr2=0x7f239813a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f239813a430 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f238c002f70 tx=0x7f239813f7c0).stop 2021-11-21T22:03:21.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 shutdown_connections 2021-11-21T22:03:21.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f23980b6770 0x7f23981374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2378053720 0x7f2378055be0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f23980b1270 0x7f239813a430 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.083 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 --2- 172.21.2.116:0/2251063520 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f23980ad580 0x7f239813b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 >> 172.21.2.116:0/2251063520 conn(0x7f2398062040 msgr2=0x7f2398138de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 shutdown_connections 2021-11-21T22:03:21.084 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.079+0000 7f239fd8c700 1 -- 172.21.2.116:0/2251063520 wait complete. 2021-11-21T22:03:21.088 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:03:21.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 Processor -- start 2021-11-21T22:03:21.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- start start 2021-11-21T22:03:21.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc94c144810 0x7fc94c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94c147e80 con 0x7fc94c0abce0 2021-11-21T22:03:21.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94c0af5a0 con 0x7fc94c0ac600 2021-11-21T22:03:21.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94c0b1490 con 0x7fc94c144810 2021-11-21T22:03:21.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc94c144810 0x7fc94c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc94c144810 0x7fc94c1478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54696/0 (socket says 172.21.2.116:54696) 2021-11-21T22:03:21.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/38030654 learned_addr learned my addr 172.21.2.116:0/38030654 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:21.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94b7fe700 1 --1- 172.21.2.116:0/38030654 >> v1:172.21.2.116:6789/0 conn(0x7fc94c0abce0 0x7fc94c0b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38420/0 (socket says 172.21.2.116:38420) 2021-11-21T22:03:21.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/38030654 >> v1:172.21.2.117:6789/0 conn(0x7fc94c0ac600 legacy=0x7fc94c0aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:21.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/38030654 >> v1:172.21.2.116:6789/0 conn(0x7fc94c0abce0 legacy=0x7fc94c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/38030654 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c0b1fd0 con 0x7fc94c144810 2021-11-21T22:03:21.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 --2- 172.21.2.116:0/38030654 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c144810 0x7fc94c1478c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7fc94c05d170 tx=0x7fc93c006eb0).ready entity=mon.0 client_cookie=d22640fa52525ff6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94affd700 1 -- 172.21.2.116:0/38030654 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c00d610 con 0x7fc94c144810 2021-11-21T22:03:21.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94affd700 1 -- 172.21.2.116:0/38030654 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c000e90 con 0x7fc94c144810 2021-11-21T22:03:21.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94affd700 1 -- 172.21.2.116:0/38030654 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c00d960 con 0x7fc94c144810 2021-11-21T22:03:21.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- 172.21.2.116:0/38030654 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c144810 msgr2=0x7fc94c1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 --2- 172.21.2.116:0/38030654 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c144810 0x7fc94c1478c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7fc94c05d170 tx=0x7fc93c006eb0).stop 2021-11-21T22:03:21.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- 172.21.2.116:0/38030654 shutdown_connections 2021-11-21T22:03:21.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 --2- 172.21.2.116:0/38030654 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c144810 0x7fc94c1478c0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.150 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- 172.21.2.116:0/38030654 >> 172.21.2.116:0/38030654 conn(0x7fc94c05bd20 msgr2=0x7fc94c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- 172.21.2.116:0/38030654 shutdown_connections 2021-11-21T22:03:21.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- 172.21.2.116:0/38030654 wait complete. 2021-11-21T22:03:21.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 Processor -- start 2021-11-21T22:03:21.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- start start 2021-11-21T22:03:21.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c0abce0 0x7fc94c146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc94c0ac600 0x7fc94c13d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc94c144810 0x7fc94c13d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c0b2ab0 con 0x7fc94c0abce0 2021-11-21T22:03:21.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c147010 con 0x7fc94c0ac600 2021-11-21T22:03:21.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc952645700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c156910 con 0x7fc94c144810 2021-11-21T22:03:21.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c0abce0 0x7fc94c146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c0abce0 0x7fc94c146990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54702/0 (socket says 172.21.2.116:54702) 2021-11-21T22:03:21.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/2271494784 learned_addr learned my addr 172.21.2.116:0/2271494784 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:21.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc94c144810 msgr2=0x7fc94c13d780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:21.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc94c144810 0x7fc94c13d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc94c0ac600 msgr2=0x7fc94c13d290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:21.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc94c0ac600 0x7fc94c13d290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.139+0000 7fc94bfff700 1 -- 172.21.2.116:0/2271494784 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c00b1e0 con 0x7fc94c0abce0 2021-11-21T22:03:21.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc94bfff700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c0abce0 0x7fc94c146990 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7fc94c05d170 tx=0x7fc93c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc9497fa700 1 -- 172.21.2.116:0/2271494784 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c00d290 con 0x7fc94c0abce0 2021-11-21T22:03:21.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc9497fa700 1 -- 172.21.2.116:0/2271494784 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c002f70 con 0x7fc94c0abce0 2021-11-21T22:03:21.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc9497fa700 1 -- 172.21.2.116:0/2271494784 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c01e460 con 0x7fc94c0abce0 2021-11-21T22:03:21.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c144dd0 con 0x7fc94c0abce0 2021-11-21T22:03:21.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c145310 con 0x7fc94c0abce0 2021-11-21T22:03:21.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94c0af060 con 0x7fc94c0abce0 2021-11-21T22:03:21.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc9497fa700 1 -- 172.21.2.116:0/2271494784 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc93c025030 con 0x7fc94c0abce0 2021-11-21T22:03:21.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc9497fa700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc92c053610 0x7fc92c055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc9497fa700 1 -- 172.21.2.116:0/2271494784 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc93c06cde0 con 0x7fc94c0abce0 2021-11-21T22:03:21.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc94b7fe700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc92c053610 0x7fc92c055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc94b7fe700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc92c053610 0x7fc92c055ad0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fc94c0b4e00 tx=0x7fc940007780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.143+0000 7fc9497fa700 1 -- 172.21.2.116:0/2271494784 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc93c043e30 con 0x7fc94c0abce0 2021-11-21T22:03:21.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc94c0573d0 con 0x7fc94c0abce0 2021-11-21T22:03:21.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc9497fa700 1 -- 172.21.2.116:0/2271494784 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc93c0230a0 con 0x7fc94c0abce0 2021-11-21T22:03:21.232 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:21.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc92c053610 msgr2=0x7fc92c055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc92c053610 0x7fc92c055ad0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fc94c0b4e00 tx=0x7fc940007780).stop 2021-11-21T22:03:21.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c0abce0 msgr2=0x7fc94c146990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c0abce0 0x7fc94c146990 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7fc94c05d170 tx=0x7fc93c015000).stop 2021-11-21T22:03:21.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 shutdown_connections 2021-11-21T22:03:21.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc94c144810 0x7fc94c13d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc94c0ac600 0x7fc94c13d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc92c053610 0x7fc92c055ad0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 --2- 172.21.2.116:0/2271494784 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc94c0abce0 0x7fc94c146990 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 >> 172.21.2.116:0/2271494784 conn(0x7fc94c05bd20 msgr2=0x7fc94c149540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 shutdown_connections 2021-11-21T22:03:21.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.227+0000 7fc952645700 1 -- 172.21.2.116:0/2271494784 wait complete. 2021-11-21T22:03:21.239 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:03:21.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca70aae700 1 Processor -- start 2021-11-21T22:03:21.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca70aae700 1 -- start start 2021-11-21T22:03:21.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca70aae700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fca6c144810 0x7fca6c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca70aae700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0abce0 0x7fca6c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca70aae700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6c147e80 con 0x7fca6c0b1db0 2021-11-21T22:03:21.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca70aae700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6c0b1490 con 0x7fca6c144810 2021-11-21T22:03:21.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca70aae700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0af060 con 0x7fca6c0abce0 2021-11-21T22:03:21.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fca6c0b1db0 0x7fca6c0ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38436/0 (socket says 172.21.2.116:38436) 2021-11-21T22:03:21.293 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 -- 172.21.2.116:0/3866983535 learned_addr learned my addr 172.21.2.116:0/3866983535 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:21.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789649815 0 0) 0x7fca6c147e80 con 0x7fca6c0b1db0 2021-11-21T22:03:21.299 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca4c003260 con 0x7fca6c0b1db0 2021-11-21T22:03:21.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6a59c700 1 --2- 172.21.2.116:0/3866983535 >> v2:172.21.2.117:3300/0 conn(0x7fca6c144810 0x7fca6c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca69d9b700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0abce0 0x7fca6c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 563806117 0 0) 0x7fca4c003260 con 0x7fca6c0b1db0 2021-11-21T22:03:21.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0abce0 msgr2=0x7fca6c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:21.300 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0abce0 0x7fca6c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 >> v2:172.21.2.117:3300/0 conn(0x7fca6c144810 msgr2=0x7fca6c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:21.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 --2- 172.21.2.116:0/3866983535 >> v2:172.21.2.117:3300/0 conn(0x7fca6c144810 0x7fca6c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c0af5a0 con 0x7fca6c0b1db0 2021-11-21T22:03:21.301 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fca64001a00 con 0x7fca6c0b1db0 2021-11-21T22:03:21.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 >> v1:172.21.2.116:6789/0 conn(0x7fca6c0b1db0 legacy=0x7fca6c0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca4c0037d0 0x7fca4c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0abce0 0x7fca4c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca4c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.302 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0b1490 con 0x7fca4c0049f0 2021-11-21T22:03:21.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0af060 con 0x7fca4c0037d0 2021-11-21T22:03:21.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c00b8b0 con 0x7fca6c0abce0 2021-11-21T22:03:21.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca4c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 -- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0abce0 msgr2=0x7fca4c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:21.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0abce0 0x7fca4c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 -- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca4c0037d0 msgr2=0x7fca4c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:21.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca4c0037d0 0x7fca4c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 -- 172.21.2.116:0/3866983535 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c003260 con 0x7fca4c0049f0 2021-11-21T22:03:21.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.287+0000 7fca6ad9d700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca4c004e70 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7fca64001710 tx=0x7fca6c0aee70).ready entity=mon.0 client_cookie=5208ead0c6be2e29 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca6400cb70 con 0x7fca4c0049f0 2021-11-21T22:03:21.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6400ccd0 con 0x7fca4c0049f0 2021-11-21T22:03:21.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6959a700 1 -- 172.21.2.116:0/3866983535 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca64004b50 con 0x7fca4c0049f0 2021-11-21T22:03:21.305 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/3866983535 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 msgr2=0x7fca4c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 --2- 172.21.2.116:0/3866983535 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca4c004e70 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7fca64001710 tx=0x7fca6c0aee70).stop 2021-11-21T22:03:21.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6ad9d700 1 -- 172.21.2.116:0/3866983535 reap_dead start 2021-11-21T22:03:21.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/3866983535 shutdown_connections 2021-11-21T22:03:21.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/3866983535 >> 172.21.2.116:0/3866983535 conn(0x7fca6c05bd20 msgr2=0x7fca6c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.306 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/3866983535 shutdown_connections 2021-11-21T22:03:21.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/3866983535 wait complete. 2021-11-21T22:03:21.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 Processor -- start 2021-11-21T22:03:21.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- start start 2021-11-21T22:03:21.307 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca6c0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0b5740 0x7fca6c146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca6c146750 0x7fca6c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0b5310 con 0x7fca4c0049f0 2021-11-21T22:03:21.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c062b40 con 0x7fca6c146750 2021-11-21T22:03:21.308 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0b2e10 con 0x7fca6c0b5740 2021-11-21T22:03:21.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca6c0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca6c0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54724/0 (socket says 172.21.2.116:54724) 2021-11-21T22:03:21.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 -- 172.21.2.116:0/2858136933 learned_addr learned my addr 172.21.2.116:0/2858136933 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:21.309 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 -- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0b5740 msgr2=0x7fca6c146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca69d9b700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0b5740 0x7fca6c146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6ad9d700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca6c146750 0x7fca6c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0b5740 0x7fca6c146260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.310 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 -- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca6c146750 msgr2=0x7fca6c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca6c146750 0x7fca6c159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 -- 172.21.2.116:0/2858136933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c143960 con 0x7fca4c0049f0 2021-11-21T22:03:21.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca6a59c700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca6c0b4830 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7fca5400a000 tx=0x7fca54008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca637fe700 1 -- 172.21.2.116:0/2858136933 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca5400de60 con 0x7fca4c0049f0 2021-11-21T22:03:21.311 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca637fe700 1 -- 172.21.2.116:0/2858136933 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca54000e90 con 0x7fca4c0049f0 2021-11-21T22:03:21.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca637fe700 1 -- 172.21.2.116:0/2858136933 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca540116f0 con 0x7fca4c0049f0 2021-11-21T22:03:21.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c145a00 con 0x7fca4c0049f0 2021-11-21T22:03:21.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c1433a0 con 0x7fca4c0049f0 2021-11-21T22:03:21.312 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c0af300 con 0x7fca4c0049f0 2021-11-21T22:03:21.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca637fe700 1 -- 172.21.2.116:0/2858136933 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fca54019020 con 0x7fca4c0049f0 2021-11-21T22:03:21.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca637fe700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca50053690 0x7fca50055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.291+0000 7fca637fe700 1 -- 172.21.2.116:0/2858136933 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fca5406b1c0 con 0x7fca4c0049f0 2021-11-21T22:03:21.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.295+0000 7fca69d9b700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca50053690 0x7fca50055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.313 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.295+0000 7fca69d9b700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca50053690 0x7fca50055b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fca5c008000 tx=0x7fca5c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.314 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.295+0000 7fca637fe700 1 -- 172.21.2.116:0/2858136933 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca54042210 con 0x7fca4c0049f0 2021-11-21T22:03:21.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca6c0573d0 con 0x7fca4c0049f0 2021-11-21T22:03:21.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca637fe700 1 -- 172.21.2.116:0/2858136933 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+10 (secure 0 0 0) 0x7fca54017020 con 0x7fca4c0049f0 2021-11-21T22:03:21.384 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:21.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca50053690 msgr2=0x7fca50055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca50053690 0x7fca50055b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fca5c008000 tx=0x7fca5c008040).stop 2021-11-21T22:03:21.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 msgr2=0x7fca6c0b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca6c0b4830 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7fca5400a000 tx=0x7fca54008040).stop 2021-11-21T22:03:21.387 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 shutdown_connections 2021-11-21T22:03:21.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fca6c146750 0x7fca6c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.388 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fca6c0b5740 0x7fca6c146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fca50053690 0x7fca50055b50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.389 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 --2- 172.21.2.116:0/2858136933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fca4c0049f0 0x7fca6c0b4830 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 >> 172.21.2.116:0/2858136933 conn(0x7fca6c05bd20 msgr2=0x7fca6c146ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.390 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 shutdown_connections 2021-11-21T22:03:21.391 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.379+0000 7fca70aae700 1 -- 172.21.2.116:0/2858136933 wait complete. 2021-11-21T22:03:21.393 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:03:21.394 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:21.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 Processor -- start 2021-11-21T22:03:21.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- start start 2021-11-21T22:03:21.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45640af7e0 0x7f456405f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4564143f70 con 0x7f45640ad780 2021-11-21T22:03:21.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45640adb60 con 0x7f45640affc0 2021-11-21T22:03:21.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4564143160 con 0x7f45640af7e0 2021-11-21T22:03:21.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f45637fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45640af7e0 0x7f456405f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f45637fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f45640af7e0 0x7f456405f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54732/0 (socket says 172.21.2.116:54732) 2021-11-21T22:03:21.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f45637fe700 1 -- 172.21.2.116:0/2421257661 learned_addr learned my addr 172.21.2.116:0/2421257661 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:21.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f45637fe700 1 -- 172.21.2.116:0/2421257661 >> v1:172.21.2.117:6789/0 conn(0x7f45640affc0 legacy=0x7f456413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:21.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f45637fe700 1 -- 172.21.2.116:0/2421257661 >> v1:172.21.2.116:6789/0 conn(0x7f45640ad780 legacy=0x7f4564143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f45637fe700 1 -- 172.21.2.116:0/2421257661 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45640b0ce0 con 0x7f45640af7e0 2021-11-21T22:03:21.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f45637fe700 1 --2- 172.21.2.116:0/2421257661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640af7e0 0x7f456405f730 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 rx=0x7f45640685b0 tx=0x7f455c00b530).ready entity=mon.0 client_cookie=412e30a87a415204 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f4561ffb700 1 -- 172.21.2.116:0/2421257661 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f455c00be70 con 0x7f45640af7e0 2021-11-21T22:03:21.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f4561ffb700 1 -- 172.21.2.116:0/2421257661 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c010470 con 0x7f45640af7e0 2021-11-21T22:03:21.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f4561ffb700 1 -- 172.21.2.116:0/2421257661 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f455c0107d0 con 0x7f45640af7e0 2021-11-21T22:03:21.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- 172.21.2.116:0/2421257661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640af7e0 msgr2=0x7f456405f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 --2- 172.21.2.116:0/2421257661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640af7e0 0x7f456405f730 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 rx=0x7f45640685b0 tx=0x7f455c00b530).stop 2021-11-21T22:03:21.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- 172.21.2.116:0/2421257661 shutdown_connections 2021-11-21T22:03:21.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 --2- 172.21.2.116:0/2421257661 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640af7e0 0x7f456405f730 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- 172.21.2.116:0/2421257661 >> 172.21.2.116:0/2421257661 conn(0x7f45640b6000 msgr2=0x7f45640b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- 172.21.2.116:0/2421257661 shutdown_connections 2021-11-21T22:03:21.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- 172.21.2.116:0/2421257661 wait complete. 2021-11-21T22:03:21.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 Processor -- start 2021-11-21T22:03:21.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.443+0000 7f456932e700 1 -- start start 2021-11-21T22:03:21.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640ad780 0x7f4564140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45640af7e0 0x7f45640aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45640affc0 0x7f4564141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564064ef0 con 0x7f45640ad780 2021-11-21T22:03:21.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45640669b0 con 0x7f45640affc0 2021-11-21T22:03:21.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4564065870 con 0x7f45640af7e0 2021-11-21T22:03:21.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640ad780 0x7f4564140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640ad780 0x7f4564140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54736/0 (socket says 172.21.2.116:54736) 2021-11-21T22:03:21.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 -- 172.21.2.116:0/3271617475 learned_addr learned my addr 172.21.2.116:0/3271617475 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:21.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 -- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45640af7e0 msgr2=0x7f45640aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45640af7e0 0x7f45640aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 -- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45640affc0 msgr2=0x7f4564141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45640affc0 0x7f4564141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 -- 172.21.2.116:0/3271617475 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c00b7d0 con 0x7f45640ad780 2021-11-21T22:03:21.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f4562ffd700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640ad780 0x7f4564140af0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f4564068820 tx=0x7f4554007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f455bfff700 1 -- 172.21.2.116:0/3271617475 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f455400ce30 con 0x7f45640ad780 2021-11-21T22:03:21.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f455bfff700 1 -- 172.21.2.116:0/3271617475 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400f470 con 0x7f45640ad780 2021-11-21T22:03:21.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f455bfff700 1 -- 172.21.2.116:0/3271617475 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f455400f7d0 con 0x7f45640ad780 2021-11-21T22:03:21.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4564066be0 con 0x7f45640ad780 2021-11-21T22:03:21.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456405bd20 con 0x7f45640ad780 2021-11-21T22:03:21.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45640b0a40 con 0x7f45640ad780 2021-11-21T22:03:21.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f455bfff700 1 -- 172.21.2.116:0/3271617475 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f4554019030 con 0x7f45640ad780 2021-11-21T22:03:21.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f455bfff700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45480536a0 0x7f4548055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:21.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f455bfff700 1 -- 172.21.2.116:0/3271617475 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f455406ad80 con 0x7f45640ad780 2021-11-21T22:03:21.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.447+0000 7f45627fc700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45480536a0 0x7f4548055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:21.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.451+0000 7f45627fc700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45480536a0 0x7f4548055b60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f454c002ef0 tx=0x7f454c006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:21.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.451+0000 7f455bfff700 1 -- 172.21.2.116:0/3271617475 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4554041e80 con 0x7f45640ad780 2021-11-21T22:03:21.562 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45640573d0 con 0x7f45640ad780 2021-11-21T22:03:21.563 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f455bfff700 1 -- 172.21.2.116:0/3271617475 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f455406c070 con 0x7f45640ad780 2021-11-21T22:03:21.564 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:21.564 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":77,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1070380430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1070380430},{"type":"v1","addr":"172.21.2.117:6835","nonce":1070380430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75},{"gid":8796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/27380863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":27380863},{"type":"v1","addr":"172.21.2.116:6833","nonce":27380863}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:03:18.322670+0000","modified":"2021-11-21T22:03:19.327237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8799":{"gid":8799,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/403152574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":403152574},{"type":"v1","addr":"172.21.2.116:6835","nonce":403152574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-11-21T22:03:21.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45480536a0 msgr2=0x7f4548055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45480536a0 0x7f4548055b60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f454c002ef0 tx=0x7f454c006910).stop 2021-11-21T22:03:21.565 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640ad780 msgr2=0x7f4564140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:21.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640ad780 0x7f4564140af0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f4564068820 tx=0x7f4554007c50).stop 2021-11-21T22:03:21.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 shutdown_connections 2021-11-21T22:03:21.566 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f45640affc0 0x7f4564141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f45640af7e0 0x7f45640aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:21.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f45480536a0 0x7f4548055b60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 --2- 172.21.2.116:0/3271617475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f45640ad780 0x7f4564140af0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:21.567 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 >> 172.21.2.116:0/3271617475 conn(0x7f45640b6000 msgr2=0x7f45640b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:21.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 shutdown_connections 2021-11-21T22:03:21.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:21.559+0000 7f456932e700 1 -- 172.21.2.116:0/3271617475 wait complete. 2021-11-21T22:03:21.568 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 77 2021-11-21T22:03:21.873 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:03:18.322670+0000', 'modified': '2021-11-21T22:03:19.327237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8799': {'gid': 8799, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/403152574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 403152574}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 403152574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:03:21.873 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:03:21.874 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:03:21.940 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8799,"want_state":"up:active","state":"up:active","rank_uptime":3.6098601860000001,"mdsmap_epoch":77,"osdmap_epoch":162,"osdmap_epoch_barrier":162,"uptime":7.9052407330000003} 2021-11-21T22:03:21.940 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8799, "mdsmap_epoch": 77, "osdmap_epoch": 162, "osdmap_epoch_barrier": 162, "rank_uptime": 3.609860186, "state": "up:active", "uptime": 7.905240733, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:03:21.941 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:03:21.941 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-11-21T22:03:21.942 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-11-21T22:03:21.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:21.943 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:21.954 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:03:21.965 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:21.966 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:22.017 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:22.018 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:22.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:22.019 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:03:22.024 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:03:22.025 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:22.028 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-11-21T22:03:22.038 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:22.039 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:22.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-11-21T22:03:22.066+0000 7fccd5870080 -1 init, newargv = 0x559e48a53070 newargc=9 2021-11-21T22:03:22.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15364]: starting ceph client 2021-11-21T22:03:22.072 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:22.073 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:22.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:22.075 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-11-21T22:03:22.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15364]: starting fuse 2021-11-21T22:03:22.089 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-11-21T22:03:22.089 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:03:22.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:22.090 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-11-21T22:03:22.091 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-11-21T22:03:22.091 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-11-21T22:03:22.092 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-11-21T22:03:22.092 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-11-21T22:03:22.093 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:22.093 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-11-21T22:03:22.093 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-11-21T22:03:22.094 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-11-21T22:03:22.094 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:22.095 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-11-21T22:03:22.095 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-11-21T22:03:22.096 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-11-21T22:03:22.096 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-11-21T22:03:22.096 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:22.097 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-11-21T22:03:22.097 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:03:22.098 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:03:22.098 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-11-21T22:03:22.098 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:03:22.099 DEBUG:teuthology.orchestra.run.gibba019:> 2021-11-21T22:03:22.100 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-11-21T22:03:22.100 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-11-21T22:03:22.169 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15364.asok 2021-11-21T22:03:22.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15364.asok 2021-11-21T22:03:22.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:22.173 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15364.asok status 2021-11-21T22:03:22.222 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-11-21T22:03:22.223 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-11-21T22:03:22.223 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:03:22.223 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:03:22.223 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-11-21T22:03:22.223 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-11-21T22:03:22.224 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-11-21T22:03:22.224 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15364", 2021-11-21T22:03:22.224 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-11-21T22:03:22.224 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:22.225 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-11-21T22:03:22.225 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-11-21T22:03:22.225 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5494, 2021-11-21T22:03:22.225 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-11-21T22:03:22.225 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-11-21T22:03:22.226 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-11-21T22:03:22.226 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5494 2021-11-21T22:03:22.226 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:22.226 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:03:22.227 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:03:22.227 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:03:22.227 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 4213742764 2021-11-21T22:03:22.227 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-11-21T22:03:22.227 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:22.228 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-11-21T22:03:22.228 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-11-21T22:03:22.228 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-11-21T22:03:22.228 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 4213742764 2021-11-21T22:03:22.228 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-11-21T22:03:22.229 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5494 172.21.2.119:0/4213742764", 2021-11-21T22:03:22.229 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/4213742764", 2021-11-21T22:03:22.229 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-11-21T22:03:22.229 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 77, 2021-11-21T22:03:22.230 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 162, 2021-11-21T22:03:22.230 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:03:22.230 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-11-21T22:03:22.230 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-11-21T22:03:22.230 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-11-21T22:03:22.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:22.232 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:22.233 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:22.238 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-11-21T22:03:22.238 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-11-21T22:03:22.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:22.239 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:03:22.290 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:03:22.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 Processor -- start 2021-11-21T22:03:22.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- start start 2021-11-21T22:03:22.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd8dc0a86b0 0x7fd8dc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.351 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 0x7fd8dc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8dc0a6e80 con 0x7fd8dc0a6aa0 2021-11-21T22:03:22.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8dc0a8bc0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc14a7b0 con 0x7fd8dc062410 2021-11-21T22:03:22.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8db7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fd8dc0a6aa0 0x7fd8dc1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38468/0 (socket says 172.21.2.116:38468) 2021-11-21T22:03:22.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8db7fe700 1 -- 172.21.2.116:0/3539935114 learned_addr learned my addr 172.21.2.116:0/3539935114 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:22.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8daffd700 1 --2- 172.21.2.116:0/3539935114 >> v2:172.21.2.116:3300/0 conn(0x7fd8dc0a86b0 0x7fd8dc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8dbfff700 1 --2- 172.21.2.116:0/3539935114 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 0x7fd8dc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8da7fc700 1 -- 172.21.2.116:0/3539935114 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065322704 0 0) 0x7fd8dc0a6e80 con 0x7fd8dc0a6aa0 2021-11-21T22:03:22.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8da7fc700 1 -- 172.21.2.116:0/3539935114 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8bc003260 con 0x7fd8dc0a6aa0 2021-11-21T22:03:22.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8daffd700 1 -- 172.21.2.116:0/3539935114 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 msgr2=0x7fd8dc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8daffd700 1 --2- 172.21.2.116:0/3539935114 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 0x7fd8dc1443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8daffd700 1 -- 172.21.2.116:0/3539935114 >> v1:172.21.2.116:6789/0 conn(0x7fd8dc0a6aa0 legacy=0x7fd8dc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8daffd700 1 -- 172.21.2.116:0/3539935114 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc14aa10 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8daffd700 1 --2- 172.21.2.116:0/3539935114 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc14b150 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7fd8cc002f70 tx=0x7fd8cc006df0).ready entity=mon.0 client_cookie=8b729e53f6291add server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8da7fc700 1 -- 172.21.2.116:0/3539935114 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8cc00d4a0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8da7fc700 1 -- 172.21.2.116:0/3539935114 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc000e90 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8da7fc700 1 -- 172.21.2.116:0/3539935114 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8cc00d7f0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- 172.21.2.116:0/3539935114 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 msgr2=0x7fd8dc14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/3539935114 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc14b150 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7fd8cc002f70 tx=0x7fd8cc006df0).stop 2021-11-21T22:03:22.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- 172.21.2.116:0/3539935114 shutdown_connections 2021-11-21T22:03:22.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/3539935114 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 0x7fd8dc1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/3539935114 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc14b150 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- 172.21.2.116:0/3539935114 >> 172.21.2.116:0/3539935114 conn(0x7fd8dc05bd20 msgr2=0x7fd8dc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- 172.21.2.116:0/3539935114 shutdown_connections 2021-11-21T22:03:22.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.343+0000 7fd8e1d24700 1 -- 172.21.2.116:0/3539935114 wait complete. 2021-11-21T22:03:22.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 Processor -- start 2021-11-21T22:03:22.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 -- start start 2021-11-21T22:03:22.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 0x7fd8dc141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd8dc0a6aa0 0x7fd8dc1396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc139ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc141090 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc141760 con 0x7fd8dc0a6aa0 2021-11-21T22:03:22.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc13c4d0 con 0x7fd8dc062410 2021-11-21T22:03:22.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc139ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc139ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54754/0 (socket says 172.21.2.116:54754) 2021-11-21T22:03:22.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 -- 172.21.2.116:0/2879905265 learned_addr learned my addr 172.21.2.116:0/2879905265 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:22.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 -- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 msgr2=0x7fd8dc141ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 0x7fd8dc141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 -- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd8dc0a6aa0 msgr2=0x7fd8dc1396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd8dc0a6aa0 0x7fd8dc1396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 -- 172.21.2.116:0/2879905265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc00b180 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8dbfff700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc139ba0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 rx=0x7fd8d4006e00 tx=0x7fd8d40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8d8ff9700 1 -- 172.21.2.116:0/2879905265 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d400c850 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8d8ff9700 1 -- 172.21.2.116:0/2879905265 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d400c9b0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8dc13c890 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8d8ff9700 1 -- 172.21.2.116:0/2879905265 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d400cd10 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8dc1412c0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8dc0573d0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8d8ff9700 1 -- 172.21.2.116:0/2879905265 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd8d40123f0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8d8ff9700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8c0053640 0x7fd8c0055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8d8ff9700 1 -- 172.21.2.116:0/2879905265 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd8d406bcb0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8db7fe700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8c0053640 0x7fd8c0055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.347+0000 7fd8db7fe700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8c0053640 0x7fd8c0055b00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fd8c4001110 tx=0x7fd8c40076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.365 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.351+0000 7fd8d8ff9700 1 -- 172.21.2.116:0/2879905265 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8d4042db0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.431+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8dc05db70 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.431+0000 7fd8d8ff9700 1 -- 172.21.2.116:0/2879905265 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16052 (secure 0 0 0) 0x7fd8d40463d0 con 0x7fd8dc0a86b0 2021-11-21T22:03:22.437 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:22.437 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":162,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:19.325611+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-11-21T22:03:16.171372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-11-21T22:03:17.340772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:03:22.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8c0053640 msgr2=0x7fd8c0055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8c0053640 0x7fd8c0055b00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fd8c4001110 tx=0x7fd8c40076c0).stop 2021-11-21T22:03:22.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 msgr2=0x7fd8dc139ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc139ba0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 rx=0x7fd8d4006e00 tx=0x7fd8d40091d0).stop 2021-11-21T22:03:22.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 shutdown_connections 2021-11-21T22:03:22.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd8c0053640 0x7fd8c0055b00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd8dc0a86b0 0x7fd8dc139ba0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd8dc0a6aa0 0x7fd8dc1396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 --2- 172.21.2.116:0/2879905265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd8dc062410 0x7fd8dc141ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 >> 172.21.2.116:0/2879905265 conn(0x7fd8dc05bd20 msgr2=0x7fd8dc0b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 shutdown_connections 2021-11-21T22:03:22.441 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.435+0000 7fd8e1d24700 1 -- 172.21.2.116:0/2879905265 wait complete. 2021-11-21T22:03:22.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-21T21:54:13.059512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-11-21T21:54:20.960577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2021-11-21T22:03:16.171372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2021-11-21T22:03:17.340772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-21T22:03:22.446 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-21T22:03:22.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73cee7700 1 Processor -- start 2021-11-21T22:03:22.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73cee7700 1 -- start start 2021-11-21T22:03:22.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73cee7700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe738144810 0x7fe7381478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73cee7700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe738147e80 con 0x7fe7380abce0 2021-11-21T22:03:22.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73cee7700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7380af5a0 con 0x7fe7380ac600 2021-11-21T22:03:22.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73cee7700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7380b1490 con 0x7fe738144810 2021-11-21T22:03:22.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe735d9b700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe7380abce0 0x7fe7380b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38484/0 (socket says 172.21.2.116:38484) 2021-11-21T22:03:22.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe735d9b700 1 -- 172.21.2.116:0/4014367821 learned_addr learned my addr 172.21.2.116:0/4014367821 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:22.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354910568 0 0) 0x7fe738147e80 con 0x7fe7380abce0 2021-11-21T22:03:22.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.495+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe71c003260 con 0x7fe7380abce0 2021-11-21T22:03:22.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1586439788 0 0) 0x7fe71c003260 con 0x7fe7380abce0 2021-11-21T22:03:22.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 >> v2:172.21.2.117:3300/0 conn(0x7fe738144810 msgr2=0x7fe7381478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 --2- 172.21.2.116:0/4014367821 >> v2:172.21.2.117:3300/0 conn(0x7fe738144810 0x7fe7381478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 >> v1:172.21.2.117:6789/0 conn(0x7fe7380ac600 legacy=0x7fe7380aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380b1fd0 con 0x7fe7380abce0 2021-11-21T22:03:22.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe728001660 con 0x7fe7380abce0 2021-11-21T22:03:22.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 >> v1:172.21.2.116:6789/0 conn(0x7fe7380abce0 legacy=0x7fe7380b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe71c003750 0x7fe71c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe71c004500 0x7fe71c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe71c004f90 0x7fe71c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380b1490 con 0x7fe71c004f90 2021-11-21T22:03:22.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c00be80 con 0x7fe71c003750 2021-11-21T22:03:22.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c00c040 con 0x7fe71c004500 2021-11-21T22:03:22.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe71c004f90 0x7fe71c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/4014367821 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe71c004500 msgr2=0x7fe71c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe71c004500 0x7fe71c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/4014367821 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe71c003750 msgr2=0x7fe71c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe71c003750 0x7fe71c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/4014367821 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c003260 con 0x7fe71c004f90 2021-11-21T22:03:22.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/4014367821 reap_dead start 2021-11-21T22:03:22.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe71c004f90 0x7fe71c00b750 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7fe7300012f0 tx=0x7fe730007d60).ready entity=mon.0 client_cookie=827a6da85d82fbf1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe730006870 con 0x7fe71c004f90 2021-11-21T22:03:22.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7300069d0 con 0x7fe71c004f90 2021-11-21T22:03:22.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73559a700 1 -- 172.21.2.116:0/4014367821 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe730006d30 con 0x7fe71c004f90 2021-11-21T22:03:22.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/4014367821 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe71c004f90 msgr2=0x7fe71c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe71c004f90 0x7fe71c00b750 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7fe7300012f0 tx=0x7fe730007d60).stop 2021-11-21T22:03:22.512 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/4014367821 shutdown_connections 2021-11-21T22:03:22.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 --2- 172.21.2.116:0/4014367821 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe71c004f90 0x7fe71c00b750 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/4014367821 >> 172.21.2.116:0/4014367821 conn(0x7fe73805bd20 msgr2=0x7fe73805c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/4014367821 shutdown_connections 2021-11-21T22:03:22.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/4014367821 wait complete. 2021-11-21T22:03:22.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 Processor -- start 2021-11-21T22:03:22.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- start start 2021-11-21T22:03:22.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe71c004f90 0x7fe738156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7380b4730 0x7fe73815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7380b0a20 0x7fe7380629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe738156810 con 0x7fe7380b0a20 2021-11-21T22:03:22.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380b3de0 con 0x7fe7380b4730 2021-11-21T22:03:22.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe738159a00 con 0x7fe71c004f90 2021-11-21T22:03:22.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7380b0a20 0x7fe7380629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7380b0a20 0x7fe7380629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54774/0 (socket says 172.21.2.116:54774) 2021-11-21T22:03:22.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/3187767531 learned_addr learned my addr 172.21.2.116:0/3187767531 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:22.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe71c004f90 msgr2=0x7fe738156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe71c004f90 0x7fe738156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7380b4730 msgr2=0x7fe73815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7380b4730 0x7fe73815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 -- 172.21.2.116:0/3187767531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380b42b0 con 0x7fe7380b0a20 2021-11-21T22:03:22.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe736d9d700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7380b0a20 0x7fe7380629c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7fe7300012f0 tx=0x7fe730016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe7277fe700 1 -- 172.21.2.116:0/3187767531 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe730014860 con 0x7fe7380b0a20 2021-11-21T22:03:22.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe7277fe700 1 -- 172.21.2.116:0/3187767531 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe730019070 con 0x7fe7380b0a20 2021-11-21T22:03:22.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7380b4010 con 0x7fe7380b0a20 2021-11-21T22:03:22.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7380b2f20 con 0x7fe7380b0a20 2021-11-21T22:03:22.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe7277fe700 1 -- 172.21.2.116:0/3187767531 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe730014bb0 con 0x7fe7380b0a20 2021-11-21T22:03:22.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe710003410 con 0x7fe7380b0a20 2021-11-21T22:03:22.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.499+0000 7fe7277fe700 1 -- 172.21.2.116:0/3187767531 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe73001e070 con 0x7fe7380b0a20 2021-11-21T22:03:22.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.503+0000 7fe7277fe700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe714053720 0x7fe714055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.503+0000 7fe7277fe700 1 -- 172.21.2.116:0/3187767531 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe73006cfc0 con 0x7fe7380b0a20 2021-11-21T22:03:22.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.503+0000 7fe73659c700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe714053720 0x7fe714055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.503+0000 7fe73659c700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe714053720 0x7fe714055be0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fe72c002f70 tx=0x7fe72c006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.503+0000 7fe7277fe700 1 -- 172.21.2.116:0/3187767531 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe730044010 con 0x7fe7380b0a20 2021-11-21T22:03:22.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.587+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe710003c40 con 0x7fe7380b0a20 2021-11-21T22:03:22.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.587+0000 7fe7277fe700 1 -- 172.21.2.116:0/3187767531 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v162) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe73001b070 con 0x7fe7380b0a20 2021-11-21T22:03:22.593 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 1 2021-11-21T22:03:22.593 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe714053720 msgr2=0x7fe714055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe714053720 0x7fe714055be0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fe72c002f70 tx=0x7fe72c006a00).stop 2021-11-21T22:03:22.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7380b0a20 msgr2=0x7fe7380629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7380b0a20 0x7fe7380629c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7fe7300012f0 tx=0x7fe730016000).stop 2021-11-21T22:03:22.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 shutdown_connections 2021-11-21T22:03:22.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe714053720 0x7fe714055be0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe7380b0a20 0x7fe7380629c0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe7380b4730 0x7fe73815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 --2- 172.21.2.116:0/3187767531 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe71c004f90 0x7fe738156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 >> 172.21.2.116:0/3187767531 conn(0x7fe73805bd20 msgr2=0x7fe7381483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 shutdown_connections 2021-11-21T22:03:22.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.591+0000 7fe73cee7700 1 -- 172.21.2.116:0/3187767531 wait complete. 2021-11-21T22:03:22.600 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-11-21T22:03:22.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 Processor -- start 2021-11-21T22:03:22.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- start start 2021-11-21T22:03:22.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ffb641426e0 0x7ffb64142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb640afae0 con 0x7ffb64062e60 2021-11-21T22:03:22.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb640b0f60 con 0x7ffb64069e70 2021-11-21T22:03:22.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb64063470 con 0x7ffb641426e0 2021-11-21T22:03:22.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb69254700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7ffb64062e60 0x7ffb640af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38502/0 (socket says 172.21.2.116:38502) 2021-11-21T22:03:22.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb69254700 1 -- 172.21.2.116:0/512542870 learned_addr learned my addr 172.21.2.116:0/512542870 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:22.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505375177 0 0) 0x7ffb640afae0 con 0x7ffb64062e60 2021-11-21T22:03:22.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb4c003260 con 0x7ffb64062e60 2021-11-21T22:03:22.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310971465 0 0) 0x7ffb4c003260 con 0x7ffb64062e60 2021-11-21T22:03:22.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 >> v2:172.21.2.117:3300/0 conn(0x7ffb641426e0 msgr2=0x7ffb64142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 --2- 172.21.2.116:0/512542870 >> v2:172.21.2.117:3300/0 conn(0x7ffb641426e0 0x7ffb64142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 >> v1:172.21.2.117:6789/0 conn(0x7ffb64069e70 legacy=0x7ffb640ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb64143050 con 0x7ffb64062e60 2021-11-21T22:03:22.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7ffb54001660 con 0x7ffb64062e60 2021-11-21T22:03:22.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 >> v1:172.21.2.116:6789/0 conn(0x7ffb64062e60 legacy=0x7ffb640af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb4c003750 0x7ffb4c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb4c004500 0x7ffb4c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb4c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb64063470 con 0x7ffb4c004f90 2021-11-21T22:03:22.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c00be80 con 0x7ffb4c003750 2021-11-21T22:03:22.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c00c040 con 0x7ffb4c004500 2021-11-21T22:03:22.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb4c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 -- 172.21.2.116:0/512542870 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb4c004500 msgr2=0x7ffb4c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb4c004500 0x7ffb4c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 -- 172.21.2.116:0/512542870 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb4c003750 msgr2=0x7ffb4c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb4c003750 0x7ffb4c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 -- 172.21.2.116:0/512542870 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c003260 con 0x7ffb4c004f90 2021-11-21T22:03:22.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 -- 172.21.2.116:0/512542870 reap_dead start 2021-11-21T22:03:22.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6a256700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb4c00b750 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7ffb5c0012f0 tx=0x7ffb5c007d60).ready entity=mon.0 client_cookie=9bccc8e6d405855 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb5c006870 con 0x7ffb4c004f90 2021-11-21T22:03:22.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0069d0 con 0x7ffb4c004f90 2021-11-21T22:03:22.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb68a53700 1 -- 172.21.2.116:0/512542870 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb5c006d30 con 0x7ffb4c004f90 2021-11-21T22:03:22.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/512542870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 msgr2=0x7ffb4c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.671 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb4c00b750 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7ffb5c0012f0 tx=0x7ffb5c007d60).stop 2021-11-21T22:03:22.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/512542870 shutdown_connections 2021-11-21T22:03:22.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/512542870 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb4c00b750 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/512542870 >> 172.21.2.116:0/512542870 conn(0x7ffb640b5730 msgr2=0x7ffb640b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/512542870 shutdown_connections 2021-11-21T22:03:22.672 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/512542870 wait complete. 2021-11-21T22:03:22.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 Processor -- start 2021-11-21T22:03:22.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.651+0000 7ffb6bcb9700 1 -- start start 2021-11-21T22:03:22.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb64067010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb640675a0 0x7ffb64143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb64144070 0x7ffb641277a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb64065fe0 con 0x7ffb4c004f90 2021-11-21T22:03:22.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb640668d0 con 0x7ffb640675a0 2021-11-21T22:03:22.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb64066b70 con 0x7ffb64144070 2021-11-21T22:03:22.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb64067010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb64067010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54790/0 (socket says 172.21.2.116:54790) 2021-11-21T22:03:22.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 -- 172.21.2.116:0/3585249099 learned_addr learned my addr 172.21.2.116:0/3585249099 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:22.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 -- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb64144070 msgr2=0x7ffb641277a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:22.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb64144070 0x7ffb641277a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 -- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb640675a0 msgr2=0x7ffb64143800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:22.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb640675a0 0x7ffb64143800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 -- 172.21.2.116:0/3585249099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb64143cf0 con 0x7ffb4c004f90 2021-11-21T22:03:22.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69a55700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb64067010 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7ffb60002f70 tx=0x7ffb60006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb52ffd700 1 -- 172.21.2.116:0/3585249099 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb60013460 con 0x7ffb4c004f90 2021-11-21T22:03:22.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb52ffd700 1 -- 172.21.2.116:0/3585249099 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb600135c0 con 0x7ffb4c004f90 2021-11-21T22:03:22.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb64146150 con 0x7ffb4c004f90 2021-11-21T22:03:22.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb64127eb0 con 0x7ffb4c004f90 2021-11-21T22:03:22.677 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb52ffd700 1 -- 172.21.2.116:0/3585249099 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb60013920 con 0x7ffb4c004f90 2021-11-21T22:03:22.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb640b1e00 con 0x7ffb4c004f90 2021-11-21T22:03:22.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb52ffd700 1 -- 172.21.2.116:0/3585249099 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ffb6001a060 con 0x7ffb4c004f90 2021-11-21T22:03:22.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb52ffd700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb44053720 0x7ffb44055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb52ffd700 1 -- 172.21.2.116:0/3585249099 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ffb6006b440 con 0x7ffb4c004f90 2021-11-21T22:03:22.678 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.655+0000 7ffb69254700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb44053720 0x7ffb44055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.659+0000 7ffb69254700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb44053720 0x7ffb44055be0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7ffb64062db0 tx=0x7ffb640b07d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.679 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.659+0000 7ffb52ffd700 1 -- 172.21.2.116:0/3585249099 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb60042490 con 0x7ffb4c004f90 2021-11-21T22:03:22.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.739+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffb640573d0 con 0x7ffb4c004f90 2021-11-21T22:03:22.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb52ffd700 1 -- 172.21.2.116:0/3585249099 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v162) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffb60018020 con 0x7ffb4c004f90 2021-11-21T22:03:22.746 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:22.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb44053720 msgr2=0x7ffb44055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb44053720 0x7ffb44055be0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7ffb64062db0 tx=0x7ffb640b07d0).stop 2021-11-21T22:03:22.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 msgr2=0x7ffb64067010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb64067010 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7ffb60002f70 tx=0x7ffb60006e40).stop 2021-11-21T22:03:22.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 shutdown_connections 2021-11-21T22:03:22.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ffb64144070 0x7ffb641277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ffb640675a0 0x7ffb64143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ffb44053720 0x7ffb44055be0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 --2- 172.21.2.116:0/3585249099 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ffb4c004f90 0x7ffb64067010 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 >> 172.21.2.116:0/3585249099 conn(0x7ffb640b5730 msgr2=0x7ffb6413c310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 shutdown_connections 2021-11-21T22:03:22.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.743+0000 7ffb6bcb9700 1 -- 172.21.2.116:0/3585249099 wait complete. 2021-11-21T22:03:22.753 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-11-21T22:03:22.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 Processor -- start 2021-11-21T22:03:22.806 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- start start 2021-11-21T22:03:22.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe2181426e0 0x7fe218142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2180afae0 con 0x7fe2180ad470 2021-11-21T22:03:22.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2180b0f60 con 0x7fe218062e60 2021-11-21T22:03:22.807 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180ae030 con 0x7fe2181426e0 2021-11-21T22:03:22.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21ec35700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe2180ad470 0x7fe2180ad920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38526/0 (socket says 172.21.2.116:38526) 2021-11-21T22:03:22.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21ec35700 1 -- 172.21.2.116:0/3963982665 learned_addr learned my addr 172.21.2.116:0/3963982665 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:22.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283906947 0 0) 0x7fe2180afae0 con 0x7fe2180ad470 2021-11-21T22:03:22.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe200003260 con 0x7fe2180ad470 2021-11-21T22:03:22.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2912752049 0 0) 0x7fe200003260 con 0x7fe2180ad470 2021-11-21T22:03:22.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe2181426e0 msgr2=0x7fe218142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe2181426e0 0x7fe218142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 >> v1:172.21.2.117:6789/0 conn(0x7fe218062e60 legacy=0x7fe2180af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.814 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218063470 con 0x7fe2180ad470 2021-11-21T22:03:22.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe2100019e0 con 0x7fe2180ad470 2021-11-21T22:03:22.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 >> v1:172.21.2.116:6789/0 conn(0x7fe2180ad470 legacy=0x7fe2180ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200003800 0x7fe200003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.815 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe200004500 0x7fe200004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe200005070 0x7fe200007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180ae030 con 0x7fe200004500 2021-11-21T22:03:22.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200007ed0 con 0x7fe200005070 2021-11-21T22:03:22.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe200008090 con 0x7fe200003800 2021-11-21T22:03:22.816 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21dc33700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe200004500 0x7fe200004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21dc33700 1 -- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200003800 msgr2=0x7fe200003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21dc33700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200003800 0x7fe200003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21dc33700 1 -- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe200005070 msgr2=0x7fe200007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21dc33700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe200005070 0x7fe200007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21dc33700 1 -- 172.21.2.116:0/3963982665 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200003260 con 0x7fe200004500 2021-11-21T22:03:22.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21dc33700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe200004500 0x7fe200004980 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7fe214000f70 tx=0x7fe2140079e0).ready entity=mon.0 client_cookie=65580a7458ce11ba server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe214007cf0 con 0x7fe200004500 2021-11-21T22:03:22.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214014410 con 0x7fe200004500 2021-11-21T22:03:22.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21d432700 1 -- 172.21.2.116:0/3963982665 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe214014770 con 0x7fe200004500 2021-11-21T22:03:22.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- 172.21.2.116:0/3963982665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe200004500 msgr2=0x7fe200004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 --2- 172.21.2.116:0/3963982665 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe200004500 0x7fe200004980 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7fe214000f70 tx=0x7fe2140079e0).stop 2021-11-21T22:03:22.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe21ec35700 1 -- 172.21.2.116:0/3963982665 reap_dead start 2021-11-21T22:03:22.819 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- 172.21.2.116:0/3963982665 shutdown_connections 2021-11-21T22:03:22.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- 172.21.2.116:0/3963982665 >> 172.21.2.116:0/3963982665 conn(0x7fe2180b5730 msgr2=0x7fe2180b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- 172.21.2.116:0/3963982665 shutdown_connections 2021-11-21T22:03:22.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.803+0000 7fe220698700 1 -- 172.21.2.116:0/3963982665 wait complete. 2021-11-21T22:03:22.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 Processor -- start 2021-11-21T22:03:22.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 -- start start 2021-11-21T22:03:22.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200004500 0x7fe21814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2180637a0 0x7fe2180b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe218145e60 0x7fe218145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.821 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21813f950 con 0x7fe218145e60 2021-11-21T22:03:22.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2181448d0 con 0x7fe2180637a0 2021-11-21T22:03:22.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe218151a30 con 0x7fe200004500 2021-11-21T22:03:22.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe218145e60 0x7fe218145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe218145e60 0x7fe218145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54814/0 (socket says 172.21.2.116:54814) 2021-11-21T22:03:22.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 -- 172.21.2.116:0/3737545339 learned_addr learned my addr 172.21.2.116:0/3737545339 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:22.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21dc33700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2180637a0 0x7fe2180b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21e434700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200004500 0x7fe21814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 -- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200004500 msgr2=0x7fe21814d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200004500 0x7fe21814d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 -- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2180637a0 msgr2=0x7fe2180b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2180637a0 0x7fe2180b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 -- 172.21.2.116:0/3737545339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21813fb80 con 0x7fe218145e60 2021-11-21T22:03:22.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe21ec35700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe218145e60 0x7fe218145340 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7fe2100016f0 tx=0x7fe2180af2f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe2077fe700 1 -- 172.21.2.116:0/3737545339 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe210004ab0 con 0x7fe218145e60 2021-11-21T22:03:22.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe2077fe700 1 -- 172.21.2.116:0/3737545339 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe210004c10 con 0x7fe218145e60 2021-11-21T22:03:22.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2180b4950 con 0x7fe218145e60 2021-11-21T22:03:22.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe2077fe700 1 -- 172.21.2.116:0/3737545339 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe210007170 con 0x7fe218145e60 2021-11-21T22:03:22.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe218145830 con 0x7fe218145e60 2021-11-21T22:03:22.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21813fd10 con 0x7fe218145e60 2021-11-21T22:03:22.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe2077fe700 1 -- 172.21.2.116:0/3737545339 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe21001d070 con 0x7fe218145e60 2021-11-21T22:03:22.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe2077fe700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1f80536d0 0x7fe1f8055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.807+0000 7fe2077fe700 1 -- 172.21.2.116:0/3737545339 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe21006b0c0 con 0x7fe218145e60 2021-11-21T22:03:22.826 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.811+0000 7fe21e434700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1f80536d0 0x7fe1f8055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.827 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.811+0000 7fe21e434700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1f80536d0 0x7fe1f8055b90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fe20800a000 tx=0x7fe208008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.828 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.811+0000 7fe2077fe700 1 -- 172.21.2.116:0/3737545339 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe210042220 con 0x7fe218145e60 2021-11-21T22:03:22.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe218143050 con 0x7fe218145e60 2021-11-21T22:03:22.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe2077fe700 1 -- 172.21.2.116:0/3737545339 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe210045840 con 0x7fe218145e60 2021-11-21T22:03:22.899 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:22.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1f80536d0 msgr2=0x7fe1f8055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1f80536d0 0x7fe1f8055b90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fe20800a000 tx=0x7fe208008040).stop 2021-11-21T22:03:22.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe218145e60 msgr2=0x7fe218145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe218145e60 0x7fe218145340 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7fe2100016f0 tx=0x7fe2180af2f0).stop 2021-11-21T22:03:22.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 shutdown_connections 2021-11-21T22:03:22.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe1f80536d0 0x7fe1f8055b90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe218145e60 0x7fe218145340 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2180637a0 0x7fe2180b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 --2- 172.21.2.116:0/3737545339 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe200004500 0x7fe21814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 >> 172.21.2.116:0/3737545339 conn(0x7fe2180b5730 msgr2=0x7fe21805bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 shutdown_connections 2021-11-21T22:03:22.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.895+0000 7fe220698700 1 -- 172.21.2.116:0/3737545339 wait complete. 2021-11-21T22:03:22.906 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-11-21T22:03:22.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 Processor -- start 2021-11-21T22:03:22.959 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- start start 2021-11-21T22:03:22.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02600abce0 0x7f02601478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f02600ac600 0x7f02600acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0260147e80 con 0x7f0260144810 2021-11-21T22:03:22.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02600b1490 con 0x7f02600ac600 2021-11-21T22:03:22.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02600b1fd0 con 0x7f02600abce0 2021-11-21T22:03:22.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265a4a700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f02600ac600 0x7f02600acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265249700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0260144810 0x7f02600b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38542/0 (socket says 172.21.2.116:38542) 2021-11-21T22:03:22.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265249700 1 -- 172.21.2.116:0/1813767439 learned_addr learned my addr 172.21.2.116:0/1813767439 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:22.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265a4a700 1 -- 172.21.2.116:0/1813767439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02600abce0 msgr2=0x7f02601478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:22.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265a4a700 1 --2- 172.21.2.116:0/1813767439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02600abce0 0x7f02601478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265a4a700 1 -- 172.21.2.116:0/1813767439 >> v1:172.21.2.116:6789/0 conn(0x7f0260144810 legacy=0x7f02600b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265a4a700 1 -- 172.21.2.116:0/1813767439 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600af5a0 con 0x7f02600ac600 2021-11-21T22:03:22.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0253fff700 1 -- 172.21.2.116:0/1813767439 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873979320 0 0) 0x7f0260147e80 con 0x7f0260144810 2021-11-21T22:03:22.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0265a4a700 1 --2- 172.21.2.116:0/1813767439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600ac600 0x7f02600acab0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f02580032f0 tx=0x7f02580091d0).ready entity=mon.0 client_cookie=1f0c1c972c222bcb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0253fff700 1 -- 172.21.2.116:0/1813767439 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0258009ea0 con 0x7f02600ac600 2021-11-21T22:03:22.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0253fff700 1 -- 172.21.2.116:0/1813767439 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025800b8a0 con 0x7f02600ac600 2021-11-21T22:03:22.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f0253fff700 1 -- 172.21.2.116:0/1813767439 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f025800ba10 con 0x7f02600ac600 2021-11-21T22:03:22.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- 172.21.2.116:0/1813767439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600ac600 msgr2=0x7f02600acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 --2- 172.21.2.116:0/1813767439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600ac600 0x7f02600acab0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f02580032f0 tx=0x7f02580091d0).stop 2021-11-21T22:03:22.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- 172.21.2.116:0/1813767439 shutdown_connections 2021-11-21T22:03:22.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 --2- 172.21.2.116:0/1813767439 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600ac600 0x7f02600acab0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:22.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 --2- 172.21.2.116:0/1813767439 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f02600abce0 0x7f02601478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- 172.21.2.116:0/1813767439 >> 172.21.2.116:0/1813767439 conn(0x7f026005bd20 msgr2=0x7f026005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:22.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- 172.21.2.116:0/1813767439 shutdown_connections 2021-11-21T22:03:22.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.955+0000 7f02674ad700 1 -- 172.21.2.116:0/1813767439 wait complete. 2021-11-21T22:03:22.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 Processor -- start 2021-11-21T22:03:22.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 -- start start 2021-11-21T22:03:22.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600abce0 0x7f0260143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02600ac600 0x7f02600b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0260144810 0x7f02600b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0260149860 con 0x7f02600abce0 2021-11-21T22:03:22.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0260159960 con 0x7f02600ac600 2021-11-21T22:03:22.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02601455e0 con 0x7f0260144810 2021-11-21T22:03:22.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600abce0 0x7f0260143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600abce0 0x7f0260143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54826/0 (socket says 172.21.2.116:54826) 2021-11-21T22:03:22.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 -- 172.21.2.116:0/3433446991 learned_addr learned my addr 172.21.2.116:0/3433446991 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:22.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 -- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0260144810 msgr2=0x7f02600b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0260144810 0x7f02600b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 -- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02600ac600 msgr2=0x7f02600b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:22.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02600ac600 0x7f02600b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:22.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 -- 172.21.2.116:0/3433446991 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0258009470 con 0x7f02600abce0 2021-11-21T22:03:22.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0265249700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600abce0 0x7f0260143790 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7f025c000ed0 tx=0x7f025c007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02527fc700 1 -- 172.21.2.116:0/3433446991 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f025c00ce20 con 0x7f02600abce0 2021-11-21T22:03:22.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02527fc700 1 -- 172.21.2.116:0/3433446991 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c0072e0 con 0x7f02600abce0 2021-11-21T22:03:22.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02527fc700 1 -- 172.21.2.116:0/3433446991 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f025c005610 con 0x7f02600abce0 2021-11-21T22:03:22.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0260145880 con 0x7f02600abce0 2021-11-21T22:03:22.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0260145dc0 con 0x7f02600abce0 2021-11-21T22:03:22.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02600af060 con 0x7f02600abce0 2021-11-21T22:03:22.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02527fc700 1 -- 172.21.2.116:0/3433446991 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f025c010070 con 0x7f02600abce0 2021-11-21T22:03:22.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02527fc700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0240053640 0x7f0240055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:22.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f02527fc700 1 -- 172.21.2.116:0/3433446991 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f025c06b1c0 con 0x7f02600abce0 2021-11-21T22:03:22.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.959+0000 7f0264a48700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0240053640 0x7f0240055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:22.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.963+0000 7f0264a48700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0240053640 0x7f0240055b00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f0254002f70 tx=0x7f0254006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:22.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:22.963+0000 7f02527fc700 1 -- 172.21.2.116:0/3433446991 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f025c042210 con 0x7f02600abce0 2021-11-21T22:03:23.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.043+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02600573d0 con 0x7f02600abce0 2021-11-21T22:03:23.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.043+0000 7f02527fc700 1 -- 172.21.2.116:0/3433446991 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+10 (secure 0 0 0) 0x7f025c045830 con 0x7f02600abce0 2021-11-21T22:03:23.049 INFO:teuthology.orchestra.run.gibba016.stdout:pg_num: 8 2021-11-21T22:03:23.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0240053640 msgr2=0x7f0240055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:23.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0240053640 0x7f0240055b00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f0254002f70 tx=0x7f0254006a80).stop 2021-11-21T22:03:23.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600abce0 msgr2=0x7f0260143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:23.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600abce0 0x7f0260143790 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7f025c000ed0 tx=0x7f025c007ce0).stop 2021-11-21T22:03:23.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 shutdown_connections 2021-11-21T22:03:23.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0260144810 0x7f02600b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f02600ac600 0x7f02600b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0240053640 0x7f0240055b00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:23.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 --2- 172.21.2.116:0/3433446991 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f02600abce0 0x7f0260143790 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:23.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 >> 172.21.2.116:0/3433446991 conn(0x7f026005bd20 msgr2=0x7f0260141610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:23.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 shutdown_connections 2021-11-21T22:03:23.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.047+0000 7f02674ad700 1 -- 172.21.2.116:0/3433446991 wait complete. 2021-11-21T22:03:23.057 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-11-21T22:03:23.058 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:23.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.103+0000 7f51b1683700 1 Processor -- start 2021-11-21T22:03:23.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.103+0000 7f51b1683700 1 -- start start 2021-11-21T22:03:23.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f51ac0a6aa0 0x7f51ac14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:23.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f51ac062410 0x7f51ac1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:23.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0a6e80 con 0x7f51ac0a86b0 2021-11-21T22:03:23.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac0a8bc0 con 0x7f51ac062410 2021-11-21T22:03:23.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac14a7b0 con 0x7f51ac0a6aa0 2021-11-21T22:03:23.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f51ac062410 0x7f51ac1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:23.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f51ac062410 0x7f51ac1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54836/0 (socket says 172.21.2.116:54836) 2021-11-21T22:03:23.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 -- 172.21.2.116:0/1875118162 learned_addr learned my addr 172.21.2.116:0/1875118162 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:23.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 -- 172.21.2.116:0/1875118162 >> v2:172.21.2.117:3300/0 conn(0x7f51ac0a6aa0 msgr2=0x7f51ac14b150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:23.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 --2- 172.21.2.116:0/1875118162 >> v2:172.21.2.117:3300/0 conn(0x7f51ac0a6aa0 0x7f51ac14b150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 -- 172.21.2.116:0/1875118162 >> v1:172.21.2.117:6789/0 conn(0x7f51ac0a86b0 legacy=0x7f51ac1470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:23.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 -- 172.21.2.116:0/1875118162 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac14aa10 con 0x7f51ac062410 2021-11-21T22:03:23.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51ab7fe700 1 --2- 172.21.2.116:0/1875118162 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac1443f0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7f51ac061ab0 tx=0x7f51a4009290).ready entity=mon.0 client_cookie=240681193df1c47a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:23.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51a9ffb700 1 -- 172.21.2.116:0/1875118162 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a400b830 con 0x7f51ac062410 2021-11-21T22:03:23.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51a9ffb700 1 -- 172.21.2.116:0/1875118162 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a400b990 con 0x7f51ac062410 2021-11-21T22:03:23.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51a9ffb700 1 -- 172.21.2.116:0/1875118162 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a400bb40 con 0x7f51ac062410 2021-11-21T22:03:23.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- 172.21.2.116:0/1875118162 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 msgr2=0x7f51ac1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:23.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- 172.21.2.116:0/1875118162 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac1443f0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7f51ac061ab0 tx=0x7f51a4009290).stop 2021-11-21T22:03:23.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- 172.21.2.116:0/1875118162 shutdown_connections 2021-11-21T22:03:23.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- 172.21.2.116:0/1875118162 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac1443f0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:23.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- 172.21.2.116:0/1875118162 >> v2:172.21.2.117:3300/0 conn(0x7f51ac0a6aa0 0x7f51ac14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- 172.21.2.116:0/1875118162 >> 172.21.2.116:0/1875118162 conn(0x7f51ac05bd20 msgr2=0x7f51ac05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:23.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- 172.21.2.116:0/1875118162 shutdown_connections 2021-11-21T22:03:23.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- 172.21.2.116:0/1875118162 wait complete. 2021-11-21T22:03:23.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 Processor -- start 2021-11-21T22:03:23.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- start start 2021-11-21T22:03:23.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac0641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:23.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51ac0a6aa0 0x7f51ac0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:23.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51ac0a86b0 0x7f51ac066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:23.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac064bb0 con 0x7f51ac062410 2021-11-21T22:03:23.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1571a0 con 0x7f51ac0a86b0 2021-11-21T22:03:23.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac144f50 con 0x7f51ac0a6aa0 2021-11-21T22:03:23.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac0641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:23.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac0641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54842/0 (socket says 172.21.2.116:54842) 2021-11-21T22:03:23.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 -- 172.21.2.116:0/1326476336 learned_addr learned my addr 172.21.2.116:0/1326476336 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:23.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 -- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51ac0a6aa0 msgr2=0x7f51ac0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:23.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51ac0a6aa0 0x7f51ac0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 -- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51ac0a86b0 msgr2=0x7f51ac066d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:23.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51ac0a86b0 0x7f51ac066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 -- 172.21.2.116:0/1326476336 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4009530 con 0x7f51ac062410 2021-11-21T22:03:23.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51aaffd700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac0641d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7f51ac063f80 tx=0x7f5194006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:23.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51a3fff700 1 -- 172.21.2.116:0/1326476336 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5194013460 con 0x7f51ac062410 2021-11-21T22:03:23.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51a3fff700 1 -- 172.21.2.116:0/1326476336 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940135c0 con 0x7f51ac062410 2021-11-21T22:03:23.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac1573d0 con 0x7f51ac062410 2021-11-21T22:03:23.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51a3fff700 1 -- 172.21.2.116:0/1326476336 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5194013920 con 0x7f51ac062410 2021-11-21T22:03:23.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.107+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac1451f0 con 0x7f51ac062410 2021-11-21T22:03:23.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.111+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac0573d0 con 0x7f51ac062410 2021-11-21T22:03:23.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.111+0000 7f51a3fff700 1 -- 172.21.2.116:0/1326476336 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f519400f070 con 0x7f51ac062410 2021-11-21T22:03:23.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.111+0000 7f51a3fff700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51900535b0 0x7f5190055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:23.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.111+0000 7f51a3fff700 1 -- 172.21.2.116:0/1326476336 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f519406b360 con 0x7f51ac062410 2021-11-21T22:03:23.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.111+0000 7f51aa7fc700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51900535b0 0x7f5190055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:23.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.111+0000 7f51aa7fc700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51900535b0 0x7f5190055a70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f519c002ef0 tx=0x7f519c006910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:23.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.111+0000 7f51a3fff700 1 -- 172.21.2.116:0/1326476336 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51940424c0 con 0x7f51ac062410 2021-11-21T22:03:23.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.219+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51ac14b470 con 0x7f51ac062410 2021-11-21T22:03:23.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.219+0000 7f51a3fff700 1 -- 172.21.2.116:0/1326476336 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f5194045ae0 con 0x7f51ac062410 2021-11-21T22:03:23.225 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:23.225 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":77,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1070380430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1070380430},{"type":"v1","addr":"172.21.2.117:6835","nonce":1070380430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75},{"gid":8796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/27380863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":27380863},{"type":"v1","addr":"172.21.2.116:6833","nonce":27380863}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:03:18.322670+0000","modified":"2021-11-21T22:03:19.327237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8799":{"gid":8799,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/403152574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":403152574},{"type":"v1","addr":"172.21.2.116:6835","nonce":403152574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-11-21T22:03:23.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51900535b0 msgr2=0x7f5190055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:23.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51900535b0 0x7f5190055a70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f519c002ef0 tx=0x7f519c006910).stop 2021-11-21T22:03:23.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 msgr2=0x7f51ac0641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:23.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac0641d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7f51ac063f80 tx=0x7f5194006df0).stop 2021-11-21T22:03:23.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 shutdown_connections 2021-11-21T22:03:23.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f51ac0a86b0 0x7f51ac066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f51ac0a6aa0 0x7f51ac0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:23.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f51900535b0 0x7f5190055a70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:23.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 --2- 172.21.2.116:0/1326476336 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f51ac062410 0x7f51ac0641d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:23.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 >> 172.21.2.116:0/1326476336 conn(0x7f51ac05bd20 msgr2=0x7f51ac1441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:23.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 shutdown_connections 2021-11-21T22:03:23.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:23.223+0000 7f51b1683700 1 -- 172.21.2.116:0/1326476336 wait complete. 2021-11-21T22:03:23.229 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 77 2021-11-21T22:03:23.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-11-21T22:03:18.322670+0000', 'modified': '2021-11-21T22:03:19.327237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8799': {'gid': 8799, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.116:6835/403152574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.116:6834', 'nonce': 403152574}, {'type': 'v1', 'addr': '172.21.2.116:6835', 'nonce': 403152574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-11-21T22:03:23.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-11-21T22:03:23.536 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-11-21T22:03:23.600 INFO:teuthology.orchestra.run.gibba016.stdout:{"cluster_fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","whoami":0,"id":8799,"want_state":"up:active","state":"up:active","rank_uptime":5.2703517440000001,"mdsmap_epoch":77,"osdmap_epoch":162,"osdmap_epoch_barrier":162,"uptime":9.5657326190000003} 2021-11-21T22:03:23.601 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", "id": 8799, "mdsmap_epoch": 77, "osdmap_epoch": 162, "osdmap_epoch_barrier": 162, "rank_uptime": 5.270351744, "state": "up:active", "uptime": 9.565732619, "want_state": "up:active", "whoami": 0 } 2021-11-21T22:03:23.602 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-11-21T22:03:23.602 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-11-21T22:03:23.602 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba036.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-11-21T22:03:23.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.603 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:23.615 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:03:23.627 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:23.628 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:23.680 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:23.681 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:23.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.682 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:03:23.688 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-11-21T22:03:23.689 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:23.691 DEBUG:teuthology.orchestra.run.gibba036:> sudo modprobe fuse 2021-11-21T22:03:23.701 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-11-21T22:03:23.702 DEBUG:teuthology.orchestra.run.gibba036:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-11-21T22:03:23.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:2021-11-21T22:03:23.729+0000 7f03f631a080 -1 init, newargv = 0x5563e9dc9bd0 newargc=9 2021-11-21T22:03:23.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14858]: starting ceph client 2021-11-21T22:03:23.736 INFO:teuthology.orchestra.run.gibba036.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-11-21T22:03:23.736 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-11-21T22:03:23.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.738 DEBUG:teuthology.orchestra.run.gibba036:> ls /sys/fs/fuse/connections 2021-11-21T22:03:23.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14858]: starting fuse 2021-11-21T22:03:23.745 INFO:teuthology.orchestra.run.gibba036.stdout:52 2021-11-21T22:03:23.745 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-11-21T22:03:23.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.746 DEBUG:teuthology.orchestra.run.gibba036:> sudo python3 -c ' 2021-11-21T22:03:23.747 DEBUG:teuthology.orchestra.run.gibba036:> import glob 2021-11-21T22:03:23.747 DEBUG:teuthology.orchestra.run.gibba036:> import re 2021-11-21T22:03:23.747 DEBUG:teuthology.orchestra.run.gibba036:> import os 2021-11-21T22:03:23.748 DEBUG:teuthology.orchestra.run.gibba036:> import subprocess 2021-11-21T22:03:23.748 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:23.749 DEBUG:teuthology.orchestra.run.gibba036:> def find_socket(client_name): 2021-11-21T22:03:23.749 DEBUG:teuthology.orchestra.run.gibba036:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-11-21T22:03:23.749 DEBUG:teuthology.orchestra.run.gibba036:> files = glob.glob(asok_path) 2021-11-21T22:03:23.750 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:23.750 DEBUG:teuthology.orchestra.run.gibba036:> # Given a non-glob path, it better be there 2021-11-21T22:03:23.751 DEBUG:teuthology.orchestra.run.gibba036:> if "*" not in asok_path: 2021-11-21T22:03:23.751 DEBUG:teuthology.orchestra.run.gibba036:> assert(len(files) == 1) 2021-11-21T22:03:23.751 DEBUG:teuthology.orchestra.run.gibba036:> return files[0] 2021-11-21T22:03:23.752 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:23.752 DEBUG:teuthology.orchestra.run.gibba036:> for f in files: 2021-11-21T22:03:23.753 DEBUG:teuthology.orchestra.run.gibba036:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-11-21T22:03:23.753 DEBUG:teuthology.orchestra.run.gibba036:> if os.path.exists("/proc/{0}".format(pid)): 2021-11-21T22:03:23.754 DEBUG:teuthology.orchestra.run.gibba036:> return f 2021-11-21T22:03:23.754 DEBUG:teuthology.orchestra.run.gibba036:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-11-21T22:03:23.754 DEBUG:teuthology.orchestra.run.gibba036:> 2021-11-21T22:03:23.755 DEBUG:teuthology.orchestra.run.gibba036:> print(find_socket("client.1")) 2021-11-21T22:03:23.755 DEBUG:teuthology.orchestra.run.gibba036:> ' 2021-11-21T22:03:23.816 INFO:teuthology.orchestra.run.gibba036.stdout:/var/run/ceph/ceph-client.1.14858.asok 2021-11-21T22:03:23.819 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14858.asok 2021-11-21T22:03:23.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.820 DEBUG:teuthology.orchestra.run.gibba036:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14858.asok status 2021-11-21T22:03:23.868 INFO:teuthology.orchestra.run.gibba036.stdout:{ 2021-11-21T22:03:23.869 INFO:teuthology.orchestra.run.gibba036.stdout: "metadata": { 2021-11-21T22:03:23.869 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_sha1": "911bc85f89d5c4ff92555237a37b5b774d7b1332", 2021-11-21T22:03:23.869 INFO:teuthology.orchestra.run.gibba036.stdout: "ceph_version": "ceph version 15.2.15-36-g911bc85f (911bc85f89d5c4ff92555237a37b5b774d7b1332) octopus (stable)", 2021-11-21T22:03:23.869 INFO:teuthology.orchestra.run.gibba036.stdout: "entity_id": "1", 2021-11-21T22:03:23.870 INFO:teuthology.orchestra.run.gibba036.stdout: "hostname": "gibba036", 2021-11-21T22:03:23.870 INFO:teuthology.orchestra.run.gibba036.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-11-21T22:03:23.870 INFO:teuthology.orchestra.run.gibba036.stdout: "pid": "14858", 2021-11-21T22:03:23.870 INFO:teuthology.orchestra.run.gibba036.stdout: "root": "/" 2021-11-21T22:03:23.871 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:23.871 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_count": 0, 2021-11-21T22:03:23.871 INFO:teuthology.orchestra.run.gibba036.stdout: "dentry_pinned_count": 0, 2021-11-21T22:03:23.871 INFO:teuthology.orchestra.run.gibba036.stdout: "id": 9000, 2021-11-21T22:03:23.871 INFO:teuthology.orchestra.run.gibba036.stdout: "inst": { 2021-11-21T22:03:23.872 INFO:teuthology.orchestra.run.gibba036.stdout: "name": { 2021-11-21T22:03:23.872 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "client", 2021-11-21T22:03:23.872 INFO:teuthology.orchestra.run.gibba036.stdout: "num": 9000 2021-11-21T22:03:23.872 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:23.873 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:03:23.873 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:03:23.873 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:03:23.873 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1217093081 2021-11-21T22:03:23.873 INFO:teuthology.orchestra.run.gibba036.stdout: } 2021-11-21T22:03:23.874 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:23.874 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": { 2021-11-21T22:03:23.874 INFO:teuthology.orchestra.run.gibba036.stdout: "type": "v1", 2021-11-21T22:03:23.875 INFO:teuthology.orchestra.run.gibba036.stdout: "addr": "172.21.2.136:0", 2021-11-21T22:03:23.875 INFO:teuthology.orchestra.run.gibba036.stdout: "nonce": 1217093081 2021-11-21T22:03:23.875 INFO:teuthology.orchestra.run.gibba036.stdout: }, 2021-11-21T22:03:23.875 INFO:teuthology.orchestra.run.gibba036.stdout: "inst_str": "client.9000 172.21.2.136:0/1217093081", 2021-11-21T22:03:23.875 INFO:teuthology.orchestra.run.gibba036.stdout: "addr_str": "172.21.2.136:0/1217093081", 2021-11-21T22:03:23.876 INFO:teuthology.orchestra.run.gibba036.stdout: "inode_count": 1, 2021-11-21T22:03:23.876 INFO:teuthology.orchestra.run.gibba036.stdout: "mds_epoch": 77, 2021-11-21T22:03:23.876 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch": 162, 2021-11-21T22:03:23.876 INFO:teuthology.orchestra.run.gibba036.stdout: "osd_epoch_barrier": 0, 2021-11-21T22:03:23.877 INFO:teuthology.orchestra.run.gibba036.stdout: "blacklisted": false, 2021-11-21T22:03:23.877 INFO:teuthology.orchestra.run.gibba036.stdout: "fs_name": "cephfs" 2021-11-21T22:03:23.877 INFO:teuthology.orchestra.run.gibba036.stdout:} 2021-11-21T22:03:23.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.879 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:23.880 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:23.886 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:03:23.886 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:03:23.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.887 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:23.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.938 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:23.938 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:23.944 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:03:23.944 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:03:23.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:23.945 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:23.946 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:23.950 INFO:teuthology.orchestra.run.gibba036.stdout:fuseblk 2021-11-21T22:03:23.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-11-21T22:03:23.952 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba036.front.sepia.ceph.com... 2021-11-21T22:03:23.952 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-21T22:03:23.953 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:24.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba036.stderr:ceph-fuse[14858]: fuse finished with error 0 and tester_r 0 2021-11-21T22:03:24.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:03:24.017 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:03:24.018 DEBUG:teuthology.orchestra.run.gibba036:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:24.027 INFO:teuthology.orchestra.run.gibba036.stdout:ext2/ext3 2021-11-21T22:03:24.028 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-11-21T22:03:24.029 INFO:teuthology.orchestra.run:waiting for 900 2021-11-21T22:03:30.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-21T22:03:30.132 DEBUG:teuthology.orchestra.run.gibba036:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-11-21T22:03:30.137 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-11-21T22:03:30.137 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2021-11-21T22:03:30.152 INFO:teuthology.orchestra.run.gibba019.stdout: File: "." 2021-11-21T22:03:30.153 INFO:teuthology.orchestra.run.gibba019.stdout: ID: 0 Namelen: 255 Type: fuseblk 2021-11-21T22:03:30.153 INFO:teuthology.orchestra.run.gibba019.stdout:Block size: 4194304 Fundamental block size: 4194304 2021-11-21T22:03:30.153 INFO:teuthology.orchestra.run.gibba019.stdout:Blocks: Total: 80930 Free: 80930 Available: 80930 2021-11-21T22:03:30.154 INFO:teuthology.orchestra.run.gibba019.stdout:Inodes: Total: 1 Free: 0 2021-11-21T22:03:30.155 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-11-21T22:03:30.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe268104700 1 Processor -- start 2021-11-21T22:03:30.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe268104700 1 -- start start 2021-11-21T22:03:30.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe268104700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fe2600a6aa0 0x7fe26014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe268104700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2600a6e80 con 0x7fe260062410 2021-11-21T22:03:30.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe268104700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26014a7b0 con 0x7fe2600a86b0 2021-11-21T22:03:30.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe268104700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2600a8bc0 con 0x7fe2600a6aa0 2021-11-21T22:03:30.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe2666a1700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fe260062410 0x7fe2601443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38572/0 (socket says 172.21.2.116:38572) 2021-11-21T22:03:30.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe2666a1700 1 -- 172.21.2.116:0/173053305 learned_addr learned my addr 172.21.2.116:0/173053305 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:30.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594745744 0 0) 0x7fe2600a6e80 con 0x7fe260062410 2021-11-21T22:03:30.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.207+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe248003260 con 0x7fe260062410 2021-11-21T22:03:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735309844 0 0) 0x7fe248003260 con 0x7fe260062410 2021-11-21T22:03:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 >> v2:172.21.2.117:3300/0 conn(0x7fe2600a6aa0 msgr2=0x7fe26014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 --2- 172.21.2.116:0/173053305 >> v2:172.21.2.117:3300/0 conn(0x7fe2600a6aa0 0x7fe26014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 >> v1:172.21.2.117:6789/0 conn(0x7fe2600a86b0 legacy=0x7fe2601470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:30.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26014aa10 con 0x7fe260062410 2021-11-21T22:03:30.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fe258001a00 con 0x7fe260062410 2021-11-21T22:03:30.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 >> v1:172.21.2.116:6789/0 conn(0x7fe260062410 legacy=0x7fe2601443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe248003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe248004500 0x7fe248004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe248005050 0x7fe24800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600a8bc0 con 0x7fe248003820 2021-11-21T22:03:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24800bf40 con 0x7fe248005050 2021-11-21T22:03:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24800c100 con 0x7fe248004500 2021-11-21T22:03:30.223 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe248003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 -- 172.21.2.116:0/173053305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe248004500 msgr2=0x7fe248004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:30.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe248004500 0x7fe248004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 -- 172.21.2.116:0/173053305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe248005050 msgr2=0x7fe24800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:30.224 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe248005050 0x7fe24800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 -- 172.21.2.116:0/173053305 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe248003260 con 0x7fe248003820 2021-11-21T22:03:30.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe2666a1700 1 -- 172.21.2.116:0/173053305 reap_dead start 2021-11-21T22:03:30.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe248003c20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7fe25c002f70 tx=0x7fe25c006ca0).ready entity=mon.0 client_cookie=acd6a8894e4dd2ba server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe25c012460 con 0x7fe248003820 2021-11-21T22:03:30.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c0125c0 con 0x7fe248003820 2021-11-21T22:03:30.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe264e9e700 1 -- 172.21.2.116:0/173053305 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe25c012920 con 0x7fe248003820 2021-11-21T22:03:30.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/173053305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 msgr2=0x7fe248003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe248003c20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7fe25c002f70 tx=0x7fe25c006ca0).stop 2021-11-21T22:03:30.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/173053305 shutdown_connections 2021-11-21T22:03:30.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 --2- 172.21.2.116:0/173053305 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe248003c20 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/173053305 >> 172.21.2.116:0/173053305 conn(0x7fe26005bd20 msgr2=0x7fe26005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:30.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/173053305 shutdown_connections 2021-11-21T22:03:30.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/173053305 wait complete. 2021-11-21T22:03:30.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 Processor -- start 2021-11-21T22:03:30.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- start start 2021-11-21T22:03:30.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe260159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe260064690 0x7fe260067090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2600655c0 0x7fe2600b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600a90a0 con 0x7fe248003820 2021-11-21T22:03:30.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26015a840 con 0x7fe2600655c0 2021-11-21T22:03:30.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe260147460 con 0x7fe260064690 2021-11-21T22:03:30.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe260159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe260159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54858/0 (socket says 172.21.2.116:54858) 2021-11-21T22:03:30.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 -- 172.21.2.116:0/3276705787 learned_addr learned my addr 172.21.2.116:0/3276705787 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:30.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 -- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe260064690 msgr2=0x7fe260067090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe260064690 0x7fe260067090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 -- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2600655c0 msgr2=0x7fe2600b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2600655c0 0x7fe2600b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 -- 172.21.2.116:0/3276705787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26015a3a0 con 0x7fe248003820 2021-11-21T22:03:30.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe265ea0700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe260159cf0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7fe25c002f70 tx=0x7fe25c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe252ffd700 1 -- 172.21.2.116:0/3276705787 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe25c012460 con 0x7fe248003820 2021-11-21T22:03:30.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe252ffd700 1 -- 172.21.2.116:0/3276705787 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c000e90 con 0x7fe248003820 2021-11-21T22:03:30.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe260067580 con 0x7fe248003820 2021-11-21T22:03:30.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2600638c0 con 0x7fe248003820 2021-11-21T22:03:30.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe252ffd700 1 -- 172.21.2.116:0/3276705787 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe25c01e5f0 con 0x7fe248003820 2021-11-21T22:03:30.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c003410 con 0x7fe248003820 2021-11-21T22:03:30.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.211+0000 7fe252ffd700 1 -- 172.21.2.116:0/3276705787 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fe25c01c070 con 0x7fe248003820 2021-11-21T22:03:30.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.215+0000 7fe252ffd700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe240053600 0x7fe240055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.215+0000 7fe252ffd700 1 -- 172.21.2.116:0/3276705787 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe25c06c5c0 con 0x7fe248003820 2021-11-21T22:03:30.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.215+0000 7fe26569f700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe240053600 0x7fe240055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.215+0000 7fe26569f700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe240053600 0x7fe240055ac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fe254000f70 tx=0x7fe2540069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.215+0000 7fe252ffd700 1 -- 172.21.2.116:0/3276705787 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe25c043650 con 0x7fe248003820 2021-11-21T22:03:30.303 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.299+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe23c0036c0 con 0x7fe248003820 2021-11-21T22:03:30.304 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.299+0000 7fe252ffd700 1 -- 172.21.2.116:0/3276705787 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v162) v1 ==== 81+0+33659 (secure 0 0 0) 0x7fe25c034390 con 0x7fe248003820 2021-11-21T22:03:30.304 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:30.305 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:03:30.305 INFO:teuthology.orchestra.run.gibba016.stdout: "epoch": 162, 2021-11-21T22:03:30.305 INFO:teuthology.orchestra.run.gibba016.stdout: "fsid": "fe3c4e1d-e687-4491-9a93-95ac7299e2b3", 2021-11-21T22:03:30.305 INFO:teuthology.orchestra.run.gibba016.stdout: "created": "2021-11-21T21:54:10.975984+0000", 2021-11-21T22:03:30.306 INFO:teuthology.orchestra.run.gibba016.stdout: "modified": "2021-11-21T22:03:19.325611+0000", 2021-11-21T22:03:30.306 INFO:teuthology.orchestra.run.gibba016.stdout: "last_up_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:03:30.306 INFO:teuthology.orchestra.run.gibba016.stdout: "last_in_change": "2021-11-21T21:54:15.911408+0000", 2021-11-21T22:03:30.306 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-11-21T22:03:30.307 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_num": 5799936, 2021-11-21T22:03:30.307 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_set": [ 2021-11-21T22:03:30.307 INFO:teuthology.orchestra.run.gibba016.stdout: "pglog_hardlimit", 2021-11-21T22:03:30.307 INFO:teuthology.orchestra.run.gibba016.stdout: "purged_snapdirs", 2021-11-21T22:03:30.307 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_deletes", 2021-11-21T22:03:30.308 INFO:teuthology.orchestra.run.gibba016.stdout: "sortbitwise" 2021-11-21T22:03:30.308 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:30.308 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_version": 4, 2021-11-21T22:03:30.308 INFO:teuthology.orchestra.run.gibba016.stdout: "full_ratio": 0.94999998807907104, 2021-11-21T22:03:30.309 INFO:teuthology.orchestra.run.gibba016.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-11-21T22:03:30.309 INFO:teuthology.orchestra.run.gibba016.stdout: "nearfull_ratio": 0.85000002384185791, 2021-11-21T22:03:30.309 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_snapshot": "", 2021-11-21T22:03:30.309 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_max": 30, 2021-11-21T22:03:30.309 INFO:teuthology.orchestra.run.gibba016.stdout: "max_osd": 8, 2021-11-21T22:03:30.310 INFO:teuthology.orchestra.run.gibba016.stdout: "require_min_compat_client": "jewel", 2021-11-21T22:03:30.310 INFO:teuthology.orchestra.run.gibba016.stdout: "min_compat_client": "jewel", 2021-11-21T22:03:30.310 INFO:teuthology.orchestra.run.gibba016.stdout: "require_osd_release": "octopus", 2021-11-21T22:03:30.310 INFO:teuthology.orchestra.run.gibba016.stdout: "pools": [ 2021-11-21T22:03:30.311 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.311 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 1, 2021-11-21T22:03:30.311 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "device_health_metrics", 2021-11-21T22:03:30.311 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:13.059512+0000", 2021-11-21T22:03:30.311 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:30.312 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:30.312 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:30.312 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:30.312 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:30.313 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:30.313 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:30.313 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:30.313 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 1, 2021-11-21T22:03:30.313 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 1, 2021-11-21T22:03:30.314 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 1, 2021-11-21T22:03:30.314 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 1, 2021-11-21T22:03:30.314 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 1, 2021-11-21T22:03:30.314 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:30.315 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:30.315 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:30.315 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:30.315 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:30.315 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:30.316 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:30.316 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.316 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "15", 2021-11-21T22:03:30.316 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:30.316 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:30.317 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:30.317 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:30.317 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:30.317 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:30.317 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:30.318 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:30.318 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:30.318 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:30.318 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:30.319 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:30.319 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:30.319 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:30.319 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:30.319 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:30.320 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:30.320 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:30.320 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:30.320 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:30.320 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:30.321 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:30.321 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:30.321 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:30.321 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:30.321 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:30.322 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.322 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:30.322 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:30.322 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:30.323 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:30.323 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:30.323 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:30.323 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:30.323 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:30.324 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:30.324 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:30.324 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:30.324 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:03:30.324 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 1 2021-11-21T22:03:30.325 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.325 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:30.325 INFO:teuthology.orchestra.run.gibba016.stdout: "mgr_devicehealth": {} 2021-11-21T22:03:30.325 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.326 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.326 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.326 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 2, 2021-11-21T22:03:30.326 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "rbd", 2021-11-21T22:03:30.326 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T21:54:20.960577+0000", 2021-11-21T22:03:30.327 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:30.327 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:30.327 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:30.327 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:30.327 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:30.328 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:30.328 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:30.328 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:30.328 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:03:30.329 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:03:30.329 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:03:30.329 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:03:30.329 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:03:30.329 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:30.330 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:30.330 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:30.330 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:30.330 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:30.330 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:30.331 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:30.331 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.331 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "18", 2021-11-21T22:03:30.331 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:30.332 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:30.332 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:30.332 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:30.332 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:30.332 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:30.333 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:30.333 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:30.333 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:30.333 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:30.333 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:30.334 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:30.334 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:30.334 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:30.334 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:30.335 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:30.335 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:30.335 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:30.335 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:30.335 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:30.336 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:30.336 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:30.336 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:30.336 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:30.336 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:30.337 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:30.337 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.337 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:30.337 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:30.337 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:30.338 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:30.338 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:30.338 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:30.338 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:30.339 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:30.339 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:30.339 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:30.339 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:30.339 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:03:30.340 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:30.340 INFO:teuthology.orchestra.run.gibba016.stdout: "rbd": {} 2021-11-21T22:03:30.340 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.340 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.340 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.341 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 29, 2021-11-21T22:03:30.341 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_metadata", 2021-11-21T22:03:30.341 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:03:16.171372+0000", 2021-11-21T22:03:30.341 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:30.341 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:30.342 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:30.342 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:30.342 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:30.342 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:30.343 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:30.343 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:30.343 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:03:30.343 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:03:30.343 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:03:30.344 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:03:30.344 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:03:30.344 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:30.344 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:30.344 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:30.345 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:30.345 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:30.345 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:30.345 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:30.345 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.346 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "161", 2021-11-21T22:03:30.346 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:30.346 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:30.346 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:30.347 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:30.347 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:30.347 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:30.347 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:30.347 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:30.348 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:30.348 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:30.348 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:30.348 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:30.348 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:30.349 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:30.349 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:30.349 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:30.349 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:30.349 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:30.350 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:30.350 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:30.350 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:30.350 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:30.351 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:30.351 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:30.351 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:30.351 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:30.351 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.352 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:30.352 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:30.352 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:30.352 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:30.352 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:30.353 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:30.353 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:30.353 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:30.353 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:30.353 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:30.354 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:30.354 INFO:teuthology.orchestra.run.gibba016.stdout: "options": { 2021-11-21T22:03:30.354 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_bias": 4, 2021-11-21T22:03:30.354 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_min": 16, 2021-11-21T22:03:30.355 INFO:teuthology.orchestra.run.gibba016.stdout: "recovery_priority": 5 2021-11-21T22:03:30.355 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.355 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:30.355 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:03:30.355 INFO:teuthology.orchestra.run.gibba016.stdout: "metadata": "cephfs" 2021-11-21T22:03:30.356 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.356 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.356 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.356 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.357 INFO:teuthology.orchestra.run.gibba016.stdout: "pool": 30, 2021-11-21T22:03:30.357 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_name": "cephfs_data", 2021-11-21T22:03:30.357 INFO:teuthology.orchestra.run.gibba016.stdout: "create_time": "2021-11-21T22:03:17.340772+0000", 2021-11-21T22:03:30.357 INFO:teuthology.orchestra.run.gibba016.stdout: "flags": 1, 2021-11-21T22:03:30.358 INFO:teuthology.orchestra.run.gibba016.stdout: "flags_names": "hashpspool", 2021-11-21T22:03:30.358 INFO:teuthology.orchestra.run.gibba016.stdout: "type": 1, 2021-11-21T22:03:30.358 INFO:teuthology.orchestra.run.gibba016.stdout: "size": 2, 2021-11-21T22:03:30.358 INFO:teuthology.orchestra.run.gibba016.stdout: "min_size": 1, 2021-11-21T22:03:30.358 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_rule": 0, 2021-11-21T22:03:30.359 INFO:teuthology.orchestra.run.gibba016.stdout: "object_hash": 2, 2021-11-21T22:03:30.359 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_autoscale_mode": "off", 2021-11-21T22:03:30.359 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num": 8, 2021-11-21T22:03:30.359 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num": 8, 2021-11-21T22:03:30.360 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_placement_num_target": 8, 2021-11-21T22:03:30.360 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_target": 8, 2021-11-21T22:03:30.360 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_num_pending": 8, 2021-11-21T22:03:30.360 INFO:teuthology.orchestra.run.gibba016.stdout: "last_pg_merge_meta": { 2021-11-21T22:03:30.360 INFO:teuthology.orchestra.run.gibba016.stdout: "source_pgid": "0.0", 2021-11-21T22:03:30.361 INFO:teuthology.orchestra.run.gibba016.stdout: "ready_epoch": 0, 2021-11-21T22:03:30.361 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_started": 0, 2021-11-21T22:03:30.361 INFO:teuthology.orchestra.run.gibba016.stdout: "last_epoch_clean": 0, 2021-11-21T22:03:30.361 INFO:teuthology.orchestra.run.gibba016.stdout: "source_version": "0'0", 2021-11-21T22:03:30.361 INFO:teuthology.orchestra.run.gibba016.stdout: "target_version": "0'0" 2021-11-21T22:03:30.362 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.362 INFO:teuthology.orchestra.run.gibba016.stdout: "last_change": "162", 2021-11-21T22:03:30.362 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend": "0", 2021-11-21T22:03:30.362 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_prenautilus": "0", 2021-11-21T22:03:30.363 INFO:teuthology.orchestra.run.gibba016.stdout: "last_force_op_resend_preluminous": "0", 2021-11-21T22:03:30.363 INFO:teuthology.orchestra.run.gibba016.stdout: "auid": 0, 2021-11-21T22:03:30.363 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_mode": "selfmanaged", 2021-11-21T22:03:30.363 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_seq": 0, 2021-11-21T22:03:30.363 INFO:teuthology.orchestra.run.gibba016.stdout: "snap_epoch": 0, 2021-11-21T22:03:30.364 INFO:teuthology.orchestra.run.gibba016.stdout: "pool_snaps": [], 2021-11-21T22:03:30.364 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps": "[]", 2021-11-21T22:03:30.364 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_bytes": 0, 2021-11-21T22:03:30.364 INFO:teuthology.orchestra.run.gibba016.stdout: "quota_max_objects": 0, 2021-11-21T22:03:30.364 INFO:teuthology.orchestra.run.gibba016.stdout: "tiers": [], 2021-11-21T22:03:30.365 INFO:teuthology.orchestra.run.gibba016.stdout: "tier_of": -1, 2021-11-21T22:03:30.365 INFO:teuthology.orchestra.run.gibba016.stdout: "read_tier": -1, 2021-11-21T22:03:30.365 INFO:teuthology.orchestra.run.gibba016.stdout: "write_tier": -1, 2021-11-21T22:03:30.365 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_mode": "none", 2021-11-21T22:03:30.365 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_bytes": 0, 2021-11-21T22:03:30.366 INFO:teuthology.orchestra.run.gibba016.stdout: "target_max_objects": 0, 2021-11-21T22:03:30.366 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-11-21T22:03:30.366 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-11-21T22:03:30.366 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_target_full_ratio_micro": 800000, 2021-11-21T22:03:30.367 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_flush_age": 0, 2021-11-21T22:03:30.367 INFO:teuthology.orchestra.run.gibba016.stdout: "cache_min_evict_age": 0, 2021-11-21T22:03:30.367 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profile": "", 2021-11-21T22:03:30.367 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_params": { 2021-11-21T22:03:30.367 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "none" 2021-11-21T22:03:30.368 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.368 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_period": 0, 2021-11-21T22:03:30.368 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_count": 0, 2021-11-21T22:03:30.368 INFO:teuthology.orchestra.run.gibba016.stdout: "use_gmt_hitset": true, 2021-11-21T22:03:30.368 INFO:teuthology.orchestra.run.gibba016.stdout: "min_read_recency_for_promote": 0, 2021-11-21T22:03:30.369 INFO:teuthology.orchestra.run.gibba016.stdout: "min_write_recency_for_promote": 0, 2021-11-21T22:03:30.369 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_grade_decay_rate": 0, 2021-11-21T22:03:30.369 INFO:teuthology.orchestra.run.gibba016.stdout: "hit_set_search_last_n": 0, 2021-11-21T22:03:30.369 INFO:teuthology.orchestra.run.gibba016.stdout: "grade_table": [], 2021-11-21T22:03:30.370 INFO:teuthology.orchestra.run.gibba016.stdout: "stripe_width": 0, 2021-11-21T22:03:30.370 INFO:teuthology.orchestra.run.gibba016.stdout: "expected_num_objects": 0, 2021-11-21T22:03:30.370 INFO:teuthology.orchestra.run.gibba016.stdout: "fast_read": false, 2021-11-21T22:03:30.370 INFO:teuthology.orchestra.run.gibba016.stdout: "options": {}, 2021-11-21T22:03:30.370 INFO:teuthology.orchestra.run.gibba016.stdout: "application_metadata": { 2021-11-21T22:03:30.371 INFO:teuthology.orchestra.run.gibba016.stdout: "cephfs": { 2021-11-21T22:03:30.371 INFO:teuthology.orchestra.run.gibba016.stdout: "data": "cephfs" 2021-11-21T22:03:30.371 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.371 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.371 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.372 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:30.372 INFO:teuthology.orchestra.run.gibba016.stdout: "osds": [ 2021-11-21T22:03:30.372 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.372 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:03:30.373 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c3baac6c-e283-460c-b082-9b135ec136dc", 2021-11-21T22:03:30.373 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.373 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.373 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.373 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.374 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.374 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.374 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.374 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 160, 2021-11-21T22:03:30.374 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.375 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.375 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.375 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.375 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.375 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.376 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6800", 2021-11-21T22:03:30.376 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.376 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.377 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.377 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.378 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6801", 2021-11-21T22:03:30.378 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.378 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.378 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.379 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.379 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.379 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.379 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.380 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.380 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6802", 2021-11-21T22:03:30.380 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.380 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.380 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.381 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.381 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6803", 2021-11-21T22:03:30.381 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.381 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.382 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.382 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.382 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.382 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.382 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.383 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.383 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6806", 2021-11-21T22:03:30.383 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.383 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.384 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.384 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.384 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6807", 2021-11-21T22:03:30.384 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.384 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.385 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.385 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.385 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.385 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.386 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.386 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.386 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6804", 2021-11-21T22:03:30.386 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.387 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.387 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.387 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.388 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6805", 2021-11-21T22:03:30.388 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14707 2021-11-21T22:03:30.388 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.388 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.388 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.389 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6801/14707", 2021-11-21T22:03:30.389 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6803/14707", 2021-11-21T22:03:30.389 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6807/14707", 2021-11-21T22:03:30.389 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6805/14707", 2021-11-21T22:03:30.390 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.390 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.390 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.390 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.390 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.391 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.391 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:03:30.391 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "61210302-784a-4f36-a7bd-7c05c0e31483", 2021-11-21T22:03:30.391 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.392 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.392 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.392 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.392 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.392 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.393 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.393 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 159, 2021-11-21T22:03:30.393 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.393 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.394 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.394 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.394 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.394 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.394 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6808", 2021-11-21T22:03:30.395 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.395 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.395 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.395 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.396 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6809", 2021-11-21T22:03:30.396 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.396 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.396 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.396 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.397 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.397 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.397 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.397 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.398 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6810", 2021-11-21T22:03:30.398 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.398 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.398 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.398 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.399 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6811", 2021-11-21T22:03:30.399 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.399 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.399 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.400 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.400 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.400 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.400 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.400 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.401 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6814", 2021-11-21T22:03:30.401 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.401 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.401 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.401 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.402 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6815", 2021-11-21T22:03:30.402 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.402 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.402 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.403 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.403 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.403 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.403 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.404 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.404 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6812", 2021-11-21T22:03:30.404 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.404 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.405 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.405 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.405 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6813", 2021-11-21T22:03:30.405 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14708 2021-11-21T22:03:30.405 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.406 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.406 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.406 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6809/14708", 2021-11-21T22:03:30.406 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6811/14708", 2021-11-21T22:03:30.407 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6815/14708", 2021-11-21T22:03:30.407 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6813/14708", 2021-11-21T22:03:30.407 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.407 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.407 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.408 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.408 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.408 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.408 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:03:30.408 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56", 2021-11-21T22:03:30.409 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.409 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.409 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.410 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.410 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.410 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.410 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.410 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 160, 2021-11-21T22:03:30.411 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.411 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.411 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.411 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.412 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.412 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.412 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6816", 2021-11-21T22:03:30.412 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.412 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.413 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.413 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.413 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6817", 2021-11-21T22:03:30.413 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.414 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.414 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.414 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.414 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.414 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.415 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.415 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.415 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6818", 2021-11-21T22:03:30.415 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.416 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.416 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.416 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.416 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6819", 2021-11-21T22:03:30.416 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.417 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.417 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.417 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.417 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.418 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.418 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.418 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.418 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6822", 2021-11-21T22:03:30.418 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.419 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.419 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.419 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.419 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6823", 2021-11-21T22:03:30.420 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.420 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.420 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.420 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.420 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.421 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.421 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.421 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.421 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6820", 2021-11-21T22:03:30.422 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.422 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.422 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.422 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.423 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6821", 2021-11-21T22:03:30.423 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14709 2021-11-21T22:03:30.423 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.423 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.423 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.424 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6817/14709", 2021-11-21T22:03:30.424 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6819/14709", 2021-11-21T22:03:30.424 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6823/14709", 2021-11-21T22:03:30.424 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6821/14709", 2021-11-21T22:03:30.425 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.425 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.425 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.425 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.426 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.426 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.426 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:03:30.426 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "dcebb53a-56e7-4d2f-9ac5-a61776613ba6", 2021-11-21T22:03:30.427 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.427 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.427 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.427 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.428 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.428 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.428 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.428 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 160, 2021-11-21T22:03:30.428 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.429 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.429 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.429 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.429 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.430 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.430 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6824", 2021-11-21T22:03:30.430 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.430 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.431 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.431 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.431 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6825", 2021-11-21T22:03:30.431 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.431 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.432 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.432 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.432 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.432 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.432 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.433 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.433 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6826", 2021-11-21T22:03:30.433 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.433 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.434 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.434 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.435 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6827", 2021-11-21T22:03:30.435 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.435 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.436 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.436 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.436 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.436 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.437 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.437 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.437 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6830", 2021-11-21T22:03:30.437 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.437 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.438 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.438 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.438 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6831", 2021-11-21T22:03:30.438 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.439 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.439 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.439 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.439 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.439 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.440 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.440 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.440 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6828", 2021-11-21T22:03:30.440 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.440 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.441 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.441 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.441 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.116:6829", 2021-11-21T22:03:30.441 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14710 2021-11-21T22:03:30.442 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.442 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.442 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.442 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.116:6825/14710", 2021-11-21T22:03:30.442 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.116:6827/14710", 2021-11-21T22:03:30.443 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.116:6831/14710", 2021-11-21T22:03:30.443 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.116:6829/14710", 2021-11-21T22:03:30.443 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.443 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.444 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.444 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.444 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.444 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.444 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:03:30.445 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "a7251112-9008-418e-a7b6-4501e7b9365d", 2021-11-21T22:03:30.445 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.445 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.445 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.446 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.446 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.446 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.446 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.446 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 159, 2021-11-21T22:03:30.447 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.447 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.447 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.447 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.448 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.448 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.448 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6800", 2021-11-21T22:03:30.448 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.448 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.449 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.449 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.449 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6801", 2021-11-21T22:03:30.449 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.449 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.450 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.450 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.450 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.450 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.451 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.451 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.451 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6802", 2021-11-21T22:03:30.451 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.451 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.452 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.452 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.452 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6803", 2021-11-21T22:03:30.452 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.453 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.453 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.453 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.453 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.453 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.454 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.454 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.454 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6806", 2021-11-21T22:03:30.454 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.455 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.455 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.455 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.455 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6807", 2021-11-21T22:03:30.455 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.456 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.456 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.456 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.456 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.457 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.457 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.457 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.457 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6804", 2021-11-21T22:03:30.458 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.458 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.458 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.458 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.458 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6805", 2021-11-21T22:03:30.459 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14956 2021-11-21T22:03:30.459 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.459 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.459 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.460 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6801/14956", 2021-11-21T22:03:30.460 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6803/14956", 2021-11-21T22:03:30.460 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6807/14956", 2021-11-21T22:03:30.460 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6805/14956", 2021-11-21T22:03:30.460 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.461 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.461 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.461 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.461 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.462 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.462 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:03:30.462 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "0c8cf701-2ace-4542-a985-c4435cd023fb", 2021-11-21T22:03:30.462 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.462 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.463 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.463 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.463 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.463 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.464 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.464 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 160, 2021-11-21T22:03:30.464 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.464 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.464 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.465 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.465 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.465 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.465 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6808", 2021-11-21T22:03:30.465 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.466 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.466 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.466 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.466 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6809", 2021-11-21T22:03:30.467 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.467 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.467 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.467 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.467 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.468 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.468 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.468 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.468 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6810", 2021-11-21T22:03:30.469 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.469 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.469 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.469 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.469 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6811", 2021-11-21T22:03:30.470 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.470 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.470 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.470 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.471 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.471 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.471 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.471 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.471 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6814", 2021-11-21T22:03:30.472 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.472 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.472 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.472 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.473 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6815", 2021-11-21T22:03:30.473 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.473 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.473 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.473 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.474 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.474 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.474 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.474 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.475 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6812", 2021-11-21T22:03:30.475 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.475 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.475 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.475 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.476 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6813", 2021-11-21T22:03:30.476 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14957 2021-11-21T22:03:30.476 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.476 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.477 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.477 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6809/14957", 2021-11-21T22:03:30.477 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6811/14957", 2021-11-21T22:03:30.477 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6815/14957", 2021-11-21T22:03:30.478 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6813/14957", 2021-11-21T22:03:30.478 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.478 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.478 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.479 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.479 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.479 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.479 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:03:30.479 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "66ee78f9-53ac-45a6-a441-8a71e4e06bac", 2021-11-21T22:03:30.480 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.480 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.480 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.480 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.481 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.481 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.481 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.481 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 160, 2021-11-21T22:03:30.481 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.482 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.482 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.482 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.482 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.483 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.483 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6816", 2021-11-21T22:03:30.483 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.483 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.483 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.484 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.484 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6817", 2021-11-21T22:03:30.484 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.484 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.485 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.485 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.485 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.485 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.485 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.486 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.486 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6818", 2021-11-21T22:03:30.486 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.486 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.487 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.487 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.487 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6819", 2021-11-21T22:03:30.487 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.487 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.488 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.488 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.488 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.488 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.488 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.489 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.489 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6822", 2021-11-21T22:03:30.489 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.489 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.490 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.490 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.490 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6824", 2021-11-21T22:03:30.490 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.491 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.491 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.491 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.491 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.491 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.492 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.492 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.492 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6820", 2021-11-21T22:03:30.492 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.493 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.493 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.493 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.493 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6821", 2021-11-21T22:03:30.493 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14958 2021-11-21T22:03:30.494 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.494 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.494 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.494 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6817/14958", 2021-11-21T22:03:30.495 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6819/14958", 2021-11-21T22:03:30.495 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6824/14958", 2021-11-21T22:03:30.495 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6821/14958", 2021-11-21T22:03:30.495 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.495 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.496 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.496 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.496 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.496 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.496 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:03:30.497 INFO:teuthology.orchestra.run.gibba016.stdout: "uuid": "7c1bf6d9-c24a-4373-9eee-5488fd33e157", 2021-11-21T22:03:30.497 INFO:teuthology.orchestra.run.gibba016.stdout: "up": 1, 2021-11-21T22:03:30.497 INFO:teuthology.orchestra.run.gibba016.stdout: "in": 1, 2021-11-21T22:03:30.497 INFO:teuthology.orchestra.run.gibba016.stdout: "weight": 1, 2021-11-21T22:03:30.498 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_affinity": 1, 2021-11-21T22:03:30.498 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_begin": 0, 2021-11-21T22:03:30.498 INFO:teuthology.orchestra.run.gibba016.stdout: "last_clean_end": 0, 2021-11-21T22:03:30.498 INFO:teuthology.orchestra.run.gibba016.stdout: "up_from": 13, 2021-11-21T22:03:30.498 INFO:teuthology.orchestra.run.gibba016.stdout: "up_thru": 160, 2021-11-21T22:03:30.499 INFO:teuthology.orchestra.run.gibba016.stdout: "down_at": 0, 2021-11-21T22:03:30.499 INFO:teuthology.orchestra.run.gibba016.stdout: "lost_at": 0, 2021-11-21T22:03:30.499 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addrs": { 2021-11-21T22:03:30.499 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.500 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.500 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.500 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6823", 2021-11-21T22:03:30.500 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.500 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.501 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.501 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.501 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6825", 2021-11-21T22:03:30.501 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.501 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.502 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.502 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.502 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addrs": { 2021-11-21T22:03:30.502 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.503 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.503 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.503 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6826", 2021-11-21T22:03:30.503 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.503 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.504 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.504 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.504 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6827", 2021-11-21T22:03:30.504 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.505 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.505 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.505 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.505 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addrs": { 2021-11-21T22:03:30.505 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.506 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.506 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.506 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6830", 2021-11-21T22:03:30.506 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.507 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.507 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.507 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.507 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6831", 2021-11-21T22:03:30.507 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.508 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.508 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.508 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.508 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addrs": { 2021-11-21T22:03:30.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addrvec": [ 2021-11-21T22:03:30.509 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.509 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v2", 2021-11-21T22:03:30.509 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6828", 2021-11-21T22:03:30.509 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.510 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.510 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.510 INFO:teuthology.orchestra.run.gibba016.stdout: "type": "v1", 2021-11-21T22:03:30.510 INFO:teuthology.orchestra.run.gibba016.stdout: "addr": "172.21.2.117:6829", 2021-11-21T22:03:30.511 INFO:teuthology.orchestra.run.gibba016.stdout: "nonce": 14960 2021-11-21T22:03:30.511 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.511 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.511 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.511 INFO:teuthology.orchestra.run.gibba016.stdout: "public_addr": "172.21.2.117:6825/14960", 2021-11-21T22:03:30.512 INFO:teuthology.orchestra.run.gibba016.stdout: "cluster_addr": "172.21.2.117:6827/14960", 2021-11-21T22:03:30.512 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_back_addr": "172.21.2.117:6831/14960", 2021-11-21T22:03:30.512 INFO:teuthology.orchestra.run.gibba016.stdout: "heartbeat_front_addr": "172.21.2.117:6829/14960", 2021-11-21T22:03:30.512 INFO:teuthology.orchestra.run.gibba016.stdout: "state": [ 2021-11-21T22:03:30.512 INFO:teuthology.orchestra.run.gibba016.stdout: "exists", 2021-11-21T22:03:30.513 INFO:teuthology.orchestra.run.gibba016.stdout: "up" 2021-11-21T22:03:30.513 INFO:teuthology.orchestra.run.gibba016.stdout: ] 2021-11-21T22:03:30.513 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.513 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:30.514 INFO:teuthology.orchestra.run.gibba016.stdout: "osd_xinfo": [ 2021-11-21T22:03:30.514 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.514 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 0, 2021-11-21T22:03:30.514 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.514 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.515 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.515 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.515 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.515 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.929221+0000", 2021-11-21T22:03:30.516 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.516 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.516 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.516 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 1, 2021-11-21T22:03:30.516 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.517 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.517 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.517 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.517 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.518 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.917817+0000", 2021-11-21T22:03:30.518 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.518 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.518 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.518 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 2, 2021-11-21T22:03:30.519 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.519 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.519 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.519 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.520 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.520 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.955321+0000", 2021-11-21T22:03:30.520 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.520 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.520 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.521 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 3, 2021-11-21T22:03:30.521 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.521 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.521 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.521 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.522 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.522 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.949148+0000", 2021-11-21T22:03:30.522 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.522 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.523 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.523 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 4, 2021-11-21T22:03:30.523 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.523 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.523 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.524 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.524 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.524 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.005322+0000", 2021-11-21T22:03:30.524 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.525 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.525 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.525 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 5, 2021-11-21T22:03:30.525 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.525 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.526 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.526 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.526 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.526 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.943674+0000", 2021-11-21T22:03:30.527 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.527 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.527 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.527 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 6, 2021-11-21T22:03:30.527 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.528 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.528 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.528 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.528 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.529 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:13.990443+0000", 2021-11-21T22:03:30.529 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.529 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.529 INFO:teuthology.orchestra.run.gibba016.stdout: { 2021-11-21T22:03:30.530 INFO:teuthology.orchestra.run.gibba016.stdout: "osd": 7, 2021-11-21T22:03:30.530 INFO:teuthology.orchestra.run.gibba016.stdout: "down_stamp": "0.000000", 2021-11-21T22:03:30.530 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_probability": 0, 2021-11-21T22:03:30.530 INFO:teuthology.orchestra.run.gibba016.stdout: "laggy_interval": 0, 2021-11-21T22:03:30.530 INFO:teuthology.orchestra.run.gibba016.stdout: "features": 4540138292840890367, 2021-11-21T22:03:30.531 INFO:teuthology.orchestra.run.gibba016.stdout: "old_weight": 0, 2021-11-21T22:03:30.531 INFO:teuthology.orchestra.run.gibba016.stdout: "last_purged_snaps_scrub": "2021-11-21T21:54:14.011952+0000", 2021-11-21T22:03:30.531 INFO:teuthology.orchestra.run.gibba016.stdout: "dead_epoch": 0 2021-11-21T22:03:30.531 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.532 INFO:teuthology.orchestra.run.gibba016.stdout: ], 2021-11-21T22:03:30.532 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap": [], 2021-11-21T22:03:30.532 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_upmap_items": [], 2021-11-21T22:03:30.532 INFO:teuthology.orchestra.run.gibba016.stdout: "pg_temp": [], 2021-11-21T22:03:30.532 INFO:teuthology.orchestra.run.gibba016.stdout: "primary_temp": [], 2021-11-21T22:03:30.533 INFO:teuthology.orchestra.run.gibba016.stdout: "blacklist": {}, 2021-11-21T22:03:30.533 INFO:teuthology.orchestra.run.gibba016.stdout: "erasure_code_profiles": { 2021-11-21T22:03:30.533 INFO:teuthology.orchestra.run.gibba016.stdout: "default": { 2021-11-21T22:03:30.533 INFO:teuthology.orchestra.run.gibba016.stdout: "crush-failure-domain": "osd", 2021-11-21T22:03:30.534 INFO:teuthology.orchestra.run.gibba016.stdout: "k": "2", 2021-11-21T22:03:30.534 INFO:teuthology.orchestra.run.gibba016.stdout: "m": "1", 2021-11-21T22:03:30.534 INFO:teuthology.orchestra.run.gibba016.stdout: "plugin": "jerasure", 2021-11-21T22:03:30.534 INFO:teuthology.orchestra.run.gibba016.stdout: "ruleset-failure-domain": "osd", 2021-11-21T22:03:30.534 INFO:teuthology.orchestra.run.gibba016.stdout: "technique": "reed_sol_van" 2021-11-21T22:03:30.535 INFO:teuthology.orchestra.run.gibba016.stdout: } 2021-11-21T22:03:30.535 INFO:teuthology.orchestra.run.gibba016.stdout: }, 2021-11-21T22:03:30.535 INFO:teuthology.orchestra.run.gibba016.stdout: "removed_snaps_queue": [], 2021-11-21T22:03:30.535 INFO:teuthology.orchestra.run.gibba016.stdout: "new_removed_snaps": [], 2021-11-21T22:03:30.536 INFO:teuthology.orchestra.run.gibba016.stdout: "new_purged_snaps": [], 2021-11-21T22:03:30.536 INFO:teuthology.orchestra.run.gibba016.stdout: "crush_node_flags": {}, 2021-11-21T22:03:30.536 INFO:teuthology.orchestra.run.gibba016.stdout: "device_class_flags": {} 2021-11-21T22:03:30.536 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:03:30.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.299+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe240053600 msgr2=0x7fe240055ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.299+0000 7fe268104700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe240053600 0x7fe240055ac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fe254000f70 tx=0x7fe2540069b0).stop 2021-11-21T22:03:30.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.299+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 msgr2=0x7fe260159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.538 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.299+0000 7fe268104700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe260159cf0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7fe25c002f70 tx=0x7fe25c014000).stop 2021-11-21T22:03:30.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 shutdown_connections 2021-11-21T22:03:30.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fe2600655c0 0x7fe2600b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fe260064690 0x7fe260067090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.539 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fe240053600 0x7fe240055ac0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 --2- 172.21.2.116:0/3276705787 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fe248003820 0x7fe260159cf0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 >> 172.21.2.116:0/3276705787 conn(0x7fe26005bd20 msgr2=0x7fe260148ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:30.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 shutdown_connections 2021-11-21T22:03:30.540 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.303+0000 7fe268104700 1 -- 172.21.2.116:0/3276705787 wait complete. 2021-11-21T22:03:30.541 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:30.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc8f6c700 1 Processor -- start 2021-11-21T22:03:30.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc8f6c700 1 -- start start 2021-11-21T22:03:30.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc8f6c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc40a6aa0 0x7f0bc414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc8f6c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0bc40a86b0 0x7f0bc41470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.600 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc8f6c700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc40a6e80 con 0x7f0bc41475d0 2021-11-21T22:03:30.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc8f6c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc40a8bc0 con 0x7f0bc40a86b0 2021-11-21T22:03:30.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc8f6c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc414a7b0 con 0x7f0bc40a6aa0 2021-11-21T22:03:30.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc1d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0bc40a86b0 0x7f0bc41470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc1d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f0bc40a86b0 0x7f0bc41470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54864/0 (socket says 172.21.2.116:54864) 2021-11-21T22:03:30.601 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/2968214454 learned_addr learned my addr 172.21.2.116:0/2968214454 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:30.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.591+0000 7f0bc2d9d700 1 --1- 172.21.2.116:0/2968214454 >> v1:172.21.2.116:6789/0 conn(0x7f0bc41475d0 0x7f0bc41443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38588/0 (socket says 172.21.2.116:38588) 2021-11-21T22:03:30.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/2968214454 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc40a6aa0 msgr2=0x7f0bc414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:30.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 --2- 172.21.2.116:0/2968214454 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc40a6aa0 0x7f0bc414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/2968214454 >> v1:172.21.2.116:6789/0 conn(0x7f0bc41475d0 legacy=0x7f0bc41443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/2968214454 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc414aa10 con 0x7f0bc40a86b0 2021-11-21T22:03:30.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 --2- 172.21.2.116:0/2968214454 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc41470e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f0bac002f70 tx=0x7f0bac006eb0).ready entity=mon.0 client_cookie=be37d4828de5472d server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc159a700 1 -- 172.21.2.116:0/2968214454 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bac00d5c0 con 0x7f0bc40a86b0 2021-11-21T22:03:30.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc159a700 1 -- 172.21.2.116:0/2968214454 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac000e90 con 0x7f0bc40a86b0 2021-11-21T22:03:30.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc159a700 1 -- 172.21.2.116:0/2968214454 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bac00d950 con 0x7f0bc40a86b0 2021-11-21T22:03:30.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/2968214454 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 msgr2=0x7f0bc41470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/2968214454 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc41470e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f0bac002f70 tx=0x7f0bac006eb0).stop 2021-11-21T22:03:30.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/2968214454 shutdown_connections 2021-11-21T22:03:30.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/2968214454 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc41470e0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/2968214454 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc40a6aa0 0x7f0bc414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/2968214454 >> 172.21.2.116:0/2968214454 conn(0x7f0bc405bd20 msgr2=0x7f0bc405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:30.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/2968214454 shutdown_connections 2021-11-21T22:03:30.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/2968214454 wait complete. 2021-11-21T22:03:30.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 Processor -- start 2021-11-21T22:03:30.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- start start 2021-11-21T22:03:30.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0bc40a6aa0 0x7f0bc4064350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc4064bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc41475d0 0x7f0bc40650b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40a8d20 con 0x7f0bc40a86b0 2021-11-21T22:03:30.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40a93b0 con 0x7f0bc40a6aa0 2021-11-21T22:03:30.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4156f00 con 0x7f0bc41475d0 2021-11-21T22:03:30.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc4064bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc4064bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54868/0 (socket says 172.21.2.116:54868) 2021-11-21T22:03:30.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/3097342167 learned_addr learned my addr 172.21.2.116:0/3097342167 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:30.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc41475d0 msgr2=0x7f0bc40650b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:30.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc41475d0 0x7f0bc40650b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0bc40a6aa0 msgr2=0x7f0bc4064350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:30.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0bc40a6aa0 0x7f0bc4064350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 -- 172.21.2.116:0/3097342167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac00b1e0 con 0x7f0bc40a86b0 2021-11-21T22:03:30.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc1d9b700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc4064bc0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f0bac002f70 tx=0x7f0bac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bbb7fe700 1 -- 172.21.2.116:0/3097342167 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bac006b60 con 0x7f0bc40a86b0 2021-11-21T22:03:30.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bbb7fe700 1 -- 172.21.2.116:0/3097342167 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac011ce0 con 0x7f0bc40a86b0 2021-11-21T22:03:30.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bbb7fe700 1 -- 172.21.2.116:0/3097342167 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bac01e650 con 0x7f0bc40a86b0 2021-11-21T22:03:30.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc41571a0 con 0x7f0bc40a86b0 2021-11-21T22:03:30.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc4154020 con 0x7f0bc40a86b0 2021-11-21T22:03:30.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc4157330 con 0x7f0bc40a86b0 2021-11-21T22:03:30.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.595+0000 7f0bbb7fe700 1 -- 172.21.2.116:0/3097342167 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0bac01a040 con 0x7f0bc40a86b0 2021-11-21T22:03:30.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.599+0000 7f0bbb7fe700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ba80536d0 0x7f0ba8055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.599+0000 7f0bbb7fe700 1 -- 172.21.2.116:0/3097342167 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0bac06c8e0 con 0x7f0bc40a86b0 2021-11-21T22:03:30.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.599+0000 7f0bc259c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ba80536d0 0x7f0ba8055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.599+0000 7f0bc259c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ba80536d0 0x7f0ba8055b90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f0bb4002f70 tx=0x7f0bb4006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.599+0000 7f0bbb7fe700 1 -- 172.21.2.116:0/3097342167 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0bac043a40 con 0x7f0bc40a86b0 2021-11-21T22:03:30.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.707+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bc40573d0 con 0x7f0bc40a86b0 2021-11-21T22:03:30.712 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.707+0000 7f0bbb7fe700 1 -- 172.21.2.116:0/3097342167 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f0bac023090 con 0x7f0bc40a86b0 2021-11-21T22:03:30.712 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:30.713 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":77,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1070380430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1070380430},{"type":"v1","addr":"172.21.2.117:6835","nonce":1070380430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75},{"gid":8796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/27380863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":27380863},{"type":"v1","addr":"172.21.2.116:6833","nonce":27380863}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:03:18.322670+0000","modified":"2021-11-21T22:03:19.327237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8799":{"gid":8799,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.2.116:6835/403152574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6834","nonce":403152574},{"type":"v1","addr":"172.21.2.116:6835","nonce":403152574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-11-21T22:03:30.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.707+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ba80536d0 msgr2=0x7f0ba8055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.707+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ba80536d0 0x7f0ba8055b90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f0bb4002f70 tx=0x7f0bb4006a00).stop 2021-11-21T22:03:30.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.707+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 msgr2=0x7f0bc4064bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.714 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.707+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc4064bc0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f0bac002f70 tx=0x7f0bac015000).stop 2021-11-21T22:03:30.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 shutdown_connections 2021-11-21T22:03:30.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0bc41475d0 0x7f0bc40650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0ba80536d0 0x7f0ba8055b90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.715 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0bc40a86b0 0x7f0bc4064bc0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 --2- 172.21.2.116:0/3097342167 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0bc40a6aa0 0x7f0bc4064350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 >> 172.21.2.116:0/3097342167 conn(0x7f0bc405bd20 msgr2=0x7f0bc40b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 shutdown_connections 2021-11-21T22:03:30.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.711+0000 7f0bc8f6c700 1 -- 172.21.2.116:0/3097342167 wait complete. 2021-11-21T22:03:30.717 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 77 2021-11-21T22:03:30.719 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-11-21T22:03:30.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 Processor -- start 2021-11-21T22:03:30.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- start start 2021-11-21T22:03:30.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb1c00b02e0 0x7fb1c005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00af620 0x7fb1c0142f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fb1c00afa80 0x7fb1c0143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c00b06c0 con 0x7fb1c00b02e0 2021-11-21T22:03:30.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c00adf20 con 0x7fb1c00afa80 2021-11-21T22:03:30.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00b2470 con 0x7fb1c00af620 2021-11-21T22:03:30.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb1c00b02e0 0x7fb1c005f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fb1c00b02e0 0x7fb1c005f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54876/0 (socket says 172.21.2.116:54876) 2021-11-21T22:03:30.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 -- 172.21.2.116:0/1664432376 learned_addr learned my addr 172.21.2.116:0/1664432376 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:30.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 -- 172.21.2.116:0/1664432376 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00af620 msgr2=0x7fb1c0142f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:30.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 --2- 172.21.2.116:0/1664432376 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00af620 0x7fb1c0142f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 -- 172.21.2.116:0/1664432376 >> v2:172.21.2.117:3300/0 conn(0x7fb1c00afa80 msgr2=0x7fb1c0143460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:03:30.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 --2- 172.21.2.116:0/1664432376 >> v2:172.21.2.117:3300/0 conn(0x7fb1c00afa80 0x7fb1c0143460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 -- 172.21.2.116:0/1664432376 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00b26d0 con 0x7fb1c00b02e0 2021-11-21T22:03:30.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c59e8700 1 --2- 172.21.2.116:0/1664432376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00b02e0 0x7fb1c005f730 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7fb1bc002f70 tx=0x7fb1bc006f40).ready entity=mon.0 client_cookie=80aa35f0cacff88a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c49e6700 1 -- 172.21.2.116:0/1664432376 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1bc00dc70 con 0x7fb1c00b02e0 2021-11-21T22:03:30.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c49e6700 1 -- 172.21.2.116:0/1664432376 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc000e90 con 0x7fb1c00b02e0 2021-11-21T22:03:30.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c49e6700 1 -- 172.21.2.116:0/1664432376 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1bc00be90 con 0x7fb1c00b02e0 2021-11-21T22:03:30.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/1664432376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00b02e0 msgr2=0x7fb1c005f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/1664432376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00b02e0 0x7fb1c005f730 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7fb1bc002f70 tx=0x7fb1bc006f40).stop 2021-11-21T22:03:30.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/1664432376 shutdown_connections 2021-11-21T22:03:30.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/1664432376 >> v2:172.21.2.117:3300/0 conn(0x7fb1c00afa80 0x7fb1c0143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/1664432376 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00af620 0x7fb1c0142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/1664432376 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00b02e0 0x7fb1c005f730 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/1664432376 >> 172.21.2.116:0/1664432376 conn(0x7fb1c00b5ff0 msgr2=0x7fb1c00b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:30.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/1664432376 shutdown_connections 2021-11-21T22:03:30.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.767+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/1664432376 wait complete. 2021-11-21T22:03:30.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 Processor -- start 2021-11-21T22:03:30.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 -- start start 2021-11-21T22:03:30.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00af620 0x7fb1c00a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00afa80 0x7fb1c0066b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1c00b02e0 0x7fb1c00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01558f0 con 0x7fb1c00af620 2021-11-21T22:03:30.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0155b90 con 0x7fb1c00b02e0 2021-11-21T22:03:30.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0155e30 con 0x7fb1c00afa80 2021-11-21T22:03:30.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00af620 0x7fb1c00a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00af620 0x7fb1c00a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54882/0 (socket says 172.21.2.116:54882) 2021-11-21T22:03:30.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 -- 172.21.2.116:0/489871183 learned_addr learned my addr 172.21.2.116:0/489871183 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:30.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 -- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00afa80 msgr2=0x7fb1c0066b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:30.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00afa80 0x7fb1c0066b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 -- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1c00b02e0 msgr2=0x7fb1c00a4770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:03:30.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1c00b02e0 0x7fb1c00a4770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 -- 172.21.2.116:0/489871183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc00b460 con 0x7fb1c00af620 2021-11-21T22:03:30.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c59e8700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00af620 0x7fb1c00a2a30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7fb1bc002f70 tx=0x7fb1c00b7000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1b2ffd700 1 -- 172.21.2.116:0/489871183 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1bc01d070 con 0x7fb1c00af620 2021-11-21T22:03:30.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1b2ffd700 1 -- 172.21.2.116:0/489871183 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc022600 con 0x7fb1c00af620 2021-11-21T22:03:30.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c01560d0 con 0x7fb1c00af620 2021-11-21T22:03:30.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1b2ffd700 1 -- 172.21.2.116:0/489871183 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1bc022960 con 0x7fb1c00af620 2021-11-21T22:03:30.786 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c0156570 con 0x7fb1c00af620 2021-11-21T22:03:30.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c0156260 con 0x7fb1c00af620 2021-11-21T22:03:30.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1b2ffd700 1 -- 172.21.2.116:0/489871183 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fb1bc029070 con 0x7fb1c00af620 2021-11-21T22:03:30.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1b2ffd700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1a00536d0 0x7fb1a0055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:30.787 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1b2ffd700 1 -- 172.21.2.116:0/489871183 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb1bc06cdf0 con 0x7fb1c00af620 2021-11-21T22:03:30.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.771+0000 7fb1c51e7700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1a00536d0 0x7fb1a0055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:30.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.775+0000 7fb1c51e7700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1a00536d0 0x7fb1a0055b90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fb1b4002fb0 tx=0x7fb1b4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:30.788 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.775+0000 7fb1b2ffd700 1 -- 172.21.2.116:0/489871183 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1bc043ec0 con 0x7fb1c00af620 2021-11-21T22:03:30.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.883+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb1c00b0da0 con 0x7fb1c00af620 2021-11-21T22:03:30.963 INFO:tasks.ceph.mon.a.gibba016.stderr:2021-11-21T22:03:30.963+0000 7f1e52db9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-21T22:03:30.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.971+0000 7fb1b2ffd700 1 -- 172.21.2.116:0/489871183 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb1bc027030 con 0x7fb1c00af620 2021-11-21T22:03:30.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1a00536d0 msgr2=0x7fb1a0055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1a00536d0 0x7fb1a0055b90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fb1b4002fb0 tx=0x7fb1b4006930).stop 2021-11-21T22:03:30.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00af620 msgr2=0x7fb1c00a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:30.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00af620 0x7fb1c00a2a30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7fb1bc002f70 tx=0x7fb1c00b7000).stop 2021-11-21T22:03:30.980 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 shutdown_connections 2021-11-21T22:03:30.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fb1c00b02e0 0x7fb1c00a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fb1c00afa80 0x7fb1c0066b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:30.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fb1a00536d0 0x7fb1a0055b90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 --2- 172.21.2.116:0/489871183 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fb1c00af620 0x7fb1c00a2a30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:30.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 >> 172.21.2.116:0/489871183 conn(0x7fb1c00b5ff0 msgr2=0x7fb1c005bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:30.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 shutdown_connections 2021-11-21T22:03:30.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:30.975+0000 7fb1c7c4c700 1 -- 172.21.2.116:0/489871183 wait complete. 2021-11-21T22:03:30.986 INFO:teuthology.orchestra.run.gibba016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-11-21T22:03:30.987 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-21T22:03:30.990 INFO:tasks.ceph.mds.c.gibba016.stdout:starting mds.c at 2021-11-21T22:03:31.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 Processor -- start 2021-11-21T22:03:31.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- start start 2021-11-21T22:03:31.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49dc0a86b0 0x7f49dc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f49dc062410 0x7f49dc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49dc0a6e80 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.047 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0a8bc0 con 0x7f49dc0a86b0 2021-11-21T22:03:31.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc14a7b0 con 0x7f49dc062410 2021-11-21T22:03:31.048 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49dc0a86b0 0x7f49dc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:31.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f49dc0a86b0 0x7f49dc14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54910/0 (socket says 172.21.2.116:54910) 2021-11-21T22:03:31.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 -- 172.21.2.116:0/631571304 learned_addr learned my addr 172.21.2.116:0/631571304 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:31.049 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 -- 172.21.2.116:0/631571304 >> v2:172.21.2.117:3300/0 conn(0x7f49dc062410 msgr2=0x7f49dc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:31.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 --2- 172.21.2.116:0/631571304 >> v2:172.21.2.117:3300/0 conn(0x7f49dc062410 0x7f49dc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.050 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 -- 172.21.2.116:0/631571304 >> v1:172.21.2.117:6789/0 conn(0x7f49dc0a6aa0 legacy=0x7f49dc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:31.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 -- 172.21.2.116:0/631571304 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc14aa10 con 0x7f49dc0a86b0 2021-11-21T22:03:31.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d9d9b700 1 --2- 172.21.2.116:0/631571304 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a86b0 0x7f49dc14b150 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 rx=0x7f49dc061ab0 tx=0x7f49c4006eb0).ready entity=mon.0 client_cookie=c5f867acd6de98eb server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:31.051 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d959a700 1 -- 172.21.2.116:0/631571304 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c400d5c0 con 0x7f49dc0a86b0 2021-11-21T22:03:31.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d959a700 1 -- 172.21.2.116:0/631571304 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4000e90 con 0x7f49dc0a86b0 2021-11-21T22:03:31.052 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49d959a700 1 -- 172.21.2.116:0/631571304 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c400d910 con 0x7f49dc0a86b0 2021-11-21T22:03:31.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- 172.21.2.116:0/631571304 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a86b0 msgr2=0x7f49dc14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 --2- 172.21.2.116:0/631571304 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a86b0 0x7f49dc14b150 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 rx=0x7f49dc061ab0 tx=0x7f49c4006eb0).stop 2021-11-21T22:03:31.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- 172.21.2.116:0/631571304 shutdown_connections 2021-11-21T22:03:31.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 --2- 172.21.2.116:0/631571304 >> v2:172.21.2.117:3300/0 conn(0x7f49dc062410 0x7f49dc1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 --2- 172.21.2.116:0/631571304 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a86b0 0x7f49dc14b150 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:31.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- 172.21.2.116:0/631571304 >> 172.21.2.116:0/631571304 conn(0x7f49dc05bd20 msgr2=0x7f49dc05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:31.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- 172.21.2.116:0/631571304 shutdown_connections 2021-11-21T22:03:31.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.035+0000 7f49e0f76700 1 -- 172.21.2.116:0/631571304 wait complete. 2021-11-21T22:03:31.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 Processor -- start 2021-11-21T22:03:31.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 -- start start 2021-11-21T22:03:31.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49dc062410 0x7f49dc0671d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a6aa0 0x7f49dc156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49dc0a86b0 0x7f49dc156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0a8d90 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc1574a0 con 0x7f49dc062410 2021-11-21T22:03:31.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0a9910 con 0x7f49dc0a86b0 2021-11-21T22:03:31.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a6aa0 0x7f49dc156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:31.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a6aa0 0x7f49dc156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54914/0 (socket says 172.21.2.116:54914) 2021-11-21T22:03:31.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 -- 172.21.2.116:0/3967710933 learned_addr learned my addr 172.21.2.116:0/3967710933 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:31.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 -- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49dc0a86b0 msgr2=0x7f49dc156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49dc0a86b0 0x7f49dc156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 -- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49dc062410 msgr2=0x7f49dc0671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49dc062410 0x7f49dc0671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 -- 172.21.2.116:0/3967710933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c400b1e0 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d9d9b700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a6aa0 0x7f49dc156ac0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f49dc061ab0 tx=0x7f49c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:31.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d37fe700 1 -- 172.21.2.116:0/3967710933 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c4002f70 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d37fe700 1 -- 172.21.2.116:0/3967710933 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c40068b0 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d37fe700 1 -- 172.21.2.116:0/3967710933 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c401e460 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0a9bb0 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc0aa110 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc0573d0 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d37fe700 1 -- 172.21.2.116:0/3967710933 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f49c4025030 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d37fe700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49c00536f0 0x7f49c0055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49d37fe700 1 -- 172.21.2.116:0/3967710933 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f49c406cdb0 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.039+0000 7f49da59c700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49c00536f0 0x7f49c0055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:31.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.043+0000 7f49da59c700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49c00536f0 0x7f49c0055bb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f49cc002f90 tx=0x7f49cc0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:31.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.043+0000 7f49d37fe700 1 -- 172.21.2.116:0/3967710933 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49c4043db0 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49dc05db70 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49d37fe700 1 -- 172.21.2.116:0/3967710933 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f49c4023020 con 0x7f49dc0a6aa0 2021-11-21T22:03:31.156 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:03:31.156 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":78,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.117:6835/1070380430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6834","nonce":1070380430},{"type":"v1","addr":"172.21.2.117:6835","nonce":1070380430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75},{"gid":8796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.116:6833/27380863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6832","nonce":27380863},{"type":"v1","addr":"172.21.2.116:6833","nonce":27380863}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-21T22:03:18.322670+0000","modified":"2021-11-21T22:03:30.963990+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":163,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-11-21T22:03:31.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49c00536f0 msgr2=0x7f49c0055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49c00536f0 0x7f49c0055bb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f49cc002f90 tx=0x7f49cc0069b0).stop 2021-11-21T22:03:31.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a6aa0 msgr2=0x7f49dc156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a6aa0 0x7f49dc156ac0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f49dc061ab0 tx=0x7f49c4015000).stop 2021-11-21T22:03:31.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 shutdown_connections 2021-11-21T22:03:31.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f49dc0a86b0 0x7f49dc156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f49c00536f0 0x7f49c0055bb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:31.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f49dc0a6aa0 0x7f49dc156ac0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:31.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 --2- 172.21.2.116:0/3967710933 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f49dc062410 0x7f49dc0671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 >> 172.21.2.116:0/3967710933 conn(0x7f49dc05bd20 msgr2=0x7f49dc0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:31.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 shutdown_connections 2021-11-21T22:03:31.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.151+0000 7f49e0f76700 1 -- 172.21.2.116:0/3967710933 wait complete. 2021-11-21T22:03:31.161 INFO:teuthology.orchestra.run.gibba016.stderr:dumped fsmap epoch 78 2021-11-21T22:03:31.163 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-11-21T22:03:31.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 Processor -- start 2021-11-21T22:03:31.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 -- start start 2021-11-21T22:03:31.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f1d140adc10 0x7f1d140b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d14144a10 0x7f1d14141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d140b3c50 con 0x7f1d14141510 2021-11-21T22:03:31.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d140b3ef0 con 0x7f1d14144a10 2021-11-21T22:03:31.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d140b4190 con 0x7f1d140adc10 2021-11-21T22:03:31.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d14144a10 0x7f1d14141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:31.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f1d14144a10 0x7f1d14141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54924/0 (socket says 172.21.2.116:54924) 2021-11-21T22:03:31.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 -- 172.21.2.116:0/3513960456 learned_addr learned my addr 172.21.2.116:0/3513960456 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:31.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 -- 172.21.2.116:0/3513960456 >> v2:172.21.2.117:3300/0 conn(0x7f1d140adc10 msgr2=0x7f1d140b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:31.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 --2- 172.21.2.116:0/3513960456 >> v2:172.21.2.117:3300/0 conn(0x7f1d140adc10 0x7f1d140b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 -- 172.21.2.116:0/3513960456 >> v1:172.21.2.117:6789/0 conn(0x7f1d14141510 legacy=0x7f1d1413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:31.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 -- 172.21.2.116:0/3513960456 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d14140b80 con 0x7f1d14144a10 2021-11-21T22:03:31.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d12ffd700 1 --2- 172.21.2.116:0/3513960456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14141020 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7f1d1405f430 tx=0x7f1d04006eb0).ready entity=mon.0 client_cookie=d6f072292befa976 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:31.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d127fc700 1 -- 172.21.2.116:0/3513960456 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0400d5c0 con 0x7f1d14144a10 2021-11-21T22:03:31.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d127fc700 1 -- 172.21.2.116:0/3513960456 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04000e90 con 0x7f1d14144a10 2021-11-21T22:03:31.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d127fc700 1 -- 172.21.2.116:0/3513960456 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0400d910 con 0x7f1d14144a10 2021-11-21T22:03:31.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 -- 172.21.2.116:0/3513960456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 msgr2=0x7f1d14141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.211+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3513960456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14141020 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7f1d1405f430 tx=0x7f1d04006eb0).stop 2021-11-21T22:03:31.225 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- 172.21.2.116:0/3513960456 shutdown_connections 2021-11-21T22:03:31.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3513960456 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14141020 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:31.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3513960456 >> v2:172.21.2.117:3300/0 conn(0x7f1d140adc10 0x7f1d140b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- 172.21.2.116:0/3513960456 >> 172.21.2.116:0/3513960456 conn(0x7f1d1405bd20 msgr2=0x7f1d1405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:31.226 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- 172.21.2.116:0/3513960456 shutdown_connections 2021-11-21T22:03:31.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- 172.21.2.116:0/3513960456 wait complete. 2021-11-21T22:03:31.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 Processor -- start 2021-11-21T22:03:31.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- start start 2021-11-21T22:03:31.227 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d140adc10 0x7f1d14143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d14141510 0x7f1d141525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.228 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d141161d0 con 0x7f1d14144a10 2021-11-21T22:03:31.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14155340 con 0x7f1d140adc10 2021-11-21T22:03:31.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140afae0 con 0x7f1d14141510 2021-11-21T22:03:31.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:31.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54926/0 (socket says 172.21.2.116:54926) 2021-11-21T22:03:31.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 -- 172.21.2.116:0/3251428953 learned_addr learned my addr 172.21.2.116:0/3251428953 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:31.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 -- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d14141510 msgr2=0x7f1d141525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d14141510 0x7f1d141525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 -- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d140adc10 msgr2=0x7f1d14143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d140adc10 0x7f1d14143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 -- 172.21.2.116:0/3251428953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0400b1e0 con 0x7f1d14144a10 2021-11-21T22:03:31.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d13fff700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14155730 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f1d140aecf0 tx=0x7f1d0c009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:31.233 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d10ff9700 1 -- 172.21.2.116:0/3251428953 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c001500 con 0x7f1d14144a10 2021-11-21T22:03:31.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d10ff9700 1 -- 172.21.2.116:0/3251428953 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c001660 con 0x7f1d14144a10 2021-11-21T22:03:31.234 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d140afd80 con 0x7f1d14144a10 2021-11-21T22:03:31.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d10ff9700 1 -- 172.21.2.116:0/3251428953 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c0019c0 con 0x7f1d14144a10 2021-11-21T22:03:31.235 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d14151d70 con 0x7f1d14144a10 2021-11-21T22:03:31.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d14155490 con 0x7f1d14144a10 2021-11-21T22:03:31.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d10ff9700 1 -- 172.21.2.116:0/3251428953 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f1d0c018030 con 0x7f1d14144a10 2021-11-21T22:03:31.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d10ff9700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf40536a0 0x7f1cf4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:31.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.215+0000 7f1d10ff9700 1 -- 172.21.2.116:0/3251428953 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f1d0c06b450 con 0x7f1d14144a10 2021-11-21T22:03:31.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.219+0000 7f1d137fe700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf40536a0 0x7f1cf4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:31.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.219+0000 7f1d137fe700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf40536a0 0x7f1cf4055b60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f1d08002f70 tx=0x7f1d08006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:31.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.219+0000 7f1d10ff9700 1 -- 172.21.2.116:0/3251428953 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d14155490 con 0x7f1d14144a10 2021-11-21T22:03:31.330 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.327+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1d14155490 con 0x7f1d14144a10 2021-11-21T22:03:31.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d10ff9700 1 -- 172.21.2.116:0/3251428953 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1d14155490 con 0x7f1d14144a10 2021-11-21T22:03:31.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf40536a0 msgr2=0x7f1cf4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf40536a0 0x7f1cf4055b60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f1d08002f70 tx=0x7f1d08006a80).stop 2021-11-21T22:03:31.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 msgr2=0x7f1d14155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:31.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14155730 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f1d140aecf0 tx=0x7f1d0c009120).stop 2021-11-21T22:03:31.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 shutdown_connections 2021-11-21T22:03:31.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f1cf40536a0 0x7f1cf4055b60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:31.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1d14144a10 0x7f1d14155730 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:31.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f1d14141510 0x7f1d141525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 --2- 172.21.2.116:0/3251428953 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f1d140adc10 0x7f1d14143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:31.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 >> 172.21.2.116:0/3251428953 conn(0x7f1d1405bd20 msgr2=0x7f1d1413fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:31.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 shutdown_connections 2021-11-21T22:03:31.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:31.979+0000 7f1d1a091700 1 -- 172.21.2.116:0/3251428953 wait complete. 2021-11-21T22:03:31.990 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-11-21T22:03:32.044 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.039+0000 7fde553be700 1 Processor -- start 2021-11-21T22:03:32.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.039+0000 7fde553be700 1 -- start start 2021-11-21T22:03:32.045 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.039+0000 7fde553be700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fde500b11c0 0x7fde50147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.039+0000 7fde553be700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500b1940 con 0x7fde50143e60 2021-11-21T22:03:32.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.039+0000 7fde553be700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500b15a0 con 0x7fde500a8670 2021-11-21T22:03:32.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.039+0000 7fde553be700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500b1b70 con 0x7fde500b11c0 2021-11-21T22:03:32.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7fde50143e60 0x7fde501442d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38662/0 (socket says 172.21.2.116:38662) 2021-11-21T22:03:32.046 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 -- 172.21.2.116:0/1347124742 learned_addr learned my addr 172.21.2.116:0/1347124742 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:03:32.053 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519132523 0 0) 0x7fde500b1940 con 0x7fde50143e60 2021-11-21T22:03:32.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde30003260 con 0x7fde50143e60 2021-11-21T22:03:32.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4effd700 1 --2- 172.21.2.116:0/1347124742 >> v2:172.21.2.117:3300/0 conn(0x7fde500b11c0 0x7fde50147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:32.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2733168432 0 0) 0x7fde30003260 con 0x7fde50143e60 2021-11-21T22:03:32.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 >> v2:172.21.2.117:3300/0 conn(0x7fde500b11c0 msgr2=0x7fde50147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:32.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 --2- 172.21.2.116:0/1347124742 >> v2:172.21.2.117:3300/0 conn(0x7fde500b11c0 0x7fde50147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:32.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 >> v1:172.21.2.117:6789/0 conn(0x7fde500a8670 legacy=0x7fde50147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:32.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500b1e10 con 0x7fde50143e60 2021-11-21T22:03:32.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7fde480019e0 con 0x7fde50143e60 2021-11-21T22:03:32.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 >> v1:172.21.2.116:6789/0 conn(0x7fde50143e60 legacy=0x7fde501442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:32.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30003750 0x7fde30003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde30004500 0x7fde30004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde30004ec0 0x7fde3000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500b1b70 con 0x7fde30004ec0 2021-11-21T22:03:32.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3000be30 con 0x7fde30003750 2021-11-21T22:03:32.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3000bff0 con 0x7fde30004500 2021-11-21T22:03:32.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde30004ec0 0x7fde3000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:32.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4effd700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30003750 0x7fde30003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:32.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4e7fc700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde30004500 0x7fde30004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:32.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 -- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde30004500 msgr2=0x7fde30004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:32.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde30004500 0x7fde30004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:32.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 -- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30003750 msgr2=0x7fde30003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:32.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30003750 0x7fde30003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:32.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 -- 172.21.2.116:0/1347124742 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30003260 con 0x7fde30004ec0 2021-11-21T22:03:32.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4e7fc700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde30004500 0x7fde30004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:32.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 -- 172.21.2.116:0/1347124742 reap_dead start 2021-11-21T22:03:32.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4f7fe700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde30004ec0 0x7fde3000b750 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 rx=0x7fde50068600 tx=0x7fde500b1fa0).ready entity=mon.0 client_cookie=336d4389c725dbf1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:32.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4800cd40 con 0x7fde30004ec0 2021-11-21T22:03:32.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde48005990 con 0x7fde30004ec0 2021-11-21T22:03:32.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4dffb700 1 -- 172.21.2.116:0/1347124742 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4800cea0 con 0x7fde30004ec0 2021-11-21T22:03:32.061 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- 172.21.2.116:0/1347124742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde30004ec0 msgr2=0x7fde3000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:32.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde30004ec0 0x7fde3000b750 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 rx=0x7fde50068600 tx=0x7fde500b1fa0).stop 2021-11-21T22:03:32.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- 172.21.2.116:0/1347124742 shutdown_connections 2021-11-21T22:03:32.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 --2- 172.21.2.116:0/1347124742 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde30004ec0 0x7fde3000b750 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:32.062 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- 172.21.2.116:0/1347124742 >> 172.21.2.116:0/1347124742 conn(0x7fde5005bd20 msgr2=0x7fde5005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:32.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- 172.21.2.116:0/1347124742 shutdown_connections 2021-11-21T22:03:32.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- 172.21.2.116:0/1347124742 wait complete. 2021-11-21T22:03:32.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 Processor -- start 2021-11-21T22:03:32.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- start start 2021-11-21T22:03:32.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30004ec0 0x7fde5013d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde5013d190 0x7fde50143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde5014a530 0x7fde50159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5013f030 con 0x7fde5013d190 2021-11-21T22:03:32.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500b5b40 con 0x7fde30004ec0 2021-11-21T22:03:32.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde553be700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde50148df0 con 0x7fde5014a530 2021-11-21T22:03:32.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde5013d190 0x7fde50143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:32.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4e7fc700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde5013d190 0x7fde50143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54950/0 (socket says 172.21.2.116:54950) 2021-11-21T22:03:32.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.043+0000 7fde4e7fc700 1 -- 172.21.2.116:0/819761466 learned_addr learned my addr 172.21.2.116:0/819761466 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:32.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4e7fc700 1 -- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde5014a530 msgr2=0x7fde50159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:32.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4e7fc700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde5014a530 0x7fde50159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:32.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4e7fc700 1 -- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30004ec0 msgr2=0x7fde5013d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:32.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4effd700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30004ec0 0x7fde5013d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:32.066 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4e7fc700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30004ec0 0x7fde5013d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:32.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4e7fc700 1 -- 172.21.2.116:0/819761466 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50149090 con 0x7fde5013d190 2021-11-21T22:03:32.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4effd700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30004ec0 0x7fde5013d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:03:32.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4e7fc700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde5013d190 0x7fde50143230 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7fde38008000 tx=0x7fde38009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:32.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde47fff700 1 -- 172.21.2.116:0/819761466 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde38007e80 con 0x7fde5013d190 2021-11-21T22:03:32.067 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde47fff700 1 -- 172.21.2.116:0/819761466 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde38000e90 con 0x7fde5013d190 2021-11-21T22:03:32.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde50141350 con 0x7fde5013d190 2021-11-21T22:03:32.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5014ab80 con 0x7fde5013d190 2021-11-21T22:03:32.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde47fff700 1 -- 172.21.2.116:0/819761466 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde380106d0 con 0x7fde5013d190 2021-11-21T22:03:32.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde500b5d40 con 0x7fde5013d190 2021-11-21T22:03:32.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde47fff700 1 -- 172.21.2.116:0/819761466 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fde38019030 con 0x7fde5013d190 2021-11-21T22:03:32.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde47fff700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde34053720 0x7fde34055be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:32.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde47fff700 1 -- 172.21.2.116:0/819761466 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fde3806af30 con 0x7fde5013d190 2021-11-21T22:03:32.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4effd700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde34053720 0x7fde34055be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:32.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde4effd700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde34053720 0x7fde34055be0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fde4000c000 tx=0x7fde4000b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:32.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.047+0000 7fde47fff700 1 -- 172.21.2.116:0/819761466 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde38041f30 con 0x7fde5013d190 2021-11-21T22:03:32.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.131+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde5013d710 con 0x7fde5013d190 2021-11-21T22:03:33.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde47fff700 1 -- 172.21.2.116:0/819761466 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) v1 ==== 185+0+0 (secure 0 0 0) 0x7fde38045550 con 0x7fde5013d190 2021-11-21T22:03:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde34053720 msgr2=0x7fde34055be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde34053720 0x7fde34055be0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fde4000c000 tx=0x7fde4000b040).stop 2021-11-21T22:03:33.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde5013d190 msgr2=0x7fde50143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde5013d190 0x7fde50143230 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7fde38008000 tx=0x7fde38009040).stop 2021-11-21T22:03:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 shutdown_connections 2021-11-21T22:03:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fde5014a530 0x7fde50159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:33.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fde34053720 0x7fde34055be0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fde5013d190 0x7fde50143230 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 --2- 172.21.2.116:0/819761466 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fde30004ec0 0x7fde5013d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 >> 172.21.2.116:0/819761466 conn(0x7fde5005bd20 msgr2=0x7fde5014adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 shutdown_connections 2021-11-21T22:03:33.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:32.999+0000 7fde553be700 1 -- 172.21.2.116:0/819761466 wait complete. 2021-11-21T22:03:33.006 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_metadata' removed 2021-11-21T22:03:33.010 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-11-21T22:03:33.063 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e7048700 1 Processor -- start 2021-11-21T22:03:33.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e7048700 1 -- start start 2021-11-21T22:03:33.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e7048700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc6e00b11c0 0x7fc6e0147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:33.064 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e7048700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 0x7fc6e0147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:33.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e7048700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e00b1940 con 0x7fc6e0147c10 2021-11-21T22:03:33.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e7048700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e00b15a0 con 0x7fc6e00b11c0 2021-11-21T22:03:33.065 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e7048700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e00b1b70 con 0x7fc6e00a8670 2021-11-21T22:03:33.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc6e00b11c0 0x7fc6e0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:33.068 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc6e00b11c0 0x7fc6e0147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54956/0 (socket says 172.21.2.116:54956) 2021-11-21T22:03:33.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 -- 172.21.2.116:0/619647723 learned_addr learned my addr 172.21.2.116:0/619647723 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:33.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e55e5700 1 --1- 172.21.2.116:0/619647723 >> v1:172.21.2.116:6789/0 conn(0x7fc6e0147c10 0x7fc6e014a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38682/0 (socket says 172.21.2.116:38682) 2021-11-21T22:03:33.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 -- 172.21.2.116:0/619647723 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 msgr2=0x7fc6e0147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:03:33.069 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 --2- 172.21.2.116:0/619647723 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 0x7fc6e0147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:33.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 -- 172.21.2.116:0/619647723 >> v1:172.21.2.116:6789/0 conn(0x7fc6e0147c10 legacy=0x7fc6e014a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:33.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 -- 172.21.2.116:0/619647723 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e00b1e10 con 0x7fc6e00b11c0 2021-11-21T22:03:33.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6e4de4700 1 --2- 172.21.2.116:0/619647723 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e00b11c0 0x7fc6e0147230 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fc6d0002f70 tx=0x7fc6d0006eb0).ready entity=mon.0 client_cookie=5ad39eef0b81a1d9 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:33.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6df7fe700 1 -- 172.21.2.116:0/619647723 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d000d5c0 con 0x7fc6e00b11c0 2021-11-21T22:03:33.070 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6df7fe700 1 -- 172.21.2.116:0/619647723 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d0000e90 con 0x7fc6e00b11c0 2021-11-21T22:03:33.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.059+0000 7fc6df7fe700 1 -- 172.21.2.116:0/619647723 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d000d950 con 0x7fc6e00b11c0 2021-11-21T22:03:33.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/619647723 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e00b11c0 msgr2=0x7fc6e0147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:33.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 --2- 172.21.2.116:0/619647723 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e00b11c0 0x7fc6e0147230 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fc6d0002f70 tx=0x7fc6d0006eb0).stop 2021-11-21T22:03:33.071 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/619647723 shutdown_connections 2021-11-21T22:03:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 --2- 172.21.2.116:0/619647723 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 0x7fc6e0147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 --2- 172.21.2.116:0/619647723 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e00b11c0 0x7fc6e0147230 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/619647723 >> 172.21.2.116:0/619647723 conn(0x7fc6e005bd20 msgr2=0x7fc6e005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/619647723 shutdown_connections 2021-11-21T22:03:33.072 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/619647723 wait complete. 2021-11-21T22:03:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 Processor -- start 2021-11-21T22:03:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- start start 2021-11-21T22:03:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 0x7fc6e0067350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:33.073 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6e00b11c0 0x7fc6e01414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e0147c10 0x7fc6e01419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e0149800 con 0x7fc6e0147c10 2021-11-21T22:03:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e0140b60 con 0x7fc6e00b11c0 2021-11-21T22:03:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e0063f10 con 0x7fc6e00a8670 2021-11-21T22:03:33.074 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e0147c10 0x7fc6e01419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e0147c10 0x7fc6e01419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:54962/0 (socket says 172.21.2.116:54962) 2021-11-21T22:03:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 -- 172.21.2.116:0/3354293251 learned_addr learned my addr 172.21.2.116:0/3354293251 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:03:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 -- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 msgr2=0x7fc6e0067350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:33.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 0x7fc6e0067350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 -- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6e00b11c0 msgr2=0x7fc6e01414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6e00b11c0 0x7fc6e01414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 -- 172.21.2.116:0/3354293251 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d000b1e0 con 0x7fc6e0147c10 2021-11-21T22:03:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e55e5700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e0147c10 0x7fc6e01419c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fc6e00a8390 tx=0x7fc6d80013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:33.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6ddffb700 1 -- 172.21.2.116:0/3354293251 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d8005ae0 con 0x7fc6e0147c10 2021-11-21T22:03:33.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6ddffb700 1 -- 172.21.2.116:0/3354293251 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8005c40 con 0x7fc6e0147c10 2021-11-21T22:03:33.077 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e00641b0 con 0x7fc6e0147c10 2021-11-21T22:03:33.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e0141eb0 con 0x7fc6e0147c10 2021-11-21T22:03:33.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6ddffb700 1 -- 172.21.2.116:0/3354293251 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d80070d0 con 0x7fc6e0147c10 2021-11-21T22:03:33.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e00b20b0 con 0x7fc6e0147c10 2021-11-21T22:03:33.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6ddffb700 1 -- 172.21.2.116:0/3354293251 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc6d801a060 con 0x7fc6e0147c10 2021-11-21T22:03:33.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6ddffb700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6c40535b0 0x7fc6c4055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:03:33.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.063+0000 7fc6ddffb700 1 -- 172.21.2.116:0/3354293251 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fc6d806b180 con 0x7fc6e0147c10 2021-11-21T22:03:33.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.067+0000 7fc6e4de4700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6c40535b0 0x7fc6c4055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:03:33.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.067+0000 7fc6e4de4700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6c40535b0 0x7fc6c4055a70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fc6d0002f70 tx=0x7fc6d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:03:33.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.067+0000 7fc6ddffb700 1 -- 172.21.2.116:0/3354293251 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc6d8042440 con 0x7fc6e0147c10 2021-11-21T22:03:33.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:33.151+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc6e00573d0 con 0x7fc6e0147c10 2021-11-21T22:03:34.021 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.007+0000 7fc6ddffb700 1 -- 172.21.2.116:0/3354293251 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc6d8018070 con 0x7fc6e0147c10 2021-11-21T22:03:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6c40535b0 msgr2=0x7fc6c4055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6c40535b0 0x7fc6c4055a70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fc6d0002f70 tx=0x7fc6d0015000).stop 2021-11-21T22:03:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e0147c10 msgr2=0x7fc6e01419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:03:34.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e0147c10 0x7fc6e01419c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fc6e00a8390 tx=0x7fc6d80013a0).stop 2021-11-21T22:03:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 shutdown_connections 2021-11-21T22:03:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc6c40535b0 0x7fc6c4055a70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc6e0147c10 0x7fc6e01419c0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:03:34.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc6e00b11c0 0x7fc6e01414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:34.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 --2- 172.21.2.116:0/3354293251 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc6e00a8670 0x7fc6e0067350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:03:34.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 >> 172.21.2.116:0/3354293251 conn(0x7fc6e005bd20 msgr2=0x7fc6e00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:03:34.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 shutdown_connections 2021-11-21T22:03:34.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:03:34.011+0000 7fc6e7048700 1 -- 172.21.2.116:0/3354293251 wait complete. 2021-11-21T22:03:34.025 INFO:teuthology.orchestra.run.gibba016.stderr:pool 'cephfs_data' removed 2021-11-21T22:03:34.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2021-11-21T22:03:34.026 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2021-11-21T22:03:34.633 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:03:34.637 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:03:34.640 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:03:34.643 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:04.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:04:04.132 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:04:04.133 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:04:04.685 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:04.688 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:04.692 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:04.695 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:34.033 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-11-21T22:04:34.035 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-21T22:04:34.758 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:34.762 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:34.766 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:04:34.769 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:04.808 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:04.813 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:04.816 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:04.819 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:34.860 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:34.864 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:34.868 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:05:34.871 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:04.910 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:04.914 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:04.918 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:04.921 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:34.962 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:34.967 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:34.970 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:06:34.973 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:05.027 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:05.032 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:05.034 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:05.038 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:35.086 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:35.091 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:35.095 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:07:35.099 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:05.138 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:05.142 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:05.145 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:05.149 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:35.197 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:35.202 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:35.207 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:08:35.210 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:05.265 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:05.269 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:05.272 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:05.275 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:35.340 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:35.344 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:35.348 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:09:35.352 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:05.418 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:05.423 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:05.426 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:05.430 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:35.475 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:35.479 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:35.482 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:10:35.485 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:05.526 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:05.530 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:05.534 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:05.538 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:35.581 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:35.585 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:35.589 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:11:35.593 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:05.636 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:05.640 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:05.648 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:05.652 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:35.695 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:35.699 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:35.702 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:12:35.706 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:05.745 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:05.749 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:05.752 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:05.756 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:35.797 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:35.802 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:35.806 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:13:35.809 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:05.848 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:05.852 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:05.855 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:05.858 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:35.957 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:35.962 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:35.967 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:14:35.972 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:06.017 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:06.021 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:06.025 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:06.028 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:36.073 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:36.077 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:36.081 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:15:36.084 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:06.124 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:06.218 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:06.221 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:06.225 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:36.267 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:36.271 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:36.274 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:16:36.277 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:06.376 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:06.380 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:06.382 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:06.386 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:36.426 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:36.430 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:36.433 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:17:36.436 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:06.497 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:06.501 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:06.504 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:06.511 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:36.565 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:36.569 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:36.573 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:18:36.576 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:04.139 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-11-21T22:19:04.140 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-21T22:19:04.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:19:04.152 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-11-21T22:19:04.154 DEBUG:teuthology.orchestra.run.gibba016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:19:04.168 DEBUG:teuthology.orchestra.run.gibba017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:19:04.181 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:19:04.195 DEBUG:teuthology.orchestra.run.gibba036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-11-21T22:19:04.209 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-11-21T22:19:04.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:19:04.210 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:19:04.210 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:19:06.613 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:06.617 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:06.621 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:06.625 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:36.686 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:36.690 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:36.693 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:19:36.696 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:06.741 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:06.745 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:06.748 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:06.752 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:36.803 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:36.807 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:36.810 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:20:36.813 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:06.855 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:06.859 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:06.862 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:06.866 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:36.904 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:36.908 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:36.911 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:21:36.914 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:06.997 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:07.001 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:07.005 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:07.009 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:37.048 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:37.052 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:37.056 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:22:37.059 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:07.124 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:07.127 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:07.130 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:07.133 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:37.176 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:37.180 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:37.183 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:23:37.187 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:07.246 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:07.250 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:07.254 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:07.259 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:37.326 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:37.330 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:37.333 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:24:37.337 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:07.384 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:07.388 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:07.391 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:07.395 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:37.436 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:37.440 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:37.444 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:25:37.447 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:07.486 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:07.490 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:07.494 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:07.497 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:37.539 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:37.543 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:37.547 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:26:37.551 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:07.597 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:07.601 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:07.605 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:07.608 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:37.649 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:37.653 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:37.656 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:27:37.660 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:07.717 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:07.721 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:07.724 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:07.727 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:37.777 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:37.782 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:37.786 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:28:37.789 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:07.834 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:07.838 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:07.841 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:07.845 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:37.918 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:37.922 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:37.925 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:29:37.928 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:08.008 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:08.012 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:08.015 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:08.018 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:38.062 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:38.066 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:38.070 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:30:38.073 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:08.130 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:08.135 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:08.140 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:08.145 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:38.189 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:38.193 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:38.197 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:31:38.200 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:08.251 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:08.281 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:08.285 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:08.295 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:38.334 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:38.338 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:38.341 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:32:38.344 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:08.390 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:08.394 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:08.397 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:08.401 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:38.458 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:38.461 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:38.465 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:33:38.468 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:04.214 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-11-21T22:34:04.216 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-21T22:34:04.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:34:04.222 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2021-11-21T22:34:04.223 INFO:tasks.cephfs_test_runner:ERROR 2021-11-21T22:34:04.224 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.224 INFO:tasks.cephfs_test_runner:====================================================================== 2021-11-21T22:34:04.225 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:34:04.225 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-11-21T22:34:04.226 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.226 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-11-21T22:34:04.227 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-11-21T22:34:04.227 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-11-21T22:34:04.227 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-11-21T22:34:04.227 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-11-21T22:34:04.228 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.228 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-11-21T22:34:04.228 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.229 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.229 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2021-11-21T22:34:04.229 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2021-11-21T22:34:04.230 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2021-11-21T22:34:04.230 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2021-11-21T22:34:04.230 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-11-21T22:34:04.230 INFO:tasks.cephfs_test_runner: proc.wait() 2021-11-21T22:34:04.231 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 148, in wait 2021-11-21T22:34:04.231 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-11-21T22:34:04.231 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-11-21T22:34:04.232 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-11-21T22:34:04.232 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-11-21T22:34:04.232 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-11-21T22:34:04.233 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-11-21T22:34:04.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-11-21T22:34:04.233 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-11-21T22:34:04.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-11-21T22:34:04.234 INFO:tasks.cephfs_test_runner: for line in f: 2021-11-21T22:34:04.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-11-21T22:34:04.234 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-11-21T22:34:04.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-11-21T22:34:04.235 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-11-21T22:34:04.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-11-21T22:34:04.236 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-11-21T22:34:04.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-11-21T22:34:04.236 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-11-21T22:34:04.236 INFO:tasks.cephfs_test_runner:socket.timeout 2021-11-21T22:34:04.237 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.237 INFO:tasks.cephfs_test_runner:====================================================================== 2021-11-21T22:34:04.237 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:34:04.238 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-11-21T22:34:04.238 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.239 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-11-21T22:34:04.239 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-11-21T22:34:04.240 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-11-21T22:34:04.240 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-11-21T22:34:04.241 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-11-21T22:34:04.241 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.242 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-11-21T22:34:04.242 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.243 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.243 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2021-11-21T22:34:04.244 INFO:tasks.cephfs_test_runner: m.teardown() 2021-11-21T22:34:04.244 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2021-11-21T22:34:04.245 INFO:tasks.cephfs_test_runner: self.umount() 2021-11-21T22:34:04.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2021-11-21T22:34:04.246 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2021-11-21T22:34:04.246 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-11-21T22:34:04.247 INFO:tasks.cephfs_test_runner: proc.wait() 2021-11-21T22:34:04.247 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 148, in wait 2021-11-21T22:34:04.247 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-11-21T22:34:04.248 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-11-21T22:34:04.249 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-11-21T22:34:04.249 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-11-21T22:34:04.249 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-11-21T22:34:04.249 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-11-21T22:34:04.250 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-11-21T22:34:04.250 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-11-21T22:34:04.250 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-11-21T22:34:04.251 INFO:tasks.cephfs_test_runner: for line in f: 2021-11-21T22:34:04.251 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-11-21T22:34:04.251 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-11-21T22:34:04.252 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-11-21T22:34:04.252 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-11-21T22:34:04.252 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-11-21T22:34:04.252 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-11-21T22:34:04.253 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-11-21T22:34:04.253 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-11-21T22:34:04.253 INFO:tasks.cephfs_test_runner:socket.timeout 2021-11-21T22:34:04.254 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.254 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-11-21T22:34:04.254 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2363.901s 2021-11-21T22:34:04.255 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.255 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2021-11-21T22:34:04.255 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.256 INFO:tasks.cephfs_test_runner:====================================================================== 2021-11-21T22:34:04.256 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:34:04.257 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-11-21T22:34:04.257 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-11-21T22:34:04.259 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-11-21T22:34:04.259 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-11-21T22:34:04.260 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-11-21T22:34:04.260 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-11-21T22:34:04.261 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.261 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-11-21T22:34:04.262 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.262 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.263 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2021-11-21T22:34:04.263 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2021-11-21T22:34:04.264 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2021-11-21T22:34:04.265 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2021-11-21T22:34:04.265 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-11-21T22:34:04.266 INFO:tasks.cephfs_test_runner: proc.wait() 2021-11-21T22:34:04.266 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 148, in wait 2021-11-21T22:34:04.266 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-11-21T22:34:04.267 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-11-21T22:34:04.268 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-11-21T22:34:04.268 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-11-21T22:34:04.269 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-11-21T22:34:04.269 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-11-21T22:34:04.270 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-11-21T22:34:04.270 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-11-21T22:34:04.271 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-11-21T22:34:04.271 INFO:tasks.cephfs_test_runner: for line in f: 2021-11-21T22:34:04.272 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-11-21T22:34:04.272 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-11-21T22:34:04.273 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-11-21T22:34:04.274 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-11-21T22:34:04.274 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-11-21T22:34:04.275 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-11-21T22:34:04.275 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-11-21T22:34:04.276 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-11-21T22:34:04.276 INFO:tasks.cephfs_test_runner:socket.timeout 2021-11-21T22:34:04.277 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.277 INFO:tasks.cephfs_test_runner:====================================================================== 2021-11-21T22:34:04.278 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:34:04.279 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-11-21T22:34:04.279 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.280 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-11-21T22:34:04.281 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-11-21T22:34:04.281 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-11-21T22:34:04.282 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-11-21T22:34:04.282 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-11-21T22:34:04.283 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.283 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-11-21T22:34:04.284 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.284 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-11-21T22:34:04.285 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2021-11-21T22:34:04.286 INFO:tasks.cephfs_test_runner: m.teardown() 2021-11-21T22:34:04.286 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2021-11-21T22:34:04.287 INFO:tasks.cephfs_test_runner: self.umount() 2021-11-21T22:34:04.287 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2021-11-21T22:34:04.288 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2021-11-21T22:34:04.288 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-11-21T22:34:04.289 INFO:tasks.cephfs_test_runner: proc.wait() 2021-11-21T22:34:04.289 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 148, in wait 2021-11-21T22:34:04.290 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-11-21T22:34:04.290 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-11-21T22:34:04.291 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-11-21T22:34:04.291 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-11-21T22:34:04.292 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-11-21T22:34:04.292 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-11-21T22:34:04.293 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-11-21T22:34:04.293 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-11-21T22:34:04.294 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-11-21T22:34:04.294 INFO:tasks.cephfs_test_runner: for line in f: 2021-11-21T22:34:04.294 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-11-21T22:34:04.295 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-11-21T22:34:04.295 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-11-21T22:34:04.296 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-11-21T22:34:04.296 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-11-21T22:34:04.296 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-11-21T22:34:04.296 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-11-21T22:34:04.297 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-11-21T22:34:04.297 INFO:tasks.cephfs_test_runner:socket.timeout 2021-11-21T22:34:04.297 INFO:tasks.cephfs_test_runner: 2021-11-21T22:34:04.298 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:34:04.408 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=681aaa383b684a0ca9ebc987224d079d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-11-21T22:34:04.410 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2021-11-21T22:34:04.418 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2021-11-21T22:34:04.442 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2021-11-21T22:34:04.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-11-21T22:34:04.444 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-11-21T22:34:04.444 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-11-21T22:34:08.516 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:08.520 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:08.524 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:08.527 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:38.593 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:38.597 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:38.601 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:34:38.605 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:08.650 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:08.655 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:08.658 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:08.662 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:38.709 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:38.713 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:38.718 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:35:38.721 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:08.769 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:08.773 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:08.779 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:08.782 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:38.849 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:38.852 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:38.856 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:36:38.860 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:08.921 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:08.926 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:08.931 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:08.935 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:38.983 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:38.987 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:38.991 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:37:38.994 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:09.064 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:09.067 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:09.069 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:09.076 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:39.121 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:39.124 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:39.127 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:38:39.131 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:09.174 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:09.178 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:09.181 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:09.184 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:39.310 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:39.384 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:39.387 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:39:39.391 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:09.434 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:09.438 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:09.442 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:09.446 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:39.518 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:39.522 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:39.526 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:40:39.529 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:09.585 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:09.590 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:09.593 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:09.596 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:39.670 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:39.674 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:39.677 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:41:39.681 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:09.734 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:09.738 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:09.742 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:09.745 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:39.795 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:39.799 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:39.803 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:42:39.806 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:09.853 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:09.858 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:09.861 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:09.865 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:39.914 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:39.918 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:39.921 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:43:39.925 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:09.983 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:09.987 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:09.990 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:09.994 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:40.062 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:40.067 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:40.070 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:44:40.072 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:10.135 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:10.139 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:10.143 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:10.146 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:40.204 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:40.210 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:40.215 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:45:40.226 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:10.271 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:10.275 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:10.279 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:10.282 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:40.341 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:40.345 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:40.349 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:46:40.352 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:10.412 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:10.417 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:10.420 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:10.424 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:40.505 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:40.508 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:40.512 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:47:40.517 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:10.558 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:10.563 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:10.566 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:10.569 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:40.633 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:40.638 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:40.642 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:48:40.646 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:04.457 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out 2021-11-21T22:49:04.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:49:04.460 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-21T22:49:04.476 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-21T22:49:04.486 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2021-11-21T22:49:04.497 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2021-11-21T22:49:04.498 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:49:04.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 Processor -- start 2021-11-21T22:49:04.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 -- start start 2021-11-21T22:49:04.568 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f25140a6aa0 0x7f251414b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25140a86b0 0x7f25141470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.569 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f25141475d0 0x7f25141443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140a8bc0 con 0x7f25140a6aa0 2021-11-21T22:49:04.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140a6e80 con 0x7f25141475d0 2021-11-21T22:49:04.570 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f251b522700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251414a7b0 con 0x7f25140a86b0 2021-11-21T22:49:04.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f25140a6aa0 0x7f251414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f25140a6aa0 0x7f251414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55010/0 (socket says 172.21.2.116:55010) 2021-11-21T22:49:04.571 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 -- 172.21.2.116:0/2010906486 learned_addr learned my addr 172.21.2.116:0/2010906486 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:04.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 -- 172.21.2.116:0/2010906486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25140a86b0 msgr2=0x7f25141470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:04.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 --2- 172.21.2.116:0/2010906486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25140a86b0 0x7f25141470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.572 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 -- 172.21.2.116:0/2010906486 >> v2:172.21.2.117:3300/0 conn(0x7f25141475d0 msgr2=0x7f25141443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:04.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 --2- 172.21.2.116:0/2010906486 >> v2:172.21.2.117:3300/0 conn(0x7f25141475d0 0x7f25141443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.573 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 -- 172.21.2.116:0/2010906486 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251414aa10 con 0x7f25140a6aa0 2021-11-21T22:49:04.576 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f25192be700 1 --2- 172.21.2.116:0/2010906486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a6aa0 0x7f251414b150 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7f2510002f70 tx=0x7f2510006eb0).ready entity=mon.0 client_cookie=85a16e00bbf66242 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f2507fff700 1 -- 172.21.2.116:0/2010906486 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f251000d570 con 0x7f25140a6aa0 2021-11-21T22:49:04.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f2507fff700 1 -- 172.21.2.116:0/2010906486 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510000e90 con 0x7f25140a6aa0 2021-11-21T22:49:04.577 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.559+0000 7f2507fff700 1 -- 172.21.2.116:0/2010906486 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f251000d900 con 0x7f25140a6aa0 2021-11-21T22:49:04.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/2010906486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a6aa0 msgr2=0x7f251414b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.578 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 --2- 172.21.2.116:0/2010906486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a6aa0 0x7f251414b150 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7f2510002f70 tx=0x7f2510006eb0).stop 2021-11-21T22:49:04.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/2010906486 shutdown_connections 2021-11-21T22:49:04.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 --2- 172.21.2.116:0/2010906486 >> v2:172.21.2.117:3300/0 conn(0x7f25141475d0 0x7f25141443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 --2- 172.21.2.116:0/2010906486 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25140a86b0 0x7f25141470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 --2- 172.21.2.116:0/2010906486 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a6aa0 0x7f251414b150 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/2010906486 >> 172.21.2.116:0/2010906486 conn(0x7f251405bd20 msgr2=0x7f251405c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:04.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/2010906486 shutdown_connections 2021-11-21T22:49:04.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/2010906486 wait complete. 2021-11-21T22:49:04.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 Processor -- start 2021-11-21T22:49:04.581 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- start start 2021-11-21T22:49:04.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25140a6aa0 0x7f251414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a86b0 0x7f25141597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25141475d0 0x7f2514159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b3b30 con 0x7f25140a86b0 2021-11-21T22:49:04.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514065640 con 0x7f25140a6aa0 2021-11-21T22:49:04.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b3890 con 0x7f25141475d0 2021-11-21T22:49:04.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a86b0 0x7f25141597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a86b0 0x7f25141597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55018/0 (socket says 172.21.2.116:55018) 2021-11-21T22:49:04.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 -- 172.21.2.116:0/4276275841 learned_addr learned my addr 172.21.2.116:0/4276275841 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:04.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f25192be700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25140a6aa0 0x7f251414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 -- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25141475d0 msgr2=0x7f2514159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25141475d0 0x7f2514159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 -- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25140a6aa0 msgr2=0x7f251414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25140a6aa0 0x7f251414ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 -- 172.21.2.116:0/4276275841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251000b1e0 con 0x7f25140a86b0 2021-11-21T22:49:04.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f2518abd700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a86b0 0x7f25141597f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7f2508002f70 tx=0x7f2508006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f25067fc700 1 -- 172.21.2.116:0/4276275841 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2508013460 con 0x7f25140a86b0 2021-11-21T22:49:04.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f25067fc700 1 -- 172.21.2.116:0/4276275841 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25080135c0 con 0x7f25140a86b0 2021-11-21T22:49:04.589 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25140658a0 con 0x7f25140a86b0 2021-11-21T22:49:04.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f25067fc700 1 -- 172.21.2.116:0/4276275841 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2508013920 con 0x7f25140a86b0 2021-11-21T22:49:04.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2514153dd0 con 0x7f25140a86b0 2021-11-21T22:49:04.590 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25140b3540 con 0x7f25140a86b0 2021-11-21T22:49:04.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.563+0000 7f25067fc700 1 -- 172.21.2.116:0/4276275841 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f250800f070 con 0x7f25140a86b0 2021-11-21T22:49:04.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.567+0000 7f25067fc700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24f40536a0 0x7f24f4055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.591 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.567+0000 7f25067fc700 1 -- 172.21.2.116:0/4276275841 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f250806b0c0 con 0x7f25140a86b0 2021-11-21T22:49:04.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.567+0000 7f25192be700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24f40536a0 0x7f24f4055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.567+0000 7f25192be700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24f40536a0 0x7f24f4055b60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f2510017000 tx=0x7f2510015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.567+0000 7f25067fc700 1 -- 172.21.2.116:0/4276275841 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f25080423f0 con 0x7f25140a86b0 2021-11-21T22:49:04.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2514065a30 con 0x7f25140a86b0 2021-11-21T22:49:04.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f25067fc700 1 -- 172.21.2.116:0/4276275841 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+13318 (secure 0 0 0) 0x7f2508045a10 con 0x7f25140a86b0 2021-11-21T22:49:04.659 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:49:04.660 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":165,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:34.000323+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.116:6835/403152574":"2021-11-22T22:03:30.888079+0000","172.21.2.116:6834/403152574":"2021-11-22T22:03:30.888079+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:49:04.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24f40536a0 msgr2=0x7f24f4055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24f40536a0 0x7f24f4055b60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f2510017000 tx=0x7f2510015000).stop 2021-11-21T22:49:04.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a86b0 msgr2=0x7f25141597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a86b0 0x7f25141597f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7f2508002f70 tx=0x7f2508006e90).stop 2021-11-21T22:49:04.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 shutdown_connections 2021-11-21T22:49:04.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f25141475d0 0x7f2514159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f24f40536a0 0x7f24f4055b60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f25140a86b0 0x7f25141597f0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 --2- 172.21.2.116:0/4276275841 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f25140a6aa0 0x7f251414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.665 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 >> 172.21.2.116:0/4276275841 conn(0x7f251405bd20 msgr2=0x7f25141490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:04.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 shutdown_connections 2021-11-21T22:49:04.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.651+0000 7f251b522700 1 -- 172.21.2.116:0/4276275841 wait complete. 2021-11-21T22:49:04.669 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-11-21T22:49:04.764 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.755+0000 7f70b54f9700 1 Processor -- start 2021-11-21T22:49:04.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.755+0000 7f70b54f9700 1 -- start start 2021-11-21T22:49:04.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.755+0000 7f70b54f9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f70b00a6aa0 0x7f70b014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.765 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.755+0000 7f70b54f9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b00a6e80 con 0x7f70b0062410 2021-11-21T22:49:04.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.755+0000 7f70b54f9700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b014a7b0 con 0x7f70b00a86b0 2021-11-21T22:49:04.766 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.755+0000 7f70b54f9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b00a8bc0 con 0x7f70b00a6aa0 2021-11-21T22:49:04.769 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70af7fe700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f70b0062410 0x7f70b01443f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38748/0 (socket says 172.21.2.116:38748) 2021-11-21T22:49:04.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70af7fe700 1 -- 172.21.2.116:0/2535822225 learned_addr learned my addr 172.21.2.116:0/2535822225 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:04.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 --2- 172.21.2.116:0/2535822225 >> v2:172.21.2.116:3300/0 conn(0x7f70b00a6aa0 0x7f70b014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.770 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 -- 172.21.2.116:0/2535822225 >> v1:172.21.2.117:6789/0 conn(0x7f70b00a86b0 legacy=0x7f70b01470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:04.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 -- 172.21.2.116:0/2535822225 >> v1:172.21.2.116:6789/0 conn(0x7f70b0062410 legacy=0x7f70b01443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 -- 172.21.2.116:0/2535822225 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b014aa10 con 0x7f70b00a6aa0 2021-11-21T22:49:04.771 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 --2- 172.21.2.116:0/2535822225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b00a6aa0 0x7f70b014b150 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f70b0061b50 tx=0x7f7098006df0).ready entity=mon.0 client_cookie=76ccee90b552eb05 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70adffb700 1 -- 172.21.2.116:0/2535822225 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f709800d5c0 con 0x7f70b00a6aa0 2021-11-21T22:49:04.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70adffb700 1 -- 172.21.2.116:0/2535822225 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7098000e90 con 0x7f70b00a6aa0 2021-11-21T22:49:04.772 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70adffb700 1 -- 172.21.2.116:0/2535822225 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f709800d910 con 0x7f70b00a6aa0 2021-11-21T22:49:04.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- 172.21.2.116:0/2535822225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b00a6aa0 msgr2=0x7f70b014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.773 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2535822225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b00a6aa0 0x7f70b014b150 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f70b0061b50 tx=0x7f7098006df0).stop 2021-11-21T22:49:04.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- 172.21.2.116:0/2535822225 shutdown_connections 2021-11-21T22:49:04.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2535822225 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b00a6aa0 0x7f70b014b150 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.774 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- 172.21.2.116:0/2535822225 >> 172.21.2.116:0/2535822225 conn(0x7f70b005bd20 msgr2=0x7f70b005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:04.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- 172.21.2.116:0/2535822225 shutdown_connections 2021-11-21T22:49:04.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- 172.21.2.116:0/2535822225 wait complete. 2021-11-21T22:49:04.775 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 Processor -- start 2021-11-21T22:49:04.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- start start 2021-11-21T22:49:04.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b0062410 0x7f70b0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.776 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70b00a6aa0 0x7f70b0064aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70b00a86b0 0x7f70b015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00b42f0 con 0x7f70b0062410 2021-11-21T22:49:04.777 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0156910 con 0x7f70b00a86b0 2021-11-21T22:49:04.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00b4050 con 0x7f70b00a6aa0 2021-11-21T22:49:04.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b0062410 0x7f70b0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.778 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b0062410 0x7f70b0156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55028/0 (socket says 172.21.2.116:55028) 2021-11-21T22:49:04.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 -- 172.21.2.116:0/2278939259 learned_addr learned my addr 172.21.2.116:0/2278939259 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:04.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 -- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70b00a6aa0 msgr2=0x7f70b0064aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.779 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70b00a6aa0 0x7f70b0064aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 -- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70b00a86b0 msgr2=0x7f70b015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70b00a86b0 0x7f70b015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.780 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 -- 172.21.2.116:0/2278939259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709800b180 con 0x7f70b0062410 2021-11-21T22:49:04.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70aeffd700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b0062410 0x7f70b0156420 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7f70b0061b50 tx=0x7f7098012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70a7fff700 1 -- 172.21.2.116:0/2278939259 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7098016070 con 0x7f70b0062410 2021-11-21T22:49:04.781 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70a7fff700 1 -- 172.21.2.116:0/2278939259 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7098002f50 con 0x7f70b0062410 2021-11-21T22:49:04.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70a7fff700 1 -- 172.21.2.116:0/2278939259 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f709801e450 con 0x7f70b0062410 2021-11-21T22:49:04.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b0159b00 con 0x7f70b0062410 2021-11-21T22:49:04.782 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.759+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b0064f90 con 0x7f70b0062410 2021-11-21T22:49:04.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.763+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b015a8d0 con 0x7f70b0062410 2021-11-21T22:49:04.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.763+0000 7f70a7fff700 1 -- 172.21.2.116:0/2278939259 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7098025030 con 0x7f70b0062410 2021-11-21T22:49:04.783 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.763+0000 7f70a7fff700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7094053580 0x7f7094055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.784 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.763+0000 7f70a7fff700 1 -- 172.21.2.116:0/2278939259 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f709806c870 con 0x7f70b0062410 2021-11-21T22:49:04.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.763+0000 7f70ae7fc700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7094053580 0x7f7094055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.763+0000 7f70ae7fc700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7094053580 0x7f7094055a40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f70a8003310 tx=0x7f70a8006d30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.785 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.763+0000 7f70a7fff700 1 -- 172.21.2.116:0/2278939259 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7098043cb0 con 0x7f70b0062410 2021-11-21T22:49:04.854 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.847+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70b00573d0 con 0x7f7094053580 2021-11-21T22:49:04.863 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.855+0000 7f70a7fff700 1 -- 172.21.2.116:0/2278939259 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66173 (secure 0 0 0) 0x7f70b00573d0 con 0x7f7094053580 2021-11-21T22:49:04.865 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:49:04.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7094053580 msgr2=0x7f7094055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.866 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7094053580 0x7f7094055a40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f70a8003310 tx=0x7f70a8006d30).stop 2021-11-21T22:49:04.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b0062410 msgr2=0x7f70b0156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b0062410 0x7f70b0156420 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7f70b0061b50 tx=0x7f7098012000).stop 2021-11-21T22:49:04.867 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 shutdown_connections 2021-11-21T22:49:04.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f70b00a86b0 0x7f70b015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f70b00a6aa0 0x7f70b0064aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:04.868 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7094053580 0x7f7094055a40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 --2- 172.21.2.116:0/2278939259 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f70b0062410 0x7f70b0156420 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 >> 172.21.2.116:0/2278939259 conn(0x7f70b005bd20 msgr2=0x7f70b00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:04.869 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 shutdown_connections 2021-11-21T22:49:04.870 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.859+0000 7f70b54f9700 1 -- 172.21.2.116:0/2278939259 wait complete. 2021-11-21T22:49:04.870 INFO:teuthology.orchestra.run.gibba016.stderr:dumped all 2021-11-21T22:49:04.871 INFO:teuthology.orchestra.run.gibba016.stdout:{"pg_ready":true,"pg_map":{"version":1802,"stamp":"2021-11-21T22:49:04.429085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":704299008,"kb_used":6104448,"kb_used_data":698194560,"kb_used_omap":27875,"kb_used_meta":0,"kb_avail":698194560,"statfs":{"total":721202184192,"available":714951229440,"internally_reserved":0,"allocated":714951229440,"data_stored":714951229440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28544708,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004580"},"pg_stats":[{"pgid":"1.0","version":"14'2","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.433989+0000","last_change":"2021-11-21T21:54:16.981838+0000","last_active":"2021-11-21T22:03:33.433989+0000","last_peered":"2021-11-21T22:03:33.433989+0000","last_clean":"2021-11-21T22:03:33.433989+0000","last_became_active":"2021-11-21T21:54:16.981409+0000","last_became_peered":"2021-11-21T21:54:16.981409+0000","last_unstale":"2021-11-21T22:03:33.433989+0000","last_undegraded":"2021-11-21T22:03:33.433989+0000","last_fullsized":"2021-11-21T22:03:33.433989+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_clean_scrub_stamp":"2021-11-21T21:54:13.901475+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.002389+0000","last_change":"2021-11-21T21:54:22.713500+0000","last_active":"2021-11-21T22:03:33.002389+0000","last_peered":"2021-11-21T22:03:33.002389+0000","last_clean":"2021-11-21T22:03:33.002389+0000","last_became_active":"2021-11-21T21:54:22.713118+0000","last_became_peered":"2021-11-21T21:54:22.713118+0000","last_unstale":"2021-11-21T22:03:33.002389+0000","last_undegraded":"2021-11-21T22:03:33.002389+0000","last_fullsized":"2021-11-21T22:03:33.002389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.434053+0000","last_change":"2021-11-21T21:54:22.713430+0000","last_active":"2021-11-21T22:03:33.434053+0000","last_peered":"2021-11-21T22:03:33.434053+0000","last_clean":"2021-11-21T22:03:33.434053+0000","last_became_active":"2021-11-21T21:54:22.712862+0000","last_became_peered":"2021-11-21T21:54:22.712862+0000","last_unstale":"2021-11-21T22:03:33.434053+0000","last_undegraded":"2021-11-21T22:03:33.434053+0000","last_fullsized":"2021-11-21T22:03:33.434053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.594014+0000","last_change":"2021-11-21T21:54:22.711838+0000","last_active":"2021-11-21T22:03:33.594014+0000","last_peered":"2021-11-21T22:03:33.594014+0000","last_clean":"2021-11-21T22:03:33.594014+0000","last_became_active":"2021-11-21T21:54:22.711687+0000","last_became_peered":"2021-11-21T21:54:22.711687+0000","last_unstale":"2021-11-21T22:03:33.594014+0000","last_undegraded":"2021-11-21T22:03:33.594014+0000","last_fullsized":"2021-11-21T22:03:33.594014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.002388+0000","last_change":"2021-11-21T21:54:22.713478+0000","last_active":"2021-11-21T22:03:33.002388+0000","last_peered":"2021-11-21T22:03:33.002388+0000","last_clean":"2021-11-21T22:03:33.002388+0000","last_became_active":"2021-11-21T21:54:22.713254+0000","last_became_peered":"2021-11-21T21:54:22.713254+0000","last_unstale":"2021-11-21T22:03:33.002388+0000","last_undegraded":"2021-11-21T22:03:33.002388+0000","last_fullsized":"2021-11-21T22:03:33.002388+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:34.289098+0000","last_change":"2021-11-21T21:54:22.786234+0000","last_active":"2021-11-21T22:03:34.289098+0000","last_peered":"2021-11-21T22:03:34.289098+0000","last_clean":"2021-11-21T22:03:34.289098+0000","last_became_active":"2021-11-21T21:54:22.786041+0000","last_became_peered":"2021-11-21T21:54:22.786041+0000","last_unstale":"2021-11-21T22:03:34.289098+0000","last_undegraded":"2021-11-21T22:03:34.289098+0000","last_fullsized":"2021-11-21T22:03:34.289098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.434097+0000","last_change":"2021-11-21T21:54:22.788120+0000","last_active":"2021-11-21T22:03:33.434097+0000","last_peered":"2021-11-21T22:03:33.434097+0000","last_clean":"2021-11-21T22:03:33.434097+0000","last_became_active":"2021-11-21T21:54:22.787305+0000","last_became_peered":"2021-11-21T21:54:22.787305+0000","last_unstale":"2021-11-21T22:03:33.434097+0000","last_undegraded":"2021-11-21T22:03:33.434097+0000","last_fullsized":"2021-11-21T22:03:33.434097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:34.289171+0000","last_change":"2021-11-21T21:54:22.713690+0000","last_active":"2021-11-21T22:03:34.289171+0000","last_peered":"2021-11-21T22:03:34.289171+0000","last_clean":"2021-11-21T22:03:34.289171+0000","last_became_active":"2021-11-21T21:54:22.713554+0000","last_became_peered":"2021-11-21T21:54:22.713554+0000","last_unstale":"2021-11-21T22:03:34.289171+0000","last_undegraded":"2021-11-21T22:03:34.289171+0000","last_fullsized":"2021-11-21T22:03:34.289171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:34.290472+0000","last_change":"2021-11-21T21:54:22.714589+0000","last_active":"2021-11-21T22:03:34.290472+0000","last_peered":"2021-11-21T22:03:34.290472+0000","last_clean":"2021-11-21T22:03:34.290472+0000","last_became_active":"2021-11-21T21:54:22.714383+0000","last_became_peered":"2021-11-21T21:54:22.714383+0000","last_unstale":"2021-11-21T22:03:34.290472+0000","last_undegraded":"2021-11-21T22:03:34.290472+0000","last_fullsized":"2021-11-21T22:03:34.290472+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3638,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3726197,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.333,"15min":1.335},"min":{"1min":0.753,"5min":0.65700000000000003,"15min":0.57699999999999996},"max":{"1min":1.532,"5min":1.841,"15min":1.841},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3109999999999999,"15min":1.3080000000000001},"min":{"1min":0.69299999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7030000000000001,"5min":1.921,"15min":1.921},"last":0.746}]},{"osd":1,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.254,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.60699999999999998,"15min":0.56000000000000005},"max":{"1min":1.546,"5min":1.7010000000000001,"15min":1.724},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2490000000000001,"15min":1.228},"min":{"1min":0.79600000000000004,"5min":0.59299999999999997,"15min":0.56499999999999995},"max":{"1min":1.5780000000000001,"5min":1.696,"15min":1.7070000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2849999999999999,"15min":1.278},"min":{"1min":0.70599999999999996,"5min":0.64300000000000002,"15min":0.623},"max":{"1min":1.6020000000000001,"5min":1.97,"15min":1.97},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2869999999999999,"15min":1.302},"min":{"1min":0.76800000000000002,"5min":0.63300000000000001,"15min":0.59799999999999998},"max":{"1min":1.6140000000000001,"5min":1.881,"15min":1.921},"last":0.752}]},{"osd":3,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.68799999999999994,"5min":0.624,"15min":0.55300000000000005},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.6830000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.202,"5min":1.2689999999999999,"15min":1.24},"min":{"1min":0.78100000000000003,"5min":0.64700000000000002,"15min":0.624},"max":{"1min":1.5509999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.73099999999999998}]},{"osd":4,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84199999999999997,"15min":0.88},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.23899999999999999},"max":{"1min":1.284,"5min":1.5940000000000001,"15min":1.728},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.36799999999999999,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":1.3340000000000001,"5min":1.3540000000000001,"15min":1.3819999999999999},"last":0.40200000000000002}]},{"osd":5,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89500000000000002,"15min":0.92800000000000005},"min":{"1min":0.33800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.6559999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98099999999999998,"15min":0.98599999999999999},"min":{"1min":0.307,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":1.472,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.44900000000000001}]},{"osd":6,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.028,"15min":1.028},"min":{"1min":0.33100000000000002,"5min":0.313,"15min":0.253},"max":{"1min":1.395,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.373},{"interface":"front","average":{"1min":1.02,"5min":1.01,"15min":1.02},"min":{"1min":0.38900000000000001,"5min":0.26700000000000002,"15min":0.23799999999999999},"max":{"1min":1.3560000000000001,"5min":1.744,"15min":1.744},"last":0.438}]}]},{"osd":6,"up_from":13,"seq":55834575509,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":4048,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4145786,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3069999999999999,"15min":1.2789999999999999},"min":{"1min":0.871,"5min":0.372,"15min":0.372},"max":{"1min":1.5009999999999999,"5min":1.7609999999999999,"15min":1.8029999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.75700000000000001,"5min":0.442,"15min":0.38100000000000001},"max":{"1min":1.619,"5min":1.708,"15min":1.708},"last":1.5329999999999999}]},{"osd":1,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.294,"15min":1.28},"min":{"1min":0.84699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.639,"5min":1.708,"15min":1.7130000000000001},"last":1.163},{"interface":"front","average":{"1min":1.238,"5min":1.294,"15min":1.266},"min":{"1min":0.77300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5389999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.113}]},{"osd":2,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3120000000000001,"15min":1.298},"min":{"1min":0.82099999999999995,"5min":0.435,"15min":0.34599999999999997},"max":{"1min":1.7090000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.724},{"interface":"front","average":{"1min":1.194,"5min":1.25,"15min":1.2410000000000001},"min":{"1min":0.79800000000000004,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4950000000000001,"5min":1.843,"15min":1.903},"last":1.266}]},{"osd":3,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2310000000000001,"15min":1.2170000000000001},"min":{"1min":0.77800000000000002,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.738},"last":1.101},{"interface":"front","average":{"1min":1.171,"5min":1.232,"15min":1.21},"min":{"1min":0.76600000000000001,"5min":0.437,"15min":0.34699999999999998},"max":{"1min":1.476,"5min":1.657,"15min":1.8999999999999999},"last":1.6299999999999999}]},{"osd":4,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.83999999999999997},"min":{"1min":0.32000000000000001,"5min":0.25,"15min":0.187},"max":{"1min":1.0089999999999999,"5min":1.417,"15min":1.621},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91400000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.25900000000000001,"15min":0.217},"max":{"1min":1.1910000000000001,"5min":1.26,"15min":1.7110000000000001},"last":1.0249999999999999}]},{"osd":5,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0700000000000001,"15min":1.0469999999999999},"min":{"1min":0.42399999999999999,"5min":0.28199999999999997,"15min":0.19700000000000001},"max":{"1min":1.5249999999999999,"5min":1.55,"15min":1.7150000000000001},"last":1.496},{"interface":"front","average":{"1min":0.871,"5min":0.92300000000000004,"15min":0.91200000000000003},"min":{"1min":0.374,"5min":0.25600000000000001,"15min":0.192},"max":{"1min":1.645,"5min":1.669,"15min":1.669},"last":0.90500000000000003}]},{"osd":7,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94899999999999995,"15min":0.93300000000000005},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.20899999999999999},"max":{"1min":1.5820000000000001,"5min":1.6479999999999999,"15min":1.77},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1419999999999999,"15min":1.1140000000000001},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.25700000000000001},"max":{"1min":1.4530000000000001,"5min":1.5920000000000001,"15min":1.7769999999999999},"last":1.3460000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":2746,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812147,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.2110000000000001,"15min":1.1890000000000001},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.29699999999999999},"max":{"1min":2.012,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2789999999999999,"15min":1.2390000000000001},"min":{"1min":0.46300000000000002,"5min":0.441,"15min":0.36399999999999999},"max":{"1min":1.9710000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3160000000000001,"15min":1.3089999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.36799999999999999},"max":{"1min":1.637,"5min":1.946,"15min":1.946},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.25,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.379},"max":{"1min":1.6839999999999999,"5min":2,"15min":2},"last":0.74099999999999999}]},{"osd":2,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.2110000000000001,"15min":1.2030000000000001},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.33800000000000002},"max":{"1min":1.5209999999999999,"5min":1.7689999999999999,"15min":1.8129999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.147,"15min":1.1679999999999999},"min":{"1min":0.46899999999999997,"5min":0.36299999999999999,"15min":0.34399999999999997},"max":{"1min":1.4259999999999999,"5min":1.456,"15min":1.7170000000000001},"last":0.83699999999999997}]},{"osd":3,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40899999999999997},"max":{"1min":1.4870000000000001,"5min":1.724,"15min":1.875},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.48999999999999999,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":1.665,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.217,"5min":0.217,"15min":0.20699999999999999},"max":{"1min":0.93999999999999995,"5min":1.278,"15min":1.3560000000000001},"last":0.217},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.22500000000000001},"max":{"1min":1.1000000000000001,"5min":1.5129999999999999,"15min":1.73},"last":0.29799999999999999}]},{"osd":6,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91400000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.20300000000000001},"max":{"1min":1.1439999999999999,"5min":1.3380000000000001,"15min":1.552},"last":0.247},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.93100000000000005},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":1.454,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.33300000000000002}]},{"osd":7,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":1.0109999999999999,"15min":0.996},"min":{"1min":0.31,"5min":0.26500000000000001,"15min":0.19400000000000001},"max":{"1min":1.429,"5min":1.5620000000000001,"15min":1.74},"last":0.317},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.318,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.252,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.32100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":762944,"kb_used_data":87274432,"kb_used_omap":3477,"kb_used_meta":0,"kb_avail":87274432,"statfs":{"total":90150273024,"available":89369018368,"internally_reserved":0,"allocated":89369018368,"data_stored":89369018368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3561237,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.292,"15min":1.2410000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.53200000000000003},"max":{"1min":1.552,"5min":1.718,"15min":1.718},"last":1.462},{"interface":"front","average":{"1min":1.286,"5min":1.2569999999999999,"15min":1.24},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.56200000000000006},"max":{"1min":1.6220000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.4430000000000001}]},{"osd":1,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.304,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.61099999999999999},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.75},"last":1.391},{"interface":"front","average":{"1min":1.222,"5min":1.2450000000000001,"15min":1.218},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.59199999999999997},"max":{"1min":1.4430000000000001,"5min":1.601,"15min":1.7410000000000001},"last":1.175}]},{"osd":2,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.24,"15min":1.2350000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.54800000000000004},"max":{"1min":1.579,"5min":1.579,"15min":2.2629999999999999},"last":1.266},{"interface":"front","average":{"1min":1.353,"5min":1.3640000000000001,"15min":1.3220000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.58599999999999997},"max":{"1min":1.673,"5min":1.7569999999999999,"15min":2.226},"last":1.488}]},{"osd":3,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2769999999999999,"15min":1.2330000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.54300000000000004},"max":{"1min":1.7210000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.57399999999999995},"max":{"1min":1.492,"5min":1.992,"15min":1.992},"last":1.3080000000000001}]},{"osd":5,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.91300000000000003,"15min":0.86199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23000000000000001},"max":{"1min":1.123,"5min":1.28,"15min":1.5720000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97999999999999998,"15min":0.91500000000000004},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":1.5569999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.204}]},{"osd":6,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":0.88300000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.21099999999999999},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0129999999999999,"15min":0.96999999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23699999999999999},"max":{"1min":1.4319999999999999,"5min":1.5349999999999999,"15min":1.5780000000000001},"last":1.3520000000000001}]},{"osd":7,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1160000000000001,"15min":1.0609999999999999},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.24099999999999999},"max":{"1min":1.71,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.03},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.071,"15min":0.98699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23400000000000001,"15min":0.22800000000000001},"max":{"1min":1.2909999999999999,"5min":1.609,"15min":1.716},"last":1.149}]}]},{"osd":3,"up_from":13,"seq":55834575507,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3422,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3504138,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.38800000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.93999999999999995,"5min":1.034,"15min":1.161},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.35999999999999999,"5min":0.249,"15min":0.21199999999999999},"max":{"1min":0.95199999999999996,"5min":1.0469999999999999,"15min":1.0569999999999999},"last":0.84699999999999998}]},{"osd":1,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73499999999999999,"15min":0.746},"min":{"1min":0.33900000000000002,"5min":0.23000000000000001,"15min":0.19400000000000001},"max":{"1min":0.93400000000000005,"5min":1.238,"15min":1.238},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.80100000000000005,"15min":0.79600000000000004},"min":{"1min":0.42399999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.2130000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.90300000000000002}]},{"osd":2,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.38100000000000001,"5min":0.22800000000000001,"15min":0.20999999999999999},"max":{"1min":1.1080000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.012},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.91600000000000004,"15min":0.90100000000000002},"min":{"1min":0.44900000000000001,"5min":0.248,"15min":0.23100000000000001},"max":{"1min":1.353,"5min":1.3660000000000001,"15min":1.6359999999999999},"last":1.0009999999999999}]},{"osd":4,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3080000000000001,"15min":1.349},"min":{"1min":0.79400000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.9730000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.90500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5580000000000001,"5min":1.7809999999999999,"15min":1.833},"last":1.45}]},{"osd":5,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.323,"15min":1.327},"min":{"1min":0.97199999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.593,"5min":1.843,"15min":1.917},"last":1.583},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3460000000000001,"15min":1.355},"min":{"1min":0.92000000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.7130000000000001,"5min":1.7789999999999999,"15min":1.9470000000000001},"last":1.405}]},{"osd":6,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.333,"15min":1.349},"min":{"1min":1.03,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6870000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.379},{"interface":"front","average":{"1min":1.347,"5min":1.349,"15min":1.3340000000000001},"min":{"1min":1.127,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.657,"5min":1.7,"15min":1.891},"last":1.482}]},{"osd":7,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3440000000000001,"15min":1.3480000000000001},"min":{"1min":1.075,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.359},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.375,"15min":1.3799999999999999},"min":{"1min":0.93600000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.651,"5min":1.8839999999999999,"15min":1.8999999999999999},"last":1.5449999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":2820,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2887910,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70099999999999996,"15min":0.70999999999999996},"min":{"1min":0.29299999999999998,"5min":0.221,"15min":0.19400000000000001},"max":{"1min":0.96999999999999997,"5min":1.1000000000000001,"15min":1.351},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.32800000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.97099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.03}]},{"osd":1,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.223,"15min":0.188},"max":{"1min":1.075,"5min":1.075,"15min":1.1519999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84799999999999998,"15min":0.86499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1479999999999999,"5min":1.155,"15min":1.5820000000000001},"last":0.77500000000000002}]},{"osd":3,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.79800000000000004},"min":{"1min":0.30399999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.115,"5min":1.319,"15min":1.319},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81000000000000005,"15min":0.84499999999999997},"min":{"1min":0.40000000000000002,"5min":0.218,"15min":0.20499999999999999},"max":{"1min":1.0609999999999999,"5min":1.302,"15min":1.302},"last":0.81799999999999995}]},{"osd":4,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.296,"15min":1.357},"min":{"1min":0.94499999999999995,"5min":0.629,"15min":0.503},"max":{"1min":1.6319999999999999,"5min":1.7230000000000001,"15min":1.8420000000000001},"last":1.319},{"interface":"front","average":{"1min":1.345,"5min":1.331,"15min":1.377},"min":{"1min":0.86399999999999999,"5min":0.66800000000000004,"15min":0.58999999999999997},"max":{"1min":1.716,"5min":1.829,"15min":2.0710000000000002},"last":1.4750000000000001}]},{"osd":5,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.367,"15min":1.409},"min":{"1min":0.84399999999999997,"5min":0.62,"15min":0.61299999999999999},"max":{"1min":1.6659999999999999,"5min":1.901,"15min":1.901},"last":1.46},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.423},"min":{"1min":0.82999999999999996,"5min":0.65600000000000003,"15min":0.55500000000000005},"max":{"1min":1.7430000000000001,"5min":1.784,"15min":1.913},"last":1.5529999999999999}]},{"osd":6,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3819999999999999,"15min":1.4259999999999999},"min":{"1min":0.88700000000000001,"5min":0.68200000000000005,"15min":0.57599999999999996},"max":{"1min":1.8240000000000001,"5min":1.861,"15min":1.861},"last":1.605},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3360000000000001,"15min":1.377},"min":{"1min":0.81799999999999995,"5min":0.64800000000000002,"15min":0.54900000000000004},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":2.512},"last":1.2929999999999999}]},{"osd":7,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3540000000000001,"15min":1.379},"min":{"1min":0.98699999999999999,"5min":0.63700000000000001,"15min":0.59599999999999997},"max":{"1min":1.6719999999999999,"5min":1.8009999999999999,"15min":1.925},"last":1.383},{"interface":"front","average":{"1min":1.264,"5min":1.274,"15min":1.3300000000000001},"min":{"1min":0.89700000000000002,"5min":0.625,"15min":0.51100000000000001},"max":{"1min":1.7669999999999999,"5min":1.7689999999999999,"15min":1.806},"last":1.3580000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575507,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3806,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3897559,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":1.254,"5min":1.278,"15min":1.3069999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79600000000000004,"15min":0.81699999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.224},"max":{"1min":0.98699999999999999,"5min":1.093,"15min":1.4730000000000001},"last":0.86699999999999999}]},{"osd":2,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.29599999999999999,"5min":0.28299999999999997,"15min":0.19500000000000001},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.82899999999999996,"15min":0.81100000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.19},"max":{"1min":0.97699999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.107}]},{"osd":3,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.17599999999999999},"max":{"1min":1.135,"5min":1.423,"15min":1.7010000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.27500000000000002,"5min":0.26900000000000002,"15min":0.20899999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0449999999999999}]},{"osd":4,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.345,"15min":1.3540000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.643,"5min":1.792,"15min":1.9990000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.361},"min":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.30099999999999999},"max":{"1min":1.6319999999999999,"5min":1.929,"15min":1.9299999999999999},"last":1.321}]},{"osd":5,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4219999999999999},"min":{"1min":0.66800000000000004,"5min":0.54500000000000004,"15min":0.28999999999999998},"max":{"1min":1.7609999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.4019999999999999,"15min":1.3899999999999999},"min":{"1min":0.60399999999999998,"5min":0.54800000000000004,"15min":0.35499999999999998},"max":{"1min":1.661,"5min":1.839,"15min":1.875},"last":1.3640000000000001}]},{"osd":6,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.3540000000000001},"min":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.27800000000000002},"max":{"1min":1.6779999999999999,"5min":1.6919999999999999,"15min":1.8879999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.345,"15min":1.357},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":1.641,"5min":1.8560000000000001,"15min":1.915},"last":1.3939999999999999}]},{"osd":7,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.353,"15min":1.375},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.7689999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.292},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3700000000000001,"15min":1.361},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":1.655,"5min":1.819,"15min":1.9350000000000001},"last":1.5249999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3915,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4009734,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.27800000000000002,"5min":0.22500000000000001,"15min":0.186},"max":{"1min":0.95899999999999996,"5min":1.1100000000000001,"15min":1.1299999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77300000000000002,"15min":0.754},"min":{"1min":0.217,"5min":0.217,"15min":0.21199999999999999},"max":{"1min":0.97999999999999998,"5min":1.0880000000000001,"15min":1.327},"last":0.875}]},{"osd":2,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.22600000000000001,"5min":0.217,"15min":0.216},"max":{"1min":0.96299999999999997,"5min":1.1160000000000001,"15min":1.25},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.20100000000000001},"max":{"1min":0.97199999999999998,"5min":1.161,"15min":1.161},"last":0.83499999999999996}]},{"osd":3,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.92600000000000005,"15min":0.89400000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.24099999999999999},"max":{"1min":1.2190000000000001,"5min":1.3460000000000001,"15min":1.6299999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.216},"max":{"1min":1.016,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3140000000000001,"15min":1.3120000000000001},"min":{"1min":0.70999999999999996,"5min":0.40200000000000002,"15min":0.34599999999999997},"max":{"1min":1.663,"5min":1.6990000000000001,"15min":1.77},"last":1.415},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.365,"15min":1.3520000000000001},"min":{"1min":0.67700000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.712,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.321}]},{"osd":5,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.361,"15min":1.367},"min":{"1min":0.64800000000000002,"5min":0.41399999999999998,"15min":0.38100000000000001},"max":{"1min":1.657,"5min":1.9370000000000001,"15min":1.9399999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3959999999999999,"15min":1.3620000000000001},"min":{"1min":0.78500000000000003,"5min":0.35599999999999998,"15min":0.34599999999999997},"max":{"1min":1.8640000000000001,"5min":1.887,"15min":2.141},"last":1.288}]},{"osd":6,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3320000000000001,"15min":1.3180000000000001},"min":{"1min":0.69499999999999995,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":1.5940000000000001,"5min":1.8,"15min":1.8400000000000001},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.399,"15min":1.3520000000000001},"min":{"1min":0.86399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.7769999999999999,"5min":1.903,"15min":1.903},"last":1.474}]},{"osd":7,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.367,"15min":1.361},"min":{"1min":0.748,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":1.766,"5min":1.7989999999999999,"15min":1.921},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.4039999999999999,"15min":1.369},"min":{"1min":0.66300000000000003,"5min":0.36499999999999999,"15min":0.35999999999999999},"max":{"1min":1.841,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.383}]}]}]}} 2021-11-21T22:49:04.874 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-21T22:49:04.928 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.919+0000 7f15104a0700 1 Processor -- start 2021-11-21T22:49:04.929 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.919+0000 7f15104a0700 1 -- start start 2021-11-21T22:49:04.940 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f150813d470 0x7f15080af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.941 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15080adb90 0x7f150813fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080ac090 con 0x7f15080abcb0 2021-11-21T22:49:04.942 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080b1590 con 0x7f150813d470 2021-11-21T22:49:04.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15080ae7c0 con 0x7f15080adb90 2021-11-21T22:49:04.943 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150e23c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f15080abcb0 0x7f150813cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38760/0 (socket says 172.21.2.116:38760) 2021-11-21T22:49:04.944 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150e23c700 1 -- 172.21.2.116:0/4108510432 learned_addr learned my addr 172.21.2.116:0/4108510432 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:04.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004081232 0 0) 0x7f15080ac090 con 0x7f15080abcb0 2021-11-21T22:49:04.945 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f0003260 con 0x7f15080abcb0 2021-11-21T22:49:04.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 --2- 172.21.2.116:0/4108510432 >> v2:172.21.2.117:3300/0 conn(0x7f150813d470 0x7f15080af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.946 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2715434029 0 0) 0x7f14f0003260 con 0x7f15080abcb0 2021-11-21T22:49:04.947 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15080adb90 msgr2=0x7f150813fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:04.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150ea3d700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15080adb90 0x7f150813fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.948 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15080adb90 0x7f150813fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 >> v2:172.21.2.117:3300/0 conn(0x7f150813d470 msgr2=0x7f15080af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:04.949 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 --2- 172.21.2.116:0/4108510432 >> v2:172.21.2.117:3300/0 conn(0x7f150813d470 0x7f15080af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508062590 con 0x7f15080abcb0 2021-11-21T22:49:04.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f14fc001660 con 0x7f15080abcb0 2021-11-21T22:49:04.950 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 >> v1:172.21.2.116:6789/0 conn(0x7f15080abcb0 legacy=0x7f150813cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f00037d0 0x7f14f0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14f0004500 0x7f14f0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.951 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14f0005070 0x7f14f000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15080ae7c0 con 0x7f14f0004500 2021-11-21T22:49:04.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f000bf20 con 0x7f14f0005070 2021-11-21T22:49:04.952 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f000c0e0 con 0x7f14f00037d0 2021-11-21T22:49:04.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 --2- 172.21.2.116:0/4108510432 >> v2:172.21.2.117:3300/0 conn(0x7f150813d470 0x7f15080af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:49:04.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150ea3d700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f15080adb90 0x7f150813fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:49:04.953 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14f0004500 0x7f14f0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150e23c700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f00037d0 0x7f14f0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.954 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150ea3d700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14f0005070 0x7f14f000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 -- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f00037d0 msgr2=0x7f14f0003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:04.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f00037d0 0x7f14f0003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.955 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 -- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14f0005070 msgr2=0x7f14f000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:04.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14f0005070 0x7f14f000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.956 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 -- 172.21.2.116:0/4108510432 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0003260 con 0x7f14f0004500 2021-11-21T22:49:04.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150ea3d700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f14f0005070 0x7f14f000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:49:04.957 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150da3b700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14f0004500 0x7f14f0004980 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7f1504007000 tx=0x7f1504008040).ready entity=mon.0 client_cookie=aa1f183596631ba8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1504015070 con 0x7f14f0004500 2021-11-21T22:49:04.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1504011410 con 0x7f14f0004500 2021-11-21T22:49:04.960 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150d23a700 1 -- 172.21.2.116:0/4108510432 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1504011770 con 0x7f14f0004500 2021-11-21T22:49:04.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- 172.21.2.116:0/4108510432 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14f0004500 msgr2=0x7f14f0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 --2- 172.21.2.116:0/4108510432 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f14f0004500 0x7f14f0004980 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7f1504007000 tx=0x7f1504008040).stop 2021-11-21T22:49:04.961 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f150ea3d700 1 -- 172.21.2.116:0/4108510432 reap_dead start 2021-11-21T22:49:04.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- 172.21.2.116:0/4108510432 shutdown_connections 2021-11-21T22:49:04.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- 172.21.2.116:0/4108510432 >> 172.21.2.116:0/4108510432 conn(0x7f150805bd20 msgr2=0x7f150805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:04.962 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- 172.21.2.116:0/4108510432 shutdown_connections 2021-11-21T22:49:04.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- 172.21.2.116:0/4108510432 wait complete. 2021-11-21T22:49:04.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 Processor -- start 2021-11-21T22:49:04.963 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.923+0000 7f15104a0700 1 -- start start 2021-11-21T22:49:04.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f0004500 0x7f15080b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1508142cb0 0x7f1508063850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.964 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15081424c0 0x7f1508063d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1508143150 con 0x7f1508142cb0 2021-11-21T22:49:04.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1508065870 con 0x7f15081424c0 2021-11-21T22:49:04.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508065b10 con 0x7f14f0004500 2021-11-21T22:49:04.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1508142cb0 0x7f1508063850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1508142cb0 0x7f1508063850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55050/0 (socket says 172.21.2.116:55050) 2021-11-21T22:49:04.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 -- 172.21.2.116:0/640968335 learned_addr learned my addr 172.21.2.116:0/640968335 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:04.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150e23c700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f0004500 0x7f15080b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 -- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f0004500 msgr2=0x7f15080b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150ea3d700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15081424c0 0x7f1508063d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f0004500 0x7f15080b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 -- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15081424c0 msgr2=0x7f1508063d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:04.969 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15081424c0 0x7f1508063d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:04.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 -- 172.21.2.116:0/640968335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508065db0 con 0x7f1508142cb0 2021-11-21T22:49:04.970 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150ea3d700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15081424c0 0x7f1508063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:49:04.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150da3b700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1508142cb0 0x7f1508063850 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f1504007000 tx=0x7f1504008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.971 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f14fb7fe700 1 -- 172.21.2.116:0/640968335 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150400ae70 con 0x7f1508142cb0 2021-11-21T22:49:04.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f14fb7fe700 1 -- 172.21.2.116:0/640968335 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1504000e90 con 0x7f1508142cb0 2021-11-21T22:49:04.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1508066250 con 0x7f1508142cb0 2021-11-21T22:49:04.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f14fb7fe700 1 -- 172.21.2.116:0/640968335 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150401c620 con 0x7f1508142cb0 2021-11-21T22:49:04.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150813fd60 con 0x7f1508142cb0 2021-11-21T22:49:04.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508065f40 con 0x7f1508142cb0 2021-11-21T22:49:04.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f14fb7fe700 1 -- 172.21.2.116:0/640968335 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f150401c780 con 0x7f1508142cb0 2021-11-21T22:49:04.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f14fb7fe700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14e80534f0 0x7f14e80559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:04.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f14fb7fe700 1 -- 172.21.2.116:0/640968335 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f150406cf80 con 0x7f1508142cb0 2021-11-21T22:49:04.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150e23c700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14e80534f0 0x7f14e80559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:04.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.927+0000 7f150e23c700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14e80534f0 0x7f14e80559b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f14fc00d000 tx=0x7f14fc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:04.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:04.931+0000 7f14fb7fe700 1 -- 172.21.2.116:0/640968335 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f15040443a0 con 0x7f1508142cb0 2021-11-21T22:49:05.019 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.011+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15080573d0 con 0x7f1508142cb0 2021-11-21T22:49:05.020 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.011+0000 7f14fb7fe700 1 -- 172.21.2.116:0/640968335 <== mon.0 v2:172.21.2.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+13318 (secure 0 0 0) 0x7f1504047980 con 0x7f1508142cb0 2021-11-21T22:49:05.021 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:49:05.021 INFO:teuthology.orchestra.run.gibba016.stdout:{"epoch":165,"fsid":"fe3c4e1d-e687-4491-9a93-95ac7299e2b3","created":"2021-11-21T21:54:10.975984+0000","modified":"2021-11-21T22:03:34.000323+0000","last_up_change":"2021-11-21T21:54:15.911408+0000","last_in_change":"2021-11-21T21:54:15.911408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-21T21:54:13.059512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-11-21T21:54:20.960577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3baac6c-e283-460c-b082-9b135ec136dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6800","nonce":14707},{"type":"v1","addr":"172.21.2.116:6801","nonce":14707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6802","nonce":14707},{"type":"v1","addr":"172.21.2.116:6803","nonce":14707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6806","nonce":14707},{"type":"v1","addr":"172.21.2.116:6807","nonce":14707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6804","nonce":14707},{"type":"v1","addr":"172.21.2.116:6805","nonce":14707}]},"public_addr":"172.21.2.116:6801/14707","cluster_addr":"172.21.2.116:6803/14707","heartbeat_back_addr":"172.21.2.116:6807/14707","heartbeat_front_addr":"172.21.2.116:6805/14707","state":["exists","up"]},{"osd":1,"uuid":"61210302-784a-4f36-a7bd-7c05c0e31483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6808","nonce":14708},{"type":"v1","addr":"172.21.2.116:6809","nonce":14708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6810","nonce":14708},{"type":"v1","addr":"172.21.2.116:6811","nonce":14708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6814","nonce":14708},{"type":"v1","addr":"172.21.2.116:6815","nonce":14708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6812","nonce":14708},{"type":"v1","addr":"172.21.2.116:6813","nonce":14708}]},"public_addr":"172.21.2.116:6809/14708","cluster_addr":"172.21.2.116:6811/14708","heartbeat_back_addr":"172.21.2.116:6815/14708","heartbeat_front_addr":"172.21.2.116:6813/14708","state":["exists","up"]},{"osd":2,"uuid":"c6ace5b1-b40d-4f54-8a3d-baa8dd7ace56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6816","nonce":14709},{"type":"v1","addr":"172.21.2.116:6817","nonce":14709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6818","nonce":14709},{"type":"v1","addr":"172.21.2.116:6819","nonce":14709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6822","nonce":14709},{"type":"v1","addr":"172.21.2.116:6823","nonce":14709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6820","nonce":14709},{"type":"v1","addr":"172.21.2.116:6821","nonce":14709}]},"public_addr":"172.21.2.116:6817/14709","cluster_addr":"172.21.2.116:6819/14709","heartbeat_back_addr":"172.21.2.116:6823/14709","heartbeat_front_addr":"172.21.2.116:6821/14709","state":["exists","up"]},{"osd":3,"uuid":"dcebb53a-56e7-4d2f-9ac5-a61776613ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6824","nonce":14710},{"type":"v1","addr":"172.21.2.116:6825","nonce":14710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6826","nonce":14710},{"type":"v1","addr":"172.21.2.116:6827","nonce":14710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6830","nonce":14710},{"type":"v1","addr":"172.21.2.116:6831","nonce":14710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.116:6828","nonce":14710},{"type":"v1","addr":"172.21.2.116:6829","nonce":14710}]},"public_addr":"172.21.2.116:6825/14710","cluster_addr":"172.21.2.116:6827/14710","heartbeat_back_addr":"172.21.2.116:6831/14710","heartbeat_front_addr":"172.21.2.116:6829/14710","state":["exists","up"]},{"osd":4,"uuid":"a7251112-9008-418e-a7b6-4501e7b9365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6800","nonce":14956},{"type":"v1","addr":"172.21.2.117:6801","nonce":14956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6802","nonce":14956},{"type":"v1","addr":"172.21.2.117:6803","nonce":14956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6806","nonce":14956},{"type":"v1","addr":"172.21.2.117:6807","nonce":14956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6804","nonce":14956},{"type":"v1","addr":"172.21.2.117:6805","nonce":14956}]},"public_addr":"172.21.2.117:6801/14956","cluster_addr":"172.21.2.117:6803/14956","heartbeat_back_addr":"172.21.2.117:6807/14956","heartbeat_front_addr":"172.21.2.117:6805/14956","state":["exists","up"]},{"osd":5,"uuid":"0c8cf701-2ace-4542-a985-c4435cd023fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6808","nonce":14957},{"type":"v1","addr":"172.21.2.117:6809","nonce":14957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6810","nonce":14957},{"type":"v1","addr":"172.21.2.117:6811","nonce":14957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6814","nonce":14957},{"type":"v1","addr":"172.21.2.117:6815","nonce":14957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6812","nonce":14957},{"type":"v1","addr":"172.21.2.117:6813","nonce":14957}]},"public_addr":"172.21.2.117:6809/14957","cluster_addr":"172.21.2.117:6811/14957","heartbeat_back_addr":"172.21.2.117:6815/14957","heartbeat_front_addr":"172.21.2.117:6813/14957","state":["exists","up"]},{"osd":6,"uuid":"66ee78f9-53ac-45a6-a441-8a71e4e06bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6816","nonce":14958},{"type":"v1","addr":"172.21.2.117:6817","nonce":14958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6818","nonce":14958},{"type":"v1","addr":"172.21.2.117:6819","nonce":14958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6822","nonce":14958},{"type":"v1","addr":"172.21.2.117:6824","nonce":14958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6820","nonce":14958},{"type":"v1","addr":"172.21.2.117:6821","nonce":14958}]},"public_addr":"172.21.2.117:6817/14958","cluster_addr":"172.21.2.117:6819/14958","heartbeat_back_addr":"172.21.2.117:6824/14958","heartbeat_front_addr":"172.21.2.117:6821/14958","state":["exists","up"]},{"osd":7,"uuid":"7c1bf6d9-c24a-4373-9eee-5488fd33e157","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6823","nonce":14960},{"type":"v1","addr":"172.21.2.117:6825","nonce":14960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6826","nonce":14960},{"type":"v1","addr":"172.21.2.117:6827","nonce":14960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6830","nonce":14960},{"type":"v1","addr":"172.21.2.117:6831","nonce":14960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.117:6828","nonce":14960},{"type":"v1","addr":"172.21.2.117:6829","nonce":14960}]},"public_addr":"172.21.2.117:6825/14960","cluster_addr":"172.21.2.117:6827/14960","heartbeat_back_addr":"172.21.2.117:6831/14960","heartbeat_front_addr":"172.21.2.117:6829/14960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.929221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.917817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.955321+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.949148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.005322+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.943674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:13.990443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-11-21T21:54:14.011952+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.116:6835/403152574":"2021-11-22T22:03:30.888079+0000","172.21.2.116:6834/403152574":"2021-11-22T22:03:30.888079+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-11-21T22:49:05.022 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14e80534f0 msgr2=0x7f14e80559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:05.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14e80534f0 0x7f14e80559b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f14fc00d000 tx=0x7f14fc009040).stop 2021-11-21T22:49:05.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1508142cb0 msgr2=0x7f1508063850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:05.023 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1508142cb0 0x7f1508063850 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f1504007000 tx=0x7f1504008040).stop 2021-11-21T22:49:05.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 shutdown_connections 2021-11-21T22:49:05.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f15081424c0 0x7f1508063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:05.024 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f14e80534f0 0x7f14e80559b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:05.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f1508142cb0 0x7f1508063850 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:05.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 --2- 172.21.2.116:0/640968335 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f14f0004500 0x7f15080b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:05.025 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 >> 172.21.2.116:0/640968335 conn(0x7f150805bd20 msgr2=0x7f15080af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:05.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 shutdown_connections 2021-11-21T22:49:05.026 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:05.015+0000 7f15104a0700 1 -- 172.21.2.116:0/640968335 wait complete. 2021-11-21T22:49:06.029 INFO:tasks.ceph:Scrubbing osd.0 2021-11-21T22:49:06.030 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:06.096 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69e11700 1 Processor -- start 2021-11-21T22:49:06.097 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69e11700 1 -- start start 2021-11-21T22:49:06.098 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69e11700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6409a200 0x7f7a6409a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69e11700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a640b11c0 0x7f7a64097df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.099 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69e11700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a64148040 con 0x7f7a640b0ae0 2021-11-21T22:49:06.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69e11700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a64147da0 con 0x7f7a640b11c0 2021-11-21T22:49:06.100 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69e11700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640affb0 con 0x7f7a6409a200 2021-11-21T22:49:06.101 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69610700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a640b11c0 0x7f7a64097df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69610700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f7a640b11c0 0x7f7a64097df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55060/0 (socket says 172.21.2.116:55060) 2021-11-21T22:49:06.102 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a63fff700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f7a640b0ae0 0x7f7a641445c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38782/0 (socket says 172.21.2.116:38782) 2021-11-21T22:49:06.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.079+0000 7f7a69610700 1 -- 172.21.2.116:0/3134776588 learned_addr learned my addr 172.21.2.116:0/3134776588 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.103 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69610700 1 -- 172.21.2.116:0/3134776588 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6409a200 msgr2=0x7f7a6409a5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:06.104 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69610700 1 --2- 172.21.2.116:0/3134776588 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6409a200 0x7f7a6409a5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69610700 1 -- 172.21.2.116:0/3134776588 >> v1:172.21.2.116:6789/0 conn(0x7f7a640b0ae0 legacy=0x7f7a641445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.105 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69610700 1 -- 172.21.2.116:0/3134776588 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a640b0250 con 0x7f7a640b11c0 2021-11-21T22:49:06.106 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69610700 1 --2- 172.21.2.116:0/3134776588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b11c0 0x7f7a64097df0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f7a5c0032f0 tx=0x7f7a5c009290).ready entity=mon.0 client_cookie=58eaca43596063d1 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a637fe700 1 -- 172.21.2.116:0/3134776588 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5c00b8a0 con 0x7f7a640b11c0 2021-11-21T22:49:06.107 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a637fe700 1 -- 172.21.2.116:0/3134776588 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c00ba00 con 0x7f7a640b11c0 2021-11-21T22:49:06.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a637fe700 1 -- 172.21.2.116:0/3134776588 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5c00bbb0 con 0x7f7a640b11c0 2021-11-21T22:49:06.108 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/3134776588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b11c0 msgr2=0x7f7a64097df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.109 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 --2- 172.21.2.116:0/3134776588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b11c0 0x7f7a64097df0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f7a5c0032f0 tx=0x7f7a5c009290).stop 2021-11-21T22:49:06.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/3134776588 shutdown_connections 2021-11-21T22:49:06.110 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 --2- 172.21.2.116:0/3134776588 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b11c0 0x7f7a64097df0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 --2- 172.21.2.116:0/3134776588 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6409a200 0x7f7a6409a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.111 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/3134776588 >> 172.21.2.116:0/3134776588 conn(0x7f7a640b4d70 msgr2=0x7f7a640b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.112 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/3134776588 shutdown_connections 2021-11-21T22:49:06.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/3134776588 wait complete. 2021-11-21T22:49:06.113 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 Processor -- start 2021-11-21T22:49:06.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- start start 2021-11-21T22:49:06.114 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6409a200 0x7f7a64144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.115 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b0ae0 0x7f7a64148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a640b11c0 0x7f7a64065480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.116 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64149210 con 0x7f7a640b0ae0 2021-11-21T22:49:06.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64065970 con 0x7f7a640b11c0 2021-11-21T22:49:06.117 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64065bc0 con 0x7f7a6409a200 2021-11-21T22:49:06.118 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b0ae0 0x7f7a64148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b0ae0 0x7f7a64148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55064/0 (socket says 172.21.2.116:55064) 2021-11-21T22:49:06.119 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 -- 172.21.2.116:0/1506668563 learned_addr learned my addr 172.21.2.116:0/1506668563 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.120 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 -- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6409a200 msgr2=0x7f7a64144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f7a6409a200 0x7f7a64144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.121 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 -- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a640b11c0 msgr2=0x7f7a64065480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f7a640b11c0 0x7f7a64065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.122 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 -- 172.21.2.116:0/1506668563 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c009530 con 0x7f7a640b0ae0 2021-11-21T22:49:06.123 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a63fff700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b0ae0 0x7f7a64148d20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f7a640626a0 tx=0x7f7a58007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5800ce30 con 0x7f7a640b0ae0 2021-11-21T22:49:06.124 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a580072e0 con 0x7f7a640b0ae0 2021-11-21T22:49:06.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a64065e60 con 0x7f7a640b0ae0 2021-11-21T22:49:06.125 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a641598e0 con 0x7f7a640b0ae0 2021-11-21T22:49:06.126 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a58007450 con 0x7f7a640b0ae0 2021-11-21T22:49:06.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.083+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a64159dd0 con 0x7f7a640b0ae0 2021-11-21T22:49:06.127 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f7a58010070 con 0x7f7a640b0ae0 2021-11-21T22:49:06.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a61ffb700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a44053660 0x7f7a44055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7a5806b0c0 con 0x7f7a640b0ae0 2021-11-21T22:49:06.128 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a61ffb700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7f7a44059e60 0x7f7a4405c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 --> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a4405c840 con 0x7f7a44059e60 2021-11-21T22:49:06.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a5806b3d0 con 0x7f7a640b0ae0 2021-11-21T22:49:06.129 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a69610700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7f7a44059e60 0x7f7a4405c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a69610700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7f7a44059e60 0x7f7a4405c280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a68e0f700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a44053660 0x7f7a44055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.130 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== osd.0 v2:172.21.2.116:6800/14707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f7a4405c840 con 0x7f7a44059e60 2021-11-21T22:49:06.131 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.087+0000 7f7a68e0f700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a44053660 0x7f7a44055b20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f7a54002fb0 tx=0x7f7a540069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 --> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7a6415a6e0 con 0x7f7a44059e60 2021-11-21T22:49:06.132 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a61ffb700 1 -- 172.21.2.116:0/1506668563 <== osd.0 v2:172.21.2.116:6800/14707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7a6415a6e0 con 0x7f7a44059e60 2021-11-21T22:49:06.133 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:06.133 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:06.134 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:06.135 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7f7a44059e60 msgr2=0x7f7a4405c280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:6800/14707,v1:172.21.2.116:6801/14707] conn(0x7f7a44059e60 0x7f7a4405c280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a44053660 msgr2=0x7f7a44055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.136 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f7a44053660 0x7f7a44055b20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f7a54002fb0 tx=0x7f7a540069b0).stop 2021-11-21T22:49:06.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b0ae0 msgr2=0x7f7a64148d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 --2- 172.21.2.116:0/1506668563 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f7a640b0ae0 0x7f7a64148d20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f7a640626a0 tx=0x7f7a58007c40).stop 2021-11-21T22:49:06.137 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69610700 1 -- 172.21.2.116:0/1506668563 reap_dead start 2021-11-21T22:49:06.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 shutdown_connections 2021-11-21T22:49:06.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 >> 172.21.2.116:0/1506668563 conn(0x7f7a640b4d70 msgr2=0x7f7a640b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 shutdown_connections 2021-11-21T22:49:06.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.095+0000 7f7a69e11700 1 -- 172.21.2.116:0/1506668563 wait complete. 2021-11-21T22:49:06.139 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2021-11-21T22:49:06.201 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 Processor -- start 2021-11-21T22:49:06.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- start start 2021-11-21T22:49:06.202 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55f4144810 0x7f55f41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 0x7f55f40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f4147e80 con 0x7f55f40b1db0 2021-11-21T22:49:06.203 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40b1490 con 0x7f55f4144810 2021-11-21T22:49:06.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40af060 con 0x7f55f40abce0 2021-11-21T22:49:06.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55f4144810 0x7f55f41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.204 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f55f4144810 0x7f55f41478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55072/0 (socket says 172.21.2.116:55072) 2021-11-21T22:49:06.205 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 -- 172.21.2.116:0/216016265 learned_addr learned my addr 172.21.2.116:0/216016265 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.207 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 -- 172.21.2.116:0/216016265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 msgr2=0x7f55f40b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:06.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 --2- 172.21.2.116:0/216016265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 0x7f55f40b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 -- 172.21.2.116:0/216016265 >> v1:172.21.2.117:6789/0 conn(0x7f55f40b1db0 legacy=0x7f55f40ac600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:06.208 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 -- 172.21.2.116:0/216016265 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40af5a0 con 0x7f55f4144810 2021-11-21T22:49:06.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f8e23700 1 --2- 172.21.2.116:0/216016265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f41478c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7f55f40b2a30 tx=0x7f55e4006eb0).ready entity=mon.0 client_cookie=dbc8c6bd45bfdd36 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f37fe700 1 -- 172.21.2.116:0/216016265 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55e400d570 con 0x7f55f4144810 2021-11-21T22:49:06.209 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f37fe700 1 -- 172.21.2.116:0/216016265 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e4000e90 con 0x7f55f4144810 2021-11-21T22:49:06.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f37fe700 1 -- 172.21.2.116:0/216016265 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55e400d8c0 con 0x7f55f4144810 2021-11-21T22:49:06.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- 172.21.2.116:0/216016265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 msgr2=0x7f55f41478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.210 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- 172.21.2.116:0/216016265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f41478c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7f55f40b2a30 tx=0x7f55e4006eb0).stop 2021-11-21T22:49:06.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- 172.21.2.116:0/216016265 shutdown_connections 2021-11-21T22:49:06.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- 172.21.2.116:0/216016265 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 0x7f55f40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- 172.21.2.116:0/216016265 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f41478c0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.211 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- 172.21.2.116:0/216016265 >> 172.21.2.116:0/216016265 conn(0x7f55f405bd20 msgr2=0x7f55f405c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- 172.21.2.116:0/216016265 shutdown_connections 2021-11-21T22:49:06.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- 172.21.2.116:0/216016265 wait complete. 2021-11-21T22:49:06.212 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 Processor -- start 2021-11-21T22:49:06.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- start start 2021-11-21T22:49:06.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 0x7f55f413c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.213 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f4159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55f41465e0 0x7f55f415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f415a690 con 0x7f55f4144810 2021-11-21T22:49:06.214 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4145ba0 con 0x7f55f41465e0 2021-11-21T22:49:06.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55fb087700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4145fd0 con 0x7f55f40abce0 2021-11-21T22:49:06.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f4159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.215 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f4159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55082/0 (socket says 172.21.2.116:55082) 2021-11-21T22:49:06.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.195+0000 7f55f3fff700 1 -- 172.21.2.116:0/163872068 learned_addr learned my addr 172.21.2.116:0/163872068 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f3fff700 1 -- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 msgr2=0x7f55f413c8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:06.216 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f3fff700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 0x7f55f413c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f3fff700 1 -- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55f41465e0 msgr2=0x7f55f415a1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:06.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f3fff700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55f41465e0 0x7f55f415a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f3fff700 1 -- 172.21.2.116:0/163872068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e400b1e0 con 0x7f55f4144810 2021-11-21T22:49:06.217 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f3fff700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f4159cb0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7f55dc002fb0 tx=0x7f55dc006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f1ffb700 1 -- 172.21.2.116:0/163872068 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc013460 con 0x7f55f4144810 2021-11-21T22:49:06.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f1ffb700 1 -- 172.21.2.116:0/163872068 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc0135c0 con 0x7f55f4144810 2021-11-21T22:49:06.218 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40b4180 con 0x7f55f4144810 2021-11-21T22:49:06.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f4144f00 con 0x7f55f4144810 2021-11-21T22:49:06.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f1ffb700 1 -- 172.21.2.116:0/163872068 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc013920 con 0x7f55f4144810 2021-11-21T22:49:06.219 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f40b3fe0 con 0x7f55f4144810 2021-11-21T22:49:06.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f1ffb700 1 -- 172.21.2.116:0/163872068 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f55dc00f070 con 0x7f55f4144810 2021-11-21T22:49:06.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f1ffb700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55d8053610 0x7f55d8055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.220 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f1ffb700 1 -- 172.21.2.116:0/163872068 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f55dc06b1b0 con 0x7f55f4144810 2021-11-21T22:49:06.221 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f8e23700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55d8053610 0x7f55d8055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.199+0000 7f55f8e23700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55d8053610 0x7f55d8055ad0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f55f40b2a30 tx=0x7f55e4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.222 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.203+0000 7f55f1ffb700 1 -- 172.21.2.116:0/163872068 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55f40b3fe0 con 0x7f55f4144810 2021-11-21T22:49:06.291 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.283+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f40b4310 con 0x7f55d8053610 2021-11-21T22:49:06.292 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.283+0000 7f55f1ffb700 1 -- 172.21.2.116:0/163872068 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55e400d600 con 0x7f55d8053610 2021-11-21T22:49:06.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55d8053610 msgr2=0x7f55d8055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.294 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55d8053610 0x7f55d8055ad0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f55f40b2a30 tx=0x7f55e4015000).stop 2021-11-21T22:49:06.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 msgr2=0x7f55f4159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f4159cb0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7f55dc002fb0 tx=0x7f55dc006de0).stop 2021-11-21T22:49:06.295 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 shutdown_connections 2021-11-21T22:49:06.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f55f41465e0 0x7f55f415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f55d8053610 0x7f55d8055ad0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.296 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f55f4144810 0x7f55f4159cb0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 --2- 172.21.2.116:0/163872068 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f55f40abce0 0x7f55f413c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 >> 172.21.2.116:0/163872068 conn(0x7f55f405bd20 msgr2=0x7f55f4140a80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.297 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 shutdown_connections 2021-11-21T22:49:06.298 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.287+0000 7f55fb087700 1 -- 172.21.2.116:0/163872068 wait complete. 2021-11-21T22:49:06.298 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 0 to deep-scrub 2021-11-21T22:49:06.300 INFO:tasks.ceph:Scrubbing osd.1 2021-11-21T22:49:06.301 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:06.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 Processor -- start 2021-11-21T22:49:06.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- start start 2021-11-21T22:49:06.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f109c0b11c0 0x7f109c147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f109c0a8670 0x7f109c147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f109c147c10 0x7f109c14a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0b1940 con 0x7f109c0b11c0 2021-11-21T22:49:06.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0b15a0 con 0x7f109c147c10 2021-11-21T22:49:06.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0b1b70 con 0x7f109c0a8670 2021-11-21T22:49:06.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f109c0b11c0 0x7f109c147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f109c0b11c0 0x7f109c147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55086/0 (socket says 172.21.2.116:55086) 2021-11-21T22:49:06.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 -- 172.21.2.116:0/481150442 learned_addr learned my addr 172.21.2.116:0/481150442 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 -- 172.21.2.116:0/481150442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f109c0a8670 msgr2=0x7f109c147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 --2- 172.21.2.116:0/481150442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f109c0a8670 0x7f109c147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 -- 172.21.2.116:0/481150442 >> v2:172.21.2.117:3300/0 conn(0x7f109c147c10 msgr2=0x7f109c14a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 --2- 172.21.2.116:0/481150442 >> v2:172.21.2.117:3300/0 conn(0x7f109c147c10 0x7f109c14a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 -- 172.21.2.116:0/481150442 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0b1e10 con 0x7f109c0b11c0 2021-11-21T22:49:06.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a0dc1700 1 --2- 172.21.2.116:0/481150442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c0b11c0 0x7f109c147230 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7f108c002f70 tx=0x7f108c006eb0).ready entity=mon.0 client_cookie=11ace3eb149b76b6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f109b7fe700 1 -- 172.21.2.116:0/481150442 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c00d570 con 0x7f109c0b11c0 2021-11-21T22:49:06.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f109b7fe700 1 -- 172.21.2.116:0/481150442 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c000e90 con 0x7f109c0b11c0 2021-11-21T22:49:06.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f109b7fe700 1 -- 172.21.2.116:0/481150442 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c00d900 con 0x7f109c0b11c0 2021-11-21T22:49:06.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- 172.21.2.116:0/481150442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c0b11c0 msgr2=0x7f109c147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 --2- 172.21.2.116:0/481150442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c0b11c0 0x7f109c147230 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7f108c002f70 tx=0x7f108c006eb0).stop 2021-11-21T22:49:06.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- 172.21.2.116:0/481150442 shutdown_connections 2021-11-21T22:49:06.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 --2- 172.21.2.116:0/481150442 >> v2:172.21.2.117:3300/0 conn(0x7f109c147c10 0x7f109c14a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 --2- 172.21.2.116:0/481150442 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f109c0a8670 0x7f109c147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 --2- 172.21.2.116:0/481150442 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c0b11c0 0x7f109c147230 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.363 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- 172.21.2.116:0/481150442 >> 172.21.2.116:0/481150442 conn(0x7f109c05bd20 msgr2=0x7f109c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- 172.21.2.116:0/481150442 shutdown_connections 2021-11-21T22:49:06.364 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- 172.21.2.116:0/481150442 wait complete. 2021-11-21T22:49:06.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 Processor -- start 2021-11-21T22:49:06.367 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.347+0000 7f10a3025700 1 -- start start 2021-11-21T22:49:06.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f109c0a8670 0x7f109c1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f109c0b11c0 0x7f109c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.368 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c147c10 0x7f109c15a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c144a60 con 0x7f109c147c10 2021-11-21T22:49:06.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c144230 con 0x7f109c0b11c0 2021-11-21T22:49:06.369 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c13f950 con 0x7f109c0a8670 2021-11-21T22:49:06.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c147c10 0x7f109c15a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c147c10 0x7f109c15a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55094/0 (socket says 172.21.2.116:55094) 2021-11-21T22:49:06.370 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 -- 172.21.2.116:0/4238415554 learned_addr learned my addr 172.21.2.116:0/4238415554 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 -- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f109c0a8670 msgr2=0x7f109c1497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f109c0a8670 0x7f109c1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.371 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 -- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f109c0b11c0 msgr2=0x7f109c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f109c0b11c0 0x7f109c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 -- 172.21.2.116:0/4238415554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c00b1e0 con 0x7f109c147c10 2021-11-21T22:49:06.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a15c2700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c147c10 0x7f109c15a020 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7f109c1423e0 tx=0x7f10940091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.372 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109400f760 con 0x7f109c147c10 2021-11-21T22:49:06.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109400f8c0 con 0x7f109c147c10 2021-11-21T22:49:06.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109400fc20 con 0x7f109c147c10 2021-11-21T22:49:06.373 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c13f6b0 con 0x7f109c147c10 2021-11-21T22:49:06.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c13fde0 con 0x7f109c147c10 2021-11-21T22:49:06.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f109c141fb0 con 0x7f109c147c10 2021-11-21T22:49:06.374 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f109401a460 con 0x7f109c147c10 2021-11-21T22:49:06.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10800536f0 0x7f1080055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f109406c890 con 0x7f109c147c10 2021-11-21T22:49:06.375 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f1080059ef0 0x7f108005c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 --> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f108005c8d0 con 0x7f1080059ef0 2021-11-21T22:49:06.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f109406cba0 con 0x7f109c147c10 2021-11-21T22:49:06.376 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f109bfff700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f1080059ef0 0x7f108005c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f10a0dc1700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10800536f0 0x7f1080055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f109bfff700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f1080059ef0 0x7f108005c310 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.377 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.351+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== osd.1 v2:172.21.2.116:6808/14708 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f108005c8d0 con 0x7f1080059ef0 2021-11-21T22:49:06.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.355+0000 7f10a0dc1700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10800536f0 0x7f1080055bb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f108c002f70 tx=0x7f108c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.378 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 --> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f109c1598d0 con 0x7f1080059ef0 2021-11-21T22:49:06.379 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f1099ffb700 1 -- 172.21.2.116:0/4238415554 <== osd.1 v2:172.21.2.116:6808/14708 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f109c1598d0 con 0x7f1080059ef0 2021-11-21T22:49:06.379 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:06.380 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:06.380 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:06.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f1080059ef0 msgr2=0x7f108005c310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.382 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:6808/14708,v1:172.21.2.116:6809/14708] conn(0x7f1080059ef0 0x7f108005c310 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10800536f0 msgr2=0x7f1080055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f10800536f0 0x7f1080055bb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f108c002f70 tx=0x7f108c015000).stop 2021-11-21T22:49:06.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c147c10 msgr2=0x7f109c15a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 --2- 172.21.2.116:0/4238415554 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f109c147c10 0x7f109c15a020 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7f109c1423e0 tx=0x7f10940091d0).stop 2021-11-21T22:49:06.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a15c2700 1 -- 172.21.2.116:0/4238415554 reap_dead start 2021-11-21T22:49:06.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 shutdown_connections 2021-11-21T22:49:06.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 >> 172.21.2.116:0/4238415554 conn(0x7f109c05bd20 msgr2=0x7f109c145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 shutdown_connections 2021-11-21T22:49:06.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.359+0000 7f10a3025700 1 -- 172.21.2.116:0/4238415554 wait complete. 2021-11-21T22:49:06.386 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2021-11-21T22:49:06.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7cc84700 1 Processor -- start 2021-11-21T22:49:06.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7cc84700 1 -- start start 2021-11-21T22:49:06.445 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7cc84700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a7805efb0 0x7f2a780b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7cc84700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a780b0500 con 0x7f2a780b5380 2021-11-21T22:49:06.446 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7cc84700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a780b1b50 con 0x7f2a780b13b0 2021-11-21T22:49:06.447 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7cc84700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a780adc70 con 0x7f2a7805efb0 2021-11-21T22:49:06.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7659c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f2a780b5380 0x7f2a7813c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38826/0 (socket says 172.21.2.116:38826) 2021-11-21T22:49:06.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7659c700 1 -- 172.21.2.116:0/2282813586 learned_addr learned my addr 172.21.2.116:0/2282813586 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:06.448 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392395563 0 0) 0x7f2a780b0500 con 0x7f2a780b5380 2021-11-21T22:49:06.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a58003260 con 0x7f2a780b5380 2021-11-21T22:49:06.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2155375998 0 0) 0x7f2a58003260 con 0x7f2a780b5380 2021-11-21T22:49:06.449 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a7805efb0 msgr2=0x7f2a780b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a7805efb0 0x7f2a780b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 >> v1:172.21.2.117:6789/0 conn(0x7f2a780b13b0 legacy=0x7f2a781424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.450 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a780afd20 con 0x7f2a780b5380 2021-11-21T22:49:06.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f2a60001660 con 0x7f2a780b5380 2021-11-21T22:49:06.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 >> v1:172.21.2.116:6789/0 conn(0x7f2a780b5380 legacy=0x7f2a7813c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a7805efb0 0x7f2a58003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.451 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2a58003e20 0x7f2a58004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a58004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5800b930 con 0x7f2a58004b10 2021-11-21T22:49:06.452 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5800baa0 con 0x7f2a58003e20 2021-11-21T22:49:06.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5800bc60 con 0x7f2a7805efb0 2021-11-21T22:49:06.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.431+0000 7f2a76d9d700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a58004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.453 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a76d9d700 1 -- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a7805efb0 msgr2=0x7f2a58003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a76d9d700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a7805efb0 0x7f2a58003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a76d9d700 1 -- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2a58003e20 msgr2=0x7f2a58004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.454 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a76d9d700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2a58003e20 0x7f2a58004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a76d9d700 1 -- 172.21.2.116:0/2282813586 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58003260 con 0x7f2a58004b10 2021-11-21T22:49:06.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a76d9d700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a58004f90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7f2a700012f0 tx=0x7f2a70007d60).ready entity=mon.0 client_cookie=3d445d713194d974 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.455 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a700014c0 con 0x7f2a58004b10 2021-11-21T22:49:06.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a70006870 con 0x7f2a58004b10 2021-11-21T22:49:06.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7559a700 1 -- 172.21.2.116:0/2282813586 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a70006bd0 con 0x7f2a58004b10 2021-11-21T22:49:06.456 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/2282813586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 msgr2=0x7f2a58004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 --2- 172.21.2.116:0/2282813586 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a58004f90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7f2a700012f0 tx=0x7f2a70007d60).stop 2021-11-21T22:49:06.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a76d9d700 1 -- 172.21.2.116:0/2282813586 reap_dead start 2021-11-21T22:49:06.457 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/2282813586 shutdown_connections 2021-11-21T22:49:06.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/2282813586 >> 172.21.2.116:0/2282813586 conn(0x7f2a78061900 msgr2=0x7f2a78069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/2282813586 shutdown_connections 2021-11-21T22:49:06.458 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/2282813586 wait complete. 2021-11-21T22:49:06.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 Processor -- start 2021-11-21T22:49:06.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- start start 2021-11-21T22:49:06.459 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a78129050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2a7812a410 0x7f2a78134670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a78134b60 0x7f2a7812cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.460 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a78140ae0 con 0x7f2a58004b10 2021-11-21T22:49:06.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a78133820 con 0x7f2a7812a410 2021-11-21T22:49:06.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a78141f90 con 0x7f2a78134b60 2021-11-21T22:49:06.461 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a78129050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a78129050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55116/0 (socket says 172.21.2.116:55116) 2021-11-21T22:49:06.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 -- 172.21.2.116:0/3446015420 learned_addr learned my addr 172.21.2.116:0/3446015420 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.462 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 -- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a78134b60 msgr2=0x7f2a7812cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a78134b60 0x7f2a7812cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 -- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2a7812a410 msgr2=0x7f2a78134670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.463 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2a7812a410 0x7f2a78134670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 -- 172.21.2.116:0/3446015420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a780abf90 con 0x7f2a58004b10 2021-11-21T22:49:06.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7659c700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a78129050 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 rx=0x7f2a78129930 tx=0x7f2a781421d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.464 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a6f7fe700 1 -- 172.21.2.116:0/3446015420 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a60007170 con 0x7f2a58004b10 2021-11-21T22:49:06.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a6f7fe700 1 -- 172.21.2.116:0/3446015420 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a600072d0 con 0x7f2a58004b10 2021-11-21T22:49:06.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a78134290 con 0x7f2a58004b10 2021-11-21T22:49:06.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a6f7fe700 1 -- 172.21.2.116:0/3446015420 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a60004920 con 0x7f2a58004b10 2021-11-21T22:49:06.465 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a78129b00 con 0x7f2a58004b10 2021-11-21T22:49:06.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7812a100 con 0x7f2a58004b10 2021-11-21T22:49:06.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.435+0000 7f2a6f7fe700 1 -- 172.21.2.116:0/3446015420 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f2a60010070 con 0x7f2a58004b10 2021-11-21T22:49:06.466 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.439+0000 7f2a6f7fe700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2a5c053520 0x7f2a5c0559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.439+0000 7f2a6f7fe700 1 -- 172.21.2.116:0/3446015420 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2a6006b3e0 con 0x7f2a58004b10 2021-11-21T22:49:06.467 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.439+0000 7f2a75d9b700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2a5c053520 0x7f2a5c0559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.439+0000 7f2a75d9b700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2a5c053520 0x7f2a5c0559e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f2a68002f70 tx=0x7f2a68006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.468 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.439+0000 7f2a6f7fe700 1 -- 172.21.2.116:0/3446015420 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2a60042780 con 0x7f2a58004b10 2021-11-21T22:49:06.531 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.523+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a78133400 con 0x7f2a5c053520 2021-11-21T22:49:06.532 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a6f7fe700 1 -- 172.21.2.116:0/3446015420 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2a68012450 con 0x7f2a5c053520 2021-11-21T22:49:06.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2a5c053520 msgr2=0x7f2a5c0559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2a5c053520 0x7f2a5c0559e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f2a68002f70 tx=0x7f2a68006a00).stop 2021-11-21T22:49:06.534 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 msgr2=0x7f2a78129050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a78129050 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 rx=0x7f2a78129930 tx=0x7f2a781421d0).stop 2021-11-21T22:49:06.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 shutdown_connections 2021-11-21T22:49:06.535 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f2a78134b60 0x7f2a7812cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f2a7812a410 0x7f2a78134670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f2a5c053520 0x7f2a5c0559e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.536 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 --2- 172.21.2.116:0/3446015420 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f2a58004b10 0x7f2a78129050 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 >> 172.21.2.116:0/3446015420 conn(0x7f2a78061900 msgr2=0x7f2a78142400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 shutdown_connections 2021-11-21T22:49:06.537 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.527+0000 7f2a7cc84700 1 -- 172.21.2.116:0/3446015420 wait complete. 2021-11-21T22:49:06.538 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 1 to deep-scrub 2021-11-21T22:49:06.540 INFO:tasks.ceph:Scrubbing osd.2 2021-11-21T22:49:06.541 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:06.602 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db8cb8700 1 Processor -- start 2021-11-21T22:49:06.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db8cb8700 1 -- start start 2021-11-21T22:49:06.603 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db8cb8700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f5db40aed90 0x7f5db40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db8cb8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db40af360 0x7f5db413c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db8cb8700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40b00d0 con 0x7f5db40b24a0 2021-11-21T22:49:06.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db8cb8700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db40adba0 con 0x7f5db40aed90 2021-11-21T22:49:06.604 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db8cb8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b2880 con 0x7f5db40af360 2021-11-21T22:49:06.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db259c700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f5db40b24a0 0x7f5db40ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38846/0 (socket says 172.21.2.116:38846) 2021-11-21T22:49:06.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db259c700 1 -- 172.21.2.116:0/3854345669 learned_addr learned my addr 172.21.2.116:0/3854345669 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:06.605 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753213949 0 0) 0x7f5db40b00d0 con 0x7f5db40b24a0 2021-11-21T22:49:06.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d98003260 con 0x7f5db40b24a0 2021-11-21T22:49:06.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2773342848 0 0) 0x7f5d98003260 con 0x7f5db40b24a0 2021-11-21T22:49:06.606 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db40af360 msgr2=0x7f5db413c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db40af360 0x7f5db413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 >> v2:172.21.2.117:3300/0 conn(0x7f5db40aed90 msgr2=0x7f5db40b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.607 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 --2- 172.21.2.116:0/3854345669 >> v2:172.21.2.117:3300/0 conn(0x7f5db40aed90 0x7f5db40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40b14a0 con 0x7f5db40b24a0 2021-11-21T22:49:06.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f5da4001660 con 0x7f5db40b24a0 2021-11-21T22:49:06.608 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 >> v1:172.21.2.116:6789/0 conn(0x7f5db40b24a0 legacy=0x7f5db40ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d980037d0 0x7f5d98003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.609 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5d98004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db40af360 0x7f5d98007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40adba0 con 0x7f5d98004500 2021-11-21T22:49:06.610 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b2880 con 0x7f5d980037d0 2021-11-21T22:49:06.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98005300 con 0x7f5db40af360 2021-11-21T22:49:06.611 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db1d9b700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5d98004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.612 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db1d9b700 1 -- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db40af360 msgr2=0x7f5d98007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db1d9b700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db40af360 0x7f5d98007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.613 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db1d9b700 1 -- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d980037d0 msgr2=0x7f5d98003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db1d9b700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5d980037d0 0x7f5d98003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.614 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.587+0000 7f5db1d9b700 1 -- 172.21.2.116:0/3854345669 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d98003260 con 0x7f5d98004500 2021-11-21T22:49:06.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db1d9b700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5d98004980 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f5da8002f70 tx=0x7f5da8006ca0).ready entity=mon.0 client_cookie=27b9b6840414fa34 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.615 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5da8012410 con 0x7f5d98004500 2021-11-21T22:49:06.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da8012570 con 0x7f5d98004500 2021-11-21T22:49:06.616 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db159a700 1 -- 172.21.2.116:0/3854345669 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5da80128d0 con 0x7f5d98004500 2021-11-21T22:49:06.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/3854345669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 msgr2=0x7f5d98004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.617 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 --2- 172.21.2.116:0/3854345669 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5d98004980 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f5da8002f70 tx=0x7f5da8006ca0).stop 2021-11-21T22:49:06.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db2d9d700 1 -- 172.21.2.116:0/3854345669 reap_dead start 2021-11-21T22:49:06.618 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/3854345669 shutdown_connections 2021-11-21T22:49:06.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/3854345669 >> 172.21.2.116:0/3854345669 conn(0x7f5db405fd00 msgr2=0x7f5db40600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.619 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/3854345669 shutdown_connections 2021-11-21T22:49:06.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/3854345669 wait complete. 2021-11-21T22:49:06.620 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 Processor -- start 2021-11-21T22:49:06.621 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- start start 2021-11-21T22:49:06.622 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5db4139300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5db40646c0 0x7f5db414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db4135f80 0x7f5db4136400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.623 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41368f0 con 0x7f5d98004500 2021-11-21T22:49:06.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4140420 con 0x7f5db40646c0 2021-11-21T22:49:06.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41519b0 con 0x7f5db4135f80 2021-11-21T22:49:06.624 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5db4139300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5db4139300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55134/0 (socket says 172.21.2.116:55134) 2021-11-21T22:49:06.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 -- 172.21.2.116:0/66428794 learned_addr learned my addr 172.21.2.116:0/66428794 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.625 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 -- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db4135f80 msgr2=0x7f5db4136400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f5db4135f80 0x7f5db4136400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 -- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5db40646c0 msgr2=0x7f5db414d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.626 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f5db40646c0 0x7f5db414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 -- 172.21.2.116:0/66428794 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4064220 con 0x7f5d98004500 2021-11-21T22:49:06.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db259c700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5db4139300 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 rx=0x7f5da4001370 tx=0x7f5da4001ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.627 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5da4007170 con 0x7f5d98004500 2021-11-21T22:49:06.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40072d0 con 0x7f5d98004500 2021-11-21T22:49:06.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db41516d0 con 0x7f5d98004500 2021-11-21T22:49:06.628 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5da4004920 con 0x7f5d98004500 2021-11-21T22:49:06.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4139c00 con 0x7f5d98004500 2021-11-21T22:49:06.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5db40640a0 con 0x7f5d98004500 2021-11-21T22:49:06.629 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f5da4004a80 con 0x7f5d98004500 2021-11-21T22:49:06.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d90053690 0x7f5d90055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5da406bbb0 con 0x7f5d98004500 2021-11-21T22:49:06.630 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f5d90059e10 0x7f5d9005c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 --> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d9005c7f0 con 0x7f5d90059e10 2021-11-21T22:49:06.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.591+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5da406bf10 con 0x7f5d98004500 2021-11-21T22:49:06.631 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.595+0000 7f5db2d9d700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f5d90059e10 0x7f5d9005c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.595+0000 7f5db1d9b700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d90053690 0x7f5d90055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.595+0000 7f5db2d9d700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f5d90059e10 0x7f5d9005c230 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.632 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.595+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== osd.2 v2:172.21.2.116:6816/14709 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f5d9005c7f0 con 0x7f5d90059e10 2021-11-21T22:49:06.633 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.595+0000 7f5db1d9b700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d90053690 0x7f5d90055b50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5da8002f70 tx=0x7f5da8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 --> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5db414dde0 con 0x7f5d90059e10 2021-11-21T22:49:06.634 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5da37fe700 1 -- 172.21.2.116:0/66428794 <== osd.2 v2:172.21.2.116:6816/14709 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5db414dde0 con 0x7f5d90059e10 2021-11-21T22:49:06.635 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:06.635 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:06.635 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:06.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f5d90059e10 msgr2=0x7f5d9005c230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.637 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:6816/14709,v1:172.21.2.116:6817/14709] conn(0x7f5d90059e10 0x7f5d9005c230 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d90053690 msgr2=0x7f5d90055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f5d90053690 0x7f5d90055b50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5da8002f70 tx=0x7f5da8015000).stop 2021-11-21T22:49:06.638 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 msgr2=0x7f5db4139300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 --2- 172.21.2.116:0/66428794 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f5d98004500 0x7f5db4139300 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 rx=0x7f5da4001370 tx=0x7f5da4001ae0).stop 2021-11-21T22:49:06.639 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db2d9d700 1 -- 172.21.2.116:0/66428794 reap_dead start 2021-11-21T22:49:06.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 shutdown_connections 2021-11-21T22:49:06.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 >> 172.21.2.116:0/66428794 conn(0x7f5db405fd00 msgr2=0x7f5db4069920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.640 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 shutdown_connections 2021-11-21T22:49:06.641 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.603+0000 7f5db8cb8700 1 -- 172.21.2.116:0/66428794 wait complete. 2021-11-21T22:49:06.641 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2021-11-21T22:49:06.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 Processor -- start 2021-11-21T22:49:06.716 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- start start 2021-11-21T22:49:06.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa59c0ac2f0 0x7fa59c0ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.717 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fa59c0ae1c0 0x7fa59c142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c1433b0 con 0x7fa59c1426c0 2021-11-21T22:49:06.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c143110 con 0x7fa59c0ac2f0 2021-11-21T22:49:06.718 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c0ada80 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa59c0ac2f0 0x7fa59c0ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.719 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fa59c0ac2f0 0x7fa59c0ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55148/0 (socket says 172.21.2.116:55148) 2021-11-21T22:49:06.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 -- 172.21.2.116:0/3514090373 learned_addr learned my addr 172.21.2.116:0/3514090373 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 -- 172.21.2.116:0/3514090373 >> v2:172.21.2.117:3300/0 conn(0x7fa59c0ae1c0 msgr2=0x7fa59c142aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:06.720 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 --2- 172.21.2.116:0/3514090373 >> v2:172.21.2.117:3300/0 conn(0x7fa59c0ae1c0 0x7fa59c142aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 -- 172.21.2.116:0/3514090373 >> v1:172.21.2.117:6789/0 conn(0x7fa59c1426c0 legacy=0x7fa59c0af850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:06.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 -- 172.21.2.116:0/3514090373 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c0add20 con 0x7fa59c0ac2f0 2021-11-21T22:49:06.721 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59affd700 1 --2- 172.21.2.116:0/3514090373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ac2f0 0x7fa59c0ac800 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7fa59c062500 tx=0x7fa58c006e40).ready entity=mon.0 client_cookie=2a388ba223ccf16f server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59a7fc700 1 -- 172.21.2.116:0/3514090373 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58c00bcb0 con 0x7fa59c0ac2f0 2021-11-21T22:49:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59a7fc700 1 -- 172.21.2.116:0/3514090373 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c000e90 con 0x7fa59c0ac2f0 2021-11-21T22:49:06.722 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa59a7fc700 1 -- 172.21.2.116:0/3514090373 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58c0153f0 con 0x7fa59c0ac2f0 2021-11-21T22:49:06.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/3514090373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ac2f0 msgr2=0x7fa59c0ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/3514090373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ac2f0 0x7fa59c0ac800 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7fa59c062500 tx=0x7fa58c006e40).stop 2021-11-21T22:49:06.726 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/3514090373 shutdown_connections 2021-11-21T22:49:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/3514090373 >> v2:172.21.2.117:3300/0 conn(0x7fa59c0ae1c0 0x7fa59c142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/3514090373 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ac2f0 0x7fa59c0ac800 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/3514090373 >> 172.21.2.116:0/3514090373 conn(0x7fa59c05bd20 msgr2=0x7fa59c05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/3514090373 shutdown_connections 2021-11-21T22:49:06.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.707+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/3514090373 wait complete. 2021-11-21T22:49:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 Processor -- start 2021-11-21T22:49:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 -- start start 2021-11-21T22:49:06.729 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa59c0ac2f0 0x7fa59c14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ae1c0 0x7fa59c14f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.730 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa59c1426c0 0x7fa59c1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c13f840 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c13cba0 con 0x7fa59c0ac2f0 2021-11-21T22:49:06.731 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1458e0 con 0x7fa59c1426c0 2021-11-21T22:49:06.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ae1c0 0x7fa59c14f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ae1c0 0x7fa59c14f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55150/0 (socket says 172.21.2.116:55150) 2021-11-21T22:49:06.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 -- 172.21.2.116:0/909339795 learned_addr learned my addr 172.21.2.116:0/909339795 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 -- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa59c1426c0 msgr2=0x7fa59c1510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa59c1426c0 0x7fa59c1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 -- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa59c0ac2f0 msgr2=0x7fa59c14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa59c0ac2f0 0x7fa59c14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 -- 172.21.2.116:0/909339795 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c00d4d0 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59affd700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ae1c0 0x7fa59c14f100 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7fa59c062500 tx=0x7fa58c018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa598ff9700 1 -- 172.21.2.116:0/909339795 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58c009070 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa598ff9700 1 -- 172.21.2.116:0/909339795 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c015c70 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c13cdd0 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa598ff9700 1 -- 172.21.2.116:0/909339795 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58c00be60 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c13ec80 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c0573d0 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa598ff9700 1 -- 172.21.2.116:0/909339795 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fa58c01e3f0 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa598ff9700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5800535b0 0x7fa580055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa598ff9700 1 -- 172.21.2.116:0/909339795 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa58c06c6c0 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.711+0000 7fa59b7fe700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5800535b0 0x7fa580055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.715+0000 7fa59b7fe700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5800535b0 0x7fa580055a70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fa584000ed0 tx=0x7fa584006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.715+0000 7fa598ff9700 1 -- 172.21.2.116:0/909339795 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa58c043b50 con 0x7fa59c0ae1c0 2021-11-21T22:49:06.804 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.795+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa59c13f140 con 0x7fa5800535b0 2021-11-21T22:49:06.805 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa598ff9700 1 -- 172.21.2.116:0/909339795 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa584012450 con 0x7fa5800535b0 2021-11-21T22:49:06.809 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5800535b0 msgr2=0x7fa580055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5800535b0 0x7fa580055a70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fa584000ed0 tx=0x7fa584006930).stop 2021-11-21T22:49:06.810 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ae1c0 msgr2=0x7fa59c14f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ae1c0 0x7fa59c14f100 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7fa59c062500 tx=0x7fa58c018000).stop 2021-11-21T22:49:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 shutdown_connections 2021-11-21T22:49:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fa59c1426c0 0x7fa59c1510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.811 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fa5800535b0 0x7fa580055a70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fa59c0ae1c0 0x7fa59c14f100 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 --2- 172.21.2.116:0/909339795 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fa59c0ac2f0 0x7fa59c14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.812 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 >> 172.21.2.116:0/909339795 conn(0x7fa59c05bd20 msgr2=0x7fa59c1405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 shutdown_connections 2021-11-21T22:49:06.813 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.799+0000 7fa5a1cd2700 1 -- 172.21.2.116:0/909339795 wait complete. 2021-11-21T22:49:06.814 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 2 to deep-scrub 2021-11-21T22:49:06.817 INFO:tasks.ceph:Scrubbing osd.3 2021-11-21T22:49:06.818 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:06.871 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44f0800700 1 Processor -- start 2021-11-21T22:49:06.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44f0800700 1 -- start start 2021-11-21T22:49:06.873 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44f0800700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e8147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44f0800700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f44e8143e60 0x7f44e81442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.874 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44f0800700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e80b1940 con 0x7f44e80a8670 2021-11-21T22:49:06.875 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44f0800700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80b15a0 con 0x7f44e8143e60 2021-11-21T22:49:06.876 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44f0800700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80b1b70 con 0x7f44e80b11c0 2021-11-21T22:49:06.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44eed9d700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f44e8143e60 0x7f44e81442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44edd9b700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7f44e80a8670 0x7f44e8147720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:37794/0 (socket says 172.21.2.116:37794) 2021-11-21T22:49:06.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44edd9b700 1 -- 172.21.2.116:0/913838723 learned_addr learned my addr 172.21.2.116:0/913838723 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:06.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.863+0000 7f44ee59c700 1 --2- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e8147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 -- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 msgr2=0x7f44e8147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 --2- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e8147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 -- 172.21.2.116:0/913838723 >> v1:172.21.2.117:6789/0 conn(0x7f44e80a8670 legacy=0x7f44e8147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 -- 172.21.2.116:0/913838723 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80b1e10 con 0x7f44e8143e60 2021-11-21T22:49:06.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 --2- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e8143e60 0x7f44e81442f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f44e40032f0 tx=0x7f44e40091d0).ready entity=mon.1 client_cookie=57991e082ca26869 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44ed59a700 1 -- 172.21.2.116:0/913838723 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e400b6c0 con 0x7f44e8143e60 2021-11-21T22:49:06.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44ed59a700 1 -- 172.21.2.116:0/913838723 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e400b820 con 0x7f44e8143e60 2021-11-21T22:49:06.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44ed59a700 1 -- 172.21.2.116:0/913838723 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e400b990 con 0x7f44e8143e60 2021-11-21T22:49:06.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e8143e60 msgr2=0x7f44e81442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 --2- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e8143e60 0x7f44e81442f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f44e40032f0 tx=0x7f44e40091d0).stop 2021-11-21T22:49:06.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- 172.21.2.116:0/913838723 shutdown_connections 2021-11-21T22:49:06.889 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 --2- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e8143e60 0x7f44e81442f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.890 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 --2- 172.21.2.116:0/913838723 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e8147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- 172.21.2.116:0/913838723 >> 172.21.2.116:0/913838723 conn(0x7f44e805bd20 msgr2=0x7f44e805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- 172.21.2.116:0/913838723 shutdown_connections 2021-11-21T22:49:06.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- 172.21.2.116:0/913838723 wait complete. 2021-11-21T22:49:06.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 Processor -- start 2021-11-21T22:49:06.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- start start 2021-11-21T22:49:06.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e80a8670 0x7f44e8141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e80b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44e8143e60 0x7f44e80b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80b5520 con 0x7f44e8143e60 2021-11-21T22:49:06.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e814a8e0 con 0x7f44e80a8670 2021-11-21T22:49:06.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80b3610 con 0x7f44e80b11c0 2021-11-21T22:49:06.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44e8143e60 0x7f44e80b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44e8143e60 0x7f44e80b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55168/0 (socket says 172.21.2.116:55168) 2021-11-21T22:49:06.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 -- 172.21.2.116:0/2990656365 learned_addr learned my addr 172.21.2.116:0/2990656365 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:06.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44ee59c700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e80a8670 0x7f44e8141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44edd9b700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e80b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 -- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 msgr2=0x7f44e80b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e80b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 -- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e80a8670 msgr2=0x7f44e8141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f44e80a8670 0x7f44e8141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 -- 172.21.2.116:0/2990656365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009470 con 0x7f44e8143e60 2021-11-21T22:49:06.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44edd9b700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f44e80b11c0 0x7f44e80b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:49:06.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44eed9d700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44e8143e60 0x7f44e80b46d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 rx=0x7f44e40032f0 tx=0x7f44e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e400b6c0 con 0x7f44e8143e60 2021-11-21T22:49:06.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e40203f0 con 0x7f44e8143e60 2021-11-21T22:49:06.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e80b3d40 con 0x7f44e8143e60 2021-11-21T22:49:06.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e814ae00 con 0x7f44e8143e60 2021-11-21T22:49:06.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.867+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e4020750 con 0x7f44e8143e60 2021-11-21T22:49:06.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f44e80b3490 con 0x7f44e8143e60 2021-11-21T22:49:06.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f44e401e070 con 0x7f44e8143e60 2021-11-21T22:49:06.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44e37fe700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d0053660 0x7f44d0055b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f44e406cb90 con 0x7f44e8143e60 2021-11-21T22:49:06.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44e37fe700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f44d0059de0 0x7f44d005c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 --> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44d005c7c0 con 0x7f44d0059de0 2021-11-21T22:49:06.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44e406cef0 con 0x7f44e8143e60 2021-11-21T22:49:06.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44edd9b700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f44d0059de0 0x7f44d005c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44edd9b700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f44d0059de0 0x7f44d005c200 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44ee59c700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d0053660 0x7f44d0055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== osd.3 v2:172.21.2.116:6824/14710 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f44d005c7c0 con 0x7f44d0059de0 2021-11-21T22:49:06.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.871+0000 7f44ee59c700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d0053660 0x7f44d0055b20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f44dc00b000 tx=0x7f44dc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:06.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 --> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f44e80b4eb0 con 0x7f44d0059de0 2021-11-21T22:49:06.907 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44e37fe700 1 -- 172.21.2.116:0/2990656365 <== osd.3 v2:172.21.2.116:6824/14710 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f44e80b4eb0 con 0x7f44d0059de0 2021-11-21T22:49:06.908 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:06.908 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:06.909 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:06.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f44d0059de0 msgr2=0x7f44d005c200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:6824/14710,v1:172.21.2.116:6825/14710] conn(0x7f44d0059de0 0x7f44d005c200 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d0053660 msgr2=0x7f44d0055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f44d0053660 0x7f44d0055b20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f44dc00b000 tx=0x7f44dc008040).stop 2021-11-21T22:49:06.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44e8143e60 msgr2=0x7f44e80b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 --2- 172.21.2.116:0/2990656365 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f44e8143e60 0x7f44e80b46d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 rx=0x7f44e40032f0 tx=0x7f44e4016000).stop 2021-11-21T22:49:06.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44eed9d700 1 -- 172.21.2.116:0/2990656365 reap_dead start 2021-11-21T22:49:06.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 shutdown_connections 2021-11-21T22:49:06.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 >> 172.21.2.116:0/2990656365 conn(0x7f44e805bd20 msgr2=0x7f44e80a6760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:06.913 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 shutdown_connections 2021-11-21T22:49:06.914 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.879+0000 7f44f0800700 1 -- 172.21.2.116:0/2990656365 wait complete. 2021-11-21T22:49:06.914 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2021-11-21T22:49:06.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.975+0000 7f46e85a3700 1 Processor -- start 2021-11-21T22:49:06.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.975+0000 7f46e85a3700 1 -- start start 2021-11-21T22:49:06.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.975+0000 7f46e85a3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e0144810 0x7f46e01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.989 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.975+0000 7f46e85a3700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f46e00ac600 0x7f46e00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.975+0000 7f46e85a3700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e0147e80 con 0x7f46e00abce0 2021-11-21T22:49:06.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.975+0000 7f46e85a3700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e00b1490 con 0x7f46e00ac600 2021-11-21T22:49:06.990 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.975+0000 7f46e85a3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00b1fd0 con 0x7f46e0144810 2021-11-21T22:49:06.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e5b3e700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f46e00abce0 0x7f46e00b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38898/0 (socket says 172.21.2.116:38898) 2021-11-21T22:49:06.991 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e5b3e700 1 -- 172.21.2.116:0/3048463597 learned_addr learned my addr 172.21.2.116:0/3048463597 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:06.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414387308 0 0) 0x7f46e0147e80 con 0x7f46e00abce0 2021-11-21T22:49:06.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c8003260 con 0x7f46e00abce0 2021-11-21T22:49:06.992 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e0144810 0x7f46e01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 --2- 172.21.2.116:0/3048463597 >> v2:172.21.2.117:3300/0 conn(0x7f46e00ac600 0x7f46e00aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033865486 0 0) 0x7f46c8003260 con 0x7f46e00abce0 2021-11-21T22:49:06.993 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e0144810 msgr2=0x7f46e01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e0144810 0x7f46e01478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 >> v2:172.21.2.117:3300/0 conn(0x7f46e00ac600 msgr2=0x7f46e00aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.994 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 --2- 172.21.2.116:0/3048463597 >> v2:172.21.2.117:3300/0 conn(0x7f46e00ac600 0x7f46e00aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:06.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00af5a0 con 0x7f46e00abce0 2021-11-21T22:49:06.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f46d0001660 con 0x7f46e00abce0 2021-11-21T22:49:06.995 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 >> v1:172.21.2.116:6789/0 conn(0x7f46e00abce0 legacy=0x7f46e00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:06.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46c80037d0 0x7f46c8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46c8004500 0x7f46c8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.996 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46c8004f90 0x7f46c800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:06.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00b1490 con 0x7f46c80037d0 2021-11-21T22:49:06.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00b1fd0 con 0x7f46c8004f90 2021-11-21T22:49:06.997 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c800be20 con 0x7f46c8004500 2021-11-21T22:49:06.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e0144810 0x7f46e01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-21T22:49:06.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46c80037d0 0x7f46c8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:06.998 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 -- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46c8004500 msgr2=0x7f46c8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46c8004500 0x7f46c8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:06.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 -- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46c8004f90 msgr2=0x7f46c800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:06.999 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46c8004f90 0x7f46c800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 -- 172.21.2.116:0/3048463597 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8003260 con 0x7f46c80037d0 2021-11-21T22:49:07.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e633f700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46c80037d0 0x7f46c8003bf0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7f46dc007000 tx=0x7f46dc008040).ready entity=mon.0 client_cookie=1c97dce779f0b066 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.000 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46dc012430 con 0x7f46c80037d0 2021-11-21T22:49:07.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc00c030 con 0x7f46c80037d0 2021-11-21T22:49:07.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e533d700 1 -- 172.21.2.116:0/3048463597 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46dc012780 con 0x7f46c80037d0 2021-11-21T22:49:07.001 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- 172.21.2.116:0/3048463597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46c80037d0 msgr2=0x7f46c8003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 --2- 172.21.2.116:0/3048463597 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46c80037d0 0x7f46c8003bf0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7f46dc007000 tx=0x7f46dc008040).stop 2021-11-21T22:49:07.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 -- 172.21.2.116:0/3048463597 reap_dead start 2021-11-21T22:49:07.002 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- 172.21.2.116:0/3048463597 shutdown_connections 2021-11-21T22:49:07.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- 172.21.2.116:0/3048463597 >> 172.21.2.116:0/3048463597 conn(0x7f46e005bd20 msgr2=0x7f46e005c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- 172.21.2.116:0/3048463597 shutdown_connections 2021-11-21T22:49:07.003 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- 172.21.2.116:0/3048463597 wait complete. 2021-11-21T22:49:07.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 Processor -- start 2021-11-21T22:49:07.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- start start 2021-11-21T22:49:07.004 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e01439f0 0x7f46e015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46e0063b80 0x7f46e015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46e0066b70 0x7f46e0067020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.005 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e015aba0 con 0x7f46e0066b70 2021-11-21T22:49:07.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0143dd0 con 0x7f46e0063b80 2021-11-21T22:49:07.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0064100 con 0x7f46e01439f0 2021-11-21T22:49:07.006 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46e0066b70 0x7f46e0067020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46e0066b70 0x7f46e0067020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55188/0 (socket says 172.21.2.116:55188) 2021-11-21T22:49:07.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 -- 172.21.2.116:0/2885683444 learned_addr learned my addr 172.21.2.116:0/2885683444 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.007 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 -- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e01439f0 msgr2=0x7f46e015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e01439f0 0x7f46e015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 -- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46e0063b80 msgr2=0x7f46e015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.008 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46e0063b80 0x7f46e015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 -- 172.21.2.116:0/2885683444 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00643a0 con 0x7f46e0066b70 2021-11-21T22:49:07.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e6b40700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46e0066b70 0x7f46e0067020 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f46d8000b60 tx=0x7f46d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.009 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46cf7fe700 1 -- 172.21.2.116:0/2885683444 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46d800c070 con 0x7f46e0066b70 2021-11-21T22:49:07.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46cf7fe700 1 -- 172.21.2.116:0/2885683444 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d80135e0 con 0x7f46e0066b70 2021-11-21T22:49:07.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e0064640 con 0x7f46e0066b70 2021-11-21T22:49:07.010 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46cf7fe700 1 -- 172.21.2.116:0/2885683444 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46d8013940 con 0x7f46e0066b70 2021-11-21T22:49:07.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.979+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e0067510 con 0x7f46e0066b70 2021-11-21T22:49:07.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.983+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e0156850 con 0x7f46e0066b70 2021-11-21T22:49:07.011 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.983+0000 7f46cf7fe700 1 -- 172.21.2.116:0/2885683444 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f46d8013aa0 con 0x7f46e0066b70 2021-11-21T22:49:07.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.983+0000 7f46cf7fe700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46c00536d0 0x7f46c0055b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.012 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.983+0000 7f46cf7fe700 1 -- 172.21.2.116:0/2885683444 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f46d806aab0 con 0x7f46e0066b70 2021-11-21T22:49:07.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.983+0000 7f46e633f700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46c00536d0 0x7f46c0055b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.013 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.983+0000 7f46e633f700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46c00536d0 0x7f46c0055b90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f46dc007000 tx=0x7f46dc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.014 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:06.983+0000 7f46cf7fe700 1 -- 172.21.2.116:0/2885683444 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46d8041e90 con 0x7f46e0066b70 2021-11-21T22:49:07.075 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.067+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f46e0159f20 con 0x7f46c00536d0 2021-11-21T22:49:07.076 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.067+0000 7f46cf7fe700 1 -- 172.21.2.116:0/2885683444 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f46dc012430 con 0x7f46c00536d0 2021-11-21T22:49:07.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46c00536d0 msgr2=0x7f46c0055b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.078 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46c00536d0 0x7f46c0055b90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f46dc007000 tx=0x7f46dc008040).stop 2021-11-21T22:49:07.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46e0066b70 msgr2=0x7f46e0067020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46e0066b70 0x7f46e0067020 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f46d8000b60 tx=0x7f46d8008040).stop 2021-11-21T22:49:07.079 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 shutdown_connections 2021-11-21T22:49:07.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f46c00536d0 0x7f46c0055b90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f46e0066b70 0x7f46e0067020 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.080 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f46e0063b80 0x7f46e015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 --2- 172.21.2.116:0/2885683444 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f46e01439f0 0x7f46e015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 >> 172.21.2.116:0/2885683444 conn(0x7f46e005bd20 msgr2=0x7f46e0148570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.081 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 shutdown_connections 2021-11-21T22:49:07.082 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.071+0000 7f46e85a3700 1 -- 172.21.2.116:0/2885683444 wait complete. 2021-11-21T22:49:07.082 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 3 to deep-scrub 2021-11-21T22:49:07.084 INFO:tasks.ceph:Scrubbing osd.4 2021-11-21T22:49:07.085 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:07.138 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 Processor -- start 2021-11-21T22:49:07.139 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- start start 2021-11-21T22:49:07.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f6b9c0b24a0 0x7f6b9c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.140 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6b9c0af360 0x7f6b9c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b9c0aeb70 0x7f6b9c069e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0adba0 con 0x7f6b9c0af360 2021-11-21T22:49:07.141 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0af7a0 con 0x7f6b9c0b24a0 2021-11-21T22:49:07.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0b00d0 con 0x7f6b9c0aeb70 2021-11-21T22:49:07.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6b9c0af360 0x7f6b9c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.142 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6b9c0af360 0x7f6b9c0ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55196/0 (socket says 172.21.2.116:55196) 2021-11-21T22:49:07.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 -- 172.21.2.116:0/3304447629 learned_addr learned my addr 172.21.2.116:0/3304447629 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 -- 172.21.2.116:0/3304447629 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b9c0aeb70 msgr2=0x7f6b9c069e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.143 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 --2- 172.21.2.116:0/3304447629 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b9c0aeb70 0x7f6b9c069e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 -- 172.21.2.116:0/3304447629 >> v2:172.21.2.117:3300/0 conn(0x7f6b9c0b24a0 msgr2=0x7f6b9c0b1a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 --2- 172.21.2.116:0/3304447629 >> v2:172.21.2.117:3300/0 conn(0x7f6b9c0b24a0 0x7f6b9c0b1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.144 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 -- 172.21.2.116:0/3304447629 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0b2880 con 0x7f6b9c0af360 2021-11-21T22:49:07.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9bfff700 1 --2- 172.21.2.116:0/3304447629 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c0ae680 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7f6b8c002f70 tx=0x7f6b8c006eb0).ready entity=mon.0 client_cookie=8fc47cf39abf17e6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9b7fe700 1 -- 172.21.2.116:0/3304447629 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8c00d570 con 0x7f6b9c0af360 2021-11-21T22:49:07.145 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9b7fe700 1 -- 172.21.2.116:0/3304447629 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c000e90 con 0x7f6b9c0af360 2021-11-21T22:49:07.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6b9b7fe700 1 -- 172.21.2.116:0/3304447629 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8c00d900 con 0x7f6b9c0af360 2021-11-21T22:49:07.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/3304447629 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 msgr2=0x7f6b9c0ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.146 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 --2- 172.21.2.116:0/3304447629 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c0ae680 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7f6b8c002f70 tx=0x7f6b8c006eb0).stop 2021-11-21T22:49:07.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/3304447629 shutdown_connections 2021-11-21T22:49:07.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 --2- 172.21.2.116:0/3304447629 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b9c0aeb70 0x7f6b9c069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.147 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 --2- 172.21.2.116:0/3304447629 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c0ae680 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 --2- 172.21.2.116:0/3304447629 >> v2:172.21.2.117:3300/0 conn(0x7f6b9c0b24a0 0x7f6b9c0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/3304447629 >> 172.21.2.116:0/3304447629 conn(0x7f6b9c05fd00 msgr2=0x7f6b9c0600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.148 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/3304447629 shutdown_connections 2021-11-21T22:49:07.149 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.131+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/3304447629 wait complete. 2021-11-21T22:49:07.151 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 Processor -- start 2021-11-21T22:49:07.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 -- start start 2021-11-21T22:49:07.152 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b9c0aeb70 0x7f6b9c1306d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c13dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b9c0b24a0 0x7f6b9c066210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c13e390 con 0x7f6b9c0af360 2021-11-21T22:49:07.153 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c13f110 con 0x7f6b9c0aeb70 2021-11-21T22:49:07.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c13f3b0 con 0x7f6b9c0b24a0 2021-11-21T22:49:07.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c13dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.154 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c13dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55202/0 (socket says 172.21.2.116:55202) 2021-11-21T22:49:07.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 -- 172.21.2.116:0/2309618843 learned_addr learned my addr 172.21.2.116:0/2309618843 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.155 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 -- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b9c0b24a0 msgr2=0x7f6b9c066210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6b9c0b24a0 0x7f6b9c066210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 -- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b9c0aeb70 msgr2=0x7f6b9c1306d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.156 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6b9c0aeb70 0x7f6b9c1306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 -- 172.21.2.116:0/2309618843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c00b1e0 con 0x7f6b9c0af360 2021-11-21T22:49:07.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b9bfff700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c13dea0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7f6b8c002f70 tx=0x7f6b8c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.157 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8c011a80 con 0x7f6b9c0af360 2021-11-21T22:49:07.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c011be0 con 0x7f6b9c0af360 2021-11-21T22:49:07.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9c13f650 con 0x7f6b9c0af360 2021-11-21T22:49:07.158 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8c01e460 con 0x7f6b9c0af360 2021-11-21T22:49:07.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c066f50 con 0x7f6b9c0af360 2021-11-21T22:49:07.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b9c065f70 con 0x7f6b9c0af360 2021-11-21T22:49:07.159 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6b8c01a040 con 0x7f6b9c0af360 2021-11-21T22:49:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b80053610 0x7f6b80055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6b8c06ce50 con 0x7f6b9c0af360 2021-11-21T22:49:07.160 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f6b80059d90 0x7f6b8005c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 --> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b8005c770 con 0x7f6b80059d90 2021-11-21T22:49:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.135+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b8c06d230 con 0x7f6b9c0af360 2021-11-21T22:49:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.139+0000 7f6ba09eb700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b80053610 0x7f6b80055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.161 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.139+0000 7f6ba11ec700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f6b80059d90 0x7f6b8005c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.139+0000 7f6ba11ec700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f6b80059d90 0x7f6b8005c1b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.162 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.139+0000 7f6ba09eb700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b80053610 0x7f6b80055ad0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f6b84000ed0 tx=0x7f6b84006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.139+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== osd.4 v2:172.21.2.117:6800/14956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f6b8005c770 con 0x7f6b80059d90 2021-11-21T22:49:07.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 --> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6b9c0669c0 con 0x7f6b80059d90 2021-11-21T22:49:07.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6b99ffb700 1 -- 172.21.2.116:0/2309618843 <== osd.4 v2:172.21.2.117:6800/14956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6b9c0669c0 con 0x7f6b80059d90 2021-11-21T22:49:07.164 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:07.165 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:07.165 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:07.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f6b80059d90 msgr2=0x7f6b8005c1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6800/14956,v1:172.21.2.117:6801/14956] conn(0x7f6b80059d90 0x7f6b8005c1b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.167 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b80053610 msgr2=0x7f6b80055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6b80053610 0x7f6b80055ad0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f6b84000ed0 tx=0x7f6b84006930).stop 2021-11-21T22:49:07.168 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 msgr2=0x7f6b9c13dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 --2- 172.21.2.116:0/2309618843 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6b9c0af360 0x7f6b9c13dea0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7f6b8c002f70 tx=0x7f6b8c015000).stop 2021-11-21T22:49:07.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba11ec700 1 -- 172.21.2.116:0/2309618843 reap_dead start 2021-11-21T22:49:07.169 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 shutdown_connections 2021-11-21T22:49:07.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.147+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 >> 172.21.2.116:0/2309618843 conn(0x7f6b9c05fd00 msgr2=0x7f6b9c13c730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.151+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 shutdown_connections 2021-11-21T22:49:07.170 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.151+0000 7f6ba2c4f700 1 -- 172.21.2.116:0/2309618843 wait complete. 2021-11-21T22:49:07.171 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2021-11-21T22:49:07.229 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 Processor -- start 2021-11-21T22:49:07.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- start start 2021-11-21T22:49:07.230 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fbe4c1426e0 0x7fbe4c142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbe4c062e60 0x7fbe4c0af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c0ad470 0x7fbe4c0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.231 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c0b0f60 con 0x7fbe4c062e60 2021-11-21T22:49:07.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c063470 con 0x7fbe4c1426e0 2021-11-21T22:49:07.232 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c143050 con 0x7fbe4c0ad470 2021-11-21T22:49:07.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbe4c062e60 0x7fbe4c0af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fbe4c062e60 0x7fbe4c0af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55210/0 (socket says 172.21.2.116:55210) 2021-11-21T22:49:07.236 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 -- 172.21.2.116:0/823015589 learned_addr learned my addr 172.21.2.116:0/823015589 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 -- 172.21.2.116:0/823015589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c0ad470 msgr2=0x7fbe4c0ad900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 --2- 172.21.2.116:0/823015589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c0ad470 0x7fbe4c0ad900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.237 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 -- 172.21.2.116:0/823015589 >> v2:172.21.2.117:3300/0 conn(0x7fbe4c1426e0 msgr2=0x7fbe4c142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 --2- 172.21.2.116:0/823015589 >> v2:172.21.2.117:3300/0 conn(0x7fbe4c1426e0 0x7fbe4c142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 -- 172.21.2.116:0/823015589 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c0b1e00 con 0x7fbe4c062e60 2021-11-21T22:49:07.238 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4affd700 1 --2- 172.21.2.116:0/823015589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c062e60 0x7fbe4c0af3d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 rx=0x7fbe34002f70 tx=0x7fbe34006e40).ready entity=mon.0 client_cookie=c71dc7cd4bdd04fd server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4a7fc700 1 -- 172.21.2.116:0/823015589 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3400bb00 con 0x7fbe4c062e60 2021-11-21T22:49:07.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4a7fc700 1 -- 172.21.2.116:0/823015589 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34000e90 con 0x7fbe4c062e60 2021-11-21T22:49:07.239 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4a7fc700 1 -- 172.21.2.116:0/823015589 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3400be90 con 0x7fbe4c062e60 2021-11-21T22:49:07.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- 172.21.2.116:0/823015589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c062e60 msgr2=0x7fbe4c0af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- 172.21.2.116:0/823015589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c062e60 0x7fbe4c0af3d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 rx=0x7fbe34002f70 tx=0x7fbe34006e40).stop 2021-11-21T22:49:07.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- 172.21.2.116:0/823015589 shutdown_connections 2021-11-21T22:49:07.240 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- 172.21.2.116:0/823015589 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c0ad470 0x7fbe4c0ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- 172.21.2.116:0/823015589 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c062e60 0x7fbe4c0af3d0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.241 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- 172.21.2.116:0/823015589 >> v2:172.21.2.117:3300/0 conn(0x7fbe4c1426e0 0x7fbe4c142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- 172.21.2.116:0/823015589 >> 172.21.2.116:0/823015589 conn(0x7fbe4c0b5730 msgr2=0x7fbe4c0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- 172.21.2.116:0/823015589 shutdown_connections 2021-11-21T22:49:07.242 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- 172.21.2.116:0/823015589 wait complete. 2021-11-21T22:49:07.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 Processor -- start 2021-11-21T22:49:07.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- start start 2021-11-21T22:49:07.243 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c0ad470 0x7fbe4c141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c1426e0 0x7fbe4c14dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe4c13f750 0x7fbe4c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.244 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c13e190 con 0x7fbe4c0ad470 2021-11-21T22:49:07.245 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c13fcd0 con 0x7fbe4c13f750 2021-11-21T22:49:07.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe51eec700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c066000 con 0x7fbe4c1426e0 2021-11-21T22:49:07.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c0ad470 0x7fbe4c141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.246 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4b7fe700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c0ad470 0x7fbe4c141b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55218/0 (socket says 172.21.2.116:55218) 2021-11-21T22:49:07.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.223+0000 7fbe4b7fe700 1 -- 172.21.2.116:0/1641210323 learned_addr learned my addr 172.21.2.116:0/1641210323 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4b7fe700 1 -- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c1426e0 msgr2=0x7fbe4c14dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4b7fe700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c1426e0 0x7fbe4c14dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.247 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4b7fe700 1 -- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe4c13f750 msgr2=0x7fbe4c14dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4b7fe700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe4c13f750 0x7fbe4c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4b7fe700 1 -- 172.21.2.116:0/1641210323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3400d4d0 con 0x7fbe4c0ad470 2021-11-21T22:49:07.248 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4b7fe700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c0ad470 0x7fbe4c141b30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 rx=0x7fbe3c000ed0 tx=0x7fbe3c006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe48ff9700 1 -- 172.21.2.116:0/1641210323 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3c012410 con 0x7fbe4c0ad470 2021-11-21T22:49:07.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe48ff9700 1 -- 172.21.2.116:0/1641210323 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c012570 con 0x7fbe4c0ad470 2021-11-21T22:49:07.249 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c0662a0 con 0x7fbe4c0ad470 2021-11-21T22:49:07.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe48ff9700 1 -- 172.21.2.116:0/1641210323 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3c0128d0 con 0x7fbe4c0ad470 2021-11-21T22:49:07.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c14e4b0 con 0x7fbe4c0ad470 2021-11-21T22:49:07.250 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4c0b13c0 con 0x7fbe4c0ad470 2021-11-21T22:49:07.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe48ff9700 1 -- 172.21.2.116:0/1641210323 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fbe3c00f030 con 0x7fbe4c0ad470 2021-11-21T22:49:07.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe48ff9700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe300536a0 0x7fbe30055b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe48ff9700 1 -- 172.21.2.116:0/1641210323 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbe3c0462f0 con 0x7fbe4c0ad470 2021-11-21T22:49:07.252 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4affd700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe300536a0 0x7fbe30055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.227+0000 7fbe4affd700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe300536a0 0x7fbe30055b60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fbe34006fc0 tx=0x7fbe34006e40).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.253 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.231+0000 7fbe48ff9700 1 -- 172.21.2.116:0/1641210323 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbe3c042690 con 0x7fbe4c0ad470 2021-11-21T22:49:07.320 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.311+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe4c066740 con 0x7fbe300536a0 2021-11-21T22:49:07.321 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe48ff9700 1 -- 172.21.2.116:0/1641210323 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbe340069f0 con 0x7fbe300536a0 2021-11-21T22:49:07.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe300536a0 msgr2=0x7fbe30055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe300536a0 0x7fbe30055b60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fbe34006fc0 tx=0x7fbe34006e40).stop 2021-11-21T22:49:07.323 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c0ad470 msgr2=0x7fbe4c141b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c0ad470 0x7fbe4c141b30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 rx=0x7fbe3c000ed0 tx=0x7fbe3c006d70).stop 2021-11-21T22:49:07.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 shutdown_connections 2021-11-21T22:49:07.324 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fbe4c13f750 0x7fbe4c14dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fbe4c1426e0 0x7fbe4c14dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fbe300536a0 0x7fbe30055b60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.325 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 --2- 172.21.2.116:0/1641210323 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fbe4c0ad470 0x7fbe4c141b30 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 >> 172.21.2.116:0/1641210323 conn(0x7fbe4c0b5730 msgr2=0x7fbe4c069dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 shutdown_connections 2021-11-21T22:49:07.326 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.315+0000 7fbe51eec700 1 -- 172.21.2.116:0/1641210323 wait complete. 2021-11-21T22:49:07.327 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 4 to deep-scrub 2021-11-21T22:49:07.329 INFO:tasks.ceph:Scrubbing osd.5 2021-11-21T22:49:07.330 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:07.383 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f98470d2700 1 Processor -- start 2021-11-21T22:49:07.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f98470d2700 1 -- start start 2021-11-21T22:49:07.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f98470d2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400a8670 0x7f9840147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.384 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f98470d2700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400b1940 con 0x7f9840143e60 2021-11-21T22:49:07.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f98470d2700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400b15a0 con 0x7f98400b11c0 2021-11-21T22:49:07.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f98470d2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98400b1b70 con 0x7f98400a8670 2021-11-21T22:49:07.385 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f9840143e60 0x7f98401442f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:38946/0 (socket says 172.21.2.116:38946) 2021-11-21T22:49:07.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 -- 172.21.2.116:0/2145652253 learned_addr learned my addr 172.21.2.116:0/2145652253 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:07.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769357790 0 0) 0x7f98400b1940 con 0x7f9840143e60 2021-11-21T22:49:07.386 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9820003260 con 0x7f9840143e60 2021-11-21T22:49:07.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1452664048 0 0) 0x7f9820003260 con 0x7f9840143e60 2021-11-21T22:49:07.392 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400a8670 msgr2=0x7f9840147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:07.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400a8670 0x7f9840147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 >> v1:172.21.2.117:6789/0 conn(0x7f98400b11c0 legacy=0x7f9840147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:07.393 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400b1e10 con 0x7f9840143e60 2021-11-21T22:49:07.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f98380019e0 con 0x7f9840143e60 2021-11-21T22:49:07.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 >> v1:172.21.2.116:6789/0 conn(0x7f9840143e60 legacy=0x7f98401442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.394 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400a8670 0x7f9820003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9820003ef0 0x7f9820004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9820004bf0 0x7f982000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.395 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98400b1b70 con 0x7f9820004bf0 2021-11-21T22:49:07.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98200052f0 con 0x7f9820003ef0 2021-11-21T22:49:07.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982000bc40 con 0x7f98400a8670 2021-11-21T22:49:07.396 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9820004bf0 0x7f982000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 -- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400a8670 msgr2=0x7f9820003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:07.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f98400a8670 0x7f9820003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.397 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 -- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9820003ef0 msgr2=0x7f9820004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:07.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f9820003ef0 0x7f9820004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 -- 172.21.2.116:0/2145652253 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9820003260 con 0x7f9820004bf0 2021-11-21T22:49:07.398 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f984566f700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9820004bf0 0x7f982000b750 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 rx=0x7f98400b26d0 tx=0x7f98400b1fa0).ready entity=mon.0 client_cookie=6e8b99e2e41b5e41 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f983800cc00 con 0x7f9820004bf0 2021-11-21T22:49:07.399 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983800cd60 con 0x7f9820004bf0 2021-11-21T22:49:07.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.375+0000 7f983f7fe700 1 -- 172.21.2.116:0/2145652253 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9838005b80 con 0x7f9820004bf0 2021-11-21T22:49:07.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/2145652253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9820004bf0 msgr2=0x7f982000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.400 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 --2- 172.21.2.116:0/2145652253 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f9820004bf0 0x7f982000b750 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 rx=0x7f98400b26d0 tx=0x7f98400b1fa0).stop 2021-11-21T22:49:07.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 -- 172.21.2.116:0/2145652253 reap_dead start 2021-11-21T22:49:07.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/2145652253 shutdown_connections 2021-11-21T22:49:07.401 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/2145652253 >> 172.21.2.116:0/2145652253 conn(0x7f984005bd20 msgr2=0x7f984005c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/2145652253 shutdown_connections 2021-11-21T22:49:07.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/2145652253 wait complete. 2021-11-21T22:49:07.402 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 Processor -- start 2021-11-21T22:49:07.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- start start 2021-11-21T22:49:07.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9820004bf0 0x7f9840141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.403 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98400b4700 0x7f984015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401562d0 0x7f98400b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9840140f80 con 0x7f98401562d0 2021-11-21T22:49:07.404 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9840159860 con 0x7f98400b4700 2021-11-21T22:49:07.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984015a6f0 con 0x7f9820004bf0 2021-11-21T22:49:07.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401562d0 0x7f98400b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.405 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401562d0 0x7f98400b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55238/0 (socket says 172.21.2.116:55238) 2021-11-21T22:49:07.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 -- 172.21.2.116:0/3559081984 learned_addr learned my addr 172.21.2.116:0/3559081984 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 -- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9820004bf0 msgr2=0x7f9840141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.406 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f9820004bf0 0x7f9840141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 -- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98400b4700 msgr2=0x7f984015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f98400b4700 0x7f984015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.407 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 -- 172.21.2.116:0/3559081984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840140c80 con 0x7f98401562d0 2021-11-21T22:49:07.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f984566f700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401562d0 0x7f98400b3b20 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 rx=0x7f984015a880 tx=0x7f9838015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98380074d0 con 0x7f98401562d0 2021-11-21T22:49:07.408 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983800ced0 con 0x7f98401562d0 2021-11-21T22:49:07.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98401411b0 con 0x7f98401562d0 2021-11-21T22:49:07.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9838007820 con 0x7f98401562d0 2021-11-21T22:49:07.409 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9840159d60 con 0x7f98401562d0 2021-11-21T22:49:07.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9840159f60 con 0x7f98401562d0 2021-11-21T22:49:07.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f983802f070 con 0x7f98401562d0 2021-11-21T22:49:07.410 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9824053690 0x7f9824055b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f983806ce50 con 0x7f98401562d0 2021-11-21T22:49:07.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7f9824059e90 0x7f982405c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.411 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 --> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f982405c870 con 0x7f9824059e90 2021-11-21T22:49:07.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.379+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f983806d160 con 0x7f98401562d0 2021-11-21T22:49:07.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.383+0000 7f9844e6e700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9824053690 0x7f9824055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.412 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.383+0000 7f983ffff700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7f9824059e90 0x7f982405c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.383+0000 7f983ffff700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7f9824059e90 0x7f982405c2b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.383+0000 7f9844e6e700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9824053690 0x7f9824055b50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f9828000f70 tx=0x7f98280069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.413 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.383+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== osd.5 v2:172.21.2.117:6808/14957 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f982405c870 con 0x7f9824059e90 2021-11-21T22:49:07.414 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.391+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 --> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f98400b20b0 con 0x7f9824059e90 2021-11-21T22:49:07.415 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.391+0000 7f983dffb700 1 -- 172.21.2.116:0/3559081984 <== osd.5 v2:172.21.2.117:6808/14957 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f98400b20b0 con 0x7f9824059e90 2021-11-21T22:49:07.415 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:07.416 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:07.416 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:07.417 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7f9824059e90 msgr2=0x7f982405c2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6808/14957,v1:172.21.2.117:6809/14957] conn(0x7f9824059e90 0x7f982405c2b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9824053690 msgr2=0x7f9824055b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.418 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f9824053690 0x7f9824055b50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f9828000f70 tx=0x7f98280069b0).stop 2021-11-21T22:49:07.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401562d0 msgr2=0x7f98400b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 --2- 172.21.2.116:0/3559081984 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f98401562d0 0x7f98400b3b20 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 rx=0x7f984015a880 tx=0x7f9838015000).stop 2021-11-21T22:49:07.419 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f984566f700 1 -- 172.21.2.116:0/3559081984 reap_dead start 2021-11-21T22:49:07.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 shutdown_connections 2021-11-21T22:49:07.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 >> 172.21.2.116:0/3559081984 conn(0x7f984005bd20 msgr2=0x7f9840142bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.420 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 shutdown_connections 2021-11-21T22:49:07.421 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.395+0000 7f98470d2700 1 -- 172.21.2.116:0/3559081984 wait complete. 2021-11-21T22:49:07.421 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2021-11-21T22:49:07.488 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.479+0000 7f6655fd9700 1 Processor -- start 2021-11-21T22:49:07.489 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.479+0000 7f6655fd9700 1 -- start start 2021-11-21T22:49:07.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f66500aceb0 0x7f66501430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 0x7f66501435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.491 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66500b0690 con 0x7f66500b1830 2021-11-21T22:49:07.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665013cfd0 con 0x7f66500aceb0 2021-11-21T22:49:07.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6650143a90 con 0x7f6650146cb0 2021-11-21T22:49:07.492 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f66500aceb0 0x7f66501430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f66500aceb0 0x7f66501430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55244/0 (socket says 172.21.2.116:55244) 2021-11-21T22:49:07.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 -- 172.21.2.116:0/1656004916 learned_addr learned my addr 172.21.2.116:0/1656004916 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.493 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 -- 172.21.2.116:0/1656004916 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 msgr2=0x7f66501435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:07.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 --2- 172.21.2.116:0/1656004916 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 0x7f66501435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 -- 172.21.2.116:0/1656004916 >> v1:172.21.2.116:6789/0 conn(0x7f66500b1830 legacy=0x7f66501465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.494 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 -- 172.21.2.116:0/1656004916 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66500ad290 con 0x7f66500aceb0 2021-11-21T22:49:07.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664f7fe700 1 --2- 172.21.2.116:0/1656004916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500aceb0 0x7f66501430b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x7f6638002f70 tx=0x7f6638006eb0).ready entity=mon.0 client_cookie=c11800d519609320 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664e7fc700 1 -- 172.21.2.116:0/1656004916 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663800d5c0 con 0x7f66500aceb0 2021-11-21T22:49:07.495 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664e7fc700 1 -- 172.21.2.116:0/1656004916 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6638000e90 con 0x7f66500aceb0 2021-11-21T22:49:07.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664e7fc700 1 -- 172.21.2.116:0/1656004916 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663800d950 con 0x7f66500aceb0 2021-11-21T22:49:07.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- 172.21.2.116:0/1656004916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500aceb0 msgr2=0x7f66501430b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.496 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- 172.21.2.116:0/1656004916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500aceb0 0x7f66501430b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x7f6638002f70 tx=0x7f6638006eb0).stop 2021-11-21T22:49:07.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- 172.21.2.116:0/1656004916 shutdown_connections 2021-11-21T22:49:07.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- 172.21.2.116:0/1656004916 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 0x7f66501435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.497 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- 172.21.2.116:0/1656004916 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500aceb0 0x7f66501430b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- 172.21.2.116:0/1656004916 >> 172.21.2.116:0/1656004916 conn(0x7f66500b5730 msgr2=0x7f66500b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- 172.21.2.116:0/1656004916 shutdown_connections 2021-11-21T22:49:07.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- 172.21.2.116:0/1656004916 wait complete. 2021-11-21T22:49:07.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 Processor -- start 2021-11-21T22:49:07.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- start start 2021-11-21T22:49:07.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66500aceb0 0x7f6650156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500b1830 0x7f6650064a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 0x7f665013f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665013df20 con 0x7f66500b1830 2021-11-21T22:49:07.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665013e1c0 con 0x7f66500aceb0 2021-11-21T22:49:07.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665013e460 con 0x7f6650146cb0 2021-11-21T22:49:07.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500b1830 0x7f6650064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500b1830 0x7f6650064a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55254/0 (socket says 172.21.2.116:55254) 2021-11-21T22:49:07.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 -- 172.21.2.116:0/3386811506 learned_addr learned my addr 172.21.2.116:0/3386811506 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 -- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 msgr2=0x7f665013f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 0x7f665013f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 -- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66500aceb0 msgr2=0x7f6650156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66500aceb0 0x7f6650156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 -- 172.21.2.116:0/3386811506 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663800b1e0 con 0x7f66500b1830 2021-11-21T22:49:07.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664effd700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500b1830 0x7f6650064a10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 rx=0x7f6650062d10 tx=0x7f6640007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664cff9700 1 -- 172.21.2.116:0/3386811506 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664000ce30 con 0x7f66500b1830 2021-11-21T22:49:07.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664cff9700 1 -- 172.21.2.116:0/3386811506 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664000f470 con 0x7f66500b1830 2021-11-21T22:49:07.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f664cff9700 1 -- 172.21.2.116:0/3386811506 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664000f7d0 con 0x7f66500b1830 2021-11-21T22:49:07.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665013e700 con 0x7f66500b1830 2021-11-21T22:49:07.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.483+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665013eba0 con 0x7f66500b1830 2021-11-21T22:49:07.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.487+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665013e890 con 0x7f66500b1830 2021-11-21T22:49:07.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.487+0000 7f664cff9700 1 -- 172.21.2.116:0/3386811506 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6640019030 con 0x7f66500b1830 2021-11-21T22:49:07.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.487+0000 7f664cff9700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6634053580 0x7f6634055a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.487+0000 7f664cff9700 1 -- 172.21.2.116:0/3386811506 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f664006abd0 con 0x7f66500b1830 2021-11-21T22:49:07.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.487+0000 7f664f7fe700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6634053580 0x7f6634055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.487+0000 7f664f7fe700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6634053580 0x7f6634055a40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f6638002f70 tx=0x7f6638015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.511 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.487+0000 7f664cff9700 1 -- 172.21.2.116:0/3386811506 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6640041ea0 con 0x7f66500b1830 2021-11-21T22:49:07.579 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.571+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6650065240 con 0x7f6634053580 2021-11-21T22:49:07.580 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.571+0000 7f664cff9700 1 -- 172.21.2.116:0/3386811506 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6650065240 con 0x7f6634053580 2021-11-21T22:49:07.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6634053580 msgr2=0x7f6634055a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.582 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6634053580 0x7f6634055a40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f6638002f70 tx=0x7f6638015000).stop 2021-11-21T22:49:07.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500b1830 msgr2=0x7f6650064a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.583 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500b1830 0x7f6650064a10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 rx=0x7f6650062d10 tx=0x7f6640007c50).stop 2021-11-21T22:49:07.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 shutdown_connections 2021-11-21T22:49:07.584 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6650146cb0 0x7f665013f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.585 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6634053580 0x7f6634055a40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f66500b1830 0x7f6650064a10 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.586 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 --2- 172.21.2.116:0/3386811506 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f66500aceb0 0x7f6650156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 >> 172.21.2.116:0/3386811506 conn(0x7f66500b5730 msgr2=0x7f66501471e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 shutdown_connections 2021-11-21T22:49:07.588 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.575+0000 7f6655fd9700 1 -- 172.21.2.116:0/3386811506 wait complete. 2021-11-21T22:49:07.589 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 5 to deep-scrub 2021-11-21T22:49:07.589 INFO:tasks.ceph:Scrubbing osd.6 2021-11-21T22:49:07.590 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:07.643 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 Processor -- start 2021-11-21T22:49:07.644 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 -- start start 2021-11-21T22:49:07.647 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff7680b11c0 0x7ff768147230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7ff7680a8670 0x7ff768147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.648 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff768147c10 0x7ff76814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680b1940 con 0x7ff7680b11c0 2021-11-21T22:49:07.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680b15a0 con 0x7ff7680a8670 2021-11-21T22:49:07.649 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76f57f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680b1b70 con 0x7ff768147c10 2021-11-21T22:49:07.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff7680b11c0 0x7ff768147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7ff7680b11c0 0x7ff768147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55258/0 (socket says 172.21.2.116:55258) 2021-11-21T22:49:07.650 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 -- 172.21.2.116:0/986557521 learned_addr learned my addr 172.21.2.116:0/986557521 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 -- 172.21.2.116:0/986557521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff768147c10 msgr2=0x7ff76814a6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 --2- 172.21.2.116:0/986557521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff768147c10 0x7ff76814a6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.651 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 -- 172.21.2.116:0/986557521 >> v2:172.21.2.117:3300/0 conn(0x7ff7680a8670 msgr2=0x7ff768147720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 --2- 172.21.2.116:0/986557521 >> v2:172.21.2.117:3300/0 conn(0x7ff7680a8670 0x7ff768147720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 -- 172.21.2.116:0/986557521 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680b1e10 con 0x7ff7680b11c0 2021-11-21T22:49:07.652 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff76d31b700 1 --2- 172.21.2.116:0/986557521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7680b11c0 0x7ff768147230 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 rx=0x7ff764002f70 tx=0x7ff764006eb0).ready entity=mon.0 client_cookie=9082c2632f473964 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff757fff700 1 -- 172.21.2.116:0/986557521 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff76400d570 con 0x7ff7680b11c0 2021-11-21T22:49:07.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff757fff700 1 -- 172.21.2.116:0/986557521 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff764000e90 con 0x7ff7680b11c0 2021-11-21T22:49:07.653 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.635+0000 7ff757fff700 1 -- 172.21.2.116:0/986557521 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff76400d900 con 0x7ff7680b11c0 2021-11-21T22:49:07.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/986557521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7680b11c0 msgr2=0x7ff768147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 --2- 172.21.2.116:0/986557521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7680b11c0 0x7ff768147230 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 rx=0x7ff764002f70 tx=0x7ff764006eb0).stop 2021-11-21T22:49:07.654 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/986557521 shutdown_connections 2021-11-21T22:49:07.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 --2- 172.21.2.116:0/986557521 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff768147c10 0x7ff76814a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 --2- 172.21.2.116:0/986557521 >> v2:172.21.2.117:3300/0 conn(0x7ff7680a8670 0x7ff768147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.655 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 --2- 172.21.2.116:0/986557521 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff7680b11c0 0x7ff768147230 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/986557521 >> 172.21.2.116:0/986557521 conn(0x7ff76805bd20 msgr2=0x7ff76805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/986557521 shutdown_connections 2021-11-21T22:49:07.656 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/986557521 wait complete. 2021-11-21T22:49:07.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 Processor -- start 2021-11-21T22:49:07.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- start start 2021-11-21T22:49:07.657 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff7680a8670 0x7ff76814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff7680b11c0 0x7ff7681401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff768147c10 0x7ff7680b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.658 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76814c740 con 0x7ff768147c10 2021-11-21T22:49:07.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680b5d70 con 0x7ff7680a8670 2021-11-21T22:49:07.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76814bc30 con 0x7ff7680b11c0 2021-11-21T22:49:07.659 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff768147c10 0x7ff7680b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff768147c10 0x7ff7680b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55264/0 (socket says 172.21.2.116:55264) 2021-11-21T22:49:07.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 -- 172.21.2.116:0/2116358734 learned_addr learned my addr 172.21.2.116:0/2116358734 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.660 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 -- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff7680b11c0 msgr2=0x7ff7681401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7ff7680b11c0 0x7ff7681401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 -- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff7680a8670 msgr2=0x7ff76814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.661 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7ff7680a8670 0x7ff76814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 -- 172.21.2.116:0/2116358734 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76400b1e0 con 0x7ff768147c10 2021-11-21T22:49:07.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76db1c700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff768147c10 0x7ff7680b5400 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7ff760001250 tx=0x7ff7600090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.662 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7600014e0 con 0x7ff768147c10 2021-11-21T22:49:07.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff760001640 con 0x7ff768147c10 2021-11-21T22:49:07.663 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7600019a0 con 0x7ff768147c10 2021-11-21T22:49:07.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76814c230 con 0x7ff768147c10 2021-11-21T22:49:07.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76814b720 con 0x7ff768147c10 2021-11-21T22:49:07.664 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff76814c900 con 0x7ff768147c10 2021-11-21T22:49:07.666 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7ff760018030 con 0x7ff768147c10 2021-11-21T22:49:07.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7480535d0 0x7ff748055a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff76006b100 con 0x7ff768147c10 2021-11-21T22:49:07.667 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7ff748059dd0 0x7ff74805c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 --> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff74805c850 con 0x7ff748059dd0 2021-11-21T22:49:07.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.639+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76006d6c0 con 0x7ff768147c10 2021-11-21T22:49:07.668 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.643+0000 7ff76d31b700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7480535d0 0x7ff748055a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.643+0000 7ff76cb1a700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7ff748059dd0 0x7ff74805c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.643+0000 7ff76d31b700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7480535d0 0x7ff748055a90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7ff764002f70 tx=0x7ff764015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.669 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.643+0000 7ff76cb1a700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7ff748059dd0 0x7ff74805c1f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.647+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== osd.6 v2:172.21.2.117:6816/14958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7ff74805c850 con 0x7ff748059dd0 2021-11-21T22:49:07.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.651+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 --> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff7680b5b60 con 0x7ff748059dd0 2021-11-21T22:49:07.670 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff7567fc700 1 -- 172.21.2.116:0/2116358734 <== osd.6 v2:172.21.2.117:6816/14958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff7680b5b60 con 0x7ff748059dd0 2021-11-21T22:49:07.671 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:07.671 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:07.672 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:07.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7ff748059dd0 msgr2=0x7ff74805c1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.673 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6816/14958,v1:172.21.2.117:6817/14958] conn(0x7ff748059dd0 0x7ff74805c1f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7480535d0 msgr2=0x7ff748055a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7ff7480535d0 0x7ff748055a90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7ff764002f70 tx=0x7ff764015000).stop 2021-11-21T22:49:07.674 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff768147c10 msgr2=0x7ff7680b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 --2- 172.21.2.116:0/2116358734 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7ff768147c10 0x7ff7680b5400 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7ff760001250 tx=0x7ff7600090f0).stop 2021-11-21T22:49:07.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76db1c700 1 -- 172.21.2.116:0/2116358734 reap_dead start 2021-11-21T22:49:07.675 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 shutdown_connections 2021-11-21T22:49:07.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 >> 172.21.2.116:0/2116358734 conn(0x7ff76805bd20 msgr2=0x7ff7681471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 shutdown_connections 2021-11-21T22:49:07.676 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.655+0000 7ff76f57f700 1 -- 172.21.2.116:0/2116358734 wait complete. 2021-11-21T22:49:07.677 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2021-11-21T22:49:07.727 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd12b977700 1 Processor -- start 2021-11-21T22:49:07.728 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd12b977700 1 -- start start 2021-11-21T22:49:07.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd12b977700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fd124062e60 0x7fd124142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.732 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd12b977700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd124069e70 0x7fd1240ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd12b977700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1240afae0 con 0x7fd1241426e0 2021-11-21T22:49:07.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd12b977700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1240b0f60 con 0x7fd124069e70 2021-11-21T22:49:07.733 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd12b977700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd124063470 con 0x7fd124062e60 2021-11-21T22:49:07.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd124069e70 0x7fd1240ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fd124069e70 0x7fd1240ad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55278/0 (socket says 172.21.2.116:55278) 2021-11-21T22:49:07.734 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 -- 172.21.2.116:0/1691920119 learned_addr learned my addr 172.21.2.116:0/1691920119 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 -- 172.21.2.116:0/1691920119 >> v2:172.21.2.117:3300/0 conn(0x7fd124062e60 msgr2=0x7fd124142ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 --2- 172.21.2.116:0/1691920119 >> v2:172.21.2.117:3300/0 conn(0x7fd124062e60 0x7fd124142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.735 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 -- 172.21.2.116:0/1691920119 >> v1:172.21.2.117:6789/0 conn(0x7fd1241426e0 legacy=0x7fd1240af3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:07.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 -- 172.21.2.116:0/1691920119 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124143050 con 0x7fd124069e70 2021-11-21T22:49:07.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.719+0000 7fd129f14700 1 --2- 172.21.2.116:0/1691920119 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd124069e70 0x7fd1240ad470 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7fd124062d10 tx=0x7fd11c009290).ready entity=mon.0 client_cookie=70ce5c7b34fb6882 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.736 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd113fff700 1 -- 172.21.2.116:0/1691920119 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd11c00b830 con 0x7fd124069e70 2021-11-21T22:49:07.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd113fff700 1 -- 172.21.2.116:0/1691920119 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c00b990 con 0x7fd124069e70 2021-11-21T22:49:07.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd113fff700 1 -- 172.21.2.116:0/1691920119 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd11c00bb40 con 0x7fd124069e70 2021-11-21T22:49:07.737 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/1691920119 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd124069e70 msgr2=0x7fd1240ad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 --2- 172.21.2.116:0/1691920119 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd124069e70 0x7fd1240ad470 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7fd124062d10 tx=0x7fd11c009290).stop 2021-11-21T22:49:07.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/1691920119 shutdown_connections 2021-11-21T22:49:07.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 --2- 172.21.2.116:0/1691920119 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd124069e70 0x7fd1240ad470 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.738 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 --2- 172.21.2.116:0/1691920119 >> v2:172.21.2.117:3300/0 conn(0x7fd124062e60 0x7fd124142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/1691920119 >> 172.21.2.116:0/1691920119 conn(0x7fd1240b5730 msgr2=0x7fd1240b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/1691920119 shutdown_connections 2021-11-21T22:49:07.739 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/1691920119 wait complete. 2021-11-21T22:49:07.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 Processor -- start 2021-11-21T22:49:07.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- start start 2021-11-21T22:49:07.740 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd124069e70 0x7fd124067140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1241426e0 0x7fd1240641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd124066100 0x7fd124155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.741 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd124064900 con 0x7fd1241426e0 2021-11-21T22:49:07.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1241418a0 con 0x7fd124069e70 2021-11-21T22:49:07.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd124064af0 con 0x7fd124066100 2021-11-21T22:49:07.742 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1241426e0 0x7fd1240641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1241426e0 0x7fd1240641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55280/0 (socket says 172.21.2.116:55280) 2021-11-21T22:49:07.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 -- 172.21.2.116:0/2412476460 learned_addr learned my addr 172.21.2.116:0/2412476460 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.743 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 -- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd124066100 msgr2=0x7fd124155490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd124066100 0x7fd124155490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 -- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd124069e70 msgr2=0x7fd124067140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:07.744 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd124069e70 0x7fd124067140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 -- 172.21.2.116:0/2412476460 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009530 con 0x7fd1241426e0 2021-11-21T22:49:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd128f12700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1241426e0 0x7fd1240641b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7fd114002fb0 tx=0x7fd114006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.745 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd1127fc700 1 -- 172.21.2.116:0/2412476460 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd114013460 con 0x7fd1241426e0 2021-11-21T22:49:07.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd1127fc700 1 -- 172.21.2.116:0/2412476460 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1140135c0 con 0x7fd1241426e0 2021-11-21T22:49:07.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd124063d10 con 0x7fd1241426e0 2021-11-21T22:49:07.746 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd1127fc700 1 -- 172.21.2.116:0/2412476460 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd114013920 con 0x7fd1241426e0 2021-11-21T22:49:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12413f140 con 0x7fd1241426e0 2021-11-21T22:49:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd124063ea0 con 0x7fd1241426e0 2021-11-21T22:49:07.747 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd1127fc700 1 -- 172.21.2.116:0/2412476460 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fd11400f070 con 0x7fd1241426e0 2021-11-21T22:49:07.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd1127fc700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1040535b0 0x7fd104055a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.748 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.723+0000 7fd1127fc700 1 -- 172.21.2.116:0/2412476460 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd11406b050 con 0x7fd1241426e0 2021-11-21T22:49:07.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.727+0000 7fd129713700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1040535b0 0x7fd104055a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.749 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.727+0000 7fd129713700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1040535b0 0x7fd104055a70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fd120002ef0 tx=0x7fd120006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.750 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.727+0000 7fd1127fc700 1 -- 172.21.2.116:0/2412476460 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd114042490 con 0x7fd1241426e0 2021-11-21T22:49:07.817 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fd124063b90 con 0x7fd1040535b0 2021-11-21T22:49:07.818 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd1127fc700 1 -- 172.21.2.116:0/2412476460 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd12000beb0 con 0x7fd1040535b0 2021-11-21T22:49:07.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1040535b0 msgr2=0x7fd104055a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.820 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1040535b0 0x7fd104055a70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fd120002ef0 tx=0x7fd120006890).stop 2021-11-21T22:49:07.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1241426e0 msgr2=0x7fd1240641b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.822 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1241426e0 0x7fd1240641b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7fd114002fb0 tx=0x7fd114006df0).stop 2021-11-21T22:49:07.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 shutdown_connections 2021-11-21T22:49:07.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fd124066100 0x7fd124155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.823 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fd1040535b0 0x7fd104055a70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fd1241426e0 0x7fd1240641b0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 --2- 172.21.2.116:0/2412476460 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fd124069e70 0x7fd124067140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.824 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 >> 172.21.2.116:0/2412476460 conn(0x7fd1240b5730 msgr2=0x7fd1240b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 shutdown_connections 2021-11-21T22:49:07.825 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.811+0000 7fd12b977700 1 -- 172.21.2.116:0/2412476460 wait complete. 2021-11-21T22:49:07.826 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 6 to deep-scrub 2021-11-21T22:49:07.827 INFO:tasks.ceph:Scrubbing osd.7 2021-11-21T22:49:07.827 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2021-11-21T22:49:07.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda60ed1700 1 Processor -- start 2021-11-21T22:49:07.881 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda60ed1700 1 -- start start 2021-11-21T22:49:07.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda60ed1700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fda5c0a8670 0x7fda5c147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.882 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda60ed1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 0x7fda5c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda60ed1700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c0b1940 con 0x7fda5c0b11c0 2021-11-21T22:49:07.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda60ed1700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0b15a0 con 0x7fda5c0a8670 2021-11-21T22:49:07.883 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda60ed1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0b1b70 con 0x7fda5c143e60 2021-11-21T22:49:07.884 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda59d9b700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7fda5c0a8670 0x7fda5c147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda5a59c700 1 --1- >> v1:172.21.2.117:6789/0 conn(0x7fda5c0b11c0 0x7fda5c147230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.117:6789/0 says I am v1:172.21.2.116:37922/0 (socket says 172.21.2.116:37922) 2021-11-21T22:49:07.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda5a59c700 1 -- 172.21.2.116:0/2008823401 learned_addr learned my addr 172.21.2.116:0/2008823401 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:07.885 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda5ad9d700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 0x7fda5c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda59d9b700 1 -- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 msgr2=0x7fda5c1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:07.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda59d9b700 1 --2- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 0x7fda5c1442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.886 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda59d9b700 1 -- 172.21.2.116:0/2008823401 >> v1:172.21.2.117:6789/0 conn(0x7fda5c0b11c0 legacy=0x7fda5c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda5959a700 1 -- 172.21.2.116:0/2008823401 <== mon.1 v1:172.21.2.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325245593 0 0) 0x7fda5c0b1940 con 0x7fda5c0b11c0 2021-11-21T22:49:07.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda59d9b700 1 -- 172.21.2.116:0/2008823401 --> v2:172.21.2.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0b1e10 con 0x7fda5c0a8670 2021-11-21T22:49:07.887 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.875+0000 7fda5ad9d700 1 --2- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 0x7fda5c1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-21T22:49:07.888 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda59d9b700 1 --2- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fda5c0a8670 0x7fda5c147940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44006df0).ready entity=mon.1 client_cookie=ae80e519e68147b4 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5959a700 1 -- 172.21.2.116:0/2008823401 <== mon.1 v2:172.21.2.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400d570 con 0x7fda5c0a8670 2021-11-21T22:49:07.891 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5959a700 1 -- 172.21.2.116:0/2008823401 <== mon.1 v2:172.21.2.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44000e90 con 0x7fda5c0a8670 2021-11-21T22:49:07.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5959a700 1 -- 172.21.2.116:0/2008823401 <== mon.1 v2:172.21.2.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400d8c0 con 0x7fda5c0a8670 2021-11-21T22:49:07.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fda5c0a8670 msgr2=0x7fda5c147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.892 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 --2- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fda5c0a8670 0x7fda5c147940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44006df0).stop 2021-11-21T22:49:07.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- 172.21.2.116:0/2008823401 shutdown_connections 2021-11-21T22:49:07.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 --2- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 0x7fda5c1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.893 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 --2- 172.21.2.116:0/2008823401 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fda5c0a8670 0x7fda5c147940 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- 172.21.2.116:0/2008823401 >> 172.21.2.116:0/2008823401 conn(0x7fda5c05bd20 msgr2=0x7fda5c05c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- 172.21.2.116:0/2008823401 shutdown_connections 2021-11-21T22:49:07.894 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- 172.21.2.116:0/2008823401 wait complete. 2021-11-21T22:49:07.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 Processor -- start 2021-11-21T22:49:07.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- start start 2021-11-21T22:49:07.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fda5c0a8670 0x7fda5c143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.895 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fda5c0b11c0 0x7fda5c063850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 0x7fda5c13e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c14a7f0 con 0x7fda5c0a8670 2021-11-21T22:49:07.896 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0af180 con 0x7fda5c0b11c0 2021-11-21T22:49:07.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c141150 con 0x7fda5c143e60 2021-11-21T22:49:07.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fda5c0a8670 0x7fda5c143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.897 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fda5c0a8670 0x7fda5c143390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55294/0 (socket says 172.21.2.116:55294) 2021-11-21T22:49:07.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 -- 172.21.2.116:0/3938203465 learned_addr learned my addr 172.21.2.116:0/3938203465 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 -- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 msgr2=0x7fda5c13e300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:07.898 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fda5c143e60 0x7fda5c13e300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 -- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fda5c0b11c0 msgr2=0x7fda5c063850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:07.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fda5c0b11c0 0x7fda5c063850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.899 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 -- 172.21.2.116:0/3938203465 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54009470 con 0x7fda5c0a8670 2021-11-21T22:49:07.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda5a59c700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fda5c0a8670 0x7fda5c143390 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 rx=0x7fda4c000bc0 tx=0x7fda4c007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4c00ce50 con 0x7fda5c0a8670 2021-11-21T22:49:07.900 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c0072e0 con 0x7fda5c0a8670 2021-11-21T22:49:07.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4400b180 con 0x7fda5c0a8670 2021-11-21T22:49:07.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4c005800 con 0x7fda5c0a8670 2021-11-21T22:49:07.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.879+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c1413f0 con 0x7fda5c0a8670 2021-11-21T22:49:07.901 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda5c14ac20 con 0x7fda5c0a8670 2021-11-21T22:49:07.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fda4c01a060 con 0x7fda5c0a8670 2021-11-21T22:49:07.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda537fe700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fda40053610 0x7fda40055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.902 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fda4c06aed0 con 0x7fda5c0a8670 2021-11-21T22:49:07.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda537fe700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fda40059d90 0x7fda4005c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 --> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda4005c770 con 0x7fda40059d90 2021-11-21T22:49:07.903 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda4c06d560 con 0x7fda5c0a8670 2021-11-21T22:49:07.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda59d9b700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fda40053610 0x7fda40055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda5ad9d700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fda40059d90 0x7fda4005c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.904 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda59d9b700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fda40053610 0x7fda40055ad0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.905 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.883+0000 7fda5ad9d700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fda40059d90 0x7fda4005c1b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.887+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== osd.7 v2:172.21.2.117:6823/14960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fda4005c770 con 0x7fda40059d90 2021-11-21T22:49:07.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 --> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fda5c0b20b0 con 0x7fda40059d90 2021-11-21T22:49:07.906 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda537fe700 1 -- 172.21.2.116:0/3938203465 <== osd.7 v2:172.21.2.117:6823/14960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fda5c0b20b0 con 0x7fda40059d90 2021-11-21T22:49:07.907 INFO:teuthology.orchestra.run.gibba016.stdout:{ 2021-11-21T22:49:07.907 INFO:teuthology.orchestra.run.gibba016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-11-21T22:49:07.908 INFO:teuthology.orchestra.run.gibba016.stdout:} 2021-11-21T22:49:07.909 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fda40059d90 msgr2=0x7fda4005c1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6823/14960,v1:172.21.2.117:6825/14960] conn(0x7fda40059d90 0x7fda4005c1b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fda40053610 msgr2=0x7fda40055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.910 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fda40053610 0x7fda40055ad0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44014000).stop 2021-11-21T22:49:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fda5c0a8670 msgr2=0x7fda5c143390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 --2- 172.21.2.116:0/3938203465 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fda5c0a8670 0x7fda5c143390 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 rx=0x7fda4c000bc0 tx=0x7fda4c007a60).stop 2021-11-21T22:49:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda5ad9d700 1 -- 172.21.2.116:0/3938203465 reap_dead start 2021-11-21T22:49:07.911 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 shutdown_connections 2021-11-21T22:49:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 >> 172.21.2.116:0/3938203465 conn(0x7fda5c05bd20 msgr2=0x7fda5c1442d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 shutdown_connections 2021-11-21T22:49:07.912 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.895+0000 7fda60ed1700 1 -- 172.21.2.116:0/3938203465 wait complete. 2021-11-21T22:49:07.913 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2021-11-21T22:49:07.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.955+0000 7fdde9b7f700 1 Processor -- start 2021-11-21T22:49:07.965 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.955+0000 7fdde9b7f700 1 -- start start 2021-11-21T22:49:07.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdde405fc40 0x7fdde4144af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde40b10b0 con 0x7fdde40b2600 2021-11-21T22:49:07.966 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde4060500 con 0x7fdde40b0cd0 2021-11-21T22:49:07.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde4060d80 con 0x7fdde405fc40 2021-11-21T22:49:07.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde2ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdde405fc40 0x7fdde4144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.967 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde2ffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fdde405fc40 0x7fdde4144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55306/0 (socket says 172.21.2.116:55306) 2021-11-21T22:49:07.968 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde2ffd700 1 -- 172.21.2.116:0/381824682 learned_addr learned my addr 172.21.2.116:0/381824682 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde2ffd700 1 -- 172.21.2.116:0/381824682 >> v1:172.21.2.117:6789/0 conn(0x7fdde40b0cd0 legacy=0x7fdde41443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:07.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde2ffd700 1 -- 172.21.2.116:0/381824682 >> v1:172.21.2.116:6789/0 conn(0x7fdde40b2600 legacy=0x7fdde4142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.972 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde2ffd700 1 -- 172.21.2.116:0/381824682 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4144fe0 con 0x7fdde405fc40 2021-11-21T22:49:07.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde2ffd700 1 --2- 172.21.2.116:0/381824682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde405fc40 0x7fdde4144af0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 rx=0x7fdde4063390 tx=0x7fddcc006eb0).ready entity=mon.0 client_cookie=4e8de6f15233ad7c server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde27fc700 1 -- 172.21.2.116:0/381824682 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc00d610 con 0x7fdde405fc40 2021-11-21T22:49:07.973 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde27fc700 1 -- 172.21.2.116:0/381824682 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc000e90 con 0x7fdde405fc40 2021-11-21T22:49:07.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde27fc700 1 -- 172.21.2.116:0/381824682 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc00d960 con 0x7fdde405fc40 2021-11-21T22:49:07.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- 172.21.2.116:0/381824682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde405fc40 msgr2=0x7fdde4144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.974 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/381824682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde405fc40 0x7fdde4144af0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 rx=0x7fdde4063390 tx=0x7fddcc006eb0).stop 2021-11-21T22:49:07.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- 172.21.2.116:0/381824682 shutdown_connections 2021-11-21T22:49:07.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/381824682 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde405fc40 0x7fdde4144af0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:07.975 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- 172.21.2.116:0/381824682 >> 172.21.2.116:0/381824682 conn(0x7fdde40b5ff0 msgr2=0x7fdde40b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:07.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- 172.21.2.116:0/381824682 shutdown_connections 2021-11-21T22:49:07.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- 172.21.2.116:0/381824682 wait complete. 2021-11-21T22:49:07.976 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 Processor -- start 2021-11-21T22:49:07.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- start start 2021-11-21T22:49:07.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdde405fc40 0x7fdde4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.977 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdde40b0cd0 0x7fdde40ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde40b2600 0x7fdde41405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.978 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde40afa60 con 0x7fdde40b2600 2021-11-21T22:49:07.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde4156c50 con 0x7fdde405fc40 2021-11-21T22:49:07.979 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde40ae4f0 con 0x7fdde40b0cd0 2021-11-21T22:49:07.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde40b2600 0x7fdde41405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.981 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde40b2600 0x7fdde41405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55312/0 (socket says 172.21.2.116:55312) 2021-11-21T22:49:07.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 -- 172.21.2.116:0/2810138869 learned_addr learned my addr 172.21.2.116:0/2810138869 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:07.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 -- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdde40b0cd0 msgr2=0x7fdde40ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.982 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdde40b0cd0 0x7fdde40ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 -- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdde405fc40 msgr2=0x7fdde4156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:07.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdde405fc40 0x7fdde4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:07.983 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 -- 172.21.2.116:0/2810138869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc00b1e0 con 0x7fdde40b2600 2021-11-21T22:49:07.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde3fff700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde40b2600 0x7fdde41405f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7fdde409ef30 tx=0x7fdddc007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde0ff9700 1 -- 172.21.2.116:0/2810138869 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdddc00e4a0 con 0x7fdde40b2600 2021-11-21T22:49:07.984 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde0ff9700 1 -- 172.21.2.116:0/2810138869 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc00e600 con 0x7fdde40b2600 2021-11-21T22:49:07.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde40af750 con 0x7fdde40b2600 2021-11-21T22:49:07.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde0ff9700 1 -- 172.21.2.116:0/2810138869 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdddc00e960 con 0x7fdde40b2600 2021-11-21T22:49:07.985 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.959+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde40af4b0 con 0x7fdde40b2600 2021-11-21T22:49:07.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.963+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde4060ae0 con 0x7fdde40b2600 2021-11-21T22:49:07.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.963+0000 7fdde0ff9700 1 -- 172.21.2.116:0/2810138869 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fdddc019030 con 0x7fdde40b2600 2021-11-21T22:49:07.986 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.963+0000 7fdde0ff9700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fddc8053610 0x7fddc8055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.963+0000 7fdde0ff9700 1 -- 172.21.2.116:0/2810138869 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdddc06ae20 con 0x7fdde40b2600 2021-11-21T22:49:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.963+0000 7fdde37fe700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fddc8053610 0x7fddc8055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:07.987 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.963+0000 7fdde37fe700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fddc8053610 0x7fddc8055ad0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fddd4002f90 tx=0x7fddd40069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:07.988 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:07.963+0000 7fdde0ff9700 1 -- 172.21.2.116:0/2810138869 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdddc042150 con 0x7fdde40b2600 2021-11-21T22:49:08.054 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fdde40ae6b0 con 0x7fddc8053610 2021-11-21T22:49:08.055 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde0ff9700 1 -- 172.21.2.116:0/2810138869 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fddd4012450 con 0x7fddc8053610 2021-11-21T22:49:08.056 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fddc8053610 msgr2=0x7fddc8055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:08.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fddc8053610 0x7fddc8055ad0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fddd4002f90 tx=0x7fddd40069b0).stop 2021-11-21T22:49:08.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde40b2600 msgr2=0x7fdde41405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:08.057 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde40b2600 0x7fdde41405f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7fdde409ef30 tx=0x7fdddc007f00).stop 2021-11-21T22:49:08.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 shutdown_connections 2021-11-21T22:49:08.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fddc8053610 0x7fddc8055ad0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:08.058 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fdde40b2600 0x7fdde41405f0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:08.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fdde40b0cd0 0x7fdde40ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 --2- 172.21.2.116:0/2810138869 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fdde405fc40 0x7fdde4156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.059 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 >> 172.21.2.116:0/2810138869 conn(0x7fdde40b5ff0 msgr2=0x7fdde40b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:08.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 shutdown_connections 2021-11-21T22:49:08.060 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.047+0000 7fdde9b7f700 1 -- 172.21.2.116:0/2810138869 wait complete. 2021-11-21T22:49:08.061 INFO:teuthology.orchestra.run.gibba016.stderr:instructed osd(s) 7 to deep-scrub 2021-11-21T22:49:08.062 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-11-21T22:49:08.163 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b36aea700 1 Processor -- start 2021-11-21T22:49:08.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b36aea700 1 -- start start 2021-11-21T22:49:08.164 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b36aea700 1 --2- >> v2:172.21.2.117:3300/0 conn(0x7f0b300ad780 0x7f0b3005f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b36aea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b300affc0 0x7f0b30143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b36aea700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300adb60 con 0x7f0b300af7e0 2021-11-21T22:49:08.165 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b36aea700 1 -- --> v2:172.21.2.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b30143160 con 0x7f0b300ad780 2021-11-21T22:49:08.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b36aea700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b0ce0 con 0x7f0b300affc0 2021-11-21T22:49:08.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b35087700 1 --1- >> v1:172.21.2.116:6789/0 conn(0x7f0b300af7e0 0x7f0b300b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:39046/0 (socket says 172.21.2.116:39046) 2021-11-21T22:49:08.166 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b35087700 1 -- 172.21.2.116:0/4148741078 learned_addr learned my addr 172.21.2.116:0/4148741078 (peer_addr_for_me v1:172.21.2.116:0/0) 2021-11-21T22:49:08.171 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277541791 0 0) 0x7f0b300adb60 con 0x7f0b300af7e0 2021-11-21T22:49:08.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 --> v1:172.21.2.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b10003260 con 0x7f0b300af7e0 2021-11-21T22:49:08.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 <== mon.0 v1:172.21.2.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 347025726 0 0) 0x7f0b10003260 con 0x7f0b300af7e0 2021-11-21T22:49:08.172 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b300affc0 msgr2=0x7f0b30143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b300affc0 0x7f0b30143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 >> v2:172.21.2.117:3300/0 conn(0x7f0b300ad780 msgr2=0x7f0b3005f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:08.173 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 --2- 172.21.2.116:0/4148741078 >> v2:172.21.2.117:3300/0 conn(0x7f0b300ad780 0x7f0b3005f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 --> v1:172.21.2.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300b0a40 con 0x7f0b300af7e0 2021-11-21T22:49:08.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 <== mon.0 v1:172.21.2.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2626934221 0 0) 0x7f0b28001a00 con 0x7f0b300af7e0 2021-11-21T22:49:08.174 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 >> v1:172.21.2.116:6789/0 conn(0x7f0b300af7e0 legacy=0x7f0b300b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b100037d0 0x7f0b10003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b10004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b300affc0 0x7f0b1000d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.175 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30143160 con 0x7f0b10004500 2021-11-21T22:49:08.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b0ce0 con 0x7f0b100037d0 2021-11-21T22:49:08.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10005300 con 0x7f0b300affc0 2021-11-21T22:49:08.176 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.155+0000 7f0b2ffff700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b10004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:08.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2ffff700 1 -- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b300affc0 msgr2=0x7f0b1000d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:08.177 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2ffff700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b300affc0 0x7f0b1000d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2ffff700 1 -- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b100037d0 msgr2=0x7f0b10003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-11-21T22:49:08.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2ffff700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b100037d0 0x7f0b10003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.178 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2ffff700 1 -- 172.21.2.116:0/4148741078 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10003260 con 0x7f0b10004500 2021-11-21T22:49:08.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2ffff700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b10004980 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7f0b20002f70 tx=0x7f0b20006ca0).ready entity=mon.0 client_cookie=4b0ce5b74b485f7a server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:08.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b20012410 con 0x7f0b10004500 2021-11-21T22:49:08.179 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b20012570 con 0x7f0b10004500 2021-11-21T22:49:08.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2f7fe700 1 -- 172.21.2.116:0/4148741078 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b200128d0 con 0x7f0b10004500 2021-11-21T22:49:08.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/4148741078 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 msgr2=0x7f0b10004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:08.180 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 --2- 172.21.2.116:0/4148741078 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b10004980 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7f0b20002f70 tx=0x7f0b20006ca0).stop 2021-11-21T22:49:08.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b35087700 1 -- 172.21.2.116:0/4148741078 reap_dead start 2021-11-21T22:49:08.181 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/4148741078 shutdown_connections 2021-11-21T22:49:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/4148741078 >> 172.21.2.116:0/4148741078 conn(0x7f0b300b6000 msgr2=0x7f0b300b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/4148741078 shutdown_connections 2021-11-21T22:49:08.182 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/4148741078 wait complete. 2021-11-21T22:49:08.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 Processor -- start 2021-11-21T22:49:08.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- start start 2021-11-21T22:49:08.183 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b3013e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b30067360 0x7f0b30159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b30159d80 0x7f0b301562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.184 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3015a770 con 0x7f0b10004500 2021-11-21T22:49:08.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3015a4d0 con 0x7f0b30067360 2021-11-21T22:49:08.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30141e80 con 0x7f0b30159d80 2021-11-21T22:49:08.185 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b3013e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:08.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b3013e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55330/0 (socket says 172.21.2.116:55330) 2021-11-21T22:49:08.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 -- 172.21.2.116:0/3557783886 learned_addr learned my addr 172.21.2.116:0/3557783886 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:08.186 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 -- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b30159d80 msgr2=0x7f0b301562c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:08.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b30159d80 0x7f0b301562c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 -- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b30067360 msgr2=0x7f0b30159890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-11-21T22:49:08.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b30067360 0x7f0b30159890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.187 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 -- 172.21.2.116:0/3557783886 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30142120 con 0x7f0b10004500 2021-11-21T22:49:08.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b34886700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b3013e330 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 rx=0x7f0b18002f70 tx=0x7f0b18006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:08.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2dffb700 1 -- 172.21.2.116:0/3557783886 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b18013460 con 0x7f0b10004500 2021-11-21T22:49:08.188 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2dffb700 1 -- 172.21.2.116:0/3557783886 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b180135c0 con 0x7f0b10004500 2021-11-21T22:49:08.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3015a200 con 0x7f0b10004500 2021-11-21T22:49:08.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b2dffb700 1 -- 172.21.2.116:0/3557783886 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b18013920 con 0x7f0b10004500 2021-11-21T22:49:08.189 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b30153d40 con 0x7f0b10004500 2021-11-21T22:49:08.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.159+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b301422b0 con 0x7f0b10004500 2021-11-21T22:49:08.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.163+0000 7f0b2dffb700 1 -- 172.21.2.116:0/3557783886 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f0b1800f070 con 0x7f0b10004500 2021-11-21T22:49:08.190 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.163+0000 7f0b2dffb700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b14053640 0x7f0b14055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:08.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.163+0000 7f0b2dffb700 1 -- 172.21.2.116:0/3557783886 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0b1806aef0 con 0x7f0b10004500 2021-11-21T22:49:08.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.163+0000 7f0b2ffff700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b14053640 0x7f0b14055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:08.191 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.163+0000 7f0b2ffff700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b14053640 0x7f0b14055b00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f0b20002f70 tx=0x7f0b20015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:08.192 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.163+0000 7f0b2dffb700 1 -- 172.21.2.116:0/3557783886 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b18042330 con 0x7f0b10004500 2021-11-21T22:49:08.251 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.243+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b301433c0 con 0x7f0b14053640 2021-11-21T22:49:08.260 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.251+0000 7f0b2dffb700 1 -- 172.21.2.116:0/3557783886 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66173 (secure 0 0 0) 0x7f0b301433c0 con 0x7f0b14053640 2021-11-21T22:49:08.261 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:49:08.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b14053640 msgr2=0x7f0b14055b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:08.263 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b14053640 0x7f0b14055b00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f0b20002f70 tx=0x7f0b20015000).stop 2021-11-21T22:49:08.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 msgr2=0x7f0b3013e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:08.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b3013e330 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 rx=0x7f0b18002f70 tx=0x7f0b18006ec0).stop 2021-11-21T22:49:08.264 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 shutdown_connections 2021-11-21T22:49:08.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f0b30159d80 0x7f0b301562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f0b30067360 0x7f0b30159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:08.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f0b14053640 0x7f0b14055b00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:08.265 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 --2- 172.21.2.116:0/3557783886 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f0b10004500 0x7f0b3013e330 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:08.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 >> 172.21.2.116:0/3557783886 conn(0x7f0b300b6000 msgr2=0x7f0b300b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:08.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 shutdown_connections 2021-11-21T22:49:08.266 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:08.255+0000 7f0b36aea700 1 -- 172.21.2.116:0/3557783886 wait complete. 2021-11-21T22:49:08.267 INFO:teuthology.orchestra.run.gibba016.stderr:dumped all 2021-11-21T22:49:08.268 INFO:teuthology.orchestra.run.gibba016.stdout:{"pg_ready":true,"pg_map":{"version":1803,"stamp":"2021-11-21T22:49:06.430048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":704299008,"kb_used":6104448,"kb_used_data":698194560,"kb_used_omap":27875,"kb_used_meta":0,"kb_avail":698194560,"statfs":{"total":721202184192,"available":714951229440,"internally_reserved":0,"allocated":714951229440,"data_stored":714951229440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28544708,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004971"},"pg_stats":[{"pgid":"1.0","version":"14'2","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.433989+0000","last_change":"2021-11-21T21:54:16.981838+0000","last_active":"2021-11-21T22:03:33.433989+0000","last_peered":"2021-11-21T22:03:33.433989+0000","last_clean":"2021-11-21T22:03:33.433989+0000","last_became_active":"2021-11-21T21:54:16.981409+0000","last_became_peered":"2021-11-21T21:54:16.981409+0000","last_unstale":"2021-11-21T22:03:33.433989+0000","last_undegraded":"2021-11-21T22:03:33.433989+0000","last_fullsized":"2021-11-21T22:03:33.433989+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:13.901475+0000","last_clean_scrub_stamp":"2021-11-21T21:54:13.901475+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.002389+0000","last_change":"2021-11-21T21:54:22.713500+0000","last_active":"2021-11-21T22:03:33.002389+0000","last_peered":"2021-11-21T22:03:33.002389+0000","last_clean":"2021-11-21T22:03:33.002389+0000","last_became_active":"2021-11-21T21:54:22.713118+0000","last_became_peered":"2021-11-21T21:54:22.713118+0000","last_unstale":"2021-11-21T22:03:33.002389+0000","last_undegraded":"2021-11-21T22:03:33.002389+0000","last_fullsized":"2021-11-21T22:03:33.002389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.434053+0000","last_change":"2021-11-21T21:54:22.713430+0000","last_active":"2021-11-21T22:03:33.434053+0000","last_peered":"2021-11-21T22:03:33.434053+0000","last_clean":"2021-11-21T22:03:33.434053+0000","last_became_active":"2021-11-21T21:54:22.712862+0000","last_became_peered":"2021-11-21T21:54:22.712862+0000","last_unstale":"2021-11-21T22:03:33.434053+0000","last_undegraded":"2021-11-21T22:03:33.434053+0000","last_fullsized":"2021-11-21T22:03:33.434053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.594014+0000","last_change":"2021-11-21T21:54:22.711838+0000","last_active":"2021-11-21T22:03:33.594014+0000","last_peered":"2021-11-21T22:03:33.594014+0000","last_clean":"2021-11-21T22:03:33.594014+0000","last_became_active":"2021-11-21T21:54:22.711687+0000","last_became_peered":"2021-11-21T21:54:22.711687+0000","last_unstale":"2021-11-21T22:03:33.594014+0000","last_undegraded":"2021-11-21T22:03:33.594014+0000","last_fullsized":"2021-11-21T22:03:33.594014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.002388+0000","last_change":"2021-11-21T21:54:22.713478+0000","last_active":"2021-11-21T22:03:33.002388+0000","last_peered":"2021-11-21T22:03:33.002388+0000","last_clean":"2021-11-21T22:03:33.002388+0000","last_became_active":"2021-11-21T21:54:22.713254+0000","last_became_peered":"2021-11-21T21:54:22.713254+0000","last_unstale":"2021-11-21T22:03:33.002388+0000","last_undegraded":"2021-11-21T22:03:33.002388+0000","last_fullsized":"2021-11-21T22:03:33.002388+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:34.289098+0000","last_change":"2021-11-21T21:54:22.786234+0000","last_active":"2021-11-21T22:03:34.289098+0000","last_peered":"2021-11-21T22:03:34.289098+0000","last_clean":"2021-11-21T22:03:34.289098+0000","last_became_active":"2021-11-21T21:54:22.786041+0000","last_became_peered":"2021-11-21T21:54:22.786041+0000","last_unstale":"2021-11-21T22:03:34.289098+0000","last_undegraded":"2021-11-21T22:03:34.289098+0000","last_fullsized":"2021-11-21T22:03:34.289098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:33.434097+0000","last_change":"2021-11-21T21:54:22.788120+0000","last_active":"2021-11-21T22:03:33.434097+0000","last_peered":"2021-11-21T22:03:33.434097+0000","last_clean":"2021-11-21T22:03:33.434097+0000","last_became_active":"2021-11-21T21:54:22.787305+0000","last_became_peered":"2021-11-21T21:54:22.787305+0000","last_unstale":"2021-11-21T22:03:33.434097+0000","last_undegraded":"2021-11-21T22:03:33.434097+0000","last_fullsized":"2021-11-21T22:03:33.434097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:34.289171+0000","last_change":"2021-11-21T21:54:22.713690+0000","last_active":"2021-11-21T22:03:34.289171+0000","last_peered":"2021-11-21T22:03:34.289171+0000","last_clean":"2021-11-21T22:03:34.289171+0000","last_became_active":"2021-11-21T21:54:22.713554+0000","last_became_peered":"2021-11-21T21:54:22.713554+0000","last_unstale":"2021-11-21T22:03:34.289171+0000","last_undegraded":"2021-11-21T22:03:34.289171+0000","last_fullsized":"2021-11-21T22:03:34.289171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2021-11-21T22:03:34.290472+0000","last_change":"2021-11-21T21:54:22.714589+0000","last_active":"2021-11-21T22:03:34.290472+0000","last_peered":"2021-11-21T22:03:34.290472+0000","last_clean":"2021-11-21T22:03:34.290472+0000","last_became_active":"2021-11-21T21:54:22.714383+0000","last_became_peered":"2021-11-21T21:54:22.714383+0000","last_unstale":"2021-11-21T22:03:34.290472+0000","last_undegraded":"2021-11-21T22:03:34.290472+0000","last_fullsized":"2021-11-21T22:03:34.290472+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T21:54:21.693142+0000","last_clean_scrub_stamp":"2021-11-21T21:54:21.693142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3638,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3726197,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.333,"15min":1.335},"min":{"1min":0.753,"5min":0.65700000000000003,"15min":0.57699999999999996},"max":{"1min":1.532,"5min":1.841,"15min":1.841},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3109999999999999,"15min":1.3080000000000001},"min":{"1min":0.69299999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7030000000000001,"5min":1.921,"15min":1.921},"last":0.746}]},{"osd":1,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.254,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.60699999999999998,"15min":0.56000000000000005},"max":{"1min":1.546,"5min":1.7010000000000001,"15min":1.724},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2490000000000001,"15min":1.228},"min":{"1min":0.79600000000000004,"5min":0.59299999999999997,"15min":0.56499999999999995},"max":{"1min":1.5780000000000001,"5min":1.696,"15min":1.7070000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2849999999999999,"15min":1.278},"min":{"1min":0.70599999999999996,"5min":0.64300000000000002,"15min":0.623},"max":{"1min":1.6020000000000001,"5min":1.97,"15min":1.97},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2869999999999999,"15min":1.302},"min":{"1min":0.76800000000000002,"5min":0.63300000000000001,"15min":0.59799999999999998},"max":{"1min":1.6140000000000001,"5min":1.881,"15min":1.921},"last":0.752}]},{"osd":3,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.68799999999999994,"5min":0.624,"15min":0.55300000000000005},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.6830000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.202,"5min":1.2689999999999999,"15min":1.24},"min":{"1min":0.78100000000000003,"5min":0.64700000000000002,"15min":0.624},"max":{"1min":1.5509999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.73099999999999998}]},{"osd":4,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84199999999999997,"15min":0.88},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.23899999999999999},"max":{"1min":1.284,"5min":1.5940000000000001,"15min":1.728},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.36799999999999999,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":1.3340000000000001,"5min":1.3540000000000001,"15min":1.3819999999999999},"last":0.40200000000000002}]},{"osd":5,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89500000000000002,"15min":0.92800000000000005},"min":{"1min":0.33800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.6559999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98099999999999998,"15min":0.98599999999999999},"min":{"1min":0.307,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":1.472,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.44900000000000001}]},{"osd":6,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.028,"15min":1.028},"min":{"1min":0.33100000000000002,"5min":0.313,"15min":0.253},"max":{"1min":1.395,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.373},{"interface":"front","average":{"1min":1.02,"5min":1.01,"15min":1.02},"min":{"1min":0.38900000000000001,"5min":0.26700000000000002,"15min":0.23799999999999999},"max":{"1min":1.3560000000000001,"5min":1.744,"15min":1.744},"last":0.438}]}]},{"osd":6,"up_from":13,"seq":55834575509,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":4048,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4145786,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3069999999999999,"15min":1.2789999999999999},"min":{"1min":0.871,"5min":0.372,"15min":0.372},"max":{"1min":1.5009999999999999,"5min":1.7609999999999999,"15min":1.8029999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.75700000000000001,"5min":0.442,"15min":0.38100000000000001},"max":{"1min":1.619,"5min":1.708,"15min":1.708},"last":1.5329999999999999}]},{"osd":1,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.294,"15min":1.28},"min":{"1min":0.84699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.639,"5min":1.708,"15min":1.7130000000000001},"last":1.163},{"interface":"front","average":{"1min":1.238,"5min":1.294,"15min":1.266},"min":{"1min":0.77300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5389999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.113}]},{"osd":2,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3120000000000001,"15min":1.298},"min":{"1min":0.82099999999999995,"5min":0.435,"15min":0.34599999999999997},"max":{"1min":1.7090000000000001,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.724},{"interface":"front","average":{"1min":1.194,"5min":1.25,"15min":1.2410000000000001},"min":{"1min":0.79800000000000004,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4950000000000001,"5min":1.843,"15min":1.903},"last":1.266}]},{"osd":3,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2310000000000001,"15min":1.2170000000000001},"min":{"1min":0.77800000000000002,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.738},"last":1.101},{"interface":"front","average":{"1min":1.171,"5min":1.232,"15min":1.21},"min":{"1min":0.76600000000000001,"5min":0.437,"15min":0.34699999999999998},"max":{"1min":1.476,"5min":1.657,"15min":1.8999999999999999},"last":1.6299999999999999}]},{"osd":4,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.83999999999999997},"min":{"1min":0.32000000000000001,"5min":0.25,"15min":0.187},"max":{"1min":1.0089999999999999,"5min":1.417,"15min":1.621},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91400000000000003,"15min":0.90800000000000003},"min":{"1min":0.34899999999999998,"5min":0.25900000000000001,"15min":0.217},"max":{"1min":1.1910000000000001,"5min":1.26,"15min":1.7110000000000001},"last":1.0249999999999999}]},{"osd":5,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0700000000000001,"15min":1.0469999999999999},"min":{"1min":0.42399999999999999,"5min":0.28199999999999997,"15min":0.19700000000000001},"max":{"1min":1.5249999999999999,"5min":1.55,"15min":1.7150000000000001},"last":1.496},{"interface":"front","average":{"1min":0.871,"5min":0.92300000000000004,"15min":0.91200000000000003},"min":{"1min":0.374,"5min":0.25600000000000001,"15min":0.192},"max":{"1min":1.645,"5min":1.669,"15min":1.669},"last":0.90500000000000003}]},{"osd":7,"last update":"Sun Nov 21 22:48:01 2021","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94899999999999995,"15min":0.93300000000000005},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.20899999999999999},"max":{"1min":1.5820000000000001,"5min":1.6479999999999999,"15min":1.77},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1419999999999999,"15min":1.1140000000000001},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.25700000000000001},"max":{"1min":1.4530000000000001,"5min":1.5920000000000001,"15min":1.7769999999999999},"last":1.3460000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":2746,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812147,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.2110000000000001,"15min":1.1890000000000001},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.29699999999999999},"max":{"1min":2.012,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2789999999999999,"15min":1.2390000000000001},"min":{"1min":0.46300000000000002,"5min":0.441,"15min":0.36399999999999999},"max":{"1min":1.9710000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3160000000000001,"15min":1.3089999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.36799999999999999},"max":{"1min":1.637,"5min":1.946,"15min":1.946},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.25,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.379},"max":{"1min":1.6839999999999999,"5min":2,"15min":2},"last":0.74099999999999999}]},{"osd":2,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.2110000000000001,"15min":1.2030000000000001},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.33800000000000002},"max":{"1min":1.5209999999999999,"5min":1.7689999999999999,"15min":1.8129999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.147,"15min":1.1679999999999999},"min":{"1min":0.46899999999999997,"5min":0.36299999999999999,"15min":0.34399999999999997},"max":{"1min":1.4259999999999999,"5min":1.456,"15min":1.7170000000000001},"last":0.83699999999999997}]},{"osd":3,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40899999999999997},"max":{"1min":1.4870000000000001,"5min":1.724,"15min":1.875},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.48999999999999999,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":1.665,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.217,"5min":0.217,"15min":0.20699999999999999},"max":{"1min":0.93999999999999995,"5min":1.278,"15min":1.3560000000000001},"last":0.217},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.22500000000000001},"max":{"1min":1.1000000000000001,"5min":1.5129999999999999,"15min":1.73},"last":0.29799999999999999}]},{"osd":6,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91400000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.20300000000000001},"max":{"1min":1.1439999999999999,"5min":1.3380000000000001,"15min":1.552},"last":0.247},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.93100000000000005},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":1.454,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.33300000000000002}]},{"osd":7,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":1.0109999999999999,"15min":0.996},"min":{"1min":0.31,"5min":0.26500000000000001,"15min":0.19400000000000001},"max":{"1min":1.429,"5min":1.5620000000000001,"15min":1.74},"last":0.317},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.318,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.252,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.32100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":762944,"kb_used_data":87274432,"kb_used_omap":3477,"kb_used_meta":0,"kb_avail":87274432,"statfs":{"total":90150273024,"available":89369018368,"internally_reserved":0,"allocated":89369018368,"data_stored":89369018368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3561237,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.292,"15min":1.2410000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.53200000000000003},"max":{"1min":1.552,"5min":1.718,"15min":1.718},"last":1.462},{"interface":"front","average":{"1min":1.286,"5min":1.2569999999999999,"15min":1.24},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.56200000000000006},"max":{"1min":1.6220000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.4430000000000001}]},{"osd":1,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.304,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.61099999999999999},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.75},"last":1.391},{"interface":"front","average":{"1min":1.222,"5min":1.2450000000000001,"15min":1.218},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.59199999999999997},"max":{"1min":1.4430000000000001,"5min":1.601,"15min":1.7410000000000001},"last":1.175}]},{"osd":2,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.24,"15min":1.2350000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.54800000000000004},"max":{"1min":1.579,"5min":1.579,"15min":2.2629999999999999},"last":1.266},{"interface":"front","average":{"1min":1.353,"5min":1.3640000000000001,"15min":1.3220000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.58599999999999997},"max":{"1min":1.673,"5min":1.7569999999999999,"15min":2.226},"last":1.488}]},{"osd":3,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2769999999999999,"15min":1.2330000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.54300000000000004},"max":{"1min":1.7210000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.57399999999999995},"max":{"1min":1.492,"5min":1.992,"15min":1.992},"last":1.3080000000000001}]},{"osd":5,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.91300000000000003,"15min":0.86199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23000000000000001},"max":{"1min":1.123,"5min":1.28,"15min":1.5720000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97999999999999998,"15min":0.91500000000000004},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":1.5569999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.204}]},{"osd":6,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":0.88300000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.21099999999999999},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0129999999999999,"15min":0.96999999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23699999999999999},"max":{"1min":1.4319999999999999,"5min":1.5349999999999999,"15min":1.5780000000000001},"last":1.3520000000000001}]},{"osd":7,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1160000000000001,"15min":1.0609999999999999},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.24099999999999999},"max":{"1min":1.71,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.03},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.071,"15min":0.98699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23400000000000001,"15min":0.22800000000000001},"max":{"1min":1.2909999999999999,"5min":1.609,"15min":1.716},"last":1.149}]}]},{"osd":3,"up_from":13,"seq":55834575507,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3422,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3504138,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.38800000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.93999999999999995,"5min":1.034,"15min":1.161},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.35999999999999999,"5min":0.249,"15min":0.21199999999999999},"max":{"1min":0.95199999999999996,"5min":1.0469999999999999,"15min":1.0569999999999999},"last":0.84699999999999998}]},{"osd":1,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73499999999999999,"15min":0.746},"min":{"1min":0.33900000000000002,"5min":0.23000000000000001,"15min":0.19400000000000001},"max":{"1min":0.93400000000000005,"5min":1.238,"15min":1.238},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.80100000000000005,"15min":0.79600000000000004},"min":{"1min":0.42399999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.2130000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.90300000000000002}]},{"osd":2,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.38100000000000001,"5min":0.22800000000000001,"15min":0.20999999999999999},"max":{"1min":1.1080000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.012},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.91600000000000004,"15min":0.90100000000000002},"min":{"1min":0.44900000000000001,"5min":0.248,"15min":0.23100000000000001},"max":{"1min":1.353,"5min":1.3660000000000001,"15min":1.6359999999999999},"last":1.0009999999999999}]},{"osd":4,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3080000000000001,"15min":1.349},"min":{"1min":0.79400000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.9730000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.90500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5580000000000001,"5min":1.7809999999999999,"15min":1.833},"last":1.45}]},{"osd":5,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.323,"15min":1.327},"min":{"1min":0.97199999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.593,"5min":1.843,"15min":1.917},"last":1.583},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3460000000000001,"15min":1.355},"min":{"1min":0.92000000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.7130000000000001,"5min":1.7789999999999999,"15min":1.9470000000000001},"last":1.405}]},{"osd":6,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.333,"15min":1.349},"min":{"1min":1.03,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6870000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.379},{"interface":"front","average":{"1min":1.347,"5min":1.349,"15min":1.3340000000000001},"min":{"1min":1.127,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.657,"5min":1.7,"15min":1.891},"last":1.482}]},{"osd":7,"last update":"Sun Nov 21 22:48:09 2021","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3440000000000001,"15min":1.3480000000000001},"min":{"1min":1.075,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.359},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.375,"15min":1.3799999999999999},"min":{"1min":0.93600000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.651,"5min":1.8839999999999999,"15min":1.8999999999999999},"last":1.5449999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":2820,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2887910,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70099999999999996,"15min":0.70999999999999996},"min":{"1min":0.29299999999999998,"5min":0.221,"15min":0.19400000000000001},"max":{"1min":0.96999999999999997,"5min":1.1000000000000001,"15min":1.351},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.32800000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.97099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.03}]},{"osd":1,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.223,"15min":0.188},"max":{"1min":1.075,"5min":1.075,"15min":1.1519999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84799999999999998,"15min":0.86499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1479999999999999,"5min":1.155,"15min":1.5820000000000001},"last":0.77500000000000002}]},{"osd":3,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.79800000000000004},"min":{"1min":0.30399999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.115,"5min":1.319,"15min":1.319},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81000000000000005,"15min":0.84499999999999997},"min":{"1min":0.40000000000000002,"5min":0.218,"15min":0.20499999999999999},"max":{"1min":1.0609999999999999,"5min":1.302,"15min":1.302},"last":0.81799999999999995}]},{"osd":4,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.296,"15min":1.357},"min":{"1min":0.94499999999999995,"5min":0.629,"15min":0.503},"max":{"1min":1.6319999999999999,"5min":1.7230000000000001,"15min":1.8420000000000001},"last":1.319},{"interface":"front","average":{"1min":1.345,"5min":1.331,"15min":1.377},"min":{"1min":0.86399999999999999,"5min":0.66800000000000004,"15min":0.58999999999999997},"max":{"1min":1.716,"5min":1.829,"15min":2.0710000000000002},"last":1.4750000000000001}]},{"osd":5,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.367,"15min":1.409},"min":{"1min":0.84399999999999997,"5min":0.62,"15min":0.61299999999999999},"max":{"1min":1.6659999999999999,"5min":1.901,"15min":1.901},"last":1.46},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.423},"min":{"1min":0.82999999999999996,"5min":0.65600000000000003,"15min":0.55500000000000005},"max":{"1min":1.7430000000000001,"5min":1.784,"15min":1.913},"last":1.5529999999999999}]},{"osd":6,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3819999999999999,"15min":1.4259999999999999},"min":{"1min":0.88700000000000001,"5min":0.68200000000000005,"15min":0.57599999999999996},"max":{"1min":1.8240000000000001,"5min":1.861,"15min":1.861},"last":1.605},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3360000000000001,"15min":1.377},"min":{"1min":0.81799999999999995,"5min":0.64800000000000002,"15min":0.54900000000000004},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":2.512},"last":1.2929999999999999}]},{"osd":7,"last update":"Sun Nov 21 22:48:04 2021","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3540000000000001,"15min":1.379},"min":{"1min":0.98699999999999999,"5min":0.63700000000000001,"15min":0.59599999999999997},"max":{"1min":1.6719999999999999,"5min":1.8009999999999999,"15min":1.925},"last":1.383},{"interface":"front","average":{"1min":1.264,"5min":1.274,"15min":1.3300000000000001},"min":{"1min":0.89700000000000002,"5min":0.625,"15min":0.51100000000000001},"max":{"1min":1.7669999999999999,"5min":1.7689999999999999,"15min":1.806},"last":1.3580000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575507,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3806,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3897559,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":1.254,"5min":1.278,"15min":1.3069999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79600000000000004,"15min":0.81699999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.224},"max":{"1min":0.98699999999999999,"5min":1.093,"15min":1.4730000000000001},"last":0.86699999999999999}]},{"osd":2,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.29599999999999999,"5min":0.28299999999999997,"15min":0.19500000000000001},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.82899999999999996,"15min":0.81100000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.19},"max":{"1min":0.97699999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.107}]},{"osd":3,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.17599999999999999},"max":{"1min":1.135,"5min":1.423,"15min":1.7010000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.27500000000000002,"5min":0.26900000000000002,"15min":0.20899999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0449999999999999}]},{"osd":4,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.345,"15min":1.3540000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.643,"5min":1.792,"15min":1.9990000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.361},"min":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.30099999999999999},"max":{"1min":1.6319999999999999,"5min":1.929,"15min":1.9299999999999999},"last":1.321}]},{"osd":5,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4219999999999999},"min":{"1min":0.66800000000000004,"5min":0.54500000000000004,"15min":0.28999999999999998},"max":{"1min":1.7609999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.4019999999999999,"15min":1.3899999999999999},"min":{"1min":0.60399999999999998,"5min":0.54800000000000004,"15min":0.35499999999999998},"max":{"1min":1.661,"5min":1.839,"15min":1.875},"last":1.3640000000000001}]},{"osd":6,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.3540000000000001},"min":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.27800000000000002},"max":{"1min":1.6779999999999999,"5min":1.6919999999999999,"15min":1.8879999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.345,"15min":1.357},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":1.641,"5min":1.8560000000000001,"15min":1.915},"last":1.3939999999999999}]},{"osd":7,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.353,"15min":1.375},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.7689999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.292},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3700000000000001,"15min":1.361},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":1.655,"5min":1.819,"15min":1.9350000000000001},"last":1.5249999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3915,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4009734,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.27800000000000002,"5min":0.22500000000000001,"15min":0.186},"max":{"1min":0.95899999999999996,"5min":1.1100000000000001,"15min":1.1299999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77300000000000002,"15min":0.754},"min":{"1min":0.217,"5min":0.217,"15min":0.21199999999999999},"max":{"1min":0.97999999999999998,"5min":1.0880000000000001,"15min":1.327},"last":0.875}]},{"osd":2,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72799999999999998,"15min":0.72099999999999997},"min":{"1min":0.22600000000000001,"5min":0.217,"15min":0.216},"max":{"1min":0.96299999999999997,"5min":1.1160000000000001,"15min":1.25},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.20100000000000001},"max":{"1min":0.97199999999999998,"5min":1.161,"15min":1.161},"last":0.83499999999999996}]},{"osd":3,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.92600000000000005,"15min":0.89400000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.24099999999999999},"max":{"1min":1.2190000000000001,"5min":1.3460000000000001,"15min":1.6299999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.216},"max":{"1min":1.016,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3140000000000001,"15min":1.3120000000000001},"min":{"1min":0.70999999999999996,"5min":0.40200000000000002,"15min":0.34599999999999997},"max":{"1min":1.663,"5min":1.6990000000000001,"15min":1.77},"last":1.415},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.365,"15min":1.3520000000000001},"min":{"1min":0.67700000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.712,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.321}]},{"osd":5,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.361,"15min":1.367},"min":{"1min":0.64800000000000002,"5min":0.41399999999999998,"15min":0.38100000000000001},"max":{"1min":1.657,"5min":1.9370000000000001,"15min":1.9399999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3959999999999999,"15min":1.3620000000000001},"min":{"1min":0.78500000000000003,"5min":0.35599999999999998,"15min":0.34599999999999997},"max":{"1min":1.8640000000000001,"5min":1.887,"15min":2.141},"last":1.288}]},{"osd":6,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3320000000000001,"15min":1.3180000000000001},"min":{"1min":0.69499999999999995,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":1.5940000000000001,"5min":1.8,"15min":1.8400000000000001},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.399,"15min":1.3520000000000001},"min":{"1min":0.86399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.7769999999999999,"5min":1.903,"15min":1.903},"last":1.474}]},{"osd":7,"last update":"Sun Nov 21 22:48:08 2021","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.367,"15min":1.361},"min":{"1min":0.748,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":1.766,"5min":1.7989999999999999,"15min":1.921},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.4039999999999999,"15min":1.369},"min":{"1min":0.66300000000000003,"5min":0.36499999999999999,"15min":0.35999999999999999},"max":{"1min":1.841,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.383}]}]}]}} 2021-11-21T22:49:08.274 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2021-11-21T21:54:13.901475+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=13, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.275 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.275 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.276 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.276 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.277 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.278 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.278 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.279 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2021-11-21T21:54:21.693142+0000 time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=21, tm_min=54, tm_sec=21, tm_wday=6, tm_yday=325, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=11, tm_mday=21, tm_hour=22, tm_min=49, tm_sec=5, tm_wday=6, tm_yday=325, tm_isdst=0) 2021-11-21T22:49:08.280 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2021-11-21T22:49:10.698 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:10.702 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:10.705 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:10.708 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:28.281 DEBUG:teuthology.orchestra.run.gibba016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-11-21T22:49:28.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 Processor -- start 2021-11-21T22:49:28.338 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- start start 2021-11-21T22:49:28.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc8280a6aa0 0x7fc82814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a86b0 0x7fc8281470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.339 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- --> v1:172.21.2.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8280a6e80 con 0x7fc8281475d0 2021-11-21T22:49:28.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8280a8bc0 con 0x7fc8280a6aa0 2021-11-21T22:49:28.340 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82814a7b0 con 0x7fc8280a86b0 2021-11-21T22:49:28.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc8280a6aa0 0x7fc82814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:28.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7fc8280a6aa0 0x7fc82814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55338/0 (socket says 172.21.2.116:55338) 2021-11-21T22:49:28.341 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 -- 172.21.2.116:0/1740433475 learned_addr learned my addr 172.21.2.116:0/1740433475 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:28.345 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 -- 172.21.2.116:0/1740433475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a86b0 msgr2=0x7fc8281470e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:28.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 --2- 172.21.2.116:0/1740433475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a86b0 0x7fc8281470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 -- 172.21.2.116:0/1740433475 >> v1:172.21.2.117:6789/0 conn(0x7fc8281475d0 legacy=0x7fc8281443f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:28.346 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 -- 172.21.2.116:0/1740433475 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82814aa10 con 0x7fc8280a6aa0 2021-11-21T22:49:28.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82d211700 1 --2- 172.21.2.116:0/1740433475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8280a6aa0 0x7fc82814b150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 rx=0x7fc828061ab0 tx=0x7fc824006eb0).ready entity=mon.0 client_cookie=b527700a0bbe47e6 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:28.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc81bfff700 1 -- 172.21.2.116:0/1740433475 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc82400d570 con 0x7fc8280a6aa0 2021-11-21T22:49:28.347 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc81bfff700 1 -- 172.21.2.116:0/1740433475 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824000e90 con 0x7fc8280a6aa0 2021-11-21T22:49:28.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc81bfff700 1 -- 172.21.2.116:0/1740433475 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc82400d8c0 con 0x7fc8280a6aa0 2021-11-21T22:49:28.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- 172.21.2.116:0/1740433475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8280a6aa0 msgr2=0x7fc82814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.348 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- 172.21.2.116:0/1740433475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8280a6aa0 0x7fc82814b150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 rx=0x7fc828061ab0 tx=0x7fc824006eb0).stop 2021-11-21T22:49:28.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- 172.21.2.116:0/1740433475 shutdown_connections 2021-11-21T22:49:28.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- 172.21.2.116:0/1740433475 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a86b0 0x7fc8281470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.349 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- 172.21.2.116:0/1740433475 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8280a6aa0 0x7fc82814b150 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:28.350 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- 172.21.2.116:0/1740433475 >> 172.21.2.116:0/1740433475 conn(0x7fc82805bd20 msgr2=0x7fc82805c130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:28.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- 172.21.2.116:0/1740433475 shutdown_connections 2021-11-21T22:49:28.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- 172.21.2.116:0/1740433475 wait complete. 2021-11-21T22:49:28.352 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 Processor -- start 2021-11-21T22:49:28.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- start start 2021-11-21T22:49:28.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a6aa0 0x7fc828065da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.353 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc8280a86b0 0x7fc828156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8281475d0 0x7fc8280a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828065be0 con 0x7fc8281475d0 2021-11-21T22:49:28.354 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828156510 con 0x7fc8280a86b0 2021-11-21T22:49:28.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.331+0000 7fc82f475700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8281567b0 con 0x7fc8280a6aa0 2021-11-21T22:49:28.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8281475d0 0x7fc8280a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:28.355 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8281475d0 0x7fc8280a8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55348/0 (socket says 172.21.2.116:55348) 2021-11-21T22:49:28.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 -- 172.21.2.116:0/220684826 learned_addr learned my addr 172.21.2.116:0/220684826 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:28.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 -- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a6aa0 msgr2=0x7fc828065da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.356 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a6aa0 0x7fc828065da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 -- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc8280a86b0 msgr2=0x7fc828156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc8280a86b0 0x7fc828156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.357 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 -- 172.21.2.116:0/220684826 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82400b1e0 con 0x7fc8281475d0 2021-11-21T22:49:28.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82da12700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8281475d0 0x7fc8280a8ec0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fc820001250 tx=0x7fc8200090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:28.358 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc81a7fc700 1 -- 172.21.2.116:0/220684826 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8200014e0 con 0x7fc8281475d0 2021-11-21T22:49:28.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc81a7fc700 1 -- 172.21.2.116:0/220684826 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc820001640 con 0x7fc8281475d0 2021-11-21T22:49:28.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc828156a50 con 0x7fc8281475d0 2021-11-21T22:49:28.359 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc828146a10 con 0x7fc8281475d0 2021-11-21T22:49:28.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc81a7fc700 1 -- 172.21.2.116:0/220684826 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8200019a0 con 0x7fc8281475d0 2021-11-21T22:49:28.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc804003410 con 0x7fc8281475d0 2021-11-21T22:49:28.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc81a7fc700 1 -- 172.21.2.116:0/220684826 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7fc820018030 con 0x7fc8281475d0 2021-11-21T22:49:28.360 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc81a7fc700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc808053610 0x7fc808055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc81a7fc700 1 -- 172.21.2.116:0/220684826 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc82006b0b0 con 0x7fc8281475d0 2021-11-21T22:49:28.361 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.335+0000 7fc82d211700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc808053610 0x7fc808055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:28.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.339+0000 7fc82d211700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc808053610 0x7fc808055ad0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fc828061ab0 tx=0x7fc824015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:28.362 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.339+0000 7fc81a7fc700 1 -- 172.21.2.116:0/220684826 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc820042490 con 0x7fc8281475d0 2021-11-21T22:49:28.425 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.419+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 --> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc804002070 con 0x7fc808053610 2021-11-21T22:49:28.433 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc81a7fc700 1 -- 172.21.2.116:0/220684826 <== mgr.4100 v2:172.21.2.117:6832/14582 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66516 (secure 0 0 0) 0x7fc804002070 con 0x7fc808053610 2021-11-21T22:49:28.434 INFO:teuthology.orchestra.run.gibba016.stdout: 2021-11-21T22:49:28.435 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc808053610 msgr2=0x7fc808055ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc808053610 0x7fc808055ad0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fc828061ab0 tx=0x7fc824015000).stop 2021-11-21T22:49:28.436 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8281475d0 msgr2=0x7fc8280a8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8281475d0 0x7fc8280a8ec0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fc820001250 tx=0x7fc8200090f0).stop 2021-11-21T22:49:28.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 shutdown_connections 2021-11-21T22:49:28.437 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7fc808053610 0x7fc808055ad0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:28.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7fc8281475d0 0x7fc8280a8ec0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:28.438 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7fc8280a86b0 0x7fc828156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 --2- 172.21.2.116:0/220684826 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7fc8280a6aa0 0x7fc828065da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 >> 172.21.2.116:0/220684826 conn(0x7fc82805bd20 msgr2=0x7fc828149b00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:28.439 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 shutdown_connections 2021-11-21T22:49:28.440 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.427+0000 7fc82f475700 1 -- 172.21.2.116:0/220684826 wait complete. 2021-11-21T22:49:28.440 INFO:teuthology.orchestra.run.gibba016.stderr:dumped all 2021-11-21T22:49:28.441 INFO:teuthology.orchestra.run.gibba016.stdout:{"pg_ready":true,"pg_map":{"version":1813,"stamp":"2021-11-21T22:49:26.438453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9848,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":704299008,"kb_used":6104472,"kb_used_data":698194560,"kb_used_omap":27897,"kb_used_meta":0,"kb_avail":698194536,"statfs":{"total":721202184192,"available":714951204864,"internally_reserved":24576,"allocated":714951229440,"data_stored":714951229440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28567176,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9848,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005076"},"pg_stats":[{"pgid":"1.0","version":"165'4","reported_seq":143,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:17.018495+0000","last_change":"2021-11-21T22:49:17.018617+0000","last_active":"2021-11-21T22:49:17.018495+0000","last_peered":"2021-11-21T22:49:17.018495+0000","last_clean":"2021-11-21T22:49:17.018495+0000","last_became_active":"2021-11-21T21:54:16.981409+0000","last_became_peered":"2021-11-21T21:54:16.981409+0000","last_unstale":"2021-11-21T22:49:17.018495+0000","last_undegraded":"2021-11-21T22:49:17.018495+0000","last_fullsized":"2021-11-21T22:49:17.018495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'4","last_scrub_stamp":"2021-11-21T22:49:17.018495+0000","last_deep_scrub":"165'4","last_deep_scrub_stamp":"2021-11-21T22:49:17.018495+0000","last_clean_scrub_stamp":"2021-11-21T22:49:17.018495+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9848,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:18.548401+0000","last_change":"2021-11-21T22:49:18.548447+0000","last_active":"2021-11-21T22:49:18.548401+0000","last_peered":"2021-11-21T22:49:18.548401+0000","last_clean":"2021-11-21T22:49:18.548401+0000","last_became_active":"2021-11-21T21:54:22.713118+0000","last_became_peered":"2021-11-21T21:54:22.713118+0000","last_unstale":"2021-11-21T22:49:18.548401+0000","last_undegraded":"2021-11-21T22:49:18.548401+0000","last_fullsized":"2021-11-21T22:49:18.548401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:18.548401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:18.548401+0000","last_clean_scrub_stamp":"2021-11-21T22:49:18.548401+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:19.979196+0000","last_change":"2021-11-21T22:49:19.979315+0000","last_active":"2021-11-21T22:49:19.979196+0000","last_peered":"2021-11-21T22:49:19.979196+0000","last_clean":"2021-11-21T22:49:19.979196+0000","last_became_active":"2021-11-21T21:54:22.712862+0000","last_became_peered":"2021-11-21T21:54:22.712862+0000","last_unstale":"2021-11-21T22:49:19.979196+0000","last_undegraded":"2021-11-21T22:49:19.979196+0000","last_fullsized":"2021-11-21T22:49:19.979196+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:19.979195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:19.979195+0000","last_clean_scrub_stamp":"2021-11-21T22:49:19.979195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:08.127240+0000","last_change":"2021-11-21T22:49:08.127274+0000","last_active":"2021-11-21T22:49:08.127240+0000","last_peered":"2021-11-21T22:49:08.127240+0000","last_clean":"2021-11-21T22:49:08.127240+0000","last_became_active":"2021-11-21T21:54:22.711687+0000","last_became_peered":"2021-11-21T21:54:22.711687+0000","last_unstale":"2021-11-21T22:49:08.127240+0000","last_undegraded":"2021-11-21T22:49:08.127240+0000","last_fullsized":"2021-11-21T22:49:08.127240+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:08.127239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:08.127239+0000","last_clean_scrub_stamp":"2021-11-21T22:49:08.127239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:16.531625+0000","last_change":"2021-11-21T22:49:16.531653+0000","last_active":"2021-11-21T22:49:16.531625+0000","last_peered":"2021-11-21T22:49:16.531625+0000","last_clean":"2021-11-21T22:49:16.531625+0000","last_became_active":"2021-11-21T21:54:22.713254+0000","last_became_peered":"2021-11-21T21:54:22.713254+0000","last_unstale":"2021-11-21T22:49:16.531625+0000","last_undegraded":"2021-11-21T22:49:16.531625+0000","last_fullsized":"2021-11-21T22:49:16.531625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:16.531624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:16.531624+0000","last_clean_scrub_stamp":"2021-11-21T22:49:16.531624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":147,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:07.527801+0000","last_change":"2021-11-21T22:49:07.527840+0000","last_active":"2021-11-21T22:49:07.527801+0000","last_peered":"2021-11-21T22:49:07.527801+0000","last_clean":"2021-11-21T22:49:07.527801+0000","last_became_active":"2021-11-21T21:54:22.786041+0000","last_became_peered":"2021-11-21T21:54:22.786041+0000","last_unstale":"2021-11-21T22:49:07.527801+0000","last_undegraded":"2021-11-21T22:49:07.527801+0000","last_fullsized":"2021-11-21T22:49:07.527801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:07.527801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:07.527801+0000","last_clean_scrub_stamp":"2021-11-21T22:49:07.527801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:21.972594+0000","last_change":"2021-11-21T22:49:21.972723+0000","last_active":"2021-11-21T22:49:21.972594+0000","last_peered":"2021-11-21T22:49:21.972594+0000","last_clean":"2021-11-21T22:49:21.972594+0000","last_became_active":"2021-11-21T21:54:22.787305+0000","last_became_peered":"2021-11-21T21:54:22.787305+0000","last_unstale":"2021-11-21T22:49:21.972594+0000","last_undegraded":"2021-11-21T22:49:21.972594+0000","last_fullsized":"2021-11-21T22:49:21.972594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:21.972594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:21.972594+0000","last_clean_scrub_stamp":"2021-11-21T22:49:21.972594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":147,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:09.528329+0000","last_change":"2021-11-21T22:49:09.528355+0000","last_active":"2021-11-21T22:49:09.528329+0000","last_peered":"2021-11-21T22:49:09.528329+0000","last_clean":"2021-11-21T22:49:09.528329+0000","last_became_active":"2021-11-21T21:54:22.713554+0000","last_became_peered":"2021-11-21T21:54:22.713554+0000","last_unstale":"2021-11-21T22:49:09.528329+0000","last_undegraded":"2021-11-21T22:49:09.528329+0000","last_fullsized":"2021-11-21T22:49:09.528329+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:09.528329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:09.528329+0000","last_clean_scrub_stamp":"2021-11-21T22:49:09.528329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":165,"state":"active+clean","last_fresh":"2021-11-21T22:49:09.618301+0000","last_change":"2021-11-21T22:49:09.618377+0000","last_active":"2021-11-21T22:49:09.618301+0000","last_peered":"2021-11-21T22:49:09.618301+0000","last_clean":"2021-11-21T22:49:09.618301+0000","last_became_active":"2021-11-21T21:54:22.714383+0000","last_became_peered":"2021-11-21T21:54:22.714383+0000","last_unstale":"2021-11-21T22:49:09.618301+0000","last_undegraded":"2021-11-21T22:49:09.618301+0000","last_fullsized":"2021-11-21T22:49:09.618301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-21T22:49:09.618300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-21T22:49:09.618300+0000","last_clean_scrub_stamp":"2021-11-21T22:49:09.618300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9848,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575511,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763080,"kb_used_data":87274304,"kb_used_omap":3644,"kb_used_meta":0,"kb_avail":87274296,"statfs":{"total":90150273024,"available":89368879104,"internally_reserved":8192,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3731507,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.333,"15min":1.335},"min":{"1min":0.753,"5min":0.65700000000000003,"15min":0.57699999999999996},"max":{"1min":1.532,"5min":1.841,"15min":1.841},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3109999999999999,"15min":1.3080000000000001},"min":{"1min":0.69299999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7030000000000001,"5min":1.921,"15min":1.921},"last":1.5069999999999999}]},{"osd":1,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.254,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.60699999999999998,"15min":0.56000000000000005},"max":{"1min":1.546,"5min":1.7010000000000001,"15min":1.724},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2490000000000001,"15min":1.228},"min":{"1min":0.79600000000000004,"5min":0.59299999999999997,"15min":0.56499999999999995},"max":{"1min":1.5780000000000001,"5min":1.696,"15min":1.7070000000000001},"last":1.403}]},{"osd":2,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2849999999999999,"15min":1.278},"min":{"1min":0.70599999999999996,"5min":0.64300000000000002,"15min":0.623},"max":{"1min":1.6020000000000001,"5min":1.97,"15min":1.97},"last":1.365},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2869999999999999,"15min":1.302},"min":{"1min":0.76800000000000002,"5min":0.63300000000000001,"15min":0.59799999999999998},"max":{"1min":1.6140000000000001,"5min":1.881,"15min":1.921},"last":1.1160000000000001}]},{"osd":3,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.68799999999999994,"5min":0.624,"15min":0.55300000000000005},"max":{"1min":1.645,"5min":1.6739999999999999,"15min":1.6830000000000001},"last":1.534},{"interface":"front","average":{"1min":1.202,"5min":1.2689999999999999,"15min":1.24},"min":{"1min":0.78100000000000003,"5min":0.64700000000000002,"15min":0.624},"max":{"1min":1.5509999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.3100000000000001}]},{"osd":4,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84199999999999997,"15min":0.88},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.23899999999999999},"max":{"1min":1.284,"5min":1.5940000000000001,"15min":1.728},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.92000000000000004,"15min":0.92100000000000004},"min":{"1min":0.36799999999999999,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":1.3340000000000001,"5min":1.3540000000000001,"15min":1.3819999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89500000000000002,"15min":0.92800000000000005},"min":{"1min":0.33800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.6559999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98099999999999998,"15min":0.98599999999999999},"min":{"1min":0.307,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":1.472,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Sun Nov 21 22:48:55 2021","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.028,"15min":1.028},"min":{"1min":0.33100000000000002,"5min":0.313,"15min":0.253},"max":{"1min":1.395,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.081},{"interface":"front","average":{"1min":1.02,"5min":1.01,"15min":1.02},"min":{"1min":0.38900000000000001,"5min":0.26700000000000002,"15min":0.23799999999999999},"max":{"1min":1.3560000000000001,"5min":1.744,"15min":1.744},"last":1.1919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575513,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":4050,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4148121,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:49:05 2021","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.71699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.714,"5min":1.75,"15min":1.8029999999999999},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.2450000000000001,"15min":1.2509999999999999},"min":{"1min":0.74099999999999999,"5min":0.442,"15min":0.39700000000000002},"max":{"1min":1.5329999999999999,"5min":1.708,"15min":1.708},"last":1.5980000000000001}]},{"osd":1,"last update":"Sun Nov 21 22:49:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.276,"15min":1.2809999999999999},"min":{"1min":0.747,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2490000000000001,"15min":1.262},"min":{"1min":0.72099999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.518,"5min":1.702,"15min":1.7869999999999999},"last":1.425}]},{"osd":2,"last update":"Sun Nov 21 22:49:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3069999999999999,"15min":1.298},"min":{"1min":0.70799999999999996,"5min":0.435,"15min":0.42399999999999999},"max":{"1min":1.724,"5min":1.7689999999999999,"15min":1.8149999999999999},"last":1.526},{"interface":"front","average":{"1min":1.147,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.63200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.5549999999999999,"5min":1.7010000000000001,"15min":1.903},"last":1.147}]},{"osd":3,"last update":"Sun Nov 21 22:49:05 2021","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.2050000000000001,"15min":1.21},"min":{"1min":0.66300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.5700000000000001,"5min":1.645,"15min":1.738},"last":1.236},{"interface":"front","average":{"1min":1.151,"5min":1.2150000000000001,"15min":1.214},"min":{"1min":0.64600000000000002,"5min":0.437,"15min":0.34699999999999998},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.8999999999999999},"last":0.93000000000000005}]},{"osd":4,"last update":"Sun Nov 21 22:49:05 2021","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":0.84099999999999997},"min":{"1min":0.33000000000000002,"5min":0.25,"15min":0.187},"max":{"1min":1.155,"5min":1.417,"15min":1.621},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.90200000000000002,"15min":0.91200000000000003},"min":{"1min":0.28399999999999997,"5min":0.25900000000000001,"15min":0.217},"max":{"1min":1.464,"5min":1.464,"15min":1.7110000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Sun Nov 21 22:49:05 2021","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.048,"15min":1.052},"min":{"1min":0.38100000000000001,"5min":0.28199999999999997,"15min":0.19700000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.7150000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89200000000000002,"15min":0.91800000000000004},"min":{"1min":0.34300000000000003,"5min":0.25600000000000001,"15min":0.192},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Sun Nov 21 22:49:05 2021","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92800000000000005,"15min":0.93600000000000005},"min":{"1min":0.35699999999999998,"5min":0.27400000000000002,"15min":0.20899999999999999},"max":{"1min":1.52,"5min":1.5820000000000001,"15min":1.77},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.25700000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.7769999999999999},"last":1.016}]}]},{"osd":5,"up_from":13,"seq":55834575511,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763080,"kb_used_data":87274304,"kb_used_omap":2748,"kb_used_meta":0,"kb_avail":87274296,"statfs":{"total":90150273024,"available":89368879104,"internally_reserved":8192,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2814409,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.2110000000000001,"15min":1.1890000000000001},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.29699999999999999},"max":{"1min":2.012,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2789999999999999,"15min":1.2390000000000001},"min":{"1min":0.46300000000000002,"5min":0.441,"15min":0.36399999999999999},"max":{"1min":1.9710000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.335}]},{"osd":1,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3160000000000001,"15min":1.3089999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.36799999999999999},"max":{"1min":1.637,"5min":1.946,"15min":1.946},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.379},"max":{"1min":1.6839999999999999,"5min":2,"15min":2},"last":1.3779999999999999}]},{"osd":2,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.2110000000000001,"15min":1.2030000000000001},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.33800000000000002},"max":{"1min":1.5209999999999999,"5min":1.7689999999999999,"15min":1.8129999999999999},"last":1.202},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.147,"15min":1.1679999999999999},"min":{"1min":0.46899999999999997,"5min":0.36299999999999999,"15min":0.34399999999999997},"max":{"1min":1.4259999999999999,"5min":1.456,"15min":1.7170000000000001},"last":0.97999999999999998}]},{"osd":3,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40899999999999997},"max":{"1min":1.4870000000000001,"5min":1.724,"15min":1.875},"last":1.141},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.48999999999999999,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":1.665,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.0549999999999999}]},{"osd":4,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.217,"5min":0.217,"15min":0.20699999999999999},"max":{"1min":0.93999999999999995,"5min":1.278,"15min":1.3560000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.22500000000000001},"max":{"1min":1.1000000000000001,"5min":1.5129999999999999,"15min":1.73},"last":0.88300000000000001}]},{"osd":6,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91400000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.20300000000000001},"max":{"1min":1.1439999999999999,"5min":1.3380000000000001,"15min":1.552},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.93100000000000005},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":1.454,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Sun Nov 21 22:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":1.0109999999999999,"15min":0.996},"min":{"1min":0.31,"5min":0.26500000000000001,"15min":0.19400000000000001},"max":{"1min":1.429,"5min":1.5620000000000001,"15min":1.74},"last":1.4359999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.318,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.252,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.85099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":762944,"kb_used_data":87274432,"kb_used_omap":3477,"kb_used_meta":0,"kb_avail":87274432,"statfs":{"total":90150273024,"available":89369018368,"internally_reserved":0,"allocated":89369018368,"data_stored":89369018368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3561237,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.292,"15min":1.2410000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.53200000000000003},"max":{"1min":1.552,"5min":1.718,"15min":1.718},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.2569999999999999,"15min":1.24},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.56200000000000006},"max":{"1min":1.6220000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.4399999999999999}]},{"osd":1,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.304,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.61099999999999999},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.75},"last":1.389},{"interface":"front","average":{"1min":1.222,"5min":1.2450000000000001,"15min":1.218},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.59199999999999997},"max":{"1min":1.4430000000000001,"5min":1.601,"15min":1.7410000000000001},"last":1.1599999999999999}]},{"osd":2,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.24,"15min":1.2350000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.54800000000000004},"max":{"1min":1.579,"5min":1.579,"15min":2.2629999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.3640000000000001,"15min":1.3220000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.58599999999999997},"max":{"1min":1.673,"5min":1.7569999999999999,"15min":2.226},"last":1.2210000000000001}]},{"osd":3,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2769999999999999,"15min":1.2330000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.54300000000000004},"max":{"1min":1.7210000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.57399999999999995},"max":{"1min":1.492,"5min":1.992,"15min":1.992},"last":1.0509999999999999}]},{"osd":5,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.91300000000000003,"15min":0.86199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23000000000000001},"max":{"1min":1.123,"5min":1.28,"15min":1.5720000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97999999999999998,"15min":0.91500000000000004},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":1.5569999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.119}]},{"osd":6,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":0.88300000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.21099999999999999},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0129999999999999,"15min":0.96999999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23699999999999999},"max":{"1min":1.4319999999999999,"5min":1.5349999999999999,"15min":1.5780000000000001},"last":0.999}]},{"osd":7,"last update":"Sun Nov 21 22:48:45 2021","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1160000000000001,"15min":1.0609999999999999},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.24099999999999999},"max":{"1min":1.71,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.071,"15min":0.98699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23400000000000001,"15min":0.22800000000000001},"max":{"1min":1.2909999999999999,"5min":1.609,"15min":1.716},"last":1.3280000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575511,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3423,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3505367,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:49:09 2021","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.28699999999999998,"5min":0.183,"15min":0.183},"max":{"1min":0.86899999999999999,"5min":1.034,"15min":1.161},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.35799999999999998,"5min":0.249,"15min":0.21199999999999999},"max":{"1min":0.92900000000000005,"5min":1.024,"15min":1.0569999999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Sun Nov 21 22:49:09 2021","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73799999999999999,"15min":0.74299999999999999},"min":{"1min":0.27200000000000002,"5min":0.23000000000000001,"15min":0.19400000000000001},"max":{"1min":0.91200000000000003,"5min":1.238,"15min":1.238},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80800000000000005,"15min":0.79800000000000004},"min":{"1min":0.26300000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.96999999999999997,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.84899999999999998}]},{"osd":2,"last update":"Sun Nov 21 22:49:09 2021","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.20999999999999999},"max":{"1min":1.04,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92300000000000004,"15min":0.90300000000000002},"min":{"1min":0.30299999999999999,"5min":0.28999999999999998,"15min":0.23100000000000001},"max":{"1min":1.073,"5min":1.353,"15min":1.6359999999999999},"last":0.97899999999999998}]},{"osd":4,"last update":"Sun Nov 21 22:49:09 2021","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3160000000000001,"15min":1.345},"min":{"1min":0.88600000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.714,"5min":1.7709999999999999,"15min":1.9730000000000001},"last":1.484},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.375,"15min":1.3560000000000001},"min":{"1min":0.96799999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.831,"5min":1.831,"15min":1.833},"last":1.4990000000000001}]},{"osd":5,"last update":"Sun Nov 21 22:49:09 2021","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3460000000000001,"15min":1.335},"min":{"1min":0.92900000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6850000000000001,"5min":1.843,"15min":1.917},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.357,"15min":1.3540000000000001},"min":{"1min":0.92300000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.9470000000000001},"last":1.413}]},{"osd":6,"last update":"Sun Nov 21 22:49:09 2021","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.3460000000000001,"15min":1.3520000000000001},"min":{"1min":0.84199999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7470000000000001,"5min":1.7749999999999999,"15min":1.8180000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.3660000000000001,"15min":1.3340000000000001},"min":{"1min":0.82499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.891},"last":1.3640000000000001}]},{"osd":7,"last update":"Sun Nov 21 22:49:09 2021","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3540000000000001,"15min":1.351},"min":{"1min":0.93999999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.663,"5min":1.843,"15min":1.8859999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.3959999999999999,"15min":1.381},"min":{"1min":0.875,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.6339999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575511,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763076,"kb_used_data":87274304,"kb_used_omap":2822,"kb_used_meta":0,"kb_avail":87274300,"statfs":{"total":90150273024,"available":89368883200,"internally_reserved":4096,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2890245,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:49:06 2021","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.90800000000000003,"5min":1.081,"15min":1.351},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.21199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":1.03,"5min":1.155,"15min":1.1890000000000001},"last":0.92800000000000005}]},{"osd":1,"last update":"Sun Nov 21 22:49:06 2021","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.188},"max":{"1min":1.0309999999999999,"5min":1.075,"15min":1.1519999999999999},"last":1.034},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.052,"5min":1.155,"15min":1.5820000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Sun Nov 21 22:49:06 2021","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.0960000000000001,"5min":1.319,"15min":1.319},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81799999999999995,"15min":0.83699999999999997},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20499999999999999},"max":{"1min":1.002,"5min":1.302,"15min":1.302},"last":0.85099999999999998}]},{"osd":4,"last update":"Sun Nov 21 22:49:06 2021","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.335,"15min":1.3620000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.503},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.403},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.347,"15min":1.377},"min":{"1min":0.72499999999999998,"5min":0.66800000000000004,"15min":0.58999999999999997},"max":{"1min":1.909,"5min":1.909,"15min":2.0710000000000002},"last":1.4299999999999999}]},{"osd":5,"last update":"Sun Nov 21 22:49:06 2021","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.387,"15min":1.4099999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.61299999999999999},"max":{"1min":1.8160000000000001,"5min":1.901,"15min":1.901},"last":1.333},{"interface":"front","average":{"1min":1.462,"5min":1.387,"15min":1.4239999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.55500000000000005},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.913},"last":1.5580000000000001}]},{"osd":6,"last update":"Sun Nov 21 22:49:06 2021","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4079999999999999,"15min":1.4239999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.57599999999999996},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.625},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3600000000000001,"15min":1.375},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.54900000000000004},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.512},"last":1.288}]},{"osd":7,"last update":"Sun Nov 21 22:49:06 2021","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.369,"15min":1.381},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.59599999999999997},"max":{"1min":1.7849999999999999,"5min":1.8009999999999999,"15min":1.925},"last":1.222},{"interface":"front","average":{"1min":1.385,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.746,"5min":0.69299999999999995,"15min":0.51100000000000001},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.363}]}]},{"osd":0,"up_from":13,"seq":55834575511,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763072,"kb_used_data":87274304,"kb_used_omap":3810,"kb_used_meta":0,"kb_avail":87274304,"statfs":{"total":90150273024,"available":89368887296,"internally_reserved":0,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3901836,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.21099999999999999},"max":{"1min":1.254,"5min":1.278,"15min":1.3069999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79600000000000004,"15min":0.81699999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.224},"max":{"1min":0.98699999999999999,"5min":1.093,"15min":1.4730000000000001},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.29599999999999999,"5min":0.28299999999999997,"15min":0.19500000000000001},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.82899999999999996,"15min":0.81100000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.19},"max":{"1min":0.97699999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.98399999999999999}]},{"osd":3,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.17599999999999999},"max":{"1min":1.135,"5min":1.423,"15min":1.7010000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.27500000000000002,"5min":0.26900000000000002,"15min":0.20899999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.82599999999999996}]},{"osd":4,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.345,"15min":1.3540000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.643,"5min":1.792,"15min":1.9990000000000001},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.361},"min":{"1min":0.53000000000000003,"5min":0.50700000000000001,"15min":0.30099999999999999},"max":{"1min":1.6319999999999999,"5min":1.929,"15min":1.9299999999999999},"last":1.544}]},{"osd":5,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4219999999999999},"min":{"1min":0.66800000000000004,"5min":0.54500000000000004,"15min":0.28999999999999998},"max":{"1min":1.7609999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.4019999999999999,"15min":1.3899999999999999},"min":{"1min":0.60399999999999998,"5min":0.54800000000000004,"15min":0.35499999999999998},"max":{"1min":1.661,"5min":1.839,"15min":1.875},"last":1.5169999999999999}]},{"osd":6,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.361,"15min":1.3540000000000001},"min":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.27800000000000002},"max":{"1min":1.6779999999999999,"5min":1.6919999999999999,"15min":1.8879999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.345,"15min":1.357},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":1.641,"5min":1.8560000000000001,"15min":1.915},"last":1.296}]},{"osd":7,"last update":"Sun Nov 21 22:48:49 2021","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.353,"15min":1.375},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.7689999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.363},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3700000000000001,"15min":1.361},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":1.655,"5min":1.819,"15min":1.9350000000000001},"last":1.603}]}]},{"osd":1,"up_from":13,"seq":55834575511,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":88037376,"kb_used":763076,"kb_used_data":87274304,"kb_used_omap":3920,"kb_used_meta":0,"kb_avail":87274300,"statfs":{"total":90150273024,"available":89368883200,"internally_reserved":4096,"allocated":89368887296,"data_stored":89368887296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4014454,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 21 22:49:08 2021","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.90400000000000003,"5min":1.1100000000000001,"15min":1.1299999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.751,"15min":0.753},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.95599999999999996,"5min":1.0880000000000001,"15min":1.327},"last":0.80200000000000005}]},{"osd":2,"last update":"Sun Nov 21 22:49:08 2021","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.218,"5min":0.218,"15min":0.216},"max":{"1min":0.92300000000000004,"5min":1.1160000000000001,"15min":1.25},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.97599999999999998,"5min":1.161,"15min":1.161},"last":0.91600000000000004}]},{"osd":3,"last update":"Sun Nov 21 22:49:08 2021","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.0669999999999999,"5min":1.3460000000000001,"15min":1.6299999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.216},"max":{"1min":1.073,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Nov 21 22:49:08 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.306,"15min":1.3109999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.34599999999999997},"max":{"1min":1.6180000000000001,"5min":1.6990000000000001,"15min":1.77},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3620000000000001,"15min":1.3480000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.36199999999999999},"max":{"1min":1.837,"5min":1.837,"15min":1.9019999999999999},"last":1.3939999999999999}]},{"osd":5,"last update":"Sun Nov 21 22:49:08 2021","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3400000000000001,"15min":1.3660000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.39300000000000002},"max":{"1min":1.923,"5min":1.923,"15min":1.9399999999999999},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3839999999999999,"15min":1.361},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.35499999999999998},"max":{"1min":1.931,"5min":1.931,"15min":2.141},"last":1.581}]},{"osd":6,"last update":"Sun Nov 21 22:49:08 2021","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.325,"15min":1.3180000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.34999999999999998},"max":{"1min":1.7490000000000001,"5min":1.788,"15min":1.8400000000000001},"last":1.421},{"interface":"front","average":{"1min":1.286,"5min":1.3740000000000001,"15min":1.349},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.32500000000000001},"max":{"1min":1.627,"5min":1.7769999999999999,"15min":1.903},"last":1.4630000000000001}]},{"osd":7,"last update":"Sun Nov 21 22:49:08 2021","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.349,"15min":1.3560000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.379},"max":{"1min":1.7310000000000001,"5min":1.766,"15min":1.921},"last":1.613},{"interface":"front","average":{"1min":1.333,"5min":1.385,"15min":1.3700000000000001},"min":{"1min":0.68000000000000005,"5min":0.66300000000000003,"15min":0.35999999999999999},"max":{"1min":1.7989999999999999,"5min":1.841,"15min":2.0409999999999999},"last":1.629}]}]}]}} 2021-11-21T22:49:28.444 DEBUG:teuthology.orchestra.run.gibba016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2021-11-21T22:49:28.498 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cd12eb700 1 Processor -- start 2021-11-21T22:49:28.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cd12eb700 1 -- start start 2021-11-21T22:49:28.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cd12eb700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6ccc144810 0x7f6ccc1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.499 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cd12eb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0abce0 0x7f6ccc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cd12eb700 1 -- --> v1:172.21.2.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc147e80 con 0x7f6ccc0b1db0 2021-11-21T22:49:28.500 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cd12eb700 1 -- --> v2:172.21.2.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc0b1490 con 0x7f6ccc144810 2021-11-21T22:49:28.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cd12eb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0af060 con 0x7f6ccc0abce0 2021-11-21T22:49:28.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccaffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6ccc144810 0x7f6ccc1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:28.501 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccaffd700 1 --2- >> v2:172.21.2.116:3300/0 conn(0x7f6ccc144810 0x7f6ccc1478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55352/0 (socket says 172.21.2.116:55352) 2021-11-21T22:49:28.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccaffd700 1 -- 172.21.2.116:0/4030044342 learned_addr learned my addr 172.21.2.116:0/4030044342 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:28.502 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccb7fe700 1 --1- 172.21.2.116:0/4030044342 >> v1:172.21.2.116:6789/0 conn(0x7f6ccc0b1db0 0x7f6ccc0ac600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.116:6789/0 says I am v1:172.21.2.116:39078/0 (socket says 172.21.2.116:39078) 2021-11-21T22:49:28.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccaffd700 1 -- 172.21.2.116:0/4030044342 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0abce0 msgr2=0x7f6ccc0b18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-11-21T22:49:28.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccaffd700 1 --2- 172.21.2.116:0/4030044342 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0abce0 0x7f6ccc0b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.503 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccaffd700 1 -- 172.21.2.116:0/4030044342 >> v1:172.21.2.116:6789/0 conn(0x7f6ccc0b1db0 legacy=0x7f6ccc0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6ccaffd700 1 -- 172.21.2.116:0/4030044342 --> v2:172.21.2.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0af5a0 con 0x7f6ccc144810 2021-11-21T22:49:28.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.487+0000 7f6cc9ffb700 1 -- 172.21.2.116:0/4030044342 <== mon.0 v1:172.21.2.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583631729 0 0) 0x7f6ccc147e80 con 0x7f6ccc0b1db0 2021-11-21T22:49:28.504 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 --2- 172.21.2.116:0/4030044342 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc144810 0x7f6ccc1478c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7f6cb4002f70 tx=0x7f6cb4006eb0).ready entity=mon.0 client_cookie=ca14e9451d6d1cb8 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:28.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cc9ffb700 1 -- 172.21.2.116:0/4030044342 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb400d5c0 con 0x7f6ccc144810 2021-11-21T22:49:28.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cc9ffb700 1 -- 172.21.2.116:0/4030044342 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4000e90 con 0x7f6ccc144810 2021-11-21T22:49:28.505 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cc9ffb700 1 -- 172.21.2.116:0/4030044342 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb400d910 con 0x7f6ccc144810 2021-11-21T22:49:28.506 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/4030044342 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc144810 msgr2=0x7f6ccc1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/4030044342 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc144810 0x7f6ccc1478c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7f6cb4002f70 tx=0x7f6cb4006eb0).stop 2021-11-21T22:49:28.507 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/4030044342 shutdown_connections 2021-11-21T22:49:28.508 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/4030044342 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0abce0 0x7f6ccc0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/4030044342 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc144810 0x7f6ccc1478c0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:28.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/4030044342 >> 172.21.2.116:0/4030044342 conn(0x7f6ccc05bd20 msgr2=0x7f6ccc05c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:28.509 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/4030044342 shutdown_connections 2021-11-21T22:49:28.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/4030044342 wait complete. 2021-11-21T22:49:28.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 Processor -- start 2021-11-21T22:49:28.510 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- start start 2021-11-21T22:49:28.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc0abce0 0x7f6ccc146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 --2- >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ccc144810 0x7f6ccc15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.513 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 --2- >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0b3720 0x7f6ccc15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0b2ee0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.514 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- --> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0b3180 con 0x7f6ccc144810 2021-11-21T22:49:28.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- --> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc159d00 con 0x7f6ccc0b3720 2021-11-21T22:49:28.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc0abce0 0x7f6ccc146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:28.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 --2- >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc0abce0 0x7f6ccc146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.116:3300/0 says I am v2:172.21.2.116:55358/0 (socket says 172.21.2.116:55358) 2021-11-21T22:49:28.515 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 -- 172.21.2.116:0/2545889406 learned_addr learned my addr 172.21.2.116:0/2545889406 (peer_addr_for_me v2:172.21.2.116:0/0) 2021-11-21T22:49:28.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 -- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0b3720 msgr2=0x7f6ccc15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0b3720 0x7f6ccc15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.516 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 -- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ccc144810 msgr2=0x7f6ccc15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ccc144810 0x7f6ccc15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 -- 172.21.2.116:0/2545889406 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb400b1e0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.517 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6ccaffd700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc0abce0 0x7f6ccc146070 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7f6cb4002f70 tx=0x7f6cb4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:28.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb400d940 con 0x7f6ccc0abce0 2021-11-21T22:49:28.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4011ca0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.518 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc0aff90 con 0x7f6ccc0abce0 2021-11-21T22:49:28.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb401e460 con 0x7f6ccc0abce0 2021-11-21T22:49:28.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc0b2c50 con 0x7f6ccc0abce0 2021-11-21T22:49:28.519 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.491+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc0b32f0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.495+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 65700+0+0 (secure 0 0 0) 0x7f6cb4025030 con 0x7f6ccc0abce0 2021-11-21T22:49:28.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.495+0000 7f6cc3fff700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6cb00536f0 0x7f6cb0055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-21T22:49:28.520 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.495+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6cb406cb90 con 0x7f6ccc0abce0 2021-11-21T22:49:28.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.495+0000 7f6cca7fc700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6cb00536f0 0x7f6cb0055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-21T22:49:28.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.495+0000 7f6cca7fc700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6cb00536f0 0x7f6cb0055bb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f6cbc002fb0 tx=0x7f6cbc0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-21T22:49:28.521 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.495+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6ccc0b32f0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.587 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.579+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 --> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6ccc0b32f0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.583+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6cb4043fd0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.592 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.583+0000 7f6cc3fff700 1 -- 172.21.2.116:0/2545889406 <== mon.0 v2:172.21.2.116:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6ccc0b32f0 con 0x7f6ccc0abce0 2021-11-21T22:49:28.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6cb00536f0 msgr2=0x7f6cb0055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.594 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6cb00536f0 0x7f6cb0055bb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f6cbc002fb0 tx=0x7f6cbc0069b0).stop 2021-11-21T22:49:28.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc0abce0 msgr2=0x7f6ccc146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-21T22:49:28.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc0abce0 0x7f6ccc146070 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7f6cb4002f70 tx=0x7f6cb4015000).stop 2021-11-21T22:49:28.595 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 shutdown_connections 2021-11-21T22:49:28.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:3301/0,v1:172.21.2.117:6790/0] conn(0x7f6ccc0b3720 0x7f6ccc15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:3300/0,v1:172.21.2.117:6789/0] conn(0x7f6ccc144810 0x7f6ccc15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-21T22:49:28.596 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.117:6832/14582,v1:172.21.2.117:6833/14582] conn(0x7f6cb00536f0 0x7f6cb0055bb0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:28.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 --2- 172.21.2.116:0/2545889406 >> [v2:172.21.2.116:3300/0,v1:172.21.2.116:6789/0] conn(0x7f6ccc0abce0 0x7f6ccc146070 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-21T22:49:28.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 >> 172.21.2.116:0/2545889406 conn(0x7f6ccc05bd20 msgr2=0x7f6ccc1417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-21T22:49:28.597 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 shutdown_connections 2021-11-21T22:49:28.598 INFO:teuthology.orchestra.run.gibba016.stderr:2021-11-21T22:49:28.587+0000 7f6cd12eb700 1 -- 172.21.2.116:0/2545889406 wait complete. 2021-11-21T22:49:28.600 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/ceph.py", line 1839, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-21T22:49:28.603 INFO:teuthology.misc:Shutting down mds daemons... 2021-11-21T22:49:28.604 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-11-21T22:49:28.605 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.612 INFO:tasks.ceph.mds.a:Stopped 2021-11-21T22:49:28.613 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-11-21T22:49:28.613 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.619 INFO:tasks.ceph.mds.c:Stopped 2021-11-21T22:49:28.621 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-11-21T22:49:28.622 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.631 INFO:tasks.ceph.mds.b:Stopped 2021-11-21T22:49:28.632 INFO:teuthology.misc:Shutting down osd daemons... 2021-11-21T22:49:28.633 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2021-11-21T22:49:28.634 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.641 INFO:tasks.ceph.osd.0:Stopped 2021-11-21T22:49:28.643 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2021-11-21T22:49:28.644 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.696 INFO:tasks.ceph.osd.1:Stopped 2021-11-21T22:49:28.697 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2021-11-21T22:49:28.698 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.710 INFO:tasks.ceph.osd.2:Stopped 2021-11-21T22:49:28.711 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2021-11-21T22:49:28.712 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.730 INFO:tasks.ceph.osd.3:Stopped 2021-11-21T22:49:28.732 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2021-11-21T22:49:28.733 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.750 INFO:tasks.ceph.osd.4:Stopped 2021-11-21T22:49:28.751 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2021-11-21T22:49:28.752 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.758 INFO:tasks.ceph.osd.5:Stopped 2021-11-21T22:49:28.759 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2021-11-21T22:49:28.760 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.768 INFO:tasks.ceph.osd.6:Stopped 2021-11-21T22:49:28.769 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2021-11-21T22:49:28.769 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.776 INFO:tasks.ceph.osd.7:Stopped 2021-11-21T22:49:28.777 INFO:teuthology.misc:Shutting down mgr daemons... 2021-11-21T22:49:28.778 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2021-11-21T22:49:28.779 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.809 INFO:tasks.ceph.mgr.y:Stopped 2021-11-21T22:49:28.810 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2021-11-21T22:49:28.811 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.834 INFO:tasks.ceph.mgr.x:Stopped 2021-11-21T22:49:28.835 INFO:teuthology.misc:Shutting down mon daemons... 2021-11-21T22:49:28.835 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2021-11-21T22:49:28.836 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.889 INFO:tasks.ceph.mon.a:Stopped 2021-11-21T22:49:28.890 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2021-11-21T22:49:28.891 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.905 INFO:tasks.ceph.mon.b:Stopped 2021-11-21T22:49:28.906 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2021-11-21T22:49:28.906 INFO:teuthology.orchestra.run:waiting for 300 2021-11-21T22:49:28.913 INFO:tasks.ceph.mon.c:Stopped 2021-11-21T22:49:28.914 INFO:tasks.ceph:Checking cluster log for badness... 2021-11-21T22:49:28.915 DEBUG:teuthology.orchestra.run.gibba016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2021-11-21T22:49:28.932 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T22:49:28.933 DEBUG:teuthology.orchestra.run.gibba016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2021-11-21T22:49:29.183 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T22:49:29.184 DEBUG:teuthology.orchestra.run.gibba016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2021-11-21T22:49:29.421 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T22:49:29.422 DEBUG:teuthology.orchestra.run.gibba016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2021-11-21T22:49:29.574 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T22:49:29.575 DEBUG:teuthology.orchestra.run.gibba016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2021-11-21T22:49:29.741 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T22:49:29.743 DEBUG:teuthology.orchestra.run.gibba017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2021-11-21T22:49:29.905 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T22:49:29.906 DEBUG:teuthology.orchestra.run.gibba017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2021-11-21T22:49:30.055 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T22:49:30.056 DEBUG:teuthology.orchestra.run.gibba017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2021-11-21T22:49:30.169 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T22:49:30.171 DEBUG:teuthology.orchestra.run.gibba017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2021-11-21T22:49:30.275 INFO:tasks.ceph:Archiving mon data... 2021-11-21T22:49:30.277 DEBUG:teuthology.misc:Transferring archived files from gibba016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/data/mon.a.tgz 2021-11-21T22:49:30.278 DEBUG:teuthology.orchestra.run.gibba016:> mktemp 2021-11-21T22:49:30.287 INFO:teuthology.orchestra.run.gibba016.stdout:/tmp/tmp.ndPfg12TTg 2021-11-21T22:49:30.288 DEBUG:teuthology.orchestra.run.gibba016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ndPfg12TTg 2021-11-21T22:49:30.465 DEBUG:teuthology.orchestra.run.gibba016:> sudo chmod 0666 /tmp/tmp.ndPfg12TTg 2021-11-21T22:49:30.481 DEBUG:teuthology.orchestra.remote:gibba016:/tmp/tmp.ndPfg12TTg is 474KB 2021-11-21T22:49:30.545 DEBUG:teuthology.orchestra.run.gibba016:> rm -fr /tmp/tmp.ndPfg12TTg 2021-11-21T22:49:30.550 DEBUG:teuthology.misc:Transferring archived files from gibba017:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/data/mon.b.tgz 2021-11-21T22:49:30.551 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T22:49:30.560 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.9nM2zSxwYz 2021-11-21T22:49:30.561 DEBUG:teuthology.orchestra.run.gibba017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.9nM2zSxwYz 2021-11-21T22:49:30.703 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.9nM2zSxwYz 2021-11-21T22:49:30.758 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.9nM2zSxwYz is 564KB 2021-11-21T22:49:30.823 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.9nM2zSxwYz 2021-11-21T22:49:30.829 DEBUG:teuthology.misc:Transferring archived files from gibba017:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/data/mon.c.tgz 2021-11-21T22:49:30.829 DEBUG:teuthology.orchestra.run.gibba017:> mktemp 2021-11-21T22:49:30.834 INFO:teuthology.orchestra.run.gibba017.stdout:/tmp/tmp.9g5JE4YQ41 2021-11-21T22:49:30.835 DEBUG:teuthology.orchestra.run.gibba017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9g5JE4YQ41 2021-11-21T22:49:30.984 DEBUG:teuthology.orchestra.run.gibba017:> sudo chmod 0666 /tmp/tmp.9g5JE4YQ41 2021-11-21T22:49:30.997 DEBUG:teuthology.orchestra.remote:gibba017:/tmp/tmp.9g5JE4YQ41 is 564KB 2021-11-21T22:49:31.061 DEBUG:teuthology.orchestra.run.gibba017:> rm -fr /tmp/tmp.9g5JE4YQ41 2021-11-21T22:49:31.067 INFO:tasks.ceph:Cleaning ceph cluster... 2021-11-21T22:49:31.068 DEBUG:teuthology.orchestra.run.gibba016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-11-21T22:49:31.074 DEBUG:teuthology.orchestra.run.gibba017:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-11-21T22:49:31.110 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-11-21T22:49:31.114 DEBUG:teuthology.orchestra.run.gibba036:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-11-21T22:49:31.126 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2021-11-21T22:49:31.127 DEBUG:teuthology.orchestra.run.gibba016:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-11-21T22:49:31.130 DEBUG:teuthology.orchestra.run.gibba017:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-11-21T22:49:31.132 DEBUG:teuthology.orchestra.run.gibba019:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-11-21T22:49:31.146 INFO:teuthology.orchestra.run.gibba017.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-11-21T22:49:31.155 INFO:teuthology.orchestra.run.gibba016.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-11-21T22:49:31.165 DEBUG:teuthology.orchestra.run.gibba036:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-11-21T22:49:31.192 INFO:teuthology.orchestra.run.gibba019.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-11-21T22:49:31.193 INFO:teuthology.orchestra.run.gibba036.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-11-21T22:49:31.202 INFO:tasks.ceph:Archiving crash dumps... 2021-11-21T22:49:31.205 DEBUG:teuthology.misc:Transferring archived files from gibba016:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba016/crash 2021-11-21T22:49:31.208 DEBUG:teuthology.orchestra.run.gibba016:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-11-21T22:49:31.236 DEBUG:teuthology.misc:Transferring archived files from gibba017:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba017/crash 2021-11-21T22:49:31.238 DEBUG:teuthology.orchestra.run.gibba017:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-11-21T22:49:31.251 DEBUG:teuthology.misc:Transferring archived files from gibba019:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba019/crash 2021-11-21T22:49:31.252 DEBUG:teuthology.orchestra.run.gibba019:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-11-21T22:49:31.266 DEBUG:teuthology.misc:Transferring archived files from gibba036:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba036/crash 2021-11-21T22:49:31.268 DEBUG:teuthology.orchestra.run.gibba036:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-11-21T22:49:31.280 INFO:tasks.ceph:Shutting down logrotate 2021-11-21T22:49:31.282 DEBUG:teuthology.orchestra.run.gibba016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.286 DEBUG:teuthology.orchestra.run.gibba017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.288 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.291 DEBUG:teuthology.orchestra.run.gibba036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.357 DEBUG:teuthology.orchestra.run.gibba016:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.366 DEBUG:teuthology.orchestra.run.gibba017:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.388 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.409 DEBUG:teuthology.orchestra.run.gibba036:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-11-21T22:49:31.431 INFO:tasks.ceph:Compressing logs... 2021-11-21T22:49:31.432 DEBUG:teuthology.orchestra.run.gibba016:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:31.436 DEBUG:teuthology.orchestra.run.gibba017:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:31.439 DEBUG:teuthology.orchestra.run.gibba019:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:31.443 DEBUG:teuthology.orchestra.run.gibba036:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:39.426 INFO:tasks.ceph:Archiving logs... 2021-11-21T22:49:39.428 DEBUG:teuthology.misc:Transferring archived files from gibba016:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba016/log 2021-11-21T22:49:39.429 DEBUG:teuthology.orchestra.run.gibba016:> sudo tar cz -f - -C /var/log/ceph -- . 2021-11-21T22:49:41.729 DEBUG:teuthology.misc:Transferring archived files from gibba017:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba017/log 2021-11-21T22:49:41.730 DEBUG:teuthology.orchestra.run.gibba017:> sudo tar cz -f - -C /var/log/ceph -- . 2021-11-21T22:49:42.568 DEBUG:teuthology.misc:Transferring archived files from gibba019:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba019/log 2021-11-21T22:49:42.569 DEBUG:teuthology.orchestra.run.gibba019:> sudo tar cz -f - -C /var/log/ceph -- . 2021-11-21T22:49:42.891 DEBUG:teuthology.misc:Transferring archived files from gibba036:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba036/log 2021-11-21T22:49:42.893 DEBUG:teuthology.orchestra.run.gibba036:> sudo tar cz -f - -C /var/log/ceph -- . 2021-11-21T22:49:42.921 DEBUG:teuthology.run_tasks:Unwinding manager install 2021-11-21T22:49:42.936 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_911bc85f89d5c4ff92555237a37b5b774d7b1332/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-21T22:49:42.940 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2021-11-21T22:49:42.941 DEBUG:teuthology.orchestra.run.gibba016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-11-21T22:49:42.945 DEBUG:teuthology.orchestra.run.gibba017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-11-21T22:49:42.949 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-11-21T22:49:42.953 DEBUG:teuthology.orchestra.run.gibba036:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-11-21T22:49:43.068 DEBUG:teuthology.run_tasks:Unwinding manager clock 2021-11-21T22:49:43.212 INFO:teuthology.task.clock:Checking final clock skew... 2021-11-21T22:49:43.213 DEBUG:teuthology.orchestra.run.gibba016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T22:49:43.217 DEBUG:teuthology.orchestra.run.gibba017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T22:49:43.221 DEBUG:teuthology.orchestra.run.gibba019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T22:49:43.224 DEBUG:teuthology.orchestra.run.gibba036:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-21T22:49:43.233 INFO:teuthology.orchestra.run.gibba016.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T22:49:43.233 INFO:teuthology.orchestra.run.gibba016.stdout:============================================================================== 2021-11-21T22:49:43.234 INFO:teuthology.orchestra.run.gibba016.stdout:-hv01.front.sepi 216.240.36.24 3 u 97 128 377 0.230 9.310 0.248 2021-11-21T22:49:43.234 INFO:teuthology.orchestra.run.gibba016.stdout:*hv02.front.sepi 163.237.218.18 2 u 101 128 377 0.231 4.928 0.447 2021-11-21T22:49:43.234 INFO:teuthology.orchestra.run.gibba016.stdout:+hv03.front.sepi 216.229.4.69 3 u 121 128 377 0.240 4.775 0.247 2021-11-21T22:49:43.235 INFO:teuthology.orchestra.run.gibba016.stdout:+hv04.front.sepi 96.245.170.99 3 u 37 128 377 0.188 4.401 0.372 2021-11-21T22:49:43.237 INFO:teuthology.orchestra.run.gibba017.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T22:49:43.237 INFO:teuthology.orchestra.run.gibba017.stdout:============================================================================== 2021-11-21T22:49:43.238 INFO:teuthology.orchestra.run.gibba017.stdout:-hv01.front.sepi 216.240.36.24 3 u 40 128 377 0.175 5.594 0.637 2021-11-21T22:49:43.238 INFO:teuthology.orchestra.run.gibba017.stdout:*hv02.front.sepi 163.237.218.18 2 u 102 128 377 0.241 0.211 0.063 2021-11-21T22:49:43.238 INFO:teuthology.orchestra.run.gibba017.stdout:+hv03.front.sepi 216.229.4.69 3 u 56 128 377 0.256 0.445 0.070 2021-11-21T22:49:43.239 INFO:teuthology.orchestra.run.gibba017.stdout:+hv04.front.sepi 96.245.170.99 3 u 47 128 377 0.257 -0.097 0.196 2021-11-21T22:49:43.240 INFO:teuthology.orchestra.run.gibba019.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T22:49:43.240 INFO:teuthology.orchestra.run.gibba019.stdout:============================================================================== 2021-11-21T22:49:43.241 INFO:teuthology.orchestra.run.gibba019.stdout:-hv01.front.sepi 216.240.36.24 3 u 111 128 377 0.242 6.754 0.188 2021-11-21T22:49:43.241 INFO:teuthology.orchestra.run.gibba019.stdout:*hv02.front.sepi 163.237.218.18 2 u 24 128 377 0.270 1.883 0.170 2021-11-21T22:49:43.241 INFO:teuthology.orchestra.run.gibba019.stdout:+hv03.front.sepi 216.229.4.69 3 u 110 128 377 0.264 2.367 0.257 2021-11-21T22:49:43.242 INFO:teuthology.orchestra.run.gibba019.stdout:+hv04.front.sepi 96.245.170.99 3 u 98 128 377 0.230 1.661 0.214 2021-11-21T22:49:43.244 INFO:teuthology.orchestra.run.gibba036.stdout: remote refid st t when poll reach delay offset jitter 2021-11-21T22:49:43.244 INFO:teuthology.orchestra.run.gibba036.stdout:============================================================================== 2021-11-21T22:49:43.245 INFO:teuthology.orchestra.run.gibba036.stdout:-hv01.front.sepi 216.240.36.24 3 u 50 128 377 0.158 4.815 0.226 2021-11-21T22:49:43.245 INFO:teuthology.orchestra.run.gibba036.stdout:*hv02.front.sepi 163.237.218.18 2 u 53 128 377 0.118 -0.023 0.038 2021-11-21T22:49:43.245 INFO:teuthology.orchestra.run.gibba036.stdout:+hv03.front.sepi 216.229.4.69 3 u 116 128 377 0.144 0.249 0.042 2021-11-21T22:49:43.246 INFO:teuthology.orchestra.run.gibba036.stdout:+hv04.front.sepi 96.245.170.99 3 u 62 128 377 0.150 -0.458 0.019 2021-11-21T22:49:43.246 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2021-11-21T22:49:43.291 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2021-11-21T22:49:43.292 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2021-11-21T22:49:43.321 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-11-21T22:49:43.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-11-21T22:49:43.413 INFO:teuthology.task.internal:Duration was 3641.798747 seconds 2021-11-21T22:49:43.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2021-11-21T22:49:43.447 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2021-11-21T22:49:43.449 DEBUG:teuthology.orchestra.run.gibba016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-11-21T22:49:43.453 DEBUG:teuthology.orchestra.run.gibba017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-11-21T22:49:43.456 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-11-21T22:49:43.460 DEBUG:teuthology.orchestra.run.gibba036:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-11-21T22:49:43.497 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2021-11-21T22:49:43.498 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba016.front.sepia.ceph.com 2021-11-21T22:49:43.498 DEBUG:teuthology.orchestra.run.gibba016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-11-21T22:49:43.535 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba017.front.sepia.ceph.com 2021-11-21T22:49:43.536 DEBUG:teuthology.orchestra.run.gibba017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-11-21T22:49:43.544 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba019.front.sepia.ceph.com 2021-11-21T22:49:43.545 DEBUG:teuthology.orchestra.run.gibba019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-11-21T22:49:43.555 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba036.front.sepia.ceph.com 2021-11-21T22:49:43.555 DEBUG:teuthology.orchestra.run.gibba036:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-11-21T22:49:43.569 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2021-11-21T22:49:43.570 DEBUG:teuthology.orchestra.run.gibba016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:43.573 DEBUG:teuthology.orchestra.run.gibba017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:43.576 DEBUG:teuthology.orchestra.run.gibba019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:43.578 DEBUG:teuthology.orchestra.run.gibba036:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-11-21T22:49:43.600 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2021-11-21T22:49:43.601 DEBUG:teuthology.orchestra.run.gibba016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2021-11-21T22:49:43.604 DEBUG:teuthology.orchestra.run.gibba017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2021-11-21T22:49:43.638 DEBUG:teuthology.orchestra.run.gibba019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2021-11-21T22:49:43.642 DEBUG:teuthology.orchestra.run.gibba036:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2021-11-21T22:49:43.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2021-11-21T22:49:43.716 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2021-11-21T22:49:43.718 DEBUG:teuthology.orchestra.run.gibba016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-11-21T22:49:43.727 DEBUG:teuthology.orchestra.run.gibba017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-11-21T22:49:43.734 DEBUG:teuthology.orchestra.run.gibba019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-11-21T22:49:43.744 DEBUG:teuthology.orchestra.run.gibba036:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-11-21T22:49:43.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2021-11-21T22:49:43.767 DEBUG:teuthology.orchestra.run.gibba016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.771 DEBUG:teuthology.orchestra.run.gibba017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.774 DEBUG:teuthology.orchestra.run.gibba019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.777 DEBUG:teuthology.orchestra.run.gibba036:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.781 INFO:teuthology.orchestra.run.gibba017.stdout:kernel.core_pattern = core 2021-11-21T22:49:43.785 INFO:teuthology.orchestra.run.gibba016.stdout:kernel.core_pattern = core 2021-11-21T22:49:43.786 INFO:teuthology.orchestra.run.gibba019.stdout:kernel.core_pattern = core 2021-11-21T22:49:43.794 INFO:teuthology.orchestra.run.gibba036.stdout:kernel.core_pattern = core 2021-11-21T22:49:43.824 DEBUG:teuthology.orchestra.run.gibba016:> test -e /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:49:43.873 DEBUG:teuthology.orchestra.run.gibba017:> test -e /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:49:43.882 DEBUG:teuthology.orchestra.run.gibba019:> test -e /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:49:43.891 DEBUG:teuthology.orchestra.run.gibba036:> test -e /home/ubuntu/cephtest/archive/coredump 2021-11-21T22:49:43.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-21T22:49:43.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2021-11-21T22:49:43.917 INFO:teuthology.task.internal:Transferring archived files... 2021-11-21T22:49:43.919 DEBUG:teuthology.misc:Transferring archived files from gibba016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba016 2021-11-21T22:49:43.920 DEBUG:teuthology.orchestra.run.gibba016:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-11-21T22:49:43.943 DEBUG:teuthology.misc:Transferring archived files from gibba017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba017 2021-11-21T22:49:43.944 DEBUG:teuthology.orchestra.run.gibba017:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-11-21T22:49:43.966 DEBUG:teuthology.misc:Transferring archived files from gibba019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba019 2021-11-21T22:49:43.967 DEBUG:teuthology.orchestra.run.gibba019:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-11-21T22:49:43.989 DEBUG:teuthology.misc:Transferring archived files from gibba036:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-11-19_04:10:03-fs-octopus-distro-default-gibba/6512819/remote/gibba036 2021-11-21T22:49:43.990 DEBUG:teuthology.orchestra.run.gibba036:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-11-21T22:49:44.037 INFO:teuthology.task.internal:Removing archive directory... 2021-11-21T22:49:44.038 DEBUG:teuthology.orchestra.run.gibba016:> rm -rf -- /home/ubuntu/cephtest/archive 2021-11-21T22:49:44.042 DEBUG:teuthology.orchestra.run.gibba017:> rm -rf -- /home/ubuntu/cephtest/archive 2021-11-21T22:49:44.045 DEBUG:teuthology.orchestra.run.gibba019:> rm -rf -- /home/ubuntu/cephtest/archive 2021-11-21T22:49:44.048 DEBUG:teuthology.orchestra.run.gibba036:> rm -rf -- /home/ubuntu/cephtest/archive 2021-11-21T22:49:44.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2021-11-21T22:49:44.153 INFO:teuthology.task.internal:Not uploading archives. 2021-11-21T22:49:44.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2021-11-21T22:49:44.206 INFO:teuthology.task.internal:Tidying up after the test... 2021-11-21T22:49:44.208 DEBUG:teuthology.orchestra.run.gibba016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-11-21T22:49:44.212 DEBUG:teuthology.orchestra.run.gibba017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-11-21T22:49:44.216 DEBUG:teuthology.orchestra.run.gibba019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-11-21T22:49:44.219 INFO:teuthology.orchestra.run.gibba016.stdout: 655401 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 21 22:49 /home/ubuntu/cephtest 2021-11-21T22:49:44.220 DEBUG:teuthology.orchestra.run.gibba036:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-11-21T22:49:44.223 INFO:teuthology.orchestra.run.gibba017.stdout: 655401 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 21 22:49 /home/ubuntu/cephtest 2021-11-21T22:49:44.231 INFO:teuthology.orchestra.run.gibba036.stdout: 655401 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 21 22:49 /home/ubuntu/cephtest 2021-11-22T09:46:30.768 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-11-22T09:46:30.800 DEBUG:teuthology.task.console_log:Killing console logger for gibba016 2021-11-22T09:46:30.801 DEBUG:teuthology.task.console_log:Killing console logger for gibba017 2021-11-22T09:46:30.802 DEBUG:teuthology.task.console_log:Killing console logger for gibba019 2021-11-22T09:46:30.802 DEBUG:teuthology.task.console_log:Killing console logger for gibba036 2021-11-22T09:46:30.803 DEBUG:teuthology.exit:Finished running handlers